This is a transcript of one of the videos in the plain language training course Writing well at work. There are references to slides and visual material that are only available by viewing the video.
You can access the full web based course at writertraining.online.
Information about face to face courses is at thinkwrite.com.au
Well, we’ve completed our document. We started by doing all that thinking about PAM. We generated some content, we organised it in a helpful way. We wrote it using those 5 plain language techniques. We’ve reviewed it, we’ve refined it.
Gaining confidence your document will be effective
But how can we have confidence that this document is actually going to do the job? I mean, sure, once we publish it or pass it on to wherever it needs to go to, we can make some measure about the effectiveness of the document. But before that, how can we be sure I have some degree of confidence that this document’s going to do its job?
Well, I’m going to suggest we get that confidence by testing. You see, during this course I’ve described a document as an information product. Now, every product around us has been tested before it goes into use, or at least a prototype has been tested. I mean, my computer has tested the camera that I’m using has been tested. All these products have been tested before they go into use.
But somehow we’ve developed the attitude that we don’t need to test information products. All we need to do is to do the best we can and maybe pass it around some people in the office for their feedback and review. Maybe send it to the legal team to see what they think. But then let’s just get it out there and see how it goes.
It seems to me that taking an attitude like that publishing something, putting out a product without testing, might be reckless. In fact, I’d defined reckless writing as preparing a document without a deliberate and considered concern for the reader or a writer failing to apply their mind to consider how a document will be understood. When we push documents out there without testing, I think we could be described as reckless.
So my plea to you is to don’t assume that the document is going to work just because you’ve spent a whole lot of effort on it. Test the document. Choosing not to test a document is actually choosing to take on some risk. You’re choosing to take on the risk that the document won’t work, that perhaps it might be misunderstood that perhaps people won’t find the information they need, and perhaps they won’t be able to use that information.
So don’t assume it’s going to work. Test the document to check that it will. And when we test, we’re actually testing for usability. We’re testing for things that get in the way of people being able to find the information, being able to understand that information and then being able to use it to do something.
Now, while I’m talking about testing towards the end of this course, testing is something that can happen right from the very beginning. In fact, I think we should be running some form of testing right through the document development process. You see, if we leave it right to the very end, the cost of making a change, if we find that something’s not working, we need to change it, the cost of making that change is pretty extreme. And the number of design options that we have, well, we don’t have so many. But if we did that testing early on in the process, well, it doesn’t cost us too much to make changes and we’ve got lots of different choices that we can make.
And so I think the testing process is something like this.
We’re going to have a best guess of how this document will work and then we’ll build it. We’ll craft it. We’ll put the words together.
And then we’ll test it. And if we’re confident, it’s OK, we might publish.
But if we’re not, we might go in and refine it and then we’ll rebuild.
And we’ll retest and refine and rebuild and retest until we have some confidence that the document is going to work OK and then we’ll publish, then we’ll get it out there.
Let me tell you this story, just to demonstrate the importance of genuine user testing. There was an organisation that was asked to redesign a water bill for a water utility. So they redesigned it to make it look good and, you know, put different line items in different places. So we’d have the name and the address and billing metres, the amount of water charged, all that sort of thing was redesigned to make sure that this invoice could be easily understood by people.
One of the line items on this invoice was a thing called a water access charge. All the computer programming was done to be able to generate these invoices, and then they went out to people. When they went out to people, when these invoices hit people’s letterbox, the call centre just lit up. Lots of people calling, lots of really angry people, calling. Saying “what’s this water access charge doing on my bill? I never ever, ever use any excess water. I’ve never seen a charge like this before. What’s going on here? Please get that charge off my bill.”
After they’d listened to a few phone calls, they’d figured out that what was happening is that some people were misreading the word access as excess. And so people thought they were being charged for excess water.
Now there is no amount of internal testing, internal review that would have picked that up. You only discover that type of misreading when you put it out for use, when you test it with some real users. And once you discover it, it’s a really simple thing to do. You just need to change its description. Maybe call it a water connection charge or something like that.
So testing is really valuable. Testing with real users is really valuable before the document goes out somewhere for use.
The good news is that testing a document is not difficult or expensive or time consuming. In fact, you can get good information from just six test subjects. Now we’ve got a graph here from the Norman Nielsen group. They’re perhaps one of the worldwide experts on usability. Now I know a lot of their work is based around web work, but I think we can use their thinking for other types of artefacts.
The graph I’ve got here plots the number of usability problems they find against the number of tests they run. Perhaps one of the most instructive points on the graph is right down in the bottom left corner, which tells us if we don’t test, we’re not going to find any problems. Hmm. Yeah, that is true. We won’t. But as we test with one user, they’ll all have a number of usability problems that we’ll discover. As we test with a second user, they will repeat some of the first users problems and come up with a few of more of their own. The third user, likewise they’ll repeat some of the earlier problems and maybe a handful of their own. But they’ve found that by the time you get to 5 or 6 users, you’ve found most of the usability problems, around 95% of the usability problems.
So testing a document doesn’t mean going out and running a test with 300 or 400 people. You can get good information from just six test subjects.
Access the complete course at writertraining.online