This is a transcript of one of the videos in the plain language training course Writing well at work. There are references to slides and visual material that are only available by viewing the video.
You can access the full web based course at writertraining.online.
Information about face to face courses is at thinkwrite.com.au
Now there’s a whole bunch of different ways that we can test documents, and I’m not going to go through all of those. In fact, that would be a course on its own, but here are some ideas of how you could test a document.
Firstly, you could simply ask for comments. You could give somebody a document and ask them what do they think. You’ll get some good information out of that.
Another type of test might be to ask somebody to paraphrase the document. Ask them how they would explain this idea to a 12 year old, for example.
Or perhaps you might set them some problems to solve, especially if it’s a process type document. You might ask them to do this thing by using this document. And you’ll see how well they’re able to follow this document.
You could create some understanding or comprehension tests. That would be another way.
Or you could do a thing called a structured interview. Or maybe you could use the think aloud protocol. Let me dive down into those last two just a little bit.
Using a structured interview to test a document
Suppose I was wanting to test that indoor air fact sheet that we looked at in a previous unit. What I’d do is I’d sit down with my test subject, perhaps I’d buy them a cup of coffee and we’d be sitting together and I’d ask them to read out the text aloud.
As they were reading, I’d be marking on my copy any words or phrases that they stumbled on. Of course, if they get stuck, I’m going to help them. I’d want to minimise any embarrassment. But I do want to know what are some of the words or phrases that are causing people some problems.
And when we’d finished reading that document I might ask them a bunch of questions. I might ask them things like:
What was your initial impression?
What did you think of that document?
Are there any words or phrases or ideas that you think some people might find difficult? Not that you found difficult, but that you think some other people might find difficult.
If I weren’t testing this document, how much of it would you actually read? That might give me an idea of how engaging this document is.
I might ask, as a result of reading that document, what do you think the writer wants you to do? And then I might follow that up with, well, what do you think you would actually do? Because sometimes even though what the writer wants them to do is clear, they’re not going to do it. So the document won’t be effective.
I might ask them some comprehension type questions, things like, well, what are some ways to reduce indoor air pollution? And is this something that you’re concerned about? Do you think this is an important issue?
Those types of questions to get an idea of how engaging and how important this document is to the person that will give me some really good information about how that document is going to work, how it’s going to be received by users and how they’re going to respond to it.
A think aloud protocol to articulate thinking
The other test approach I’ll talk about is the thing called the Think Aloud protocol. Now the way this works is we give the documents to our test subject and we ask them to read a paragraph or so and then ask them to tell us what they’re thinking, how they are responding to that that paragraph.
So in this example, they’d read that first paragraph, and that was that opening quote. And I think that they’d be likely saying things like, well, what’s that all about? I found that really hard to understand. It looks like this whole document’s gonna be hard work. And why are you talking about US laws and and what are ozone and sulphur dioxide? Are they particularly bad? Are they worse than other things?
And then I’d ask them to read that first paragraph and their response might be something like, do I have to remember that acronym? I hate them. Then as they read that sentence defining indoor air, their response might be. Indoors is obvious. Why do you feel you have to define it? You’ve got another committee wasting taxpayers dollars just talking about the obvious. And look, why is an industrial space not indoors? I’d reckon my workshop is an indoor space. And what’s so significant about an hour? Can’t pollution hit me in just a few minutes?
So they might have those types of questions. So we’d continue through the document in that way, getting people’s response to what they’ve read.
That would be another way of testing a document and getting some insights into how people are responding to the things that we have written.
Now, after you’ve done one of those tests, you won’t do them both. You’ll pick the test that you think is most appropriate in your situation.
Use the test results
After you’ve done that test, you will use those results.
If you’ve discovered that there are words or phrases that people struggle with, well find some alternatives.
If people are recalling information inaccurately, well, that would be a clue that we need to rewrite that. Make it a bit clearer.
If people say that they’re not likely to read it, or they’ll see that document as pretty unimportant, or we might need to highlight those key messages, maybe we’ll include some graphics that might help engage people.
And after we’ve made all those changes that we think are necessary to address the problems that we’ve found, then we’re going to retest. And we’ll keep on doing that: testing and retesting and testing and retesting until the majority of concerns are removed and we’ve got a reasonably high level of confidence that the document is going to do what it’s intended to do.
Now I know you’re not going to test every single document that you write, but think about those more critical documents where implementing a test regime would be really helpful to give you confidence that this document will work and will achieve the objectives that it’s set out to.
Sadly, this idea of document testing is not a popular one. It’s perhaps the thing that’s missed most in many plain language projects. People object, they say. Now look, it’s going to cost too much. It’s going to slow us down. We just don’t have time for all this testing stuff. We know ourselves what’s going to work and what’s not going to work.
There’s a bit of arrogance in choosing not to test. And my observation is that there’s never enough time or money to do it right, but there’s always enough time and money to do it again.
Hmm.
And you’ve possibly seen something like this. We’ve got a communication project and we decide not to test. We just get it out there. It doesn’t work so well, I don’t worry about that. We’ll just have another go.
So let me plead with you to think through whether testing your document would be a worthwhile thing to do. And how you’d go about testing that so that you’ve got confidence that this document is going to work.
Access the complete course at writertraining.online