Tag Archives: Document testing

How to mitigate document risk: Test with real users

Plain language / Plain English, writing tip, ,

Publishing a document that nobody reads, or that people don’t understand, or that users don’t act on, is a waste.

All the investment in crafting and reviewing will have zero return.

It’s very risky to assume a document will achieve purpose on the sole basis of internal review. Merely passing a document around the office to get your colleagues or boss’s opinion is not enough – it’s best to test with end users.

A document is an information product. Users are the only ones who can tell you if your information product is working.

Test early, test frequently to check your document is hitting the mark with users to get confidence that it will be effective.

5 steps for effective document review

Plain language / Plain English, , ,

You’ve slaved away for weeks on an important document. You’ve researched and talked to people. You’ve worked hard to understand your audience, their needs and how they will use your work. You’ve carefully structured the content into sensible chunks and you’ve used plain language techniques as you’ve crafted the text.

But, will the document work? Will it achieve the goals you had in mind?

It’s time for the document review and test phase. Time to find out what other people think of your work. Or perhaps time to send it ‘upstairs’ for sign-off.

Too often authors simply e-mail the document around to colleagues in the office with a message like “I’ve written this. What do you think?” If they don’t hear anything back they assume it’s all OK. But a review process like this rarely provides confidence that the document will communicate effectively. In fact, such a sloppy document review process could be reckless writing. (Our writer training course can reduce costly review churn.)

All communication products should be tested well before release, just like any other product. (Why don’t we test information products?)

So, when you ask people to review a document focus on assessing how well the document will achieve purpose.

Try these 5 steps:

1. Develop thick skin.

People may suggest improvements to your writing (and, by implication, improvements to your thinking). It’s easy to become defensive about the structure and words you have used. Don’t be. If someone finds your message is unclear then you need to improve it.

2. State your purpose clearly; identify your audience.

People can only provide useful review comments when they know what the document is intending to do, and who it is aimed at. When asking for review, explain who the document is for, and how you would like people to respond to your document. Judge the feedback you receive in the light of your purpose and audience.

3. Identify real users.

Representatives of your target audience are the best people to tell you how well your document will work. Anybody else is guessing (although experienced reviewers usually guess well).

4. Prepare document review questions.

Questions help reviewers engage with the material. For example:

  • Does this document contain information or ideas that are important to you?
  • Can you easily find the information you need?
  • Once you find the information, can you understand it?
  • Does it tell you too much? Does it tell you too little?
  • Is the information correct? Could it be misleading?
  • Does it comply with the law (if relevant) and our internal standards (if they exist)?
  • Do you know what to do now that you have read this?

5. Make feedback easy

Tell people how to get their comments to you. Do you want them to talk to you about the document, email you some ideas, or insert their comments in your text? And tell reviewers when you expect their comments.

You don’t need to act on all the review comments. Exercise judgement – some comments will be helpful, some will not. Use feedback to improve your writing, but don’t be bullied by it. After all, it’s your document, so take ownership of it.

How to write standard letters

Plain language / Plain English,

Crafting a letter once and then sending it to multiple customers or members is a smart way to communicate. Standard letters are especially useful in organisations with repetitive processes and a need to send an identical message to many people.

Standard letters reduce costs because the work is done once only and the benefit enjoyed many times. Standard letters also make sure the message is always consistent.

However, just because a letter is ‘standard’ does not mean it is a good piece of communication. Frequently readers misunderstand letters or fail to appreciate their importance.

One common consequence is increased call centre costs. Recipients call to find out what the letter really means.

Tips when writing standard letters

 1. Know your purpose in writing

Having a clear outcome in mind is essential when writing anything. You must know why you are writing and the impact you intend to have.

Before putting pen to paper, you need to be able to complete the sentence: “As a result of reading this letter, I want Mr Smith to …..”. It may be that you want Mr Smith to do something, think something, or feel something.

Defining the purpose of the letter helps you determine the content, the style and tone, and the call to action.

2. Understand your readers

The better you know and understand your readers, the more likely your letter will ‘contact’ them. Understand their situation, their current understanding of the material, their fears and desires.

It’s helpful to write down all the questions your reader is likely to have. You can probably guess some of these questions, but the only way you’ll know for sure is to ask people. User research (users – people who will use your letter) is essential when planning to write.

3. Know the situation

Standard letters are sent in response to some previous action, often by the member or customer, or because some threshold has been reached.

Knowing what has triggered the letter provides the context for writing and reading. It is helpful to explain to the reader why they are receiving this letter. The context may be obvious to you but your reader may not.

4. Write to your least sophisticated reader

Standard letters are read across a range of readability levels and across a range of familiarity with the content. Write so your less able readers will comprehend. Writing this way won’t disadvantage your more able readers; it just makes it easier for everybody.

Use plain language techniques: familiar words, sentences that make just one point, active voice, etc.

Your readers will appreciate being able to understand your message at first glance, and your call centre will benefit from fewer enquiries generated by unclear letters.

5. Be brief, write point first

Most people are busy. They want to read and understand your letter in the shortest time possible.

Put the main point of your letter at the very beginning. Don’t hold people in suspense. Some readers may only read the first line or two, so don’t keep vital information until the end.

Use as few words as possible to get your message across. If your letter is more than one page long, you reduce the likelihood of it being read completely.

6. Use appropriate variable material

Powerful standard letters include content drawn from your underlying data systems. For example, a superannuation fund may include different information for members aged under 65 than for those over 65.

Including specific information gives your reader a letter that is closely tailored to their needs. They don’t need to figure out for themselves what is relevant and what is not.

Including variable content in standard letters reduces the total number of letters to manage and may reduce system complexity.

7. Test, test, test

You will only know if your letter is working by testing it.

While you are crafting the letter, test early drafts with real users. Ask if they are likely to read the letter, ask if they can understand what it says, ask what they would do as a result of reading it. Refine in response to feedback and re-test.

When your letters are in production, measure if they are achieving purpose. Are people doing what you want them to do when they receive your letter?

The risk of writing functional documents and web content

Plain language / Plain English,

A functional document (including web content) is a document that readers have to do something with. Functional documents include contracts, reports, advice, disclosures, fact sheets, policies, procedures, terms and conditions, letters. Most documents written by businesses and government agencies are functional documents.

Whenever you write a functional document you are taking on risk.

Whenever you write a document that people have to rely on in some way, there is a risk they may not understand, or may not get the full picture, and so act in a detrimental way.

A fundamental principle of plain language is that the writer takes prime responsibility for the communication. You can’t blame the reader if they don’t get it. You may not be able to blame the reader even if they don’t read the document. The question is shifting from “Did you read the document?”, aimed at the reader, to “Is the document readable?”, aimed at the writer.

We generally consider risk by thinking about likelihood and consequence; thinking about the likelihood a reader may misunderstand your document, or not read all of it; thinking about the consequence of a reader acting on missing or confused information.

You can probably predict likely consequences. For many documents, risk will be low because the consequences of misunderstanding are not that bad.

But you can only know about likelihood by testing the document. You can only appreciate how people are likely to read and understand your document when you run it past a sample of real users and observe their reactions and thoughts.

You can prove you have taken document risk seriously by having your document certified.

Due diligence in public documents – test before publishing

Plain language / Plain English, ,

Due diligence: action that is considered reasonable for people to be expected to take in order to keep themselves or others and their property safe. (Cambridge English Dictionary – my emphasis)

Businesses and government agencies write public documents to inform people about products, services, obligations and opportunities. People may be harmed if they do not fully understand fact sheets, letters, contracts, websites and the like. They may not receive the product or service they expect, they may act in a damaging way, or they miss out on something due to them.

The impact of misunderstanding may be serious or trivial. If the document is about a work process, misunderstanding may result in death or injury. If it is about a financial product, misunderstanding may lead to poverty and disadvantage.

The development process for most public documents is something like

  1. a junior person writes a first draft
  2. a manager reviews and edits the draft
  3. the marketing people provide input
  4. the legal department review, to make sure there is no risk to the organisation
  5. final graphic design and publish.

Organisations may perform these steps with skill and care, but that is not due diligence. It is merely people internal to the organisation talking to each other about the document. They make untested assumptions about how the target audience will understand and react to the document.

Document testing is both reasonable and necessary for public documents

Testing helps keep other people safe. Document testing checks the information can be read, understood and acted on before it is published.

Choosing not to test is reckless.Reckless writing: not caring about readers Simply passing a document around the organisation and throwing it out to the public is not reasonable or responsible. We would never allow a physical product to enter the public arena that way – why do we tolerate it with information products?

Document testing is not difficult, expensive or time consuming. You can find most usability problems by testing with just a handful of potential users.