The Goodall Technology Reading Ease Index – How Complex is Your Marketing Copy?

I’ve often wondered what sort of ego people must have in order to name a discovery or invention after themselves. Well it appears I’ve got one of those egos myself as I’m about to introduce something similar.

[EDIT: My thoughts on this subject have matured a little. See my posts on the Arcanicity Index]

I read a lot of marketing copy as part of the REPAMA studies that I carry out. And I can tell you that some vendor’s marketing copy is very easy to consume whilst others’ copy is impenetrable. I’m not sitting in judgement here because I realise that my own natural writing style is quite formal and where possible I try to use the right words regardless of whether they are the easiest to read and understand. As long as you think your audience will be able to understand then there is no problem.

Anyway as part of the REPAMATron project which looks to automatically create competitive marketing intelligence studies by inferring the marketing strategy behind a vendor’s marketing copy, I’ve decided to try to rate the ease with which technology marketing copy can be read. This concept is not new, far from it in fact. A number of indexes exist that will look at text and rate how difficult it is to read and comprehend. Some, such as the Flesch-Kincaid Reading Ease test produces an arbitrary score that has to be interpreted, whilst others such as Gunning fog,  Coleman Liau, SMOG and The Automated Reading Index all attempt to predict the number of years education (equivalent to the US scholastic grade system) a reader would need to have had in order to understand the text.

This is all well and good and the formulas are all pretty straightforward and they all reach a similar conclusion. However, as they are general reading indexes they cannot take into account the domain knowledge that is required to understand the text.

For example, if I were to write:

In Cloud Computing it is important that a provider offers IAAS, PAAS as well as SAAS solutions.

Would a lay-person be able to understand that? Perhaps not and perhaps it is not intended for a lay person to read. Perhaps it’s aimed at an audience that should know what these things mean. And that’s fair enough.

But what about those visitors to a web site, or those readers of a product brochure that are interested but not expert. What is the impact of excluding the “interested but not expert” majority? As a technology vendor’s sales mature away from technology-led sales and it needs to reach a broader audience, will that audience understand? Does a vendor even know or realise that their marketing copy is inscrutable to all but the experts it targets?

If we use the traditional reading ease indexes they suggest that the text above is relatively easy to understand. Using the excellent online readability test at addedbytes.com, the sentence above produces the following readability scores.

Readability Formula Grade
Flesch-Kincaid Grade Level (Wikipedia) 10.50
Gunning-Fog Score (Wikipedia) 13.90
Coleman-Liau Index (Wikipedia) 10.20
SMOG Index (Wikipedia) 12.90
Automated Readability Index (Wikipedia) 7.80
Average Grade Level 11.06

So these results suggest that on average, 11 years education is what is needed to be able to understand that text. Of course that is just not the case. Without understanding the terms IAAS, PAAS and SAAS, the meaning behind the text remains hidden. Whilst these indexes allow us to look at sentence structure and the complexity of words that are used, they fail to take into account the domain knowledge that is required to understand the meaning. It’s unfair to criticise the existing indexes as they are generic and concern themselves with the ability of the reader to ‘read‘ the text. They don’t set out to examine the domain knowledge required to ‘understand‘ the text.

In my marketing studies, I do however have the luxury of focussing on words that are used to describe technology. So I am able to look for text that might require domain-specific knowledge to understand. One measure of that domain knowledge is how often technological acronyms  (IAAS, PAAS and SAAS in the text above) are found within the text. So by taking one ( or an average ) of the existing reading indexes and then applying some form of factor based on the the number and density of acronyms present in a text, we are starting to get close to a judgement on how much a specific piece of text relies on technological domain knowledge.

<tongue-in-cheek>And what am I to do about naming this index? I mean, Messrs Flesch and Kincaid used their own names as did Gunning, Coleman, Liau and SMOG (OK perhaps my example runs out at SMOG). But as it appears to be the done thing then I give the world the Goodall Technology Reading Ease Index.

But then it hit me. What if these eminent linguists and readability experts didn’t name their studies after themselves but instead others that followed christened them after their creator? It would be a bit arrogant of me add my own name to that list just yet. So for now I give you the Technology Reading Ease Index</tongue-in-cheek>

I’ll keep you posted as I develop the forumula.

Danny Goodall

P.S. The image in the top right is created by Wordle.net using the most complex font I could find and shows the 500 most common words used on the front page of this blog.

Posted in All Blog Categories, marketing, natural language processing, REPAMATron and tagged , , , , , , , , , , .

4 Comments

  1. Pingback: The Characteristics of a Technology Reading Ease Index | lustratus REPAMA

  2. Pingback: Eureka – Goodall Arcanicity and the Technology Reading Ease Index | lustratus REPAMA

Leave a Reply

Your email address will not be published. Required fields are marked *