Who doesn’t love a nice top-50 story?
Some of our favourite examples of projects for PR clients are the top 50s (The Sun Example) and rank orders. They are a neat and easily digestible method to present data.
However, in an industry where time is of the essence, here is a simple and effective thought process to help reduce the amount of back and forth involved.
You do not need a Rank Order question to get a Ranking
We try to steer our clients away from ‘Rank Order’ questions and guide them to a better, more respondent-friendly, alternative.
A Rank Order is an output as opposed to an input. It should be something that comes out of the research results rather than a type of question that is put out to respondents in the first place.
The most well-known Rank Order is the Top Ten singles chart, as seen on "Top of the Pops". Number ten to number one - all neatly ranked.
They counted the sales of (or 'votes' for) each single and generated a ranking as an output. Each purchase gave a single vote - just as though it were a single code question - it was the mass of sales/ the mass of responses which served to generate the Rank Order output.
Mathematical issues with Ranking Data
Maths only works on Cardinal numbers. A 'rank order' becomes just the same as any other data the moment you look at anything other than the individual responses.
•If respondent A rates item one 1st and item two 2nd, that is meaningful.
•If respondent B rates item one 2nd and item two 1st, that also has a meaning.
However, when you average the data, item one and item two have the same score (one first and one second each). There is not anything between 1st and 2nd, which is why we see "1st Equal" results in sports. The 'rank order' data must become simple numerical data if we are to average them out. Therefore, it is much quicker and easier to collect that numerical data in the first place.
Put simply, if you ask a Rank Order question, the mathematical output is unlikely to produce a Rank Order.
Respondent Friendly?
Rank Order questions force the respondent to make unfamiliar and unrealistic comparisons and trade-offs. They must decide between elements of a good or service which they either do not normally consider or which they rank as equally important (or unimportant).
If researching food, for example, do people really have different rankings for "flavour" and "aroma"? Are smell and taste not binary for most consumers? Ice cream needs to have both a "nice flavour" and "nice smell" to qualify for purchase. Most consumers do not rank the taste as more important than the smell or vice versa.
Market Research has spent a lot of time and effort on Conjoint techniques. The thesis underlying Conjoint research is that consumers evaluate goods and services as a whole; a "gestalt evaluation". If this concept is right then asking consumers to consciously disaggregate their buying decisions: to Rank, the elements of the good or service, is sub-optimal market research.
Better than Ranking Questions
When a client asks for a Rank Order, a simple multi-code or single-code question will be the correct thing to offer in most cases. Very rarely, a client really does need a Rank Order question, in those instances, specialist market research expertise should be sought at the outset. These questions are significantly more complex and may not produce better or even different data, running the risk of producing of a body of data that is unrankable.
About OnePoll
Established in 2003 OnePoll is a leading provider of international market research and data communication solutions, powering online research with inspiring human insights. Collaborating with more than 550 organisations, OnePoll delivers market research consultancy, data visualisation & storytelling and audience-ready insights to global brands and ambitious start-ups alike. As an MRS Company Partner and Corporate member of ESOMAR, OnePoll is the second most-mentioned market research agency across the UK press and media with around 48,000 mentions annually.
OnePoll is a trading style of 72Point Limited, part of SWNS Media Group which consists of SWNS, 72Point, OnePoll, PLAY and Oath Studio.