Reporting importance

In our previous article, we looked at the different ways that you can measure how important something is to someone. The article suggested the following methods for directly measuring importance:

  • Ranking
  • Money spend
  • Likelihood of use
  • Frequency of use
  • Impact of non-existence
  • Attitude

Ways to report importance

Below are some of the options for reporting the results from each of these activities. For simplicity in describing the methods, I will use the symbols 'O' and 'N', where 'O' is the option we were measuring the importance of, and 'N' is the number of participants who gave a particular answer.

Ranking

  • "N people put O in their top 3."
  • "In the rankings of importance, N people put O first".

Money spend

  • "The average amount of money spent on O was ..."
  • "The total amount of money spent on O was ..."
  • "N people put their $50 note against O."

Likelihood of use

  • "N people said they were likely (either somewhat or extremely) to use O."
  • "N people said they were extremely unlikely to use O."

Frequency of use

  • "N people said they use O 'often'."
  • "N people said the would use O 'never' or 'rarely'."

Impact of non-existence

  • "N people indicated ... would happen if O didn't exist."

Attitude

  • "N people strongly agree with the statement: 'It would be very hard to do my job without O'."

Context is king

The way in which the results from each of these activities is reported can have a considerable impact on the story you're telling.

For example, suppose we have conducted a money spend activity and found that 8 out of 10 participants put $50 against option D. How important, then, is option O?

If participants were given five $50 notes, option D may not seem as important than if participants were given one $50 note, two $20 notes and a $10 note. Assigning $50 is not as significant if you've got another four $50 notes to spend, whereas assigning $50 out of $100 is a fairly big deal.

As this example shows, you need to include relevant contextual information when reporting importance. A good rule of thumb is to ensure that you describe any part of the methodology that could influence interpretation of the results. For each of the above methods, this is likely to mean providing the following information.

Ranking

  • Number of items respondents had to choose from.
  • How many rankings respondents were asked to give (e.g. top 3 or top 5).

Money spend

  • How much money, in total, respondents had to spend.
  • In what denominations this money was provided.

All other direct measures

  • The exact question asked.
  • The different response options provided (i.e. the scale respondents had to use when answering the question).

Indirect measurement of importance

So far we've only discussed direct measures of importance. There is one other popular method for ascertaining how important something is which doesn't measure this construct directly. This method derives a measure of importance from another, more reliable statistic, such as satisfaction.

The derivation of importance from satisfaction is done by estimating the relationship between satisfaction with the individual item and satisfaction overall.

If there's a strong relationship between an individual item and overall satisfaction, we can say that the item is relatively important. Conversely, if there isn't a strong relationship between an individual item and overall satisfaction, the item is relatively unimportant. To put it another way, the amount of influence each item has on overall importance is considered a measure of the item's importance.

The technique we use to measure the relationship is calculation of 'correlation', and certain conditions need to hold for this to be a valid approach. It's best to get assistance from someone with appropriate statistical skills if you're interested in using this approach. But beware that you need to ask the satisfaction questions in a certain way to enable correlations to be calculated. Therefore, it's worth considering the way you plan to report importance well before the questionnaire is finalised.

Importance untangled

Importance is a curious creature. On the one hand, it's something we want to measure frequently. On the other hand, it could well be one of the hardest things to measure accurately.

However, all is not lost. The techniques discussed here are easy to implement and should considerably improve the quality of importance data you collect. Good luck!