KPIs 101.5: Statistics, Lies and Damn Lies...

As discussed in "KPIs 101: How Much Can Your Business Improve?", every quantifiable Key Performance Indicator that has ever been devised falls into the following categories:

1. Size - how big is the deal?

2. Number of - how many deals are on the table?

3. Speed - how fast does the deal take to work through?

4. Capacity - how many deals can we actually do?

5. Statistics - averages, conversion rates, ratios, standard deviations, skew, correlation etc. etc. etc.

If you use them together, you can quickly work out how much your business can improve.

Today we will discuss #5: Statistics - averages, conversion rates, ratios, distributions, correlations etc.

The first thing to realise is that a huge amount can be learned about a business without getting into the really hardcore statistical measures. For example conversion rates and averages.

Conversion rates are very similar in make up to velocity discussed in 101.4. Conversion rates deal with how many of one thing are required to produce another. For example, how many contacts does it take to generate a sales meeting? How many sales meetings does it take to generate a quote? How many quotes does it take to generate a sale? What % of a Sale is Gross Profit? What % of Sales are Fixed Costs? What % of Sales is Net Profit?

Conversion rates can be expressed either in percentage terms, as ratios, or as simple numbers. Every area of a business has its own key ratios that need to be tracked. For example, in production we might look at set-up times, velocities, capacities and wastage rates. In accounting, as well as looking at the % of Sales measures mentioned above, we might look at Accounts Payable and Receivables Turnover Rates, and how we went against budgets; while in marketing, we might look at the Lifetime Value of a Customer and Average Cost of Acquiring a Customer.

From there, we can build models which describe how our business works, and we can start to play with these models to work out where best to spend our money to get the biggest improvement bang for our buck.

Lies, Damn Lies and Statistics

Statistics are often trotted out to support points of view. For example, the average house price today is $500,000. However, in statistics, there are actually 3 different measures of the average. The most common measure is called the mean which is defined simply as the total value of the population divided by the total number of the population. However, the mean can be skewed by a small number of huge values. For example, if the population is 10, and 8 have houses worth $200,000, 1 has a house worth $1m, while the remaining person has a house worth $2.4m, the mean produces a skewed and, in this instance arguably unrepresentative result.

So statisticians came up with 2 other averages as an attempt to provide further information. The first is the mode which is the most common value. In this instance, it would be $200,000.

The second is called the median which is the value at which 50% of the population lies. In this instance, that would be $2.5m.

Taken separately, each average tells a completely different story. Taken together, the mean, median and mode give us a far better understanding of the subject.

The key to making statistics and in fact, any KPI work is to understand what questions you want them to help you answer.

For example, correlation is the process of establishing whether 2 things are related. But the first thing that any science or statistics major will tell you is that correlation does not equate to causation. I did some work recently for a big NGO, and I discovered that there was a 75% correlation between rainfall and donations. Now, the sample was only 1 year, and the rates were based on averages, but the correlation still existed.

Questions immediately arose:

1. What happens to the correlation if we expand the sample?

2. What happens if we drop the averages and use the actual daily data?

3. Are there any offsets e.g. lags in the data?

4. Are there other factors involved e.g. do the wetter months correspond with the end of the tax year when donors want to maximise their tax deductions?

5. If this correlation holds,

• how can it be used to optimise marketing? AND

• how can we drought-proof donations?

This is where other statistical measures that look at the spread of the distributed values e.g. standard deviation and skewness come in really handy. Essentially, we are trying to understand the impact of outliers on the data with a view to determining whether we need to direct our marketing spend across the whole spectrum of our customers and markets, or whether we can significantly reduce our spending by focusing it in specific, high volume or value areas that generate most of our results. This process is all about getting rid of the outliers.

On the other hand, in the world of sales and risk management, it is all about the outliers. For example, if our top sales person is consistently generating greater volume and better Gross Profit Margins, we need to look at why that is, and whether it can be replicated across the rest of the sales team e.g. by extra training or by better sales qualification processes.

In risk management, the outliers point to situations that may have slipped through the risk management framework gaps which need to be investigated to understand if, and how, the risk management frameworks and practices need to be changed to effectively catch these outliers in future.

Without context, KPIs make no sense.

If you do not understand why you are trying to measure what you are trying to measure, the numbers produced will mean nothing. Context is everything.

In the next article, we will look at a simple business KPI model where small changes can have a massive impact on the net profit results.