Market data and unrealistic expectations

By Katharine Turner, CIPD Vice President Reward

Median base pay policies - in fact any pay policies based on a specific market posture - should be abolished because they encourage companies to be over-reliant on market data and to expect from it a level of perfection and order that simply does not exist in the real world.

There. I have said it.

I have been analysing market data from salary surveys for almost thirty years now and I have developed a (healthy) scepticism of it over that time.

Good data is a prerequisite for good decisions. The data itself isn’t the main problem very often.

It’s the expectations of the data that cause the trouble.

Even the best quality data is imperfect, so why then are we surprised when data changes from year to year if not even more frequently?  Does this mean that the data is wrong?

The rational and analytical compensation expert who is constantly looking for inaccuracies and “errors in the spread sheet” should continue to do so but don’t also search for perfection.

There are many reasons why data can change. In my experience, market data – even over short periods - move quite significantly for perfectly sound reasons.

Here are three:

  • Peer groups which are based on market capitalisation will change often – unless you ask for the same companies to be in the sample as last year.
  • People (believe it or not) move jobs – so the data change.  NB. This should be much less relevant where benchmarked roles have multiple incumbents but can be highly relevant where role has one incumbent per company and the overall sample is small.
  • Data sources may not be comparable.  A chief executive’s job could be benchmarked against other chief executives on the basis of data drawn from annual reports and/or from proprietary sources.  Here timings, approach to analysis and sample can all affect the output.

But these technical explanations miss the real point.

Market data – where it reflects the market – is and should be dynamic.  It goes up and down; it changes.  Be warned.   If you have a median base pay policy, for example,  and are at pains to match the base pay of your employees to market medians expect the market medians to change and not always upwards. 

Sometimes market data can assume a primacy when we all know that drawing on multiple inputs and considerations to make informed and good decisions on pay levels is crucial. 

So if you want my opinion, it is this. Get the best pay data that you can. But don’t then be over-reliant on it and don’t be surprised if it changes.
Most important, then apply your judgement.

Remember – what others do should be but one part of formalising a view on what you should do.  Don’t be guilty of outsourcing your judgement to the market.

Thank you for your comments. There may be a short delay in this going live on the blog page as we moderate the comments added to our blogs.

  • I remember a pay planning report some years ago from a subsidiary company that said that their market position was 98.73% of the market.  I asked when they had done the analysis and they said last Monday.  I asked what their pay position had become by the Tuesday.  They failed to see the joke.

    Is market benchmarking a science or an art?  Discuss.....

  • I agree with Katharine and Clive.  Pay benchmarking is a tool not an answer.  My experience is that there is very limited understanding by people involved in pay determination that benchmarking is an art not a science, to pick up Clive’s point.  There are so many variables in both the data and the methodology that it can never be more than a guide.  Many managers think that the benchmark is an absolute, both in numeric terms and as an “answer” to what a role should be paid.

    The issue on judgement is also key.  A good practitioner will use knowledge build up over many years together with a good sense for what an organisation is trying to achieve and how the reward strategy is congruent to those outcomes to develop a view on the “right” level of pay rather that an understandable but blind reliance on the data to “prove” the pay proposal.

    Another issue that I believe causes some confusion is that benchmarking is about jobs or roles, not people.  Individuals in a role will have different levels of experience, qualification, motivation and so on.  The benchmark, by definition, looks at the median role, not necessarily the median performer.

    The issue raised by Katharine on sample size becomes particularly relevant when considering executive pay.  The sample sizes tend to be very small and with a large spread of outcomes.  Further, as performance plays a big role in executive pay the concept of a median, even for base pay has much less meaning.

    Having said the above the obvious question becomes how are decisions made on pay, particularly for executives?  Therein, as Shakespeare said, “lies the rub”.

  • Thank you Katherine for reminding us that "data" is not "information", and it is in no way "normative" - that is unless the company in question chooses to make it normative.

    Too often the "easy" solution is to fall back on the crass explanation "that's what the market says", when we should really be taking proactive decisions on HOW we want to make use of this data to transform it into something useful in the specific context of our organisations. For example :

    - determining a pay structure policy (size, scale, scope of salary bands based around a "chosen" set of market data)

    - determining how to make this structure evolve over time in the light of future data

    - determining how to position individuals within this chosen reference frame (the factors we take into account)

    - etc.

    I agree with Ian that calibration becomes critical for senior executives, and the application of sound judgement, in this specific context becomes a critical issue - moving away from the "what" and the "how" to the "why" is what good governance is about!

    In answer to Clive's essay question, my short answer is that it is clearly both, requiring a systematic analytical approach up front and the application of judgement further downstream in order to balance (without being able to precisely measure) the contradictory forces and pressures involved.