Samu: Measuring for Management
As BI (Business Intelligence) services move out of the corporate sector into the world of SMEs so the metrics of KPIs (Key Performance Indicators), and their inherent problems should become a management concern. This is a serious issue for all company staff, investors and stakeholders, as many career fortunes (or lack of) are tied to these numbers, along with public perceptions.
Prior to adopting BI many organisations will already be utilising KPIs, using these statistical results to inform their management practice. So when it’s seen that KPI facilities are baked into most BI applications there is the temptation for management to reuse their companies existing metrics, without first reassessing their fitness for purpose. To help assess a metrics continued suitability it’s vital to critically examine their functions and purposes.
Anatomy of a measure
For a measure to be meaningful it has to have a purpose that is central to business functions. It should be related to specific objectives. The objectives need to be achievable.
For example you may have a KPI based on unique visitor to your site. This is fairly common in organisations that are dealing with inbound or content marketing, such as media organisations. But is it telling you, or anyone else, anything meaningful?
KPIs are used for a disparate array of purposes, typically to monitor cost performance, efficiency, inventory, but they can be, and often are deceptive of a true companies (Or governments) performance. Christoph Roser of All About Lean.com has written an excellent series of articles starting with ‘Lies, Damned Lies, and KPI – Part 1: Examples of Fudging Within these articles he demonstrates how industry uses these measures, where they often fail and offers some excellent advice on when and where KPIs can actually offer some meaningful value.
I’d add that another reason KPIs can fail is that they can be too wrapped up in Paul Drucker’s MBO (Management by Objectives) thinking,which has seeped through MBA courses to dominate too much management practice.
Disadvantages with MBO (From Communication Theory)
It has to be systematically done
There is more importance given to the setting of the goals than on the actual outcome or course of action
It may lead to polarization of efforts, whereby, people or departments are not motivated to look beyond their own targets and help others
It does not take into consideration, the environment in which the goals are set, like available resources, stakeholders, etc.
There is no stimulation of innovation
Managers start believing in the concept of an “ideal employee” and evaluate their subordinates based on what they expect they should be
Targets can be misreported and objective setting may become counter –productive to the organisation
Quality of goals set may be poorer or too unrealistic
It is time-consuming to implement and difficult to maintain
Setting production targets may encourage resources to meet those targets through whatever means necessary, which usually results in poor quality
It is not easy to identify all the objectives
Several alternatives to MBO exist, an example is Dr Deming’s 14 point methodologies, arising from Lean production, explored usefully in this blog.
At some stage I shall write about utilising Deming’s System of Profound Knowledge (SoPK) to communications, but for the rest of this post I am seeking to add to what Roser has written and explore the roles, and questionable uses of KPIs in measuring communications and look at how Roser’s suggestions can be put into practice,
I’ve seen KPIs used in relation to ‘Brand Health’, Marketing Optimisation’, ‘Operational Efficiency’, ‘Impact’ and even ‘Innovation’’, all important, indeed ‘key’ areas of a business’s communication goals. I’ve seen the results of these KPIs used to justify ‘Change Management’ and ‘Investment’’ decisions to boards and shareholders, and used as the basis to communicate a company’s ‘Successes’ to stakeholders. So no doubt they are a central cog in the machinery of management. Yet it’s fair to say I’ve also seen a lot of spurious nonsense.
First off, using KPIs that are intrinsically tied into a company’s mission and objectives is the correct approach. So there should be little wrong with the above?
Let’s briefly explore one of these, ‘Brand Health‘.
There are a range of methodologies used to analyse this, for example:
A brand can be measured competitively against similar brands to produce statistics to determine a ‘share of voice’ – total brand mentions, a total of a competitors brand mentions are divided to reach a percentage. So the metrics collection can be automated by recording the brand mentions in social channels, YouTube, Facebook, Twitter, blogs and monitored over time. These can be compared to brand mentions of competitors A, B, C… Across these same channels.
Never mind the potential problems of fake followers, what if your brand is actually falling behind the times and competitor ‘A’s’ strategy has been working to capture the WhatsApp crowd (A space where metrics are in contention)? If you are not counting, and you can’t count on all channels, you are not being adequately informed.
Roser states that you should only measure what you really need to, and that metrics are best used to learn from, Good advice. So should they always be key to your organization’s communication strategies?
Roser’s good practice
I have used his suggested titles here as a starting point and added a few.
- Temporary KPIs
These can be very useful. As an example of usage if your site is suffering from 404 errors due to lots of broken backlinks a temporary KPI is important whilst mitigation or changes are put into place to rectify this issue. 404 errors after all can lead to customer dissatisfaction and reputational damage can ensue.
- Reassessing KPIs
Do you really need 44 KPIs to assess the effectiveness of your impact reporting? I’ve seen it done and would question their collective effectiveness. It is beholden of management to occasionally reassess what is ‘key’ and the indicators fitness for purpose. It’s also in everyone’s interests to simplify where possible.
- Cascading structure
A KPI should make sense to the staff it affects, the senior management team, the governance, and to external stakeholders, even if the results are only communicated internally.
- SMART KPI
SMART KPIs are limited, A KPI should be Specific, Measurable, Achievable and Relevant with a Target in mind. A SMARTER KPI would consider the Environment too, of stakeholders, available resources etc. A HARD KPI, Heartfelt, Animated (easy to picture) Required or necessary Difficult (requiring moving out of your comfort zone) may provide for more relevant indicators on the path to achieving goals.
Are you using up to date data? I have witnessed an organisation using three year old ABC figures to determine their reach. Of course media sales had slumped in the interim, so their reach was greatly inflated. If this form of metric usage is really ‘Key’’ then numbers should not be fudged, manipulated or sources neglected, they should be verifiable and stand up to challenges.
- Basing reporting on KPIs
Claims of success, based solely on the evidence of KPIs is common, but I suggest they shouldn’t be. It’s the meaning that can be gained, rather than inferred, that’s really important. Statistics can be intentionally manipulated or unintentionally used to make false claims. Ask your self will your public trust you sufficiently to change their values based only on a quantitative analysis? Are you neglecting qualitative methodologies?
- Communicating Values
Values are not confined to numbers. To hold something in regard as important or having worth, requires engaging feelings of respect and empathy which are not always rationally derived. A number value should only ever be a part of the narrative you are communicating rather than an end in itself. This is known by most value driven organisations, yet not always sufficiently comprehended beyond the sectors in which they are found.