Joan Hanson was sick and hurt. She was 81-years old, dehydrated, suffered from skin ulcers, and had broken a few of her ribs in a fall. She called 911. An ambulance arrived at her home to whisk her away to the hospital. But, when they arrived, she was surprised that they did not immediately check her in. Instead, they waited in the hospital parking lot for an hour and a half.
Were the emergency room doors blocked by other ambulances? No, the door was free of obstructions. Was the hospital expecting to receive a large number of critically ill patients from an accident or a shooting? No, it was a typical weekday with the average patient volumes. The reason for the delay — which had become a routine practice — was to help the hospital improve patient wait-time performance metrics and avoid financial penalties.
When Joan was finally seen and stabilized, an emergency room nurse opined that Joan was in no shape to take care of herself and should be admitted to the hospital for further treatment. However, a hospital administrator overruled the nurse.
Was there no bed for Joan? No, the hospital had plenty of beds. Was there no money or medicine to care for her? No, and no. Joan was turned away because she was too sick. Patients like Joan threaten the hospital’s performance on government mandated outcome metrics. So, Joan was loaded in an ambulance and hauled to another hospital across town. After being admitted, several doctors refused to take her on as a patient for fear that a bad outcome would harm their grade on published “physician report cards.”
This fictionalized account of Joan’s travails demonstrate very real but perverse by-products of metrics aimed at improving patient care. Numerous studies have shown that the imposition of pay for performance metrics in healthcare leads to these and other behaviors at odds with patient wellbeing. Metrics maladies are not limited to healthcare.
When pay for performance testing was mandated for grade schools students, wide-spread cheating by teachers was documented in Atlanta, Chicago, Cleveland, Dallas, Houston, Washington, and other cities. In other cases, mayors and governors moved the goalposts by diminishing the difficulty of tests or lowering the grades required to pass them. This raised the pass rate, thereby demonstrating the success of their educational reforms. When politicians insist on meeting crime reduction targets, police departments often respond by misclassifying cases or recording incidents as lesser crimes. Or they skew their performance rates by directing resources to focus on the easier-to-solve crimes.
And, of course, the misapplication of metrics has taken a significant toll on businesses. When performance is judged by only one indicator — short-term profits — rather than other factors like reputation, market share, customer satisfaction, and employee morale — disaster inevitably follows. Continued revelations about the depth and breadth of Wells Fargo’s illegal activities are the latest example of a phenomenon that has plagued corporations for decades. On April 20, 2018, the bank agreed to pay a fine of US$1 billion for a variety of fraudulent activities associated with their handling of auto loans and mortgage interest rates. This penalty is on top of the US$1.5 billion already paid for offenses that included opening millions of false customer accounts, punishing whistleblowers, and unlawfully repossessing military service members’ cars. This all started when the bank set quotas for its employees for signing up customers for additional services and threatening disciplinary action or termination for those who failed to make their numbers.
Despite their possible shortcomings, performance metrics — including profit and loss — are vital to helping businesses track and report results to shareholders. The question is, how can we craft meaningful performance metrics that incentivize desirable behaviors while avoiding potential negative consequences? This question is asked and answered in Jerry Muller’s recently published book The Tyranny of Metrics. Muller observes that “[t]he problem is not measurement, but excessive measurement and inappropriate measurement — not metrics, but metric fixation…. The most characteristic feature of metric fixation is the aspiration to replace judgment based on experience with standardized measurement.”
Muller’s prescribed cure for this ubiquitous malady is twofold. First, recognize metrics’ inherent downsides and limitations. Second, use a checklist to ensure metrics are properly applied.
Metrics Downsides and Limitations
- Goal displacement: The tendency to focus on what is being measured at the expense of other, more important organizational goals.
- Short-termism: Advancing short-term goals at the expense of long-range considerations.
- Costs in employee time: The transactional costs associated with compiling, processing, and reading metrics.
- Diminishing utility: The reduction of metrics’ benefits over time.
- Rule cascades: The reduction in organizational efficiency because of controls to prevent gaming, cheating, and goal diversion.
- Rewarding luck: Measuring outcomes for results over which people have little control.
- Discouraging innovation: Stagnation that may result from a focus on established metrics at the expense of trying something new.
- Discouraging cooperation and common purpose: The tendency of individual incentives to induce competition at the expense of cooperation.
- Degradation of work: Employee misery caused by “filling in the boxes” at the expense of innovative problem-solving.
- Costs to productivity: The cumulative negative impact of poorly designed metrics on employee time, morale, and initiative.
The “Checklist”
- What kind of information do you seek to measure? Measuring objects influenced by the process of measurement are less reliable and more prone to unintended consequences than measuring inanimate matter.
- How useful is the information? Just because some activities can be measured does not make them worth measuring.
- How useful are more metrics? The fact that metrics can be helpful doesn’t mean that more metrics are more helpful.
- What are the costs of not relying on standardized measurement? Are there other sources of information about performance based on judgment and experience?
- To what purposes will the measurement be put? Data used for internal monitoring are less likely to pervert behavior than those used by external parties for reward and punishment.
- What are the costs of acquiring the metrics? Collecting, processing, and analyzing data costs money and takes time.
- Why are the people at the top demanding performance metrics? Consider hiring from within to avoid metrics aimed at educating ignorant executives.
- How and by whom are the performance measures developed? Measurements are more likely to be meaningful when they are developed from the bottom up rather than top down.
- Remember that even the best measures are subject to corruption or goal diversion.
- Remember that, sometimes, recognizing the limits of the possible is the beginning of wisdom. Not all problems can be solved by metrics.
Given the unrelenting parade of corporate scandals — often induced by ill-conceived performance metrics — tame the tyranny of metrics at your firm to avoid becoming the next victim.
