The Volkswagen “dieselgate” story, in which the company helped their cars pass emissions tests under special conditions that they later failed when the car was driven normally, is not an isolated incident – there has long been a culture of gaming metrics in the automotive industry and other sectors do it too.
Metrics are shorthand for good performance – they focus the mind of the setter and the tested and are costly and embarrassing if targets are missed. So why take the risk?
This gap between the rhetoric of good practice and the reality falling short in manufacturing has been noted elsewhere. And research I co-authored back in 2004 showed how another car supplier which had multiple sites across the world gamed the standards placed on it by customers. Winning business depended on performing well on various measures, with the company’s customers assessing each delivery of parts on quality, cost and whether it was on time and in full. One of us spent six weeks in two different sites carrying out participant observation. We found that the company gamed the system rather than admit issues with performance, presenting an image of good practice which was a facade.
Quarterly or biannually, customer procurement staff visited the supplier and undertook a complex evaluation process of the company we studied and the rest of their supply base. The quality and delivery performance over an extended time period was assessed as were cost increases, manufacturing and the supply chain capabilities and staff performance. League tables were produced and further business was awarded on good performance, or business was lost if the supplier scored badly.
The customers wanted to trust the supplier manufacturing process. This meant believing it was capable of producing good parts and meeting variations in demand. We found three examples of gaming to meet process measures.
1. Presenting a smooth-running operation
The semblance of high quality at every stage of the manufacturing process gives an indication to customers of the eventual product quality. It is common practice for suppliers to provide statistical process control charts where samples of car parts are checked to see if they match up to various metrics. Ideally, all the machinery and equipment should be measured and in control so that it does not produce bad parts. In our research, the supplier sent the charts taken from places in the process they knew were performing best, not a general sample and most certainly not from areas that they knew would make the graphs look bad.
2. Hiding stoppages and delays
The second measure was about stoppages or downtime in the process. Minimising unexpected breakdowns or downtime in the manufacturing process is key to being seen as a capable supplier. In order to achieve this, the best manufacturers plan maintenance periods where the line is stopped deliberately to service machines.
When the supplier process we studied stopped unexpectedly, management immediately reallocated the time to “planned maintenance”, so they met the downtime target and looked like an advanced manufacturer to the customers.
3. Minimising the appearance of waste
A good manufacturing process does not produce much wasted product. On the factory floor the designated container for our study company had an acceptable level of scrap product, which was measured and recorded. Material which could have been counted as scrap was disposed of via conveyor belts intended for other purposes. We even found that it was placed in employee trouser and coat pockets to be hidden and taken out the factory, so that waste targets were met.
Buyers would visit and audit the supplier, so the supplier was well practised at turning the factory floor into a stage where ideal manufacturing practices were displayed and performed during the visit. Examples include the accident reporting chart beginning after the last accident, records of employee skills being up to date, even if the employee went on the course a long time ago and might not be able still to perform the tasks in question. Machines were hidden, cleaned and floor spaces painted, all to create the impression of a competent organisation which lapsed as soon as the buyers left.
The major car manufacturing customers were not above gaming measures themselves. When an employee was honest with a customer about having problems with an aspect of the process they were told to “move the curve up” on the chart, rather than investigate the root cause and improve the process. Perfectly good batches of parts were rejected one week then accepted the next so that customer stock levels met their target.
A Volkswagen car during a test at a technical and testing centre.Reuters/Dado Ruvic
Beyond the car industry
It would be wrong to just demonise VW and the car industry. The downside of having measurement and metrics on performance is a far wider phenomenon. I have witnessed similar processes in the food sector and it seems very similar to the distorted affect of performance measures at the failing Mid-Staffordshire hospitals and even in football. People focus on the measure, not the wider good the measure was intended to represent. Plus, the setters of the measure are equally culpable.
I respectfully disagree with Edward Queen’s article on The Conversation, which makes the case for teaching business students better ethics – this is too little, too late. The problem is far wider and systemic.
Trying to meet measures with no extra effort is considered an ethical activity in many organisations and rewarded at annual appraisal time – another example of targets shaping performance. The attractiveness of metrics lies in their appearing to give managers control; the problem is that they cannot cover all aspects of performance and leave scope for short cuts, interpretation and gaming – by every party involved.
Returning to the car industry, it’s worth paying heed to the ideas of management expert W Edwards Deming. His ideas are widely attributed to be responsible for improving the reliability and quality of cars and achieving market dominance for companies who followed them. He believed that organisations should not be run on targets, quotas, or objectives, as they are usually a distraction from improving processes. In the case of VW and the car industry, the overarching goal is to lower emissions, which is a far riskier option to ignore in the long term.
Author: Lynne Frances Baxter, Senior Lecturer in Management Systems, University of York
