Managing Metric Management in ISO 9001
Rody Ryan, Director, Goldcert Management Systems Ltd, Ireland.
Since its revision in the year 2000, the ISO 9001 standard for Quality Management Systems has placed a significant emphasis on “Measurement, Analysis and Improvement”. This emphasis (like many other elements of the Standard) reflects the growing maturity of quality management experience around the world. There is a strong trend amongst the better companies towards the integration of the quality system and the management system. The resulting “Business Model” is firmly based on the collection and analysis of appropriate metrics in significant areas of the company’s activities. In this article we explore the opportunities and challenges involved in using the ISO 9001 standard as a framework for implementing an effective metric management system.
All companies have metrics - activities that are measured, reviewed and then adjusted as necessary in the light of the results. For many companies the only formal metrics are financial ones such as annual turnover, taxation returns or end-of-quarter results. Other companies have sophisticated systems in place to define, measure, analyse and improve their “Key Performance Indicators”. Such indicators typically go beyond the strictly financial measures to address operational issues such as on-time delivery, design lead-time, rework levels, customer satisfaction etc.
Even in organisations registered to ISO 9001 it is not unusual for these metric management activities to be initiated and controlled outside the quality system, as if they lived in some kind of “parallel universe”. In many cases the quality system was simply not designed to support an effective metric management system. As a result, senior management often regard their metrics as essential information for running the business, while quality system records are regarded as “a necessary evil”, maintained only “to keep the Registrar happy”.
A growing number of companies, on the other hand, have managed to integrate their management and quality systems so that the two are practically indistinguishable. Metric management plays a key role in the development of such a “Business Model” approach to the use of the ISO 9001 standard.
Since the revision of ISO 9001in the year 2000, the Standard has encouraged many more companies to follow this “Business Model” approach. Starting with a strong customer focus, the overall approach has been to align the quality management system with the business needs of the organisation. The Standard places a great emphasis on the use of the quality system to drive continual improvement and requires measurable quality objectives to be “established at relevant functions and levels within the organisation”. This requirement poses a significant, and hopefully positive, challenge for many companies.
In implementing ISO 9001, some companies need to introduce, for the first time, a formal system to measure and improve their own performance. Others need to integrate existing metric management systems into the quality management system. Even the companies that are already doing what the standard requires still face the ongoing challenge of ensuring that their metrics continue to provide adequate insight into key aspects of their business.
The Balanced Scorecard is a powerful tool for companies who wish to develop a comprehensive metric management system. It starts from the perspective that all stakeholders, by definition, have interests that the organisation needs to address in a balanced way. This approach provides an integrated framework within which different levels and functions of the organisation can identify, implement and monitor their objectives.
A typical Balanced Scorecard might include metrics on
- Financial performance
- Customer satisfaction
- Product/service quality
- Process and operational performance
- Supplier performance
- Staff training and development
- Employee satisfaction
- Community/Environment impact
The challenge is to identify suitable metrics under some or all of these headings that reflect the interests of different stakeholders and can be consciously aligned with the organisation’s strategic goals.
Getting the balance right
Implementing a metric management system raises many challenging issues:
- Too many measures quickly defeats the purpose of the exercise by creating confusion rather than clarity. Too few can lead to dangerous “blind spots”
- It is important that the data reviewed is not too short-term. Where possible, the metrics should show past-present-future characteristics. (Eg: Cumulative Sales up to the end of last month, Sales for last month, and projected sales for the next six months).
- The level of detail in both target and performance data should be appropriate to the user. Generally speaking, the higher up the organisation, the more summarised the information needs to be. The process by which high-level metrics and goals are decomposed or “cascaded” down to lower levels needs to be continually reviewed.
- Targets and goals need to be based on rational criteria and proper research. Poorly designed metric management systems end up as little more than glorified “number games”, where ill-considered targets lead to the creation of misleading or meaningless reports.
- Metrics need to distinguish between process efficiency and performance outcomes. If outcomes are based purely on the basis of short-term performance (eg working excessive hours “just to meet the target”) they will not last and the reported results, accurate as they may be in one sense, can give a very misleading impression of the efficiency of the underlying processes.
Balanced Scoreboard and ISO 9001
A major challenge facing many companies after registration to ISO 9001 is how to align the quality system with the key business goals of the organisation and thus avoid the phenomenon of “maintenance fatigue”. A powerful approach is to ensure that the Balanced Scorecard (or equivalent metric management system) provides the direction and that the quality management system provides the engine to get there.
With the benefit of hindsight, it can be seen that early versions of ISO 9001 described the “components” of a quality management system, as reflected in the famous “20 clauses” of the original Standard. The year 2000 version of the Standard, for the first time, described how these components fitted together to form an “engine” for systematic continual improvement that can be deployed in any part of the organisation.
Let us look at how the core ISO 9001 components (documentation, action tracking, records, audits, management review) can be used to drive systematic continual improvement in the metric management system itself.
The first point to recognise is that metric management is itself a process and should be mapped accordingly, with well-defined inputs and outputs, together with criteria for the selection of suitable metrics and for the review of performance. Like any other part of the business, if the metric management process is not defined, it cannot be systematically improved.
Many documented procedures are written without a clear sense of purpose beyond the old adage of “say what you do . . .”. From a Balanced Scorecard point of view, the purpose of a documented procedure is to capture the practice that results in effective outcomes. If performance is acceptable, the procedure remains unchanged. Otherwise it is updated to capture the changed practices and/or processes that lead to better results. A simple but powerful way to focus on this is to include a heading like “Measures of Effectiveness” in all procedures. This simple addition can help strengthen the link between the quality and metric management systems.
Mechanical implementations of ISO 9001 include “Corrective Action Request” (CAR) systems to address things that go wrong in their quality systems (ie non-conformances). Such naming conventions reflect and reinforce a rather negative mindset in relation to the quality system. By definition they are purely reactive and cannot easily be used to track the pro-active changes that result from the Balanced Scorecard approach. Many companies have derived significant benefit from renaming these reactive mechanisms with more pro-active titles such as “Controlled Action Request” or simply “Action Tracking” systems and using them accordingly.
Such systems can be used to cascade high-level objectives down to “relevant functions and levels within the organisation” (see ISO 9001:2008 5.4.1). In fact they can track the entire life-cycle (planning, implementation, verification) of a target or goal related to the Balanced Scorecard. Higher-level CARs (Controlled Action Requests) can be broken down into lower-level (“child”) CARs. This top-down planning process can be cascaded down to the smallest units of activity necessary to achieve the target.
The implementation of the plan is carried out in a bottom-up manner. Layer by layer, starting from the smallest units of activity, each CAR in the family is monitored, tracked, reviewed, recorded and closed. This cycle is repeated until the original, high-level target (hopefully) has been achieved.
This pro-active use of the corrective action mechanism can help invigorate the quality management system and move it from “maintenance mode” towards being a key driver for continual improvement and a core component of a successful “Business Model”.
In many ways the Balanced Scoreboard is no more than a formalised way of collecting and analysing “records… to provide evidence … of the effective operation of the quality management system” as has always been a requirement of ISO 9001. But more importantly, it can provide the “missing link” that so often prevents senior management from grasping the full potential of the quality management system.
Using the Balanced Scorecard literally (i.e. like a scoreboard at a football match) provides the organisation from top to bottom with a clear, visible focus for the collection and analysis of records. Properly designed, it helps organisations to face up to failure in a systematic, constructive manner, replacing the traditional “blame culture” with a strong focus on process improvement.
Internal audits are supposed to “determine whether the quality management system …. is being effectively implemented and maintained” (ISO 9001:2008 8.2.2b). Most internal audits focus on fairly stable aspects of the business (as reflected in documented procedures) and not on areas that are going through major change. Using Controlled Action Requests to track the implementation of goals and targets allows auditors to confirm that related changes are being planned, implemented and verified in a controlled manner. This is not only reassuring to senior management but also fulfils the ISO 9001 requirement that “the integrity of the QMS is maintained when changes to the QMS are planned and implemented”. Under a Balanced Scorecard approach, such audits can provide senior management with powerful, objective, timely, independent feedback on the current stage of implementation of the various changes that result from their high-level objectives.
Of course, an important area to audit would be the metric management system itself - to ensure that the organisation is following best practice and that performance improvements are being implemented in a systematic and sustainable way.
The assumption behind ISO 9001 is that senior management own and sponsor the quality management system and take an active leading role in its development and improvement long after registration. In practice, senior management tend to withdraw from the quality system shortly after certification, to get on with “the real business” of running the organisation. Their direct involvement in ISO 9001 tends to be limited to the “Management Review” (as required by the Standard) - typically once or twice a year. Such reviews tend to be rather mechanical, largely because the quality system under review is not well aligned with the change management issues that concern senior management.
Using ISO 9001 to implement the Balanced Scorecard approach can transform the Management Review and, more importantly, the entire relationship between the management system and the quality system. In fact they can both merge into something far more powerful than either on its own – a Business Model. In such a model, the Management Review includes issues of direct interest to senior management, such as:
- Did we achieve our business objectives as reflected in the Balanced Scorecard?
- Did we achieve them because of the quality system/business model – or was it in spite of it (ie performance-driven rather than process-based)?
- Do we need to improve our process for identifying and achieving the goals and targets defined on our Balanced Scorecard?
From Components to System...
The ISO 9001 process model for continual improvement provides an excellent framework for the implementation of an effective metric management system. Having reviewed the “components” of the ISO 9001 system, let us have a look at how the major clauses of the Standard can be used for that purpose:
Quality Management System (ISO 9001:2008 Clause 4)
The opening section on “General Requirements” (Clause 4.1) spells out a Plan-Do-Check-Act approach that clearly illustrates the common ground between ISO 9001 and the Balanced Scorecard. This is further strengthened by the Standards emphasis on the need for documented processes and well-managed records.
Management Responsibility (ISO 9001:2008 Clause 5)
Under this heading management have an opportunity to articulate their policy of putting a Balanced Scorecard approach at the very heart of their management system. Such an approach can transform the “necessary evil” into a source of pride and inspiration for the organisation and all its staff.
Resource Management (ISO 9001:2008 Clause 6)
Issues of resources apply as much to the metric management process as to any other. The main resource is usually time - time to collect and analyse data. Many companies find themselves in the vicious circle of being so busy "achieving results" that they have no time to prove that the results they achieve are worth the effort. Other important resources for an effective metric management system include appropriate software applications and suitable training for staff. Management commitment to an effective metric management system must be followed by the allocation of sufficient resources to make that system work – especially in the early stages.
Product Realisation (ISO 9001:2008 Clause 7)
The individual sub-clauses of this clause of the standard have obvious significance for the management of a metric management system:
- Planning realisation processes: In the spirit of this clause, a documented procedure should capture how metrics are selected, identified, broken down or "cascaded" to various functions and levels, monitored, reviewed and updated at appropriate intervals. In particular it should address the key events of reporting and responding to any deviations from targets or goals.
- Customer requirements: The metric management system should be aligned with the customer focus that underlies the entire ISO 9001 standard. This would ensure that metrics address issues of concern to customers.
- Design: As a top-down design activity, the Balanced Scorecard approach would benefit from the wisdom contained in the Standard’s Design clause.
- Purchasing: The normal precautions and controls that the Standard promotes in relation to vendors and purchases have an obvious relevance where the organisation is selecting training, consulting and software services for its metric management system.
- Control and Validation: The metric management system needs to be regularly reviewed to establish its own fitness for purpose.
- Identification and Traceability: The system needs to ensure that all data sources are clearly identified and that higher-level “information” is clearly traceable to lower level “data”.
- Preservation: This requirement has an obvious relevance to how well the integrity of the metric management information is protected against misleading, corrupted or incomplete data.
- Monitoring and measuring equipment: Metric management is all about measuring. This part of the standard poses the question “How do you know that your Balanced Scorecard is accurate, complete and up-to-date?”. This question is just as relevant to the integrity of data collection as it is to the organisation’s products or services.
Measurement, Analysis and Improvement (ISO 9001:2008 Clause 8)
A mature system should "measure, analyse and improve" the metric management system itself. This will not happen unless the system is designed to make it happen. Useful things to measure might include the following:
- The number of metrics established versus the planned number
- The degree to which the high level targets/goals have been met
- The number of metric-based activities that are completed on time
- The main reasons for failure to achieve targets/goals
- Main reasons for other deviations in the metric management process.
When the continual improvement loop is closed, and senior management (at Management Review) ask the big question: "Is this metric management system working for us...?" the answer should be very obvious.
Being in a position to ask the question is the hallmark of a world-class organisation.
As the world shrinks under the influence of information technology and globalisation, new challenges emerge for the future direction and structure of quality management systems. Many large corporations have implemented ISO 9001 on a site-by-site basis, with little or no regard to sharing best practice between sites or using the ISO 9001 toolbox as a framework for continual improvement across the corporation. Many such corporations do indeed operate good metric management systems, often with a Balanced Scoreboard approach, but usually outside the framework of “the ISO system”. This presents a major challenge.
For many organisations the process of streamlining the quality system within one site seems difficult enough.
The idea of aligning all their sites is so overwhelming that few even address this issue. And yet they do operate a metric management system. They do have an established best practice in identifying, monitoring, reviewing and responding to trends in their key performance indicators. And the changes that they initiate do have a direct and predictable impact on the integrity of the quality systems in the various sites.
If they are not using a quality management system to control their metric management activities…. what on earth are they using?
(Updated by the original author in 2011 from a paper published in the Proceedings of the XXVIII Congress of IMECCA, the Mexican Institute for Quality Control, Aguascalientes, Mexico 4-7th October 2000).
This article was published in NSAI E-zine