The CIPM MRA: Past, present and future

Andy Henson
Director, International Liaison and Communication Department
BIPM

Abstract:

The CIPM Mutual Recognition Arrangement (CIPM MRA) has now been signed by the representatives of 98 institutes and covers a further 152 institutes designated by these signatories. These come from 53 Member States, 41 Associate States and Economies of the CGPM, and four international organizations. Through it the national metrology institutes can demonstrate their measurement abilities and publish internationally recognized statements of their capabilities. All the data are openly available in the CIPM MRA database (the KCDB), which has become an essential reference for the accredited laboratory community amongst others. In this paper we review the situation that led to the development of the CIPM MRA, explain its structure, mechanisms and impact, and conclude with some speculative remarks as to how it might evolve in the future.

Introduction

The CIPM Mutual Recognition of national measurement standards and of calibration and measurement certificates issued by national metrology institutes (known as the CIPM Mutual Recognition Arrangement, or CIPM MRA) [1] was signed in Paris on 14 October 1999 by the Directors of 38 National Metrology Institutes (NMIs) and two international organizations. Today, it provides a primary source to identify internationally recognized national capabilities within the NMI and wider metrology community. The underpinning science and the outcomes are openly available to all interested parties.

The CIPM MRA is built on demonstrated and peer-reviewed capabilities providing a high level of confidence and trust. Participation in the CIPM MRA has grown substantially since its launch. At the time of writing (April 2015) it had been signed by the representatives of 98 institutes – from 53 Member States, 41 Associate States and Economies of the CGPM, and four international organizations – and covers a further 152 institutes designated by the signatory bodies [2].

The requirement to participate in scientific comparisons with international counterparts, together with the peer-review process covering both the quality management system and the individually declared Calibration and Measurement Capabilities (CMCs), ensures a high degree of rigour within the CIPM MRA. This in turn provides the basis for and underpins the international mutual recognition. The names of the participants, the comparison reports and results, and the CMCs are all publicly and freely available in the BIPM key comparison database, the KCDB [3], which is maintained by the BIPM. At the time of writing, 898 Key Comparisons, 421 Supplementary Comparisons and almost 24 000 peer-reviewed CMCs are listed in the KCDB [4].

Through the Joint Committee of the Regional Metrology Organizations and the BIPM (JCRB) [5], the BIPM operates the CMC inter-regional review website; the BIPM also chairs the JCRB and provides the Executive Secretary for it. The Executive Secretary position has always been a secondment position from one of the Member State NMIs to the BIPM, the secondment typically being for a two year period.

After some 15 years of operation the CIPM MRA has matured into a well-recognized pillar of the international quality infrastructure. The CIPM MRA is signed by the Directors of NMIs and as such it is not binding on Governments. Nevertheless, over time the CIPM MRA database has become the essential reference on the internationally accepted calibration capability of the NMIs and Designated Institutes (DIs). One unforeseen aspect of this success has become evident in recent years. The Directors of the major participating laboratories, whose staff bear the brunt of the workload of piloting comparisons and reviewing CMCs, had understood that in the early phase there would be a significant workload. What they had not anticipated was just how the CIPM MRA would continue to expand in both scope and participation, with the consequent continued and ongoing high workload which remains today. After a decade and a half it is time to review the implementation and operation of the CIPM MRA and ensure its sustainability for the coming years.

This paper will look at some of the original motivations which led to the CIPM MRA being drawn up; review its structures, mechanisms and operation; and consider its impact, as well as its evolution over time. The article will conclude with the author’s speculations as to what the future might hold for the Arrangement.

The origins of the CIPM MRA

The backdrop to the CIPM MRA was the major increase in world trade triggered by the General Agreement on Tariffs and Trade [6]. GATT provided fair trade rules and led to the gradual reduction of tariffs, duties and other trade barriers, for goods, services and intellectual property. Although GATT began in 1947, the early decades focused on preventing increased tariff barriers rather than on any reduction in tariffs. However, in its later years, tariff barriers began to tumble and world trade expanded accordingly. The final round — the 1986-1994 Uruguay Round [6]— led to the creation of the World Trade Organization (WTO). As fiscal barriers to trade were reduced, non-tariff barriers, and the need to address them, were brought into far sharper focus, leading to the Technical Barriers to Trade (TBT) Agreement which first came into force alongside the WTO in 1995. The need to measure consistently, and to have those measurements accepted across trading partners, was fundamental to an increasingly globalized world.

In addition to these changes on the world stage, three different developments collectively reinforced the need for a comprehensive and coherent solution to establishing and demonstrating the degree of international equivalence at the NMI level. It is worth looking a little more closely at each of these drivers, and at the response of the international community.

The first driver related to countries trading with the largest economy in the world, the USA. Many US regulators required instruments to be specifically calibrated by the US NMI, the National Institute of Standards and Technology (NIST). Whilst comparability related to the SI base units, and particularly the metre and kilogram, were established under the auspices of the CIPM, the need for confidence in industrially relevant derived units such as pressure or force was becoming a major issue. In particular, regulators in the USA, and consequently the companies supplying equipment with their sphere of regulation, were demanding that calibrations must be ‘traceable to NIST’ (rather than ‘traceable to the SI’). That is to say the only acceptable traceability route for instrument calibration for some regulators in the USA was via the USA’s NMI, irrespective of whether or not the manufacturer was in the USA. This represented a major cost and delay hurdle for non-USA manufacturers and at the same time placed an unwelcome additional burden on NIST, which was increasingly being requested to calibrate instruments for non-USA customers. From the early 1980s onwards the NMIs of the major trading nations were able to ease the burden for their manufacturers by concluding bilateral agreements with NIST, often supported by bilateral comparisons, demonstrating the equivalence of their measurement standards with those held at NIST. NIST in turn educated USA regulators as best they could, by providing the regulators with advice and a technical understanding of the issue, often on a case-by-case basis. This enabled the regulator, if they so chose, to accept the foreign NMI’s calibration of the instrument manufacturer’s equipment.

The CIPM recognized this trend as early as 1983 [7], although it was initially reluctant to intervene in what was essentially an issue between pairs of sovereign nations. It recognized, however, that if the same approach were widely adopted by other nations it could result in an unmanageable web of bilateral relationships, and also that the NMIs and industries of smaller nations were at risk of being disadvantaged, without the ‘trade clout’ to warrant the priority of concluding a bilateral agreement and conducting the supporting comparisons for their NMI. By 1986 the first tentative and modest concepts for some sort of centrally coordinated approach were being discussed and considered within the CIPM, but it took more than ten years, and the two other key drivers, before the CIPM reached consensus to move forward.

The second driver related to the emergence of a laboratory accreditation system in the European Community. By the early 1990s certification of manufacturers, and consequently accreditation of the calibration and testing laboratories supporting them, was becoming increasingly prevalent. Laboratory accreditation in Europe expanded particularly rapidly, as a way of ensuring industrial confidence in calibrations throughout the European Community (now the European Union), with its many different languages, cultures and legal systems. Before long, the accreditation system was adopted, with some variations in implementation, world-wide. Accreditation of laboratories systemically formalized the need to have and to demonstrate metrological competence underpinned by credible metrological traceability. The International Laboratory Accreditation Cooperation (ILAC) began to appraise the international metrology community of the needs of their national accreditation bodies, accredited calibration and test laboratories (of which there are now some 49 000 world-wide). These commercial calibration labs were required to demonstrate that their equipment had been calibrated by a competent body and that they had appropriate metrological traceability. The accreditation community needed some way to be sure that the measurement capabilities claimed by the NMIs were justified, and ILAC called for a database to provide information and assurance on sources of reliable, internationally recognized metrological traceability.

The third driver arose in parallel with the emergence and strengthening of regional metrology cooperation through what are now known as Regional Metrology Organizations (RMOs). Emerging from the 1977 Commonwealth Science Council Initiative, the Asia Pacific Metrology Programme (APMP) was established in 1980. This was followed by EUROMET, the European Association of National Metrology Institutes (now EURAMET) in 1987; the Inter-American Metrology System (SIM) in 1988; the Euro-Asian Cooperation of National Metrological Institutions (COOMET) in 1991; and the Inter-Africa Metrology System (AFRIMETS) in 2007. As comparisons are the key tool enabling NMIs to benchmark their scientific progress, as well as to demonstrate their service delivery capabilities, the RMOs began to organize regional comparisons to provide evidence of comparability for their members whose capabilities were not sufficiently advanced for them to participate at the global level. There was an increasingly urgent need to somehow tie together these regional comparisons with those of the CIPM and the BIPM.

By 1994, with growing acceptance of the need for a coordinated world-wide activity, the CIPM outlined a draft Resolution to be put before the CGPM at its meeting in October 1995. This resolution, which was adopted as Resolution 2 (1995) [8], set out the principles for what would become the CIPM MRA four years later.

Thus, towards the end of 1995 the foundation for the CIPM MRA had been laid. What lay ahead was a protracted consultation and drafting exercise undertaken by the BIPM and involving the CIPM and the Directors of the NMIs (who would be the future signatories). The main elements were outlined by the time of the CIPM meeting in 1996, but fine tuning of the details, and winning support across the breadth of NMIs took a considerable amount of work and diplomatic skill from all involved. In particular, many NMIs were not used to the concept of routinely having their capabilities internationally peer reviewed, a cornerstone of the proposed process, and it took time for some of them to become comfortable with this approach. Further, at that time ISO Guide 25 (now ISO/IEC 17025) was not universally accepted for use by all NMIs, so the requirements for laboratory quality management systems had to be resolved to the satisfaction of the expected participants. The approach to establishing reference values and the meaning of equivalence had to be looked at afresh. Resolving how best to address different perspectives on a wide variety of detailed issues took time.

Launch of the CIPM MRA

In October 1999, during the 21st meeting of the CGPM, Resolution 2 was adopted, formally paving the way for signature of the CIPM MRA. In Resolution 2 (1999) the CGPM invited [8]:

• all Member States of the Metre Convention to participate in the arrangement by giving authority to the director of the designated national metrology institute in their country to sign the arrangement,
• all Member States to make every effort to implement the arrangement and to encourage other authorities in their country to recognize the equivalence of national measurement standards and calibration and measurement certificates thereby demonstrated,
• all States to use this arrangement as the basis for recognizing the national measurement standards and calibration and measurement certificates of signatory national metrology institutes.

On 14 October 1999 the Directors of the NMIs from 38 Member States and two international organizations signed the document, and the CIPM MRA was finally under way.

In parallel, following consultation with the WTO, consideration was given to ensure that the CIPM MRA did not itself become a technical barrier to trade. The CIPM created a new status of ‘Associates of the Conférence Générale des Poids et Mesures’ to allow States that were not yet ready to become Member States, (and in special cases Economies) the opportunity to participate in the CIPM MRA. Thus alongside Resolution 2 (1999) addressing the CIPM MRA, the 21st CGPM also adopted Resolution 3 (1999) [8] to lay out the basis for participation in the CIPM MRA by NMIs from Associate States or Economies. Depending on the size of a state’s economy, Associates were allowed to participate in the CIPM MRA with a subscription as low as one tenth of the minimum contribution that would be paid if they were a Member State.¹

Structure and mechanisms of the CIPM MRA

The objectives of the CIPM MRA are [1] to establish the degree of equivalence of national measurement standards maintained by NMIs; to provide for the mutual recognition of calibration and measurement certificates issued by NMIs; and thereby to provide governments and other parties with a secure technical foundation for wider agreements related to international trade, commerce and regulatory affairs.

NMI directors sign the CIPM MRA with the approval of the appropriate authorities in their own country and thereby accept the process specified in the CIPM MRA for establishing the CIPM MRA database. They agree to recognize the results of key and supplementary comparisons as stated in the CIPM MRA database and to recognize the published Calibration and Measurement Capabilities (CMCs) of other participating NMIs and DIs.²

A limited number of international organizations also participate. There are currently four such organizations [2]: the European Space Agency (ESA); the International Atomic Energy Agency (IAEA); the Institute for Reference Materials and Measurements (IRMM); and the World Meteorological Organization (WMO).

The three fundamental elements leading to approval of an institute’s CMCs are:

1. participation by the institute in reviewed and approved scientific comparisons;
2. operation by the institute of an appropriate and approved quality management system;
3. international peer-review (regional and inter-regional) of claimed calibration and measurement capabilities.

A generalized overview of the process is given below, but for full details please refer to the text of the CIPM MRA, available on the BIPM website [1].

The outcomes of the CIPM MRA are published, internationally recognized statements of the CMCs of the participants [3]. The technical basis relies on demonstrated competence through international key and supplementary comparisons, and the operation of peer-reviewed quality systems at the NMIs.

Participating institutes are required to operate an appropriate quality system (essentially this currently means ISO/IEC 17025, and for those providing reference materials, ISO Guide 34) which must cover the calibration and measurement capabilities that are to be declared through the CIPM MRA. Due to geographic, technical and organizational differences between the RMOs, each has tailored its quality management system review process to be optimal for its own region, whilst remaining within the JCRB guidelines. For example in APMP, where most laboratories are also accredited, the review process is closely integrated with the regional accreditation system. Assessors are jointly chosen and the assessment evidence used for both accreditation and the CIPM MRA, avoiding duplication of effort.

Following satisfactory participation in appropriate comparisons the participating institutes declare their CMCs, which are subject to two rounds of peer review. The CMCs are firstly reviewed within the RMO of which the declaring institute is a member. After any comments have been resolved at the RMO level, the CMCs are subject to a second round of inter-regional review by the other RMOs. This second interregional review is carried out in parallel by the various RMOs. The outcome consists of internationally recognized statements of the measurement capabilities of the participating institutes.

All the data are openly available in the CIPM MRA database [3], which is maintained by the BIPM and publicly available on the internet. The database, widely known as the KCDB (the BIPM key comparison database, which goes far beyond just details of the comparisons), comprises four parts:

• Appendix A listing the signatory NMIs together with any designated institutes;
• Appendix B with full details of the registered comparisons;
• Appendix C listing the internationally approved Calibration and Measurement Capabilities (CMCs);
• Appendix D listing Key Comparisons (although this is somewhat redundant in practice given the information in Appendix B).

The overall coordination is by the BIPM under the authority of the CIPM. The Consultative Committees of the CIPM, the RMOs and the BIPM are responsible for carrying out the key and supplementary comparisons. The Joint Committee of the Regional Metrology Organizations and the BIPM (JCRB) [5] is charged with coordinating the activities among the RMOs, particularly with regard to the inter-regional CMC review.

Key and Supplementary Comparisons

The scientific comparisons are the basic building block that enables NMIs to show they are ‘getting the right answer’ and appropriately estimating the uncertainties of their results. The subjects of key comparisons are decided by the CCs, and these same subjects are often also adopted as key comparisons by the RMOs, which may in addition also undertake supplementary comparisons to address specific measurement requirements.

At the launch of the CIPM MRA a formal transition period was defined [1] as running until the first wave of key and supplementary comparisons had been completed. In this period some flexibility was exercised, recognizing that it took some time for the processes to catch up with a backlog of reviews, of, for example, the quality systems. The transition period was deemed to have been concluded at the end of 2003 (and at the end of 2005 for Chemistry).

In these early years the NMIs, RMOs, and the CIPM Consultative Committees were hard at work organizing and conducting the comparisons and analysing the results. At the same time, whilst some NMIs had formal quality systems, many did not, and had to begin developing a suitable system from scratch. All the quality systems then had to be taken through the RMO peer-review system. The NMIs and DIs also worked on developing their own CMCs and reviewing the CMCs of other NMIs and DIs.

By May 2004, just after the end of the transition period, 470 key comparisons had been registered in the KCDB [4] , among which 324 were conducted by the CCs and BIPM, and 146 were conducted by one of the five RMOs participating in the JCRB. Of these, about one fifth were conducted before the CIPM MRA was signed and so did not necessarily fully meet all procedural aspects; however their results were considered to provide “Provisional equivalence” which allowed them to be used to support CMC declarations.

Figure 1 shows the cumulative listing of comparisons registered in the KCDB. Since 2003, the average rate of registration of new key comparisons has been roughly constant at about 40 new key comparisons per year (Figure 2). A slight reduction of this rate can be seen in recent years, probably resulting from the strategic planning exercises carried out since 2013, which have led to some rationalization. It is clear that new comparisons continue to be needed, both because new capabilities need to be underpinned, and because the original comparisons become old and need to be repeated.

The number of RMO supplementary comparisons, on the other hand, shows a modest but steady increase, perhaps driven by RMO members who are not able to operate at the highest levels of metrology and who are developing capability and needing to participate in comparisons. Supplementary comparisons are typically conducted for two main reasons. Firstly, an NMI may miss a comparison cycle and need to demonstrate its capabilities. Secondly, the RMO may have specific regional needs that are not covered by the key comparisons. Key comparisons only address the key techniques or ‘pinning points’, an RMO may wish to undertake a comparison related to a more specific technique that is not considered key. This may be because they have member NMIs who are not able to participate in the high-level metrology addressed in key comparisons, but nonetheless need to be able to support CMC claims.

Figure 1. Total number of key comparisons and supplementary comparisons registered in the KCDB [4].

Figure 2. Number of new comparisons registered in the KCDB over the one-year period ending at the date indicated on the x-axis [4].

Calibration and Measurement Capabilities (CMCs)

CMCs are initially reviewed by the NMI creating them and then by experts drawn from the relevant technical committee of the supporting RMO. Once the initiating RMO is satisfied, the CMCs are submitted into an inter-regional review process. The CMCs, usually in batches, are posted on a dedicated page of the JCRB website. The JCRB has instituted deadline requirements in the inter-regional review to prevent it from stalling due to inaction by the reviewing RMOs. RMOs must indicate their intention to review within three weeks of the CMC file being posted, using a standardized online process, otherwise their review rights are lost. There is no fixed deadline for reviews because the size of a batch and the complexity of the CMCs within any given batch vary enormously. RMOs set and post their own review deadlines, but having done so they must respect them. Historically, all batches have been reviewed by all RMOs, but more recently some CCs have organized themselves by dividing up the review work to reduce the amount of redundancy in the review process. However, all CMCs must be reviewed by experts from at least one additional RMO. In the vast majority of cases, even today, CMCs are reviewed by more than one region at the inter-regional stage.

Figure 3. Evolution of the number of CMCs listed in the KCDB [4]

Figure 3 shows the total number of published CMCs as a function of time. By May 2004, just after the end of the transition period, the KCDB contained more than 17 000 CMCs, around two thirds of the number in the database today. By the time of the ten-year anniversary in late 2009 there were just over 21 000 CMCs published in the KCDB and the number of registered comparisons had doubled to 664 key comparisons and 218 supplementary comparisons.

The number of CMCs continued on an upward trend until March 2013, when the curve has flattened off. However, interpretation of the numbers is complicated by changes that have been made to the way some of the CMCs are formulated. In late 2004 EURAMET (then EUROMET) introduced the concept of using uncertainty tables for current and voltage transfer allowing uncertainty information to be displayed in a more succinct way and reducing the need for the NMIs to make multiple line entries. This resulted in a drop in the number of CMCs, but not of course in the amount of information in the KCDB. In 2013 EURAMET decided to adopt the uncertainty tables (sometimes referred to as the uncertainty matrix) across its entire portfolio of Electricity and Magnetism CMCs, and in 2015 APMP followed suit.

By 2005 it was realized that a mechanism was needed to allow for temporary suspension of CMCs. This became known, somewhat misleadingly, as ‘greying out’ of CMCs. For the KCDB users these greyed out CMCs are invisible and inaccessible, but they remain in the database ready to be reinstated when appropriate evidence of addressing the reason for suspension has been provided. Much later it was realized that some CMCs sat in this greyed out status for long periods, and formal procedures were introduced to handle both the greying out and the reinstatement or deletion of CMCs.

In 2004 at the suggestion of a number of NMIs a CIPM MRA logo [9] was adopted. This allows those NMIs which have been granted permission to include the CIPM MRA logo on calibration certificates covered by CMCs published in the KCDB. Much more recently this has been extended to verification certificates (particularly important in COOMET), and to Certified Reference Material (CRM) documentation, again provided they are covered by CMC entries in the KCDB.

The CIPM MRA today

As mentioned previously, the CIPM MRA has been signed by the representatives of 98 institutes – from 53 Member States, 41 Associates of the CGPM, and 4 international organizations – and covers a further 152 institutes designated by the signatory bodies [2].

The KCDB website receives approximately 11 000 unique visits per month [4] (discounting minor visits where little is examined in the database). Although not surprisingly the NMI community is the single largest community visiting the KCDB (see Figure 4), there are substantial numbers of external visitors too, most notably from calibration and test laboratories.

Figure 4.Who visits the KCDB? [4]

NMIs access the database for many reasons, for example to check and benchmark their own capability, to assess the state of the art during the CMC review process, or perhaps to source traceability for national standards that are not primary. Also, in many countries the NMI acts as a ‘portal’ for regulators and other users. That is to say the third party enquiry is addressed to the NMI, which in turn uses its knowledge of and familiarity with the database and its contents, together with its expertise in understanding measurement challenges, to provide advice to the client or customer. Consequently, many of the NMI database visits may be to service external enquiries.

As of 1 March 2015, the KCDB included a total of 23 969 CMCs [4]:

• 14 180 in General Physics,
• 4 022 in Ionizing Radiation, and
• 5 767 in Chemistry

The distribution of CMCs among the RMOs is very uneven, as can be seen below:

• AFRIMETS: 446
• APMP: 5 348
• COOMET: 2 264
• EURAMET: 10 737
• SIM: 4 925

The balance not included in the RMO distribution comes from the international organizations.As of 1 March 2015, the
KCDB covered 898 key comparisons, with the distribution of [4]:

• 89 from the BIPM,
• 436 from the CCs,
• 4 from AFRIMETS,
• 127 from APMP,
• 42 from COOMET,
• 147 from EURAMET, and
• 53 from SIM.

One of the wider objectives quoted in the CIPM MRA is to provide governments and other parties with a secure technical foundation for wider agreements related to international trade, commerce and regulatory affairs. The most obvious expression of this, and the widespread recognition and acceptance of the KCDB, was reflected in its inclusion as a reliable and convenient source of internationally accepted traceable measurements in the 2013 ‘ILAC-P10:01/2013. ILAC Policy on the Traceability of Measurement Results’ [10]. In this way the 49 000 accredited calibration and testing laboratories world-wide are all linked back into the international system, helping ensure the unbroken chain of measurements used by industry and wider society to the SI.

There are many examples of where the CIPM MRA has had a practical impact and a number of these have been summarized by KRISS, the Korean NMI. One example relates to a Korean manufacturer contracted to develop, manufacture and delivers two special oil offshore platforms to an oil consortium operating in the Russian Federation. Such platforms contain thousands of instrumentation loops, and in this case some 600 loops subject to state metrological control, comprising some 10 000 measuring instruments of approximately 60 different types. The requirement was for traceability to the national measurement standards of the Russian Federation, verified by Russian Federation Verification Officers, or by some body accredited and authorized by them. VNIIMS, the Russian NMI, relied on the joint participation of Russian and Korean NMIs in the CIPM MRA as justification for calibration to take place in Korea, saving 16 million $US. A similar building project, this time for an oil major operating in the USA, relied on the participation of KRISS and NIST in the CIPM MRA, allowing in country calibration leading to a saving of some 11 million $US. A third involved a Mexican automobile parts manufacturer, maintaining SI traceability through the Korean NMI, able to call on the CIPM MRA when supplying an Indian client leading to a saving of some 5 million $US. [11]

In addition the CIPM MRA has undoubtable helped provide a basis for comparing the performance of NMIs and hence raise the general standard of metrological performance in many participating states.

The CIPM MRA review and the way forward

In 2009 a Symposium was held to celebrate the ten-year anniversary of the CIPM MRA. A wide range of presentations were given by representatives from the organizations that rely on sound and widely accepted measurements. These included, amongst others, the WTO, the International Organization for Standardization (ISO), ILAC, the International Organization of Legal Metrology (OIML), the United Nations Industrial Development Organization (UNIDO) and Boeing.

The symposium was largely a celebration of the success of the CIPM MRA, but not surprisingly thoughts also turned to the future. In a session entitled ‘The CIPM MRA today and tomorrow: How the CIPM MRA might evolve to support metrology needs in other sectors of society’ representatives from the stakeholder and metrology community outlined the challenges for the future, including some speculation around the way the CIPM MRA itself might evolve. Whilst there was recognition of the need for and value of the CIPM MRA in facilitating easily accessible internationally accepted traceability, the NMI community questioned the sustainability of the CIPM MRA workload in the longer term. In a world requiring measurements across ever wider ranges with ever decreasing uncertainties, and involving ever more counties, there was little sign of easing of the drivers for the workload..

This 2009 discussion encapsulated the concerns of the major NMIs over the coming years. It is probably true to say that when first conceived the success and take-up of the CIPM MRA had not been fully envisaged. Whilst the initial workload had been properly anticipated, the ongoing workload had not. New areas of metrology, such as chemistry, together with emerging NMIs wishing to demonstrate their capability, as well as established NMIs expanding their scopes, all add to the leadership burden which falls disproportionately on a limited number of leading NMIs. This burden not only relates to the running of comparisons, but also to the peer review of quality management systems and the examination of CMCs.

In March 2013 the JCRB held a workshop on CMC review, which brought some useful improvements in efficiency but no substantive changes in scope or implementation. A discussion at the October 2013 meeting of NMI Directors and Member State Representatives, echoed in the CIPM, concluded that it was time for a deeper look and formally concluded “There is a need to review the effectiveness and efficiency of the CIPM MRA.” [7]

Around the same time and in parallel, a strategy exercise was undertaken (and published on the BIPM webpages) by each of the CIPM Consultative Committees. This exercise has already helped manage and rationalize the number of planned CC comparisons. No doubt there will be further efforts in the review to ensure the suite of comparisons is the optimal balance of generating confidence at a sustainable level of effort.

The CIPM began planning a workshop for the CIPM MRA stakeholders to address the sustainability of the CIPM MRA and, to ensure all Member States were clear regarding the objectives, drafted a resolution which in November 2014 was adopted as Resolution 5 at the 25th CGPM meeting [8].

The text of Resolution 5 (2014) [8] is as follows:

noting:

a workshop planned for 2015 to engage in a broad discussion of the CIPM MRA, involving: Directors of National Metrology Institutes, Member States representatives, representatives of RMOs and other relevant stakeholders concerning the benefits of the CIPM MRA, as well as establishing views on what works well, and what needs to be improved regarding its implementation,

invites

the Consultative Committees and the JCRB to continue their ongoing efforts to streamline operations within the existing framework, and to prepare for and contribute to the wider review in 2015,

the CIPM to establish a working group under the chairmanship of its President, with membership to be determined at the 2015 workshop, to conduct a review of the implementation and operation of the CIPM MRA,

The review will consider whether the CIPM MRA is meeting stakeholder needs overall, look for opportunities to simplify the whole system, as well as opportunities to improve the efficiency of the processes, procedures and tools, including the KCDB.

The CCs, JCRB/RMOs and the NMI Directors and other stakeholders are preparing for the review. Certainly some NMI Directors will be pressing for their concerns over the work load to be addressed, particularly in piloting comparisons, but also in the CMC review process. The CMC review currently has considerable redundancy built into it (as the review is first done in the region of the initiating NMI, and then in a second step, carried out in parallel by selected NMI experts from the other regions), so there is probably room for efficiency savings through reduced duplication of reviews. It is also obviously important that the ‘expert base’ of NMIs that are prepared to lead comparisons is broadened. There is a general feeling that those laboratories that have completed the learning curve and participated in many comparisons, publishing many CMCs, should now be prepared to volunteer and carry a greater share of the workload, particularly in the piloting of comparisons.

It is equally clear that the KCDB will be redesigned when the new needs are better known. At this stage we can predict that a minimum requirement will be for better data input tools and an improved KCDB search capability. The search facilities for the physics and chemistry areas of the KCDB are already treated separately, but could of course be separated further if required. There may also be merit in considering alternative ways for the data to be displayed for these disciplines. No doubt other ideas will be brought to the table. The BIPM, as the operator of the system, will input its ideas, under the guidance of the CIPM, but the main input will come from the NMI Directors, for they are the signatories of the CIPM MRA and they provide most of the resources necessary to make it work.

ILAC, representing the accreditation bodies and the calibration and test laboratories worldwide that require SI traceability, is also likely to make suggestions. Views differ on whether the accredited laboratory community should consider the CIPM MRA simply as a convenient way to demonstrate an internationally acceptable metrological traceability route to the SI or the preferred way. It is clear that metrology will continue to advance, as the exploitation of wider scientific developments requires the ability to measure at ever higher levels of accuracy. Consequently there is ongoing demand from the user community for the NMIs to continue to enhance their capability.

In conclusion

In conclusion it is clear that the CIPM MRA has been a huge success, helping to underpin free trade and improving the comparability of measurement worldwide. After 15 years of operation it is time for the implementation and operational aspects to be reviewed. A review workshop is scheduled for mid-October 2015 and preparations are well under way under the supervision of a CIPM ad hoc working group. At this workshop a formal review group will be established to make the key recommendations for change to ensure the sustainability of the CIPM MRA over the coming decades.

Authors Note: Any views expressed are those of the author, and do not represent the views of the BIPM or the CIPM.