Oct 22, 2019 Last Updated 1:22 PM, Sep 18, 2019

MEASUREMENT UNCERTAINTY REQUIREMENTS FOR CONFORMITY ASSESSMENT BODIES

 

1.0      INTRODUCTION

Uncertainty of measurement is defined as a “parameter, associated with the result of a measurement that characterizes the dispersion of the values that could reasonably be attributed to the measurand” (the measurand is the particular quantity subject to measurement) (International Vocabulary of Metrology, 2008). 

There is growing recognition of the importance of Measurement Uncertainty (MU) to assuring the reliability of measurement.  This is reflected in the standards ISO/IEC 17025 and ISO 15189, which contain specific requirements concerning these issues.  The reactions to these developments range from interpretation at a minimal level, to extensive calculation of the uncertainty associated with each contributing factor. Uncertainty of measurement is best estimated within the individual laboratory/measurement environment. All factors which will have a significant influence on the test or measurement result must be included in the estimation process.  There must be suitable programs utilizing reference standards, instruments and materials to ensure on-going and adequate quality control and repeatability and reproducibility of methods and equipment over time.  In many instances, it will be possible to utilize quality control data in assessing uncertainty components such as precision.  Where these data are not available, it may be necessary to carry out precision studies or to rely on published information about the method or portions of it until the CAB can obtain its own estimates.  The ILAC-G17:2002 should be used as a guidance document as well to supplement the content of this document. 

2.0     JANAAC APPROACH 

CABs seeking accreditation are required to abide by the following:

  1. The CAB must have a procedure for identifying the sources of uncertainty associated with testing methods on their Scope of Accreditation.
  2. This procedure must identify the mechanism used for recording and identifying the major components contributing to the uncertainty and where applicable, present the calculations used for estimating the measurement uncertainty for the test method.
  3. The components of uncertainty shall be identified for all applicable methods, and accompanied by reasonable estimates of their magnitude.
  4. The estimate of the measurement uncertainty can be determined from reference or control samples, from method validation data, or from combining the individual components.
  5. The CAB must consider risks associated with any statistical assumptions made in applying the decision rule to the results reported.

Laboratories should be validating methods, using Certified Reference Materials for calibration, carrying out internal Quality Control (QC) checks, and participating in Proficiency Testing (PT), all of which help to evaluate MU.  Strategies that make use of validation, QC and/or PT data that are commonly available to testing labs can be used.  The degree of rigour required in addressing the issues should be driven by customer requirements. 

3.0     MEASUREMENT UNCERTAINTY CATEGORIES 

Following are the measurement uncertainty categories (I – V below) for the tests identified on the laboratory’s proposed scope of accreditation: 

  1. Qualitative or semi-quantitative tests for which measurement uncertainty budgets will not be required.
  2. Published or standard test methods that specify limits to the values of the major sources of uncertainty of measurement and specify the form of presentation of calculated results. In such cases, the laboratory is considered to have satisfied this clause by following the test method and reporting instructions in their entirety.
  • Chemical, environmental or biological test methods based on published regulatory or consensus methods (e.g. AOAC, ASTM, APHA/AWWA, EPA) for which the measurement uncertainty is not defined in the method. For these types of tests, uncertainty estimates are required and can be determined using appropriate, published guidance documents such as the ISO/IEC “Guide to the Expression of Uncertainty in Measurement (GUM)”, Eurochem/CITAC Guide documents, for example.  Also, there may be test methods that have no significant sources of uncertainty other than random error and when this is the case, determining the random error satisfy the requirement for uncertainty estimation. 
  1. Test methods that need identification of the major components of uncertainty and a reasonable estimate of measurement uncertainty.
  2. Test methods that need identification of all components of uncertainty and detailed measurement uncertainty budgets calculated in accordance with published methods that are consistent with those described in the ISO “Guide to the Expression of Uncertainty in Measurement”.

 

4.0     LABORATORY DEVELOPED METHODS 

Laboratory-developed methods require validation per the relevant clauses of ISO/IEC 17025, and ISO 15189 .  As part of this validation, the significance of the measurement components, or the significance of the modifications of the measurement components from the standard test method, must be considered so that the appropriate measurement uncertainty category for the laboratory-developed method can be identified. 

Note that in ISO/IEC 17025:2005 Clause 5.10.3.1 c, and in ISO/IEC 17025:2017 Clause 7.8.3.1 c in order to determine if the uncertainty affects compliance to a specification limit uncertainty must be estimated.  Categories III, IV or V apply to these types of methods. 

5.0       PROCEDURE FOR ASSESSING MEASUREMENT UNCERTAINTY DURING ACCREDITATION 

  1. Prior to scheduling the initial assessment for an applicant laboratory and as part of the document review, the assessor(s) will review the draft scope(s) of accreditation, and the estimation of uncertainty in accordance with the categories III, IV and V above.
  2. The CAB is required to provide estimations of measurement uncertainty for the test within the requested scope prior to the assessment. Similar testing methods for a given technology may share the same budget 
  3. The assessor will confirm the presence of a written procedure for applying measurement uncertainty estimations. There must also be personnel knowledgeable in estimating measurement uncertainty for the full scope(s) of accreditation.  The knowledgeable person(s) who develops the uncertainty budgets, and tools for development of software for its determination may be an employee of the larger organization or some other outside source.  If an outside source is used, they must be readily accessible to the laboratory during the assessment.  The laboratory must have personnel who are skilled in the proper application of the uncertainty budgets, and tools for its determination.  Finally, if measurement uncertainty budgets were requested prior to the assessment they will be reviewed for their technical validity and correctness. 
  4. The assessor shall document the laboratory’s level of compliance with the requirements of the standard.
  5. If the laboratory has satisfactorily addressed the requirements of the standard and has generated technically valid measurement uncertainty budgets the laboratory will be considered compliant with the standard for the purpose of accreditation.
  6. If any budgets requested are not adequate, deficiencies must be duly noted and responded to prior to accreditation.
  7. If at any stage in the accreditation cycle, there are changes in the operations that will impact the MU budget established, then the CAB is required to re-calculate the measurement uncertainties which will be evaluated at the next assessment visit.

6.0     REPORTING MEASUREMENT UNCERTAINTY 

Measurement uncertainty needs to be estimated for all methods in Category III, but it needs to be reported only when one or more of the following conditions occurs: 

  1. when requested by the client;
  2. when required by specification or regulation; and/or
  3. when the result is being used to determine conformance with a specification limit.

 

In these cases, the laboratory must report the expanded uncertainty in the same units as the measurement result and with the same number of significant digits as the reported value.  The coverage factor must be included in the uncertainty statement.  If the MU was estimated using relative standard deviations or percentage relative standard deviations, the percentage must be transformed into the reported units prior to reporting the uncertainty. 

If the method has a known bias and this bias was not adjusted (for example, adjustment for recovery), this bias should be reported in addition to the result and the uncertainty.  For example, a measurement method has an average recovery of 89% of the target analyte, and the expanded measurement uncertainty has been estimated as 2.3% at levels below 300 ppm.  A test result is 210 ppm, and the result is used to prove conformance with a specification limit of 300 ppm. 

The result could be reported as follows: 

Sample result = 210 ppm.  The expanded uncertainty of this result is ± 5 ppm, with a coverage factor of 95%.  This method has an average recovery of 92%, or at this level, a possible bias of - 23 ppm.  

Suggested References:

ISO/IEC Guide 98-3:2008 Guide to the Expression of Uncertainty in Measurement

ISO 21748:2017 Guidance to the Use of Repeatability, Reproducibility and Trueness Estimated in Measurement Uncertainty Evaluation 

ISO 5725 series – Accuracy of measurement methods and results, parts 1 - 4 & 6JCGM 106:2012- Evaluation of measurement data - the role of measurement uncertainty in conformity assessment. 

ILAC-P14:01/2013 ILAC Policy for Uncertainty in Calibration

Notice of Transition to
ISO/IEC 17011:2017

    JANAAC is currently in transition to the new ISO/IEC 17011:2017 standard. As such, all our accredited CABs and new applications will be informed in a timely manner of any accreditation requirements which may be affected. Please contact JANAAC for further information or clarifications.



    Notice of Transition to
    ISO/IEC 17025:2017

    ILAC, in consultation with ISO, has agreed on a three-year period for laboratories that demonstrate conformity to the ISO/IEC 17025 standard, to make the transition from ISO/IEC 17025:2005, to the ISO/IEC 17025:2017 version of the standard. A communique has been issued in this regard. JANAAC, in keeping with this communique, requires its accredited laboratories to make this transition by November 30, 2020. After this date, accreditation to ISO/IEC 17025:2005 will no longer be valid. During this transition period, both revisions of the standard are equally valid and applicable. Testing and calibration laboratories may continue to submit applications for accreditation to the ISO/IEC 17025:2005 standard up to December 31, 2018, in which case, initial assessments must be conducted by March 31, 2019. After December 31, 2018, JANAAC will no longer be accepting applications for ISO/IEC 17025:2005 accreditation. After March 31, 2019, all assessments (initial, re-assessments and interim assessments, including scope extensions) will be conducted using the revised standard, ISO/IEC 17025:2017. If the date of reassessment or interim on-site assessment (surveillance) occurs before Nov 30, 2020, and after December 31, 2018 the laboratory will be assessed against the ISO/IEC 17025:2017 standard, in accordance with international practice. All accredited and applicant testing and calibration laboratories are required to purchase a copy of the revised standard at the earliest opportunity and to commence the transition process. Please contact JANAAC for any further information and clarification required.