BS EN ISO 16140‑4:2020

BSI Standards Publication

Microbiology of the food chain — Method validation Part 4: Protocol for method validation in a single laboratory

BS EN ISO 16140‑4:2020

BRITISH STANDARD

National foreword This British Standard is the UK implementation of EN ISO 16140‑4:2020. It is identical to ISO 16140‑4:2020. The UK participation in its preparation was entrusted to Technical Committee AW/9, Microbiology.

A list of organizations represented on this committee can be obtained on request to its committee manager. This publication does not purport to include all the necessary provisions of a contract. Users are responsible for its correct application. © The British Standards Institution 2020 Published by BSI Standards Limited 2020 ISBN 978 0 580 89054 3 ICS 07.100.30

Compliance with a British Standard cannot confer immunity from legal obligations. This British Standard was published under the authority of the Standards Policy and Strategy Committee on 30 September 2020. Amendments/corrigenda issued since publication Date

Text affected

BS EN ISO 16140‑4:2020

EN ISO 16140-4

EUROPEAN STANDARD NORME EUROPÉENNE EUROPÄISCHE NORM

August 2020

ICS 07.100.30

English Version

Microbiology of the food chain - Method validation - Part 4: Protocol for method validation in a single laboratory (ISO 16140-4:2020) Microbiologie de la chaîne alimentaire - Validation des méthodes - Partie 4: Protocole pour la validation de méthodes dans un seul laboratoire (ISO 16140-4:2020)

Mikrobiologie der Lebensmittelkette Verfahrensvalidierung - Teil 4: Arbeitsvorschrift für Einzel-Labor-Verfahrensvalidierung (ISO 161404:2020)

This European Standard was approved by CEN on 25 May 2020. CEN members are bound to comply with the CEN/CENELEC Internal Regulations which stipulate the conditions for giving this European Standard the status of a national standard without any alteration. Up-to-date lists and bibliographical references concerning such national standards may be obtained on application to the CEN-CENELEC Management Centre or to any CEN member. This European Standard exists in three official versions (English, French, German). A version in any other language made by translation under the responsibility of a CEN member into its own language and notified to the CEN-CENELEC Management Centre has the same status as the official versions. CEN members are the national standards bodies of Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway, Poland, Portugal, Republic of North Macedonia, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey and United Kingdom.

EUROPEAN COMMITTEE FOR STANDARDIZATION COMITÉ EUROPÉEN DE NORMALISATION EUROPÄISCHES KOMITEE FÜR NORMUNG

CEN-CENELEC Management Centre: Rue de la Science 23, B-1040 Brussels © 2020 CEN

All rights of exploitation in any form and by any means reserved worldwide for CEN national Members.

Ref. No. EN ISO 16140-4:2020 E

BS EN ISO 16140‑4:2020

EN ISO 16140-4:2020 (E)

European foreword This document (EN ISO 16140-4:2020) has been prepared by Technical Committee ISO/TC 34 "Food products" in collaboration with Technical Committee CEN/TC 463 “Microbiology of the food chain” the secretariat of which is held by AFNOR. This European Standard shall be given the status of a national standard, either by publication of an identical text or by endorsement, at the latest by February 2021, and conflicting national standards shall be withdrawn at the latest by February 2021. Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. CEN shall not be held responsible for identifying any or all such patent rights. According to the CEN-CENELEC Internal Regulations, the national standards organizations of the following countries are bound to implement this European Standard: Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway, Poland, Portugal, Republic of North Macedonia, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey and the United Kingdom.

Endorsement notice

The text of ISO 16140-4:2020 has been approved by CEN as EN ISO 16140-4:2020 without any modification.

BS EN ISO 16140‑4:2020 ISO 16140-4:2020(E) 

Contents

Page

Foreword......................................................................................................................................................................................................................................... iv

Introduction...................................................................................................................................................................................................................................v 1 Scope.................................................................................................................................................................................................................................. 1 2

3 4

5

6

7

Normative references....................................................................................................................................................................................... 1 Terms and definitions...................................................................................................................................................................................... 2

General principles of the single-laboratory detection or quantification method validation ..................................................................................................................................................................................................................... 4 4.1 General............................................................................................................................................................................................................ 4 4.2 Principles of the factorial approach....................................................................................................................................... 5 4.3 Principles of the conventional approach........................................................................................................................... 5

Technical protocol for validation — Factorial approach............................................................................................. 7 5.1 Qualitative methods............................................................................................................................................................................ 7 5.1.1 Single-laboratory method validation study against a reference method......................... 7 5.1.2 Single-laboratory method validation study without a reference method.................... 13 5.2 Quantitative methods...................................................................................................................................................................... 15 5.2.1 Single-laboratory method validation study against a reference method...................... 15 5.2.2 Single-laboratory method validation study without a reference method.................... 18 Technical protocol for validation — Conventional approach..............................................................................19 6.1 Qualitative methods......................................................................................................................................................................... 19 6.1.1 Single-laboratory method validation study against a reference method...................... 19 6.1.2 Single-laboratory method validation study without a reference method.................... 20 6.2 Quantitative methods...................................................................................................................................................................... 21 6.2.1 Single-laboratory method validation study against a reference method...................... 21 6.2.2 Single-laboratory method validation study without a reference method.................... 23 Summary of acceptability limits........................................................................................................................................................26

Annex A (informative) List of factors and factor levels for factorial method validation..............................27 Annex B (informative) Calculation of in-house reproducibility for qualitative methods based on the LOD50 study described in 6.1.2.3...................................................................................................................29 Annex C (informative) Example of a factorial single-laboratory method validation study for a quantitative method against a reference method.......................................................................................................30 Annex D (informative) Example of a factorial single-laboratory method validation study for a qualitative method against a reference method...........................................................................................................36 Annex E (informative) Example of a factorial single-laboratory method validation study for the variability of the LOD50 for a qualitative method without a reference method.....................40 Annex F (informative) Determination of precision if the artificially contaminated samples are unstable.............................................................................................................................................................................................................43 Annex G (informative) Protocol for single-laboratory validation of alternative methods for microbiological confirmation and typing procedures................................................................................................45 Bibliography.............................................................................................................................................................................................................................. 46

© ISO 2020 – All rights reserved



iii

BS EN ISO 16140‑4:2020 ISO 16140-4:2020(E) 

Foreword ISO (the International Organization for Standardization) is a worldwide federation of national standards bodies (ISO member bodies). The work of preparing International Standards is normally carried out through ISO technical committees. Each member body interested in a subject for which a technical committee has been established has the right to be represented on that committee. International organizations, governmental and non-governmental, in liaison with ISO, also take part in the work. ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of electrotechnical standardization.

The procedures used to develop this document and those intended for its further maintenance are described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the different types of ISO documents should be noted. This document was drafted in accordance with the editorial rules of the ISO/IEC Directives, Part 2 (see www​.iso​.org/​directives). Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of any patent rights identified during the development of the document will be in the Introduction and/or on the ISO list of patent declarations received (see www​.iso​.org/​patents). Any trade name used in this document is information given for the convenience of users and does not constitute an endorsement.

For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and expressions related to conformity assessment, as well as information about ISO’s adherence to the World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www​.iso​.org/​ iso/​foreword​.html. This document was prepared by Technical Committee ISO/TC 34, Food products, Subcommittee SC 9, Microbiology, in collaboration with the  European Committee for Standardization (CEN) Technical Committee CEN/TC 463, Microbiology of the food chain, in accordance with the Agreement on technical cooperation between ISO and CEN (Vienna Agreement). A list of all parts in the ISO 16140 series can be found on the ISO website.

Any feedback or questions on this document should be directed to the user’s national standards body. A complete listing of these bodies can be found at www​.iso​.org/​members​.html.

iv



© ISO 2020 – All rights reserved

BS EN ISO 16140‑4:2020 ISO 16140-4:2020(E) 

Introduction 0.1   The ISO 16140 series The ISO 16140 series has been expanded in response to the need for various ways to validate or verify test methods. It is the successor to ISO 16140:2003. The ISO 16140 series consists of six parts with the general title, Microbiology of the food chain — Method validation: — Part 1: Vocabulary;

— Part 2: Protocol for the validation of alternative (proprietary) methods against a reference method;

— Part 3: Protocol for the verification of reference methods and validated alternative methods in a single laboratory; — Part 4: Protocol for method validation in a single laboratory;

— Part 5: Protocol for factorial interlaboratory validation for non-proprietary methods;

— Part 6: Protocol for the validation of alternative (proprietary) methods for microbiological confirmation and typing procedures.

ISO  17468 is a closely linked International Standard, which establishes technical rules for the development and validation of standardized methods. In general, two stages are needed before a method can be used in a laboratory.

— The first stage is the validation of the method. Validation is conducted using a study in a single laboratory followed by an interlaboratory study (see ISO 16140-2, ISO 16140-5 and ISO 16140-6). In the case when a method is validated within one laboratory (as described in this document), no interlaboratory study is conducted.

— The second stage is method verification, where a laboratory demonstrates that it can satisfactorily perform a validated method. This is described in ISO  16140-3. Verification is only applicable to methods that have been validated using an interlaboratory study. In general, two types of methods are distinguished: reference methods and alternative methods.

A reference method is defined in ISO 16140-1:2016, 2.59, as an “internationally recognized and widely accepted method”. The note to entry clarifies that “these are ISO  standards and standards jointly published by ISO and CEN or other regional/national standards of equivalent standing”.

In the ISO 16140 series, reference methods include standardized reference (ISO and CEN) methods as defined in ISO 17468:2016, 3.5, as a “reference method described in a standard”.

An alternative method (method submitted for validation) is defined in ISO  16140-1:2016, 2.4, as a “method of analysis that detects or quantifies, for a given category of products, the same analyte as is detected or quantified using the corresponding reference method”. The note to entry clarifies that: “The method can be proprietary. The term ‘alternative’ is used to refer to the entire ‘test procedure and reaction system’. This term includes all ingredients, whether material or otherwise, required for implementing the method.”. This document, ISO 16140-4, addresses validation within a single laboratory. The results are therefore only valid for the laboratory that conducted the study. In this case, verification (as described in ISO 16140-3) is not applicable. ISO 16140-5 describes protocols for non-proprietary methods where a more rapid validation is required or when the method to be validated is highly specialized and the number of participating laboratories required by ISO  16140-2 cannot be reached. This document and ISO 16140-5 can be used for validation against a reference method. This document (regarding qualitative and quantitative methods) and ISO 16140-5 (regarding quantitative methods only) can also be used for validation without a reference method. © ISO 2020 – All rights reserved



v

BS EN ISO 16140‑4:2020 ISO 16140-4:2020(E)  The flow chart in Figure 1 gives an overview of the links between the different parts mentioned above. It also guides the user in selecting the right part of the ISO 16140 series, taking into account the purpose of the study and the remarks given above.

Figure 1 — Flow chart for application of the ISO 16140 series NOTE In this document, the words “category”, “type” and/or “item” are sometimes combined with “(food)” to improve readability. However, the word “(food)” is interchangeable with “(feed)” and other areas of the food chain as mentioned in Clause 1.

ISO 16140-6 is somewhat different from the other parts in the ISO 16140 series in that it relates to a very specific situation where only the confirmation procedure of a method is to be validated [e.g. the biochemical confirmation of Enterobacteriaceae (see ISO  21528-2)]. The confirmation procedure advances a suspected (presumptive) result to a confirmed positive result. The validation of alternative typing techniques (e.g. serotyping of Salmonella) is also covered by ISO 16140-6. The validation study in ISO  16140-6 clearly defines the selective agar(s) from which strains can be confirmed using the alternative confirmation method. If  successfully validated, the alternative confirmation method can only be used if strains are recovered on an agar that was used and shown to be acceptable within the validation study. Figure 2 shows the possibilities where an alternative confirmation method validated in accordance with ISO 16140-6 can be applied (see text in the boxes).

vi



© ISO 2020 – All rights reserved

BS EN ISO 16140‑4:2020 ISO 16140-4:2020(E) 

Figure 2 — Use of validated alternative confirmation methods (see ISO 16140-6) EXAMPLE

An example application of a validated alternative confirmation method is as follows.

An alternative confirmation method based on ELISA has been validated (in accordance with ISO  16140-6) to replace the biochemical confirmation for Salmonella as described in ISO  6579-1. In the validation study, XLD (mandatory agar in accordance with ISO 6579-1) plus BGA and a specified chromogenic agar (two optional agars for second plating in accordance with ISO 6579-1) were used as the agars to start the confirmation. The validated confirmation method can be used to replace the biochemical confirmation under the following conditions: — by laboratories using the ISO 6579-1; or

— by laboratories using an ISO 16140-2 validated alternative method that refers to ISO 6579-1 for confirmation; or — by laboratories using an ISO 16140-2 validated alternative method that starts the confirmation from XLD and/or BGA agar and/or the specified chromogenic agar. The validated confirmation method cannot be used under the following conditions:

— by laboratories using an ISO 16140-2 validated alternative method that refers only to agars other than those included in the validation to start the confirmation (e.g. Hektoen agar and SS agar only); or

— by laboratories using an ISO  16140-2 validated alternative method that refers only to a confirmation procedure that does not require isolation on agar.

0.2   Validation protocols in the ISO 16140 series

An interlaboratory validation study, in accordance with ISO 16140-2, requires at least eight laboratories for quantitative methods and at least ten laboratories for qualitative methods. ISO 16140-5 is intended to be used for interlaboratory studies comprising four to seven laboratories for quantitative methods and four to nine laboratories for qualitative methods. ISO 16140-5 can only be used for non-proprietary methods. Table 1 provides an overview of the different protocols.

© ISO 2020 – All rights reserved



vii

BS EN ISO 16140‑4:2020 ISO 16140-4:2020(E)  Table 1 — Overview of different validation protocols described in the ISO 16140 series Number of participating laboratories 1

4 to 7 (quantitative method)/ 4 to 9 (qualitative method) ≥ 8 (quantitative method)/  ≥ 10 (qualitative method)

With reference method

Without reference method

This document: This document: —   factorial (see 5.1.1 and 5.2.1), or  — factorial (see 5.1.2 and 5.2.2), or  — conventional (see 6.1.1 and 6.2.1) — conventional (see 6.1.2 and 6.2.2) ISO 16140-5: for non-proprietary methods only

ISO 16140-2: for the interlaboratory study part

ISO 16140-5: for non-proprietary quantitative methods only Not applicable

The aim of this document is to assess the performance of detection or quantification methods within a single laboratory, typically across a number of (food) categories and (food) types. Single-laboratory validation of alternative methods for microbiological confirmation and typing procedures can also be performed under certain conditions: the general principles are the same as those described in ISO 16140-6 for the validation of alternative (proprietary) methods for microbiological confirmation and typing procedures (except there is no interlaboratory study). Further information is given in Annex G. The protocols in this document only validate the method for the particular laboratory. A generalization to other laboratories is not within the scope of these protocols. However, extension to other laboratories is possible if this document is used as the first phase of validation of a reference method, to be followed by an interlaboratory study as described in ISO 17468.

If a reference method is available, the validation of a method is conducted by comparing the alternative method to the reference method. This allows inclusion of naturally contaminated samples in the validation process and thus provides a more realistic picture of the performance of the method. If no reference method is available, the validation process is based on samples with known contamination levels only. This document provides protocols for both situations. The general principles for single-laboratory validations of detection and quantification methods are the same as those described in ISO 16140-2 for the validation of alternative (proprietary) methods against a reference method. This document cannot be used without ISO  16140-1 or ISO  16140-2, as many definitions and procedures are given in these International Standards. In addition to the validation parameters described in ISO 16140-2, this document describes the calculation of in-house repeatability and in-house reproducibility. Calculation of these parameters is not required if an interlaboratory study is to be conducted after the single-laboratory validation (i.e. if the single-laboratory validation is only the first phase of validation). Reliability of performance parameters obtained with this document is comparable to ISO 16140-2. This also means that the workload associated with the technical protocols for the single laboratory is comparable with the method comparison study of ISO 16140-2. This document provides two strategies for the single-laboratory method validation of detection and quantification methods. The first strategy is based on a factorial approach while the second strategy uses the conventional approach derived from the protocols of ISO 16140-2. In addition, protocols for the determination of the in-house reproducibility for quantitative methods are described. The advantages of using a factorial approach, over the conventional approach, are that it takes into account specific conditions that the laboratory encounters during routine testing and provides more information on the factors (technicians, culture media, etc.) that vary within the laboratory across relevant (food) items, while using fewer samples to assess the performance of the method. The factorial approach offers assessment of the precision of quantitative methods. It allows computation of reliable and representative single-laboratory method validation parameters such as in-house reproducibility standard deviation, LOD50 or RLOD values because it provides information on the variability of these values under different measurement conditions. The factorial approach requires fewer test results in order to obtain similar or higher levels of reliability compared to the conventional approach.

viii



© ISO 2020 – All rights reserved

BS EN ISO 16140‑4:2020

INTERNATIONAL STANDARD

ISO 16140-4:2020(E)

Microbiology of the food chain — Method validation — Part 4: Protocol for method validation in a single laboratory 1 Scope

This document specifies the general principles and the technical protocols for single-laboratory validation of methods for microbiology in the food chain. The protocols in this document only validate the method for the laboratory conducting the study. This document is applicable to single-laboratory validation of:

— methods used in the analysis (detection or quantification) of microorganisms in: — products intended for human consumption; — products intended for animal feeding;

— environmental samples in the area of food and feed production, handling; — samples from the primary production stage;

— methods for the confirmation or typing of microorganisms. This validation will replace only the confirmation or typing procedure of a specified method (see Annex G). This document is, in particular, applicable to bacteria and fungi. Some clauses can be applicable to other (micro)organisms or their metabolites, to be determined on a case-by-case basis. Single-laboratory validation is required if an interlaboratory validation in accordance with ISO 16140-2 is not appropriate. Possible applications are: — validation of an in-house method;

— method evaluation study in the validation process of a reference method in accordance with ISO 17468; — extension of the scope of an ISO 16140-2 validated method, e.g. category extension or test portion size; — modifications of existing methods.

Single-laboratory validation is the second step in the standardization of a reference method (see  ISO  17468). It is only applicable to methods that are fully specified with regard to all relevant parameters (including tolerances on temperatures and specifications on culture media) and that have already been optimized.

2 Normative references

The following documents are referred to in the text in such a way that some or all of their content constitutes requirements of this document. For dated references, only the edition cited applies. For undated references, the latest edition of the referenced document (including any amendments) applies. ISO 6887 (all parts), Microbiology of the food chain — Preparation of test samples, initial suspension and decimal dilutions for microbiological examination © ISO 2020 – All rights reserved



1

BS EN ISO 16140‑4:2020 ISO 16140-4:2020(E)  ISO  7218, Microbiology of food and animal feeding stuffs — General requirements and guidance for microbiological examinations ISO 16140-1:2016, Microbiology of the food chain — Method validation — Part 1: Vocabulary

ISO 16140-2:2016, Microbiology of the food chain — Method validation — Part 2: Protocol for the validation of alternative (proprietary) methods against a reference method

3 Terms and definitions For the purposes of this document, the terms and definitions given in ISO 16140-1 and the following apply. ISO and IEC maintain terminological ), ), data=TEST.RESULTS.DATAFRAME)

Item1 = summary(fit)$coefficients[1]

Item2 = summary(fit)$coefficients[1]+summary(fit)$coefficients[2] (…)

ItemN = summary(fit)$coefficients[1]+summary(fit)$coefficients[N] MeanAcrossItems = mean(c(Item1, Item2,…, ItemN)) LOD50 = -log(0,5)/exp(MeanAcrossItems)

NOTE 2 An Excel®-based program4) is available for performing the LOD50 calculations from https://​standards​ .iso​.org/​iso/​16140/​-4/​ed​-1/​en.

Annex  B provides further information on the LOD50 study that allows the calculation of in-house reproducibility. 6.1.2.4 Calculation and interpretation for sensitivity

Calculation for sensitivity shall be conducted in accordance with ISO 16140-2:2016, 5.1.3.4. Use the test results of the LOD50 study at zero level and at high level. Test results at low (fractional) level are not used. All reference results at zero level are treated as negative results. At high level, they are treated as positive results.

NA represents the number of negative test results of the method at zero level and PD the number of positive test results at zero level. ND represents the number of negative test results of the method at high level and PA the number of positive test results at high level. Acceptability limits (ALs) per (food) category are as follows: ND = 3, PD = 1. If ND is larger than 3, or PD is larger than 1, the method cannot be validated. 6.1.2.5 Inclusivity/exclusivity study

Inclusivity/exclusivity testing shall be conducted in accordance with 5.1.1.5.

6.2 Quantitative methods 6.2.1

Single-laboratory method validation study against a reference method

6.2.1.1 General The conventional approach for the single-laboratory validation is based on the ISO 16140-2:2016, 6.1, method comparison study design. 4) Excel® is the trade name of a product supplied by Microsoft and is an example of a suitable product available commercially. This information is given for the convenience of users of this document and does not constitute an endorsement by ISO of this product. © ISO 2020 – All rights reserved



21

BS EN ISO 16140‑4:2020 ISO 16140-4:2020(E)  6.2.1.2 Relative trueness study The relative trueness study is a comparative study between the results obtained by the reference method and that of the alternative method. It shall be conducted in accordance with ISO 16140-2:2016, 6.1.2. 6.2.1.3 Accuracy profile study

The accuracy profile study compares the results obtained by the reference method to that of the alternative method. It shall be conducted in accordance with ISO 16140-2:2016, 6.1.3. 6.2.1.4 Limit of quantification study

For some alternative methods, it is appropriate to determine the limit of quantification (LOQ). The LOQ is only relevant, and shall be determined, when the measurement principle of the alternative method is not based on the visual observation of the target microorganism. The LOQ study, if applicable, shall be conducted in accordance with ISO 16140-2:2016, 6.1.4. 6.2.1.5 In-house precision study

In-house repeatability standard deviation and in-house reproducibility standard deviation are important performance characteristics of any measurement method. They describe precision of the alternative method under in-house repeatability conditions and in-house reproducibility conditions. One (food) item is selected per (food) category for the determination of the in-house repeatability standard deviation and the in-house reproducibility standard deviation. If the artificially contaminated (food) item is sufficiently stable, microbiologically, tests are conducted on eight different days by three or more technicians. No technician shall conduct tests on more than three days. However, if only two technicians are available, then each technician shall conduct tests on four days. If only one technician is available, this technician shall conduct all tests, but the method is then validated only for this technician. On each day, one of the technicians performs tests on five  replicates using the alternative method, yielding a total of 8 × 5 = 40 tests. In addition, five replicate tests are performed using the reference method on day one as a control. The in-house repeatability standard deviation is calculated as shown by Formula (5): sr ,in-house =

where    

22

yij

yj =

1 5

5

∑ yij i =1

1 32

∑ j=1 ∑ i=1 ( yij − y j ) 8

5

2

(5)

is the log10 measurement of the ith replicate on day j;

is the average over the five log10 measurements on day j.



© ISO 2020 – All rights reserved

BS EN ISO 16140‑4:2020 ISO 16140-4:2020(E)  The in-house reproducibility standard deviation is calculated as shown by Formula (6):

2 (6) sI ,in-house = s2A + sr,in-house

where

s2A =

and

y=

1 7

1 40

8

∑( y j − y)

2

j=1

1 − sr2,in-house 5

∑ i=1 ∑ j=1 yij 5

8

is the average over all 40 log10 measurements. If s2A < 0 , set s2A = 0 .

If the artificially contaminated (food) item is not sufficiently stable, microbiologically, the following options apply.

— Conduct the precision study based on the design described above. The calculated in-house reproducibility standard deviation will very likely be larger than the actual unknown in-house reproducibility standard deviation. Therefore, it can only be used as an upper limit of the actual variation. However, the calculated in-house repeatability standard deviation is not (much) affected by sample instability and can be used without limitation. — Conduct the precision study based on the design described above, but adjust the results for the underlying trend, see F.2. — Conduct the precision study based on the design described above but replace the eight days by eight different times. These eight different times shall comprise at least three different days, and within each day, different equipment or different batches/suppliers of media shall be used. — Conduct the precision study and adjust the results of the alternative method by the results of the reference method, see F.3. The design of the precision study shall be described in the validation study report.

NOTE 1 The variance components can be calculated more efficiently, with better precision, using the restricted maximum likelihood (REML) approach[13]. NOTE 2 Evaluation of the calculated in-house reproducibility standard deviation can be conducted using the criterion proposed by Reference [11]: target value ±0,25 log10 cfu/g. Since accuracy comprises precision, no additional evaluation of precision is required in this document.

6.2.1.6 Inclusivity/exclusivity study

Inclusivity/exclusivity testing shall be conducted in accordance with 5.2.1.8. 6.2.2

Single-laboratory method validation study without a reference method

6.2.2.1 General The conventional approach for the single-laboratory validation is based on the ISO 16140-2:2016, 6.1, method comparison study design. In the case of a validation without a reference method, the study has to rely on known contamination levels of artificially contaminated samples.

© ISO 2020 – All rights reserved



23

BS EN ISO 16140‑4:2020 ISO 16140-4:2020(E)  6.2.2.2 Relative trueness study The relative trueness study is determined using the known contamination levels of the samples. It shall be conducted in accordance with ISO 16140-2:2016, 6.1.2, where each test result of the reference method is replaced with the known contamination level of the artificially contaminated sample. 6.2.2.3 Accuracy profile study

The accuracy profile study shall be conducted in accordance with ISO 16140-2:2016, 6.1.3, where each test result of the reference method is replaced with the known contamination level of the artificially contaminated sample. In addition, the following adaptations shall be made. a) The calculation of the standard deviation of the reference method in Step 5 of ISO 16140-2:2016, 6.1.3.3, can be omitted, and the value replaced by 0 since the contamination levels are known and, hence, there is no variability.

b) Step 9 of ISO 16140-2:2016, 6.1.3.3, does not apply because the standard deviation of the reference values cannot exceed the threshold of 0,125. 6.2.2.4 Limit of quantification study

The LOQ study, if applicable, shall be conducted in accordance with ISO 16140-2:2016, 6.1.4. 6.2.2.5 In-house precision study

One (food) item is selected per (food) category for the determination of the in-house repeatability standard deviation and the in-house reproducibility standard deviation. If the artificially contaminated (food) item is sufficiently stable, microbiologically, tests are conducted on eight different days by three or more technicians. No technician shall conduct tests on more than three days. However, if only two technicians are available, then each technician shall conduct tests on four days. If only one technician is available, this technician shall conduct all tests, but the method is then validated only for this technician.

On each day, the technician performs tests on five replicates, giving a total of 8 × 5 = 40 tests over the eight days. The in-house repeatability standard deviation is calculated as shown by Formula (7): sr ,in-house =

where    

24

yij

1 yj = 5

5

∑ yij i =1

1 32

8

5

∑∑ ( yij − y j ) j=1 i =1

2

(7)

is the log10 value of the test result of the ith replicate on day j; is the average over the five test results on day j.



© ISO 2020 – All rights reserved

BS EN ISO 16140‑4:2020 ISO 16140-4:2020(E)  The in-house reproducibility standard deviation is calculated as shown by Formula (8):

sI ,in-house = s2A + sr2,in-house (8)

where

s2A =

and

y=

1 7

1 40

8

∑( y j − y)

2

j=1

1 − sr2,in-house 5

∑ i=1 ∑ j=1 yij 5

8

is the average over all 40 log10 measurements. If s2A < 0 , set s2A = 0 .

If the artificially contaminated (food) item is not sufficiently stable, microbiologically, the following options apply.

— Conduct the precision study based on the design described above. The calculated in-house reproducibility standard deviation will very likely be larger than the actual unknown in-house reproducibility standard deviation. Therefore, it can only be used as an upper limit of the actual variation. However, the calculated in-house repeatability standard deviation is not (much) affected by sample instability and can be used without limitation. — Conduct the precision study based on the design described above but adjust the results for the underlying trend, see F.2. — Conduct the precision study based on the design described above, but replace the eight days by eight  different times. These eight different times shall comprise at least three different days. Different equipment or different batches/suppliers of media shall be used for each day. The design of the precision study shall be described in the validation study report.

In-house repeatability and in-house reproducibility standard deviations are used to describe precision of the alternative method. If the in-house reproducibility standard deviation is larger than 0,5, the method cannot be considered to be validated. NOTE 1 The variance components can be calculated more efficiently, with better precision, using the REML approach. NOTE 2 Evaluation of the calculated in-house reproducibility standard deviation can be conducted using the criterion proposed by Reference [11]: target value ±0,25 log10 cfu/g. Since accuracy comprises precision, no additional evaluation of precision is required in this document.

6.2.2.6 Inclusivity/exclusivity study

Inclusivity/exclusivity testing shall be conducted in accordance with 5.2.1.8.

© ISO 2020 – All rights reserved



25

BS EN ISO 16140‑4:2020 ISO 16140-4:2020(E) 

7 Summary of acceptability limits Table 8 provides the overview of acceptability limits (ALs). Criteria applied in the inclusivity/exclusivity study are not included. The organization of the table is based on the following method and study characteristics: — qualitative or quantitative method;

— factorial or conventional study design; — with or without reference method; — paired or unpaired study.

Table 8 — Overview of acceptability limits

Study

Method

Reference method

Paired/ unpaired

 

 

 

 

     

 

 

 

 

 

     

 

 

Quantitative  

 

 

 

Qualitative

 

 

  Conventional      

 

Yes

Qualitative

 

 

 

 

Factorial

26

 

     

Quantitative  

 

 

No  

Yes No    

Yes  

No  

Yes No

Acceptability limit (AL) (ND–PD) ≤ AL in accordance with Table 5

Unpaired RLOD ≤ 2,5  

Paired

Factorial difference of RLOD: |d|