1
EXPERT COMMITTEE ON BIOLOGICAL STANDARDIZATION Geneva, 17 to 20 October 2017
Collaborative Study to establish the 1st WHO International Standard for Human Herpes Virus 6B (HHV-6B) DNA for nucleic acid amplification
technique (NAT)-based assays
Sheila Govind, Jason Hockley, Clare Morris and the *Collaborative Study Group Division of Virology and Biostatistics. National Institute of Biological Standards and Control,
Blanche Lane, South Mimms, Potters Bar, Hertfordshire. EN6 3QG. United Kingdom.
*see Appendix I
NOTE:
This document has been prepared for the purpose of inviting comments and suggestions on the proposals contained therein, which will then be considered by the Expert Committee on Biological Standardization (ECBS). Comments MUST be received by 18 September 2017 and should be addressed to the World Health Organization, 1211 Geneva 27, Switzerland, attention: Technologies, Standards and Norms (TSN). Comments may also be submitted electronically to the Responsible Officer: Dr C M Nübling at email: nueblingc@who.int
© World Health Organization 2017
All rights reserved. Publications of the World Health Organization are available on the WHO web site (www.who.int) or can be purchased from WHO Press, World Health Organization, 20 Avenue Appia, 1211 Geneva 27, Switzerland (tel.: +41 22 791 3264; fax: +41 22 791 4857; e-mail:
bookorders@who.int).
Requests for permission to reproduce or translate WHO publications – whether for sale or for noncommercial distribution – should be addressed to WHO Press through the WHO web site: (http://www.who.int/about/licensing/copyright_form/en/index.html).
The designations employed and the presentation of the material in this publication do not imply the expression of any opinion whatsoever on the part of the World Health Organization concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted lines on maps represent approximate border lines for which there may not yet be full agreement.
The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or recommended by the World Health Organization in preference to others of a similar nature that are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters.
All reasonable precautions have been taken by the World Health Organization to verify the information contained in this publication. However, the published material is being distributed without warranty of any kind, either expressed or implied. The responsibility for the interpretation and use of the material lies with the reader. In no event shall the World Health Organization be liable for damages arising from its use. The named authors alone are responsible for the views expressed in this publication.
Summary
An international collaborative study was conducted to establish the 1st WHO International Standard for use in the standardisation of Human Herpes virus 6 (HHV-6) nucleic acid
amplification technology (NAT) assays. Two candidate freeze-dried HHV-6 virus preparations, species 6B Strain Z-29 (Sample A) and species 6A Strain GS (Sample B), were formulated in 10mM Tris-HCl pH 7.4, 0.5% Human serum albumin (HSA), 2.0% D-(+)-Trehalose dehydrate, and analysed by 26 laboratories from 12 countries, each using their routine NAT-detection assays for HHV-6 viral load determination.
A difference of up to 3.08 log10 copies/ mL in the quantitative mean potency estimates were reported across the laboratories for each of the candidates and their liquid equivalents, as well as for 5 clinical samples included for a limited commutability assessment. The agreement between the laboratories was improved when the potencies were expressed relative to either of the
candidate preparations. The evaluations of both candidates were very similar however of the two candidates, Sample A is most appropriate for use as an International Standard for HHV-6 DNA detection assays that are primarily employed for HHV-6B associated conditions.
The results from the accelerated thermal degradation stability studies performed at 3 months and 6 months have demonstrated that the candidate material (15/266) is stable at temperatures used for storage (-20°C) and temperatures 4°C, 20°C, 37°C and 45°C reflecting ambient temperature fluctuations encountered during global shipment. Further real-time stability analyses will ensue to assess the long-term stability of the candidate.
Based upon the the returned dataset, it is proposed that candidate Sample A (NIBSC code
15/266) be established as the 1st WHO International Standard for HHV-6B DNA for nucleic acid amplification technique (NAT)-based assays with an assigned potency of 7.75 log10 IU/ mL per ampoule.
Introduction
Human herpes viruses 6 (HHV-6) are memebers of the beta-subfamily of the Herpes viruses.
HHV-6A (strain GS) was first isolated in 1986 from the peripheral blood of patients with AIDS and lymphoproliferative disorders in the USA [1]. In subsequent years two further isolates strain U1102 and strain Z-29 were independently isolated from AIDS patients in Africa from Uganda and Zaire, respectively. Since their identification they have been catagorised as distinct variants 6A and 6B, however in 2012 they were recognised as separate viruses HHV-6A and HHV-6B [2].
The seroprevalence of HHV-6B in the human population is between 90-95%, primary infection is acquired within the first 2 years of childhood [3,4], causing in most cases a self-limiting febrile illness exanthema subitum resulting in a characteristic rash [5]. Following primary infection the virus persists in the salivary glands and establishes latency in monocytes and macrophages.
The clinical burden resides in the opportunistic reactivation of the virus, predominantly HHV- 6B, in immunocompromised transplant patients, particularly hematopoietic stem cell transplantation (HSCT) recipients. Following HSCT, HHV-6B reactivation is detected in 40% to 70% of patients. In this population HHV-6B viremia has been reported in association with a number of morbidities including organ dysfunction, delayed/impaired platelet recovery, myelosuppression, encephalitis, fever, rash, hepatitis, pneumonitis, gastroduodenitis, CMV reactivation, and graft-versus-host disease (GVHD) [6, 7].
Viral load detection in this patient group is widely achieved using quantitative polymerase chain reaction (qPCR). Many laboratories rely on in-house developed assays, whilst there are some commercially available assays. Many assays are still pan-species detection assays but a growing number of assays seek to provide a distinction between the two viral species. Due to the
heterogeneity of the assays being used there is difficulty in making meaningful inter-laboratory comparisons. This was highlighted in a study in 2008 by Flamand et al [8] and in 2010 by de Pagter et al concluding that “standardization needs to be improved to allow further elucidation of the clinical significance of HHV-6 loads” [9]. HHV-6 viral detection is further complicated by the incidence of inherited chromosomally integrated HHV-6 (iciHHV-6) that occurs in ~1% of the population, where the HHV-6 genome is present in every nucleated cell detectable as >10e6 copies/ mL, often mistaken for active infection, resulting in the administration of anti-viral therapy in the absence of actual infection [10, 11]. The call for standardized measurement of HHV-6 DNA levels has been raised in order “to define clinically-relevant viral loads and for meaningful cross-institutional comparisons of viral loads [12, 13].
This project was endorsed by the WHO ECBS in October 2010. Engagement with the HHV-6 &
7 Foundation in 2013 and 2015 permitted discussions as to the requirements for a HHV-6
reference standard. A whole virus preparation was proposed which would allow for the standard to go through the extraction as well as the amplification procedure comparable to a clinical sample. A high titre preparation was also proposed to enable the detection of high viral loads greater than106 copies/mL typically associated with iciHHV-6.
Aim of the study
The purpose of this study is to establish the 1st WHO International Standard for Human Herpes 6 virus (HHV-6) for use as a calibrant in NAT detection assays for viral load determination in clinical diagnostics in accordance to the guidelines set out in the WHO Technical report:
Recommendations for the preparation, characterization and establishment of international and other biological reference standards [14]. Two candidates were evaluated in a multicentre collaborative study, one for each HHV-6 species, and the results obtained have enabled a potency estimation to be assigned to the proposed candidate based on the range of NAT
detection assays in current use. The collated data has also been used to determine the suitability of the candidate material for use as a primary reference material by its ability to improve
agreement between the represented NAT-detection assays. Furthermore the evaluation conducted provides a preliminary assessment of the commutability of the candidate formulation.
Bulk Material and Processing Candidate Standards
Advice was sought through Kristin Loomis of the HHV-6 & 7 Foundation from scientific committee members of the HHV-6 & 7 Foundation as well as the wider HHV-6 community (Dr’s Ablashi, Flamand, Pellet, Gautheret-Dejean, Gompels, Jacobson and Agut) as to the selection criteria for suitable candidate strains. HHV-6A GS and HHV-6B Z-29 strains were recommended, as well as HHV-6B strain HST isolated from an Exanthama Subitum primary childhood infection (Yamanishi 1988) as another suitable candidate. However its propagation is best achieved in the MT4 Human T cell leukaemia line that is also co-infected with human T-cell lymphotropic virus type 1 (Hazard group 3). Collaboration with BIOCELL Diagnostics
(Baltimore, USA) was established to attempt propagation in alternative cell lines; MOLT-3, HSB-2 or SupT-1 which proved to be unsuccessful. An alternative HHV-6A strain, U1102, was also recommended that is used as a reference strain, however since HHV-6A is not the foremost species linked with reactivation associated with clinical sequelae in immunocompromised patients, this was not pursued further.
The candidate materials were prepared from viral stocks donated by Dr JL Murk, Medical Microbiology Department of Virology, University Medical Centre Utrecht, The Netherlands;
using cell culture supernatant from HHV-6B Z-29 infected MOLT-3 cells and cell culture supernatant from HHV-6A GS infected HSB-2 cells. A vial of frozen uninfected MOLT-3 cells was also donated whilst a T252 flask of growing CCRF-HSB-2 cells was sourced from the
European collection of cell cultures (Cat # NCPV 1311111v) for HHV-6A propagation. Both cell lines are non-adherent T lymphoblastoid lines derived from Human acute lymphoblastic
leukaemic patients.
Since the detection of HHV-6 is performed in multiple analytes, a universal buffer formulation was selected, to permit subsequent dilution into a clinical matrix pertinent to the analyte being
assayed by the end user. Both preparations of HHV-6 GS and Z-29 comprise of cell-free, live virus preparations from productively infected cell culture supernatants diluted into a universal buffer formulation comprising 10mM Tris-HCl pH 7.4, 0.5% Human serum albumin (HSA), 2.0% D-(+)-Trehalose dehydrate, that underwent lyophilisation to ensure long term stability of the product and to permit shipment at ambient temperatures.
Viral propagation of bulk material
Propagation of HHV-6 virus was achieved sequentially in permissive cell lines MOLT-3 and HSB-2 cells for HHV-6B and HHV-6A respectively. MOLT-3 cells (Passage 15) stored in liquid nitrogen were thawed in a 37°C water bath and resuspended in 10mL of growth medium, RPMI-1640 Medium supplemented with 10% Foetal bovine serum (FBS) (Sigma F2442 Origin USA), and 1% Antibiotics-Antimycotics and briefly centrifuged at 1250 RPM for 5 minutes to recover the cellular pellet which was resuspended in 25ml fresh media in a T75 flask and placed upright in a humidified incubator set at a 37⁰C with 5% CO2. Cell culture was maintained until sufficient doublings had occurred for viral inoculation and to maintain sufficient control cells for further use. For inoculation briefly a 100 fold dilution of the cell culture supernatant donated by Dr JL Murk was used to inoculate ~2 x 10e6 cell/ml of MOLT-3 in 2.0 ml of cell culture
medium (Infectious units were not determined). The cell culture flask was placed at an angle in a cell incubator for 4 hours, then resuspended into 10ml of fresh media with 5% FBS in an upright T25 and maintained in a humidified incubator at 37⁰C with 5% CO2 . 3 days post infection fresh MOLT-3 cells and media were added and the culture was split into five T75 flasks for bulk propagation. Initial signs of CPE (enlarged cells) were observed around 4 days post-infection.
Periodic testing of the supernatant using quantitative PCR was performed to track any increase in viral load. The infected culture was supplemented with fresh cells and maintained with fresh media until maximal >90% CPE was visible, approximately after 2 weeks post-infection. The total cell culture supernatant was stored at -80⁰C until required for bulk preparation. The propagation of HHV-6A in CCRF-HSB-2 cells was performed as described above, only cells were grown in IDMEM, 2mM Glutamine, 5% FBS and 1% Antibiotics-Antimycotics and infected cultures were harvested approximately 3 weeks post-infection.
Pre-fill testing
Viral DNA quantification
The concentration of the HHV-6 viral stocks was determined using the CE marked RealStar®
HHV-6 PCR Kit 1.0 in vitro diagnostic (IVD ) assay that quantifies and differentiates HHV-6A and HHV-6B specific DNA using a dual probe qPCR detection assay (Altona Diagnostics, Germany). Briefly, nucleic acid extractions were performed using 200 µL of test sample using the QIAamp DNA mini Kit (QIAGEN, Germany). Extractions were performed using the QIAcube, an automated extraction platform (QIAGEN, Germany). Extractions were performed with the inclusion of an internal control (IC) which is included in the RealStar® HHV-6 PCR Kit
1.0 intended to control for PCR inhibition. Purified nucleic acid samples were amplified by qPCR on the Mx3005P Real-time PCR instrument (Mx3005P QPCR System, Agilent Genomics) enabling parallel detection of HHV-6A (FAM fluorophore) and HHV-6B (Cy5 fluorophore).
Viral quantification was achieved with the inclusion of four plasmid based quantification standards, to generate two standard curves with a dynamic range of 101-104 quantifiable
copies/µL for both HHV-6A and HHV-6B (Altona Diagnostics, Germany). Reactions were set up according to manufacturer’s instruction using 10µl of test DNA sample added to 20µl of the combined master mixes A and B. Reactions conditions were performed as instructed:
denaturation at 95⁰C for 10 minutes followed by 45 cycles of two-step denaturation and
annealing at 95⁰C for 15 seconds and 58⁰C for 1 minute respectively. Data were analysed using the Mx3005P software v4.0. According to the manufacturers specification the following control parameter values of the standard curve were met to ensure valid quantitative diagnostic testing;
Slope - 3.00 / - 3.74, PCR Efficiency 85 % / 115 %, R square (R2) > 0.98. However in practice efficiencies of >95% were achieved.
Pre-fill testing
A number of excipient formulations, mainly based on previous formulations used for established WHO internationals standards, were tested at NIBSC in a non-infectious trial fill to test the ability of the formulation to form a robust homogenous cake upon lyophilisation. The formulations are listed above with the evaluation on the appearance of the “cake” post lyophilisation:
1] 10mM Tris-HCl pH 7.4, 2% HSA, 1% D-(+)-Trehalose dehydrate: loose cake, flaking; 2]
10mM Tris-HCl pH 7.4, 0.5% HSA: slightly shrunken, robust loose cakes; 3] 10mM Tris-HCl pH 7.4, 1% polygelin, 1% D-(+)-Trehalose dehydrate: Loose robust cakes; 4] 10mM Tris-HCl pH 7.4, 1% polygelin: slightly shrunken, loose robust cakes, 5] 10mM Tris-HCl pH 7.4, 0.5%
HSA, 1% D-(+)-Trehalose dehydrate: shrunken to half height; 6] 10mM Tris-HCl pH 7.4, 0.5%
HSA, 2% D-(+)-Trehalose dehydrate: robust cakes, loose; 7] 10mM Tris-HCl pH 7.4, 0.5%
HSA, 5% D-(+)-Trehalose dehydrate: adherent, robust cake, some cracking on top surface.
Whilst formulations 3 and 4 containing polygelin produced the most uniform cake appearance, polygelin is a bovine gelatin dervative that would likely cause importation issues for many countries, therefore formulations 6 and 7 were selected and further tested in a trial infectious fill using an AdVantage 2.0 BenchTop Freeze Dryer (SP Scientific). To ensure there was no
observed loss in potency upon lyophilisation viral DNA was extracted from the reconstituted freeze-dried material and the liquid unprocessed formulation and the viral load quantification was performed using the RealStar® HHV-6 PCR Kit 1.0 assay as described above. No
discernable loss of potency was observed using either formulation however it was decided to use the 10mM Tris-HCl pH 7.4, 0.5% Human serum albumin (HSA), 2.0% D-(+)-Trehalose
dehydrate formulation based on the reconsitiution appearance, compared with the 10mM Tris- HCl pH 7.4, 0.5% HSA, 5% D-(+)-Trehalose dehydrate preparation, where the former
reconstituted more efficiently into a clear colourless solution.
Preparation of bulk material and evaluation of materials
Both candidate bulks were prepared in an identical fashion. The universal buffer 10mM Tris- HCl pH 7.4, 0.5% Human serum albumin (HSA), 2.0% D-(+)-Trehalose dehydrate was prepared at NIBSC. The HSA used in the production of the candidate standards was derived from a licensed product (Zenalb® 20, a 200 g/L of human albumin solution 20% Solution) re-screened at NIBSC to be negative for anti-HIV-1, HIV-2, HBsAg, and HCV. Frozen aliquots of the viral supernatants were thawed using a 30⁰C water bath. Prior to dilution into universal buffer, the viral supernatants were cleared of cellular debris by centrifugation for 5 minutes at 1250 rpm and a 1:50 fold dilution was made in order that the bulk preparation should contain approximately ~1 x 10e8 copies/ml copies/mL in the case of the first candidate (15/266) and ~ 6 x 10e7copies/ml in the case of the second candidate (16/116), in a final volume of 6.0L of universal buffer.
200mL of each of the liquid bulks was divided into aliquots of 0.55ml, and 1.1ml in 2ml screw cap Sarstedt tubes and stored at -80⁰C for viral copy determination as well as for inclusion in the collaborative study panel to be tested alongside the equivalent lyophilised preparation. The liquid bulk was also tested for sterility. The remaining bulk volume of 5.8L was processed for
lyophilisation and designated NIBSC product code 15/266 for the HHV-6B candidate and 16/116 for the HHV-6A candidate.
Filling and lyophilisation of candidate standards
The filling and lyophilisation of both bulk materials was performed at NIBSC Standards Processing Division under ISO9001. The filling was performed in a Metall and Plastic GmbH (Radolfzell, Germany) negative pressure isolator that contains the entire filling line and is
interfaced with the freeze dryer (CS150 12m2, Serail, Argenteuil, France) through a ‘pizza door’
arrangement to maintain containment of the operation. The bulk material was kept at 18°C throughout the filling process, and stirred constantly using a magnetic stirrer. The bulk was dispensed into 5 mL screw cap glass vials in1.0 ml aliquots, using a Bausch & Strobel (Ilfshofen, Germany) filling machine FVF5060. The homogeneity of the fill was determined by on-line check-weighing of the wet weight against the dry weight, and vials outside the defined
specification were discarded. Filled vials were partially stoppered with halobutyl 14mm diameter cruciform closures and lyophilized in a CS150 freeze dryer. Vials were loaded onto the shelves at +4°C, then cooled to -50⁰C and held at this temperature for 2 hours. A vacuum was applied to 200 µb over 1 hour, followed by ramping to 100 µb over 1 hour. The temperature was then raised to -15°C, and the vacuum maintained at this temperature for 31 hours. The vacuum was lowered to 30 µb and the shelves were ramped to 25°C over 10 hours and this was then held for 20 hours before releasing the vacuum and back-filling the vials with nitrogen, produced by evaporation of liquid nitrogen with an analysis of 99.999% purity. The vials were then stoppered in the dryer, removed and capped in the isolator. The isolator was decontaminated with
formaldehyde before removal of the product. The sealed vials are stored at -20°C at NIBSC under continuous temperature monitoring for the lifetime of the product.
Post-fill testing:
Stability assessment of candidate products
To predict the stability of the freeze-dried materials, vials of the proposed HHV-6 IS candidates 15/266 and 16/116 are subject to accelerated degradation studies. This entails the storage of multiple vials of each candidate post production at -70⁰C, -20⁰C, +4⁰C, +20⁰C, +37⁰C and +45⁰C for up to 10 years. Periodically 3 vials are removed from each temperature and tested for viral potency using the real-time PCR method described above, to provide an indication of stability at the storage temperature of -20⁰C. Up to three evaluations are performed in the first year and once annually thereafter.
The assessment of residual moisture and oxygen content are critical parameters when considering the stability and shelf life of lyophilized products. Non-invasive moisture and oxygen determinations were made as follows. Vials of excipient only formulations for the proposed HHV-6 standards were prepared to be used to compare between destructive and non- invasive moisture analysis by near Infra-Red reflectance (NIR, Process Sensors MT 600P, Corby, UK). Results obtained from the non-infectious samples by NIR would then be correlated to coulometric Karl Fischer (KF, Mitsubishi CA-100, A1 Envirosciences, Cramlington, UK) to give % w/w moisture readings. Moisture determinations were compared against values from a standard curve, which was made using 10 vial replicates of non-infectious excipient only samples, which were subjected to varying exposure times (0, 5, 10, 15, 30, 45, 60 and 90 minutes) to atmospheric air, by removing screw caps and raising the stopper to the filling position for the designated period of time before the stoppers were fully re-inserted and the caps re-sealed. Subsequently, several vials of each time point were tested by coulometric Karl Fischer and the standard curve generated. Then 12 vials of the definitive batch containing lyophilised HHV-6 in excipient formulation were tested by NIR and their moistures assigned based on the calibration curve generated from the data from the non-infectious excipient vials.
Oxygen headspace content is an indicator of the success of the nitrogen back-filling process in the dryer and subsequent integrity of the seal on the vials. Oxygen headspace was measured using non-invasive headspace gas analysis (FMS-760 Lighthouse, Charlottesville, VA). This correlates the NIR absorbance at 760nm (for oxygen) based on a NIR laser source and is calibrated against equivalent vials, sealed with traceable oxygen gas standards. Calibration of the unit was achieved using 5ml screw capped vials containing oxygen standards 0% and 20%.
Homogeneity and genome integrity of candidate products
The freeze-dried candidates (15/266 and 16/116) were tested to determine the homogeneity of the detectable viral content of the lyophilised material post-production. Lyophilised samples were reconstituted in 1 mL of nuclease-free water (QIAGEN, Germany), mixed gently on a vortex and left for 20 minutes. 200 µL of reconstituted sample was used for extraction using the QIAamp Mini DNA kit and the extracted DNA used for amplification using the Altona
Diagnostics RealStar® HHV-6 PCR Kit 1.0 assay as described for Pre-fill testing.
Each candidate standard underwent full NGS analysis to verify the sequence integrity of the viral genome. To ensure a sufficient quantity of DNA for downstream NGS evaluation, DNA
extractions were pooled from 3 vials of lyophilised candidate and purified and concentrated using Agencourt AMPure XP (Beckman Coulter). The samples were mixed with 1 volume (100 µl) of AMPure beads and purified according to the manufacturer’s instructions. DNA was eluted in 12 µl of sterile DNase-free water and the concentrations were measured using the Qubit fluorometer using the Qubit dsDNA HS Kit (Invitrogen). Fragment libraries were prepared using the Nextera XT DNA Library Prep Kit (Illumina) and Nextera XT Index Kit V2, set D (Illumina) using 5 µl of each concentrated DNA sample (HHV-15/266 = 0.107 ng/µl and HHV-16/116 = 0.13 ng/µl). Libraries were sequenced on an Illumina MiSeq using a MiSeq Reagent Kit V2 500- cycle sequencing kit. Read alignment: Paired-end sequencing reads were mapped to references using CLC Genomics Workbench 10 (QIAGEN), using default read-mapping settings. Variants were detected using the Quality-based Variant Detection (1.4) tool; the minimum coverage was set to 10 and the minimum variant frequency to 50 %, variants were also required to be in at least one forward and one reverse read.
Collaborative Study Study samples
A total of nine study samples coded A-G, J and K, were prepared for evaluation in this study (Table 5). Participating laboratories were sent a questionnaire (Appendix II) prior to sample dispatch, to ascertain the types of clinical samples routinely assayed in their laboratory and to determine the quantity of sample required for their extraction procedure. Sample sets were thereby customised to each participant based on the responses received. All participants received the candidate HHV-6 materials in both the lyophilised and liquid state (Candidate 15/266;
Samples A and B, Candidate 16/116; Samples D and C respectively). In addition to the proposed IS candidates, 6 clinically relevant samples were also included in the panel for testing alongside the candidate materials. These were 3 whole blood samples; 2 of which were obtained from inherited chromosomally integrated HHV-6 individuals (Sample E iciHHV-6A and Sample F iciHHV-6B), kindly donated and aliquoted by the HHV-6 & 7 Foundation, USA. The third whole blood sample (G) positive for HHV-6 DNA was purchased from Cerba Specimen Services, France and was diluted into HHV-6 negative whole blood in order that sufficient quantity was available for distribution across the participating laboratories testing whole blood.
Information was not available regarding the clinical status of the donor. Sample J was kindly donated by the University of Washington, USA. A 0.9 mL aliquot of plasma obtained from transplant patient was diluted into negative plasma again to ensure sufficient quantity for study evaluation. The viral load of the neat sample was estimated by the donating laboratory to be ~6.5 log10copies/mL, which was diluted into 89 mL of HHV-6 negative plasma. A pseudo CSF sample was created by spiking Z-29 positive culture supernatant used for the IS preparation into HHV-6 negative CSF.
Whole blood and plasma obtained from the National Blood Service and CSF obtained from Raighmore Hospital, Scotland, were tested prior to dilution to confirm the absence of HHV-6 viral DNA and the samples were further tested post dilution with the positive clinical samples to confirm the viral load of the aliquots. Once diluted the HHV-6 positive whole blood (Sample G), plasma (Sample J) and CSF (Sample K) were aliquoted into smaller volumes 0.55ml, and 1.1ml into 2ml screw cap Sarstedt tubes, proportionate to the volumes required by participants to perform duplicate nucleic acid extractions across three separate experimental runs as instructed
Sample HHV-6 Sample ID
A Proposed HHV-6B candidate IS 15/266 B 15/266 Liquid bulk
C 16/116 Liquid bulk
D Proposed HHV-6A candidate IS 16/116 E iciHHV-6A
F iciHHV-6B
G Whole Blood HHV-6B J Plasma HHV-6B K Spiked CSF HHV-6B
in the collaborative study protocol (Appendix III). All liquid samples were stored at -80⁰C until required for dispatch to the study participants. All samples were shipped on dry-ice from NIBSC.
Design of the study protocol
Participants received six vials of each of the candidate samples and the clinical samples J and K for duplicate analysis of each sample using a fresh vial for each data point across three
independent runs. For the whole blood clinical samples E, F and G, three vials each were sent due to the limited quantity of these samples unless a quantity above 0.5ml was required per extraction. Sufficient volumes were sent to enable duplicate analysis of each sample for each data point across three independent runs.
Upon receipt, participants were directed to store the study samples either at -80⁰C (samples B, C, E, F, G, J and K) or at -20⁰C (samples A and D). Participants were directed to reconstitute the lyophilised samples (A and D) in 1ml of nuclease-free molecular grade water for a minimum of 20 minutes with occasional gentle agitation before use.
Participants performing quantitative analysis, were directed to test samples A and D for the first run, undiluted and in addition at a minimum of 3-4 serial ten-fold dilutions in a single sample matrix commonly used in their laboratory (e.g. whole blood, plasma etc.). For example, dilutions of 1/10, 1/100 etc. were suggested such that at least 2 of these dilutions should fall into the linear range of quantitation in their assay. For subsequent analysis participants were requested to test a minimum of two serial dilutions of sample A and D that fall within the quantitative linear range of their assay.
Those participants performing qualitative analysis were requested for the first assay to test samples A and D undiluted and then an additional minimum of 7 serial 1:10 fold serial dilutions in a single sample matrix commonly used in their laboratory (e.g. whole blood, plasma etc.) in order to determine the end point of detectable HHV-6 viral DNA. Participants were asked to select a single matrix for the dilution of both samples A and D such that the data would be comparable between the two lyophilised HHV-6 candidates. Participants were requested to ensure their data included at least two dilution points at which HHV-6 was no longer detectable.
For the two remaining qualitative assays, participants were requested to re-test the dilutions around the assay end point as determined in the first assay, and to include a minimum of two half-log serial dilutions either side of the determined end point dilution.
A results reporting form was provided to each participant. This included a table to provide details of extraction and amplification processes in the assays performed. Separate sheets were provided to submit data values from each assay performed. An example study protocol and results reporting form are shown in Appendix IV).
Participants
29 participants were recruited and randomly assigned a laboratory code by which to reference their data thereby assuring laboratory anonymity. However 2 of these were unable to proceed with the study. Study samples were therefore sent to 27 participants (listed in Appendix I) representing participants from research and clinical laboratories, manufacturers of HHV-6 NAT
in-vitro diagnostic (IVD) assays and reference materials, as well as an EQA provider. For the final analysis one dataset was excluded due to insufficient dilutions performed for end-point qualitative determination. The final evaluated dataset represents data from 26 participants from 12 different countries.
Statistical methods
Qualitative and quantitative assay results were evaluated separately. In the case of qualitative assays, for each laboratory and assay method, data from all assays were pooled to give a number positive out of number tested at each dilution step. A single ‘end-point’ for each dilution series was calculated, to give an estimate of ‘log10 NAT detectable units/mL` . It should be noted that these estimates are not necessarily directly equivalent to a genuine genome equivalent
number/mL [15].
In the case of quantitative assays, analysis was based on the results supplied by the participants, reported in copies/mL. For each sample and assay run, first the mean value at each dilution was calculated across the two vials. Then a single estimate of log10 copies/mL was obtained for each sample within an assay run by multiplying the responses by the corresponding dilution factors and then taking the arithmetic mean value across the results. A single estimate for each sample within the laboratory and assay method was then calculated as the arithmetic mean of the log10 estimates of copies/mL across assay runs.
Overall analysis was based on the log10 estimates of copies/mL or ‘log10 NAT detectable units/mL’. Overall mean estimates were calculated as the means of all individual laboratories.
Variation between laboratories (inter-laboratory) was expressed as standard deviations (SD) of the log10 estimates and % geometric coefficient of variation (GCV = {10s-1}×100% where s is the standard deviation of the log10 transformed estimates) of the actual estimates. Variation within laboratories and between assays (intra-laboratory) was expressed as standard deviations of the log10 estimates and %GCVs of the individual assay mean estimates.
Relative potencies were calculated as the difference in estimated log10 ‘units per mL’ (test sample – standard). All individual values are shown in Table 10 and 11.
Validity Criteria for quantitative assays
Samples A and D were diluted in a matrix, and the neat values were excluded from any calculations as not reflecting the intended use of these preparations. Dilutions below 1/10000 were also excluded as, in the five laboratories that performed dilutions that far, it appeared by visual inspection that further dilutions were not in the linear range of the assays. Nearly all laboratories performed dilutions within the range of 1/10 to 1/10000 and nearly all appeared linear by visual inspection. Thus this range was used for the calculations.
In order to determine whether it was appropriate to calculate a mean value using data from all dilutions, a GCV was calculated for the responses corrected for dilution and those samples with a GCV result of greater than 100% were deemed to not show sufficient evidence of dilutional linearity and removed from further calculations. Furthermore, the slope of sample log10
copies/mL by log10 dilution was calculated in each case and those samples where this slope did not lie between 0.80 and 1.25 were also deemed to not show sufficient evidence of dilutional linearity and were removed from further calculations.
For both of these criteria, more strict calculations were also investigated. A threshold of 50% for GCV was used, and an acceptable slope range of 0.90 to 1.11. In this case, the overall results were not found to differ greatly from those calculated but more laboratories’ results were excluded (data not shown).
Results and analysis
Validation of study samples
Post production data analysis was performed on the filled and lyophilised candidate materials sample A (15/266) and D (16/116) confirming the CV of the fill mass and the mean residual moisture and oxygen content, which were determined to be within acceptable limits for the preparation of WHO biological reference materials (Table 1 and 2).
Two thermal accelerated degradation assessments were performed at 3 and 6 month post- production on both candidate standards by assessing the change in potency if any, of detectable HHV-6 viral nucleic acid across the range of storage temperatures. 1 vial was tested at each of the storage temperatures in 3 independent assays. Each vial was tested either undiluted or then at a 10 fold dilution and a further 100 fold dilution. The values obtained at 3 and 6 months for 15/266 shows no observable drop in potency at temperatures up to +45⁰C. Instead the majority of estimates showed an increase in potency when stored at elevated temperatures. The same was also observed for data from candidate 16/116 stored for 3 months at elevated temperatures (Table 3a and 3b). However the 6 month data for candidate 16/116 indicates there may be some drop in potency with increasing temperatures (Table 3a and3b). The stability of the candidates was investigated further using Degtest-R v2 software (CombiStats, EDQM) which was unable to show any predicted loss of potency for candidate 15/266. However analysis of candidate 16/116 using the 6 month data points gives a predicted loss of 0.488% per year when stored at 20⁰C, which exceeds the WHO requirement of 0.1% per year (Table 3c). Further analysis are scheduled to be performed which will provide additional evaluations on the stability and suitability of both candidates for long term use.
The mean HHV-6 viral nucleic acid content from twelve randomly selected vials containing each of the candidate IS preparations were tested in duplicate to ensure for homogeneity across vials.
Each vial was reconstituted using nuclease-free water for 20 minutes with gentle agitation.
Aliquots of an undiluted and 1:10 diluted (using the universal buffer formulation 10mM Tris- HCl pH 7.4, 0.5% Human serum albumin (HSA), 2.0% D-(+)-Trehalose dehydrate ) were was extracted and amplified as described previously. An average of each duplicate was then used to determine the viral copy number (Table 4). An average of 8.05 log10 copies/mL (Range 8.006- 8.151) was obtained for candidate 15/266 and an average of 7.737 log10 copies/mL (Range 7.660-7.801) was obtained for candidate 16/116.
The mapping analysis for candidate 15/266 shows that 99.91 % of the genome is identical to the NCBI GenBank Accession AF157706 for HHV-6B strain Z-29. Variant calling analysis detected 88 potential variants, with a cut-off frequency of 50 % (26 in the DR regions and 62 in the central coding region). A large coverage peak in the region spanning the origin of viral
replication was evident from the mapping reads which upon further analysis was indicative of sequential tandem repeats within this region. Candidate 16/116 mapping analysis showed 99.72
% coverage to the NCBI GenBank Accession KJ123690 for HHV-6A strain GS. Variant calling analysis detected 103 potential variants, with a cut-off frequency of 50 % (79 in the DR regions
and 24 in the central coding region). Of note the DR regions could not be reliably mapped, which encompassed approximately 8,700 bp and 8, 200 bp at either end of the HHV-6B and 6A
genomes respectively. Of the SNPs identified only 1 was located to an amplification region used by one of the study participants, laboratory 28 who targeted U73 origin-binding protein where a single SNP was identified in 2231C>T (Ala744Val) at a frequency of 64.9 %.
Collaborative Study Data
Data were received from 26 laboratories from 12 different countries. From the 26 laboratories 36 datasets were returned for the lyophilised candidate materials Samples A and D, 34
quantitative and 2 qualitative datasets. For quantitative data almost all assays were conventional qPCR assays from which participants returned values as copies/mL. Two laboratories also
provided data in copies/106 cells for the whole blood samples E-G, these were not included in the analysis. Three laboratories returned digital PCR (dPCR) data reporting in copies/mL, laboratory 20 returned only dPCR data, and laboratory 25 returned duplicate dPCR analysis represented as 25b. Finally laboratory 13 performed a single dPCR analysis represented as 13b. Qualitative data evaluations were returned as either positive or negative detection and post analysis expressed as NAT detectable units /mL. Laboratory 4 data whilst assigned under quantitative, was obtained by performing an in-house qualitative assay reporting in Ct values, quantitative results were then calculated using a line equation using a stored calibration curve.
In general, participants performed their experimental runs using one assay method with the use of one matrix type for the dilution of Sample A and D. Exceptions were Participant 8 and 16 who performed their analysis using both plasma and whole blood. Participant 8 performed one dataset per analytical run as opposed to the requested duplicate analyses. Participant 16 performed duplicate analysis but in addition performed their amplification analysis on four different amplification platforms, which were considered as 4 different methods and therefore given the designation 16a -d.
Summary of assay methodologies
All participants used commercially available nucleic acid extraction kits, which included Abbott:
M2000SP DNA Extraction; Altona Diagnostics: Altostar Purification kit; BioMérieux:
NucliSENS easyMAG, NucliSENS miniMAG, ELITech Group: ELITe GALAXY 300
extraction kit; Invitrogen: Purelink Viral RNA/DNA mini kit; Roche: MagNA Pure 96 DNA and viral NA small volume kit, MagNA pure LC Total nucleic acid isolation kit; Sacace: Sacace DNA Sorb B extraction kit; Siemens: VERSANT® Sample Preparation 1.2 Reagents kit;
QIAGEN: QIAamp DNA Blood Mini kit, QIAamp DNA mini Kit, QIAamp DSP virus kit, Qiagen Ultra Sens Virus Kit, QIAamp 96 Virus QIAcube HT Kit, QIAgen EZ1 mini kit version 2.0, QIAsymphony DNA Mini Kit
The majority of the extraction protocols were performed on automated extraction platforms including: ABBOTT (M2000SP); Altona (AltoStar Automation System); BioMérieux (Nuclisens easyMAG Nuclisens miniMAG ), ELITech Group (ELITe GALAXY) Roche
( MagNa Pure 96), Siemens (VERSANT® kPCR Molecular System ), and QIAGEN:(QIAcube,
QIAcube HT, QIAGEN EZ1, QIAsymphony). Six participant’s datasets were performed manually.
15 datasets were obtained using laboratory developed amplification assays, compared with 16 datasets obtained using commercial quantitative amplification assays. Of the 31 evaluated datasets 18 were pan HHV-6 assays and 12 that were designed to distinguish between HHV-6A and HHV-6B. Six commercial assays were represented in this study, four assays are pan HHV-6 detection assays: XG-HHV6-MB ELITech, HHV6 ELIte MGB kit, Sacace HHV-6 Real TM Quant, Focus Diagnostics (DIaSorin Molecular) ASR, BioMeriuex HHV6 R-gene® kit, Progenie Molecular. The remaining two IVD assays also distinguish between HHV-6A and HHV-6B, Altona Diagnostics Realstar HHV-6 PCR Kit 1.0 Commercial and the Siemens kPCR PLX®HHV-6 DNA Assay. Of the returned datasets obtained from laboratory developed tests 11 were pan HHV-6 detection assays whilst the remaining 5 distinguished between HHV-6A and HHV-6B.
Where disclosed a number of HHV-6 genes were used as amplification targets;
Pan assays:U31, U65-67, ORF13R region, Polymerase gene HHV-6, U38, P100, U57,U6, Major capsid protein (MCP) , UL27 and small capsid protein regions and U73 gene (origin binding protein). 2 datasets did not disclose the amplification region as this information was proprietary.
For the HHV-6A/6B distinction assays: One assay combined U67 for copy number and U95 for 6A/B discrimination, U65-66, U31 and U57. For the dPCR assays U65-67, U57 and UL 67/68 were targeted. 7 datasets did not disclose the amplification region as this information was proprietary.
A range of amplification platforms were also used which include:
ABI Prism 7500 SDS, Mastercycle (Eppendorf), Sacace Sacycler-96,Bio-Rad MyiQ, 3M Integrated Cycler, Applied Biosystems StepOne, Bio-Rad CFX96, ABI 7500 Fast, Rotorgene Q (QIAGEN), ROCHE LC480 Lightcycler 480, MxPro3000P, ABI7500 Fast Real-Time PCR System, Agilent, VERSANT kPCR Molecular System (Amplification Detection Module).
The returned details on methodology highlight the heterogeneity of the method combinations in use for both extraction and amplification of HHV-6 NAT-detection assays, as well as the distinction of each viral species, although in some instances this may have been altered for the purpose of this study.
Estimated potencies of study samples Inter-laboratory variation
The individual laboratory potency estimates for the lyophilised candidates comprise of 34 quantitative and 2 qualitative datasets presented in Table 6. Quantitative potency estimates are provided in log10 copies/mL for all samples, whereas qualitative potency estimates are provided in log10 NAT detectable units/ mL for Samples A and D only. Samples A - D were assayed by all laboratories, whereas samples E-G, J and K were not received for testing by all laboratories. The qualitative potency estimates were derived from a single end-point from each dilution series as described in the statistical methods. The quantitative potency estimates for the candidate
materials A and D are based on combined corrected mean estimates across dilutions as described in the statistical methods. All other samples were tested undiluted and potency estimates were therefore derived from the mean of the neat reported values.
The overall mean potency estimates are summarised in Table 7. A broad range in the viral potency estimates is evident across all the assay formats and for most of the samples assayed.
The log10 copies/ml range for sample A the proposed HHV-6B candidate IS (sample A) combining both the qualitative and quantitative estimates is 5.14-8.89 showing a 3.75 log10
spread of the data. Excluding the qualitative estimates reduces the spread down to 2.64 log10 The corresponding liquid bulk sample (B), shows a similar log10 spread of 2.73 (range 6.16- 8.99). The difference in the quantitative mean potency estimate between the liquid and lyophilized equivalent is -0.11 log10 copies/mL.
For HHV-6A candidate IS (sample D) the quantitative range of 6.10-8.77 of estimates was observed, showing a 2.67 log10 spread of the data. Including the qualitative estimates increases the range to 5.14-8.77 log10 units an increase to 3.63 log10 units. Here the corresponding liquid bulk sample (C), shows a similar log10 spread of 2.64 (range 6.28-8.92). The difference in the quantitative mean potency estimate between the liquid and lyophilized equivalent is 0.03 log10 copies/mL.
For the clinical samples E-G, J and K the overall laboratory mean potency estimates ranges are, for sample E 5.22-7.67 log10 copies/mL a difference of 2.45 log10, for sample F a of range 5.64- 7.83 log10 copies/mL giving a difference of 2.19 log10. The greatest log10 spread was observed for Sample G, a 3.08 log10 spread based on the range 2.26-5.34 log10 copies/mL. Sample J estimates ranged from 2.28-4.73 log10 copies/mL, a difference of 2.45 log10. Finally Sample K exhibited the lowest spread of all the samples ranging from 3.99-5.61, a difference of 1.62 log10.
Intra-laboratory variation
Table 8 presents the intra-laboratory variation of the log10 copies/ mL potency estimates for each of the samples assayed using quantitative analysis. Each laboratory is represented showing the SD values across the samples assayed. A large majority of the SD values are extremely low (<
0.1) across the individual laboratories reflecting excellent single assay agreement and
repeatibility. The majority of the remaining standard deviation values were <0.24. Using >0.25
as a cut-off SD value, laboratory 8 shows consistently higher SD values across almost all of the evaluated samples compared with other laboratories. The highest standard deviation of 0.89 log10 copies/mL was seen with sample G for laboratory 2 however the SD values for the
remaining samples for this laboratory are almost all < 0.1. High SD values for Sample G potency estimates were seen for other laboratories 15, 19, 23, 25b and 29, also the exception to otherwise excellent intra-laboratory SD values for these laboratories.
Comparison of laboratory reported estimates and relative potencies
Figures 1a-b and 2a-b present histogram plots of the mean estimates returned by the study participants for the candidate samples (A-D) and Figures 3a- 7a present histogram plots for the clinical samples (E -G, J and K). For the lyophilised candidates (Figures 1a and 2a) quantitative assay estimates are represented by unshaded boxes and the qualitative datasets are represented by the shaded boxes. For the liquid candidate materials (samples B and C) (Figures 1b and 2b) and all the clinical samples (Figures 3a- 7a) only quantitative estimates are depicted. Each dataset is shown by the individually assigned laboratory number inside each box. Where laboratories provided more than one dataset a further designation of “a” or “b” alongside the laboratory number is provided. The mean estimates of each sample are plotted on the x axis either as log10
copies/mL for the quantitative assays or log10 NAT detectable units/ mL for the qualitative assays against the frequency of the estimated mean values on the Y axis. Each histogram provides a representation of where each laboratory lies in the distribution of the total dataset for each sample, highlighting positioning in relation to the consensus potency estimate.
For sample A whilst there is a spread of 3.75log10 in the mean estimates across the laboratories, 16 datasets report mean potency estimates that lie within 7.72-8.09 log10 copies/ mL representing 52% of the total datasets showing very good agreement (Figure 1a). The two qualitative assays lie to the extreme left of the histogram plot representing the lowest mean potency estimates.
From the quantitative assays, laboratory 4 reports the lowest estimate, similar to one of the qualitative assays, whilst the highest quantitative mean potencies are reported by laboratory 12 and then 15. For Sample B (Figure 1b) the liquid equivalent of Sample A) the majority of the mean potency estimates are clustered together between 7.14-8.36 log10 copies/mL with the exception of laboratory 4 under-reporting and laboratories 12 and 15 over reporting in comparison to the majority of the laboratories.
Using Sample A as a common reference, to apply a relative potency assessment for Samples B we observe an improvement in agreement across the reported mean estimates (Figure 1c). These values are obtained by taking the difference between the laboratory derived estimates for
candidate sample (A) from the laboratory derived estimates from the test sample B (the
difference values are provided in Table 10). For sample B the SD and GCV% values drop from 0.55 and 259% respectively (Table 7) to SD 0.31 and GCV 104% (Table 9). The relative potency estimation brings the discordant potency estimates of laboratory 4, 12 and 15 to within the consensus.
For sample D a spread of 3.63log10 in the mean potency estimates across the laboratories is depicted in the histogram plot (Figure 2a). The two qualitative assays lie to the extreme left of
the histogram plot representing the lowest mean potency estimates. From the quantitative assays, laboratory 4 and then 19 report the lowest estimates, whilst the highest quantitative mean
potency is reported by laboratory 12. For Sample C (Figure 2b) the liquid equivalent of Sample D laboratory 4 and 19 report the lowest mean potency estimates whilst laboratory 12 reports the highest. Using Sample D as a common reference, to apply a relative potency assessment for Samples C we observe an improvement in agreement across the reported mean estimates (Figure 2c). These values are obtained by taking the difference between the laboratory derived estimates for candidate sample (D) from the laboratory derived estimates from the test sample C (the difference values are provided in Table 11). For sample C the SD and GCV% values drop from 0.51 and 221% respectively (Table 7) to SD 0.31 and GCV 104% (Table 9). The relative potency estimation brings the discordant potency estimates of laboratory 4, 19 and 12 to within the consensus.
For the clinical samples the histogram plots of the quantitative estimates are shown in Figures 3a, 4a, 5a, 6a and 7a. For the relative potency assessment of the clinical samples (Figures 3b, 4b, 5b, 6b and 7b) the histogram plots are differentiated by colour to indicate the diluent used by each laboratory to obtain their mean estimate of the candidate sample. Where laboratories performed dilutions using two matrices e.g. laboratory 8 is represented by a pink box to show the estimate obtained using plasma and a red box to show the mean estimate obtained using whole blood.
This is also the case for laboratory 16 where methods 16a-16d appear twice as either pink boxes or red boxes. Laboratories using serum as a diluent are depicted in yellow.
Figure 3a shows the distribution of the laboratory mean estimates for Sample E, the whole blood ciHHV-6A. Using Sample D as a common reference, to apply a relative potency assessment, we observe a modest reduction in the SD from 0.53 down to 0.39, and reduction in the GCV% from 238% to 146% (Table 7 and 9). Laboratory 19, that is at the lower end of the overall estimates (Figure 3a), and laboratory 18 that is at the highest end of the overall estimates are pulled into the consensus. Laboratory 15 however was not harmonised to the consensus (Figure 3b).
Figure 4a shows the distribution of the laboratory mean estimates for Sample F, the whole blood ciHHV-6B. Using Sample A as a common reference, to apply a relative potency assessment, we observe a reduction in the SD from 0.48 down to 0.29, and reduction in the GCV% from 201%
to 94% (Table 7 and 9). Laboratory 13b and 25b, that are at the lower end of the overall estimates, and laboratory 15 that is at the highest end of the overall estimates (Figure 4a) are pulled into the consensus. Here there is a general trend for the laboratories using whole blood for their dilution series to cluster (Figure 4b).
Figure 5a shows the distribution of the laboratory mean estimates for sample G, the HHV-6B positive whole blood sample that showed the greatest inter-laboratory variation 3.08 log10 copies/
mL. Only a slight improvement in agreement was observed when a relative potency assessment was performed (Figure 5b), where the SD and GCV% values of 0.82 and 560% respectively changed moderately down to SD 0.75 and GCV 468% after the relative potency assessment (Tables 7 and 9). Whilst laboratory 15 is pulled into the group the overall spread of estimates stays the same. Here clustering is not observed for the majority of the laboratories using whole blood for their dilution series of Sample A. Laboratory 16 estimates are however both shifted to the right separating into whole blood then plasma (Figure 5b).
Figure 6a shows the distribution of the laboratory mean estimates for Sample J, the HHV-6B plasma sample. Using Sample A as a common reference, to apply a relative potency assessment, we observe a reduction in the SD from 0.48 down to 0.24, and reduction in the GCV% from 202% to 74% (Table 7 and 9) . Laboratory 4 at the lower end of the overall estimates, and laboratory 12 that is at the highest end of the overall estimates (Figure 6a ) are pulled into the consensus (Figure 6b). The 2.45 log10 copies/ mL inter-laboratory variation was reduce to a within a log10 difference, improving agreement across laboratories.
Figure 7a shows the distribution of the laboratory mean estimates for Sample K, a CSF sample spiked with candidate A, an improvement in agreement between laboratories is also observed where the SD and GCV% values drop from 0.45 and 182% to SD 0.14 and GCV 37% (Table 7 and 9). The relative potency estimation brings laboratories 10, 15 that sit to the right of the bulk of estimates in the histogram plot (Figure 7b) to within the consensus. Although the diluents are not applicable to this sample matrix, harmonisation with the other matrices is still observed.
Assessment of diluent effects
Since the proposed IS preparation is intended for use using multiple diluents Figure 8a and 8b show the laboratory mean estimates for sample A and D respectively in log10 copies/mL and log10 NAT detectable units/ mL for the qualitative datasets (22 and 28), using data obtained following the dilution of the reconstituted lyophilised candidate, using a clinical matrix. The majority of laboratories used plasma (n=12) for their dilutions followed by serum (n=7) then whole blood (n=4). Laboratories 8 and 16 used both plasma and whole blood for their
evaluations and these data have been separated for this analysis. Only 1 laboratory used a non- clinical matrix (water) to perform the dilutions. Each diluent used is represented by a colour.
There is no clear trend across the estimates where estimates obtained with either whole blood, serum or plasma are interspersed suggesting little difference in the mean estimate based on the diluent used. That withstanding the estimates obtained by laboratory 16 using either plasma or whole blood clearly do separate with both candidates Sample A and D, where whole blood dilutions cluster together to the right of the plasma dilutions using the same methodology.
Laboratory 8 also performed a similar analysis using both plasma and whole blood as diluents for Sample A and D, but no difference between the estimates is evident.
To investigate any matrix effect further interval plots were generated for the quantitative laboratory mean estimates for samples A and B, for each matrix used (see Figure (9a and 9b).
The confidence limits for the three matrices are all overlapping, suggesting no evidence for an effect on the results from using different diluents. For Sample A the mean for each diluent lies at:
Plasma= 7.81, Serum= 7.79, Whole blood= 7.63. For Sample D the mean for each diluent lies at:
Plasma= 7.49, Serum= 7.47, Whole blood= 7.38.
Discussion
The intended purpose of this study was to evaluate the suitability of the candidate preparations 15/266 and 16/116 for use as primary reference materials for the standardisation of the HHV-6 NAT-detection assays. A lack of clear distinction between 6A and 6B has contributed to the difficulty in assessing the epidemiological and etiological associations of HHV-6A, and a clear disease has yet not been associated with HHV-6A. However a clear disease association has been attributed to HHV-6B primary infection and HHV-6B reactivation in immunocompromised patients where the standardisation of the HHV-6B NAT-detection assays is required.
The proposed candidate standard (15/266) comprises of virus preparation of HHV-6 strains Z-29, prepared using high titre viral stock propagated in a permissive cell line formulated in a Tris- based universal buffer. The formulation will thus enable the dilution of the preparation into a suitable clinical matrix for the preparation of a multi-point standard curve. The candidate has undergone NGS analysis confirming that the sequenced genomes shows >99% sequence homology to the expected reference strains. No gross changes in the overall genome integrity were detected, however tandem repeats were uncovered at the ORI of replication that may be a possible consequence of successive rounds of laboratory propagation.
The production data analysis of the residual oxygen and moisture content of the lyophilised formulation is within the acceptable limits (<1.14% and <1.0% respectively) indicative of long term stability. Furthermore the fill CV and the potency estimates are indicative of a
homogeneous product. The viral copy values obtained for the HHV-6B candidate post-
production shows good homogeneity across the vial contents. The mean copies/ mL obtained for 15/266 from the in house analysis (8.0 log10 copies/ mL) is in fair agreement with the quantitative overall mean estimate of this candidate (7.75 log10 copies/ mL) obtained from the collaborative study data. The results obtained from the accelerated thermal degradation studies at 3 and 6 months indicate that the material is stable and there is no evidence for any predicted loss in potency based on the current theremal accelerated degradation data. In addition further accelerated degradation evaluations are scheduled for future time points which will provide additional data for analysis and an indication of continued stability and suitability of this candidate for long-term use.
The overall quantitative mean estimate of the lyophilised material 15/266 sample A is 7.75 log10
copies/ mL (SD 0.49 and GCV 208%), this is comparable with the quantitative mean of the equivalent liquid bulk (Sample B) 7.86 log10 copies/ mL. Whilst there is a slight difference in potency in the freeze dried candidate compared with the liquid candidate, which could be a result from the processing of the materials through lyophilisation. Consideration must be given to the data points for the liquid sample B which were obtained from undiluted samples only, whereas multiple dilution points were performed for the lyophilised candidate. The difference observed in the potency estimates could therefore be attributed to inaccuracies in the determination of high viral load samples, as this is likely to be outside of the quantitative range of most qPCR assays.
Indeed some laboratories did report that estimations performed with the undiluted candidates were problematic. Participant 5 noted that the values for the undiluted Samples A and D were “>
1 x 10e7”, likely outside of the quantitative range of their assay. Laboratory 13 noted for their
dPCR analysis that the undiluted samples resulted in assay “saturation”. Finally laboratory 25 noted “poor extraction efficiency of undiluted samples A-D” particularly for dPCR data (25b).
These would certainly be expected for the dPCR assays as they rely on positive and negative wells for copy number determination using Poisson distribution. Indeed laboratory 13 did provide data of dilution of samples B and C for their estimations (13b). This will be noted for future study designs.
Overall from the standard deviations obtained from the intra-laboratory analysis the assays show good consistency within each laboratory, indicative of good single assay validation.
Interestingly the inter-laboratory variation of the quantitative mean laboratory estimates for each of the candidate materials and the clinical samples is somewhat smaller than expected in the absence of a standardised reference, reflected by the lower than expected SD values ranging from 0.45- 0.55 and with the exception of 0.82 for sample G. The first EQA study conducted by QCMD in 2010 reported greater variation with standard deviations of the log10 number of copies/ml ranging from 0.5 to 0.7 [9]. A more recent study looking at the variability of quantitative HHV-6 NAT detection assays across laboratories participating in a proficiency testing survey, reported SD values in the range of 0.63-0.98. Only the SD value of 0.82 obtained for Sample G is comparable to this range [16]. This maybe attributed to the high proportion (50%) of commercial assays included in this data set.
Nevertheless there is variation across the laboratories for the mean estimates of the candidates and the clinical samples ranging from 1.62 to 3.08 log10 copies/mL. Agreement between the laboratories for these samples was markedly improved when the potencies for these samples were expressed relative to either of the corresponding candidate materials, reducing the SD values of the overall estimates demonstrating the suitability of the candidates to standardise assays. Interestingly Sample A the HHV-6B candidate IS was able to improve agreement for the estimates returned for sample E the iciHHV-6A sample reducing the SD from 0.55 to 0.39 the same as that obtained with Sample D the HHV-6A candidate IS (Table 12 highlighted in red).
For Sample C the liquid HHV-6A preparation, Sample A the HHV-6B candidate IS was able reduce the initial SD 0.51 down to 0.37 (Table 12 highlighted in red) but this was not to the same extent as Sample D (Table 13 values in black).
However the HHV-6B candidate IS was unsuccessful at harmonising the dataset returned for sample G. It is not entirely clear what this could be attributable to. Unlike the ciHHV-6 samples, there is little traceability of the manipulations this sample underwent prior to receipt from Cerba Specimen Services. Processing at NIBSC included thawing and dilution into negative whole blood and then re-freezing, followed by a further thaw by the participants. Indeed the highest intra-assay variability values were obtained with sample G (Table 8) but this was not consistently observed across all laboratories. Another factor relative to the ciHVV-6 samples is the difference in viral load of Sample G, the former being 6.0 log10 and sample G being approximately half the quantity. Assay performance may vary more at these lower viral loads. Interestingly laboratory 16 provided the lowest estimate for Sample G 2.26-2.68 log10 (below the 25% percentile < 2.973 log10 copies/ml) across the 4 methods performed compared with other laboratories, in contrast to the estimates they provided for samples E and F that were in line with the mean estimates of these samples. However laboratory 16 estimates were positioned within the range of mean estimates for sample J that was estimated at 3.56log10 copies/ml.
The comparison of diluent effects (Figure 9a and 9b), show that the mean estimates derived from the dilutions made of the candidate Sample A and D using plasma, serum or whole blood do not differ significantly form each across the laboratories, suggesting there is no diluent effect on the obtained mean estimates. However the data from laboratory 16 does show a difference between plasma and whole blood using the same method as indicated from the histogram plots in Figure 8a and 8b. The difference between the mean estimates for both samples A and D was significant using t-test (P= <0.0001) across the 4 methods performed by laboratoty 16. Further controlled commutability assessments should be performed within methods to compare directly for matrix effects. In order to address this issue fully a study with multiple clinical samples plasma, serum and whole blood should be conducted in order to draw robust conclusions on the commutability of the proposed standard preparations across methods. This is defined as “a reference material (RM) would be considered commutable when a measurement procedure produces the same result for a RM as it does for an authentic patient sample that contained the same analyte
concentration” as stated in Vesper et al., 2007, typically performed comparing two methods using linear regression with prediction limits or correspondence analysis for more than 2 methods [17].
This multicentre collaborative study included a good number of laboratories with a wide geographical distribution. The collective study group provides a good representation of the variety of end-users of HHV-6 NAT assays and illustrates the wide range of assays
methodologies in use. The details supplied by participants on the assay methodologies highlight the heterogeneity of the method combinations for both extraction and amplification of HHV-6 NAT-detection assays, where no two methods of the returned datasets were identical. To ensure that the calculated mean values were not biased by outliers or over-representation of particular assay methods, additional analyses were calculated as Huber’s robust mean as well as weighted by assay method (combining extraction groups or amplification groups or by laboratory (8 and 16).The values obtained demonstrated minimal effect on the alternative calculation methods, resulting in no more than 0.02 Log10 IU/mL different to the proposed value.
Quantitative PCR detection is currently the most widely employed method for rapid and sensitive HHV-6 detection in many clinical laboratories worldwide. The findings of the
collaborative study data support the use of a common primary reference material to standardise multiple detection methods and multiple clinical matrices which should ensure better
comparability across institutions and help toward setting universal clinically relevant viral load threshold limits to better direct patient diagnosis and treatment decisions. The results of the study demonstrate that the proposed candidate standards NIBSC code 15/266, would be suitable for use as a standard for the quantification of HHV-6B DNA detection assays. Based on the overall quantitative potency estimate for this candidate we would recommend a value of 7.75 log10
International Units/ mL to be assigned.
Recommendation
It is proposed that the candidate standard (NIBSC code 15/266) is established as the 1st WHO International standard for HHV-6B nucleic acid amplification technique (NAT)-based assays with an assigned potency of 7.75 log10 International Units when reconstituted in 1 mL of nuclease-free water. The proposed standard is intended for use by IVD manufactures for kit calibration and for use by clinical, reference and research laboratories for the calibration of secondary reference reagents used in routine NAT-assays for HHV-6B detection. A draft instruction for use (IFU) for the product is included in Appendix IV.
