Thesis
Reference
Point-of-Care Testing in Pediatric Emergency Care
LACROIX, Laurence Elisabeth
Abstract
Les progrès récents de la médecine d'urgence pédiatrique ont mené à l'implémentation au sein des services d'urgence de tests de diagnostic rapide (Point-of-Care Testing, POCT). Ces tests montrent de nombreux avantages: réduction du délai diagnostique et du temps de séjour aux urgences, réduction du nombre de tests additionnels et traitements non nécessaires, réduction du délai jusqu'à la prescription du traitement approprié, réduction du volume sanguin nécessaire, et possibilité pour un personnel non entrainé de réaliser le test facilement. Cependant, plusieurs éléments sont à prendre en considération: le contexte clinique dans lequel le test est effectué afin de pouvoir en interpréter le résultat, son exactitude par rapport à un test de référence, l'assurance du maintien de la qualité, ainsi que la mise en place de protocoles cliniques de prise en charge, afin de pouvoir offrir aux patients pédiatriques une médecine d'urgence sécurisée et plus personnalisée.
LACROIX, Laurence Elisabeth. Point-of-Care Testing in Pediatric Emergency Care. Thèse de privat-docent : Univ. Genève, 2017
DOI : 10.13097/archive-ouverte/unige:100279
Available at:
http://archive-ouverte.unige.ch/unige:100279
Clinical Medicine Section Department of Pediatrics
"Point-Of-Care Testing in Pediatric Emergency Care"
Thesis submitted to the Faculty of Medicine of the University of Geneva
for the degree of Privat-Docent by
Dr Laurence LACROIX-DUCARDONNOY
Table of contents
Abstract………..1
Abbreviations……….2
I. Pediatric emergency care: a new medical specialty in Switzerland! ... 3
1. Recent advances in pediatric emergency medicine ... 3
2. Improvement strategies to reduce diagnostic delays in pediatric emergency settings .... 5
3. European standards for Pediatric Emergency Care ... 6
II. The basics of Point-Of-Care Testing (POCT) ... 7
1. Pathways of diagnostic approach to the ill patient ... 7
2. Variations in diagnostic test statistics ... 9
3. The gold standard test ... 9
4. The Point-Of-Care test ... 10
III. Advantages of Point-of-Care Testing ... 11
1. Improved diagnostic delays ... 11
2. Length of stay ... 11
2. Reduced delay in treatment prescription ... 12
a. Reduction in drop out patients ... 12
4. Improved traceability of patient sample ... 14
5. Host connectivity ... 14
IV. Challenges to implementation of Point-of-Care Testing ... 15
1. Adequacy ... 15
2. Quality assessment ... 15
3. Environmental influence ... 16
4. Errors in POCT testing ... 17
5. Implementation of clinical pathways and PED logistic ... 18
6. Costs ... 18
V. Personal contribution in the area of research ... 20
1. Procalcitonin Measurement for Detection of Serious Bacterial Infection in Febrile in Children: comparison between two automated immunoassays. ... 22
2. Impact of the Lab-score on antibiotic prescription rate in children with fever without source: a randomized controlled trial. ... 26
3. Validation of the Step by Step approach in the management of young infants with fever without source ... 44
4. Validation of a novel assay to distinguish bacterial and viral infections ... 57
VI. Perspectives ... 75
1. Expertise and knowledge ... 75
2. Implementation of specific protocols ... 76
a. Fever without source ... 76
b. Diagnosis and management of Group A Streptoccoccal pharyngitis ... 77
3. POCUS, a new type of POCT ... 78
4. Broadening the applicability of POCT ... 78
5. Combination of host biomarkers and/or pathogen specific signatures... 80
6. Emerging technologies ... 81
VII. Conclusion ... 82
VIII. References ... 83
IX. Acknowledgements ... 85
Abstract
In many pediatric emergency departments throughout developed countries, recent advances in pediatric emergency care have led to the implementation of Point-Of-Care Testing (POCT), which refers to any laboratory test performed outside the central laboratory, more commonly at the patient’s bedside. In this setting, POCT involves mainly host biomarkers and pathogen- specific tests.
These tests show many advantages: improved diagnostic delays and hence positive effect on patient flow, reduced unnecessary tests or treatments, reduced delay in treatment prescription, reduced blood sample volume, and possibility even for an untrained staff to perform the test easily.
However, these tests also face numerous challenges. First, the emergency care physician should carefully assess the clinical context in which the test will be performed in order to adequately interpret the corresponding results. Then, accuracy with a reference method should be examined. Maintenance of quality should be assessed throughout the testing process and clinical pathways and corresponding protocols should be implemented.
Continuous improvements in POCT technologies, through serial or multiplex emergent testing, invariably offer patients a secured and more personalized emergency care medicine.
Abbreviations
CRP - C-reactive protein ED - emergency department FWS - fever without source IBI - invasive bacterial infection
IP-10 - Interferon gamma-induced protein-10 LOS - length of stay
PED - pediatric emergency department PCT - procalcitonin
POC - point-of-care
POCT - point-of-care testing SBI - serious bacterial infection
TRAIL- TNF (tumor necrosis factor) related apoptosis-inducing ligand
I. Pediatric emergency care: a new medical specialty in Switzerland!
1. Recent advances in pediatric emergency medicine
Pediatric emergency care has shown amazing advances over the past 20 years. Although pediatric patients have for a long time been and are still treated by general care practitioners or adult emergency physicians, pediatric emergency medicine has been recognized as a distinct subspecialty of pediatric medicine in 2014 in Switzerland.
In our country over the past 20 years, the pediatric population has remained rather stable or has only moderately increased (Fig.1). However, the number of visits to the Pediatric Emergency Department (PED) has increased by almost 50% between 1997 and 2016 (Fig.2).
0 5'000 10'000 15'000 20'000 25'000 30'000 35'000 40'000
1997 2000 2005 2010 2013 2016
Nombre
< 1 year old 1-4 years 5-9 years 10-14 years 15-19 years TOTAL
Figure 2. Evolution of the annual rate of visits to the PED, Geneva University Hospital, between 1997 and 2016 (personal data)
Moreover, besides the increasing presentation rate to the PED, pediatric emergency physicians also face the challenge to deal with recent advances in ambulatory care. In the past, patients showing moderate to severe affections were frequently admitted. Both diagnosis and treatment initiation were mostly performed in admission units. In this setting, laboratory analysis could be easily performed in purpose-built laboratories, usually centralizing different analyses. Nowadays, improvement in ambulatory care facilities prompts a rapid and accurate diagnosis to be made in the PED itself, or even in prehospital settings, in order to rapidly initiate the most accurate treatment.
Actual concepts in emergency care include a notion of timing that is clearly elicited in the updated definition of Emergency Medicine in Europe, defined as follows by the European Society for Emergency Medicine (EUSEM) Professional Committee:
“Emergency Medicine is a primary specialty established using the knowledge and skills required for the prevention diagnosis and management of urgent and emergency aspects of illness and injury, affecting patients of all age groups with a full spectrum of undifferentiated physical and behavioral disorders.
18000 20000 22000 24000 26000 28000 30000
1997 1999 2001 2003 2005 2007 2009 2011 2013 2015
This includes organizing the proper medical response for patients looking for urgent medical care.
Time and timing in this setting may be critical either from a medical or from the patient’s point of view.
The practice of Emergency Medicine encompasses the in-hospital as well as out-of-hospital triage, resuscitation, initial assessment, telemedicine and the management of undifferentiated urgent and emergency patients until discharge or transfer to the care of another health care professional.”
It has therefore become essential for this emerging subspecialty to develop various strategies in order to reduce the delay before diagnosis and treatment initiation in pediatric emergency care.
2. Improvement strategies to reduce diagnostic delays in pediatric emergency settings
The frequent overcrowding of PEDs has been described internationally1, showing negative side effects for the patient: longer length of stay for the pediatric patient, parental dissatisfaction with reduced confidence in the health care system. Moreover, it may also represent an emerging threat to patient safety. Hence, finding potential solutions for PED overcrowding may directly affect health care delivery to patients.
Enhancing patient flow through the emergency department (ED) is one of the main targets for the emergency practitioner. Patient flow depends on different variables: rapid and efficient
Besides the above-mentioned improvements in reducing the amount of time patients spend in the ED and in order to improve patient flow through the ED, the development of more decentralized analysis tools, showing rapid availability of results for many commonly ordered tests at the bedside of the patient, and easiness to use even for untrained staff, so–called point- of-care testing (POCT), also appeared promising.
The term POCT first appeared in the literature in year 1994, and rapidly, an increasing number of papers concerning its use were published4. POCT has been defined by numerous authors, but an effective common denominator for the myriad of definitions is: patient specimens assayed at or near the patient with the assumption that test results will be available instantly or in a very short timeframe to assist caregivers with immediate diagnosis and/or clinical intervention5.
3. European standards for Pediatric Emergency Care
Standards for Children and Young People in Emergency Care Settings have recently been published in 2012 by the Royal college of Paediatric and Child Health (http://www.rcpch.ac.uk/emergencycare), with a new revision that is due for 2017. Here also, the authors mention that access to certain POCT devices, notably arterial blood gases and arterial or capillary blood glucose monitoring, is essential tool that should be specified in the equipment list of any emergency department or any emergency care setting.
II. The basics of Point-Of-Care Testing (POCT)
1. Pathways of diagnostic approach to the ill patient
Pediatric Emergency physicians frequently have to face diagnostic challenges with patients they are in charge of. Indeed, neonates, infants and small children often present with non specific symptoms. They are also less capable than older children or adults to explain or locate their complaint. However, it is crucial to differentiate those suffering from serious conditions such as serious bacterial infections (SBI), and necessitating the initiation of an appropriate treatment, from those presenting with less severe affections requiring only symptomatic treatment. Also, if uncertainty concerning the diagnosis still persists after a thorough history and clinical examination, it is frequently required to rely on additional testing, in order to determine whether the patient actually suffers from the suspected condition or not, whether this condition is severe or not, and which therapeutic strategy should be initiated (therapeutic abstention, versus medical or surgical therapy).
Diagnostic tests are essential in order to facilitate clinical decision making by stratifying any individual patient into different risk categories. A useful way to measure to which extent the result of a test modifies the pretest probability of suffering from a specific disease or condition is the likelihood ratio (LR), which is computed from the conventional diagnostic sensitivity (Se) and specificity (Sp) of the test itself, and therefore is independent of the prevalence of the disease. The LR represents how many times more (or less) likely patients who actually have the disease or condition are to demonstrate a particular result compared with patients who do not have the disease6. The integration of pretest probability (i.e. the prevalence of the disease in the studied population) and likelihood ratio value is easy to visualize on Fagan’s nomogram (Fig. 2)7.
Figure 3. Fagan’s nomogram for interpreting diagnostic test results: the example of serious bacterial infection risk according to the Lab-score. Adapted from Fagan7.
(LR+: positive likelihood ratio, LR-: negative likelihood ratio, SBI: serious bacterial infection)
There are two main diagnostic procedures that are encountered when additional testing is required: the gold standard test or the point-of-care test. For both of them, the impact on diagnostic decision processes depends not only on the diagnostic accuracy of the test itself, but also on the pretest probability of the specific condition.
It is therefore crucial to relate any diagnostic tool to a clinical situation that will determine the pretest probability. Indeed, the pretest probability, which is influenced by both clinical expertise and prevalence of the affection in the corresponding population, will influence both the positive and negative predictive values (respectively PPV and NPV) of a diagnostic test, and thus impact on clinical decision making.
pre-test probability
= disease (SBI) prevalence:
24.7%
Post-test probability
69%
LR- Lab-score ≥3
0.17
LR+
Lab-score ≥3
6.7
Post-test probability
5%
2. Variations in diagnostic test statistics
It is essential to take into account that diagnostic test accuracy statistics may vary according to clinical variables8. Among them, clinical heterogeneity of patients is the most important one, responsible for the so-called spectrum bias: sensitivity is expected to increase where test results become more extreme in patients with the most severe disease (i.e. more likely to test positive) whereas specificity is more commonly influenced by other comorbid conditions and alternative diagnoses in those without the target disorder that could cause false positive results.
Diagnostic test accuracy is also affected by variations in timing, in technical aspects of any equipment or materials used and, inter- and intra-observer and laboratory variations. Similar variations in the reference standards used must also be considered.
Finally, threshold effect is an important source of heterogeneity to consider in meta-analyses of diagnostic tests, as patients will be classified positive or negative according to the defined cut-off or threshold value. The higher the cut-off value chosen, the higher the specificity and lower the sensitivity estimates will be.
Therefore, before implementation of any new test device, it is recommended for the researcher to conduct sufficiently large, prospectively designed primary studies of diagnostic test accuracy that compare two or more tests for the same target disorder so that sources of heterogeneity are minimized and comparative accuracy can be established in a wide spectrum of patients8.
3. The gold standard test
A gold standard test is defined as the reference test that will determine the validity of a
have numerous disadvantages: discomfort, pain, iatrogenic risks, costs, and delay between the test and the answer.
4. The Point-Of-Care test
Point-Of-Care testing (POCT) refers to any laboratory test performed outside the central laboratory at or near the location of the patient. This modern variety of laboratory testing implies minimizing instrument sizes and facilitating assay procedures for any person susceptible to perform the test, mostly emergency physicians or nurses, who are untrained for complex laboratory technologies.
POCT analyzers can be separated into different categories: qualitative strip-based POCT methods (urine dipstick analyses, pregnancy testing, detection of infectious components in swab samples, etc.), quantitative unit-use analyzers (glucometers), bench-top POCT analyzers (using spectrophotometric substrates and enzyme-activity measurement, hematological particle counting, immunoassay, and blood-gas analyzers), hemostaseological coagulation analyzers, continuous measurements with POCT devices (such as continuous glucose monitoring), and finally molecular biology-based POCT devices to detect infectious agents.
In pediatric emergency settings, POCT are mostly used to evaluate metabolic disturbances (blood electrolytes and glucose, ketone bodies, acid-base balance, lactate level) and respiratory status (blood gases and acid-base balance), and as an aid to detect underlying bacterial versus viral infections. In the latter category, POCT can be divided into 3 groups:
biological markers of infection (urinary dipstick, white blood cell count and differential), response specific host biomarkers in relationship to various infectious agents such as C- reactive protein (CRP) or procalcitonin (PCT) and pathogen specific tests. Pathogen-specific tests include viral pathogens (respiratory pathogens such as Influenza Virus or Respiratory Syncitial Virus, enteric viruses such as Rotavirus, HIV), bacterial pathogens (such as group A Streptococcus) and parasitic agents (such as Plasmodium infections).
III. Advantages of Point-of-Care Testing
1. Improved diagnostic delays
POCT significantly shortens turnaround times, which is defined by the time elapsed between sample acquisition from the patient and result of the corresponding analysis. Indeed, not only the time for specimen transport, sample preparation (centrifugation, separation) and data entry, but also the time for validation of the result and forwarding of the report are eliminated.
Therefore, parental and/or patient satisfaction is increased.
The effect POCT has on patient care and on efficiency shows a strong dependency on the clinical context. A short delay for test results is most beneficial in cases in which delays in treatment of at least 1 hour can have significant effects on outcomes and delays in test results are the primary determining factor holding up patient management decisions 9.
For example, patients with septic shock benefit from both early recognition and prompt and aggressive treatment of their condition, including adequate fluid resuscitation, administration of antibiotics and if necessary initiation of an inotropic treatment in the first hour of management10. These recommendations have been published as clinical practice guidelines by the American College of Critical Care Medicine and incorporated into the American Heart Association Pediatric Advanced Life Support (PALS) courses11. When correctly applied, they show improvement in septic shock outcomes in both academic and community settings.
Therefore, infectious biomarkers determination through POCT devices could help in clinical diagnosis and hence participate in prompt recognition and treatment of certain life-threatening conditions.
2. Length of stay
responsible for additional delays 12. However, more recently, a large randomized controlled trial including more than 10’000 non critically ill general ED patients aged 15 years old and more and stratified according to Emergency Severity Index at triage, showed that patients allocated to the POCT group had reduced median ED LOS compared to patients randomized to the central laboratory group. This effect was largely due to reduction of LOS among acuity level 3 patients and those discharged from hospital 13. Differences among studies in the effect on LOS are probably attributable to other confounding factors such as differences in ED settings. Moreover, a pediatric randomized controlled study in a tertiary urban hospital setting evaluated effect of POCT versus routine laboratory analysis allocation. Similar waiting periods were noted in both groups for time spent in the waiting room, time waiting for first physician contact, and time waiting for blood to be drawn. Significantly less time was required for results to become available to physicians when POCT was used (65.0 minutes; P
< 0.001), with significant decrease in overall LOS, with patients randomized to the POCT group spending an average of 38.5 minutes (p< 0.001) less time in the ED14. Similar findings were found in a prospective observational study in the Netherlands comparing 2 cohorts of febrile children, aged 1 month to 16 years, in whom bedside CRP testing lowered total LOS by 30 minutes, representing 19% of total LOS and by 15% after adjusting for other determinants15. In a setting where the prevalence of fever is high, implementing bedside CRP testing could therefore have beneficial effect on patient LOS and patient flow at the PED.
2. Reduced delay in treatment prescription
a. Reduction in drop out patients
Through reduction of many pre- and post analytical steps, the result of the test is instantaneously reported to the patient after validation, with no need for a return visit or a back-up phone consultation with the patient, the parents or caregivers. Therefore, this may show a beneficial effect in reducing the number of drop out patients.
b. Reduction in disease spread
Regarding many transmissible pathogens, POCT is used not only as an aid to clinical diagnosis, but also to assist in decision-making for cohorting and infection control, which is of particular importance during RSV or influenza outbreaks16, 17. This is a significant contribution to bed management during a period when resources are most stretched. This strategy also aids in improving patient flow.
c. Reduction in absenteeism length for patient or caregivers
POCT reduces the need for follow-up consultation and the delay before treatment initiation.
This strategy is also indirectly beneficial for parents or caregivers who are less prone to professional absenteeism.
d. Reduction of unnecessary tests and treatments
Since POCT reduces the delay before the diagnosis is made, earlier management decisions can be made concerning the patient, with reduced need for additional testing or unnecessary treatments. For example, the use of POCT for the management of seasonal influenza in pediatric patients, rapid tests have been shown to reduce prescription of laboratory test, radiographic studies, and antibiotics17, 18. This effect may have a beneficial impact on the development of bacterial resistance that occurs when unnecessary antibiotic treatments are prescribed to the patient.
Indeed, development of antibiotic resistance is a main concern for public health. It i s mainly driven by antibiotic consumption. In a recent study analyzing consumption trends between
settings, limiting antibiotic prescription to patients who really need could be achieved through the use of POCT, which provides the health care practitioner with an immediate result.
3. Reduced blood sample volume
Due to miniaturization of the device, proximity to the patient, and reduced requirements for phlebotomy with easier access to capillary samples, POCT testing requires less blood volume than when the same analyte is tested in a central laboratory facility, thereby reducing iatrogenic blood losses. This aspect is particularly important when caring for fragile patients such as neonates, infants and trauma patients, in which venopuncture can be challenging, especially in critical situations, and in whom transfusion has been shown deleterious in many aspects 20.
4. Improved traceability of patient sample
POCT is generally performed by the staff directly in charge of the patient. By reducing the number of people in charge of the sample and the need for transportation, the use of POCT minimizes the risk of sample loss or mistaking different patient samples.
5. Host connectivity
Recent POCT devices show improved informatical connectivity, with associated software permitting direct transfer of patient and quality control results, patient and operator identification and instrument-specific information to a database. Electronic transfer of POCT data reduces the errors due to manual transcription of laboratory data.
IV. Challenges to implementation of Point-of-Care Testing
1. Adequacy
It is crucial that the indication for testing is appropriate, so that the test has an impact on management of the patient or on the treatment modality of the corresponding medical condition. The suggestion for using a POCT device should be raised by a professional who is aware of the clinical context and interpretation of the test result.
Moreover, the test should be used in definite epidemiology settings, since prevalence of the disease impacts on both positive and negative predictive values of the test itself.
2. Quality assessment
First, it is important to note that most POCT devices are operated by staff with limited technical background (most often nurses but also junior residents), who already frequently suffers from heavy workload and time pressure. Teaching them how to use the device is therefore an important step to adequate sampling and technical workup, in order to assure adequate quality of the test.
Second, it is important to ensure adequate quality control of the device itself using both regular internal and external quality assessment methods. Cooperation with the central laboratory is therefore necessary.
In order to evaluate the diagnostic properties of a test, it is also necessary to compare the result of a POCT with the corresponding gold standard, in order to ensure that the POCT will show adequate diagnostic performances. Indeed, the ideal test should be accurate, precise, and
Precision refers to reproducibility, through estimation of the dispersion of the test results. The more repeatable or reproducible the measurement results are, the more precise the POCT test is.
Although comparisons with the reference method are generally reported using linear regression analyses, the use of other statistical tools such as the Bland-Altman plot is more accurate since it better reflects clinical utility21.
3. Environmental influence
Environment is a significant factor that may influence the performances of a POCT device.
For instance, in emergency medical services practice, POCT devices may be used in outdoor settings or transport vehicles under specific conditions as movements and vibrations, extreme temperatures, rain and humidity, which could interfere with the electronic equipment of the device and alter the performance of the test. These conditions should also be carefully taken into account when using POCT in specific settings such as humanitarian medicine or disaster settings.
Moreover, in these settings, it is difficult to assess the quality of training of the person in charge of the test. However, if performed outside the standard hospital environment, the result of the POCT test will very often lead to a clinical decision that may have a significant influence on the quality of care to the patient (orientation, treatment decision). It is therefore crucial that devices should be simplified and equipped with inbuilt quality controls and used by a trained staff.
4. Errors in POCT testing
Errors in POCT testing may arise in the pre-analysis phase and may be related to patient preparation, sample acquisition, or pre-analytical sample preparation. They may also occur in the post analysis phase (documentation, data protection).
In order to reduce the risk of errors and facilitate convenience, it has been shown that the following features should apply to POCT devices 22:
Long-life, maintenance-free electrodes or disposable sensor packs
Touch screens as the user interface
Software that can demand user and patient identification
Built-in bar code scanners
Sample aspiration instead of injection
Reduced sample sizes
Clot detection within analysis chamber
Sample detection to prevent short samples
Liquid calibration systems instead of gas bottles
Automated calibrations
Automated quality control sampling
Sophisticated quality control programs including interpretation of data
Connectivity to information systems allowing remote monitoring and control
Built-in videos for training purposes
5. Implementation of clinical pathways and PED logistic
Shortening of the turnaround time with no additional benefit for the patient negates the effort and the cost of performing the analysis at the patient’s bedside through point-of-care testing.
Hence, the success of POCT implementation requires implementation of protocols and treatment algorithms rapidly responsive to POCT results, so that immediate procedure or treatment initiation may be applied.
6. Costs
The literature concerning POCT associated costs and benefits is scant in pediatric emergency medicine. In adults, for the majority of POCT devices the direct cost per analysis is higher than the cost per analysis in centralized laboratories. However, when taking into account indirect costs linked with the numerous manual steps in transferring samples to central laboratories, technical workup of these samples by specialized technicians and transcription of results, the total cost of POCT does not exceed that of the central analysis9, 12. More recently, a medico-economic analysis has shown that using POCT rather than traditional serum testing for children presenting with gastroenteritis results in cost savings from the points of view of the patient as well as the healthcare provider23. However, this finding needs to be analyzed at a more general point of view concerning pediatric patients.
Careful medico-economic studies examining costs and benefits in relation to selected outcomes for the patient should therefore be performed in each institution before implementation of any new POCT device.
In conclusion, considering the above mentioned challenges, some authors have stated the main required features for POCT devices24. They should show the following characteristics:
Simple to use
Reagents and consumables should be robust in storage and usage
Results should be concordant with an established laboratory method.
Both device and associated reagents and consumables should be safe to use.
More officially, the World Health Organization (WHO) has provided guidelines named under the acronym ASSURED22. They recommend that POCT devices should be:
Affordable – for those at risk of infection
Sensitive – minimal false negatives
Specific – minimal false positives
User-friendly – minimal steps to carry out test
Rapid & Robust – short turnaround time and no need for refrigerated storage
Equipment-free – no complex equipment
Delivered – to end users
Initially described for POCT devices intended for the detection of sexually transmitted infections, mainly in resource limited settings, it has rapidly become obvious that the features of the ASSURED guidelines would also benefit to the implementation of any new POCT device, even in the developed world.
V. Personal contribution in the area of research
The use of any POCT in pediatric emergency settings requires different steps before the test becomes applicable in the field.
First of all, any POCT device needs to be developed by engineers who will concentrate on the analytical part of the device. This subject is beyond the scope of this thesis.
Then, any POCT device will be tested against a reference method in order to be validated in the population of interest. Many devices are tested on the adult population. However, accuracy of the test in the pediatric population also needs to be tested, since the biological environment of a newborn or an infant may differ from that of an older child, an adolescent or an adult, depending on the age of the patient. For instance, hemoglobin level is high in a healthy term neonate (19.3± 2.2 g/dL), with a predominance of fetal hemoglobin reflecting the relative hypoxic intrauterine environment. Then following exposure to the ambient high oxygen content after birth, breakdown of red blood cells containing fetal hemoglobin occur together with the rise of adult hemoglobin percentage, and hemoglobin levels typically decrease over the first weeks of life to reach a physiologic nadir by 8 to 12 weeks of age (9 to 11 g/dL), before reaching a steady state25. POCT devices measuring hemoglobin counts in pediatrics should therefore be validated over the whole range of values. Another example of this concept is POCT measurement of pediatric bilirubin levels. Whereas adult devices are calibrated for adult-range bilirubin levels, much higher values may be measured in case of neonatal hyperbilirubinemia (which can lead to serious consequences if left untreated).
Calibration of the device for the pediatric and more precisely neonatal range of values is therefore important.
Finally, a third illustration of this is the important variation in procalcitonin (PCT) levels, which rise as a result of the increased endogenous production in the early postnatal phase during the first 2 days of life before reaching a steady state.
Besides accuracy and precision, any POCT should be compared to a gold standard reference method. This can be achieved through correlation studies between the new assay and the reference method. Correlation quantifies the strength of linear association between two
variables, and is measured by the Pearson correlation coefficient. However, it does not imply that there is good agreement between the two methods, which represent two ways of measuring the same characteristic, since when testing two different devices, neither can be said to offer the truth with certainty. This agreement is measured by a Bland-Altman analysis26.
1. Procalcitonin Measurement for Detection of Serious Bacterial Infection in Febrile in Children: comparison between two automated immunoassays.
Fever without source (FWS) frequently represents a diagnostic challenge in children with fever without source presenting to the Pediatric Emergency Department, accounting for approximately 20% of all febrile patients and up to 20% of visits among children in the 2- to 24-month age group across all years.
Among these, it is crucial to differentiate those suffering from serious bacterial infections (SBI) and necessitating immediate antibiotic treatment from those presenting with focal bacterial infections or viral infections in which only supportive symptomatic treatment is indicated.
Procalcitonin (PCT) is a frequently used biomarker that has been shown effective in better discriminating children with and without serious bacterial infection (SBI), especially in the management of fever without source. PCT measurements in these studies were initially conducted on the BRAHMS Kryptor system, a well validated automated immunoassay method for this biomarker. However, this sophisticated assay can only be performed in a laboratory by well-trained technicians. Recently, PCT methods have become available on several other automated immunoassay systems and these have been compared against the Kryptor method as the reference in a number of adult cohorts, but not on pediatric
patients.
Our goal was to test the concordance of PCT values obtained on Kryptor and on the point-of- care VIDAS systems over the three previously established cut-off ranges in children with FWS, before routinely using the newly available PCT automated immunoassay device.
In 65 samples from pediatric patients with FWS, Kryptor and VIDAS PCT results correlated remarkably well, with no significant difference in the frequency distribution over the 3 previously established PCT cut-off ranges. The strength of the agreement was good (κ=0.759) with an overall concordance of 84.6%. This finding allows clinical implementation of both techniques with the same nominal PCT cut-off values for detection of serious bacterial infection in children presenting with fever without source.
Procalcitonin measurement for detection of serious bacterial infection in febrile children: Comparison between two automated immunoassays
L. Lacroix⁎, S. Manzano, A. Galetto, A. Gervaix
Service of Pediatric Emergency Medicine, Child and Adolescent Department, University Hospitals of Geneva, Geneva, Switzerland
a b s t r a c t a r t i c l e i n f o
Article history:
Received 29 November 2011
received in revised form 30 January 2012 accepted 13 February 2012
Available online 23 February 2012 Keywords:
Procalcitonin Pediatric Bacterial infection Method comparison
Objectives:To assess the concordance of procalcitonin values at 3 cut-off ranges in a cohort of pediatric samples presenting with fever without source, using two different automated immunoassays.
Design and methods:65 frozen samples from children presenting with fever without source were thawed, tested on both Kryptor and VIDAS systems, and compared using a regression analysis, a Bland–
Altman difference plot, and analysis of concordance at the clinically relevant cut-off points.
Results:Kryptor and VIDAS PCT results correlated remarkably well (r = 0.952), with no significant dif- ference in the frequency distribution over the 3 cut-off ranges (p = 0.1384). The strength of the agreement was good (κ= 0.759) with an overall concordance of 84.6%.
Conclusion:Correlation and concordance of PCT values measured by both systems were good. This finding allows clinical implementation of both techniques with the same nominal PCT cut-off values for detection of serious bacterial infection in children presenting with fever without source.
© 2012 The Canadian Society of Clinical Chemists. Published by Elsevier Inc. All rights reserved.
Introduction
Procalcitonin (PCT) is a frequently used biomarker that has been shown effective in better discriminating children with and without serious bacterial infection (SBI)[1,2]. PCT determination is particularly well established in the management of fever without source (FWS), which is a frequent complaint in children presenting in emergency ser- vices[3]. The determination of a rapid and simple biological score to identify children with and without SBI, based on the combined determi- nation of C-reactive protein (CRP), PCT and the results of a urinary dip- stick, has also been recently described[4]. In this score, the likelihood of bacterial infection is linked to the following three PCT cut-off ranges:
b0.5μg/L (serious bacterial infection unlikely), 0.5–1.99μg/L (serious bacterial infection possible), and≥2.0μg/L (serious bacterial infection highly suspected).
PCT measurements in these studies were conducted on the BRAHMS Kryptor system, a well validated automated immunoassay method for this biomarker [5]. However, this sophisticated assay can only be performed in a laboratory by well-trained technicians.
cohorts [6–9]. Generally, there was an excellent correlation with the Kryptor reference method but with some bias (i.e. reporting dif- ferent values showing a systematic error). This bias, however, did not affect the concordance between the methods over previously established medically relevant PCT cut-off ranges[8,9]. Consequently, for these clinical applications, the same nominal cut-off values can be used on different systems. To our knowledge, there is no study com- paring PCT values using two different automated assays in pediatric patients.
Our goal was to test the concordance of PCT values obtained on Kryptor and on the point of care VIDAS systems over the three previ- ously established cut-off ranges in children with FWS. Furthermore, we also assessed the long-term stability of PCT in deep-frozen sam- ples by comparing the retested Kryptor PCT results with the initial results.
Material and methods Study sample
Clinical Biochemistry 45 (2012) 593–595
Contents lists available atSciVerse ScienceDirect
Clinical Biochemistry
j o u r n a l h o m e p a g e : w w w . e l s e v i e r . c o m / l o c a t e / c l i n b i o c h e m
analysis. These PCT values had already been determined with the Kryptor system in our previous study[10].
Laboratory analysis
After initial PCT testing, serum samples were aliquoted and frozen at−30 °C.
Each sample was thawed and clarified by centrifugation, and then retested on both the Kryptor (BRAHMS, Hennigsdorf, Germany) and the VIDAS (bioMérieux, Marcy l'Étoile, France) systems within 2 h from thawing. Methodological details of both PCT methods have been described previously[8,9].
Statistical analysis
Continuous variables are expressed as medians and interquartile ranges (IQR).
Comparison of Kryptor and VIDAS PCT methods was evaluated by regression analysis and a Bland–Altman difference plot[11]. As the standard deviation between both methods increased with the PCT concentration, the difference was plotted as a percentage of average on the y-axis[11]. Correlation was assessed by the Spearman's rank correlation. Concordance between VIDAS and Kryptor PCT values at the clinically relevant cut-off points was assessed by calculating the κcoefficient. Differences in frequency distribution over PCT ranges were analyzed for statistical significance by theχ² test. Finally, sta- bility of the PCT molecule upon prolonged storage was assessed by determining the difference between original and retested Kryptor PCT valued with analysis for statistical significance by the Wilcoxon's paired signed-rank test.
Results
The 65 stored samples (storage time 2.5 to 3.5 years, average 3 years) were successfully analyzed, permitting comparison of both methods in all our samples. Compared with the initial PCT results, the median retested Kryptor PCT value (IQR) showed a significant (pb0.001) decline with 14.4% from 0.794μg/L (0.501–1.706) to 0.680μg/L (0.360–1.520).
This drop in PCT levels was relatively constant with no correlation between storage time (r = 0.1079; p = 0.3922). Furthermore, this shift to lower values upon storage did not result in a significant
change in the frequency distribution over the 3 cut-off ranges (p = 0.1796).
Regression analysis of Kryptor (x) and VIDAS (y) PCT values showed the following parameters (95% CI): slope 1.801 (1.655– 1.947), intercept−0.302 (−0.545 to−0.059), r = 0.952.
The VIDAS system reported higher PCT values than the Kryptor with an average relative difference (2SD limits) between both methods of +26.7% (−24.2 to 77.6%) (Fig. 1). Considering the 3 clini- cally relevant cut-off ranges (b0.5μg/L; 0.5–1.99μg/L and≥2.0μg/L), there was no significant difference in the frequency distribution between both methods (p= 0.1384) and, consequently, the strength of the agreement was good (κ= 0.759) with an overall concordance of 84.6% (Table 1).
Discussion
PCT determinations in children are increasingly used in routine clinical practice. It is both helpful in emergency settings, to sort out underlying bacterial infections in young patients presenting with fever without source[12], as well as in the management of antibiotic treatments administered for documented bacterial infections[13].
Whereas the Kryptor assay has always been considered as the reference method for PCT, new point of care automated assays are now available, such as the VIDAS[8,9]. However, to our knowledge no comparison study has yet evaluated the concordance between Kryptor and other quantitative PCT methods in children. A previous comparison study between semi-quantitatively and quantitatively measured PCT in a pediatric emergency setting showed only moderate agreement which could be explained by a subjective interpretation of the manual assay[14].
The previously validated laboratory risk index score for the identi- fication of SBI in children with FWS was based on PCT values deter- mined on the Kryptor system[12]. To allow routine implementation on a wider scale it is important to validate whether the same PCT cut-off ranges can also be applied on other systems.
The mainfinding of this study is the remarkably good correlation between Kryptor and VIDAS PCT results in a small cohort of pediatric samples. Yet, we also observed a positive measurement bias for VIDAS PCT values compared to values reported by the Kryptor system. Based on the regression equation this means that the nominal cut-off values of 0.5 and 2.0 mg/L for PCT on the Kryptor system would translate into higher values of 0.6 and 3.3 mg/L on the VIDAS system. This posi- tive bias (i.e., higher PCT values on VIDAS compared to Kryptor) has
594 L. Lacroix et al. / Clinical Biochemistry 45 (2012) 593–595
already been previously reported for adult samples[8,9]. In these adult cohorts, however, the higher PCT values on the VIDAS system still resulted in a high concordance between both systems with respect to the previously established clinical cut-off values for the Kryptor system.
Consequently, for clinical decision-making in adults the same nominal cut-off values can be used. In our study, we do not consider that this bias would necessitate different cut-off ranges on the VIDAS system.
First, the average relative bias of 27% (95% CI 24% to 78%) observed in the Bland–Altman difference plot is comparable with the reported PCT reference change value of 61% established in healthy individuals[15].
The only caveat is that this RCF was established in healthy adults and could be different in children. Second, there was a good overall concor- dance of 85% between both methods when considering the previously established 3 PCT cut-off ranges.
Finally, we observed a modest decline in PCT levels of 14% after long term deep frozen storage of our cohort of pediatric samples, with a delay of 2.5 to 3.5 years between initial sampling and thawing for retesting. This decline, without an apparent effect of storage time, is comparable with the decline in PCT values reported in adult sam- ples stored for a similar period[8]. It cannot be excluded that the observed decline is due to the effect of freezing and thawing. Anal- ysis of the effect of repeated freezing and thawing on a small sample set showed a small, albeit non-significant, trend towards declining PCT values (data not shown). Importantly, the decline in PCT results in our cohort of stored pediatric samples did not result in a signifi- cant change in the frequency distribution over the 3 clinically rele- vant cut-off ranges. This observation on long term stability not only justifies the use of this cohort in our study but may also be useful for future retrospective studies in stored pediatric sample cohorts.
In conclusion, in samples from pediatric patients with FWS, there was a good correlation between PCT values measured on the VIDAS and Kryptor systems. Furthermore, the good concordance between both systems with respect to the previously established PCT cut-off ranges in the SBI laboratory risk index score allows us to use the same cut-off values on the VIDAS system for this application.
Acknowledgments
The authors would kindly like to thank Mrs. Florence Hugon and Mrs. Nathalie Grau for performing the assays andfilling in the data- base, Mrs. Patricia de Saint Jean and Mr. Carmelo Martinez for their support during the study, Dr. Wim Houdijk for his precious statistical input in the scientific discussion, and the Clinical Chemistry Laboratory of Geneva University Hospitals for their technical help.
References
[1] Fernandez Lopez A, Luaces Cubells C, Garcia Garcia JJ, Fernandez Pou J. Procalcitonin in pediatric emergency departments for the early diagnosis of invasive bacterial infections in febrile infants: results of a multicenter study and utility of a rapid qualitative test for this marker. Pediatr Infect Dis J 2003 Oct;22(10):895–903.
[2] van Rossum AM, Wulkan RW, Oudesluys-Murphy AM. Procalcitonin as an early marker of infection in neonates and children. Lancet Infect Dis 2004 Oct;4(10):
620–30.
[3] Manzano S, Bailey B, Gervaix A, Cousineau J, Delvin E, Girodias JB. Markers for bacte- rial infection in children with fever without source. Arch Dis Child 2011 May;96(5):
440–6.
[4] Lacour AG, Zamora SA, Gervaix A. A score identifying serious bacterial infections in children with fever without source. Pediatr Infect Dis J 2008 Jul;27(7):654–6.
[5] Hubl W, Krassler J, Zingler C, Pertschy A, Hentschel J, Gerhards-Reich C, et al. Eval- uation of a fully automated procalcitonin chemiluminescence immunoassay. Clin Lab 2003;49(7–8):319–27.
[6] Prieto B, Alvarez FV. Lack of transferability of results between procalcitonin assays. Clin Chem 2009 Dec;55(12):2226–7 (author reply 7–8).
[7] de Wolf HK, Gunnewiek JK, Berk Y, van den Ouweland J, de Metz M. Comparison of a new procalcitonin assay from Roche with the established method on the BRAHMS Kryptor. Clin Chem 2009 May;55(5):1043–4.
[8] Schuetz P, Christ-Crain M, Huber AR, Muller B. Long-term stability of procalcitonin in frozen samples and comparison of Kryptor and VIDAS automated immunoas- says. Clin Biochem 2010 Feb;43(3):341–4.
[9] Hausfater P, Brochet C, Freund Y, Charles V, Bernard M. Procalcitonin measure- ment in routine emergency medicine practice: comparison between two immu- noassays. Clin Chem Lab Med 2010 Apr;48(4):501–4.
[10] Manzano S, Bailey B, Girodias JB, Galetto-Lacour A, Cousineau J, Delvin E. Impact of procalcitonin on the management of children aged 1 to 36 months presenting with fever without source: a randomized controlled trial. Am J Emerg Med 2010 Jul;28(6):647–53.
[11] Dewitte K, Fierens C, Stockl D, Thienpont LM. Application of the Bland–Altman plot for interpretation of method-comparison studies: a critical investigation of its practice. Clin Chem 2002 May;48(5):799–801 (author reply−2).
[12] Galetto-Lacour A, Zamora SA, Andreola B, Bressan S, Lacroix L, Da Dalt L, et al.
Validation of a laboratory risk index score for the identification of severe bacterial infection in children with fever without source. Arch Dis Child 2010 Dec;95(12):
968–73.
[13] Esposito S, Tagliabue C, Picciolli I, Semino M, Sabatini C, Consolo S, et al. Procalci- tonin measurements for guiding antibiotic treatment in pediatric pneumonia.
Respir Med 2011 Sep 27 (Epub ahead of print).
[14] Manzano S, Bailey B, Girodias JB, Cousineau J, Delvin E, Gervaix A. Comparison of procalcitonin measurement by a semi-quantitative method and an ultra-sensitive quantitative method in a pediatric emergency department. Clin Biochem 2009 Oct;42(15):1557–60.
[15] Barassi A, Pallotti F, Melzi d'Eril G. Biological variation of procalcitonin in healthy individuals. Clin Chem 2004 Oct;50(10):1878.
Table 1
VIDAS vs. Kryptor PCT: frequency distribution and concordance.
PCT range (μg/L) Kryptor, n (%) VIDAS, n (%)
b0.5 22 (33.8) 20 (30.8)
0.5–1.99 32 (49.2) 28 (43.1)
≥2.0 11 (16.9) 17 (26.2)
Agreement (%, 95% CI): 55/65 (84.6; 73.5–92.4).
κ(95% CI): 0.759 (0.623–0.896).
p (χ2): 0.1384.
L. Lacroix et al. / Clinical Biochemistry 45 (2012) 593–595 595
2. Impact of the Lab-score on antibiotic prescription rate in children with fever without source: a randomized controlled trial.
After a POC test has been retrospectively validated among the target population, the next step should be prospective clinical implementation of the test and evaluation of the impact of the test on clinical practice.
Isolated or combined clinical signs alone are insufficient to accurately detect children less than 3 years of age at risk for SBI presenting with FWS. In order to improve the diagnostic approach of this entity, which is often challenging in infants and small children, a biological score named Lab-score had been described, based on the 3 variables that were independently associated with SBI and weighed differently according to the odds ratio in the univariate analysis in the original derivation study27. Ranging from 0 to 9, the Lab-score is calculated by the combined determination of procalcitonin (PCT), C-reactive protein (CRP) and presence of leukocyturia or nitrates on the urinary dipstick. It has shown excellent diagnostic characteristics for SBI detection in children with fever without source when tested on the validation set of the original study, with 94% sensitivity (95% CI 74–90) and 78% specificity (95% CI 64–87) for a cut-off point ≥3 27. The Lab-score had also been tested retrospectively on a large external population, showing similarly excellent results28. However, the Lab-score had never been tested prospectively so far. Besides the diagnostic characteristics of the Lab- score, it appeared essential to assess the implication of the Lab-score in clinical practice. The hypothesis was that prospective application of the Lab-score should reduce unnecessary antibiotic prescription.
Concerning PCT, implementation of the new POCT device has been assessed through a prospective impact study of the Lab-score on antibiotic prescription rate in children aged 3 months up to 3 years, presenting with fever without source.
Indeed, among various strategies used to help in the diagnosis of SBI in children, the easy-to- perform Lab-score takes into account biological variables independently associated with SBI, weighed differently according to the odds ratio in the univariate analysis in the original derivation study: CRP, PCT and urinary dipstick. All 3 biological variables composing the
Lab-score may be tested through a POCT device, permitting a rapid and accurate diagnosis to be made at the patient’s bedside.
In both the derivation set and the validation set of the initial study, these variables had been tested in a centralized academic laboratory. The purpose of this study was to prospectively evaluate the impact of the Lab-score on patient management which had not been tested so far, using the novel POCT automated immunoassay device for PCT testing.
No difference regarding antibiotic treatment rate or admission rate was observed in children aged 7 days until 3 years old presenting with FWS when using the Lab-score compared to the classical approach, due to lack of adherence to the Lab-score recommended guidelines.
However, if strictly followed, a significant 26.5 % reduction of antibiotic prescriptions would have been encountered.
RESEARCH ARTICLE
Impact of the Lab-Score on Antibiotic Prescription Rate in Children with Fever without Source: A Randomized Controlled Trial
Laurence Lacroix*, Sergio Manzano, Lynda Vandertuin, Florence Hugon, Annick Galetto-Lacour, Alain Gervaix
Pediatric Emergency Medicine Department, Child and Adolescent Medicine, Geneva University Hospital, Geneva, Switzerland
*laurence.lacroix@hcuge.ch
Abstract
Background:The Lab-score, based on the combined determination of procalcitonin, C-reactive protein and urinary dipstick results, has been shown accurate in detecting serious bacterial infections (SBI) in children with fever without source (FWS) on retrospective cohorts. We aimed to prospectively assess the utility of the Lab-score in safely decreasing antibiotic prescriptions in children with FWS and to determine its diagnostic characteristics compared to common SBI
biomarkers.
Methods:Randomized controlled trial in children 7 days to 36 months old with FWS, allocated either to the Lab-score group (Lab-score reported, blinded WBC count) or to the control group (WBC, bands and C-reactive protein determined, blinded procalcitonin and Lab-score), followed up until recovery. Demographic data, antibiotic prescription rate, admission rate and diagnostic properties of the Lab- score were analyzed.
Results: 271 children were analyzed. No statistically significant difference
concerning antibiotic prescription rate was observed: 41.2% (54 of 131) in the Lab- score group and 42.1% (59 of 140) in the control group (p51.000). If
recommendations based on the Lab-score had been strictly applied, a hypothetical 30.6% treatment rate would have been encountered, compared to the overall 41.7% observed rate (p50.0095). A Lab-score$3 showed the following
characteristics: sensitivity 85.1% (95% CI: 76.5–93.6%), specificity 87.3% (95% CI:
82.7–91.8%), positive predictive value 68.7% (95% CI: 58.7–78.7%), negative predictive value 94.1% (95% CI: 91.5–97.9%), positive and negative likelihood
OPEN ACCESS
Citation:Lacroix L, Manzano S, Vandertuin L, Hugon F, Galetto-Lacour A, et al. (2014) Impact of the Lab-Score on Antibiotic Prescription Rate in Children with Fever without Source: A Randomized Controlled Trial. PLoS ONE 9(12): e115061. doi:10.
1371/journal.pone.0115061
Editor:Susanna Esposito, Fondazione IRCCS Ca’
Granda Ospedale Maggiore Policlinico, UniversitA˜
degli Studi di Milano, Italy Received:July 14, 2014 Accepted:November 11, 2014 Published:December 11, 2014
Copyright:ß2014 Lacroix et al. This is an open- access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and repro- duction in any medium, provided the original author and source are credited.
Data Availability:The authors confirm that all data underlying the findings are fully available without restriction. Data are from the Lab-score study whose authors may be contacted at laurence.lacroix@hcuge.ch
Funding:This work was financially supported by bioMe´rieux (http://www.biomerieux.fr) for data management and statistical analysis. The funder also provided technical support through the loan of the procalcitonin assay. The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing Interests:The authors have thus provided an amended statement of competing interests that explicitly states that this commercial funder has no relationship relating to employment, consultancy, patents, products in development, marketed products, etc. with any of the authors.
This does not alter the authors’ adherence to all PLOS ONE policies on sharing data and materials, as detailed in the guide for authors (page 20, line 9-12).
