Article
Reference
Adherence to guidelines for improved quality of data reporting: where are we today?
ELIA, Nadia, TRAMER, Martin
ELIA, Nadia, TRAMER, Martin. Adherence to guidelines for improved quality of data reporting:
where are we today? European Journal of Anaesthesiology , 2011, vol. 28, no. 7, p. 478-80
DOI : 10.1097/EJA.0b013e3283461868 PMID : 21659804
Available at:
http://archive-ouverte.unige.ch/unige:25647
Disclaimer: layout of this document may differ from the published version.
1 / 1
Copyright © European Society of Anaesthesiology. Unauthorized reproduction of this article is prohibited.
Adherence to guidelines for improved quality of data reporting: where are we today?
Nadia Elia and Martin R. Trame`r
European Journal of Anaesthesiology2011,28:478–480
When scientific publications report their data in a manner that is incomplete or inappropriate, what might otherwise be an interesting and promising research project loses value because clinicians, patients and health policy makers are unable to use it for rational decision-making.
The time and effort of the research team is wasted, and for the patients who have been enrolled into a study that is not going to deliver, there is an ethical dimension. In response to these concerns, over the last 15 years, evi- dence-based reporting guidelines have been devised and published, under better known acronyms such as CON- SORT (CONsolidated Standards Of Reporting Trials), STROBE (STrengthening the Reporting of OBserva- tional studies in Epidemiology), STARD (STAndards for the Reporting of Diagnostic accuracy studies), PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses; formerly known as QUOROM) and SQUIRE (Standards for QUality Improvement Reporting Excellence). These guidelines are intended to improve quality and completeness of reporting in scientific publications. They provide useful information on how to describe research methods and findings, for example by indicating the minimum amount of methodological detail that needs to be included. The aim is to keep experimental designs transparent and to enable informed readers to understand and judge the reliability of the research presented. A recent systematic review suggested that the adoption of the CONSORT statement has been associated with an improvement in the reporting of randomised controlled trials (RCTs).1 Not surprisingly, these reporting guidelines have been endorsed by a number of major medical journals.
TheEuropean Journal of Anaesthesiology(EJA) endorsed the CONSORT statement 10 years ago.2Although the Jour- nal’s Guide for Authors has specified ever since that the CONSORT guidelines should be followed when reporting RCT, it is unclear to what extent this instruction has been obeyed, so we have set out to address that question.
We screened all issues of the EJA, from January to December 2010, and classified all original articles accord-
ing to their study design. Editorials, reviews or clinical guidelines were not considered. Studies concerning animal research were grouped under the same ‘animal research’ heading, whatever their design.
We then checked adherence of all RCT to the updated CONSORT checklist (http://www.consort-statemen- t.org/consort-statement/overview0/). In order to better identify the specific problems related to data reporting in theEJA, item 9 of the original checklist was further split into two items, 9a (Mechanism used to implement the random allocation sequence) and 9b (Describing any step taken to conceal the sequence until interventions were assigned). Similarly, item 10 was split into three items, 10a (Who generated the random allocation sequence?), 10b (Who enrolled participants?) and 10c (Who assigned participants to interventions?). This made 39 items to be checked in total.
We computed the number and proportion of RCT report- ing each item. Adherence was coded as ‘yes (1)’ or ‘no (0)’. For item 2a (Scientific background and explanation for rationale), one point was awarded if an attempt was made to justify the research, after which the quality and validity of the original authors’ justification was not further evaluated. Similarly, for item 20 (Trial limitations [bias, imprecision]), one point was attributed if limita- tions of the study design or biases were mentioned in the discussion, even if they were not exhaustive. Item 22 (Interpretation consistent with results, balancing benefits and harms, considering other relevant evidence) was not analysed. The maximum number of items per report was 39.
We found 130 original research articles that were pub- lished in the EJA in 2010; of these, one had been retracted, 50 (38.8%) were RCT, 26 (20.1%) were cohort studies, 21 (16.3%) concerned animals, 9 (7.0%) were surveys and 23 (17.8%) included other study designs.
Of the 50 RCT, only 24% were clearly labelled as
‘randomised’ in their title, 56% stated a specific study objective and 48% contained a complete and well defined prespecified primary outcome (Table 1). Setting and location of the study, as well as the dates defining the periods of patient recruitment and follow-up were described in less than one-third of the reports. The implementation of the random sequence was described with sufficient detail in less than 20% of the trials. Finally, only six trials (12%) provided a registration number and the name of the trial registry, giving access to some aspects of their study protocol.
EDITORIAL
From the Institute of Social and Preventive Medicine, Medical Faculty, University of Geneva, Geneva, Switzerland (NE), and Division of Anaesthesiology, Geneva University Hospitals, Geneva Switzerland (MRT)
Correspondence to Nadia Elia, MD, MSc, Institute of Social and Preventive Medicine, Medical Faculty, University of Geneva, 1211 Geneva 4, Switzerland E-mail: nadia.elia@unige.ch
0265-0215!2011 Copyright European Society of Anaesthesiology DOI:10.1097/EJA.0b013e3283461868
Copyright © European Society of Anaesthesiology. Unauthorized reproduction of this article is prohibited.
None of the trials reported on items 3b (Important changes to methods after commencement, with reasons), 6b (Any changes to trial outcomes after trial commenced, with reasons), or 7b (Explanation of any interim analyses and stopping guidelines). It is unclear whether these items were not reported because there were no changes to the methods or outcomes made in any of these trials, whether no interim analyses were performed, or whether this reflects incomplete reporting. Similarly, only two reports specifically described the reasons for early termination of the trial; it is uncertain whether others remained hidden.
In only 12 of the 50 RCTs (24%) could more than 50% of the 39 CONSORT items be identified (Fig. 1). The
median number of items reported was 16 (41% of all items), ranging from eight (21%) to 29 (74%). One trial each reported 293and 264items. Eight trials mentioned that they had followed the CONSORT statement for the establishment of a flow chart, but only one clearly stated that ‘the design and the description adhere to the Consolidated Standards of Reporting clinical Trial statement’.3 That trial reported 74% of expected items.
In 2010, the quality of data reporting of RCT in theEJA has been in the range of that previously described in a similar setting. In a study of the quality of data reporting in 108 obstetric anaesthesia RCTs, Halpernet al.5found
Guidelines for improved quality of data reporting 479
Table 1 Adherence to CONSORT guidelines of randomised controlled trials published in theEuropean Journal of Anaesthesiologyduring 2010
Section/Topic Item n- Item description n %
Title and abstract
1a Identification as a randomised trial in the title 12 24
1b Structured summary of trial design, methods, results, conclusion 50 100
Introduction
2a Scientific background and explanation of rationale 50 100
2b Specific objectives or hypotheses 28 56
Methods
Trial design 3a Description of trial design 29 58
3b Important changes to methods after trial commencement, with reasons 0 0
Participants 4a Eligibility criteria for participants 50 100
4b Settings and locations where the data were collected 14 28
Interventions 5 Detailed description of interventions for each group (how, when) 50 100
Outcomes 6a Completely defined prespecified primary and secondary outcome measures (how, when) 24 48
6b Any changes to trial outcomes after trial commenced, with reasons 0 0
Sample size 7a How was sample size determined? 37 74
7b Explanation of any interim analyses and stopping guidelines 0 0
Random sequence generation 8a Method used to generate the random allocation sequence 30 60
8b Type of randomisation; detail of any restriction (block size) 4 8
Allocation concealment 9aa Mechanism used to implement the random allocation sequence 25 50
9ba Description of step taken to conceal the sequence until interventions were assigned 9 18
Implementation 10aa Who generated the random allocation sequence? 8 16
10ba Who enrolled participants? 5 10
10ca Who assigned participants to interventions? 3 6
Blinding 11a Who was blinded after assignment to intervention? 27 54
11b How: description of the similarity of the interventions 14 28
Statistical methods 12a Statistical methods used to compare groups for primary and secondary outcomes 43 86
12b Methods for additional analyses (subgroup, adjusted analyses) 5 10
Results
Participant flow 13a Flow chart (or number randomised, received treatment, and analysed) 14 28
13b For each group, losses and exclusions after randomisation, with reasons 33 66
Recruitment 14a Dates defining the periods of recruitment and follow-up 16 32
14b Why the trial was ended or was stopped 2 4
Baseline data 15 A table showing baseline demographic/characteristics for each group 46 92
Number analysed 16 For each group, number of participants included in each analysis 10 20
Outcomes and estimation 17a For each primary and secondary outcome, results for each group, estimated effect size and its precision
45 90
17b For binary outcomes, presentation of both absolute and relative effect size recommended 1 2 Ancillary analyses 18 Result of any other analyses performed (subgroup, adjusted) distinguishing
prespecified from exploratory
5 10
Harms 19 All important harms or unintended effects in each group 33 66
Discussion
Limitations 20 Trial limitations (bias, imprecision) 33 66
Generalisability 21 Generalisability of the trial findings 6 12
Interpretation 22 Interpretation consistent with results, balancing benefits and harms, considering other relevant evidence
Not analysed Other information
Registration 23 Registration number and name of trial registry 6 12
Protocol 24 Protocol accessibility 6 12
Funding 25 Funding, role of funders 27 54
50 randomised controlled trials published between January 2010 and December 2010. n, number of trials reporting the item. The denominator of items 11a and 11b is 35 since 16 trials were not blinded. aItem 9 of the original CONSORT 2010 checklist was split into 9a and 9b and item 10 was split into 10a, 10b and 10c.
European Journal of Anaesthesiology 2011, Vol 28 No 7
Copyright © European Society of Anaesthesiology. Unauthorized reproduction of this article is prohibited.
the median proportion of correctly described CONSORT items was 65%.
We feel there is room for improvement in the quality of data reporting of RCT in the EJA. Some items of the CONSORT checklist deserve a higher priority for improvement than others, such as identification of the study design in the title, or a clearly defined primary endpoint.
We have to be aware though that these results focus on the quality of data reporting only and do not allow any conclusion regarding the quality of the RCTs published in theEJA. As Dechartreset al.6highlighted in a recently published systematic review ‘a study design with several biases that is well reported can receive a high-quality (of reporting) score’.
The editorial board of theEJA has introduced a num- ber of measures to improve the quality of data reporting within the journal. A first step was to change the structure of the abstracts by adding more subheadings.
We hope this will further help our authors to summarise their data. Among other measures that will follow is the creation of a new Methods, Statistics and Epidemiology section within the journal, in which we will, for instance, discuss the rationale behind the different items of the CONSORT guideline and present further reporting guidelines (for instance, PRISMA for the reporting of meta-analyses). In that section, we will also review common methodological issues related to statistics, epidemiology and study validity. We intend to remain short, practical, comprehensible and anaes- thesiology oriented.
We hope these essays will assist authors to design their studies and write their manuscripts. Also, they should provide a practical guide for peer-reviewers. And last but not least, they shall engage our readers and help us to improve both the quality and attractiveness of theEJA.
Acknowledgements
Dr Elia is an Associate Editor and Prof Trame`r is the Editor-in- Chief of the European Journal of Anaesthesiology. The authors declare no support from any organisation for the submitted work, no financial ties with any organisations that might have an interest in the submitted work and no other relationships or activities that could have influenced the submitted work.
This article was checked and accepted by the editors, but was not sent for external peer-review.
References
1 Plint AC, Moher D, Morrison A,et al.Does the CONSORT checklist improve the quality of reports of randomised controlled trials? A systematic review.
Med J Aust2006;185:263–267.
2 Trame`r MR. CONSORT, QUOROM, and structured abstracts: new rules for authors, new tools for readers.Eur J Anaesthesiol2001;18:1–2.
3 Rasmussen ML, Mathiesen O, Dierking G,et al.Multimodal analgesia with gabapentin, ketamine and dexamethasone in combination with paracetamol and ketorolac after hip arthroplasty: a preliminary study.Eur J Anaesthesiol 2010;27:324–330.
4 Chaparro LE, Gallo T, Gonzalez NJ,et al.Effectiveness of combined haloperidol and dexamethasone versus dexamethasone only for postoperative nausea and vomiting in high-risk day surgery patients: a randomized blinded trial.Eur J Anaesthesiol2010;27:192–195.
5 Halpern SH, Darani R, Douglas MJ,et al.Compliance with the CONSORT checklist in obstetric anaesthesia randomised controlled trials.Int J Obstet Anesth2004;13:207–214.
6 Dechartres A, Charles P, Hopewell S,et al.Reviews assessing the quality or the reporting of randomized controlled trials are increasing over time but raised questions about how quality is assessed.J Clin Epidemiol2011;
64:136–144.
480 Elia and Trame`r
Fig. 1
Number of trials
% of items reported 0
2 4 6 8 10 12 14 16 18
0-10 11-20 21-30 31-40 41-50 51-60 61-70 71-80 81-90 91-100
Percentage of the 39 items of the CONSORT checklist that were reported in 50 randomised controlled trials.
European Journal of Anaesthesiology 2011, Vol 28 No 7
