Now Reading
What you read is what you get: Are orthodontic randomized clinical trials correctly titled?

What you read is what you get: Are orthodontic randomized clinical trials correctly titled?



American Journal of Orthodontics and Dentofacial Orthopedics, 2022-04-01, Volume 161, Issue 4, Pages e303-e315, Copyright © 2021


Introduction

Because of their importance in health care research, randomized controlled trials (RCTs) should be correctly labeled in the title to ensure it reflects the methodologic standards undertaken. This study aimed to assess whether orthodontic RCTs are correctly titled RCTs, and trial characteristics that influence correct labeling in titles were explored.

Methods

Orthodontic RCTs published between January 1, 2015 and September 30, 2020 in 7 orthodontic journals were identified. For a trial to be considered an RCT, random allocation sequence generation and allocation concealment mechanisms should be reported in the study methodology. Trial characteristics at the RCT level were extracted, and frequency distributions were calculated for the included trial characteristics. A 2-stage continuation ratio ordinal logistic regression model was used to assess associations between the dependent variable (non-RCT, unclear, RCT) and the trial characteristics.

Results

One-hundred and seventy-three RCTs were analyzed. Of these, 112 (64.7%) were assessed as true RCTs, with 109 (64.1%) correctly labeled as RCTs. The American Journal of Orthodontics and Dentofacial Orthopedics (AJODO) published the most trials that were correctly labeled as RCTs (90.5%). In the adjusted model, the type of journal influenced the correct labeling, with all journals compared with AJODO having lower odds of correct RCT classification. Multicenter trials were associated with higher odds of being correctly labeled as an RCT.

Conclusions

Compared with previous investigations, the correct labeling of orthodontic RCTs in the title of the report has improved substantially. The type of journal and multicenter trials influenced the correct labeling of a trial as an RCT. The number and odds of RCTs being correctly labeled as RCTs were higher in trials published in AJODO.

Highlights

  • Randomized controlled trials (RCTs) should be correctly labeled in the titles.

  • Sixty-five percent of eligible articles were assessed as being true RCTs.

  • Sixty-four percent were correctly labeled in the title as RCTs.

  • Multicenter trials were more likely to be correctly labeled as an RCT.

  • The number and odds of RCTs being correctly labeled were higher in the American Journal of Orthodontics and Dentofacial Orthopedics.

It is well-established that randomized clinical trials (RCTs) are considered the gold standard methodology for assessing the effectiveness of an intervention because of the key element of randomization, which ensures the minimization of bias. The use of randomization is viewed as the optimal method to prevent systematic differences between baseline characteristics of participants in different intervention groups in terms of both known and unknown confounders. Importantly, conclusions drawn regarding cause and effect can be based on the findings of RCTs with greater confidence than other study designs.

Randomization in trial methodology is defined by the following steps: generation of random allocation, allocation concealment, and implementation of randomization. Several approaches are available for the random number sequence generation (allocation), such as using random tables or preferably software algorithms capable of producing random sequences. Allocation concealment ensures that both trial participants and clinicians cannot predict intervention group allocation. Methods of allocation concealment include the use of sequentially numbered sealed opaque envelopes, computerized allocation, or allocation assigned from ideally an external center that does not involve anyone associated with the trial. Best practices advise that randomization be performed offsite or by an external party to ensure unbiased implementation. A lack of adequate reporting of randomization methods has been cited within the biomedical literature over the years. This systematically prevents the development of robust evidence through systematic reviews, as poor quality RCTs can skew the conclusions of meta-analyses, or the trials cannot be included in the analysis if the methods are not adequately described.

The first recommendation on the consolidated standards of reporting trials (CONSORT) checklist is that trials with randomized participant assignments should be clearly labeled in the title. This ensures the trial is appropriately indexed, and clinicians can be confident that if randomized is in the title of the report, this indicates that the participants were randomly assigned to their comparison groups. An important reason for identifying the trial design is to ensure appropriate indexing in electronic databases, thus ensuring greater ease of identifying trials that could be included and used to calculate interventional treatment estimates in systematic reviews. However, it has been shown that the correct labeling of RCTs within the dental literature is suboptimal. , In the assessment of 112 orthodontic trials, only 33 (29.5%) were classified as true RCTs, with 52 (46.4%) marked as unclear and 27 (24.1%) as not RCTs.

The last assessment of the correct labeling of orthodontic trials as RCTs was undertaken 10 years ago, and it is unknown whether this practice has improved. This current study aimed to assess whether true orthodontic trials RCTs as defined by the reporting of both random allocation and concealment are correctly labeled in their report titles as RCTs. In addition, trial characteristics that influence the correct use of RCT in the title of the report were explored.

Material and methods

Orthodontic RCTs published between January 1, 2015 and September 30, 2020 in the following 7 orthodontic journals were eligible: American Journal of Orthodontics and Dentofacial Orthopedics (AJODO), European Journal of Orthodontics (EJO), Journal of Orthodontics , Angle Orthodontist (ANGLE), Orthodontics and Craniofacial Research , Journal of Orofacial Orthopedics , and Australasian Orthodontic Journal were included in this study. These journals were selected to allow comparison with previous research findings. The term “randomized controlled trial” was screened in the title, abstract, and methodology of articles for potential inclusion. In accordance with the Cochrane criteria for the selection of RCTs, the following inclusion criteria were used: human participants, interventions related to health care, experimental studies, presence of a control or comparative group, randomization of participants to control and treatment groups, other trials with terminology in the title or abstract such as “prospective,” “comparative,” “efficacy,” or when an indication was given that a comparison of treatment groups was undertaken prospectively were analyzed to establish whether randomization was implemented. Only studies published in English were included. Case reports, review articles, editorials, systematic reviews, follow-up studies of earlier RCT reports, protocols, and retrospective studies were excluded.

Both journal Web sites and an electronic database (Medline via PubMed) were searched by 1 author (G.K) to identify eligible trials. All titles and abstracts were screened independently by 2 authors (G.K and J.S). Full-text articles of abstracts fulfilling the inclusion criteria were retrieved and further analyzed for eligibility independently by 2 authors (G.K and J.S). Any disagreements in the final articles were resolved by discussion among the authors.

To ensure calibration and consistency in interpreting each study characteristic, a pilot assessment of 10 articles was undertaken between 2 authors (G.K and J.S) (100% agreement achieved). All study characteristics were initially extracted by a single author (G.K) and entered into a prepiloted Microsoft Excel (Microsoft, Redmond, Wash) data collection sheet. A second author (J.S) independently cross-checked the collected data. Any disagreements were resolved by discussion between both authors (G.K and J.S). No disagreements were identified.

At the level of each RCT, the following study characteristics were extracted: year of publication, number of authors, journal title, continent of corresponding author (Europe, Americas, Asia, and other), center (single or multi), design (crossover, parallel, split-mouth, split crossover), involvement of statistician which was inferred from the list of author roles and affiliations, or if there was any mention in the full text of a statistician or data analyst (yes or no). Based on previous literature, , , for a trial to be correctly described as a true RCT, random allocation sequence generation with additional details of any restriction methods (eg, blocking and block size and random concealment mechanisms) should be reported. On this basis, the correct labeling of a trial as an RCT was defined on the correct description of the randomization process (RCT, non-RCT, and unclear).

Statistical analysis

Frequency distributions were calculated for the included trial characteristics. The feature selection algorithm Boruta was used to identify significant predictors associated with the correct labeling of the trial as an RCT. A 2-step continuation ratio ordinal logistic model was fit with the significant predictors from the feature selection stage. This model is useful when there is a progression through stages which is the case here and is an alternative to models based on cumulative probabilities. In the first stage, a comparison of RCTs or unclear vs non-RCT was conducted, and in the second stage, a comparison of RCTs vs unclear given the study was not a non-RCT. Odd ratio, 95% confidence intervals, and P values were calculated. A 2-tailed P value of 0.05 was considered statistically significant. The Hosmer-Lemeshow test was used to assess model fit. All analyses were performed using Stata (version 16.1; StataCorp, College Station, Tex) and R Program (version 4.0.3; R Foundation for Statistical Computing, Vienna, Austria)

Results

A total of 173 RCTs fulfilled the eligibility criteria ( Fig 1 ; excluded trials with reasons; Supplementary Table ). Trial characteristics are reported in Table I . In terms of journal type, EJO published the most interventional trials over the examined period, followed by ANGLE ( Table I ). However, not all of these were deemed true RCTs, with only 57.1% of trials from EJO considered true RCTs and 51.9% from ANGLE. In contrast, AJODO published the most trials which were correctly labeled as RCTs (90.5%). True RCTs also frequently had authors based in Europe (68.2%), in single centers (60%), and with parallel design type trials (68.1%). A statistician was involved in 82 of the 173 trials (47.4%), of which 62 (75.6%) of these were considered true RCTs. The distribution of trials by journal title and RCT status (RCT, unclear, and non-RCT) is shown in Figure 2 . Of the included studies, 112 (64.7%) were assessed as being true RCTs, with 51 (29.5%) and 10 (5.9%) having an unclear and non-RCT status, respectively. Within this sample of 112 true RCTs, 109 (64.1%) were correctly labeled as RCTs in the title ( Table I ).

Study identification flow diagram. JO, Journal of Orthodontics .
Fig 1
Study identification flow diagram. JO, Journal of Orthodontics .

Table I
Trial characteristics and status (n = 173)
Characteristic RCT Unclear Non-RCT Total
Year of publication
2015 9 (75.0) 3 (25.0) 0 (0.0) 12 (6.9)
2016 23 (71.9) 7 (21.9) 2 (6.2) 32 (18.6)
2017 16 (57.1) 11 (39.3) 1 (3.6) 28 (16.2)
2018 27 (69.2) 8 (20.5) 4 (10.3) 39 (22.5)
2019 20 (50.0) 18 (45.0) 2 (5.0) 40 (23.1)
2020 17 (77.3) 4 (18.2) 1 (4.5) 22 (12.7)
Journal title
AJODO 38 (90.5) 4 (9.5) 0 (0.0) 42 (24.3)
ANGLE 27 (51.9) 20 (38.5) 5 (9.6) 52 (30.1)
EJO 32 (57.1) 22 (39.3) 2 (3.6) 46 (32.4)
Other 9 (69.2) 3 (23.1) 1 (7.7) 13 (7.5)
JO 9 (90.0) 1 (10.0) 0 (0.0) 10 (5.7)
Continent of the corresponding author
Europe 58 (68.2) 25 (29.4) 2 (2.4) 85 (49.1)
Americas 15 (48.4) 13 (41.9) 3 (9.7) 31 (17.9)
Asia or other 39 (68.4) 13 (22.8) 5 (8.8) 57 (33.0)
Center
Single 84 (60.0) 47 (33.6) 9 (6.4) 140 (80.9)
Multiple 28 (84.8) 4 (12.1) 1 (3.1) 33 (19.1)
Study design
Crossover 2 (40.0) 2 (40.0) 1 (20.0) 5 (2.9)
Parallel 94 (68.1) 40 (29.0) 4 (2.9) 138 (79.8)
Spilt-mouth 16 (53.3) 9 (30.0) 5 (16.7) 30 (17.3)
Involvement of statistician
No 50 (54.9) 34 (37.4) 7 (7.7) 91 (52.6)
Yes 62 (75.6) 17 (20.7) 3 (3.7) 82 (47.4)
Labeled as RCT in title
No 3 (100.0) 0 (0.0) 0 (0.0) 3 (1.7)
Yes 109 (64.1) 51 (30.0) 10 (5.9) 170 (98.3)
Total 112 (64.7) 51 (29.5) 10 (5.8) 173 (100)
JO, Journal of Orthodontics.

Row totals and column percentages for column total.

Distribution of trials by journal title and RCT status (RCT, Unclear and Non-RCT) for the current study period (2015-2020) and previous research 9 (1979-2011). JO, Journal of Orthodontics.
Fig 2
Distribution of trials by journal title and RCT status (RCT, Unclear and Non-RCT) for the current study period (2015-2020) and previous research (1979-2011). JO, Journal of Orthodontics.

The feature selection algorithm identified journal, center, and geographic location as significant predictors used those for the multivariable analysis ( Fig 3 ). During the first stage, all journals were used, but in the second stage, AJODO and Journal of Orthodontics were excluded because they did not contribute to the analysis having zero non-RCTs while causing estimation problems. An exact logistic model that handles zeros was numerically infeasible.

Boxplot from the feature selection algorithm. Green indicates significant outcome predictors.
Fig 3
Boxplot from the feature selection algorithm. Green indicates significant outcome predictors.

In the multivariable analysis, the type of journal and center maintained their significance, whereas the corresponding author’s continent did not. All journals compared with AJODO showed lower odds of rating a trial as an RCT, and so did single-center trials in both stages of the continuation ratio model ( Table II ). The corresponding Hosmer-Lemeshow tests for model fit were not significant for both stages of the continuation ratio model ( P = 0.32 and P = 0.31).

Table II
Odds ratio, 95% confidence interval, and P values from the 2-stage continuation ratio model
Covariate Odds ratio 95% confidence interval P value
Comparison: RCT or unclear vs non-RCT
Journal
AJODO Reference
ANGLE 0.13 0.04-0.45 <0.01
EJO 0.1 0.03-0.33 <0.01
Other 0.18 0.04-0.91 0.04
JO 0.58 0.05-6.21 0.66
Continent of the corresponding author
Europe Reference
Americas 0.42 0.14-1.23 0.12
Asia or other 1.17 0.50-2.71 0.72
Center
Single Reference
Multiple 3.7 1.22-11.22 0.02
Comparison: RCT vs unclear
Journal
AJODO Reference
ANGLE 0.17 0.05-0.57 <0.01
EJO 0.11 0.03-0.36 <0.01
Other 0.23 0.04-1.29 0.09
JO 0.55 0.05-5.91 0.62
Continent of the corresponding author
Europe Reference
Americas 0.44 0.15-1.32 0.14
Asia or other 1.57 0.64-3.84 0.33
Center
Single Reference
Multiple 4.44 1.34-14.69 0.01

You're Reading a Preview

Become a DentistryKey membership for Full access and enjoy Unlimited articles

Become membership

If you are a member. Log in here