We analyze all significant studies concerning the use of HCQ (or CQ) for COVID-19. Search methods, inclusion criteria, effect extraction criteria (more serious outcomes have priority), all individual study data, PRISMA answers, and statistical methods are detailed in Appendix 1. We present random-effects meta-analysis results for all studies, studies within each treatment stage, mortality, hospitalization, cases, viral clearance, after exclusion of studies with critical bias, and for Randomized Controlled Trials (RCTs).

Figure 2 shows stages of possible treatment for COVID-19. Pre-Exposure Prophylaxis (PrEP) refers to regularly taking medication before being infected, in order to prevent or minimize infection. In Post-Exposure Prophylaxis (PEP), medication is taken after exposure but before symptoms appear. Early Treatment refers to treatment immediately or soon after symptoms appear, while Late Treatment refers to more delayed treatment.

Results restricted to RCTs are shown in Figure 9 and Figure 10, showing 17% [3‑29%] improvement for all RCTs, and 23% [8‑36%] improvement when excluding late treatment studies.

Bias in clinical research may be defined as something that tends to make conclusions differ systematically from the truth. RCTs help to make study groups more similar and can provide a higher level of evidence, however they are subject to many biases [Jadad], and analysis of double-blind RCTs has identified extreme levels of bias [Gøtzsche]. For COVID-19, the overhead may delay treatment, dramatically compromising efficacy; they may encourage monotherapy for simplicity at the cost of efficacy which may rely on combined or synergistic effects; the participants that sign up may not reflect real world usage or the population that benefits most in terms of age, comorbidities, severity of illness, or other factors; standard of care may be compromised and unable to evolve quickly based on emerging research for new diseases; errors may be made in randomization and medication delivery; and investigators may have hidden agendas or vested interests influencing design, operation, analysis, and the potential for fraud. All of these biases have been observed with COVID-19 RCTs. There is no guarantee that a specific RCT provides a higher level of evidence.

High quality RCTs for novel acute diseases are more challenging, with increased ethical issues due to the urgency of treatment, increased risk due to enrollment delays, and more difficult design with a rapidly evolving evidence base. For COVID-19, the most common site of initial infection is the upper respiratory tract. Immediate treatment is likely to be most successful and may prevent or slow progression to other parts of the body. For a non-prophylaxis RCT, it makes sense to provide treatment in advance and instruct patients to use it immediately on symptoms, just as some governments have done by providing medication kits in advance. Unfortunately, no RCTs have been done in this way. Every treatment RCT to date involves delayed treatment. Among the 51 treatments we have analyzed, 64% of RCTs involve very late treatment 5+ days after onset. No non-prophylaxis COVID-19 RCTs match the potential real-world use of early treatments (they may more accurately represent results for treatments that require visiting a medical facility, e.g., those requiring intravenous administration).

RCTs have a bias against finding an effect for interventions that are widely available — patients that believe they need the intervention are more likely to decline participation and take the intervention. RCTs for hydroxychloroquine are more likely to enroll low-risk participants that do not need treatment to recover, making the results less applicable to clinical practice. This bias is likely to be greater for widely known treatments, and may be greater when the risk of a serious outcome is overstated. This bias does not apply to the typical pharmaceutical trial of a new drug that is otherwise unavailable.

Evidence shows that non-RCT trials can also provide reliable results. [Concato] find that well-designed observational studies do not systematically overestimate the magnitude of the effects of treatment compared to RCTs. [Anglemyer] summarized reviews comparing RCTs to observational studies and found little evidence for significant differences in effect estimates. [Lee] shows that only 14% of the guidelines of the Infectious Diseases Society of America were based on RCTs. Evaluation of studies relies on an understanding of the study and potential biases. Limitations in an RCT can outweigh the benefits, for example excessive dosages, excessive treatment delays, or Internet survey bias could have a greater effect on results. Ethical issues may also prevent running RCTs for known effective treatments. For more on issues with RCTs see [Deaton, Nichol].

Currently, 36 of the treatments we analyze show statistically significant efficacy or harm, defined as ≥10% decreased risk or >0% increased risk from ≥3 studies. Of the 36 treatments with statistically significant efficacy/harm, 23 have been confirmed in RCTs, with a mean delay of 4.4 months. For the 13 unconfirmed treatments, 4 have zero RCTs to date. The point estimates for the remaining 9 are all consistent with the overall results (benefit or harm), with 8 showing >20%. The only treatment showing >10% efficacy for all studies, but <10% for RCTs is aspirin.

We need to evaluate each trial on its own merits. RCTs for a given medication and disease may be more reliable, however they may also be less reliable. For off-patent medications, very high conflict of interest trials may be more likely to be RCTs, and more likely to be large trials that dominate meta analyses.