HCQ for COVID-19: real-time meta analysis of 419 studies

11 In Silico studies support the efficacy of hydroxychloroquine32-42.

24 In Vitro studies support the efficacy of hydroxychloroquine32,43-65.

3 In Vivo animal studies support the efficacy of hydroxychloroquine47,57,66.

5 studies investigate novel formulations of hydroxychloroquine that may be more effective for COVID-1946,52,67-69.

Preclinical research is an important part of the development of treatments, however results may be very different in clinical trials. Preclinical results are not used in this paper.

92% of early treatment studies report a positive effect, with an estimated improvement of 66% in random effects meta analysis.

Late treatment studies are mixed, with 70% showing positive effects, and an estimated improvement of 22%. Negative studies typically fall into the following categories: they show evidence of significant unadjusted confounding, including confounding by indication; usage is extremely late; or they use an excessively high dosage.

82% of PrEP studies show positive effects, with an estimated improvement of 34%. The majority of negative studies analyze systemic autoimmune disease patients and either do not adjust for the different baseline risk of these patients at all, or do not adjust for the highly variable risk within this group.

88% of PEP studies report positive effects, with an estimated improvement of 30%.

Table 1 summarizes the results for all stages combined, for Randomized Controlled Trials, with different exclusions, and for specific outcomes. Table 2 shows results by treatment stage. Figure 3 plots individual results by treatment stage. Figure 4, 5, 6, 7, and 8 show forest plots for treatment studies with pooled effects, and for studies reporting mortality, hospitalization, case, and viral clearance results.

Many meta-analyses for HCQ have been written, most of which have become obselete due to the continuing stream of more recent studies. More recent analyses with positive conclusions include IHU Marseille which considers significant bias from an understanding of each trial, and Prodromos, Ladapo, García-Albéniz which focus on early or prophylactic use studies.

Meta analyses reporting negative conclusions focus on late treatment studies, tend to disregard treatment delay, tend to follow formulaic evaluations which overlook major issues with various studies, and end up with weighting disproportionate to a reasoned analysis of each study's contribution. For example, Axfors assigns 87% weight to a single trial, the RECOVERY trial81, thereby producing the same result. However, the RECOVERY trial may be the most biased of the studies they included, due to the excessive dosage used, close to the level shown to be very dangerous in Borba (OR 2.8), and with extremely sick late stage patients (60% requiring oxygen, 17% ventilation/ECMO, and a very high mortality rate in both arms). There is little reason to suggest that the results from this trial are applicable to more typical dosages or to earlier treatment (10/22: the second version of this study released 10/22 assigns 74% to RECOVERY and 15% to SOLIDARITY83, which is the only other trial using a similar excessive dosage).

We include all studies in the main analysis, however there are major issues with several studies that could significantly alter the results. Here, we present an analysis excluding studies with significant issues, including indication of significant unadjusted group differences or confouding by indication, extremely late stage usage >14 days post symptoms or >50% on oxygen at baseline, very minimal detail provided, excessive dosages which have been shown to be dangerous, significant issues with adjustments that could reasonably make substantial differences, and reliance on PCR which may be inaccurate and less indicative of severity than symptoms. The aim here is not to exclude studies on technicalities, but to exclude studies that clearly have major issues that may significantly change the outcome. We welcome feedback on improvements or corrections to this. The studies excluded are as follows, and the resulting forest plot is shown in Figure 15.

Heterogeneity in COVID-19 studies arises from many factors including:

The time between infection or the onset of symptoms and treatment may critically affect how well a treatment works. For example an antiviral may be very effective when used early but may not be effective in late stage disease, and may even be harmful. Oseltamivir, for example, is generally only considered effective for influenza when used within 0-36 or 0-48 hours229,230. Baloxavir marboxil studies for influenza also show that treatment delay is critical — Ikematsu et al. report an 86% reduction in cases for post-exposure prophylaxis, Hayden et al. show a 33 hour reduction in the time to alleviation of symptoms for treatment within 24 hours and a reduction of 13 hours for treatment within 24-48 hours, and Kumar et al. report only 2.5 hours improvement for inpatient treatment.

Figure 16 shows a mixed-effects meta-regression of efficacy as a function of treatment delay in HCQ COVID-19 studies, showing that efficacy declines rapidly with treatment delay. Early treatment is critical for COVID-19.

Details of the patient population including age and comorbidities may critically affect how well a treatment works. For example, many COVID-19 studies with relatively young low-comorbidity patients show all patients recovering quickly with or without treatment. In such cases, there is little room for an effective treatment to improve results, for example as in López-Medina et al.

Efficacy may depend critically on the distribution of SARS-CoV-2 variants encountered by patients. Risk varies significantly across variants235, for example the Gamma variant shows significantly different characteristics236-239. Different mechanisms of action may be more or less effective depending on variants, for example the degree to which TMPRSS2 contributes to viral entry can differ across variants240,241.

Effectiveness may depend strongly on the dosage and treatment regimen.

The quality of medications may vary significantly between manufacturers and production batches, which may significantly affect efficacy and safety. Williams et al. analyze ivermectin from 11 different sources, showing highly variable antiparasitic efficacy across different manufacturers. Xu et al. analyze a treatment from two different manufacturers, showing 9 different impurities, with significantly different concentrations for each manufacturer.

The use of other treatments may significantly affect outcomes, including supplements, other medications, or other interventions such as prone positioning. Treatments may be synergistic61,244-254, therefore efficacy may depend strongly on combined treatments.

Across all studies there is a strong association between different outcomes, for example improved recovery is strongly associated with lower mortality. However, efficacy may differ depending on the effect measured, for example a treatment may be more effective against secondary complications and have minimal effect on viral clearance.

The distribution of studies will alter the outcome of a meta analysis. Consider a simplified example where everything is equal except for the treatment delay, and effectiveness decreases to zero or below with increasing delay. If there are many studies using very late treatment, the outcome may be negative, even though early treatment is very effective. All meta analyses combine heterogeneous studies, varying in population, variants, and potentially all factors above, and therefore may obscure efficacy by including studies where treatment is less effective. Generally, we expect the estimated effect size from meta analysis to be less than that for the optimal case. Looking at all studies is valuable for providing an overview of all research, important to avoid cherry-picking, and informative when a positive result is found despite combining less-optimal situations. However, the resulting estimate does not apply to specific cases such as early treatment in high-risk populations. While we present results for all studies, we also present treatment time and individual outcome analyses, which may be more informative for specific use cases.

HCQ studies vary widely in all the factors above. We find a significant effect based on treatment delay. Early treatment shows consistently positive results, while late treatment results are very mixed. Closer analysis may identify factors related to efficacy among this group, for example treatment may be more effective in certain popuations, or more fine-grained analysis of treatment delay may identify a point after which treatment is ineffective.

This section validates the use of pooled effects for COVID-19, which enables earlier detection of efficacy, however pooled effects are no longer required for hydroxychloroquine as of May 2020. Efficacy is now known based on specific outcomes for all studies and when restricted to RCTs. Efficacy based on specific outcomes was delayed by 2.4 months compared to using pooled outcomes. Efficacy based on specific outcomes in RCTs was delayed by 2.6 months compared to using pooled outcomes in RCTs.

For COVID-19, delay in clinical results translates into additional death and morbidity, as well as additional economic and societal damage. Combining the results of studies reporting different outcomes is required. There may be no mortality in a trial with low-risk patients, however a reduction in severity or improved viral clearance may translate into lower mortality in a high-risk population. Different studies may report lower severity, improved recovery, and lower mortality, and the significance may be very high when combining the results. "The studies reported different outcomes" is not a good reason for disregarding results. Pooling the results of studies reporting different outcomes allows us to use more of the available information. Logically we should, and do, use additional information when evaluating treatments—for example dose-response and treatment delay-response relationships provide additional evidence of efficacy that is considered when reviewing the evidence for a treatment.

We present both specific outcome and pooled analyses. In order to combine the results of studies reporting different outcomes we use the most serious outcome reported in each study, based on the thesis that improvement in the most serious outcome provides comparable measures of efficacy for a treatment. A critical advantage of this approach is simplicity and transparency. There are many other ways to combine evidence for different outcomes, along with additional evidence such as dose-response relationships, however these increase complexity.

Trials with high-risk patients may be restricted due to ethics for treatments that are known or expected to be effective, and they increase difficulty for recruiting. Using less severe outcomes as a proxy for more serious outcomes allows faster and safer collection of evidence.

For many COVID-19 treatments, a reduction in mortality logically follows from a reduction in hospitalization, which follows from a reduction in symptomatic cases, which follows from a reduction in PCR positivity. We can directly test this for COVID-19.

Publication of clinical trials is often biased based on conflicts of interest. One way to examine potential bias is to compare prospective and retrospective studies. Prospective trials that involve significant effort are more likely to be published regardless of the result, while retrospective studies are more likely to exhibit bias. For example, researchers may perform preliminary analysis with minimal effort and the results may influence their decision to continue. Retrospective studies also provide more opportunities for the specifics of data extraction and adjustments to influence results.

For HCQ, 78.4% of prospective studies report positive effects, compared to 73.8% of retrospective studies, suggesting a bias toward publishing negative results. Prospective studies show 33% [24‑41%] improvement in meta analysis, compared to 27% [23‑31%] for retrospective studies. Figure 21 shows a scatter plot of results for prospective and retrospective studies.

Figure 22 shows the results by region of the world, for all regions that have > 5 studies. Studies from North America are 2.3 times more likely to report negative results than studies from the rest of the world combined, 45.3% vs. 19.7%, two-tailed z test -4.85, p = 0.0000012110. Berry performed an independent analysis which also showed bias toward negative results for US-based research.

The lack of bias towards positive results is not surprising. Both negative and positive results are very important given the use of HCQ for COVID-19 around the world, evidence of which can be found in the studies analyzed here, government protocols, and news reports, e.g., Bianet, Middle East Eye, BBC, CBS News, Filipova, Barron's, Rathi, Oneindia, Dr. Goldin, AFP, The Indian Express, Government of India, The Australian, The Tico Times, Q Costa Rica, NPR News, Teller Report, Africanews, Belayneh, A., Medical Xpress, Afrik.com, The Africa Report, Parola, Franceinfo, Medical Xpress (B), Barron's (B), Russian Government, PledgeTimes, The Moscow Times, Russian Government (B), The BL, Vanguard, Medical World Nigeria, Pilot News, Anadolu Agency, The Guardian, Nigeria News World, AfricaFeeds, Pan African Medical Journal, The East African, Al-bab, Le Nouvel Afrik, Morocco World News, The North Africa Post, Challenge, Ukrinform, Ministry of Health of Ukraine, Ministry of Health of Ukraine (B), Pleno.News, Anadolu Agency (B), Expats.cz, Ministerstva Zdravotnictví, Efecto Cocuyo, Government of Venezuela, LifeSiteNews, Mosaique Guinee, Archyde, Government of China, France 24, Voice of America, France 24 (B), Global Times, Face 2 Face Africa, Al Arabia, GulfInsider.

HCQ treatment became highly politicized and widely restricted. In many cases, physicians recommending treatment based on clinical evidence lost employment, licenses, and careers. There is a strong bias towards publishing negative results, with negative RCTs receiving priority handling at top journals, and scientists reporting difficulty publishing positive results322-324. Meeus, for example, report that their paper with 4,000 patients reporting favourable outcomes for HCQ+AZ was rejected without peer review from the editors of four different journals.

News organizations show a similar bias. Although 314 studies show positive results, The New York Times, for example, has only written articles for studies that claim HCQ is not effective325-327. As of September 10, 2020, The New York Times still claims that there is clear evidence that HCQ is not effective for COVID-19328. As of October 9, 2020, the United States National Institutes of Health recommends against HCQ for both hospitalized and non-hospitalized patients329.

37 HCQ RCTs have not reported their results, with results missing for 50% of early treatment RCTs and 51% of prophylaxis RCTs, compared to 18% for late treatment RCTs. This is consistent with the higher prevalence of positive studies for early treatment and prophylaxis, and bias against publishing positive results.

The RCTs with missing results are shown in the RCT forest plots, and do not include 65 RCTs that report terminating prior to enrolling 30 patients. The missing trials report a total of 20,747 patients, with 11 trials having actual enrollment of 3,487, and the remainder only reporting estimated numbers. Most trials are known to have started enrollment, while several may have been terminated early. A few trials may have been terminated before enrollment started. This analysis is based on the US clinicaltrials.gov registry. There may be additional missing RCTs not registered in the US. Fincham et al. found 70% of 187 HCQ trials had not reported results as of October 2022. Their analysis includes additional trials that were not registered in clinicaltrials.gov.

Unpublished results are unethical. Future patients are deprived of the ability to make informed decisions. Moreover, RCT participants make a potentially lethal sacrifice for the good of humanity. For existing medications with known efficacy and safety data, patients forego the best treatment choice based on current data. For COVID-19, they know that they may die, depending on their random assignment.

The reasons for lack of publication differ, and may be out of control of the authors. Some RCTs were submitted for publication, but have been caught in journal politicization (authors should release preprints in this case). Others may be held due to decisions of associated organizations, or decisions of only a subset of authors. Most missing RCTs have associations with organizations and/or physicians that restricted HCQ — publication would highlight their liability. Note that in many cases, trials may have been started prior to the extreme politicization.

Generally, it is easy to choose inclusion criteria and assign biased risk evaluations in order to produce any desired outcome in a meta analysis.

COVID-19 treatment studies have many sources of heterogeneity which affect the results, including treatment delay (time from infection or the onset of symptoms), patient population (age, comorbidities), the effect measured and details of the measurement, distribution of SARS-CoV-2 variants, dosage/regimen, and other treatments (anything from supplements, other medications, or other kinds of treatment like prone positioning).

If a treatment is effective early, there is no reason to expect it will also work late. Antivirals are typically only considered effective when used within a short timeframe, for example 0-36 or 0-48 hours for oseltamivir, with longer delays not being effective229,230. For HCQ, the overwhelming majority of trials involve treatment not only after 48 hours but after 5 days - results from these trials are not relevant to earlier usage.

Authors desiring to produce a negative outcome for HCQ need only focus on late treatment studies. For example, Axfors assigns 89% weight to the RECOVERY and SOLIDARITY trials, producing the same negative result. These trials used excessively high non-patient-customized dosage in very sick late stage patients, dosages comparable to those known to be harmful in that context82. The results are not generalizable to typical dosage or treatment of earlier stage hospitalized patients, and certainly not applicable to early treatment, i.e., at first glance we can see that this meta analysis is of no relevance to early treatment.

This paper also does not appear to have been done very carefully. For example, authors include Borba which is assigned 97% weight for CQ. This study has no control group, comparing two different dosages of CQ, which is clear from the abstract of the study.

Axfors approximate early treatment with outpatient use, where they list 5 trials. This is misleading because authors ignore all outcomes other than mortality, and only one of the 5 trials has mortality events, so in reality only one trial is included. Table 1 shows the 5 trials, only one with mortality. The text says something different: "among the five studies on outpatients, there were three deaths, two occurring in the one trial of 491 relatively young patients with few comorbidities and one occurring in a small trial with 27 patients". We do not know what the missing 27 patient trial is, none of the 5 outpatient trials in Table 1 show 27 patients. There is an outpatient trial with 27 patients331, however that trial reports no mortality. It does appear in the meta analysis, but is reported as being an inpatient trial with zero mortality (in reality it was a remotely conducted trial of patients quarantined at home). The supplementary appendix has another different version for outpatient trials, with only 4 trials in Table S3 and Figure S2B (only one with mortality).

Therefore, of the 39 early treatment trials, authors have included data from only one, which contains only 1 death in each of the treatment and control groups. If we read the actual study332, we find that the death in the treatment group was a non-hospitalized patient, suggesting that the death was not caused by COVID-19, or at a minimum the patient did not receive standard care and the comparison here is therefore not valid.

Table 4 shows the reported results of physicians that use early treatments for COVID-19, compared to the results for a non-treating physician (this physician reportedly prescribed early treatment for themself, but not for patients333). The treatments used vary between physicians. Almost all report using ivermectin and/or HCQ, and most use additional treatments in combination. These results are subject to selection and ascertainment bias and more accurate analysis requires details of the patient populations and followup, however results are consistently better across many teams, and consistent with the extensive controlled trial evidence that shows a significant reduction in risk with many early treatments, and improved results with the use of multiple treatments in combination.

Funnel plots have traditionally been used for analyzing publication bias. This is invalid for COVID-19 acute treatment trials — the underlying assumptions are invalid, which we can demonstrate with a simple example. Consider a set of hypothetical perfect trials with no bias. Figure 24 plot A shows a funnel plot for a simulation of 80 perfect trials, with random group sizes, and each patient's outcome randomly sampled (10% control event probability, and a 30% effect size for treatment). Analysis shows no asymmetry (p > 0.05). In plot B, we add a single typical variation in COVID-19 treatment trials — treatment delay. Consider that efficacy varies from 90% for treatment within 24 hours, reducing to 10% when treatment is delayed 3 days. In plot B, each trial's treatment delay is randomly selected. Analysis now shows highly significant asymmetry, p < 0.0001, with six variants of Egger's test all showing p < 0.05334-341. Note that these tests fail even though treatment delay is uniformly distributed. In reality treatment delay is more complex — each trial has a different distribution of delays across patients, and the distribution across trials may be biased (e.g., late treatment trials may be more common). Similarly, many other variations in trials may produce asymmetry, including dose, administration, duration of treatment, differences in SOC, comorbidities, age, variants, and bias in design, implementation, analysis, and reporting.

Summary statistics from meta analysis necessarily lose information. As with all meta analyses, studies are heterogeneous, with differences in treatment delay, treatment regimen, patient demographics, variants, conflicts of interest, standard of care, and other factors. We provide analyses for specific outcomes and by treatment delay, and we aim to identify key characteristics in the forest plots and summaries. Results should be viewed in the context of study characteristics.

Some analyses classify treatment based on early or late administration, as done here, while others distinguish between mild, moderate, and severe cases. Viral load does not indicate degree of symptoms — for example patients may have a high viral load while being asymptomatic. With regard to treatments that have antiviral properties, timing of treatment is critical — late administration may be less helpful regardless of severity.

Details of treatment delay per patient is often not available. For example, a study may treat 90% of patients relatively early, but the events driving the outcome may come from 10% of patients treated very late. Our 5 day cutoff for early treatment may be too conservative, 5 days may be too late in many cases.

Comparison across treatments is confounded by differences in the studies performed, for example dose, variants, and conflicts of interest. Trials with conflicts of interest may use designs better suited to the preferred outcome.

In some cases, the most serious outcome has very few events, resulting in lower confidence results being used in pooled analysis, however the method is simpler and more transparent. This is less critical as the number of studies increases. Restriction to outcomes with sufficient power may be beneficial in pooled analysis and improve accuracy when there are few studies, however we maintain our pre-specified method to avoid any retrospective changes.

Studies show that combinations of treatments can be highly synergistic and may result in many times greater efficacy than individual treatments alone61,244-254. Therefore standard of care may be critical and benefits may diminish or disappear if standard of care does not include certain treatments.

This real-time analysis is constantly updated based on submissions. Accuracy benefits from widespread review and submission of updates and corrections from reviewers. Less popular treatments may receive fewer reviews.

No treatment or intervention is 100% available and effective for all current and future variants. Efficacy may vary significantly with different variants and within different populations. All treatments have potential side effects. Propensity to experience side effects may be predicted in advance by qualified physicians. We do not provide medical advice. Before taking any medication, consult a qualified physician who can compare all options, provide personalized advice, and provide details of risks and benefits based on individual medical history and situations.

Many reviews cover hydroxychloroquine for COVID-19, presenting additional background on mechanisms, formulations, and related results, including342-367.

We focus here on the question of whether HCQ is effective or not for COVID-19. Studies vary significantly in terms of treatment delay, treatment regimen, patients characteristics, and (for the pooled effects analysis) outcomes, as reflected in the high degree of heterogeneity. However, early treatment consistently shows benefits. 92% of early treatment studies report a positive effect, with an estimated improvement of 66% (p < 0.0001).

SARS-CoV-2 infection and replication involves a complex interplay of 50+ host and viral proteins and other factors25-30, providing many therapeutic targets. Over 8,000 compounds have been predicted to reduce COVID-19 risk31, either by directly minimizing infection or replication, by supporting immune system function, or by minimizing secondary complications. Figure 25 shows an overview of the results for hydroxychloroquine in the context of multiple COVID-19 treatments, and Figure 26 shows a plot of efficacy vs. cost for COVID-19 treatments.