Health authorities and mainstream media have set off all alarms to stop the indiscriminate use of ivermectin. Health authorities claim they warn people because there is insufficient evidence regarding effectiveness and safety. The original claims comprised the efficacy came from small studies, the study designs are poor, and the target population was from third world countries. Not long ago, the argumentation against the use of ivermectin shifted. An incisive scrutiny revealed flaws in some studies showing positive results. Ivermectin advocates criticize studies that have shown neutral results. The criticism comprises inappropriate doses or late use. If ivermectin is effective, a standard dose of 0.2 mg, although not ideal for some, should show better results than a placebo.
Vallejos et al., is a peer reviewed and published study conducted in the province of Corrientes in Argentina. The study included 501 patients, 250 in the ivermectin arm, and 251 in the placebo. Patients received a dose on day 0 and the second one 24 hours after. The dose was from 12 to 24 mg according to the weight of the patients. Study results showed a 5.6% hospitalization rate in the ivermectin group, and 8.4% in the placebo group. Despite showing better results in the treated arm, the variation in the rate of hospitalizations did not provide statistical significance, and the conclusion was that ivermectin has no effective on preventing hospitalizations.
The flaws
The randomization process.
A randomized selection tries to generate two equivalent groups. If the groups aren’t equivalent, the study will show incomparable results and unreliable conclusions. It is faulty to compare two groups with different ages, levels of comorbidities, or progresses of the disease. At first glance, the table of base characteristics shows the groups are equivalent. However, there is an unexpected result. Patients who received ivermectin and required mechanical ventilation did it on average 5 days after starting treatment. In the placebo group, the average was 10 days. Assuming a neutral effect from ivermectin use, the only plausible explanation is that patients in the ivermectin group were sicker at the beginning of the study.
As the study includes a base measurement of oxygen saturation, it’s possible to compare the initial state of deterioration in both groups. Regarding the average saturation values, although the control group is a little better (96.31 versus 96.09), the averages are close, and the difference seems to show that they are equivalent. However, when going through the standard deviation (2.01 vs 3.16) the difference is unusual.
Oxygen saturation follows a truncated normal distribution. It is truncated because saturation values range between 0% and 99%. From the mean and standard deviation, it is possible to estimate the number of sicker patients in both groups. Sicker patients are the one having an oxygen saturation below 92%. A fair randomization should produce a similar number of sicker patients in both groups.
When doing the estimates, we find a sizeable difference between groups. The control group had an estimated of 1.8% (4 patients) below the saturation threshold, and the treated group 11.9% (29 patients). From the estimation, there were six times more patients with a low saturation level in the treated group than the control. The imbalance between groups confirms the initial red flag.
To measure the probability of obtaining two groups with such differences, we simulated both groups 10,000 times. Each simulation had 501 oxygen saturation estimations. We randomly selected a group of 250 to replicate the treated group. Within the group, we counted the number of patients with oxygen saturation under 92%. According to the simulation, the probability of obtaining 29 or more patients was 0.19%.
As a conclusion, the randomization process was likely incorrect. It produced imbalanced groups regarding oxygen saturation. The chance to get such number is 1 in over 500, which is equivalent to a p-value of 0.0019.
Variation in oxygen saturation levels
The study presents oxygen saturation for each group on day zero and day three. It is possible to estimate the change in the number of sick patients from the distributions. The estimations show a decrease in sick patients in the treated group from 29 to 10. It also shows an increase from 4 to 13 in the placebo group. These values show an improvement in the group that received ivermectin and a deterioration in the control group.
The assessment
In summary, the conclusions presented in the study by Vallejos et al. most likely do not correspond to the reality of the data. A likely inappropriate or manipulated randomization process produced a significant imbalance between the groups. The simulation indicates that the study started with a treated group having six times more sick patients than the control group. In the first three days, the number of sick patients reduced by 2/3 in the ivermectin group and increased by 3 times in the control group. The lower hospitalization rate in the treated group (5.6% vs 8.4%) despite having six times sicker patients corroborates the effectiveness of the drug.
On December 2021 I requested the study data-set. A month later, the authors shared only information about seven patients. The patients requiring medical ventilation.This data confirms that patients in the ivermectin group were sicker at the beginning of the study. It also shows several missing data points regarding oxygen saturation.
Thank you for this excellent outline on how to evaluate study data. Given the high levels of interest and attention to this issue world-wide It is now time for a paper which presents a generalised model on how to identify flaws in the method, approach, and presentation of results of clinical trial data. I find myself asking "how did you arrive at these conclusions" ? Hopeful for more detail, while understanding that much of the 'misinformation' in narratives today arise out of a lack of appreciation how best to apply statistics techniques within a framework. One which might prove to be both self-validating and capable of revealing errors when they happen.