It is easy to be appalled and a bit morbidly amused by suggestions to sunbathe, inject disinfectant and swallow Clorox.
This magical thinking and irresponsible advice reflects a banality and ignorance of science that is easy to recognise. However, we appreciate less commonly the questionable validity of much of the information widely disseminated on Covid-19.
First think about comparisons of death rates from Covid-19 between countries and over time. This seems like a reliable number. However, there has been an overall increase in death rates coincident with, but unattributed to the virus; these patients were not tested but we speculate many have suffered from Covid-19. Besides this numerator problem, we have a denominator problem.
Depending on how widespread is testing and the true prevalence of the disease, the estimate may be undermined by a failure to test widely.
The same constraints apply to estimates of incidence of infection. Here, it depends on how widely applied is testing and how reliable are the tests. If testing is confined only to people with suspect symptoms, we know it is an underestimate of the true incidence; most positive cases are asymptomatic.
Recurrence of infection
Even if testing is widespread, it depends on its validity. Consider the apparent recurrence of infection. As people recover from infections, viral load declines towards the limit of sensitivity of the assay. Depending either on the constancy of the limit of detection or whether the load is varying, patients may be found positive, then negative, then positive again during recovery, confounding the ability to discriminate between the original infection and a new one.
Similar caution should be applied to reports of neutralising antibodies and discussion of the “post-infectious elite”. For now, we have no reliable estimates of the incidence of antibody formation, whether the antibodies formed are protective against subsequent infection and if so, how high is a protective titer or for how long that protection lasts. In this case, we must characterise the antibodies that are neutralising, then estimate their prevalence and persistence.
The challenge will then be to devise experiments to test their effectiveness, perhaps first in monkeys. We then need to consider the ethics of virus re-challenge of asymptomatic patients with neutralising antibodies to establish their immunity. Commercial tests for these antibodies until recently have false positive rates as high as 15 per cent. Such a complex approach will be necessary to use plasma from recovered patients rationally as a therapeutic or the presence of antibodies to guide return to the workplace and social behaviour.
Finally, our understanding of drugs is undermined by our ability to discriminate anecdote from evidence. Anecdotes fuelled concern about anti-hypertensive and nonsteroidal anti-inflammatory drugs.
Plausibility and evidence
Theories emerged about their risk or benefit. Plausibility is one thing; evidence is another and there was none. The World Health Organisation and the European Medicines Agency eventually withdrew their warnings about these drugs.
Recently, the market was moved by a series of uninterpretable news releases about the Gilead drug, Remdesivir. First, the New England Journal of Medicine chose to publish an uncontrolled study of patients treated with the drug. Some got better, some did not. This provided no interpretable information on whether Remdesivir made them better, worse, or neither.
Next, we had a “leak”; a physician giving her impression of what was happening in one centre of a multicentre trial: completely uninterpretable information with respect to the drug’s efficacy or safety. The markets moved again. Another “leak” on the WHO website of a controlled trial stopped early with unbalanced numbers between the groups was similarly uninterpretable. The markets reversed their earlier moves.
At last some clarity. The results of a small double-blind placebo-controlled trial: neither the doctor nor the patient knows whether they have received the active drug. This trial design minimises bias and delivered believable results.
Rigorous science
Remdesivir reduced the time in hospital by about a third. The US Food and Drug Administration gave a stamp of approval and this becomes standard of care. Yet we are still waiting for the peer-reviewed paper that provides all the results. This experience highlights the need for rigorous science in clinical trial design and allowing the time for believable results to emerge.
The scientific community has been mobilised internationally to address the crisis that is Covid-19. However, given the pressing circumstances, many papers have been rushed into print thus creating uninformative and distracting “noise”. Speed is no substitute for science.
At present, we are encouraged by the diversified approaches being taken to the development of vaccines, but the air is thick with promissory notes and dates of delivery. This is absurd. Who can know when a vaccine will have been shown safely to confer protection?
Despite the headlines we are in the early stages of assessing the tolerability, not the effectiveness of vaccines. That comes next and must be subject to rigorous scientific evaluation. Once those criteria are satisfied, we can turn to the challenges of scale up and democratisation of access to an intervention that may bring this crisis under control.
Prof Garret A FitzGerald directs the Institute for Translational Medicine and Therapeutics at the Perelman school of Medicine at the University of Pennsylvania