Malaria doesn’t need much of an introduction. It’s big (91 countries with ongoing transmission, millions of cases per year), and bad (hundreds of thousands of deaths each year, two thirds which are of children under 5 years)(World Health Organization, 2017b).
Neither does antimicrobial resistance, first commonly recognised in the 1940s (penicillin-resistant staphylococcus, streptomycin-resistant tuberculosis) and now widespread to treatments for tuberculosis, HIV/AIDS, malaria, sexually transmitted diseases, urinary tract infections, pneumonia, blood-stream infections and food poisoning (World Health Organization, 2015).
The good news is that malaria is still preventable and treatable. Some parasites are resistant to one or more antimalarial drugs (but not all) and resistance is developing and spreading. Surveillance of resistance is essential to inform malaria prevention and treatment policies; most surveillance is carried out by monitoring genetic markers (specific mutations in one of a few genes) of resistance (World Health Organization, 2017a).
There is no scientific consensus on the best methods to detect these genetic markers; different labs and different individuals within the same labs use different methods to detect these markers. Using substandard methods to detect these markers of resistance will be generating less data, and less accurate data, and misinform antimalarial prevention and treatment policies – leading to more cases and deaths from malaria.
I have worked in malaria in different labs for a number of years, and have been frustrated to find out that we do not know how best to monitor resistance or have an estimate of how inaccurate our existing data might be. I wanted to answer these questions!
So I undertook this systematic review (the first of its kind) to collate and compare estimates of test accuracy for detection of antimalarial resistance markers in malaria (Burrow, 2017).
I structured my research question using the PIRT framework:
blood samples from patients with, or suspected of having malaria (Patient group)
how accurate is
any molecular method (Index text)
any other molecular method (Reference test)
antimalarial drug resistance markers (Target condition)
I included all diagnostic accuracy studies that examined at least two molecular methods for detecting selected markers of antimalarial resistance in blood samples from patients diagnosed with, or suspected of having malaria. I searched MEDLINE, EMBASE, BIOSIS, and Science Citation Index. I used QUADAS-2 to evaluate methodological quality. I calculated sensitivity and specificity for each and synthesized and compared the results through narrative.
I had a horrifying 27,575 search results, of which 36 studies were potentially eligible. Of these, 15 were eligible and extra information successfully sought for another three. That’s a total of 18 studies, examining 13 index tests, against five reference tests, for their accuracy in detecting 24 molecular markers. The PRISMA flow diagram and overall risk of bias are in Figures 1 and 2.
Figure 1. PRISMA Flow Diagram
Figure 2. Overall Risk of Bias and Applicability Assessment
I have a lot of results tables, so you’ll have to read my dissertation if you want to know the sensitivities and specificities I found. Perhaps more interesting is what I didn’t find…
Half of the 36 studies were not eligible for inclusion as data allowing calculations of sensitivity and specificity were not made available. What difference would these results have made? This could be evidence of publication bias (abstracts were published but not full length papers) and/or of poor reporting practises. No studies were registered, prospectively or retrospectively, so the number of studies reported could not be compared to the number of studies planned. Of the 18 eligible studies, none were reported according to STARD guidelines; most papers were published before the development of the guidelines. However, they still failed to report many details that are difficult to define as anything other than essential (for example, results).
Most studies were underpowered for detection of at least one molecular marker.
No exact replication of studies was done, and very few studies replicated either the combination of reference and index tests or any one test for a specific molecular marker.
The design of studies, often the selection of patients and samples in particular, made it seem as though diagnostic accuracy studies had been added on to other types of studies as an afterthought. Studies were almost all either designed in a way that made them susceptible to large biases, or the design was not reported and biases and applicability were unclear.
Estimates of sensitivity and specificity were calculated, but confidence intervals were wide (partly owing to the above issues) and the direction, size, and effect of biases is mostly unclear.
In addition, most studies were carried out in non-endemic countries using equipment that is not available in most surveillance situations.
I don’t think my findings should change clinical practise, but they should change research practise. I think researchers need to be deliberately asking questions about test accuracy, focussing on methods that are being used and will be used in endemic countries, planning high-quality studies to answer those specific questions, using sample size calculations, prospectively registering their studies, reporting studies according to STARD, responding to requests for further information, replicating these studies, and all of this led by, and in partnership with, researchers in malaria-endemic countries.
Systematic Review Registration: PROSPERO CRD42017056249
BURROW, R. 2017. Sensitivity and specificity of molecular methods for detecting markers of antimalarial drug resistance in clinical samples of Plasmodium falciparum: a systematic review. MSc, University of Oxford.
WORLD HEALTH ORGANIZATION 2015. Global Action Plan on Antimicrobial Resistance. In: WORLD HEALTH ORGANIZATION (ed.). http://www.who.int/antimicrobial-resistance/en/: World Health Organization.
WORLD HEALTH ORGANIZATION 2017a. Status report on artemisinin and ACT resistance (April 2017). In: WORLD HEALTH ORGANIZATION (ed.) Status report on artemisinin and ACT resistance. http://www.who.int/malaria/publications/atoz/artemisinin-resistance-april2017/en/: World Health Organization.
WORLD HEALTH ORGANIZATION 2017b. World Malaria Report 2017. In: WORLD HEALTH ORGANIZATION (ed.) World Malaria Report. http://www.who.int/malaria/publications/world-malaria-report-2017/report/en/: World Health Organization.