Blog dr.Eki

Let us consider the use of a rapid antigen detection test for group A streptococcal infection in throat swabs. The first question to ask is whether there was a blinded comparison against an accepted reference standard. By blinded, we mean that the measurements with the new test were done without knowledge of the results of the reference standard. Next, we would assess the results. Traditionally, we are interested in the sensitivity (proportion of reference-standard positives correctly identified as positive by the new test) and specificity (the proportion of reference-standard negatives correctly identified as negative by the new test). Ideally, we would also like to have a measure of the precision of this estimate, such as a 95% confidence interval on the sensitivity and specificity, although such measures are rarely reported in the infectious diseases literature. Note, however, that while the sensitivity and specificity may help a laboratory to choose the best test to offer for routine testing, they do not necessarily help the clinician. Thus, faced with a positive test with known 95% sensitivity and specificity, we cannot infer that our patient with a positive test for group A streptococcal infection has a 95% likelihood of being infected. For this, we need a positive predictive value, which is calculated as the percentage of true positives among all those who test positive. If the positive predictive value is 90%, then a positive test would suggest a 90% likelihood that the person is truly infected. Similarly, the negative predictive value is the percentage of true negatives among all those who test negative. Both positive and negative predictive value change with the underlying prevalence of the disease, hence such numbers cannot be generalized to other settings. A more sophisticated way to summarize diagnostic accuracy, which combines the advantages of positive and negative predictive values while solving the problem of varying prevalence, is to quantify the results using likelihood ratios. Like sensitivity and specificity, likelihood ratios are a constant characteristic of a diagnostic test, and independent of prevalence. However, to estimate the probability of a disease using likelihood ratios, we additionally need to estimate the probability of the target condition (based on prevalence or clinical signs). Diagnostic tests then help us to shift our suspicion (pretest probability) about a condition depending on the result. Likelihood ratios tell us how much we should increase the probability of a condition for a positive test (positive likelihood ratio) or reduce the probability for a negative test (negative likelihood ratio). A positive likelihood ratio is also defined as follows: sensitivity/(1-specificity). Let us assume, hypothetically, that the sensitivity of the rapid antigen test is 80% and the specificity 90%. The positive likelihood ratio for the antigen test is (0•8/0•1) or 8. This would mean that a patient with a positive antigen test would have 8 times the odds of being positive compared with a patient without group A streptococcal infection. The tricky part in using likelihood ratios is to convert the pretest probability (say 20% based on our expected prevalence among patients with pharyngitis in our clinic) to odds: these represent 1:4 odds. After multiplying by 8, we have odds of 8:4, or a 67% post-test probability of disease. Thus, our patient probably has group A streptococcus, and it would be reasonable to treat with antibiotics. The negative likelihood ratio, defined as (1−sensitivity)/specificity, tells us how much we should reduce the probability for disease given a negative test. In this case, the negative likelihood ratio is 0•22, which can be interpreted as follows: a patient with pharyngitis and a negative antigen test would have their odds of disease multiplied by 0•22. In this case, a pretest probability of 20% (odds 1:4) would fall to an odds of 0•22 to 4, or about 5%, following a negative test. Nomograms have been published to aid in the calculation of post-test probabilities for various likelihood ratios.8 Having found that the results of the diagnostic test appear favorable for both diagnosing or ruling out disease, we ask whether the results of a study can be generalized to the type of patients we would be seeing. We might also call this “external validity” of the study. Here we are asking the question: “Am I likely to get the same good results as in this study in my own patients.” This includes such factors as the severity and spectrum of patients studied versus those we will encounter in our own practice, and technical issues in how the test is performed outside of the research setting.
To summarize, to assess a study of a new diagnostic test, we identify a study in which the new test is compared with an independent reference standard; we examine its sensitivity, specificity, and positive and negative likelihood ratios; and we determine whether the spectrum of patients and technical details of the test can be generalized to our own setting. In applying these guidelines in infectious diseases, there are some important caveats.There may be no appropriate reference standard. The spectrum of illness may dramatically change the test characteristics, as may other co-interventions such as antibiotics. For example, let us assume that we are interested in estimating the diagnostic accuracy of a new commercially available polymerase chain reaction (PCR) test for the rapid detection of Neisseria meningitidis in spinal fluid. The reference standard of culture may not be completely sensitive. Therefore, use of an expanded reference (“gold”) standard might be used. For example, the reference standard may be growth of N. meningitidis from the spinal fluid, demonstration of an elevated white blood cell count in the spinal fluid along with Gram negative bacilli with typical morphology on Gram stain, or elevated white blood cell count along with isolation of N. meningitidis in the blood. It is also important to know in what type of patients the test was evaluated, such as the inclusion and exclusion criteria as well as the spectrum of illness. Given that growth of micro-organisms is usually progressive, test characteristics in infectious diseases can change depending when the tests are conducted. For example, PCR conducted in patients who are early in their course of meningitis may not be sensitive as compared to patients that presented with late stage disease. This addresses the issue of spectrum in test evaluation.

Reference: Loeb Mark, Smieja Marek, Smaill Fiona . Introduction to evidence-based infectious diseases. In EVIDENCE-BASED INFECTIOUS DISEASES. © BMJ Publishing Group Ltd 2004,p.3-4