THE RELATIONSHIP BETWEEN SENSITIVITY AND SPECIFICITY.

05 Jun THE RELATIONSHIP BETWEEN SENSITIVITY AND SPECIFICITY.

The relationship between sensitivity and specificity.

Week 8 e-Activity Go to your state’s DOH Website, and review the statistics and research data information on screening for cancer and other prevalent diseases.

“Disease Screening Methodology” Please respond to the following:

From the case study and e-Activity, analyze and evaluate the various measures of screening tests validation, and comment on the efficacy of screening programs in your community. From the case study and applying the concepts of sensitivity and specificity to prostate cancer screening, evaluate the cost effectiveness of having screenings for men over 50 years of age.

APA format no plagerism include sources 1 page Due tomorrow 12pm eastern time Course HSA 535 Managerial Epidimiology

Case Study

 

Table 11-1 Effects of Disease Prevalence on the Predictive Value of a Screening Test

Cell Values
Row	a	b	c	d	Total (a + b + c + d)	Prevalence	No. of Cases of Disease (a + c)	Predictive Value (+)	Predictive Value (-)
A	240	25	15	220	500	51%	255	90.6%	93.6%
B	47	45	3	405	500	10%	50	51.1%	99.3%

When the prevalence of a disease decreases the predictive value (+) falls and the predictive value (–) rises. The clinical implications of low predictive value (+) are that any individual who has a positive screening test would have low probability of having the disease; an invasive diagnostic procedure would probably not be warranted for this patient. (Refer to Exhibit 11–4 for more information in the context of the positive value [+] of the prostate cancer screening test.) Table 11-1demonstrates the effects of changing the prevalence of disease upon predictive values. When the prevalence of disease drops from about 51% to 10%, the predictive value (+) drops from about 91% to 51%, and the predictive value (–) increases from about 94% to 99%.

EXHIBIT 11–4 The Importance of Positive Predictive Value for Prostate Cancer Screening

An excellent illustration of the importance of positive predictive value [predictive value (+)] is the prostate cancer screening controversy. Prostate cancer is the most common cancer among men inthe United States; according to the National Cancer Institute over 240,000 men will be diagnosed with the disease in 2012 and 28,000 will die from it. The disease can be detected early through a simple blood test to measure levels of prostate-specific antigen (PSA). PSA is a molecule found in cells that make up the prostate gland and is released when tumors disrupt the prostate cells. The PSA test was originally developed to tell whether prostate cancer was coming back in men already treated for prostate cancer. However, doctors began giving the test to healthy men with no symptoms of prostate cancer. Routine PSA screening became widespread in the United States by 1991, a year before the start of the first large clinical trial designed to determine if PSA screening actually saved lives. Although the sensitivity of the PSA test is good, it is not perfect as not all men with prostate cancer have elevated PSA levels. The specificity is more problematic because PSA blood levels can be elevated for other reasons besides cancer. For example, as men age, their prostate glands tend to enlarge and even benign conditions of the prostate can cause elevated PSA levels, as can an infection of the prostate, a condition called prostatitis. This means that a lot of men will have definitive diagnostic tests that turn out to reveal that cancer doesn’t exist. Indeed, although PSA screening can detect prostate cancer in its early, curable stages, the positive predictive value is low and 1,000 men must be screened to save 1 man’s life from prostate cancer. Men with suspicious PSA levels may go on to have a prostate biopsy. This is done with a needle; usually about a dozen small “cores” are taken. It’s unpleasant, but usually uneventful. Even so, about 70 out of 10,000 biopsies result in infection, bleeding, or urinary difficulties. Men found to have prostate cancer—about 25% to 35% of men biopsied—have a number of options. One is to closely watch the cancer to see if it gets worse. In this case, the harm is anxiety and possibly waiting too long to get treatment. In the United States, most men opt for one of several available treatments for prostate cancer. These treatments are very effective at curing the cancer but they have a high rate of side effects including impotence, incontinence, heart attacks, and occasionally even death from treatment of tiny tumors that never would have killed them. After comparing those harms to the benefit of saving one life, the U.S. Preventive Services Task Force calculated that the harms of PSA screening outweigh the benefits.

Relationship Between Sensitivity and Specificity

Figure 11–5 illustrates the relationship between sensitivity and specificity. When the screening test result is a continuous or ordered variable with several levels, then the choice for a cut point that discriminates optimally between suspected diseased and normal individuals is a trade-off. Figure 11–5 demonstrates the effects of choosing various cut points. The figure shows a hypothetical distribution of trait values (e.g., fasting blood glucose, an indicator of diabetes) for normal individuals and a distribution curve for the diseased population that overlaps the curve for the normal population. For example, fasting blood glucose levels may approximate the normal distribution with a mean of 100 mg/dL. A subject may have an elevated glucose level in the high range for a population (e.g., 120 mg/dL) and not be diabetic. Some diabetic individuals who are at the lower end of the curve for the diseased group also may have glucose levels in the high normal range. Thus, the two distributions may overlap: Some nondiseased individuals may have elevated glucose levels, and some diseased individuals may have glucose levels in the lower ranges for the abnormal group. The cut point may be set at B to maximize both sensitivity and specificity. If the cut point is moved to A by lowering the specific blood glucose level that is to be classified as abnormal, almost all of the individuals who have the disease will be screened as positive, and sensitivity will approach 100%. Specificity will be lowered because more of the nondiseased individuals will be classified as diseased. By moving the cut point to C, which represents a higher blood glucose level than point A or B, specificity will be increased at the expense of sensitivity.

FIGURE 11–5 Interrelationship between sensitivity and specificity.

Another example of establishing a cut point to distinguish between diseased and nondiseased people is setting the referral criteria for screening for glaucoma.²⁰ By using the criterion of 15 mm Hg intraocular pressure, the sensitivity of the screening test would be high, and few persons with glaucoma would be missed. At the same time, many persons who did not have the disease would be incorrectly classified. If a high referral point were selected (e.g., 33 mm Hg), the majority of those without glaucoma would not be referred, but many with the disease would be missed by the screening test. Thus, this example demonstrates that sensitivity and specificity are complementary. “The key to a successful screening is to balance the referral criteria so that both the overreferrals and underreferrals are minimized.”^{20(p 360)}

In summary, if one wishes to improve sensitivity, the cut point used to classify individuals as diseased should be moved farther in the range of the nondiseased. To improve specificity, the cut point should be moved farther in the range typically associated with the disease. There are a number of additional procedures that can improve both sensitivity and specificity:

• •Retrain screeners: If the test requires human assessment (e.g., blood pressure readings), then improving the precision of measurement through additional training sessions will reduce the amount of misclassification.
• •Recalibrate screening instrument: For those tests that utilize technology (e.g., a weighing device or a densitometer), it may be possible to reduce the amount of imprecision through refinement of the methodology.
• •Utilize a different test: In some situations there may be more than one way to measure the outcome of interest. Suppose there are two laboratory assays available to quantify serum cholesterol. If one assay performs poorly (low reliability and validity), it may be possible to replace it with a better assay.
• •Utilize more than one test: Because of the variability in some measures, it is easy to misclassify an individual as high or low. By taking more than one measure of blood pressure and averaging the results, the ability to label an individual correctly as hypertensive will be improved, resulting in better measures of sensitivity and specificity.

Evaluation of Screening Programs

Despite the intuitive appeal of screening programs, their utility should never be assumed. Rather, it is imperative that they be evaluated with the same rigor used to identify risk factors in the pathogenesis of disease. The ideal design is a randomized controlled trial. Under this approach, subjects are randomized either to receive the new screening test or program or to receive usual care. If the disease of interest is fatal, then the appropriate end point would be differences in mortality between the two groups. For nonfatal diseases (e.g., cataracts), differences in incidence between the screened and nonscreened populations should be estimated. Another approach, although less rigorous, is to conduct ecologic time trend studies in geographic areas with and without screening programs. Finally, the case-control method can be applied also: Cases are fatal (or likely to be fatal) cases of the disease, controls are nonfatal cases of the disease, and the exposure is participation in a screening program. Regardless of the approach that is taken, evaluation of screening programs is subject to several types of bias that have not yet been described fully in this chapter. Figure 11–6 depicts the natural history of disease in relation to the time of diagnosis.

Suppose the disease begins at time A and results in death at time D. A case detected as the result of screening may be picked up at time B, whereas a case that is picked up as a result of clinical signs and symptoms may not be detected until time C.

FIGURE 11–6 Natural history of disease in relation to time of diagnosis. C to D, survival time for unscreened case; B to D, survival time for screened case; B to C, lead time.

• •Lead time bias: the perception that the screen-detected case has a longer survival simply because the disease was identified earlier in the natural history of the disease. Thus, although these two individuals had identical dates of onset and death, there is an apparent increase in survival for the screen-detected case. The extent of this bias is estimated as the difference between time periods B and C.
• •Length bias: used particularly with respect to certain cancer screening programs. In illustration, tumors that are detected by a screening program tend to be slower growing and hence have an inherently better prognosis than tumors that are more rapidly growing and are detected as a result of clinical manifestation.
• •Selection bias: Although this topic was covered as an aspect of study designs, selection bias also is relevant to the evaluation of screening programs. In particular, individuals who are motivated enough to participate in screening programs may have a different probability of disease (as a result of other healthy behaviors) than individuals who refuse participation.

In conclusion, the foregoing section has discussed a number of factors that need to be taken into account in the evaluation of screening tests. Indeed, many of the issues remain controversial, as noted in Exhibit 11–1 for mammography screening and Exhibit 11–4 for the prostate-specific antigen (PSA) test.

Issues in the Classification of Morbidity and Mortality

The theme of this chapter has been screening for disease in the community and the related topics of reliability and validity of measurement. Schemes for the nomenclature and classification of disease are central to the reliable measurement of the outcome variable in epidemiologic research. The terms nomenclature and classification are defined as follows: A nomenclature is a highly specific set of terms for describing and recording clinical or pathologic diagnoses to classify ill persons into groups. A system of nomenclature must be extensive so that all conditions encountered by the practitioner in a particular health discipline can be recorded. Classification, in contrast, lends itself to the statistical compilation of groups of cases of disease by arranging disease entities into categories that share similar features.²¹

The classification systems used are in some cases purely arbitrary because they are determined by the function that is to be served by classification; nevertheless, all practitioners in a discipline need to have at their disposal a standardized system for classification of diseases. Many classification systems for diseases are theoretically possible; they might be based upon age, circumstance of onset, geographic location, or some other factor connected with the purpose for which they are to be used. The categories of disease should be general so that there will be a limited number of categories that take into account all the diseases that might be encountered. The use of general categories facilitates the epidemio-logic study of disease phenomena by giving rise to groups of interrelated morbid conditions. The classifications of disease must be distinct so that a disease falls into only one category of the classification system. Each category of the classification system must refer to diseases that are sufficiently frequent to permit several cases of disease to fall into the category. Otherwise, there would be an excessively detailed list of categories to contain the range of morbid conditions. Finally, a well-devised classification system permits standardization across different agencies and even countries so that comparisons in morbidity and mortality from disease can be made.

Two types of criteria are used for the classification of ill persons: causal and manifestational.²¹ It is possible to classify cases of disease according to a causal basis (e.g., tuberculosis or syphilis) or according to manifestation (e.g., affected anatomic site: hepatitis or breast cancer). Epidemiologic research relies primarily on manifestational criteria for classification in the hope that there will be a strong enough connection with causal factors to make possible etiologic studies.²¹

One example of a classification system is the Diagnostic and Statistical Manual of Mental Disorders-IV-TR, now in its fourth edition (text revision); it provides for standardization of the classification of psychiatric diagnoses.²² Clinicians, researchers, insurance companies, and other personnel who work in the mental health field make use of the DSM-IV-TR for classifying mental disorders.

A second example, the International Statistical Classification of Diseases and Related Health Problems, is one of the most widely used systems for the classification of diseases and is now in its 10th revision (ICD-10).²³ The ICD is sponsored by the World Health Organization (Collaborative Centers for Classification of Diseases). It is designed for varied uses: for both clinical and general epidemio-logic purposes and for the evaluation of health care. The ICD-10 spans three volumes; volume one provides classification of diseases into three-and four-character levels (an alphanumeric coding scheme replaces the previous numeric one).

Conclusion

This chapter discussed terminology related to the quality of measures employed in epidemiology. Measurement is a crucial issue because even the most carefully designed study may yield spurious results if premised upon faulty measures. Topics covered in this chapter included reliability and validity, screening for disease, and methods for the classification of diseases. Formulas and examples for calculation of sensitivity, specificity, and predictive value were provided. The effect of prevalence of disease upon predictive value was discussed also.

Study Questions and Exercises

1.

Are you able to define the following?

• reliability
• validity
• precision
• accuracy
• sensitivity
• specificity
• predictive value (+) and predictive value (–)

2.

What factors should govern the selection and use of a screening instrument by a health clinic?

3.

What is the relationship between reliability and validity? Is it possible for a measure to be reliable and invalid? Conversely, is it possible for a measure to be unreliable and valid?

4.

Assume that the fasting blood level of a lipid is normally distributed in the population of people who do not have disease “X.” There is a smaller distribution curve of the fasting blood levels of this lipid, which also is normal in shape, for the population of persons who have disease “X,” and the curve overlaps the upper end (right side) of the curve for people without the disease. Draw distribution curves for the diseased and non-diseased populations and discuss the effects upon sensitivity and specificity of setting the cut point for disease and nondisease at various positions on the two overlapping curves.

5.

How does the predictive value of a screening test vary according to the prevalence of disease?

6.

A serologic test is being devised to detect a hypothetical chronic disease. Three hundred individuals were referred to a laboratory for testing. One hundred diagnosed cases were among the 300. A serologic test yielded 200 positives, of which one-fourth were true positives. Calculate the sensitivity, specificity, and predictive value of this test. (Hint: After setting up the appropriate 2 by 2 table, find missing data by subtraction. The numbers for the cells should then correspond to the numbers shown in Appendix 11.)

7.

A new test was compared with a gold standard measurement with the following results:

	Gold Standard
New Test	+	–
+	18	2
–	8	72

What are the sensitivity and specificity?

8.

Using the data from question 7, what is the predictive value (+) and the predictive value (–)?

9.

A test-retest reliability study of the new test was conducted with the following results:

	Test
Retest	+	–
+	80	9
–	8	3

What is the percentage agreement (accuracy)?

10.

The prevalence of undetected diabetes in a population to be screened is approximately 1.5%, and it is assumed that 10,000 persons will be screened. The screening test will measure blood serum glucose content. A value of 180 mg% or higher is considered positive. The sensitivity and specificity associated with this screening test are 22.9% and 99.8%, respectively.

• What is the predictive value of a positive test?
• What is the predictive value of a negative test?

References

1.

Morrison AS. Screening in Chronic Disease. New York, NY: Oxford University Press; 1985.

2.

Shapiro S, Venet W, Strax P, Venet L. Periodic Screening for Breast Cancer: The Health Insurance Plan Project and Its Sequelae, 1963–1986. Baltimore, MD: Johns Hopkins University Press; 1988.

3.

Hurley SF, Kaldor JM. The benefits and risks of mammographic screening for breast cancer.Epidemiol Rev. 1992;14:101–129.

4.

Fletcher SW, Black W, Harris R, et al. Report of the International Workshop on Screening for Breast Cancer. J Natl Cancer Inst. 1993;85:1644–1656.

5.

Haynes RB. How to read clinical journals, II: to learn about a diagnostic test. Can Med Assoc J. 1981;124:703–710.

6.

Commission on Chronic Illness. Chronic Illness in the United States: Prevention of Chronic Illness. Cambridge, MA: Harvard University Press; 1957:1.

7.

Porta M., ed. A Dictionary of Epidemiology. 5th ed. New York, NY: Oxford University Press; 2008.

8.

Centers for Disease Control and Prevention. Cancer screening–United States, 2010. MMWR. 2012;61:41–45.

9.

Wilson JMG, Jungner F. Principles and Practice of Screening for Disease. Public Health Paper 34. Geneva, Switzerland: World Health Organization; 1968.

10.

Sackett DL, Holland WW. Controversy in the detection of disease. Lancet. 1975;2:357–359.

11.

World Health Organization. Mass Health Examinations. Geneva, Switzerland: World Health Organization; 1971. Public Health Paper 45.

12.

Halperin W, Baker EL Jr. Public Health Surveillance. New York, NY: Van Nostrand Reinhold; 1992.

13.

Beaglehole R, Bonita R, Kjellström T. Basic Epidemiology. Geneva, Switzerland: World Health Organization; 1993.

14.

Cochrane AL, Holland WW. Validation of screening procedures. Br Med Bull. 1971;27:3–8.

15.

Thorndike RL, Hagen E. Measurement and Evaluation in Psychology and Education. 2nd ed. New York, NY: Wiley; 1961.

16.

Abramson JH. Survey Methods in Community Medicine. 4th ed. New York, NY: Churchill Livingstone; 1991.

17.

Weiss NS. Clinical Epidemiology: The Study of the Outcome of Illness. New York, NY: Oxford University Press; 1986.

18.

Babbie E. The Practice of Social Research. 13th ed. Belmont, CA: Wadsworth; 2013.

19.

McCunney RJ. Medical surveillance: principles of establishing an effective program. In: McCunney RJ, ed. Handbook of Occupational Medicine. Boston, MA: Little, Brown; 1988:297–309.

20.

Myrowitz E. A public health perspective on vision screening. Am J Optom Physiol Opt. 1984;61:359–360.

21.

MacMahon B, Pugh TF. Epidemiology Principles and Methods. Boston, MA: Little, Brown; 1970.

22.

American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 4th ed. Text Revision: DSM-IV-TR. Washington, DC: American Psychiatric Association; 2000.

23.

World Health Organization. International Statistical Classification of Diseases and Related Health Problems. 2nd ed. 10th revision. Geneva, Switzerland: World Health Organization; 2004.

APPENDIX 11 Data for Problem 6

Given	Find by Subtraction
Total = 300	Total – (TP + FN) = FP + TN = 300 – 100 = 200
TP + FN = 100	FP = (TP + FP) – TP = 200 – 50 = 150
TP + FP = 200	FN = (TP + FN) – TP = 100 – 50 = 50
TP = 50	TN = (FP + TN) – FP = 200 – 150 = 50

TP, true positive; FN, false negative; FP, false positive; TN, true negative.