DSM5: The case for double standards

In my last post I talked about the inappropriate application of double standards in the DSM, as reflected in the preferential treatment given to behavioral over biological markers of mental disorders: as “porous” as behavioral markers are, the editors of DSM5 enshrine them as the “best” way of identifying mental disorders. On the other hand, here is no place in DSM5 for biological data, unless these data are “incontrovertible.” This double standard reflects the historical background of the DSM, and the difficulty in making institutional change.
There is, however, one arena in which the editors of the DSM ought to have advocated for a double standard, but failed to do so: Actually using the DSM to make a diagnosis.

Before discussing DSM5, we need to have a little discussion about medical screening. Don’t panic; I’ll leave out the math. (Readers who want a more detailed discussion can go here: http://en.wikipedia.org/wiki/Sensitivity_and_specificity )
Let’s discuss a hypothetical disease, which is considered to exist if the individual’s clinical or laboratory findings are outside of a defined “normal range.” Let’s also say we already have decided what test we’re going to use; now our job is to decide where to set the cutoff for “normal” vs. “abnormal.” If we’re lucky, the range of normal values and the range of abnormal values do not overlap, and we can set our cutoff value in between them, as in Case A. Persons with values to the left of the line will screen “negative,” and we can be 100% reassured that they do not have the disease. Persons whose values lie to the right of the cutoff are deemed “positive,” and we can be 100% certain that they have the disease. There are, in fact, some conditions that work this way: Inborn errors of metabolism (PKU, urea cycle disorders, etc.) are a good example. Babies with inborn metabolic errors have lab values that are hundreds or thousands of times elevated compared to the normal range.

Case A: Normal and abnormal values do not overlap, and we can set a cutoff value in between them. 100% of the subjects who screen “negative” (i.e. pass the screening test) are free of disease, and 100% of subjects who screen positive (fail the screening test) actually have the disease.

Most of the time, however, the “healthy” and “disease” groups overlap, as in Case B. There is no place we can set the cutoff value to cleanly separate them, no matter how good our test. We will inevitably generate some False Negatives (persons who pass the test, but actually have disease), and False Positives (persons who fail the test, but are actually healthy).

Case B. Clinical or laboratory values for the two populations overlap. Persons lying to the left of the cutoff fall into two sub-categories: True Negatives, and False Negatives (FN in the figure). Persons lying to the right of the line likewise fall into two sub-categories: True Positives, and False Positives (FP in the figure).

The question of where to set the cutoff comes up all the time in medicine — and the answer is “It Depends.” If you are conducting research on a particular disorder, you want to be sure that your research subjects all have the condition you’re interested in. You don’t want any False Positives watering down your study sample, so you will set your cutoff point as high as possible. (Case C). You will wind up generating a lot of false negatives (people who test negative  but really have the disorder), but that’s not a problem in this situation, because your mission is research, not clinical care.

Case C. Researchers interested in studying a disorder may set the cutoff very high, to assure that their study sample is composed completely of subjects who have the disease. The tradeoff for reducing False Positives to zero is the generation of a large number of False Negatives.

What if you are not a researcher, but a practicing clinician, or public health official (or, for that matter, a school psychologist)? Your goal is to find as many affected individuals as possible. In this scenario, you will set the cutoff value very low, in order to detect everyone who might be affected. You will inevitably generate some False Positives, but since you will be following up on all of your positive results, you can sort out the false positives from the true positives on the basis of further testing. (Readers who want a more detailed of the principles behind population screening can go here: http://en.wikipedia.org/wiki/Sensitivity_and_specificity )

Case D. In clinical practice, and in Public Health, the goal is to find all of the persons who may have the condition of interest. To achieve this goal, the cutoff point is set fairly low, in order to reduce False Negatives to zero. The inevitable tradeoff is the generation of a fair number of False Positives. Follow-up testing needs to be put into place to sort out the True Positives from the False Positives.

Now that we’ve discussed the statistical facts of life, let’s come back to the DSM….
The authors of DSM5 correctly concede that “in the absence of clear biological markers or clinically useful measurements of severity for many mental disorders, it has not been possible to completely separate normal and pathological symptom expressions.” In plain English, there is no “bright line” separating “normal” from “abnormal.” In other words, for most mental conditions, the situation is like to Case B: Normal and Abnormal overlap, and we will wind up with some False Negatives and False Positives, even with the most perfect test in the world. But then the DSM5 blithely goes on to say:
“The criteria [in DSM5] are concise and explicit and intended to facilitate an objective assessment of symptom presentations in a variety of clinical settings—inpatient, outpatient, partial hospital, consultation-liaison, clinical, private practice, and primary care—as well in general community epidemiological studies of mental disorders. DSM-5 is also a tool for collecting and communicating accurate public health statistics on mental disorder morbidity and mortality rates…Fortunately, all of these uses are mutually compatible.”
Alas, this is only half-true: These uses might have been “mutually compatible,” but only if the cutoff values had been adjusted to reflect the intentions of the user. It is simply wrong to use research criteria in a clinical care setting. This is no idle concern. I have many patients who are turned away from our local autism research center because they do not meet research criteria for ASD — even though they are clearly on the spectrum. This is a minor annoyance to me, and a loss of an opportunity to participate in research for the family, but other than that, no harm done. What happens when a set of “one size fits all” criteria are applied to the question of eligibility for public services?
DSM5 goes on to say: “Consolidation of autistic disorder, Asperger’s disorder, and pervasive developmental disorder into autism spectrum disorder… is designed to improve the sensitivity and specificity of the criteria for the diagnosis of autism spectrum disorder and to identify more focused treatment targets for the specific impairments identified.” Well, not really. You can improve both sensitivity and specificity up to a point, but after that, you have to make a choice between the two; you are always raising one at the expense of the other. As my stepmother used to say, “You can’t have your cake and eat it too.”
The World Health Organization “gets it,” and for this reason publishes two sets of criteria: One set for clinical care, and the other set for research: The ICD-10 Classification of Mental and Behavioural Disorders: Diagnostic Criteria for Research. The latter “sets out internationally agreed diagnostic criteria specifically designed for use when conducting research on mental and behavioral disorders. Deliberately restrictive, the criteria are intended to facilitate the selection of groups of individuals whose symptoms and other characteristics resemble each other in clearly stated ways, and thus to maximize the homogeneity of study groups and the comparability of findings in multi-centre and international studies.”
http://www.amazon.com/ICD-10-Classification-Mental-Behavioural-Disorders/dp/9241544554
The authors of DSM5 gloss over this issue entirely – to the detriment of clinicians, researchers, and affected individuals alike.

James Coplan, MD is an Internationally recognized clinician, author, and public speaker in the fields of early child development, early language development and autistic spectrum disorders. Stay connected, join Dr. Coplan on Facebook and Twitter.