OUP blog post, by Leonard A. Jason, August 20th 2017: Scientific progress stumbles without a valid case definition
Current estimates from the Centers for Disease Control and Prevention (CDC) of the number of people in the United States with chronic fatigue syndrome (CFS) increased from about 20,000 to as many as four million within a ten-year period. If this were true, we would be amidst an epidemic of unprecedented proportions. I believe that these increases in prevalence rates can be explained by unreliable case definitions. For example, in 1994, the CDC’s case definition did not require patients to have core symptoms of the CFS.
Making matters worse, in 2005, in an effort to operationalize their inadequate case definition, the CDC broadened the case definition so that ten times as many patients would be identified. Even though these estimates were challenged as bringing into the CFS case definition many who did not have this illness such as Major Depressive Disorder, as late as 2016, the CDC re-affirmed the merit in this broader case definition.
Another misguided effort occurred in 2015, when the Institute of Medicine (IOM) developed a revised clinical case definition that at least did specify core symptoms, but unfortunately also eliminated almost all exclusionary conditions, so those who had had previously been diagnosed with other illnesses such as Melancholic Depressive Disorders, could now be classified as meeting the new IOM criteria.
This case definition has the unfortunate consequence of again broadening the types of patients that will now be identified, thus their effort also will inappropriately select many patients with other diseases as meeting the new IOM criteria. Making matters even worse, the clinical case definition was not designed to be used for research purposes, but it is clearly being used in this way, and one group of researchers has already inaccurately reported that the new clinical case definition is as effective at selecting patients as research case definitions.
This comedy of errors becomes even more tragic with the recent development of a new pediatric case definition. As with past efforts, data were not collected to field test this new set of criteria. Even worse, medical personnel are asked to make decisions regarding symptoms without being providing any validated questionnaires, and this has the effect of introducing unacceptable levels of diagnostic unreliability. Scoring rules are so poorly developed that guidelines indicate that a child needs to have most symptoms at a moderate or severe level, but in reality, according to the flawed scoring rules of this case definition, youth can be classified as having the illness even if they report all symptoms as mild.
These criteria further suggest that “personality disorders” should be assessed in children, as these disorders are listed as psychiatric exclusionary conditions; however, personality disorders cannot be diagnosed (or reliably assessed) prior to the age of 18, as personality characteristics are not fully developed until adulthood. Finally, these authors also require the youth to have at least six months of illness duration, whereas the Canadian criteria and others suggest that children with three months duration can be diagnosed with the illness. Other significant limitation of this primer have been mentioned by others.
In summary, these authors failed to incorporate standard psychometric procedures that include first specifying symptoms and logical scoring rules, developing consistent ways to reliably assess these symptoms, and then collecting data to ensure that the proposed criteria are reliable and valid.
When a field of inquiry is either unable or unwilling to develop a valid case definition, as has occurred with CFS, the repercussions are catastrophic for the research and patient community. In a sense like a house of cards, if the bottom level is not established with a sturdy foundation, all upper levels of cards are vulnerable to collapse.
Science is based on having sound case definitions that allow investigators to determine who has and does not have an illness.
Having porous and invalid case definitions, whether clinical or research, affects not only prevalence estimates of CFS, but also has dire consequences for treatment approaches, as when individuals who have solely affective disorders are misclassified as having CFS, and when they improve from psychological interventions, it is easy to erroneously conclude that CFS is a psychiatric illness, which further stigmatizes patients.
Leonard A. Jason is a professor of clinical and community psychology at DePaul University, director of the Center for Community Research, and the author of Principles of Social Change and co-editor of the Handbook of Methodological Approaches to Community-Based Research: Qualitative, Quantitative, and Mixed Methods.
