Virology blog post, by David Tuller, 21 November 2016: Trial By Error, Continued: The New FITNET Trial for Kids
The article challenges:
- the failure of PACE researchers to acknowledge failings of PACE
- the unproven use of CBT to reverse false illness beliefs and deconditioning
- Crawley’s FITNET study
- Crawley’s earlier study that concluded CFS affects 1.9% of 16 year olds
- Dutch FITNET study
- CFS research & discussion in Netherlands
Tuller on FITNET:
Like the work of the PACE authors, Dr. Crawley’s research is fraught with misrepresentations and methodological problems. Like them, she routinely conflates the common symptom of chronic fatigue with the illness called chronic fatigue syndrome—a serious error with potentially harmful consequences. (I will mostly use chronic fatigue syndrome in describing the research because that is the term they use.)
Dr. Crawley favors subjective over objective outcomes. In PACE, of course, the objective measures–like a walking test, a step-test for fitness, and employment status—all failed to demonstrate “recovery” or reflect the reported improvements in the two primary subjective outcomes of physical function and fatigue. FITNET-NHS doesn’t even bother with such measures. The primary outcome is a self-report questionnaire assessing physical function, and almost all the secondary outcomes are also subjective.
This is particularly troubling because FITNET-NHS, like PACE, is non-blinded; that is, both participants and investigators know which intervention they are receiving. Non-blinded studies with subjective outcomes are notoriously vulnerable to bias—even more when the intervention itself involves telling participants that the treatment will make them better, as is the case with the kind of cognitive behavior therapy provided for ME/CFS patients.
The FITNET-NHS study protocol states that participants will be identified using the guidelines developed by NICE—the U.K.’s National Institute for Health and Care Excellence. The protocol describes the NICE guidelines as requiring three months of fatigue, plus one or more of nine additional symptoms: post-exertional malaise, difficulty sleeping, cognitive dysfunction, muscle and/or joint pain, headaches, painful lymph nodes, general malaise, dizziness and/or nausea, or palpitations. In other words, according to the protocol, post-exertional malaise is not required to participate in FITNET-NHS; it is clearly identified as an optional symptom. (In the U.K., the illness can be diagnosed at three months in children, rather than at six months.)
But the proposal’s claim to be following the NICE guidelines does not appear to be true. In the NICE guidelines, post-exertional malaise is not an optional symptom. It is required, as an essential element of the fatigue itself. (In addition, one or more of ten other symptoms must also be present.) To repeat: post-exertional malaise is required in the NICE guidelines, but is not required in the description of the NICE guidelines provided in the FITNET-NHS protocol.
By making this subtle but significant shift—a sleight-of-guideline, so to speak—Dr. Crawley and her colleagues have quietly transformed their prospective cohort from one in which post-exertional malaise is a cardinal characteristic of the illness to one in which it might or might not be present. And they have done this while still claiming–inaccurately–to follow NICE guidelines. As currently described, however, Dr. Crawley’s new study is NOT a study of chronic fatigue syndrome, as she maintains, but of chronic fatigue.
As a result, the actual study participants, like the PACE cohort, will likely be a heterogeneous grab bag of kids suffering from fatigue for any number of reasons, including depression–a common cause of exhaustion and a condition that often responds to psychotherapeutic interventions like CBT. Some or even many participants—an unknown number—will likely be genuine ME/CFS patients. Yet the results will be applied to ALL adolescents identified as having that illness. Since those who actually have it suffer from the required symptom of post-exertional malaise, an intervention that encourages them to increase their activity levels, like CBT, could potentially cause harm.
(I suppose it’s possible the FITNET-NHS protocol’s inaccurate description of the role of post-exertional malaise in the NICE guidelines was inadvertent, a case of sloppiness. If so, it would be an extraordinary oversight, given the number of people involved in the study and the enormous implications of the switch. It is curious that this obvious and jarring discrepancy between the NICE guidelines and the FITNESS-NHS description of them was not flagged during the review process, since it is easy to check whether the protocol language accurately reflects the recommendations.)
Dutch FITNET study:
The approach to CBT in the Dutch FITNET trial reflects that in the U.K. Of the online intervention’s 21 modules, according to the protocol for the Dutch study, fourteen “focus on cognitive behavioural strategies and include instructions and exercises on how to identify, challenge and change cognitive processes that contribute to CFS.” Of course, experts outside the CBT/GET/PACE bubble understand that ME/CFS is a physiological disease and that faulty “cognitive processes” have nothing to do with perpetuating or contributing to it.
The Dutch study found that those assigned to FITNET reported less fatigue, greater physical function, and greater school attendance than those in the comparison group, who received standard treatment–referred to as “usual care.” And using a composite definition of “recovery,” the study reported that 63% of those in the FITNET group–just shy of two-thirds–“recovered” at six months, compared to just eight percent in the comparison group. But this apparent success masks a much more complicated reality and cannot be taken at face value, for multiple reasons.
First, the subsequent 2013 paper from the Dutch team found no differences in “recovery” between participants in the two groups at long-term follow-up (on average, 2.7 years after starting). Those in the comparison group improved after the trial and had caught up to the intervention group, so the online CBT conferred no extended advantages or benefits. The researchers argued that the therapy was nonetheless useful because patients achieved gains more quickly. But they failed to consider another reasonable explanation for their results.
Those in usual care were attending in-person sessions at clinics or doctors’ offices. Depending on how often they went, how far they had to travel and how sick they were, the transportation demands could easily have triggered relapses and harmed their health. In contrast, those in the FITNET group could be treated at home. Perhaps they improved not from the treatment itself but from an unintended side effect–the sedentary nature of the intervention allowed them more time to rest. The investigators did not control for this aspect of the online CBT.
Second, the “recovery” figure in the Dutch FITNET study was a post-hoc calculation, as the authors acknowledged. The protocol for the trial included the outcomes to be measured, of course, but the authors did not identify before the trial what thresholds participants would need to meet to be considered “recovered.” The entire definition was constructed only after they saw the results—and the thresholds they selected were extremely lenient. Even two of the PACE authors, in a Lancet commentary praising the Dutch study, referred to the “recovery” criteria as “liberal” and “not stringent.” (In fact, only 36% “recovered” under a more modest definition of “recovery,” but the FITNET authors tucked this finding into an appendix and Dr. Crawley’s FITNET-NHS protocol didn’t mention it.)
Now, the fact that “recovery” was a post-hoc measure doesn’t mean it isn’t valid. But anyone citing this “recovery” rate should do so with caveats and some measure of caution. Dr. Crawley has exhibited no such reticence—in a recent radio interview, she declared flatly that the Dutch participants had made a “full recovery.” (In the same interview, she called PACE “a great, great study.” Then she completely misrepresented the results of the recent reanalyses of the PACE trial data. So, you know, take her words for what they’re worth.)
Given the hyperbole about “recovery,” the public is understandably likely to assume that Dr. Crawley’s new “landmark” study will result in similar success. A corollary of that assumption is that anyone who opposes the study’s approach, like so many in the patient and advocacy communities, could be accused of acting in ways that harm children by depriving them of needed treatment. This would be an unfair charge, since the online CBT being offered is based on the questionable premise that the children harbor untrue cognitions about their illness.
Third, the standard treatments received by the usual care group were described like this: “individual/group based rehabilitation programs, psychological support including CBT face-to-face, graded exercise therapy by a physiotherapist, etc.” In other words, pretty much the kinds of “evidence-based” strategies these Dutch experts and their U.K. colleagues had promoted for years as being effective for chronic fatigue syndrome. In the end, two-thirds of those in usual care received in-person CBT, and half received graded exercise therapy. (Many participants in this arm received more than one form of usual care.)
And yet less than one in ten of the usual care participants were found to have “recovered” at six months, according to the 2012 study. So what does that say about the effectiveness of these kinds of rehabilitative approaches in the first place? In light of the superlative findings for online CBT, why haven’t all chronic fatigue syndrome patients in the Netherlands now been removed from in-person treatments and offered this more convenient option? (Dr. Crawley’s FITNET-NHS proposal tried to explain away this embarrassing finding of the Dutch study by suggesting that those providing usual care were not trained to work with this kind of population.)
Finally, the Dutch study did not report any objective measures of physical performance. Although the study included assessments using an actometer—an ankle bracelet that monitors distance moved—the Lancet paper did not mention those results. In previous studies of cognitive and behavioral treatments for ME/CFS, reported improvements on subjective measures for fatigue or physical function were not accompanied by increases in physical movement, as measured by actometer. And in PACE, of course, the investigators dismissed their own objective measures as irrelevant or non-objective—after these outcomes failed to provide the desired results.
In response to correspondence calling for publication of the actometer data, the Dutch investigators refused, noting that “the goal of our treatment was reduction of fatigue and increase in school attendance, not increase in physical activity per se.” This is an inadequate explanation for the decision to withhold data that would shed light on whether participants actually improved in their physical performance as well as in their subjective impressions of their condition. If the actometer data demonstrated remarkable increases in activity levels in the online CBT group, is there any doubt they would have reported it?
In short, the Dutch FITNET study leaves a lot of questions unanswered. So does its U.K. version, the proposed FITNET-NHS. And Dr. Crawley’s recent media blitz—which included a “can’t-we-all-get-along” essay in The New Scientist—did little to quell any of the reasonable qualms observers might have about this latest effort to bolster the sagging fortunes of the CBT/GET/PACE paradigm.
“Patients are desperate for this trial, yet some people are still trying to stop us,” wrote Dr. Crawley in The New Scientist. “The fighting needs to end.”
However, those mysterious and sinister-sounding “some people” cited by Dr. Crawley have very thoughtful and legitimate reasons for questioning the quality of her research. The fighting, as she calls it, is likely to end when Dr. Crawley and her colleagues stop conflating chronic fatigue and chronic fatigue syndrome through the use of loose diagnostic criteria. And when they acknowledge what scientists in the U.S. and around the world now understand: The claim that cognitive and behavioral approaches are effective treatments that lead to “recovery” is based on deeply flawed research.
David Tuller is academic coordinator of the concurrent masters degree program in public health and journalism at the University of California, Berkeley.
