{"id":10854,"date":"2016-11-22T09:50:56","date_gmt":"2016-11-22T09:50:56","guid":{"rendered":"http:\/\/wames.org.uk\/cms-english\/?p=10854"},"modified":"2016-11-22T09:52:46","modified_gmt":"2016-11-22T09:52:46","slug":"tuller-says-fitnet-fraught-with-misrepresentations-methodological-problems","status":"publish","type":"post","link":"https:\/\/wames.org.uk\/cms-english\/tuller-says-fitnet-fraught-with-misrepresentations-methodological-problems\/","title":{"rendered":"Tuller says FITNET fraught with misrepresentations &#038; methodological problems"},"content":{"rendered":"<p><strong><em>Virology<\/em><\/strong> blog post, by David Tuller, 21 November 2016: <a href=\"http:\/\/www.virology.ws\/2016\/11\/21\/trial-by-error-continued-the-new-fitnet-trial-for-kids\/\" target=\"_blank\">Trial By Error, Continued: The New FITNET Trial for Kids<\/a><\/p>\n<p><strong>The article challenges:<\/strong><\/p>\n<ul>\n<li>the failure of PACE researchers to acknowledge failings of PACE<\/li>\n<li>the unproven use of CBT to reverse false illness beliefs and deconditioning<\/li>\n<li>Crawley&#8217;s FITNET study<\/li>\n<li>Crawley&#8217;s earlier study that concluded CFS affects 1.9% of 16 year olds<\/li>\n<li>Dutch FITNET study<\/li>\n<li>CFS research &amp; discussion in Netherlands<\/li>\n<\/ul>\n<p><strong>Tuller on\u00a0FITNET:<\/strong><\/p>\n<p>Like the work of the PACE authors, Dr. Crawley\u2019s research is fraught with misrepresentations and methodological problems. Like them, she routinely conflates the common symptom of chronic fatigue with the illness called chronic fatigue syndrome\u2014a serious error with potentially harmful consequences. (I will mostly use chronic fatigue syndrome in describing the research because that is the term they use.)<\/p>\n<p>Dr. Crawley favors subjective over objective outcomes. In PACE, of course, the objective measures\u2013like a walking test, a step-test for fitness, and employment status\u2014all failed to demonstrate \u201crecovery\u201d or reflect the reported improvements in the two primary subjective outcomes of physical function and fatigue. FITNET-NHS doesn\u2019t even bother with such measures. The primary outcome is a self-report questionnaire assessing physical function, and almost all the secondary outcomes are also subjective.<\/p>\n<p>This is particularly troubling because FITNET-NHS, like PACE, is non-blinded; that is, both participants and investigators know which intervention they are receiving. Non-blinded studies with subjective outcomes are notoriously vulnerable to bias\u2014even more when the intervention itself involves telling participants that the treatment will make them better, as is the case with the kind of cognitive behavior therapy provided for ME\/CFS patients.<\/p>\n<p>The FITNET-NHS study protocol states that participants will be identified using\u00a0the guidelines developed by NICE\u2014the U.K.\u2019s National Institute for Health and Care Excellence. The protocol describes the NICE guidelines as requiring three months of fatigue, plus one or more of nine additional symptoms: post-exertional malaise, difficulty sleeping, cognitive dysfunction, muscle and\/or joint pain, headaches, painful lymph nodes, general malaise, dizziness and\/or nausea, or palpitations. In other words, according to the protocol, post-exertional malaise is not required to participate in FITNET-NHS; it is clearly identified as an optional symptom. (In the U.K., the illness can be diagnosed at three months in children, rather than at six months.)<\/p>\n<p>But the proposal\u2019s claim to be following the NICE guidelines does not appear to be true. In the NICE guidelines, post-exertional malaise is not an optional symptom. It is required, as an essential element of the fatigue itself. (In addition, one or more of ten other symptoms must also be present.) To repeat: post-exertional malaise is required in the NICE guidelines, but is not required in the description of the NICE guidelines provided in the FITNET-NHS protocol.<\/p>\n<p>By making this subtle but significant shift\u2014a sleight-of-guideline, so to speak\u2014Dr. Crawley and her colleagues have quietly transformed their prospective cohort from one in which post-exertional malaise is a cardinal characteristic of the illness to one in which it might or might not be present. And they have done this while still claiming\u2013inaccurately\u2013to follow NICE guidelines. As currently described, however, Dr. Crawley\u2019s new study is NOT a study of chronic fatigue syndrome, as she maintains, but of chronic fatigue.<\/p>\n<p>As a result, the actual study participants, like the PACE cohort, will likely be a heterogeneous grab bag of kids suffering from fatigue for any number of reasons, including depression\u2013a common cause of exhaustion and a condition that often responds to psychotherapeutic interventions like CBT. Some or even many participants\u2014an unknown number\u2014will likely be genuine ME\/CFS patients. Yet the results will be applied to ALL adolescents identified as having that illness. Since those who actually have it suffer from the required symptom of post-exertional malaise, an intervention that encourages them to increase their activity levels, like CBT, could potentially cause harm.<\/p>\n<p>(I suppose it\u2019s possible the FITNET-NHS protocol\u2019s inaccurate description of the role of post-exertional malaise in the NICE guidelines was inadvertent, a case of sloppiness. If so, it would be an extraordinary oversight, given the number of people involved in the study and the enormous implications of the switch. It is curious that this obvious and jarring discrepancy between the NICE guidelines and the FITNESS-NHS description of them was not flagged during the review process, since it is easy to check whether the protocol language accurately reflects the recommendations.)<\/p>\n<p><strong>Dutch FITNET study:<\/strong><\/p>\n<p>The approach to CBT in the Dutch FITNET trial reflects that in the U.K. Of the online intervention\u2019s 21 modules, according to\u00a0the protocol for the Dutch study, fourteen \u201cfocus on cognitive behavioural strategies and include instructions and exercises on how to identify, challenge and change cognitive processes that contribute to CFS.\u201d Of course, experts outside the CBT\/GET\/PACE bubble understand that ME\/CFS is a physiological disease and that faulty \u201ccognitive processes\u201d have nothing to do with perpetuating or contributing to it.<\/p>\n<p>The Dutch study found that those assigned to FITNET reported less fatigue, greater physical function, and greater school attendance than those in the comparison group, who received standard treatment\u2013referred to as \u201cusual care.\u201d And using a composite definition of \u201crecovery,\u201d the study reported that 63% of those in the FITNET group\u2013just shy of two-thirds\u2013\u201crecovered\u201d at six months, compared to just eight percent in the comparison group. But this apparent success masks a much more complicated reality and cannot be taken at face value, for multiple reasons.<\/p>\n<p>First, the subsequent 2013 paper from the Dutch team found no differences in \u201crecovery\u201d between participants in the two groups at long-term follow-up (on average, 2.7 years after starting). Those in the comparison group improved after the trial and had caught up to the intervention group, so the online CBT conferred no extended advantages or benefits. The researchers argued that the therapy was nonetheless useful because patients achieved gains more quickly. But they failed to consider another reasonable explanation for their results.<\/p>\n<p>Those in usual care were attending in-person sessions at clinics or doctors\u2019 offices. Depending on how often they went, how far they had to travel and how sick they were, the transportation demands could easily have triggered relapses and harmed their health. In contrast, those in the FITNET group could be treated at home. Perhaps they improved not from the treatment itself but from an unintended side effect\u2013the sedentary nature of the intervention allowed them more time to rest. The investigators did not control for this aspect of the online CBT.<\/p>\n<p>Second, the \u201crecovery\u201d figure in the Dutch FITNET study was a post-hoc calculation, as the authors acknowledged. The protocol for the trial included the outcomes to be measured, of course, but the authors did not identify before the trial what thresholds participants would need to meet to be considered \u201crecovered.\u201d The entire definition was constructed only after they saw the results\u2014and the thresholds they selected were extremely lenient. Even two of the PACE authors, in a\u00a0Lancet\u00a0commentary praising the Dutch study, referred to the \u201crecovery\u201d criteria as \u201cliberal\u201d and \u201cnot stringent.\u201d (In fact, only 36% \u201crecovered\u201d under a more modest definition of \u201crecovery,\u201d but the FITNET authors tucked this finding into an appendix and Dr. Crawley\u2019s FITNET-NHS protocol didn\u2019t mention it.)<\/p>\n<p>Now, the fact that \u201crecovery\u201d was a post-hoc measure doesn\u2019t mean it isn\u2019t valid. But anyone citing this \u201crecovery\u201d rate should do so with caveats and some measure of caution. Dr. Crawley has exhibited no such reticence\u2014in a recent radio interview, she declared flatly that the Dutch participants had made a \u201cfull recovery.\u201d (In the same interview, she called PACE \u201ca great, great study.\u201d Then she completely misrepresented the results of the recent reanalyses of the PACE trial data. So, you know, take her words for what they\u2019re worth.)<\/p>\n<p>Given the hyperbole about \u201crecovery,\u201d the public is understandably likely to assume that Dr. Crawley\u2019s new \u201clandmark\u201d study will result in similar success. A corollary of that assumption is that anyone who opposes the study\u2019s approach, like so many in the patient and advocacy communities, could be accused of acting in ways that harm children by depriving them of needed treatment. This would be an unfair charge, since the online CBT being offered is based on the questionable premise that the children harbor untrue cognitions about their illness.<\/p>\n<p>Third, the standard treatments received by the usual care group were described like this: \u201cindividual\/group based rehabilitation programs, psychological support including CBT face-to-face, graded exercise therapy by a physiotherapist, etc.\u201d In other words, pretty much the kinds of \u201cevidence-based\u201d strategies these Dutch experts and their U.K. colleagues had promoted for years as being effective for chronic fatigue syndrome. In the end, two-thirds of those in usual care received in-person CBT, and half received graded exercise therapy. (Many participants in this arm received more than one form of usual care.)<\/p>\n<p>And yet less than one in ten of the usual care participants were found to have \u201crecovered\u201d at six months, according to the 2012 study. So what does that say about the effectiveness of these kinds of rehabilitative approaches in the first place? In light of the superlative findings for online CBT, why haven\u2019t all chronic fatigue syndrome patients in the Netherlands now been removed from in-person treatments and offered this more convenient option? (Dr. Crawley\u2019s FITNET-NHS proposal tried to explain away this embarrassing finding of the Dutch study by suggesting that those providing usual care were not trained to work with this kind of population.)<\/p>\n<p>Finally, the Dutch study did not report any objective measures of physical performance. Although the study included assessments using an actometer\u2014an ankle bracelet that monitors distance moved\u2014the\u00a0Lancet\u00a0paper did not mention those results. In previous studies of cognitive and behavioral treatments for ME\/CFS, reported improvements on subjective measures for fatigue or physical function were not accompanied by increases in physical movement, as measured by actometer. And in PACE, of course, the investigators dismissed their own objective measures as irrelevant or non-objective\u2014after these outcomes failed to provide the desired results.<\/p>\n<p>In response to correspondence calling for publication of the actometer data, the Dutch investigators refused, noting that \u201cthe goal of our treatment was reduction of fatigue and increase in school attendance, not increase in physical activity per se.\u201d This is an inadequate explanation for the decision to withhold data that would shed light on whether participants actually improved in their physical performance as well as in their subjective impressions of their condition. If the actometer data demonstrated remarkable increases in activity levels in the online CBT group, is there any doubt they would have reported it?<\/p>\n<p>In short, the Dutch FITNET study leaves a lot of questions unanswered. So does its U.K. version, the proposed FITNET-NHS. And Dr. Crawley\u2019s recent media blitz\u2014which included a \u201ccan\u2019t-we-all-get-along\u201d essay in The New Scientist\u2014did little to quell any of the reasonable qualms observers might have about this latest effort to bolster the sagging fortunes of the CBT\/GET\/PACE paradigm.<\/p>\n<p>\u201cPatients are desperate for this trial, yet some people are still trying to stop us,\u201d wrote Dr. Crawley in The New Scientist. \u201cThe fighting needs to end.\u201d<\/p>\n<p>However, those mysterious and sinister-sounding \u201csome people\u201d cited by Dr. Crawley have very thoughtful and legitimate reasons for questioning the quality of her research. The fighting, as she calls it, is likely to end when Dr. Crawley and her colleagues stop conflating chronic fatigue and chronic fatigue syndrome through the use of loose diagnostic criteria. And when they acknowledge what scientists in the U.S. and around the world now understand: The claim that cognitive and behavioral approaches are effective treatments that lead to \u201crecovery\u201d is based on deeply flawed research.<\/p>\n<p><a href=\"http:\/\/www.virology.ws\/2016\/11\/21\/trial-by-error-continued-the-new-fitnet-trial-for-kids\/\" target=\"_blank\">Read the full article<\/a><\/p>\n<p><em>David Tuller is academic coordinator of the concurrent masters degree program in public health and journalism at the University of California, Berkeley.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Virology blog post, by David Tuller, 21 November 2016: Trial By Error, Continued: The New FITNET Trial for Kids The article challenges: the failure of PACE researchers to acknowledge failings of PACE the unproven use of CBT to reverse false &hellip; <a href=\"https:\/\/wames.org.uk\/cms-english\/tuller-says-fitnet-fraught-with-misrepresentations-methodological-problems\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":true,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[1],"tags":[11,257,426,1000,416,425,27,2155],"class_list":["post-10854","post","type-post","status-publish","format-standard","hentry","category-news","tag-cbt","tag-children","tag-cognitive-behavioural-therapy","tag-david-tuller","tag-dr-esther-crawley","tag-fitnet","tag-pace","tag-virology-blog"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p5qkYK-2P4","_links":{"self":[{"href":"https:\/\/wames.org.uk\/cms-english\/wp-json\/wp\/v2\/posts\/10854","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wames.org.uk\/cms-english\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wames.org.uk\/cms-english\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wames.org.uk\/cms-english\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/wames.org.uk\/cms-english\/wp-json\/wp\/v2\/comments?post=10854"}],"version-history":[{"count":2,"href":"https:\/\/wames.org.uk\/cms-english\/wp-json\/wp\/v2\/posts\/10854\/revisions"}],"predecessor-version":[{"id":10856,"href":"https:\/\/wames.org.uk\/cms-english\/wp-json\/wp\/v2\/posts\/10854\/revisions\/10856"}],"wp:attachment":[{"href":"https:\/\/wames.org.uk\/cms-english\/wp-json\/wp\/v2\/media?parent=10854"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wames.org.uk\/cms-english\/wp-json\/wp\/v2\/categories?post=10854"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wames.org.uk\/cms-english\/wp-json\/wp\/v2\/tags?post=10854"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}