{"id":14343,"date":"2017-09-14T06:34:34","date_gmt":"2017-09-14T06:34:34","guid":{"rendered":"http:\/\/wames.org.uk\/cms-english\/?p=14343"},"modified":"2017-09-14T06:36:27","modified_gmt":"2017-09-14T06:36:27","slug":"trial-by-error-the-nice-topic-expert-reports-foi","status":"publish","type":"post","link":"https:\/\/wames.org.uk\/cms-english\/trial-by-error-the-nice-topic-expert-reports-foi\/","title":{"rendered":"Trial By Error: The NICE \u201cTopic Expert\u201d Reports (FOI)"},"content":{"rendered":"

Virology<\/strong> blog post, by David Tuller, 11 September 2017: Trial By Error: The NICE \u201cTopic Expert\u201d Reports<\/a><\/h3>\n

<\/h3>\n

Freedom of Information requests to NICE<\/strong><\/h3>\n

My first recent freedom of information request to the National Institute for Health and Care Excellence (NICE) was for information about the experts consulted in the current process of reviewing CG53,\u00a0the 2007 guidance<\/a>\u00a0for the illness the agency calls chronic fatigue syndrome\/myalgic encephalomyelitis. In its response, the agency explained that seven topic experts had been consulted in the process of preparing the surveillance document, which recommended leaving the guidance as is.<\/p>\n

(I have previously written about the NICE review process on CG53\u00a0here<\/a>,\u00a0here<\/a>\u00a0and\u00a0here.<\/a> My e-mail exchange with Sir Andrew Dillon, chief executive of the NICE guidance executive, is described\u00a0here.)<\/a><\/p>\n

Who are the experts?<\/strong><\/h3>\n

NICE, of course, declined to reveal the names of the seven topic experts, although it reported that four of them were members of the original 2007 guidance development committee. NICE also told me, as it told the ME Association, that the panel included three psychiatrists, two neurologists, a pediatrician, and a patient. NICE did not explain why the panel included more psychiatrists than other specialists, but the selection of topic experts certainly provides insight into NICE\u2019s perspective on the illness.<\/p>\n

The experts’ reports<\/strong><\/h3>\n

My next FOI request was for any reports prepared by these topic experts\u2013properly anonymized, of course. I recently received them and\u00a0have asked for them to be posted on Phoenix Rising.<\/a> If this is the full extent of the input provided by the topic experts, it seems like NICE made a remarkably anemic effort to seek outside advice. And it sought that advice from a relatively narrow spectrum of professional opinion, especially given the enormous complexity of the disease.<\/p>\n

In its e-mail to me, NICE explained that the topic experts provided \u201cthree types of contributions.\u201d Each person was asked to fill out two questionnaires. The first questionnaire sought suggestions on new studies or research that could impact diagnostics, interventions and other aspects of the guidance. The second sought input about two specific studies and whether the surveillance report should contain in-depth commentaries on them. These questionnaire responses were due last December. The questionnaires were sent to me as separate documents.*<\/p>\n

(The previous sentence has been corrected. The sentence originally read: \u201cThe questionnaires, which were often only partially filled out, were sent to me as separate documents.\u201d)<\/p>\n

In March of this year, NICE surveyed the same experts about three 2015 U.S. reports\u2014from the Agency for Healthcare Research and Quality, the Institute of Medicine (now the National Academy of Medicine), and the CFS\/ME Advisory Committee to the Department of Health and Human Services. NICE wanted to know about the possible impact of these reports on diagnostic criteria and other aspects of CG53. Not all of the topic experts answered. The responses were all sent to me as part of a single document.<\/p>\n

Strangely, the first questionnaire sent to all seven topic experts included this sentence in the first paragraph: \u201cAs a member of the committee that developed this guideline we welcome your views on any areas that need updating.\u201d That suggests that all seven topic experts consulted were, in fact, members of the committee that developed the 2007 guidance\u2013even though NICE had already informed me that only four of the seven were on the 2007 committee.<\/p>\n

I have no idea whether NICE misinformed me that only four of the seven were on the original committee, or whether the wrong form was sent three topic experts who were not actually on the 2007 guidance committee. Whatever. NICE obviously has glitches in its fact-checking department. I hope that problem gets worked out soon.<\/p>\n

Statements from the experts<\/strong><\/h3>\n

The documents I received include some interesting and irritating statements from the experts, but nothing particularly surprising. Two of the six medical experts (#4 and #7) expressed modest concerns that recent events had rendered the current guidance somewhat out-of-date. For example, #4 cited the \u201cconsiderable on-going public and media interest in this disorder\u201d and advised NICE to have \u201ca current and up to date view on this controversial area.\u201d The other four (#1, #2, #3, #6) found that the research supported the current recommendations.<\/p>\n

The patient in the group (#5) expressed personal opinions about the difficulties patients face as well as concerns about case definition\u2013but did not fill out the questionnaires provided by NICE and did not make recommendations related to the guidance. #5 attached two additional statements from other ME patients to his\/her\/their submission to NICE; together they included many references to the emerging science and to critiques of the PACE trial.*<\/p>\n

(The previous sentence has been corrected\u2013originally I wrote that NICE did not send me these two additional attachments as part of the FOI response, but it turned out I didn\u2019t notice them in the set of documents.)<\/p>\n

Expert #6, an apparent CBT\/GET hardliner, had particularly firm opinions about what was what: \u201cThis guidance has stood the test of time well, and has been instrumental in shifting thinking about the common problem of CFS\/ME and medically unexplained symptoms (functional) symptoms in general. I hope that a new guidance on management of functional disorders will extend this process further.\u201d<\/p>\n

In response to the CFSAC suggestion that it was important to focus on biomarkers and diagnostic tests, #6 wrote this: \u201cIn the UK I think we would see what the HHS CFSAC see as a failure to undertake rigorous research, more as a failure of the biological model to explain the condition adequately. However, I can understand that an alternative reaction to the failure of biological models to explain the condition is to try and define a subset of patients with the condition who appear to share a common biomarker. This approach has failed so far.\u201d<\/p>\n

Of the six medical professionals, #7 adopted the most moderate view, suggesting that \u201csome aspects of practice have moved on considerably so this guideline needs at least partial review in relation to interventions, diagnostics and prognostics.\u201d<\/p>\n

Moreover, #7 made some key suggestions about the guideline development group that should undertake these critical revisions: \u201cIt is important that the chair is someone who has no previous connection to CFS\/ME practice or research. That person should not be a mental health professional if the guideline is going to be accepted by NHS clinics and patient organizations. However, I think there should be a mental health professional on the guideline development group.\u201d<\/p>\n

#7 was also the only one to offer specific concerns about PACE. Here\u2019s what he\/she\/they wrote in discussing the research since the guidance came out in 2007: \u201cThere are some larger and more definitive UK RCTs of treatments recommended for CFS\/ME by NICE in 2007 but these are controversial and one in particular PACE in over 600 patients, published in the Lancet, has been subject to legal challenge on the grounds that after the study was started the primary outcome was changed.\u201d<\/p>\n

That comment isn\u2019t exactly accurate and it does not indicate a full awareness of the facts or of the widespread rejection of the PACE trial by the larger scientific community. But it\u2019s more skepticism and concern than shown by his\/her\/their colleagues, who mostly toed the standard line and responded as you\u2019d predict.<\/p>\n

In response to safety questions, #3 provided more evidence\u2014if any were needed\u2014that the Cochrane reviews of GET and CBT continue to be major obstacles. Here\u2019s what #3 wrote: \u201cWhilst the patient community remain concerned about safety, recent Cochrane reviews suggest exercise therapy is safe and patients are more likely to improve with treatment.\u201d<\/p>\n

“recent<\/em> Cochrane reviews suggest exercise therapy is safe “<\/em><\/p>\n

So Cochrane says it\u2019s safe, and that means the patients are wrong. And yet the reviews include the studies featuring the methodological flaws that patients\u2014and the worldwide scientific community\u2014are challenging in the first place. Reviews can\u2019t provide good evidence if they\u2019re based on bad studies, of course, but #3 did not point this out.<\/p>\n

In the second questionnaire, the\u00a0Dutch FITNET trial\u00a0<\/a>was one of two being considered as the subject of an in-depth commentary in the surveillance report. The questionnaire briefly described the study and asked for input. Here\u2019s what NICE included in its description of FITNET in the questionnaire:<\/p>\n

\u201cCBT is already recommended by CG53 and this study shows it can be delivered via the internet, indicating an alternative mode of delivery for this treatment for this age group. Although the study was from the Netherlands, a UK trial is underway. There were some issues with the study (blinding was not possible due to nature of interventions, and usual care varied because the quality and quantity of CBT differed according to local availability and was often combined with other treatments such as GET). These issues would be discussed in full in the commentary.\u201d<\/p>\n

It must be pointed out that lack of blinding is not just some small problem with the Dutch FITNET study. The lack of blinding combined with the reliance on subjective outcomes renders FITNET, and the bulk of studies from the CBT\/GET ideological brigades, incapable of providing reliable or valid evidence of treatment efficacy. Yet the topic experts generally agreed with NICE\u2019s positive assessment of FITNET and largely failed to address\u00a0the study\u2019s serious flaws<\/a>\u00a0in their comments.<\/p>\n

Among those problems was that the \u201cusual care\u201d group did very poorly. Unlike in PACE, usual care was not just a few meetings with a clinician. In FITNET, the usual care group mostly received CBT or GET, or both\u2014in other words, the treatments already recommended for the illness in the Netherlands as well as in CG53. And yet apparently these treatments largely failed to work. That raises this critical question: If CBT and GET are performing so poorly, why are they the standard of care in the Netherlands, not to mention the U.K.?<\/p>\n

In its comments about the study in the questionnaire, NICE fudged this enormous problem by suggesting that the usual care in the study \u201cvaried\u201d in quality\u2014suggesting that the treatments performed poorly because they were not optimally delivered. That\u2019s a very convenient and self-serving explanation, when the obvious and simpler answer is that the therapies simply do not work as promised.<\/p>\n

Citing this study, Professor Esther Crawley of Bristol University has received approval to inflict the FITNET approach on British children. But NICE, Professor Crawley, the topic experts and the Dutch investigators themselves have not actually considered one obvious reason why the patients assigned to the FITNET intervention might have reported better subjective outcomes. For sick patients, it would involve much less exertion to have online CBT at home than to run around town attending multiple in-person sessions. It seems logical that patients treated while in front of their laptops and not forced to travel weekly to clinic might do better than those in \u201cusual care\u201d for that reason alone.<\/p>\n

In the document compiling the responses whether the 2015 U.S. reports should impact the diagnostic criteria used to define the illness in the NICE guidance, this comment from #1 jumped out as particularly uninformed: \u201cI am unaware of any concerns about the inclusion criteria of trials in CFS.\u201d Wow. Given the enormous international controversy over the range of case definitions that have been developed for the illness, it is hard to know how to respond to such a statement.<\/p>\n

Not surprisingly, this person expressed satisfaction with the criteria outlined in CG53: \u201cI do not see any need to change the diagnostic criteria at present. From a clinical perspective, CG53 are pragmatic and useful criteria. In my clinical experience in a CFS\/ME clinic over the 10 years since the guidelines were published, no concerns have ever been expressed by patients attending the clinic about the diagnostic criteria used in CG53.\u201d<\/p>\n

And here\u2019s what #6, the CBT\/GET hardliner, had to say about the U.S.-based reports: \u201cI am suspect [sic] that there is a tendency in the USA to push towards an entirely biological explanation for the condition, whereas in the UK there is an increasing acceptance amongst patients and clinicians alike of a model that includes CFS\/ME in the umbrella of functional neurological disorders, i.e. that it is an emotionally driven disorder.\u201d<\/p>\n

CFS\/ME – “an emotionally driven disorder”<\/em><\/p>\n

Ok, then. The notion that CFS\/ME is \u201can emotionally driven disorder\u201d is certainly a popular concept in specific academic and medical circles in the U.K. But #6 does not cite any data to support the claim that there is an \u201cincreasing acceptance\u201d of such an idea among British patients and clinicians. That this pet theoretical project of a powerful clique of psychiatrists (and some others) is now presented to NICE as having popular appeal and clinical legitimacy is distressing.<\/p>\n

And that this set of comments is the best NICE could drum up in seeking external guidance for this critical review process is deeply worrying.<\/p>\n","protected":false},"excerpt":{"rendered":"

Virology blog post, by David Tuller, 11 September 2017: Trial By Error: The NICE \u201cTopic Expert\u201d Reports Freedom of Information requests to NICE My first recent freedom of information request to the National Institute for Health and Care Excellence (NICE) … Continue reading →<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[1],"tags":[1000,2254,2252,303,2155],"class_list":["post-14343","post","type-post","status-publish","format-standard","hentry","category-news","tag-david-tuller","tag-foi","tag-freedom-of-information-request","tag-nice-guidelines","tag-virology-blog"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p5qkYK-3Jl","_links":{"self":[{"href":"https:\/\/wames.org.uk\/cms-english\/wp-json\/wp\/v2\/posts\/14343","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wames.org.uk\/cms-english\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wames.org.uk\/cms-english\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wames.org.uk\/cms-english\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/wames.org.uk\/cms-english\/wp-json\/wp\/v2\/comments?post=14343"}],"version-history":[{"count":3,"href":"https:\/\/wames.org.uk\/cms-english\/wp-json\/wp\/v2\/posts\/14343\/revisions"}],"predecessor-version":[{"id":14360,"href":"https:\/\/wames.org.uk\/cms-english\/wp-json\/wp\/v2\/posts\/14343\/revisions\/14360"}],"wp:attachment":[{"href":"https:\/\/wames.org.uk\/cms-english\/wp-json\/wp\/v2\/media?parent=14343"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wames.org.uk\/cms-english\/wp-json\/wp\/v2\/categories?post=14343"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wames.org.uk\/cms-english\/wp-json\/wp\/v2\/tags?post=14343"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}