Trial By Error: Open letter to Dr Godlee about BMJ’s ethically bankrupt actions (2), by by David Tuller, DrPH in Virology blog, 22 November 2019


WAMES has joined 55 clinicians & scientists and 63 other organisations in signing this protest at the BMJ republishing an unreliable study of the Lightning Process in children with CFS.


In July, I sent Dr Fiona Godlee, editorial director of BMJ, a letter signed by 55 experts about her company’s perplexing decision to republish the originally reported–and unreliable–findings from the trial of the Lightning Process. She did not respond. This morning I sent the letter again, with more individual signatories along with dozens of patient and advocacy groups.


Dear Dr Godlee,

We are writing about the correction appended in June [1] to the pediatric study of the Lightning Process conducted by investigators from the University of Bristol and published by Archives of Disease in Childhood in September, 2017. The study appeared under the following title: “Clinical and cost-effectiveness of the Lightning Process in addition to specialist medical care for paediatric chronic fatigue syndrome: randomised controlled trial.” [2]

As Virology Blog documented shortly after publication [3], the investigators recruited more than half the sample before trial registration as part of a feasibility study, swapped primary and secondary outcomes after gathering data from these early participants, and did not disclose these salient details in the paper. With the correction and an accompanying editor’s note, Archives of Disease in Childhood has now acknowledged the validity of Virology Blog’s criticisms. Yet the journal has allowed the investigators to reassert their main conclusion–that the Lightning Process is “effective” in treating pediatric illness. This decision is untenable.

The Lightning Process is a commercial program combining life-coaching, neuro-linguistic programming and positive affirmations. Participants are taught that they can overcome illness by controlling and changing their thought patterns. Lightning Process practitioners have declared that they can successfully treat multiple sclerosis, eating disorders and other serious conditions. Government regulators have admonished practitioners for making misleading claims.

Beyond concerns about the Lightning Process itself, the Bristol study was an open-label trial with self-reported outcomes—a combination of design features that can lead to significant bias. Furthermore, in violating core methodological and ethical principles of scientific inquiry, the investigators also breached BMJ’s own strict policy against publishing any trials in which participants have been recruited before registration. Since all major medical journals adopted this policy in 2005, experienced investigators, including members of the Bristol team, should have been well aware of it.

BMJ has been a leader in the movement to require prospective registration as a condition for  trials to be considered for publication. In 2013, in testimony before the House of Common’s Science and Technology Committee, you touted BMJ’s success in implementing a zero-tolerance approach to such trials. “As far as we are aware, in the last two years we have not published any trial that has not been prospectively registered,” you stated—a claim you could not accurately repeat today. [4] A 2015 editorial in The BMJ called the prospective registration policy “the single most valuable tool we have to ensure unbiased reporting of research studies.” [5]

Given BMJ’s longstanding position that prospective registration is essential for safeguarding the integrity of the medical literature, it is unclear why you are now willing to exempt the Lightning Process study from this requirement. The editor’s note posted with the correction declared that lack of prospective registration was not grounds for retraction. [6]  No explanation or justification was provided for this determination, which appears to be incompatible with the zero-tolerance approach you advocated in your parliamentary testimony.

As you certainly know, a key goal of prospective registration is to prevent selective outcome reporting. Yet selective outcome reporting is exactly what occurred in the Lightning Process study when the investigators swapped primary and secondary outcomes mid-way through. In the published paper, the investigators reported positive results for the revised primary outcome of self-reported physical function at six months, which had initially been designated a secondary outcome. In contrast, they reported null results for the original primary outcome of school attendance at six months, which by the end had been demoted to a secondary outcome.

The swap thus ensured that the investigators were able to report positive rather than null results for their official primary outcome. Not surprisingly, this change had a major impact on how the information was presented in Archives of Disease in Childhood and interpreted by news organizations. The positive findings for the primary outcome that was selected after more than half the participants had provided data were prominently highlighted in the published paper. These findings received widespread media attention [7] and have recently been cited in a systematic review as evidence that the intervention is “effective.” [8] The null findings for the original primary outcome were ignored.

The editor’s note explains that the journal examined the outcome-swapping issue by “seeking assurance from the authors that the change in primary outcome was not influenced by (positive) findings in the feasibility phase.” This statement is perplexing. Individuals subject to potential bias are not generally considered impartial and authoritative arbiters of whether this potential bias has influenced their decision-making. Study design is supposed to seek to minimize bias precisely because humans tend to be blind to their own biases. Reputable science in high-impact journals should not have to rely on investigators’ “assurance” that they have resisted the natural temptation to let their perceived interests guide their selection of primary outcomes.

Moreover, in relation to any assurances provided by the Bristol investigators, it is worth remembering that they withheld important information about trial registration and outcome-swapping from their public version of events. Archives of Disease in Childhood would not have published the paper in the first place except for two major failures: 1) The investigators’ failure to provide an accurate account of how they conducted the trial; and 2) The journal’s failure to detect disqualifying flaws, despite its obligation to subject the paper to rigorous peer review and editorial oversight.

Archives of Disease in Childhood has now re-published positive primary results generated by outcome-swapping in a study that did not meet a strict publication requirement specifically intended to prevent outcome-swapping and other kinds of selective outcome reporting. In doing so, the journal has rewarded the Bristol investigators for their lack of candor and their methodological missteps and has contradicted BMJ’s and your own past statements about the critical importance of prospective registration.

In this case, BMJ’s actions are scientifically and ethically indefensible. They are also potentially harmful to the health and well-being of children. Thank you for your attention to this troubling matter.

This letter is being cc’d to many of the signatories as well as to others with an interest in this issue, including: several members of the committee from the National Institute for Health and Care Excellence currently developing new guidelines for ME/CFS; the director of legal services at the University of Bristol; the chair and vice chair of the CFS/ME Research Collaborative; the senior author of the Lightning Process study; the senior author of a review that cited the Lightning Process study and the editor of the journal that published that review; and a few journalists.


Dharam V. Ablashi, DVM, MS, Dip Bact
Scientific Director, HHV-6 Foundation, Santa Barbara, California, USA
Former Senior Investigator, National Cancer Institute, National Institutes of Health
Bethesda, Maryland, USA

Raeka Aiyar, PhD
Senior Director of Scientific Outreach, New York Stem Cell Foundation Research Institute

Michael Allen, PhD
Clinical Psychologist (retired), San Francisco, California, USA

Chris Armstrong, PhD
Science Liaison, Open Medicine Foundation, Agoura Hills, California, USA
Visiting Scholar, Stanford University, Stanford, California, USA

James N. Baraniuk, MD
Professor of Medicine Georgetown University, Washington, DC, USA

Lisa F. Barcellos, MPH, PhD
Professor of Epidemiology, School of Public Health, California Institute for Quantitative Biosciences, University of California, Berkeley, Berkeley, California, USA

Lucinda Bateman, MD
Medical Director Bateman Horne Center, Salt Lake City, Utah, USA

Alison C. Bested, MD, FRCPC
Clinical Director, Institute for Neuro-Immune Medicine
Associate Professor, Nova Southeastern University, Fort Lauderdale, Florida, USA

Charlotte Blease, PhD
Fulbright and Marie Curie Research Fellow, General Medicine and Primary Care
Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, USA

Molly Brown, PhD
Assistant Professor, Department of Psychology, DePaul University, Chicago, Illinois, USA

Robin Callender Smith, PhD
Professor of Media Law, Centre for Commercial Law Studies, Queen Mary University of London
Barrister and Information Rights Judge, London, England, UK

Joan Crawford, CPsychol, CEng, CSci, MA, MSc
Chartered Counselling Psychologist, Chronic Pain Management Service, St Helens Hospital, St Helens, England, UK

Janet L. Dafoe, PhD
Child Psychologist in Private Practice, Palo Alto, California, USA

Todd E. Davenport, PT, DPT, MPH, OCS
Professor and Program Director, Department of Physical Therapy
Thomas J. Long School of Pharmacy & Health Sciences, University of the Pacific
Stockton, California, USA
Workwell Foundation, Ripon, California, USA

Ronald W. Davis, PhD
Professor of Biochemistry and Genetics, Stanford University, Stanford, California, USA

Simon Duffy, PhD, FRSA
Director, Centre for Welfare Reform, Sheffield, England, UK

Jonathan C.W. Edwards, MD
Emeritus Professor of Medicine, University College London, London, England, UK

Valerie Eliot Smith
Barrister and Visiting Scholar, Centre for Commercial Law Studies
Queen Mary University of London, London, England, UK

Margaret C. Fernald, PhD
Clinical Child Psychologist, Clinical Associate of Psychology
University of Maine, Orono, Maine, USA

Kenneth J. Friedman, PhD
Associate Professor of Physiology and Pharmacology (retired)
New Jersey Medical School, University of Medicine and Dentistry of New Jersey, Newark, New Jersey, USA

Robert F. Garry, PhD
Professor of Microbiology and Immunology, Tulane University School of Medicine, New Orleans, Louisiana, USA

Claudia Gillberg, PhD
Fellow, Centre for Welfare Reform, Sheffield, England, UK
Senior Research Associate, National Centre for Lifelong Learning, Jonkoping University
Jonkoping, Sweden

Rebecca Goldin, PhD
Professor of Mathematics, George Mason University, Fairfax, Virginia, USA

Alan Gurwitt, MD
Psychiatrist in Private Practice (retired), Associate Clinical Professor
Yale Child Study Center (retired), New Haven, Connecticut, USA
Associate Clinical Professor, University of Connecticut Dept of Psychiatry (retired)
Storrs, Connecticut, USA
Lecturer, Harvard Medical School (retired), Boston, Massachusetts, USA

Paul Guyre, PhD
Emeritus Professor Micro Immunology, Geisel School of Medicine, Dartmouth
Hanover, New Hampshire, USA

Geoffrey Hallmann, LLB, DipLegPrac, DipFinPrac
Specialist in Disability and Compensation Law (retired), Lismore, New South Wales, Australia

Brian M. Hughes, PhD, FPsSI
Professor of Psychology, National University of Ireland, Galway, Galway, Ireland

Leonard A. Jason, PhD
Professor of Psychology, DePaul University, Chicago, Illinois, USA

Jonathan Kerr, MD, PhD, FRCPath
Consultant in Microbiology, West Suffolk Hospital, Bury St Edmunds, England, UK

Maureen Hanson, PhD
Liberty Hyde Bailey Professor, Department of Molecular Biology and Genetics, Cornell University, Ithaca, New York, USA

Keith Kahn-Harris, PhD
Associate Lecturer in Sociology, Birkbeck, University of London, London, England, UK

Mady Hornig, MD
Associate Professor of Epidemiology, Mailman School of Public Health, Columbia University, New York, New York, USA

Jon D. Kaiser, MD
Clinical Faculty, Department of Medicine, University of California Medical School
San Francisco, California, USA

Nancy Klimas, MD
Director, Institute for Neuro-Immune Medicine, Nova Southeastern University
Director, Miami VA Medical Center GWI and CFS/ME Program, Miami, Florida, USA

Eliana M. Lacerda, MD, MSc, PhD
Clinical Assistant Professor, International Centre for Evidence in Disability
Faculty of Infectious and Tropical Diseases, London School of Hygiene & Tropical Medicine, London, England, UK

Bruce Levin, PhD
Professor of Biostatistics, Columbia University, New York, New York, USA

Rogier Louwen, PhD
Assistant Professor, Department of Medical Microbiology and Infectious Diseases
Erasmus University Medical Center, Rotterdam, The Netherlands

Steven Lubet, JD
Williams Memorial Professor of Law, Northwestern University Pritzker School of Law,
Chicago, Illinois, USA

Sonya Marshall-Gradisnik
Professor of Immunology, Menzies Health Institute Queensland
Co-Director National Centre for Neuroimmunology and Emerging Diseases
Griffith University, Gold Coast, Queensland, Australia

Marlon Maus, MD, DrPH, FACS
Director, DrPH Program, School of Public Health, University of California, Berkeley
Berkeley, California, USA

Neil R. McGregor, BDS, MDSc, PhD
Clinical Associate Professor, Faculty of Medicine, Dentistry and Health Sciences
University of Melbourne, Melbourne, Victoria, Australia

Patrick E. McKnight, PhD
Professor of Psychology, George Mason University, Fairfax, Virginia, USA

Karl Morten, PhD
Laboratory Manager, Medical Services Division, University of Oxford, Oxford, England, UK

Nina Muirhead, BMBCh
Dermatologic Surgeon, Buckinghamshire NHS Trust, Buckinghamshire, England, UK

Heidi Nicholl, PhD
Chief Executive Officer Emerge Australia, Melbourne, Victoria, Australia

Steve Olson, MD
Family Physician, Large Medical Group Executive and Regional Physician Director
Oakland, California, USA

Elisa Oltra, PhD
Professor of Molecular and Cellular Biology, Catholic University of Valencia School of Medicine, Valencia, Spain

Vincent R. Racaniello, PhD
Professor of Microbiology and Immunology, Columbia University, New York,

Arthur L. Reingold, MD
Professor of Epidemiology, University of California, Berkeley, Berkeley, California, USA

Peter C. Rowe, MD
Professor of Pediatrics, Johns Hopkins University School of Medicine, Baltimore, Maryland, USA

Ola Didrik Saugstad, MD, PhD
Professor Emeritus, Division of Paediatric and Adolescent Medicine
University of Oslo, Oslo, Norway
Adjunct Professor of Pediatrics, Feinberg School of Medicine, Northwestern University
Chicago, Illinois, USA

David Scales, MD, PhD
Assistant Professor of Medicine, Division of Hospital Medicine, Weill Cornell Medicine
New York, New York, USA

Michael Scott, PhD
Clinician and Researcher, Psychological Therapies Unit, Liverpool, England, UK

Charles Shepherd, MBBS
Honorary Medical Adviser to the ME Association, Buckingham, England, UK

Donald R. Staines, MBBS, MPH, FAFPHM, FAFOEM
Clinical Professor, Menzies Health Institute Queensland
Co-Director, National Centre for Neuroimmunology and Emerging Diseases,
Griffith University, Gold Coast, Queensland, Australia

Philip B. Stark, PhD
Professor of Statistics, University of California, Berkeley, California, USA

Eleanor Stein, MD, FRCP(C)
Psychiatrist in Private Practice, Assistant Clinical Professor
University of Calgary, Calgary, Alberta, Canada

Leonie Sugarman, PhD
Emeritus Associate Professor of Applied Psychology, University of Cumbria
Carlisle, England, UK

John Swartzberg, MD
Clinical Professor Emeritus, School of Public Health, University of California, Berkeley
Berkeley, California, USA

Susan Taylor-Brown, PhD
Clinical Professor of Pediatrics (retired), Golisano Children’s Hospital
University of Rochester Medical Center, Rochester, New York, USA

Ronald G. Tompkins, MD, ScD
Summer M Redstone Professor of Surgery, Harvard Medical School
Boston, Massachusetts, USA

Samuel Tucker, MD
Assistant Clinical Professor of Psychiatry (retired), University of California, San Francisco
San Francisco, California, USA

David Tuller, DrPH
Senior Fellow in Public Health and Journalism, Center for Global Public Health
University of California, Berkeley, Berkeley, California, USA

Rosamund Vallings, MNZM, MBBS
General Practitioner, Auckland, New Zealand

Linda van Campen, MD
Cardiologist, Stichting Cardiozorg, Hoofddorp, The Netherlands

J. Mark VanNess, PhD
Professor and Chair, Health and Exercise Science
University of the Pacific, Stockton, California, USA

Mark Vink, MD
Family Physician, Soerabaja Research Center Amsterdam, The Netherlands

Frans Visser, MD
Cardiologist, Stichting Cardiozorg, Hoofddorp, The Netherlands

Tony Ward, MA (Hons), PhD, DipClinPsyc
Registered Clinical Psychologist, Professor of Clinical Psychology
School of Psychology, Victoria University of Wellington, New Zealand
Adjunct Professor, School of Psychology, University of Birmingham
Birmingham, England, UK
Adjunct Professor, School of Psychology, University of Kent
Canterbury, England, UK

William Weir, FRCP
Infectious Disease Consultant, London, England, UK

John Whiting, MD
Specialist Physician in Private Practice, Brisbane, Queensland, Australia

Carolyn Wilshire, PhD
Senior Lecturer, School of Psychology, Victoria University of Wellington
Wellington, New Zealand


#MEAction UK

#MEAction USA


25% ME GROUP – Support for severe ME sufferers

Action CIND

Action for ME

American ME and CFS Society

ANZMES (Associated New Zealand ME Society Inc)
New Zealand

Central London ME Group

Chester ME Self-help (MESH)

Deutsche Gesellschaft für ME/CFS | German Association of ME/CFS

Dr Myhill’s MAIMES Campaign

Emerge Australia

European ME Alliance (EMEA)

Belgium ME/CFS Association

Czech Republic ME/CFS Association


Suomen ME/CFS-yhdistys

Suomen lääketieteellinen ME/CFS-Yhdistys ry

ASFC – Association Française du Syndrome de Fatigue Chronique

Fatigatio e.V.

Icelandic ME Association

Irish ME Trust

Associazione CFS Veneto

Het Alternatiet
The Netherlands

Norges ME-forening

Liga SFC

RME Riksföreningen för ME-patienter

Verein ME/CFS Schweiz

Hope 4 ME & Fibro Northern Ireland

Invest in ME Research

FNC ME/CFS Association

Global Advocates 4 Myalgic Encephalomyelitis (GAME)                                                        Canada   (*Added to list on Nov 22 after letter was sent to Dr Godlee)

Groep ME-DenHaag
The Netherlands

Lost Voices Foundation

Massachusetts ME/CFS & FM Association

ME Advocacy Ireland

ME Association

ME Research UK

ME/CFS Australia Ltd.

ME/CFS & FM Association NSW

ME/CFS and Lyme Association of WA, Inc.

ME/CFS South Australia Inc

ME/CFS/FM Support Association Qld Inc.

ME/CVS Stichting Nederland (ME/CVS Foundation Netherlands)
The Netherlands

ME/cvs Vereniging
The Netherlands

ME/FM Society of British Columbia

Millions Missing Canada

Minnesota ME/CFS Alliance

National CFIDS Foundation, Inc.
USA   (*Corrected on Nov 22 from “CFIDS Foundation, Inc.”)

National ME/FM Action Network

New Jersey ME/CFS Association, Inc.

North London ME Network

Open Medicine Foundation

Opposing MEGA
(*Added to list after letter sent to Dr Godlee)

Oxfordshire ME Group for Action
(*Added to list after letter sent to Dr Godlee)

Pandora Org

Science for ME
UK (online forum with members from many countries)

Sheffield ME and Fibromyalgia Group

Solve ME/CFS Initiative

Steungroep ME en Arbeidsongeschiktheid
The Netherlands

Suomen lääketieteellinen ME/CFS-yhdistys ry

The York ME Community

The Young ME Sufferers Trust

Wake-Up Call Beweging

WAMES – Welsh Association of ME & CFS Support

Wisconsin ME and CFS Association, Inc


[1] Crawley E, Gaunt D, Garfield K, et al. Notice of correction and clarification: Clinical and cost-effectiveness of the Lightning Process in addition to specialist medical care for paediatric chronic fatigue syndrome: randomised controlled trial. Archives of Disease in Childhood; 2019. Accessed at:

[2] Crawley E, Gaunt D, Garfield K, et al. Clinical and cost-effectiveness of the Lightning Process in addition to specialist medical care for paediatric chronic fatigue syndrome: randomised controlled trial. Archives of Disease in Childhood; 2018; 103:155-164

[3] Tuller D. Trial by error: the SMILE trial’s undisclosed outcome-swapping. Virology Blog; Dec 13, 2017. Accessed at:

[4] Clinical trials: Hearings before the Science and Technology Committee, House of Commons; March 13, 2003. (Testimony of Fiona Godlee) Accessed at:

[5] Weber W, Merino J, Loder E. Editorial: trial registration 10 years on. BMJ; 2015; 351. Accessed at:

[6] Brown N. Editor’s note on correction to Crawley et al (2018). Archives of Disease in Childhood; 2019. Accessed at:

[7] Bosely, S. Controversial Lightning Process ‘helps children with chronic fatigue syndrome.’ The Guardian; September 20, 2017. Accessed at:

[8] Gregorowski A, Simpson J, Segal T. Child and adolescent chronic fatigue syndrome/myalgic encephalomyelitis; where are we now? Current Opinion in Pediatrics; 2019; 31: 462-46.

This entry was posted in News and tagged , , , , , , , , . Bookmark the permalink.

Comments are closed.