Response: Sharpe, Goldsmith and Chalder fail to restore confidence in the PACE trial findings, by Carolyn E Wilshire, Tom Kindlon in BMC Psychology 2019 7:19 [Published: 26 March 2019]
Abstract:
In a recent paper, we argued that the conclusions of the PACE trial of chronic fatigue syndrome are problematic because the pre-registered protocol was not adhered to. We showed that when the originally specific outcomes and analyses are used, the evidence for the effectiveness of CBT and graded exercise therapy is weak.
In a companion paper to this article, Sharpe, Goldsmith and Chalder dismiss the concerns we raised and maintain that the original conclusions are robust. In this rejoinder, we clarify one misconception in their commentary, and address seven additional arguments they raise in defence of their conclusions.
We conclude that none of these arguments is sufficient to justify digressing from the pre-registered trial protocol. Specifically, the PACE authors view the trial protocol as a preliminary plan, subject to honing and improvement as time progresses, whereas we view it as a contract that should not be broken except in extremely unusual circumstances.
While the arguments presented by Sharpe and colleagues inspire some interesting reflections on the scientific process, they fail to restore confidence in the PACE trial’s conclusions.
Some extracts:
To summarise, Sharpe et al. “prefer” their modified definition because it generates similar rates of recovery to previous studies, and is also more consistent with “our clinical experience” ([5], p.. 6).Clearly, it is not appropriate to loosen the definition of recovery simply because things did not go as expected based on previous studies. Researchers need to be open to the possibility that their results may not align with previous findings, nor with their own preconceptions. That is the whole point of a trial. Otherwise, the enterprise ceases to be genuinely informative, and becomes an exercise in belief confirmation…
Putting aside the erroneous criticism regarding the APT arm, Sharpe and colleagues raised three further criticisms of our reanalysis. The first was that we did not adhere to “an a priori analysis plan” ([5], p. 1). This claim is puzzling, because of course we followed the Investigators’ own analysis plan as set out in their trial protocol – or to be precise, we followed it as closely as was possible, given the data we had available.
