Quick thoughts blog post by Prof James C Coyne, 24 May 2016: The unfolding story of removal of data from a PLOS One article

• A data set included in a 2015 PLOS One article has been removed, leaving the article no longer compliant with the PLOS data sharing policy.
• The article reports secondary analyses from the FINE sister trial to PACE, but was published later than the PLOS One PACE paper and is subject to a stricter data sharing policy.
• Removal of the data set was announced with a correction notice indicating the data set had been published “in error.”
• The correction notice states that ethical permission (patient consent form) did not expressly allow data sharing.
• Examination of the consent form indicates that it does not differ from many in use in the UK.
• The correction was posted by PLOS One only a month after it had been received, substantially quicker than such corrections are typically processed.
• The data set had been previously downloaded and used to demonstrate that outcome switching in PACE trial had been the basis for claims of positive findings, substantially affecting effect sizes and reversing conclusions in a now controversial Cochrane review.
• The effective use by a nonacademic to counter misrepresentations in the Cochrane systematic review represents the kind of benefits anticipated for ready availability of data from clinical trials, but also the activity of “research parasites” condemned by foes of routine data sharing.
• Allowing removal of the data set and posting of the correction raises concerns about a retreat from PLOS One published data sharing policies.
• The author’s  making the data available to “qualified researchers”
  represents an exclusion not previously part of the published PLOS data sharing policies currently in effect.
• The article is now in noncompliance with the PLOS data sharing policy that authors be the named individuals who are responsible for ensuring data access.
• Broad issues are raised about the continued lack of transparency in negotiations between PLOS senior management and investigators in the UK, to the exclusion of other significant stakeholders, and allowing of substantial concessions to UK chronic fatigue syndrome investigators.
• A substantial proportion of studies recently published or under review for publication in PLOS One have consent forms that do not expressly allow data sharing.
• Presumably authors of a manuscript submitted today would simply include such a statement in order to sidestep the PLOS data sharing policy.
• Does this represent the collapse of the PLOS data sharing policies with a loophole being granted through which most investigators could pass?
• Despite the removal of the data set from the PLOS One article, it remains available on the Internet. Here‘s a link.
• Three actions should occur immediately: (1) restoration of the data set to the article with an apology from the FINE investigators; (2) a full explanation from senior editor Iratxe Puebla as to how the stealth change to the PLOS One article so rapidly occurred; and (3) involvement of other stakeholders in the negotiations between PLOS One and King’s College London and Queen Mary University London.
• Last week I requested a telephone conversation with Dr. Puebla and she indicated she would get back to me. But for now, academic editors, the pool of reviewers on which PLOS depends, and authors need to consider other options if the journal does not reassert a commitment to its previously published data sharing policies.

Commentary and documentation

The corrected PLOS One article from which the data were removed is here. The original statement in the article concerning availability of data:

 “The authors have prepared a dataset that fulfills requirements in terms of anonymity and confidentiality of trial participants, and which contains only those variables which are relevant to the present study. Data are available as Supplementary Information”.

The corrected statement now reads:

“S1 Dataset was published in error.  The error was corrected in the XML and PDF versions of this article on May 9, 2016. Please download this article again to view the correct version.”

What does it mean that the data set was published in error? Was the “error” a matter of accidentally striking the wrong key on a computer keyboard? A mis-sent email? Saying a data set was published in error recalls “mistakes were made.”

“The New York Times has called the phrase a “classic Washington linguistic construct.” Political scientist William Schneider suggested that this usage be referred to as the “past exonerative” tense,[1] and commentator William Safire has defined the phrase as “[a] passive-evasive way of acknowledging error while distancing the speaker from responsibility for it”.[2] A commentator at NPR declared this expression to be “the king of non-apologies“.[3] While perhaps most famous in politics, the phrase has also been used in business, sports, and entertainment.”

Manchester University quickly responded to inquiry whether the previously availability of the data in the PLOS One article represented a violation of the UK Data Privacy Act of 1998. MU offered the reassurance that the data set is “anonymised and contains no information which could identify an individual and so there is no need for us to report to the ICO [Information Commission Officer].”

The statement concerning Data Availability in the corrected article is now:

“Our ethical permission did not expressly permit us to share patient data, even anonymised patient data, in a public forum. Data will be made available to bona fide researchers on application to the principal investigator, Alison Wearden or the trial statistician, Graham Dunn. Alison Wearden can be contacted at:
Alison.wearden@manchester.ac.uk. Graham Dunn can be contacted at:
Graham.dunn@manchester.ac.uk.”

Journalist Leonid Schneider provided links to both the patient consent form from FINE and a comparison to what was used in PACE. He points out:

“The original FINE patient consent form from 2004 is available here, it is also very similar to the patient consent form of PACE. The patients were not specifically deciding on allowing or prohibiting the open sharing of their anonymised patient data, but the form makes references to UK Data Protection Act from 1998, which may or may not prevent such data release.”

The PLOS One data sharing policy in effect when the article was published is here

I asked a University of Pennsylvania bioethics professor, Jon Merz, JD, PHD for a comment on the consent forms and the PLOS one correction. He replied:

“If not prohibited, a la “we will never share your data, even if anonymized, with others.” then sharing is allowed, and should be encouraged.  The fact these authors admit sharing with others whom they deem “bona fide” researchers gives them unfettered discretion, which discretion has been abused.  By what standard do they decide?”

Journalist Leonid Schneider reports the following statement from Alison Weardon:

“We published the PLOS One paper on therapist effects and therapeutic alliance and provided a de-identified dataset containing the variables used in the analysis.

On 10th March, I received a freedom of information request for a copy of the patient consent form for the FINE trial. The request referred to the ongoing case relating to the PACE trial, and raised the issue of whether in fact we had been correct to make the data relating to the therapist paper open to the public. We did not request permission to do so in our trial consent form.

The dataset supplied to support the PLOS-One article was supplied in good faith and in the belief (still held) that no patient or therapist would be identifiable from it. The Freedom of Information request made me wonder if we had acted correctly, given our ethical permissions. In consultation with my co-authors and after discussing with various colleagues, I decided that it would be better to remove the dataset from public access (while still being prepared to supply it to bona-fide researchers). I wrote to PLOS-One on 18th April asking them whether it would be possible to do this.

The contents of the paper have not been retracted. The dataset has not been retracted. There is nothing wrong with either of them. The only issue is whether or not we were right in publishing this dataset given the consent that we had obtained from the trial participants“.

In the absence of other information, I think we can reasonably assume that the “various colleagues” included the PACE investigators.

In fending off Freedom of Information Act requests for sharing data, Peter White has repeatedly argued that it is a legitimate reason for withholding data if the data might be used to embarrass the investigators’ claims and reputations. The data from the PLOS One article was used effectively to discredit claims of the PACE investigators and raised concerns about conflicts of interest in a Cochrane systematic review in which they provided consultation.

See how one individual of this data allowed the unmasking of the effects of the day outcome switching in the PACE trial in a PubMed Commons comment by Sam Carter, Exploring changes to PACE trial outcome measures using anonymised data from the FINE trial.

I discuss the importance of Carter’s work in a blog post concerning the opening investigation of undeclared conflict of interest having produced a flawed Cochrane review in a dismissive response from the author to critics. Probing an untrustworthy Cochrane review of exercise for “chronic fatigue syndrome.”

As an academic editor of PLOS One, I am personally offended by senior management’s apparent acceptance of a restriction on data sharing to the vague category “bona fide researchers” and their encouraging further violations of data sharing policies by allowing the authors of the paper to decide themselves about release of their data on a case-by-case basis.

Peter White and the PACE investigators reject sharing data even with PhD’s who do not have formal academic appointments. Recently White made the argument that posting  on patient forums disqualified Dr. Wood from accessingthe PACE data [UK expert: AIDS data should not be shared until requesters shown to be HIV-

Many accomplished persons who do not have academic appointments effectively contribute to post publication peer review and to Cochrane reviews. Hilda Bastian is a great example. She also has delightfully funny blogs with PLOS. Hilda does not have PhD, but she played a crucial role in launching and sustaining PubMed Commons. Hilda works at the US National Library of Medicine, and has no  academic appointment. Would  she be turned down for access to the FINE data because she is a mere librarian, not a professor.

Graduate students, postdocs, and even patients effectively use PubMed Commons to comment on published work. They often use publicly available data sets from published studies PubMed Commons access only requires that someone can document that they have been an author on one of the 27 million entries in Pubmed, even a letter to the editor.
I doubt this would be sufficient to qualify as an “bona fide researcher” especially with FINE authors as the judges.  They are closely aligned with PACE, their sister trial investigators. At this point, I don’t expect them to be detached, honest brokers of access to the data.

The PLOS management is aligning itself with Richard Horton, The Lancet editor who has such a contemptuous attitude to individuals outside of the Oxbridge network, as well as The New England Journal of Medicine Editor,  Jeff Drazen:

PLOS One has also aligned itself in opposition to more progressive UK voices about data sharing like the present and former editors of BMJ.
See:

Richard Lehman’s Journal Review, Share data or be damned

Godlee, F. Call for greater involvement of patients.  BMJ 2015

Loder E, Groves T. The BMJ requires data sharing on request for all trials.

Richard Smith: QMUL and King’s college should release data from the PACE trial

Smith R and Roberts  Time for sharing data to become routine: the seven excuses for not doing so are all invalid F1000Research 2016,  5:781

Did the FINE authors get special treatment in the seemingly expedited handling of the correction?

A case can be made that they did.

I called the attention of the PLOS management to a 2015 article  from a Harvard Medical School professor that was accepted by a PLOS One Academic Editor at Harvard Medical School. The article had substantial undeclared conflicts of interest, was essentially a flawed experimercial for “integrative” medicine services offered by Harvard and violated PLOS data sharing policies by not making data available.

PLOS eventually responded to my complaint in February 2016, sharing a draft correction notice. It still has not appeared at the Journal site and these other serious issues remain unaddressed. Yet in the interim, the FINE authors were able to get their correction notice up and their actual article in less than a month.

For further background see

In the standoff over release of the PACE PLOS One trial data, has the journal just blinked?

What patients should require before consenting to participate in research…

UK expert: AIDS data should not be shared until requesters shown to be HIV-

Recognizing when “protecting patient privacy” is mere excuse for not sharing data

Postscript

I am losing patience with PLOS. My confidence in their commitment to data sharing is faltering. I remain convinced that PLOS One can reestablish its place at the forefront of open access, transparency and reporting, and availability of data from the analysis. I’m frustrated with the opaque negotiations that are been going on with UK investigators, the lack of involvement of other stakeholders, and the concessions that are been granted.
It may be too early to act, but is not too early to begin planning an action in which academic editors, the reviewers on whom they depend, and the authors who submit papers avoid involvement with PLOS One for a month. During that time, the promotional booths that PLOS sets up at various scientific meetings should be given wide berth. Or- may be walk up and say “What is the PLOS data sharing policy for someone from Arkansas, not Oxbridge?”