Week of:
And (free?) data for all
This Friday, the department of Health and Human Services released an (unpublished) version of rules that both clarify and enforce data sharing for clinical trials through guidelines for registration of trials as well as submitting summary results. NIH also concurrently released a policy on the dissemination of NIH-funded trials through clinicaltrials.gov. Both these rules have been in the works for a while, and have been released after an extensive comment period.
As already picked up by several news outlets, the rules are expected to lead to more information being available on clinical trials to everyone, including patients. Notable is the requirement to share data on trials that failed, something we have talked about in previous weeknotes. The guidelines have clarified a number of issues, including which trials it applies to, submission of results stratified by ethnic and racial groups, and the protocol and statistical methods used. The NIH has also outlined further initiatives, including requiring good clinical practice (GCP) training, suggesting the use of a standardized NIH-FDA protocol, and development of a standardized electronic system for NIH to manage all these trials.
Of course, over the past year and a half, we have been saying that making this world less “opaque,” as Rob Califf put it recently, is essential. But implementing the idea is often more important than the idea itself. As we move our project from idea generation and synthesis towards developing a portfolio of recommendations, we grapple with a number of issues that occur whenever good ideas are implemented. These issues are important, because while a top-down initiative like this can cause change, complementary efforts can often lead to smoother and quicker transitions to improve system function.
First, any implementation of a plan like this absolutely requires stakeholder buy-in. Over the past year, we have heard a prolific trialist say “You call it data sharing, I call it blackmail.” We’ve heard trial sites say that putting data in databases like clinicaltrials.gov is such a pain that often they put in the minimum required information to put in the format, which doesn’t even get to the salient points of the trial. Curt Meinert has released a number of memos that outline the concerns of many people actually doing trials. And this is just at the level of conducting the trial. Even at higher levels, following these rules properly will require trained individuals who can understand and aggregate the data, statisticians throughout the trial development process, and maybe even a change in how companies and universities do trials. The bigger question is how this will be enforced, since currently there are no extra funds available to HHS for enforcement. Rob Califf believes that it won’t be an issue, but given the previous problems we have had with reporting results from trials, and the fact that this new rule essentially forces people to share data using a system they are not happy with, one of the jobs of our team will be to think of complementary solutions that can help move this great idea of sharing information forward.
So what could be an answer to align stakeholders? That brings us to our second point, stakeholders need to know how certain policies can benefit them in the long run, and what will be done with their data in the short run. In general, is there any evidence available that sharing such data from clinical trials will vastly improve the system? Are there quantifiable impacts that can be put forward to show how these initiatives can be better for everyone? And this is where we can start thinking about complementary efforts to help the system. For example, we have developed a machine learning model that predicts success of trials using data on trials from two different databases. Other groups have worked on similar models for more targeted diseases and populations. Efforts like these show the clear advantages of data sharing, give examples to stakeholders on how the data might be used and what potential benefits can be achieved, and turn a top-down policy into a system-wide solution.
Which brings us to our third point. How exactly does HHS and NIH anticipate these actions will change the clinical trials system? In many other systems, a solution like this just doesn’t change one aspect of data sharing, it always has system-wide ramifications. These can be positive, where it encourages all stakeholders to work together and be more open about their findings. Or it could be negative, where we see stakeholders finding ways around the policy to share even less. Even at the micro-level, what ramifications does this have for the patient as they sign to be on a trial, the CRA who is actually conducting the trial, and us as the population who need the drugs? We have been working with the AACT database from clinicaltrials.gov for over a year now, and from the population’s persepctive, the problem isn’t that there are not enough trials on there. The major issue is that it is difficult to navigate, and almost impossible to understand the information.
And that, we feel, is the most important issue here. Drawing an impact line from HHS to the stakeholders in the clinical trials system to the patients. How can we leverage this rule to really help patients? More importantly, will the new equilibrium in the system created by this rule impact the patient negatively? We have to make sure that good ideas and initiatives are not lost. Developing system-wide solutions that can complement progressive efforts by the government is the best way to go.
