Week Note 2015 - 23

Week of: 

September 21, 2015

So what’s the answer?

I’ve often seen that if you want to learn about the problems that people face, ask them for their recommendations. The converse also works, if you ask for potential solutions, people start describing their problems. We’ve employed both questions from a myriad of stakeholders as described in all our weeknotes. We’ve been hearing a number of issues again and again, but the one I am thinking about most this week is enrollment.


“We need more clinical trials.” “There need to be bigger and better clinical trials.” “Trials are going abroad because it is difficult to enroll patients in the United States.” “We need an ethical way to engage more research subjects and patients.” “We need to support new clinical trials such as n = 1 and organ-on-a-chip where enrollment is not a big factor to be able to minimize costs.” Those are the types of answers we have heard from almost everyone we have talked to. It’s not just the experts, each of our team members has been circling the issue of enrollment for quite a while. From the international team who see it as a trend of trials shifting abroad to the risk team who weigh the benefits and downfalls of the requirement of high enrollment numbers, our team has been tackling this issue from a number of angles. To get an idea of how important an issue of enrollment is, the following graphic shows the distribution for enrollment numbers in clinical trials.

Figure 1: Enrollment Numbers for Clinical Trials from the CT.GOV Database


It’s funny how we see bumps at whole number enrollments (100, 200, 300, 400… 1500), which shows that enrollment number for a clinical trial is an issue that is determined by more than just math. And, in fact, it influences multiple factors and gives rise to several inefficiencies of the system. Trial cost, and time to completion are directly related to enrollment. But other issues our research group has found are not far behind. The success of CRRI that we visited over the summer was directly related to them being able to recruit relevant research subjects quickly. The issues with international trials revolve around the ethics associated with international research subjects, but also the ease with which enrollment happens abroad. Here’s a graphic showing how international trials tend to be completed quicker than US trials, even though enrollment numbers for international trials are slightly higher.

Figure 2: Time to Completion for Clinical Trials in the US and Abroad

Our data team used a machine learning algorithm to try and see what the most important factors are that influence a trial being terminated. With ten factors, we can predict with 75% accuracy if a trial will be terminated or not given our model. The most important factor that determines if a trial will be terminated? You guessed it: enrollment. Enrollment is directly relevant to our team members interested in personalized medicine, n = 1 trials, innovations that reduce requirements for research subjects, and patient-centered trial design.

So what’s the answer? “We need to tell everyone that clinical trials are the best way to get care, and they need to enroll in as many as possible to help society.” “Find a way to get research subjects in trials and keep them.” “Society needs to realize the importance of clinical research.” Again, we’ve heard these ideas several times, but they don’t really answer the question about enrollment. They say the answer to bad enrollment is more enrollment. Of course, we can come up with ideas on how to increase enrollment, but solutions like this for society rarely work. So what would I do if I could control the clinical trial system?

*The following reflect only the opinions of the author, and not necessarily the MIT Collaborative clinical trials team

Let’s divide the histogram above into the phases of clinical trials. Here are Phase 1,2, and 3 next to each other:

Figure 3: Enrollment Distribution for Phase 1 Trials


Figure 4: Enrollment Distribution for Phase 2 Trials



Figure 5: Enrollment Distribution for Phase 3 Trials


Phase 1 tests the safety of the drug. Phase 2 tests efficacy and safety. By the time a trial passes Phase 2, it is supposed to be safe and have some efficacy. Phase 3 involves safety, efficacy, and effectiveness and is by far the biggest driver of high enrollment numbers.


But our team knows that Phase 1,2,3 don’t happen one after the other. There are many reasons for this, but again, enrollment is a big issue. It’s smart for companies to begin enrollment for Phase 3 early, as those trials are the most expensive and take the longest time to complete. Not only that, but many Phase 3 trials continue well after the drug has been approved. So really, Phase 3 trials are where the largest issue with enrollment takes place, mostly because they seem to be the most important part of the process.


The first part of my proposal is to isolate Phase 3 trials so that they cannot be done before Phase 1 and 2 are complete and approved by FDA. This will ensure 1) the drugs that have been through Phase 1 and 2 are at least safe and effective by FDA standards so can be consumed by anyone, 2) Expensive Phase 3 trials do not get started and time is not spent designing them if not needed, and 3) it allows a better filter in a system clogged with clinical trials and long regulatory time.


Note that this will result in larger enrollment numbers for Phase 1 and 2, as I am sure the FDA will need to be more stringent for approving Phase 1 and 2 drugs for being safe and effective without additional data from Phase 3. This is fine, this is the entire purpose of Phase 1 and 2. They have been designed to test safety and efficacy before the entire population can consume the drug, and this is what I believe will make the system more efficient.


The second part of my proposal could be more controversial. My last post about medicine being an art and a science hinted at how each patient is different and the drug that might work “on average” wouldn’t work for an individual. Similarly, a drug that might not work “on average” could save someone’s life. I believe to improve the efficiency of the system, Phase 1 and 2 approved drugs need to be made available to patients immediately so that their physician can prescribe them. Thus, Phase 3 of a trial will turn into something like a Phase 4 trial, which tests post-market drugs. What does the histogram for Phase 4 look like?


Figure 6: Enrollment Distribution for Phase 4 Trials


This already looks more like Phase 3 trials than Phase 1 or 2 trials. Phase 4 trials are conducted primarily by physicians for their patients. Enrollment in these trials is not a big issue, because they don’t need to be held primarily at a research site. Any drugs in Phase 4 trials have already been approved for use by the general public. They further test safety, efficacy, and effectiveness like Phase 3, but for more long term. Of course, like people, all drugs are different. So medical experts will need to be careful about which drugs in Phase 3 can be made freely available vs which ones can’t. But I will repeat this here: if Phase 1 and Phase 2 do what they are designed to do, there should be no reason why these drugs cannot be consumed by the general public.


The other advantage is that we can then set a price cap for drugs in Phase 3, while still allowing companies to sell the drug for a profit. In economic terms, a drug in Phase 3 should be priced at or below standard of care, even if an official cap is not set. This allows companies to make profits earlier, patients to get access to drugs cheaply, and also encourages patients to sign up for a Phase 3 trial thus increasing enrollment.


The missing data from my analysis is the number of adverse events for a drug in Phase 3 that has been through Phase 1 and 2 without an adverse event.  This data can further enlighten the ways in which the proposal can take force. If, in fact, there are not that many adverse events in Phase 3, then Phase 1 and 2 are working, and the proposal is implementable. If not, then it shows we need to improve Phase 1 and 2 before proceeding with any such idea.


Another issue will be that doctors normally have a lot of evidence from Phase 3 trials when they decide to prescribe a particular drug. They will now have to do this without that evidence, thus making clinical practice more difficult. But I see this as a positive, as it allows doctors to be more flexible in having different therapies for different patients. In this age of big data, the results accumulated from these new Phase 3 trials can be made quickly available for doctors to get the evidence as the trial is ongoing. It also brings the doctors close to research, and allows easier participation of academics within the clinical trial system. It can also lead to more translational research as opposed to just about the drugs, as now more individuals have access to new therapies and data.


If this became reality without adverse events to patients, then we would have reduced the time for drugs to get approved, reduced the overall cost and waste in the system, satisfied every stakeholder (quicker approval to market, less regulatory time, more patient agency), and also not have a big bill on our hands.


I have to admit, this isn’t my idea. The idea of revisiting the regulation around efficacy and effectiveness came from Debashish Roychowdhury at our first round table last year in Cambridge. The idea lingered in my head for a while, and I think there is enough evidence out there to support it.


Of course, my recommendation may not be an idea that’s easily implementable. That’s the sad part, very little research is done on implementation out there. And as Justin likes to remind me, policy recommendations are not great recommendations. But we’ll keep working on from here, and any ideas on how to tackle these issues will be greatly appreciated!


Sauleh Siddiqui

Week Notes Authors

Johns Hopkins University Co-Research Lead

Co-Research Lead

Jen Bernstein has 15 years experience working on WHO-, industry-, U.S. government-funded research studies and clinical trials.  She received her Master of Public Health at the Johns Hopkins Bloomberg School of Public Health and currently works as Part-Time Faculty in the Johns Hopkins Whiting School of Engineering. Jen is also a clinical research associate and independent research consultant providing research support for deliverables at all stages of project development including IRB submissions, journal articles, presentations, and grant submissions. Jen honed her skills at a variety of institutions including The Urban Institute, the Johns Hopkins Center for Communication Programs, and The Weinberg Center for Women's Health and Medicine at Mercy Medical Center. 

MIT Collaborative Initiatives

Ellie Carlough is currently Associate Director of MIT Collaborative Initiatives (MIT-CI) and has been with the organization since September 2007.  In this role, she is actively involved in project management, writing and strategic planning for MIT-CI, which applies a system-based approach to addressing widespread societal issues reaching a crisis point.

Ellie graduated from Mount Holyoke College and earned a Masters of Science in Non-profit Management from the New School for Social Research in New York, New York.  After working in Manhattan for 10 years in banking and most recently as Director of Corporate and Foundation Relations at St. Vincent’s Hospital and Medical Center of New York, Ms. Carlough spent a time in the challenging position of full-time mother mastering the art of multi-tasking and social networking before joining MIT-CI.

SITRA, MIT Collaborative Initiatives

Based in Boston, Justin is the Finnish Innovation Fund's (Sitra) academic liaison for North America and serves as Sitra's Senior Lead for Sustainable Development. He helped design and build the Low2No urban decarbonization initiative and manage its implementation, and served as Helsinki Design Lab's expert on energy, carbon and sustainability. 

Currently he is helping Sitra better align its endowment with its mission by developing ways to place capital in businesses, projects and funds that generate measurable social and environmental impact together with financial return. As part of Sitra’s Strategic Research team, he works to support and challenge Sitra's strategy and shared assumptions. His research includes developing a more comprehensive understanding of sustainable well-being and how Nordic countries can transition toward it with practical economic and public policy measures.

Justin is also an Adjunct Professor at the Rhode Island School of Design where he teaches graduate level design courses. He holds a Master of Architecture degree from the Harvard Graduate School of Design and a Bachelor of Arts from the University of Washington. Find him @justinwcook.

Johns Hopkins University

Viva Dadwal is a Visiting Scholar at Johns Hopkins University Bloomberg School of Public Health, where she is investigating the transfer of health innovations from low to high-income countries. She is a published academic, a Senior Fellow at the Centre on Governance at the University of Ottawa and a Deputy Editor of open-access global health journal, Globalization and Health. Previously, she worked as a Trade Policy Officer for Foreign Affairs, Trade and Development Canada. Viva has held prestigious internships at the World Health Organization and Permanent Mission of Canada to the United Nations and to the World Trade Organization in Geneva, Switzerland.

In her spare time, Viva promotes public astronomy in urban cities through a project called #popscope. She also guest blogs for World Bank’s youth blog, Youthink! Viva holds a B.Sc. (Hons) in Biology from the University of Windsor and an M.A. in Public and International Affairs from the University of Ottawa. Viva is a recipient of Fulbright Canada Award 2014-2015 and a member of the World Economic Forum Global Shapers community. Follow Viva's adventures on Twitter @vivadadwal.

Johns Hopkins University

Francisco Del Canto Viterale is Postdoctoral Researcher in the Systems Institute within the Whiting School of Engineering at Johns Hopkins University. Currently, he is working on two different research projects: “Complex International Innovation Partnerships”, and “Clinical Trials Systems”.

His core area of expertise is the intersection of Science Technology Innovation and Higher Education, and International Relations.

Francisco has been working in the academia for over seventeen years in several countries. He has an extensive experience working as a university lecturer in International Studies, International Migration and Latin American Studies; and also as a researcher in topics related to International Scientific Relations, Science and Innovation Policy and Interdisciplinary, Complex and Systems Approach.

He received his Ph.D. in International and Intercultural Studies from the University of Deusto (Spain) in January 2014. Prior to his doctorate, he earned a Bachelor of Science in International Relations (1998), a Diplomat in International Affairs and Diplomacy (1999) in Argentina, and a Masters in International Migrations and Social Cohesion (2011) in Spain.

Johns Hopkins University

Felipe Feijoo is a Postdoctoral researcher at the Systems Institute in the Whiting School of Engineering, The Johns Hopkins University.

He obtained his B.S. in Engineering Sciences at the Universidad de La Frontera, Chile. Thereafter, he pursued his Master degree (2011) and Ph.D. (2015) in Industrial Engineering at the University of South Florida, Tampa.  

His current research involves the application of stochastic complementarity models and economics to analyze the opening of clinical trials and the pharmaceutical market in the U.S. 

His research also focuses on the application of mathematical programming, game theory, and statistics for decision making in sustainable electricity markets. He worked on a mathematical-statistical framework to developed Pareto designs of cap-and-trade carbon policies with high penetration of green energy via micro-grids. Other research interests include health-economics and predicting-forecast modeling using machine learning techniques. Felipe has also worked and led several projects for the Tampa Electric Company (TECO), Florida.

Johns Hopkins University

Senior Advisor

Dr. Tak Igusa is a professor at the Johns Hopkins University who has been engaged in multi-disciplinary collaborations spanning a wide range of fields.  He currently holds appointments in the Departments of Civil Engineering, International Health, Earth and Planetary Sciences, and Applied Mathematics & Statistics.  To provide rigor to his collaborative activities, Dr. Igusa has been promoting systems science and methodologies, which has led to significant, new research programs within the university.  This includes projects funded by the NIH, Robert Wood Johnson Foundation, CDC and the National Science Foundation on systemic studies of childhood obesity, rheumatic disease, community resilience to natural disasters, and technology innovation in developing nations.  His work led to the establishment of the Johns Hopkins Systems Institute in 2011.

Dr. Igusa received an A.B. in Applied Mathematics from Harvard University and a PhD in Civil Engineering from University of California, Berkeley. At Johns Hopkins, he is currently the Director for Health of the Systems Institute and the Program Lead of the Education & Training Program at the Global Obesity Prevention Center.  He is also advising the development of systems research programs that bridge public health and engineering at the University of the Witwatersrand in Johannesburg, South Africa. 


Johns Hopkins University

Gary Lin is currently working on complex systems research with the Johns Hopkins Systems Institute and pursuing a Doctor of Philosophy in Civil Engineering at the Johns Hopkins University.  He has a variety of research interests which includes healthcare systems, infrastructure systems, supply chain and transportation networks, and human population dynamics.  Using his diverse background, Gary has been involved in numerous interdisciplinary projects that range from natural gas networks to environmental impacts of population growth.  He hopes to apply the systems approach towards large-scale, health and engineering issues that face present and future society.

Gary obtained his Bachelor of Science in Civil Engineering and Bachelor of Arts in Economics from the University of Colorado at Boulder.  He is also a registered Engineer-in-Training (EIT) in the state of Colorado and student member of the American Society of Civil Engineers.  In addition, Gary has experience working in food transportation logistics, financial administration, and construction marketing.  Gary enjoys disc golfing, cycling, and hiking in his spare time.

Johns Hopkins University

Paul Locke, an environmental health scientist and attorney, is an Associate Professor at the Johns Hopkins Bloomberg School of Public Health in the Department of Environmental Health Sciences.  He holds a joint appointment in the Department of Health Policy and Management.  Dr. Locke is also the Distinguished Visiting Professor of Animal Law and Science at the Northwestern School of Law of Lewis & Clark College.   Dr. Locke has an MPH from Yale University School of Medicine, a DrPH from the Johns Hopkins University Bloomberg School of Public Health and a JD degree from Vanderbilt University School of Law. 

At Hopkins, Dr. Locke leads an integrated public health research, practice and teaching program.  His program focuses on research-to-practice initiatives to better protect public health, especially in advancing evidence-based in vitro toxicology and radiation protection policy at federal and international organizations.  He has published widely in both law reviews and scientific and policy journals, and has developed three cross-disciplinary courses in environmental law and policy and animal law.  Dr. Locke also directs the School’s Doctor of Public Health program in Environmental Health Sciences and a certificate program in Humane Sciences and Toxicology Policy.

As a member of the clinical trials initiative team, Dr. Locke is helping to better understand the laws, regulations and policies that frame clinical trials and human subjects research, and seeking opportunities to utilize rapidly evolving in vitro toxicological methodologies and other new techniques in clinical trial decision-making.      

Dr. Locke is admitted to practice law in the state of New York and the District of Columbia, the Southern District Court of New York and is a member of the bar of the United States Supreme Court.

Muchieast, Rhode Island School of Design

Systems Innovation Design Lead

Enrique is the systems innovation and design lead for the project. He is the founder and director of Muchieast LLC, an international consultancy working at the intersection of social, cultural and urban contemporary practices by incorporating the principles of systems innovation and creative thinking. He is a Visiting Lecturer at Brown University and a Senior Critic at the Rhode Island School of Design. Enrique holds a Master in Architecture from the Universidad Politécnica de Madrid, Escuela Técnica Superior de Arquitectura de Madrid; and a Master in Industrial Design, with Honors, from the Rhode Island School of Design, where he was granted the Norman Bel Geddes award in 1998.

Johns Hopkins University, Bloomberg School of Public Health

Michele is currently a Senior Research Assistant at Johns Hopkins Bloomberg School of Public Health in the Department of Environmental Health Sciences. Her interests include risk management and communication, effective translation of science into policy, and more recently, using a systems approach to fix ineffective or outdated policy. Since obtaining a Master’s in Health Science and a Certificate in Risk Science and Public Policy in 2012, she has been working on communications with congressional offices on Capitol Hill, conducting legislative analysis, producing advocacy materials and organizing consultation meetings with congressional offices.

Johns Hopkins University Co-Research Lead

Co-Research Lead

Dr. Sauleh Siddiqui is an Assistant Professor of Civil Engineering with an appointment in AppliedMathematics & Statistics at Johns Hopkins University and is affiliated with the Johns Hopkins Systems Institute. He is also a Visiting Researcher and Lecturer at the Technical University of Berlin and the German Economic Research Institute (DIW Berlin). His research is on formulating and solving optimization and game theory models applicable to large-scale systems. Such systems arise when modeling problems in energy and environmental markets, public health, and transportation. He also models engineering design and develops novel algorithms along with supporting mathematical theory.

He has received research funding from the World Bank, Johns Hopkins Health System, the Environment Energy Sustainability Health Institute, and the Norwegian Research Council. In addition, he has collaborated on projects with the International Council on Clean Transportation, Institute of Medicine, National Academy of Engineering, and Bikemore. These projects have included modeling energy and climate markets, patient flow and staffing in hospitals, global vaccination strategies, and urban transport networks.       

He received an A.B. in Mathematics and Public Policy from Franklin & Marshall College and a Ph.D. in Applied Mathematics & Statistics, and Scientific Computation from the University of Maryland, College Park.