AI PoS And ROI An Alphabet Soup Of 21st Century Drug Development

By Remco Jan Geukes Foppen, Vincenzo Gioia and Carlos N. Velez

Part one of this two-part series is available here.

PART 2: Should we spend more and take longer with AI in drug development?

AI-Focused Biotechs: Reducing Operation Costs

Implementing AI in drug discovery offers significant potential for reducing operating costs, though the extent varies widely. A Deloitte report suggests AI could cut drug development costs by up to 70%. Insilico Medicine’s case study with INS018_055, a novel TNIK inhibitor currently in Phase 2 for idiopathic pulmonary fibrosis, achieved development at 1/10 the traditional cost. Cost reductions depend on specific use cases, development stages, and company factors. Moreover, AI implementation requires substantial investments in technology and expertise, impacting overall ROI.

Many current AI successes, like those of Exscientia, rely on technologies developed over a decade ago in academic settings; long-term investment was required to achieve the impressive speed of AI-driven candidate discovery. But the crucial question remains: “Do these gains in speed and cost-efficiency produce candidates more likely to succeed clinically?” The true value of AI in drug discovery lies not just in faster, cheaper processes, but in generating more effective and safer drug candidates. Balancing speed and cost reduction with improved clinical success rates is the key challenge.

What Is The Probability Of Success And Likelihood Of Approval?

“PoS” refers to the probability that a drug candidate will successfully progress from one phase of clinical development to the next, such as moving from Phase 1 to Phase 2. It also can indicate advancement from Phase 3 to the regulatory stage, or from regulatory review to approval. This probability is usually expressed as a percentage. Another related metric is the Likelihood of Approval (LoA), which measures the probability of a drug candidate gaining regulatory approval from any given phase of development. LoA is calculated using PoS values between clinical stages. Both metrics are widely used to assess risk in valuation exercises and are detailed further in other sources.

An early study on PoS and LoA found that drug candidates entering clinical development in 1993 had a 32% LoA for large molecules and 13% for small molecules. Later research developed an algorithm to predict LoA for oncology drugs after Phase 2, based on four factors: activity, patient numbers in Phase 2 studies, Phase 2 duration, and prevalence. A prior study estimated an LoA of 13.4% for oncology candidates in Phase 1, with hematologic indications (36%) having a much higher LoA than solid tumors (9.8%), highlighting the difficulty of treating solid tumors.

In a comprehensive 2014 study, Hay et al. analyzed PoS and LoA across various therapeutic areas using a database of drug candidates in development between 2003 and 2011. They found that the success rate for candidates advancing from Phase 1 to Phase 2 was 67%, while the success rate from Phase 2 to Phase 3 was 39%. The LoA from Phase 1 to Approval varied significantly by therapeutic area, with 17% for infectious disease candidates and only 7% for oncology candidates. Overall, the study estimated an aggregate LoA of 15.3% for all clinical-stage candidates, a figure notably lower than earlier estimates by DiMasi et al.

In its most recent report, IQVIA estimated an overall LoA of 7.6% for 2019-2023, significantly lower than the figure reported by Hay almost a decade ago. The 2023 LoA of 10.8% represents an improvement over the ~6% LoA seen in 2021 and 2022, but remains considerably lower than earlier estimates by DiMasi and Hay. IQVIA also noted that Phase 2 success rates peaked at 52% in 2015, declined to 33%-35% between 2019 and 2021, and slightly rebounded to ~40% in 2022-2023.

Phase 2 success rates are critical as they provide the first meaningful insight into a drug’s clinical efficacy. They allow developers to explore different indications, patient populations, and potential market positions before moving to larger Phase 3 trials. Successful Phase 2 results also can enhance a drug’s attractiveness for licensing, particularly for novel targets and modalities. However, whether improved Phase 2 PoS translates into a higher likelihood of reimbursement, especially outside the U.S., remains a complex issue.

Why Did Phase 2 Success Rates (And Their Corresponding Phase 3 And LoA) Decline In The First Place?

Over the past decade, the industry has increasingly focused on developing riskier drug candidates. “Riskier” refers to factors such as:

• Larger, more complex small molecules
• Targeting “undruggable” or novel, unvalidated drug targets
• Developing more complex macromolecules like antibody-drug conjugates and bispecifics
• Exploring advanced modalities such as cell therapies, CRISPR, and mRNA

It is important to remember that every candidate reaching Phase 2 has demonstrated sufficient safety and efficacy in animal models to gain regulatory approval for human trials. However, despite advances in discovering novel therapies and modalities, the industry’s overall LoA has remained steady at 7-10% for decades. This indicates that while drug development has become more complex and expensive, our ability to advance these therapies through to approval has not improved significantly, reflecting a stagnant LoA across all areas.

This suggests a need for a radically new approach to drug discovery and development to enhance clinical success and approval rates. Can AI tools and technologies offer the breakthrough needed to address this long-standing challenge? AI-Driven Improvements In PoS: The Evidence To Date

The use of AI in drug discovery is still in its early stages. And yet, some techbios suggest that even small increases in PoS across clinical stages can be impactful and that improving PoS, not just speed, is crucial. Even non-AI native Big Pharma companies share this view. For example, GSK aims to increase PoS for oncology assets, using deep learning with integrated multimodality for ADCs and T-cell engagers. Sanofi is also using AI to enhance the quality and speed of small molecule drug discovery.

With AI drugs now in Phase 2 trials, early signs of AI’s impact on improving PoS and LoA are emerging. A recent study analyzed the clinical pipelines of 39 out of 114 AI-native companies and found that AI drugs’ Phase 1 PoS rose from the typical 40-65% to 80-90%, with Phase 2 PoS remaining at the standard 40%. Some drugs in the study were repurposed, which might explain the higher Phase 1 PoS due to existing toxicological and clinical data in a different modality. However, excluding repurposed drugs still shows an increase in PoS. While the sample size of the study is small, the results suggest a positive trend in PoS, that are underlined by the recent positive Phase 2a clinical trial data of Insilico Medicine’s AI drug, ISM001-055, for patients with idiopathic pulmonary fibrosis (IPF).

The Financial Impact Of AI-Driven PoS Improvement

AI is assumed to save time and money in drug discovery and development while producing drug candidates with higher chances of success and approval. This assumption, widely promoted by AI companies, leads to the question: “What is the financial impact of AI on improving the Probability of Success (PoS) and Likelihood of Approval (LoA)?”

The financial impact can be assessed from two points of view:

• For individual drug candidates or programs:

  • Cost Savings: AI reduces drug discovery costs.
  • Faster Discovery: AI accelerates discovery, leading to quicker approval and revenue.
  • Higher PoS and LoA: Improved PoS and LoA increase the chances of approval and revenue.
  • Better Drugs: AI helps develop safer, more effective drugs, enhancing market share and premiums.
  • Drug Repurposing: AI can find new uses for existing drugs, uncovering additional value.

• For portfolios or companies:

  • Fewer Failures: Reduces failures, allowing resources to be focused on promising programs.
  • Riskier Indications: Enables pursuit of riskier or economically unviable indications.
  • Increased Investor Interest: Attracts more investment, including from non-traditional sources.
  • Improved ROI: Enhances overall drug development and investor returns.

The idea of faster and cheaper development suggests AI might help reach clinical trials more quickly with better PoS and LoA. However, choosing to file an IND reflects confidence in AI models and follow-up studies. Investing additional time in AI might slightly improve models, aiding clinical trial success. The next section will address these considerations.

Modeling The Impact Of AI On PoS And LoA

Determining how much incremental spending on AI will yield marginal benefits in PoS is complex and requires specific case studies or examples. Evaluating the financial impact of AI on PoS and LoA involves detailed valuation analyses of the drug candidates under consideration. As each situation is different, exact metrics or rules of thumb, such as “an increase in Phase 2 PoS by X will result in a Y increase in project value,” are elusive. Our estimates suggest that increasing Phase 2 PoS by one or two percentage points could boost project value by 20% or more. However, this should be interpreted with caution. It is also important to note that an increase in Phase 2 PoS does not guarantee a corresponding increase in Phase 3 PoS.

Key questions to address whether AI can improve PoS, LoA, and project value focus on two main considerations about AI modeling robustness and incremental spending impact. For AI modeling robustness, the assumption is that current AI models are robust enough to deliver drug candidates that are safer and/or more effective than existing or in-development products. This assumption applies to both internal research teams and external service providers. While some claim AI-driven discovery can achieve above-average PoS, solid evidence is still developing. However, teams that deliver unique, first-in-class candidates or achieve superior PoS via AI could gain significant competitive advantages. For incremental spending impact, the crucial question is whether additional investment in AI (e.g., for more training data or improved algorithms) will generate candidates that are significantly better and more likely to succeed in later stages. For example, if an AI-driven program costs $1 million annually, would spending an extra $1 million for another year yield sufficient improvements? How would increasing this to $2 million, $5 million, or more impact the outcome and investor response?

If evidence is lacking, other approaches might be considered, such as improved translational models or a stepwise approach to gather data and build confidence in AI’s effectiveness.

Incremental spending also involves time, not just money. While AI providers often emphasize speed and cost-efficiency, the value of faster, cheaper work must be weighed against the potential for higher PoS and LoA. A quicker path to the clinic is less valuable if clinical success probabilities are low. Conversely, spending more time and money to achieve a higher PoS and LoA might be more beneficial.

To address this, one approach is to estimate the incremental PoS increase and its effect on project value. If the investment seems justified by the projected benefits, it may be a prudent choice. For large multinational companies, this decision might be straightforward. However, for early-stage, venture-backed firms, convincing investors to support higher spending and longer timelines is challenging, especially when the prevailing focus is on advancing candidates as cost-effectively and quickly as possible.

Final Thoughts

AI offers powerful tools but is not a cure-all. Its success relies on high-quality input data, advanced algorithms, and, crucially, the expertise and intuition of human researchers guiding its application. While AI is revolutionizing drug development by potentially enhancing the Probability of Success (PoS) and Likelihood of Approval (LoA), we should temper expectations with the understanding that current successes are based on a limited range of use cases. As AI continues to integrate into drug development, it is essential to adopt a cautious approach when adjusting the valuation criteria for pharmaceutical projects. The landscape will become clearer as more Phase 2 results from AI-driven biotech firms emerge. For instance, the recent Exscientia-Recursion merger is expected to deliver early development readouts for ten programs within the next 18 months.

The precise impact of AI on PoS and LoA remains uncertain. However, a 5-10% increase in Phase 2 PoS could lead to substantial project value enhancements, including cost savings, increased revenue, and faster market entry. Significant incremental improvements in PoS can have a large impact when scaled. Yet, the goals of reduced costs and accelerated timelines may not always align with improving clinical success. Cheaper and faster development does not necessarily guarantee better outcomes in the clinic, approval, or market performance. Conversely, there may be cases where investing more time and resources to enhance AI results could be beneficial if it leads to higher PoS and LoA.

Currently, strategic and investment decisions are often subjective and based on unstructured discussions of project risks. The main challenge is overcoming human bias and establishing objective, data-driven processes for portfolio risk assessment and investment. LLMs could help structure these ‘multimodal’ clinical outcome prediction discussions, with feature importance analysis for drug discovery and development, to identify novel features without bias, and iteratively predict ways to enhance PoS.

About The Authors:

Remco Jan Geukes Foppen Ph.D., is an AI and life sciences expert. He is sensitive to the impact of AI on business strategy and decision-making processes. Remco is an international business executive with proven expertise in the pharma industry. He led commercial and business initiatives in image analysis, data management, bioinformatics, clinical trial data analysis using machine learning, and federated learning for a variety of companies. Remco Jan Geukes Foppen has a Ph.D. in biology and holds a master’s degree in chemistry, both at the University of Amsterdam.

Vincenzo Gioia is an AI innovation strategist. He is a business and technology executive, with a 20-year focus on quality and precision for the commercialization of innovative tools. Vincenzo specializes in artificial intelligence applied to image analysis, business intelligence, and excellence. His focus on the human element of technology applications has led to high rates of solution implementation. He holds a master’s degree from University of Salerno in political sciences and marketing. 

Carlos N Velez, Ph.D., MBA, is a pharmaceutical and biotechnology strategic advisor, with 25 years of experience in consulting, venture capital, corporate strategy, and entrepreneurship. Carlos specializes in helping pharmaceutical and biotechnology companies develop in- and out-licensing strategies, with additional expertise and experience in portfolio assessment and prioritization, drug candidate valuation, valuation, and related services. He also develops and presents customized training programs (both live and virtual) for companies seeking to improve their in- and out-licensing processes. He holds a Ph.D. in Pharmacy from the University of North Carolina at Chapel Hill, and an MBA from the Rochester Institute of Technology.