This article was originally published on Law360.

Rapidly evolving regulatory paradigms for AI

From innovative startups developing novel medicines and devices that integrate AI technologies to digital tech firms that are laser-focused on validating AI for the life sciences industry to global drug and device manufacturers whose strategic priorities include digital transformation, these technologies have the potential to unlock solutions to improved health outcomes and earlier access to essential medicines.

As stated by the U.S. Food and Drug Administration's drug center director, Patrizia Cavazzoni, AI tools "are no longer futuristic concepts; they are now part of how we live and work."

As industries grapple with best practices for incorporating into product portfolios these cutting-edge technologies, regulatory authorities are facing novel challenges in establishing standards.

Unlike other industries, FDA has an established regulatory framework for regulatory AI software tools, and is now looking to adapt the existing paradigm to foster innovation and ensure the safe, effective incorporation of AI tools in drug development and manufacturing.

Due to the rapidly evolving nature of these potentially long-term regulatory paradigms, drug and biologic manufacturers and AI tech developers will want to engage strategically with FDA in order to ensure clarity surrounding their compliance obligations associated with drug development and manufacturing processes.

Effective stakeholder interaction with FDA

FDA offers a number of vehicles for stakeholders to engage with the agency, which represent an important opportunity to help with implementing meaningful changes to existing regulatory frameworks. Stakeholders have various opportunities to provide public comments, participate in workshops and engage in private meetings.

Effective engagement with FDA requires deep understanding of FDA's current regulatory paradigms and intra-agency decision-making processes, and full awareness of the company's objectives and broader industry headwinds.

Regarding sponsors' implementation of AI into their products, FDA focuses as much attention on how the AI tool was developed as on the outcomes it produces.

The agency has also made clear that simply demonstrating the AI tool works in representative conditions will not be sufficient to address its concerns; sponsors must also explain why the development method is scientifically valid and would be expected to function as intended so the agency can ensure the robustness of the tool.

Recent FDA discussion papers highlight profound potential of AI

In recent months, FDA has been raising its level of engagement with stakeholders on evolving regulatory paradigms for AI in drug development and manufacturing, publishing two discussion papers on "Using Artificial Intelligence and Machine Learning in the Development of Drug and Biological Products," and on "Artificial Intelligence in Drug Manufacturing."

The latter paper comes from FDA's Center for Drug Evaluation and Research (CDER) as part of its Framework for Regulatory Advanced Manufacturing Evaluation initiative.

While FDA discussion papers are expressly for discussion purposes only and are not draft or final guidances, they nevertheless provide insights into initiatives under consideration. Equally important is the opportunity for stakeholders to provide inputs as FDA develops its regulatory frameworks.

AI in drug development

A collaboration among CDER, the Center for Biologics Evaluation and Research, and the Center for Devices and Radiological Health, including its Digital Health Center of Excellence, the AI in drug development discussion paper is intended to facilitate a discussion around AI in drug development, including the development of medical devices to be used with drugs.

In the discussion paper, FDA promotes using AI in drug development for:

• Drug discovery, including target identification, selection and prioritization, as well as for compound screening and design;

• Nonclinical research, including pharmacokinetic, pharmacodynamic and toxicologic studies conducted in animals; exploratory in vitro and in vivo mechanistic studies conducted in animal models; organ-on-chip and multi-organ chip systems; and cell assay platforms;

• Clinical research, including patient recruitment; selection and stratification of trial participants; dose and dosing regimen optimization; monitoring and improving adherence to trial design; participant retention; clinical trial site selection; and clinical trial data collection, management and analysis;

• Postmarketing safety surveillance, including case processing, evaluation and submission; and,

• Advanced pharmaceutical manufacturing, including process design optimization, advanced process control, smart monitoring and maintenance, and trend monitoring.

AI in drug manufacturing

Meanwhile, FDA is also establishing its regulatory paradigms for the application of its risk-based regulatory framework to the use of AI technologies in drug manufacturing.

In its AI in drug manufacturing discussion paper, CDER requested additional feedback related to the types of AI-based models used, the elements of AI technologies in a current good manufacturing practices environment, practices for validating AI models and appropriate data management, among others.

Recognizing that there are limited industry standards, CDER highlighted the following areas of concern relating to AI use in manufacturing:

• Cloud applications may affect oversight of pharmaceutical manufacturing data and records, which may lead to challenges during inspections in ensuring that third-party AI software is updated with appropriate safeguards for data safety and security.

• The internet of things may increase the amount of data generated during pharmaceutical manufacturing, affecting existing data management practices. This means the industry may need additional clarity regarding regulatory compliance for generated data, as well as clarity regarding "data sampling rates, data compression, or other data management approaches to ensure that an accurate record of the drug manufacturing process is maintained," according to FDA.

• Whether and how the application of AI in pharmaceutical manufacturing is subject to regulatory oversight may require clarity.

• Standards for developing and validating AI models used for process control and to support release testing may require additional clarification from FDA.

• Continuously learning AI systems that adapt to real-time data may challenge regulatory assessment and oversight, due to issues with determining when an AI model can be considered an established condition of a process, as well as challenges associating with ascertaining the criteria for regulatory notification of changes to the model as a part of model maintenance over the product lifecycle.

In response to the AI in drug manufacturing discussion paper, CDER received numerous and wide-ranging comments from stakeholders.

For example, stakeholders detailed current and intended uses of AI technologies including chemistry, manufacturing and controls development and scale-up, advanced process control to allow dynamic control of the manufacturing process, autonomous systems for drug manufacturing, production planning, asset management, quality assurance, stability and shelf-life monitoring, document management and supply chain optimization.

Also capitalizing on this feedback, in the more recent AI in drug development discussion paper, FDA provides details on how AI applications have been deployed or have potential to support current good manufacturing practice requirements, and identifies four areas where AI could be applied during the drug manufacturing lifecycle, from design to commercial production:

• Optimization of process design, e.g., digital twins approach;

• Implementation of advanced process control;

• Smart monitoring and maintenance; and

• Trending activities to analyze manufacturing-related deviation trends, cluster problem areas and prioritize areas for proactive continual improvement.

Outstanding key considerations

A key takeaway from FDA's discussion papers and recent activities associated with AI is that the agency recognizes the profound potential benefits to advance drug development, manufacturing and compliance. The agency has also expressed an openness to adapting the current regulatory paradigms to make room for risk-based approaches.

FDA, however, has some concerns such as the transparency of AI models and with ensuring the security and integrity of data generated from continuous learning approaches.

Accordingly, the agency is eager to receive stakeholder input to understand AI's omnipresence in the life science industry to assess the potential risks and benefits.

FDA acknowledges the need to assess whether the use of AI in the context of use introduces increased or unique risks, such as limited explainability due to a system's underlying complexity or lack of full transparency for proprietary reasons, and the potential to amplify errors and preexisting biases.

To this end, FDA expressly solicits feedback on three key areas in the context of AI in drug development:

• Human-led governance, accountability and transparency;
• Quality, reliability and representativeness of data; and,
• Model development, performance, monitoring and validation.

To promote human-led governance, FDA urges the creation of a risk management plan that considers the context of use, in order to identify and mitigate risks associated with AI in drug development.

Regarding data quality, reliability and representativeness issues, the discussion paper spotlights how AI is particularly sensitive to the attributes or characteristics of the data used for training, testing and validation.

Lastly, the AI in drug development discussion paper emphasizes the importance of practices of prespecification steps and of clear documentation of criteria for developing and assessing models.

Taken together, FDA's spate of recent actions in this arena demonstrate the agency's commitment to supporting innovation by learning from the industry to inform how AI should be regulated, as digital health tools are currently developing faster than current regulations.

Comments on the AI in drug development discussion paper are open through Aug. 9 under docket FDA-2023-N-0743.

Stakeholders can also continue to engage with FDA on AI-related issues via several upcoming virtual workshops and other initiatives. These rapid developments in AI technologies require a careful balance between industry and government regulators.

Authored by Lowell Zeta and Blake Wilson

This article was originally published on Law360.