Implementing AI/ML in GMP: Applying ICH Q9(R1) to GenAI
One of the strongest messages from the EMA Multistakeholder Workshop on Annex 22 was that ICH Q9(R1) should provide the foundation for implementing AI/ML-supported solutions in GMP.
A key takeaway from several excellent industry presentations was the concept of formality in Quality Risk Management. According to ICH Q9(R1), the level of formality may (yes, Q9 says “may”) be determined by considering three factors:
ICH Q9 (R1): “When determining how much formality to apply to a given quality risk management activity, certain factors may be considered. These may include, for example, the following”: Uncertainty, Importance and Complexity
Uncertainty – the lack of knowledge about hazards, harms, and their associated risks.
Importance – the significance of the risk-based decision influenced by the AI/ML system.
Complexity – the complexity of both the AI/ML subsystem and the GMP process it supports.
These three factors provide a practical framework for determining the appropriate validation effort and, ultimately, whether a particular AI/ML model is suitable for a specific GMP application. In practice, this assessment should be performed as early as possible during the concept phase and documented as part of a High-Level Risk Assessment (HLRA).
What does this mean for AI?
Every new technology introduces uncertainty, and AI is no exception. Some models, particularly probabilistic models, introduce an additional intrinsic level of uncertainty because they may produce different outputs for the same input. This is a fundamental characteristic of transformer-based Large Language Models (LLMs).
When such models support GMP decisions, the importance of the decision is often fixed. However, the model's influence can be reduced by limiting its autonomy, through Human-in-the-Loop (HITL) oversight, an approach reflected in the current Annex 22 draft for lower-risk GMP applications.
The complexity of any implementation is also driven not only by the AI model itself but by the overall business process in which it operates.
How can we reduce risk, particularly for probabilistic models such as LLMs?
While Human-in-the-Loop is currently the primary regulatory expectation, I believe technical controls often provide a more objective and scalable solution. These controls are commonly referred to as guardrails. They can significantly improve model performance by increasing accuracy, providing provenance, enforcing output constraints, and reducing uncertainty.
As in other areas of Data Governance and Data Integrity, technical controls are generally preferable to procedural controls whenever feasible.
In the next two posts, I'll explore Human-in-the-Loop and Guardrails in more detail and discuss how they can help build compliant and trustworthy AI solutions in GMP.
Stay tuned and sign up for more content here.
