What does draft Annex 22 allow an AI assistant to do in GMP software?

It allows an assistant wherever its output has no direct impact on patient safety, product quality or data integrity, provided a qualified person answers for that output. The draft Annex 22 on artificial intelligence states that generative AI and large language models “should not be used in critical GMP applications”, and the assistants built into business software usually run on one. The dividing line therefore runs through the use. An assistant that answers an operator’s question about a procedure is a non-critical use, because the operator still works from the controlled procedure and the answer never becomes a record. A function that reads a manufacturer’s certificate and fills in the conformity section of your own certificate of analysis sits on the other side, because its output lands inside a record that a release decision rests on. We have built both into a platform for a pharmaceutical supply chain, and this is where we draw the line.

It covers AI models used in critical applications in the manufacture of medicinal products and active substances. The scope names those applications, the ones “with direct impact on patient safety, product quality or data integrity, e.g. to predict or classify data”, and says the document “provides additional guidance to Annex 11”. Because active substances are named, a company that imports and processes plant material for pharmaceutical use sits inside the same perimeter that decides where GMP begins for a herbal starting material.

The scope then filters in sequence: models trained on data rather than programmed, static, meaning frozen once trained, with a deterministic output, used in a critical application. Models that keep learning in use, or that can answer the same input differently, “should not be used in critical GMP applications”, and a generative model fails the deterministic filter by construction.

It is still a draft. The Commission consultation closed on 7 October 2025, the EudraLex Volume 4 index still lists Annex 11 at its January 2011 revision and contains no Annex 22, and the work plan of the EMA GMP and GDP inspectors targets the fourth quarter of 2026 for handing the Commission a final text. No clause binds anyone yet, and the text still shows how its authors reason about AI inside a GMP system.

Because its answer does not change a record, approve a step or classify material, and the operator still acts on the controlled procedure. For that case the draft sets one condition: in non-critical GMP applications, “personnel with adequate qualification and training should always be responsible for ensuring that the outputs from such models are suitable for the intended use, i.e. a human-in-the-loop (HITL)”.

The assistant we built answers operators’ questions about areas and procedures. Its non-critical status holds only while three conditions stay true, and the builder is the one most likely to wear them down release by release:

• it points to the controlled procedure it draws on, so the operator checks the source;
• it cannot write into a lot, a specification, a certificate or any other record;
• it cannot approve, sign or release anything.

Give it any of those capabilities and the use has changed. For non-critical uses the draft says its principles may be considered where applicable, and clause 3.3 fits. It carries a condition that summaries of it tend to drop: where a model gives an input to a decision made by a human operator “and where the effort to test such model has been diminished”, the description of the intended use “should include the responsibility of the operator”, whose training and consistent performance “should be monitored like any other manual process”. Write both down before go-live.

When what the model extracts goes into a GMP record, such as the conformity section of a certificate of analysis, because that section feeds the decision to release a batch. Judging that an extracted value meets a specification is the kind of classification the scope cites as its example, with direct impact on product quality and data integrity, so a generative model doing that job falls under the exclusion.

The tempting fix is a person who checks the result. The draft does not say that a human check turns a critical use into a non-critical one: it offers the human in the loop to generative models only where the use is non-critical from the start. That leaves two honest designs. The first keeps the model outside the record, so it shows where a value sits in the manufacturer’s document and a named person enters and signs it. The second removes generative AI from the step and uses extraction that returns the same result every time, validated like any other function that writes GMP data. Until a system that holds GMP records has been validated, the signed document stays the record of reference.

Not yet, because no new text exists. The EMA held a multistakeholder workshop on Annex 22 on 30 June and 1 July 2026, after the consultation showed support for enabling generative AI and large language models in manufacturing, to discuss guardrails within a risk-based approach. The agency expects a report with expert input, and on 13 September 2026 the event page carried none. The position is under reconsideration and, on paper, unchanged.

A risk-based approach for generative models cannot work without answers to the questions asked here: what the output feeds, who answers for it, and whether that was written down before go-live. Answer them now and the evidence exists whichever way the text moves, within the wider revision we described in where EU GMP is heading.

Written by

Fabrizio Longato

Quality consultant and AI specialist, co-founder of Clear S.r.l. Società Benefit and managing partner of Clear Italia

LinkedIn