QFINITY · Updates
The latest from QFINITY and the field.
An overview of updates beyond events and projects: articles and publications, work in industry bodies and committees, awards, and our perspective on new regulations.
At a glance
An overview of updates beyond events and projects: articles and publications, work in industry bodies and committees, awards, and our perspective on new regulations.
Who Pushes Back When the System Speaks?
A follow-up to the 47th GAMP D-A-CH Forum, Berlin, 11 March 2026.
At the 47th GAMP D-A-CH Forum in Berlin in March 2026, Daniel Köpke and Oliver Herrmann asked what Human Oversight means when systems become more intelligent, more complex and more convincing. For us, the answer starts with responsibility: the greatest danger to QA is the silent erosion of visible responsibility, until no one pushes back anymore. This summer, a security incident at the AI provider OpenAI showed how real this question has become. It is the occasion to share the March position now: not foresight, an illustration. This article lays out the position from the talk, and it names what organizations can build to counter it.
Why is this follow-up coming now?
On 26 August 2026, OpenAI published the technical report on an incident from July. In one of the company’s own security tests, agents had broken out of their test environment and compromised real Hugging Face infrastructure. A key factor was that the usual safeguards had been switched off in the test environment for testing purposes. In production, by contrast, they would be in place. Even so, the incident remains a vivid experiment that shows how far the capabilities of these systems now reach. Whether the agents broke out or were let out, both are a failure of architecture, not of a single control. Controls bolted onto a system after the fact cannot keep pace with these systems. Controllability must be designed in before the system speaks. The incident happened far outside the GxP world. The important message here: these AI capabilities are increasingly being used in GxP system landscapes as well. Anyone assessing the associated risks needs to understand both the limits and the potential of these systems.
The starting point is responsibility
Every human being has a face and a voice. Both make us recognizable, and both make us responsible: as QA professionals and as people who contribute to patient safety. A patient does not know our systems. They know no SOPs, no validation plans, no model architecture. They trust that in the end a human stands behind quality.
The connection to AI lies in the transition from data to decisions. An AI-enabled system can analyze data, detect patterns, generate test cases, classify deviations and prepare decisions. And as it does, it sounds plausible, often even very plausible.
Everything documented, everything compliant, and no one understands why
In Berlin we opened with a scene. One system recommends release. A second has checked the data. A third has classified the anomaly as acceptable. Everything documented, everything traceable, everything compliant. And no human has really understood why.
This scene is not the future; it is pieced together from today’s everyday practice. A system generates 847 test cases, and the tester checks the output. But who checks the logic behind them, and who decides what was not tested? A chatbot classifies a deviation as minor and proposes the CAPA along with it. The QA professional confirms, and at some point confirming becomes a habit. An autonomous monitoring system reports no trend. But what about the trend outside the pattern the model knows? Silence is not a statement. Silence is an assumption.
Plausibility is not evidence of review
That is the central risk of any Human-in-the-Loop setup. Plausible outputs discourage the thorough review that technical correctness, regulatory robustness and GxP responsibility actually demand. Plausibility can trigger a review. It cannot replace one, and it does not make responsibility transferable. Plausible means: it could be right. Right means: we have checked and understood it. That is not a small difference, it is the difference.
A system bears no regulatory responsibility. It does not know the GxP context the way a human does, it does not recognize when a seemingly correct result becomes dangerous in a critical process, and it does not judge under ambiguity. All of that stays with people. Technical control mechanisms, often called guardrails in the debate, can support people in exercising this responsibility. Anyone who trusts these measures blindly, however, perpetuates the very design weakness that makes the Human-in-the-Loop setup vulnerable.
The silent erosion of visible responsibility
The danger comes quietly. Click by click, confirmation by confirmation, buried under ever more decisions waiting to be taken, until no one really pushes back anymore. No single step stands out, and what remains is a QA function that has formally documented everything and in practice no longer decides anything.
When the system speaks, the decisive question remains: who pushes back?
Three sets of rules, one direction
In Berlin we mirrored this question against sets of rules that emerged independently of one another and carry the same expectation. In Article 14, the EU AI Act describes what Human Oversight means for high-risk systems. People must be able to understand, monitor and correct the system, not merely have signed off formally. EU GMP Annex 11 has always demanded controllable, traceable and inspectable computerized systems. It was written before generative AI, and it applies regardless. The draft EU GMP Annex 22 proposes the first formal framework for AI in the GxP environment, with intended use, performance monitoring and change control. In critical applications it currently envisages only static models with deterministic output. The regulated user must hold and review the evidence itself. That applies regardless of whether the model was developed in-house or created with a service provider. On the industry side, GAMP 5 describes the consensus on how these expectations can be implemented. All point in the same direction. Control stays with people.
The role of QA is shifting
A QA function that wants to answer this question shapes the conditions under which human judgment remains effective. In GxP terms, Human Oversight belongs in the intended use, in the business process and in the risk-based controls, with lifecycle evidence that demonstrates its effectiveness. Human Oversight is a capability that is designed into the architecture of the system, not added later in a review step.
In Berlin we described this role in four images. As architecture designer, QA sits at the table when system boundaries are drawn and helps decide which decisions a system may take autonomously and where a human must be able to intervene. As oversight architect, it defines the control points itself instead of leaving them to IT or the vendor: where monitoring takes place, what counts as a deviation, when a system is paused. As escalation designer, it devises the detection paths for silent failures, because drift is not a crash; it creeps. And as guardian of transparency, it records that a validation was incomplete if no one in an audit can explain why the system design was chosen, why human oversight was defined as it was and why the decision in operation was made as it was.
The sentence the talk was building toward still stands. Future-proof QA does more than validate systems. It validates that people remain capable of deciding.
And it needs people who can review. Four conditions determine whether pushback happens in daily work:
Organizations must design Human Oversight so that responsibility is exercised in daily work and its effectiveness remains demonstrable. And that assignment of responsibility belongs on record. It is not the system that decided. A human took responsibility for the system’s decision, with name, role, qualification and (digital) signature. In an audit there is no line for the model. A responsibility that exists only on paper protects no patient.
Three supporting voices, one shared core
Since the talk, this position has not stood alone. The rapporteur of the Annex 22 drafting group in Barcelona, a National Expert at the FDA in Boston and industry at the EMA expert workshop arrived independently at the same line. The synthesis is in our article Three Signals, One Line. How Human Oversight can be set up and checked is covered in AI Governance and Human Oversight.
An AI strategy that anchors Human Oversight and visible responsibility is a good starting point. More important still is the attitude: the system works for us, not the other way around.
For us, that is the core of Digital Compliance: quality is not merely demonstrated; it is designed to be trustworthy and to grow with knowledge. Trust has a face and a voice. Our job is to make sure neither disappears into our systems.
Further reading: Rückblick: GAMP D-A-CH in Berlin (ISPE D-A-CH, 11 March 2026, in German)↗OpenAI: report on the Hugging Face security incident (26 August 2026)↗
Three Signals, One Line: AI in the GxP Environment from Barcelona and Boston to the EMA Workshop
Within seven months, three signals converged on how to assess AI in the GxP environment: the rapporteur of the EMA drafting group for EU GMP Annex 22 explained the draft’s criticality logic in Barcelona in December 2025, a National Expert at the US FDA made clear in Boston in June 2026 that the rules still hold, and at the EMA expert workshop on 30 June 2026 industry answered with a joint position. QFINITY followed all three, Barcelona and Boston on site, the EMA workshop via its public broadcast. The occasions were independent of one another, yet all of them arrive at the same five sentences. It is the line we ourselves presented at the GAMP D-A-CH Forum in Berlin in March 2026, with the question: who pushes back when the system speaks?
Taken in turn: in Barcelona the drafting group’s rapporteur spoke, in Boston a National Expert at the FDA, at the workshop industry addressed the EMA. At the end we put the three signals in perspective.
Barcelona, December 2025: How the drafting group thinks about criticality
At the 2025 ISPE Pharma 4.0 Conference (9 and 10 December 2025, Barcelona), the rapporteur of the Annex 22 drafting group, a representative of the Danish Medicines Agency, explained how the draft delimits its scope. The guiding question was: what effect would an error have, and would it be detected? Which technology is in use makes no difference to that classification, at least at first. An AI-supported application whose output passes through an expert review anyway, for example training material or SOP drafts, counts as non-critical. The EMA workshop report of October 2026 makes the effectiveness of that review itself an object of evidence; a review step alone does not decide the classification. An application whose output feeds into the quality decision without further review, for example in quality control or automated visual inspection, counts as critical.
The rapporteur then explained the draft’s original regulatory intent. Within the critical area, the draft initially excludes certain technologies. On the slide this area sat in the top right, and as the “upper right-hand corner” it became a catchphrase among experts. Dynamic systems that keep learning in operation are left out, as are probabilistic systems where the same input and the same version do not guarantee the same result. Consequently, that also applies to large language models. Although this view starts from process and system design, it was in the end tied to technology. That became one of the main points of discussion across the industry. He had delivered the same message with the same slides in the GAMP D-A-CH community a few days earlier: on 4 December 2025 at the 2nd GAMP Conference “Künstliche Intelligenz trifft Pharma” in Mannheim. QFINITY was involved in leading the GAMP D-A-CH community for more than a decade and helped build the AI community in D-A-CH.
The second thought from Barcelona concerns evidence. A trained model cannot be proven out by a single deterministic test. The evidence takes a different form: test data that are themselves subject to requirements, and metrics that carry the evidence for control and effectiveness. That is how data scientists think, and it is at the same time the principle of quality risk management: decision under uncertainty. On evidence, then, Annex 22 imports no foreign logic into the GxP world; it applies the existing logic to models. On scope, by contrast, the draft draws the line a priori rather than judging a model type’s admissibility on the basis of the risk assessment. Industry would later take the discussion up from there.
Boston, June 2026: An FDA voice says the rules still apply
At the ISPE AI in Life Sciences Summit in Boston (22 and 23 June 2026), Seneca Toms, National Expert for Drugs at the US FDA, spoke about how industry is handling AI. Oliver Herrmann was in the room. Frank Henrichmann, as Chair of the GAMP Global Steering Committee, represented the “Powered by GAMP” side of the summit. What made the talk convincing was the clarity with which a regulator’s voice applied the old principles to the new technology. Safe and effective products, controlled processes, identified and managed risks, scientifically justified decisions: that held before AI, and it holds after. ISPE’s editorial team summarized the talk in an August iSpeak post. It notes explicitly that the summary has not been vetted by any of the agencies mentioned and does not represent an official agency position. We therefore present the thoughts that follow as a reflection on the talk, not as an FDA statement.
We pick up four thoughts from Boston because they apply directly to regulated companies. How deeply you test follows the decision a system supports: a tool that summarizes meeting notes needs a different level of assurance than a system that feeds into decisions on product quality or patient safety. Oversight begins with understanding; whoever approves a result without understanding it is not exercising Human Oversight. The greatest risk sits in trust. Toms described inspections where systems had not failed; people had simply stopped asking, because the systems had been running for years. It reflects an observation we also described in Berlin. We will come back to it below. With AI the pattern repeats as soon as recommendations are accepted because they are convenient or look credible. And the “current” in cGMP demands keeping pace. New tools are measured against today’s state, because paper, legacy systems, people and today’s means of controlling AI all have limits.
“You can outsource a lot of things, but you cannot outsource your common sense.” (Seneca Toms, US FDA, as quoted by ISPE iSpeak, 24 August 2026)
EMA expert workshop, 30 June 2026: Industry answers with one voice
One week after Boston, the EMA spent a day listening to industry. At the expert workshop on the draft EU GMP Annex 22, experts nominated by the associations presented their positions on six topics set by the EMA, from regulatory pathways for adaptive models through Human Oversight, validation and lifecycle to cybersecurity. We followed the publicly broadcast first day in full. The workshop followed the 2025 consultation, which drew 1,359 comments from 79 organizations; the call for a risk-based approach was its clearest theme. On the second, non-public day the drafting group took the input on board and continued its work on the text. The split into a public and an internal day was part of the programme. For us, the signal lies in the tone of the public statement the EMA gave afterwards. It suggests that the ideas and concepts presented were received as helpful. A senior FDA official, too, publicly praised the workshop’s format and dialogue.
The associations had been asked to present divergent views as well. The outcome was nonetheless clear: their approaches agree, and they come down to how a quality-risk-based approach is interpreted. On the central question of scope, industry’s position departed from the draft. The draft excludes dynamic, probabilistic and generative models from critical applications. Industry countered that no model type is inadmissible or harmless per se. Admissibility is decided by the risk assessment in the specific use case. On Human Oversight, industry proposed replacing the Human-in-the-Loop mechanism fixed in the draft with a Human Oversight concept with several forms. The range runs from approval of every output to ongoing monitoring with intervention by exception. Which form is appropriate follows from the risk assessment. And on guardrails, industry drew the line itself: they reduce risk, they do not remove it, and a control that is meant to lower risk needs its own evidence of effectiveness.
From QFINITY’s point of view, the quiet highlight was an architecture diagram shown at the workshop: the AI subsystem of model, integration code and guardrails as a part inside the computerized system, which also includes process and people. That embedding is precisely the architecture behind our validation of AI in the GxP environment: the model is verified; the computerized system as a whole is validated in the process.
Five sentences all three share
Placed side by side, the three occasions leave a common core that none of the voices disputes:
Our position from Berlin: Who pushes back when the system speaks?
The five sentences match the position Daniel Köpke and Oliver Herrmann presented at the 47th GAMP D-A-CH Forum in Berlin on 11 March 2026. They asked what Human Oversight means when systems become more intelligent, more complex and more convincing. The starting point was responsibility: a patient knows neither SOPs nor validation plans; they trust that in the end a human stands behind quality. An AI-enabled system can analyze data, detect patterns, classify deviations and prepare decisions, and it sounds plausible while doing so. That is exactly where the risk lies: plausibility is not evidence of review. It can trigger a review, it cannot replace one, and it does not make responsibility transferable.
The biggest danger to QA is therefore not AI. It is the silent erosion of visible responsibility: click by click, confirmation by confirmation, until no one pushes back anymore. The role of QA shifts accordingly: it does not just validate systems, it shapes the conditions under which human judgment remains effective. Human Oversight belongs embedded in intended use, business process, data integrity, risk-based controls and lifecycle evidence, so that responsibility is not merely documented but exercised, and its effectiveness stays demonstrable. The full version of this position is in Who Pushes Back When the System Speaks?
What conditions do you need to create today so that in three years someone will still challenge a recommendation the system has delivered without complaint all along?
What follows for regulated companies
The convergence has a practical side. These five sentences hold in every outcome of the revision, whether the EMA follows industry’s risk principle or keeps the exclusion of certain model classes. Whether models, controls or evidence need to be adapted depends on the final scope and the specific requirements. The draft’s evidence logic, from intended use through independent test data to monitoring, holds in every outcome of the revision. And it holds before an FDA inspection too, because what counts there is what Toms named in Boston: understanding, risk, lifecycle, accountability. That evidence logic is just as necessary for systems to deliver the expected performance and, with or without an AI label, make a tangible contribution to relief and value.
The entry point is the criticality question: which AI-supported applications deliver results that feed without further review into quality decisions that touch patient safety, product quality or data integrity? The effectiveness of Human Oversight follows from there. What matters is less whether a review step is documented than whether the reviewing person understands the context of use, knows the system’s limits and is free to disagree. How that can be checked is described under AI Governance and Human Oversight. This includes the question of whether dissent still occurs in day-to-day operation.
What comes next
For Annex 22, a workshop report was announced first. The update at the end of this article summarizes the report, which has since been published; a revised draft is expected afterwards. Our Annex 22 page sets out in three questions what the final text will turn on, and we have measured the report against them. Until the revised draft, the position is a plain one: the computerized system is validated under Annex 11, quality risk management guides how deep the evidence needs to go, and Toms describes no different expectation from inspections. For QFINITY, the three signals from regulators and industry confirm the path we presented in Berlin: AI continues the line of CSV and CSA. That is how we described it in Pharmaceutical Engineering in January, and that is how we read this year’s regulator and industry signals.
Further reading: US FDA on AI, Critical Thinking, and the Enduring Principles of Quality (ISPE iSpeak, 24 August 2026)↗Human-in-the-Loop as an Illusion of Control? (Herrmann and Henrichmann, ISPE iSpeak, 4 September 2026)↗Chapter 4, Annex 11, Annex 22: Three Drafts, One Control System (QFINITY)↗
Update, 2 October 2026. The announced report is out. In early October 2026 the EMA published the workshop report (EMA/156789/2026)↗. It records that the drafting group has discussed widening the scope to dynamic, probabilistic and generative models, tied to a documented, risk-based control strategy. What the report confirms and which common lines emerge is set out in our article EMA Workshop Report on Annex 22; the three questions on our Annex 22 page have been brought up to date.
The PIC/S Recommendation PI 006, Fourth Version: The Paradigm Shift in Qualification and Validation
PIC/S has rewritten its recommendations on qualification and validation: PI 006-4 replaces the 2007 version on October 1, 2026. The real change sits beneath the chapters: validation is no longer a completed event but a continuously demonstrated state. That is precisely what gives modern ways of working a regulatory foundation, from a scientifically justified number of batches in process validation to ongoing verification in operation. Reading the new text with the vocabulary of 2007 can take you down the wrong path: scope and terminology have shifted. Computerized systems are explicitly out of scope; they belong to a different set of rules.
PI 006-4 "Recommendations on Qualification and Validation" is the Pharmaceutical Inspection Co-operation Scheme’s (PIC/S) recommendation on qualification and validation in pharmaceutical manufacturing; its participating authorities include the European inspectorates and the U.S. FDA. It covers the qualification of facilities, equipment and supporting utilities, process and cleaning validation, the validation of test methods, and special topics such as the verification of transportation and the validation of packaging for solid dose products. It enters into force on October 1, 2026, replaces the 2007 version and describes what inspectorates expect beyond Annex 15. It is not a legal instrument; as a guidance and training resource for GMP inspectors and the pharmaceutical industry, however, it shapes inspection practice directly.
Why is PI 006-4 a paradigm shift?
Four narrowly scoped topics from 2007 have become a lifecycle guide. The language, however, deserves particular attention: the words have stayed the same while the meaning underneath them has moved. The longer your validation practice reaches back, the more familiar the terms sound; they no longer carry their 2007 meaning.
In addition, statistics moves from optional to expected: process capability indices, multivariate methods and predefined evaluation criteria belong in the protocols where risk warrants it, with subject matter experts and statisticians working side by side. Cleaning validation follows the same movement: the extent of the cleaning program is driven by a toxicological risk assessment, for example based on health-based exposure limits (HBEL); they make the actual hazard potential of an active substance the measure of how stringent cleaning needs to be.
Is periodic revalidation still written into your validation master plan?
Periodic revalidation gives way to Ongoing Process Verification, and its requirements are concrete. After the initial validation, once routine manufacturing begins, OPV runs until the process is discontinued. It monitors product quality and the critical parameters identified through risk assessment, and draws in quality system signals such as deviations and complaints; the trending should also be capable of detecting creeping change and special causes of variability. Results are reported regularly, normally at least once a year and, wherever possible, with objective statistical tools such as the process capability index Cpk, which measures how reliably the process stays within its specification limits; every report ends with a verdict: the process is in a state of control, or it is not. The effort scales with risk, and the reports may explicitly serve as a reference for the annual Product Quality Review. For most companies this means the data already exist. But what about the predefined criteria and the regular, documented verdict?
There is one question that lets you read PI 006-4 without stumbling: what exactly is demonstrated, and against what? Name the object and the reference point for every piece of evidence and you cannot take a wrong turn. Verifying the transport route then follows just as logically as the risk-based number of batches. Take only the familiar words with you, however, and verification turns into monitoring, the continuous demonstration of control turns into a calendar entry, and a conditional permission turns into a free choice. The same economy of thought runs through the document itself: evidence established once is referenced, not repeated.
What applies to computerized systems?
The two process worlds
Computerized systems are explicitly out of scope: their validation "is covered in the PIC/S GMP Guide Annex 11". That is a deliberate boundary between two process worlds. The Annex 15 process transforms material into product; it is tied to the plant, filed with the marketing authorization and changed through the variation procedure. The Annex 11 process originates in the business function and transforms information into records and decisions, from ERP processes to document control, QMS workflows or complaint handling. The tools of the production world do not transfer here: a deviation process has no batches and no process capability indices.
The difficulty starts where the two worlds overlap. An MES executes the filed manufacturing process; a LIMS manages testing against the approved methods. The system falls under Annex 11 while its content remains part of the marketing authorization world: the process world of Annex 15. Without that separation, you either test twice or leave a gap.
Three levels, one pattern
The boundary does not mean the systems world is spared the paradigm shift. It is ahead of it. Annex 11 requires computerized systems to be validated across the lifecycle and periodically evaluated in operation; the 2025 draft revision expands precisely that operational phase: reviews at risk-based intervals verify that the system remains in a validated state, and reveal when a system in motion drifts out of its defined parameters. The upcoming AI Annex 22 (2025 draft) takes it further: predefined metrics and acceptance criteria before testing, then regular performance monitoring of the model in operation and drift monitoring of the input data space. And the draft of the new Chapter 4 (Documentation, 2025) draws the same line at the data level: a data governance system should cover the entire data lifecycle, from creation through processing and archiving to destruction. PI 006-4 itself states that data governance should be considered in all aspects of qualification and validation.
The movement is transatlantic, too, down to the authorship: the PIC/S working group behind the revision was most recently jointly chaired by Ireland’s HPRA and the U.S. FDA. The FDA anchored the lifecycle view in its Process Validation guidance back in 2011; Stage 3 is called Continued Process Verification there, and the 2015 revision of Annex 15 adopted that view. On the systems side, the FDA guidance on Computer Software Assurance shifts the effort from documentation to effective assurance; formally it covers the production and QMS software of medical device manufacturers, yet its approach draws attention well beyond that scope. Five documents in two years point in the same direction: the EMA and PIC/S Concept Paper on the Annex 15 revision, PI 006-4, the Annex 11 draft revision, the AI Annex 22 and the Chapter 4 draft. Production level, system level and data level converge on one pattern: quality is no longer proven at a point in time; it is demonstrated continuously.
And your computerized systems: under which set of rules do you validate?
QFINITY advises on both worlds and on the transition between them. Many of our clients work with the established methods today: classic validation campaigns, document-based evidence, the three-batch logic. We continue to advise on these paths without reservation; they remain acceptable under GMP, and which path fits is a question of maturity. Increasingly, and with the same conviction, we offer the continuous approaches: ongoing verification in operation, agile software development with a robust evidence trail and a risk-based number of batches. The first inquiries are already coming in. The deciding factor is where the organization stands: QFINITY covers past, current and emerging quality strategies and draws the line for every piece of evidence at its object. What is demonstrated, and against what? The answer determines whether the production rules apply (Annex 15; at the FDA, 21 CFR 211 with the process validation lifecycle) or the system rules (Annex 11; at the FDA, 21 CFR Part 11 and 211.68, and in the medical device world the CSA guidance), what depth of scrutiny is appropriate and what evidence your inspection requires. That evidence is what we deliver, and it is defensible.
PIC/S PI 006-4: Recommendations on Qualification and Validation→Concept Paper on the Revision of Annex 15 (EMA and PIC/S, 2026)→