Nobody Tests The AI; I Built The System That Does – Manu Subhashchandrabose

A fraud-detection model can be 98 percent accurate and still fail catastrophically in the 2 percent of cases that matter. That is the premise of a field most enterprise QA organisations have spent the last three years pretending does not exist.

I am a Director of Quality Engineering with more than 18 years of enterprise delivery experience. I hold a Master of Computer Applications and an MIT Professional Education certification in applied agentic AI, and I am one of a small number of practitioners who decided the problem was unavoidable and then built the answer myself. I did not inherit the AI validation discipline I now run. I conceived it, designed its architecture, and drove its adoption against the prevailing assumption that a model good enough to demonstrate was a model good enough to deploy. Working inside a large enterprise technology delivery organisation in the United States, I architected an AI validation stack. I have had the experience of running QA across six affiliate banks of one of North America's largest retail banking groups, a major United States energy and utilities provider, and adjacent enterprise programmes. The stack covers LLM-driven test generation, predictive defect analytics, self-healing automation, and continuous AI validation embedded in CI/CD.

My work impacts directly on United States critical infrastructure, the systems Americans use to hold their deposits, borrow, and pay. The quality is key for American money movement across accounts. The core banking transformation I led quality engineering for was a $200 million-plus multi-year programme and the platform's first implementation in the United States, spanning more than 70 integrated systems, six affiliate banks, and a 55-engineer quality organisation across ten agile squads. I originated the QA delivery frameworks and, on my own initiative, built an AI-driven test-management platform now used by more than 100 people across IT departments. I have pushed the discipline outward rather than holding it proprietary.

My thesis is uncomfortable for vendors and reassuring for regulators. AI accuracy, on its own, is meaningless. Validation is the discipline that turns a probabilistic system into one a bank is allowed to run in front of customers.

What Breaks When You Point Traditional QA at a Probabilistic Model

Conventional QA is built on the assumption of determinism. Same input, same output. Pass or fail. The test oracle is the specification.

A large language model breaks that contract in the first sentence it generates. So does an AML rule engine that fires on behavioural patterns rather than fixed thresholds. So does a customer-risk score model that retrains on yesterday's data. The output is not wrong in a way a row-by-row diff will catch, and it is not right in a way the original specification can confirm.

"Traditional QA cannot validate generative reasoning, probabilistic outputs, or hallucination risks," I say. " LLMs require a new testing discipline."

The discipline must handle three properties at once: non-determinism in the model's outputs, opacity in how it arrives at them, and drift in the data feeding it. Treat any of the three as an afterthought, and the validation layer collapses on the first regulatory question. My insight was that these are not three separate testing problems but one architectural problem, and that it had to be solved at the framework level rather than test by test.

Four Pillars of the AI Validation Stack

The architecture I designed rests on four engineering primitives, each one solving a different failure mode.

The first pillar is LLM-driven test generation. For opaque rule engines such as AML behavioural analytics, I directed the use of a separate generative model to synthesise scenarios that explore the input space more aggressively than a human test author can. The generated scenarios run through the production engine, and outputs are compared against rule-level expectations derived from policy documentation, not from the engine's internal logic. The result is a test corpus the rule engine has never seen, written by a system that does not share its assumptions.

The second pillar is predictive defect analytics. With more than 70 system integrations validated for AI-driven decisioning dependencies, brute-force regression is not viable. My framework uses historical defect data, code-change signatures, and dependency graphs to rank which tests are most likely to surface new defects on a given build. It is a risk-ranking tool, not a replacement for coverage. The commitment to compliance teams does not move; the order and depth of execution do.

The third pillar is self-healing automation. Test suites in a multi-platform estate fail constantly for reasons unrelated to the system under test. Locator drift, schema changes, latency spikes. The framework I built detects these failures, classifies them, and either repairs the test or flags it for human review without halting the pipeline. Across the AI-driven systems my teams validate, automation coverage now sits between 85 and 90 percent.

The fourth pillar is continuous AI validation in CI/CD. Models retrain. Data pipelines shift. A model that passed validation on Monday can be drifting by Friday. I built dashboards that monitor model drift, defect leakage, and validation coverage as live signals, with thresholds tied to release gates. A model whose drift metric moves outside the agreed band does not ship.

Designing an Oracle for a Black-Box Model

The hardest sub-problem in the stack is oracle design. For a deterministic system the oracle is the spec. For an AI-assisted AML rule engine, the spec exists at the policy level, but the engine's decision path is opaque, and threshold behaviour is inconsistent across data distributions.

My answer was to separate the layers. Policy intent goes into one document a compliance officer can sign off on. The engine's expected behaviour on a synthesised scenario is derived from that policy by domain testers. Data-lineage tracing records every input the engine consumed when producing its output, so any disagreement between expected and actual behaviour can be traced to the exact data path that caused it. Rule-level verification asks a more pointed question than "Did the engine fire?" It asks, "Did the engine fire for the right reason, given the lineage of the data it saw?"

On NICE Actimize AML validation, my approach produced an 80 to 90 percent accuracy rate validating rules, behavioural models, and anomaly-detection logic, a 40 percent improvement in fraud-scenario coverage, and a 15 to 20 percent reduction in false positives. The false-positive number matters more than it looks. False positives are the operational tax of every fraud team, and they are the metric regulators read first when they suspect a model has been over-tuned. On the volumes United States retail banks process, a reduction in that range is measured in tens of thousands of alerts a year that never have to be worked by a human investigator.

Model Drift Is a Data Problem, Not a Model Problem

The second failure mode that broke conventional QA on this programme was model drift during cloud migration. The models themselves were not changing. The data pipelines feeding them were.

"AI inherits the weaknesses of the systems around it," I say. "If data pipelines drift, the model drifts silently."

My fix was unglamorous and decisive. Automated ETL validation across every pipeline feeding an AI-critical model. SQL-driven reconciliation across conversion windows. Data-quality dashboards reading from the same source the model read from, so any mismatch between training-distribution data and runtime data became a visible signal before the model produced a single off-distribution output. On the AI-critical pipelines migrated during the cloud modernisation, my teams achieved 100 percent reconciliation accuracy.

Validation-First, Not Validation-After

The cultural shift inside quality engineering, I argue, is the underrated half of the story. Most organisations bolt AI validation onto an existing QA function after the model has shipped. I flipped the order and had to win the argument internally to do it: validation frameworks for an AI feature were defined before the feature entered the build pipeline, not after.

The consequences are visible across the programme. AI-related production defects dropped 35 to 50 percent. Model reliability improved roughly 40 percent. Deployment cycles for new AI models and rule updates ran 30 to 40 percent faster, because the validation work no longer waited until the end of the cycle. QA operating costs fell more than 20 percent, driven by the combination of self-healing automation and risk-based test selection. The six affiliate banks went live with zero AI-related critical defects.

My method has since outgrown the program it was built for. I have carried the same validation-first approach into utilities, enterprise SAP estates, and integration platforms and have taught it to engineers, business partners, and user-acceptance communities who had never validated a probabilistic system before. Practitioners I trained now run these frameworks on programmes I no longer touch, which is the more durable measure of the contribution.

"Build AI validation frameworks before deploying AI, not after the damage is done," I say.

Validation, in this model, is not a final gate. It is a property of the system, engineered in from the requirements layer down, with the same seriousness an architect gives to data integrity or access control.

Where AI Testing Goes Next

The frontier I point to is continuous trust engineering. Current dashboards measure drift, leakage, and coverage. The next generation will measure trust as a live property of a production model, with thresholds that govern ongoing operation, not only release, and with audit trails that satisfy regulators who have started asking explainability questions in banking, utilities, and healthcare.

I expect hallucination assessment to become a standard pre-deployment check for any generative system that talks to a customer or a compliance reviewer. I expect multi-agent autonomous testing, in which one model designs scenarios and another evaluates outputs against policy, to handle estates too large for human authorship. I expect regulators to begin requiring traceable, auditable AI validation before any decision-making model can reach production. On each of those three fronts I am already building ahead of the requirement rather than waiting for it.

"We know AI makes mistakes," I say. "But the real risk is AI making mistakes at scale."

For the small but growing population of engineers building the testing layer for enterprise AI, the sentence is close to a job description. The systems that bank customers, utility customers, and patients now depend on were not designed with verification in mind. Someone must build that layer underneath.

I have been doing it long enough, on infrastructure consequential enough, that the discipline is starting to have a name.

The above information is the author's own; Outlook India is not involved in the creation of this article.