From Prompt to Production: Building a Repeatable AI Evaluation Pipeline

Most AI teams can tell you how they tested their model before deployment. Very few can show you the test infrastructure they will run six months later when the model has been updated, the prompt templates have drifted, and the evaluation criteria have evolved. The gap between a successful deployment test and a sustainable evaluation programme is where most enterprise AI quality stories end.

The Evaluation Gap

Industry surveys consistently find that the majority of enterprise LLM deployments are operating in an evaluation-free steady state after their first release — performing structured evaluation before initial deployment but lacking automated pipelines that run on subsequent model updates. The NIST AI Risk Management Framework (AI RMF 1.0) identifies this pattern under its Measure function: Measure 2.5 requires that "AI system testing and evaluation is performed to understand identified risks and potential harms in alignment with organisational risk tolerance" — and the framework makes clear this is continuous, not pre-deployment only.^[1]

The Four Pillars of Repeatable Evaluation

1. Scenario Versioning

Evaluation scenarios are software artefacts and should be treated with the same discipline as application code: version-controlled, reviewed, and promoted through environments. A scenario appropriate for model version 1.0 may need updating for version 1.2; without versioning, you cannot tell whether a score change reflects a real capability change or drift in what you were measuring. In a 2024 survey of LLM evaluation practices, researchers at Carnegie Mellon found that only 23% of teams version their evaluation scenarios alongside their model artifacts.^[2]

2. Deterministic Execution

LLM outputs are non-deterministic by nature, but evaluation pipelines should be as deterministic as possible at the infrastructure level. This means pinned model versions, fixed temperature settings for judge invocations, and idempotent scenario execution. The same scenario run twice under the same conditions should produce scores within a defined acceptable variance band — making genuine score changes distinguishable from noise.

3. Judge Calibration

The judge model — the LLM or scoring function that evaluates outputs — is itself a component that requires validation. A judge that assigns inflated scores creates false confidence; a judge that is systematically strict creates unnecessary development overhead. Regular calibration against human expert ground truth, with tracked Cohen's κ agreement metrics, is the only mechanism that maintains evaluation validity over time.

4. Regression Tracking

Every evaluation run should be compared against a baseline established at a defined reference point — typically the previous production deployment. Score changes should trigger an investigation protocol: is the change within acceptable variance, does it indicate genuine capability improvement or regression, and does it satisfy the criteria in your AI risk management documentation?

Anti-Patterns to Avoid

• Production-as-test-environment: Using live customer interactions as the primary quality signal. This is valuable data, but it only tells you what went wrong after it went wrong — it is not a substitute for pre-deployment testing.
• Single-metric evaluation: Reducing evaluation quality to a single score (accuracy, pass rate) that cannot distinguish between different failure modes. Enterprise AI systems typically need four to six distinct metrics to characterise performance adequately.
• Human review at scale: Designing evaluation pipelines that require human review of every test case. This is appropriate for building baselines and calibrating judges; it does not scale. The goal is human-calibrated, automated execution.

The purpose of an evaluation pipeline is not to prove that your model is good. It is to tell you — with sufficient speed and precision — when it has become worse, and why.

Reference Architecture

A repeatable evaluation pipeline for enterprise LLMs follows a consistent structure: a versioned scenario store feeds an execution engine that invokes the target model; outputs pass to a calibrated judge committee; scores and reasoning traces are written to an immutable evaluation store; a metrics layer computes aggregate and trend statistics; and an alerting layer triggers on regression conditions defined in the organisation's AI risk policy.

MLflow's Model Evaluation module provides a foundation for this architecture in teams already using MLflow for experiment tracking.^[3] ARIA Evaluator provides all layers of this pipeline — scenario management, multi-model judging, metrics, and alerting — in a single tenant-isolated deployment, pre-integrated with the AWS Bedrock model catalogue.