Two configurations belong in an upgrade review: the one already accepted for the task and the proposed replacement. Without evidence for both, the team can describe the new software but cannot explain the change in its own workflow. Our recommendation is to make an acceptance record part of the upgrade, before the old configuration disappears.
This August catch-up article was researched and published in September. It proposes a practical review method; it does not report a measured customer deployment or promise a particular improvement.
Preserve the old decision
Start with the reason the existing configuration was accepted. Locate the model identity, runtime version, test inputs and review criteria. If that record does not exist, write down what can still be verified and identify the missing parts. Do not reconstruct a successful test from memory and present it as an observation.
Keep the old configuration available through the organisation's normal change process while the candidate is evaluated. That may require recording more than the model file: the document collection, prompt template and application settings can also affect the task. Record the parts that matter to the proposed change.
For a shared AI Server deployment, name the application workflows that depend on the service. A change that helps one workflow may require a different acceptance decision for another. Treat the service's users as part of the review boundary rather than assuming that one demonstration represents them all.
State the intended improvement
An upgrade should have a concrete reason. The team may be seeking a supported runtime, a correction to a known failure, or better results on a particular task. Write that reason in a form that can be checked. “Use the newer model” names an action; it does not define the expected benefit.
NIST's AI RMF overview describes consideration of trustworthiness across the design, use and evaluation of AI systems.[1] Our application of that principle is to keep evaluation attached to the change. A previous acceptance decision should not silently become evidence for a configuration that was never part of the test.
Also state what must remain acceptable. If the workflow exports structured data, the export still needs to satisfy its receiving contract. If reviewers depend on source references, the references still need to support the answer. These conditions belong beside the desired improvement, not in a separate forgotten checklist.
Compare the same work
Use the retained input set for the old and candidate configurations. Apply the same review criteria. Record changes in output that matter to the task, including new failures. An evaluator should not have to guess whether a favourable result came from the upgrade or from replacing the difficult test inputs.
MLCommons defines inference benchmarks with specified workloads and quality conditions.[2] The transferable lesson for an internal review is the importance of conditions, not an entitlement to borrow an external performance result. Measure the actual workflow and document how it was run.
Keep correctness review separate from timing. A response that arrives sooner but omits a required item does not meet the acceptance rule. A longer response that provides no additional useful evidence should not count as an improvement simply because there is more text to display.
Use a record with a place for disagreement. If two reviewers interpret an output differently, preserve the disputed example and resolve the acceptance criterion. Averaging away the disagreement may hide an ambiguity in the task itself.
Decide before expanding the change
The review can produce a narrow decision: accept the candidate for the tested workflow, repeat a named test, or retain the existing configuration. Describe the evidence behind that decision and the limitations that remain. A successful trial for one document set should not be described as approval for every application on the server.
Define the reversal path before rollout. Name who can make the decision and what evidence would trigger it. Keep the procedure specific enough to execute under pressure. Our companion article on the model-selection record explains why exact candidate identity matters here.
The enterprise platform overview can help frame the deployment discussion. The upgrade decision should still be attached to one observed task, its acceptance rule and a record that another person can inspect.
References
- NIST. AI Risk Management Framework. Accessed 2026-09-12.
- MLCommons. MLPerf Inference: Datacenter. Accessed 2026-09-12.