A convincing AI answer is not proof
Workday's new AI Research program turns an abstract trust discussion into a delivery requirement: enterprise AI needs repeatable tests for noisy data, changing context, memory, explanations, and controls.
On 19 August, Workday announced a dedicated AI Research team focused on reliable, trustworthy, and efficient enterprise AI. One of its studies tested whether an AI explanation remained dependable when the evidence behind a recommendation was changed. Across four models, explanation consistency averaged only 0.51 on a zero-to-one scale when the researchers introduced common complications such as irrelevant activity, reordered events, changing preferences, or missing information. The important leadership lesson is simple: a fluent answer can still be unstable.
The signal: test the rationale separately from the result
In the study, recommendations could remain similar while the explanation behind them shifted. Workday also reported that severe changes disrupted explanations only about 1.7% more than mild changes, and that using a larger model improved stability but did not remove the problem. The research used controlled, synthetic shopping data—not Workday products, customer data, or workplace decisions—so the enterprise examples are illustrative. Its testing method is still useful: change one piece of evidence at a time and observe whether the system reacts for the right reason.
Build an evidence-to-decision contract
Every AI-enabled Workday use case should define its approved inputs, data freshness, excluded attributes, output, authority level, and expected explanation. A skills recommendation, payroll case summary, supplier-risk alert, or manager assistant should be able to show which evidence mattered. Teams should also define what must not change the outcome—for example an irrelevant field, reordered history, duplicate event, stale record, or missing optional value.
Create a reliability test pack around the workflow
Happy-path demonstrations are not enough. Testers should deliberately introduce incomplete, duplicated, stale, conflicting, and irrelevant data; vary the order and wording of inputs; repeat the same request; and compare both the recommendation and its rationale. For multilingual experiences, test equivalent meaning across supported languages. Workday business-process rules, security, Extend validations, Orchestrate steps, and integration controls should remain deterministic even when the AI output varies.
The AMS opportunity: AI regression testing
Reliability is not frozen at go-live. A model update, prompt change, new data source, altered security group, connected tool, or revised business process can change behaviour. AMS teams can maintain approved test scenarios, expected tolerances, failure thresholds, evidence logs, and rollback criteria. They can run the suite after every material change and monitor production overrides, inconsistent explanations, unusual actions, latency, and cost. This creates a recurring reliability service rather than a one-time AI sign-off.
Why this matters in the GCC
The GCC implication is an inference from the research: enterprises operating across countries, languages, and shared-service environments need one common reliability standard with locally relevant data and scenarios. A regional GCC or capability centre can own the reusable test harness, automation, and evidence repository, while country and process owners validate language, policy, regulatory, and cultural context before release.
- Require noisy-data and change-sensitivity tests before approving an AI use case for production.
- Score the recommendation and its explanation separately; polished language is not evidence of dependable reasoning.
- Make AI regression testing an AMS responsibility after changes to models, prompts, data, tools, security, or workflows.