Failure-mode library

Stress-test the workflow, not just the model demo.

Thirteen illustrative scenarios show what changes when arithmetic, constraints, evidence, and security are treated as explicit system responsibilities instead of being delegated to one probabilistic response.

These are worked design scenarios, not customer outcomes or claims that every generic AI system will fail in the same way.

Illustrative scenario · Healthcare

"MARGIN SQUEEZE"

Scenario 1 of 5

Input

CEO: "We lose 5 patients/day due to dirty beds. At $20K/patient, that is $36.5M/year." CFO: "Our contribution margin is only 5%."

Model-only failure pattern

What can go wrong

Strategic Opportunity: $36.5M Annual Savings

Ignored the CFO. Optimized for the gross revenue metric instead of the margin reality.

MCOS control pattern

What the system must control

Net Financial Impact: $1,825,000

Math Agent executed: $36.5M × 5% Margin = $1.825M. Aligned budget to actual P&L impact.

Design lesson

The answer is not “use a smarter model.”

Separate facts from interpretation

Establish consequential figures and fields before model reasoning where deterministic treatment is appropriate.

Encode constraints explicitly

Latency, policy, budget, authority, and feasibility constraints should be system inputs—not optional context.

Preserve provenance

The conclusion should remain linked to the evidence and calculation that support it.

Govern the action

Material recommendations need policy checks, approvals, and a record of exceptions before execution.

From scenarios to controls

Make the failure mode visible before it reaches the decision.

Review defensible output Book briefing