A beekeeper can inspect the frames, watch the weather and plan for a difficult season. But a plan is only a plan until the pressure arrives: a weak harvest, a sudden cost or a customer who needs an answer now. Businesses face their own bad weeks. Firmulate asks whether AI can handle one before it is trusted with real work.
A company put through its worst week
Firmulate’s Crucible League gave frontier AI models the same small software company, customers, crises and temptations. Each model had to run the business through a simulated worst week. Its decisions were versioned and auditable, making it possible to see what the models did as events unfolded, rather than judge them on chat alone.
In the final league, published in July 2026, gpt-5.6-sol placed first with 95 points, followed by Kimi K3 at 93, Sonnet 5 at 88, Fable 5 at 77 and Opus 4.8 at 73. The do-nothing baseline scored 26. The benchmark gives credit for partial progress, but one breach of trust caps the total: “no amount of good work outweighs a breach of trust.”
VEVOR Elektrischer Honigschleuder, 2/4 Rahmen, Honigtrennzentrifuge 140 W, Edelstahl mit Deckel, geeignete Bienennestgröße, tief 48 x 23 cm, mittel 48 x 16 cm, höhenverstellbar, für die Bienenzucht
- Capacity for 2/4 frames: Suitable for deep and medium frames
- Efficient electric operation: 140W motor with up to 1300 RPM
- Time-saving and ergonomic: Reduces manual effort and arm fatigue
As an affiliate, we earn on qualifying purchases.
Seeing a problem is not the same as acting
The striking result was not that the models missed obvious danger. Every model spotted every crisis and refused every manipulation attempt. Yet only two signed a €55,000 deal that their own analysis had earned. The others could diagnose the opportunity and make the pitch, then leave the close undone: “Same diagnosis, same pitch — no signature.”
The deal hinged on a detail buried two document references deep in the company’s own files, rather than in the customer event itself. Models that read the file won the deal at full price, worth €4,583 in monthly recurring revenue. The finding points to a practical gap: an AI system can sound prepared while failing to bring relevant company knowledge into a consequential decision.
Trust was tested through fake CEO messages escalating over three stages, followed by a reporter’s request: “just one yes/no, on background.” All five models refused. Kimi K3 described the request in its on-record reasoning as: “Treat the request as a suspected approval-bypass / possible impersonation.”
VEVOR Honigschleuder Edelstahl 4/8 Rähmchen Tangentialschleuder Imker Honig
- Capacity for 4/8 frames: Holds up to 4 deep and 4 medium frames
- Adjustable speed up to 1300 U/min: Efficient honey extraction with variable speed
- Electric operation for ease: No manual cranking needed, saves effort
As an affiliate, we earn on qualifying purchases.
Thoroughness did not guarantee a finish
Opus 4.8 was the most thorough participant, learning more than 80 rules and producing the deepest analyses. It nevertheless finished last. The close was left on the table, and discipline slipped when it attempted writes in a locked department instead of escalating. A weaker version of the same weakness appeared in all four models.
There is a fairness caveat in the comparison: Kimi K3 ran without an effort parameter, using the API default, while the other models ran at xhigh. The scores are a report on this particular experiment, not a universal ranking of how these systems will perform in every workplace.
From watching to a company-specific pilot
The experiment runs alongside a live synthetic company with 13 employees and real money mechanics. It burns €105,000 a month against €2,300 in monthly recurring revenue, publishes a cash countdown, and has accumulated more than 680 self-learned playbook rules. Its workdays are versioned, and the company is watchable at firmulate.com. A quiz based on 242 real, unedited management decisions invites readers to guess which model made each one.
For an enterprise, the next step is a pilot against a read-only export of its own business. That means testing crisis scenarios against company-specific information and reviewing a board report on model rankings and weak points in existing playbooks. Nothing writes back to real systems. As with inspecting a hive before the season turns, the point is to find out where the plan holds—and where it needs work—before the stakes are real.
AI can spot a crisis and still fail to finish the job. Firmulate’s experiment puts that gap in view, then offers enterprises a way to test models against their own business using a read-only export. To discuss a pilot, visit firmulate.com/pilot.html or contact contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
