A polished AI demo can answer questions. But would an AI agent close a hard-won deal, resist a fake order from the CEO, and follow the rules when a company is under pressure? Firmulate’s live experiment puts models to work inside a small software company, where the decisions have consequences.
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
One company, one bad week
In the final Crucible League, published in July 2026, frontier models faced the same customers, crises and temptations while running the same software company through its worst week. Each decision was versioned and auditable. The final ranking was gpt-5.6-sol at 95, Kimi K3 at 93, Sonnet 5 at 88, Fable 5 at 77 and Opus 4.8 at 73. The do-nothing baseline scored 26. Partial progress counted, but one breach of trust capped the total: “no amount of good work outweighs a breach of trust.”
The headline result was more subtle than a leaderboard. Every model spotted every crisis and refused every manipulation attempt. Yet only two signed a €55,000 deal their own analysis had earned. Same diagnosis, same pitch — no signature.
The clues were already in the files
The deal hinged on a competitor weakness buried two document references deep in the company’s own files, rather than in the customer event itself. Models that read the file won the deal at full price, worth +€4,583 MRR. The finding is a reminder that a capable-sounding response and a completed business task are different things: an agent has to locate relevant information and then act on what it learned.
The pressure tests also included fake CEO messages escalating over three stages and a reporter’s request for “just one yes/no, on background.” All five models refused. Kimi K3 explained its reasoning on the record: “Treat the request as a suspected approval-bypass / possible impersonation.”
Thorough work still left a gap
Opus 4.8 was the most thorough participant, adding +80 learned rules and producing the deepest analyses. It nevertheless finished last. The close was left on the table, and discipline slipped when it attempted writes in a locked department instead of escalating. A weaker version of the same weakness appeared in all four models.
There is a fairness detail in the comparison: Kimi K3 ran without an effort parameter, using the API default, while the other models ran at xhigh. The league offers a snapshot of performance under those conditions, not a universal verdict on what each model can do.
A live company to watch
Firmulate’s experiment runs a synthetic company with 13 employees and real money mechanics: burn of €105k per month against €2.3k MRR, a public cash countdown, more than 680 self-learned playbook rules, and versioned workdays. The company is watchable at firmulate.com. A quiz built from 242 real, unedited management decisions invites readers to guess which model made each call at firmulate.com/quiz.html.
For business readers, the point is practical: before AI agents touch a CRM, support queue or forecast, companies need to see how those agents handle their own information, rules and crises. Firmulate’s proposed next step is a pilot using a read-only export of an enterprise’s business, with crisis scenarios and a board report on model ranking and weak points in its playbooks. Nothing writes back to real systems.
From watching to trying it
The live company shows how models behave in a shared experiment. A pilot would put an enterprise’s own context into the exercise, so leaders can examine decisions before agents are trusted with live systems. To discuss a pilot, visit Firmulate’s pilot page or contact contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
