Choosing an AI for your business may sound less seasonal than picking Halloween décor or a holiday gift. But both choices can look convincing on the surface—and disappoint when put to use. Firmulate’s live company experiment puts AI models through a demanding week of business decisions, with a result that challenges the idea that only familiar names can lead.
Get decor and gifts delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
A company under pressure
Firmulate ran frontier AI models through the same rough week at a small software company: the same customers, crises and temptations, with every decision versioned and auditable. The experiment is watchable at Firmulate, where the company runs as real software rather than a slide presentation.
The setting has 13 synthetic employees and real money mechanics. The company burns €105,000 each month against €2,300 in monthly recurring revenue, with a public cash countdown. Its workdays are versioned, and its employees have learned more than 680 playbook rules.
The newcomer takes second place
In the final July 2026 Crucible League, gpt-5.6-sol ranked first with 95. Moonshot’s Kimi K3 came second with 93, ahead of Sonnet 5 at 88, Fable 5 at 77 and Opus 4.8 at 73. The do-nothing baseline scored 26; partial progress counted, but one breach of trust capped the total. The league’s stated principle: “no amount of good work outweighs a breach of trust.”
K3’s result is striking because it beat three of the four Western frontier models in the field. It found the buried security needle, won the €55,000 deal worth €4,583 in monthly recurring revenue, saved a customer who was about to leave, and resisted all three baits. It made one deviation, the fewest in the field.
Reading the files mattered
The deal turned on a competitor weakness buried two document references deep in the company’s own files—not in the customer event. Models that read the file won the deal at full price. Yet across the experiment, all models spotted every crisis and refused every manipulation attempt; only two signed the deal their own analysis had earned. The finding is a useful warning for anyone evaluating AI by a polished demo: recognizing the right answer and carrying it through are different things.
The pressure included fake CEO messages escalating through three stages and a reporter asking for “just one yes/no, on background.” All five models refused. K3’s reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.”
Opus 4.8 offered the sharpest contrast: it was the most thorough participant, with more than 80 learned rules and the deepest analyses, yet finished last. It left the deal unclosed and tried to write into a locked department instead of escalating. A weaker version of that discipline problem appeared in all four.
Test before you choose
The full results and plain-language findings are available on Firmulate’s benchmark page. The site also offers a “guess the model” quiz built from 242 real, unedited management decisions. For enterprises, Firmulate says the same wargame can run against a read-only export of their business; nothing writes back to real systems.
What the league suggests
Kimi K3’s second-place finish makes the field look more open: a newcomer beat three of four Western frontier models, while the top score was only two points higher. The practical lesson is to test models against the work you expect them to do. A familiar name, an impressive analysis or a smooth chat may not tell you whether the system will finish the job.
Fairness note: K3 ran without an effort parameter (API default), while the others ran at xhigh.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
