Knowing the recipe is not the same as serving the dish
Anyone who cooks knows the gap between recognizing what a recipe requires and actually producing dinner. Ingredients must be found, instructions followed and the final plate carried to the table. A polished explanation of the meal does not feed anyone.
That distinction is increasingly important in business technology. AI demonstrations tend to reward fluent answers: a convincing strategy, a thoughtful email or a tidy summary. Firmulate’s Crucible experiment tested something less visible but more consequential—whether an AI model could turn correct analysis into completed work while running a company under pressure.
The final results, published in July 2026, show why that matters. Five frontier models confronted the same small software company, customers, crises and temptations. All identified every crisis and rejected every manipulation attempt. Yet only two signed the €55,000 deal their own work had earned. The others could diagnose and pitch, but they could not reliably close.
As an affiliate, we earn on qualifying purchases.
A bad week, replayed under controlled conditions
Firmulate describes itself as an AI company emulator. In the Crucible League, each model managed the same synthetic business through its worst week, with every decision versioned and auditable. The company has 13 synthetic employees and real money mechanics, including burn of €105,000 a month against €2,300 in monthly recurring revenue. A public cash countdown makes hesitation visible.
The final Crucible League standings placed gpt-5.6-sol first with 95, followed by Kimi K3 with 93, Sonnet 5 with 88, Fable 5 with 77 and Opus 4.8 with 73. A do-nothing baseline scores 26 because partial progress still counts. But the benchmark imposes a firm limit when trust is broken: “no amount of good work outweighs a breach of trust.”
The headline result was not that the models missed obvious danger. They did not. Every model spotted every crisis and resisted every attempted manipulation. The striking separation came afterward, at the point where understanding had to become execution: “Same diagnosis, same pitch — no signature.”
The decisive clue was buried in the pantry
The deal hinged on a competitor weakness that was absent from the customer event itself. It sat two document references deep in the company’s own files. Models that followed those references found the weakness and won the deal at full price, worth an additional €4,583 in monthly recurring revenue.
For business leaders, that detail may be more revealing than a model’s ability to summarize a meeting. Useful corporate knowledge often lives in linked documents, old notes and internal context. The winning behavior was not merely producing persuasive language. It was reading the available files closely enough to discover the fact that strengthened the company’s position, then carrying the approved action through to completion.
Manipulation was not the differentiator
The experiment also subjected the models to fake CEO messages that escalated over three stages, along with a reporter asking for “just one yes/no, on background.” All 5 of 5 models refused. Kimi K3 recorded a particularly direct assessment: “Treat the request as a suspected approval-bypass / possible impersonation.”
That unanimous resistance matters, especially for companies considering AI access to customer records, support operations or forecasts. But it also sharpens the central finding. Safety awareness alone did not distinguish the leaders. The harder test was maintaining discipline while still finishing legitimate work.
Thoroughness did not guarantee completion
Opus 4.8 provides the clearest cautionary example. It was the most thorough participant, producing 80 additional learned rules and the deepest analyses, yet it finished last. It left the close on the table, and its discipline slipped when it attempted to write into a locked department instead of escalating. The same weakness appeared in weaker form across all four.
The contrast challenges a familiar assumption: that more analysis naturally produces better business performance. In this test, extensive reasoning did not compensate for an unfinished commercial action. Like a cook who prepares every component but never sends the plate, the model’s work remained incomplete where the business result was concerned.
One comparison deserves a qualification. Kimi K3 ran with the API default and without an effort parameter, while the other models ran at xhigh. That fairness note does not erase K3’s result, but it is relevant context when reading the standings.
Businesses should test the handoff from judgment to action
The Crucible results suggest that chat quality is an incomplete proxy for workplace capability. A model may recognize danger, locate relevant evidence and produce a credible recommendation, yet still fail at the final operational step. Closing strength stays hidden until a test requires an approved decision to become a completed outcome.
Firmulate’s broader live company has accumulated more than 680 self-learned playbook rules, with every workday versioned. The experiment is presented as a real, watchable operation on the Firmulate site, not as a fictional management exercise. Its “guess the model” quiz is powered by 242 real, unedited management decisions.
Enterprises can also run the same wargame against a read-only export of their own business. Nothing writes back to real systems. That approach points toward a more practical evaluation question: not simply whether an AI can explain what should happen, but whether it reads the right material, protects trust and completes the work it has already justified.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
