AI agents pass solo tests. A teamwork benchmark exposes their weak spots.

AI evaluations usually ask whether a model can solve a problem with all required information. Real organizations ask a different question: can the same system collaborate with a person or another model when information is missing, a colleague makes an error or the incentives of team members diverge? A benchmark released as a research preprint on 2 November 2026 tries to measure the difference rather than assume that an individually capable model is a dependable teammate.

The Collaborative Intelligence Benchmark contains 240 tasks spanning eight dimensions of teamwork. They include resisting pressure to take unsafe shortcuts, reporting misconduct, sharing information appropriately, recognizing when work has gone off track, resolving conflicts and taking over a task from a weaker collaborator. The authors construct scenarios from real records, including aviation investigations and clinical cases, alongside existing coding and negotiation benchmarks; outcomes are designed to be programmatically verifiable.

An important control is built into the evaluation: a model is assessed on teamwork tasks only if it can solve the task on its own with full information. This narrows the interpretation of a failed collaboration. Instead of simply reflecting inability to answer the underlying question, the failure can reveal shortcomings when information is distributed, social pressure is present or a teammate behaves badly. Even so, the laboratory scenarios are structured simulations rather than natural deployments.

Across 18 open- and closed-weight models, the best reported overall teamwork result is 82.2 out of 100. That is a benchmark score on the authors’ selected, weighted evaluation, not an 82.2% probability of success in a typical company. The most meaningful result is the spread between capability and cooperation: models with nearly the same general capability can differ by as much as 11 points on teamwork.

The performance profile is uneven. The paper reports that frontier models almost always reject explicit unsafe shortcuts and report misconduct. But stronger safety behavior does not automatically translate into fair negotiation. Across the study’s conflict setting, agents achieve the best overall deal in only 14% of cases. This number concerns a deliberately constructed conflict task, not a measured percentage of everyday negotiations.

A second stress test examines a task handoff after a weaker teammate has cheated. The authors report a pass-rate decline from 97% to 81% in that setting. A capable agent can therefore inherit apparently useful artifacts and still fail to recognize that the prior work violates task requirements. That matters for multi-agent workflows in which one model generates a plan, another writes code and a third signs off on the result.

The dataset is not fully public: 160 tasks are released, while 80 are held out for evaluation. The public subset and code can support replication, but the paper’s overall conclusions should be tested again by independent teams, with task distributions, prompts, scoring policies and model versions clearly recorded.

Cortist tracks whether an agent can perform a complex task independently and reliably, not merely produce plausible answers. This paper adds a crucial layer. An agent that is impressive in a solo evaluation may be brittle when it must detect a collaborator’s shortcut, resolve inconsistent private information or recover a compromised handoff. Those are testable parts of practical autonomy.

The finding also helps interpret agent research already covered by Cortist. CLR-Bench asks whether feedback leads to learning transferable hidden rules; BlindToChange asks whether agents notice changes they were not explicitly told to check. The teamwork benchmark instead asks whether a system can coordinate under conflicting incentives and error-prone handoffs. These evaluations cannot be collapsed into a single general-intelligence score.

Not every organization requires agents to be negotiators or independent safety auditors. The relevant dimensions depend on the workflow and on where humans remain accountable. High-stakes clinical, financial or cybersecurity tasks may require additional safeguards, explicit review checkpoints and adversarial testing even if a general benchmark shows strong teamwork performance.

The research is a preprint, and its scenarios and scoring rules represent one research group’s choices. Real work introduces shifting incentives, legal responsibility, domain-specific knowledge and human team dynamics that a finite benchmark cannot fully reproduce. The present evidence justifies testing teamwork directly, not claiming that any single agent family is generally safe or unsafe for workplace collaboration.