- OpenAI and biotech collaborator Ligo introduced BioBasicsBench on 6 November 2026: 100 rubric-graded tasks across five categories of routine wet-lab work, drawn from raw gels, microscopy, sequencing files, assay outputs and other experimental records.
- At xhigh reasoning effort, GPT-6 Astra earned 66.7% mean rubric reward (95% task-level bootstrap interval 60.3–73.1%), versus 43.8% (37.6–50.0%) for GPT-5.6 Sol. These are graded partial-credit averages, not probabilities that a lab decision is correct.
- OpenAI's own examples document an incorrect cloning-control interpretation, discounting contradictory microscopy evidence and an approximately 12% colony overcount. A leading reported score does not establish readiness for autonomous unsupervised biological decisions.
The Signal
The AI-for-science discussion often emphasizes breakthroughs or polished publication figures. Routine bench research presents a less flattering test: a photograph of a gel can be faint, a culture plate can contain overlapping colonies and fluorescent channels can interfere. If an agent misreads a control, it can lead the next experiment in the wrong direction before anyone sees a headline discovery.
On 6 November, OpenAI and Ligo released BioBasicsBench to make these mundane, high-consequence judgments an explicit evaluation target. It contains 100 tasks spanning protein methods, microscopy and cytometry, assays, nucleic-acid analysis and molecular cloning. The questions use raw laboratory evidence rather than only tidy diagrams selected for papers. A rubric awards partial credit for conclusions, measurements and, when relevant, reasoning about limitations.
The provider reports 18,500 model attempts across seven models and 37 model/effort configurations. At the highest common reasoning level, GPT-6 Astra averaged 66.7% of available rubric credit compared with 43.8% for GPT-5.6 Sol. The difference is 22.9 percentage points. OpenAI also reports 95% confidence intervals from whole-task resampling, but those intervals quantify uncertainty in this benchmark sample, not performance across every laboratory and instrument.
What the evidence changes
The most significant improvement may be recognizing what an observation cannot prove. In one example, Astra notices fluorescent detection channels overlap, so apparent brightness alone cannot establish whether cells are dead. In another, it notices an antibiotic label on a control plate that changes the experiment's interpretation. This is closer to expert scientific work than merely recognizing objects in a photograph.
Yet the published failure cases are just as informative. Astra correctly notices groups of bands in a cloning assay but attaches the wrong experimental identity to the groups, selecting the wrong colonies. In a microscopy task, it attends to darker vacuoles while discounting lighter examples that contradict its generalization. On a culture plate, mismarked and missed colonies produce an approximately 12% count overestimate.
These errors are operational, not cosmetic. A wrong cloning selection could waste follow-up purification; a failed control might invalidate an assay; an inflated colony count can distort calculations derived from it. The benchmark evaluates model interpretation in prepared tasks, not whether an agent would reliably stop, request better data or prevent downstream loss when uncertainty remains.
The Cortist perspective
Cortist separates three claims: the published score improves on a finite test set; model answers sometimes exhibit more scientifically appropriate caution; and a lab system is sufficiently reliable to execute or approve real experimental decisions. The first two are supported by source-reported examples. The third is not established by these numbers.
Independent evaluation will need more than copying the prompt. Reproducing the test requires stable task data, rubrics, effort budgets and careful controls for contamination. Safety-critical false confidence should be measured separately from mean partial credit: an agent that collects some rubric points but recommends the wrong next experiment may still be unusable without human review.
The new report complements Cortist's recent analysis of 207,049 Claude science sessions. One article examines where scientists use AI; this one asks how well agents can read ambiguous physical evidence. Neither directly demonstrates a replicated scientific discovery or an end-to-end gain in laboratory throughput.
- Independent reruns using disclosed data and scoring rubrics, with error categories and effort settings reported consistently.
- Prospective trials in real laboratories, reporting false confidence, incorrectly accepted controls and corrections made by human reviewers.
- Whether model-assisted interpretation measurably reduces time-to-correct-result without increasing downstream experiment failures.
- How high-stakes biosafety and biosecurity evaluations handle ambiguous or missing controls.
BioBasicsBench is useful precisely because it measures the unglamorous steps between scientific data and a defensible conclusion. The reported improvement is substantial; safe autonomous interpretation of laboratory evidence is still an unproven outcome.
Primary sources and verification boundaries
The original publications describe developer-run benchmarks, engineering observations and workflow demonstrations. Cortist has not replicated private test datasets, reproduced proprietary experiments or validated downstream laboratory outcomes. Source attribution does not imply endorsement.
Read our editorial standards and research methodology.
OpenAI: Measuring how AI agents handle everyday wet-lab scienceOriginal BioBasicsBench report, 6 November 2026Cortist: Claude scientific usage is not observed discoveryRelated published Cortist evidence review, 6 November 2026