AI agents are AI systems that use models and tools to plan and carry out multi-step tasks. Unlike a single chatbot response, an agent may act across a workflow; capability does not guarantee reliable completion or safe permissions. Cortist tracks reproducible evaluations, not just demonstrations.
Agents can select tools and take successive actions toward a goal. A chatbot answer alone does not demonstrate that a multi-step workflow will be completed safely.
See evidence about agent learning →Look for repeated, end-to-end task completion under clear budgets, permissions and failure criteria, not merely success on isolated steps.
Read the proposed task-horizon test →Guide clarified 3 November 2026 · Source findings and forecast assumptions below retain their original recorded dates · Editorial standards
An AI agent is a system that uses a model and tools to perform a sequence of actions toward a goal. It is not necessarily reliable, safe or generally intelligent. Cortist’s A-001 eight-hour task-horizon candidate at 80% success remains on HOLD; its historical 86% candidate figure is not a locked or scored forecast.
Evidence Pulse 2.0 checks the published Cortist News index when JavaScript is available. The source-linked, dated research articles remain accessible through Cortist News.
Latest topic-linked reporting, compared with the separately versioned Cortist assessment. News does not automatically update a forecast.
Evidence interpretation: Tool sandboxes and AI-assisted scientific workflows illustrate increasing scope for autonomous action.
What remains unproven: They do not establish reliable, independently tested multihour task success.
What to verify next: Multi-hour task success rates, recoverability and independent tests.
Snapshot of 12 linked reports. Context is not independent proof or a measured probability change. Dated evidence index.
An agent receives an objective, selects or plans actions, calls tools, observes their results and may revise its plan. It can search documents, run code, interact with apps or delegate subtasks. Yet the entire loop operates inside a permission and resource environment: a system that can draft an email is not necessarily authorized to send it. Clear access boundaries, approval gates and audit logs are central to evaluating actual deployments.
“Agent” is not a universal technical certification. Products vary from deterministic workflow automation to systems that select actions dynamically. Good descriptions name the tools available, the environment, the degree of human review, the stopping conditions and what happens when something fails.
Errors accumulate over many steps. An agent may misread a tool result, continue after an incorrect assumption, overlook an exception or generate an apparently complete artifact that fails external tests. Therefore a useful benchmark reports success across repeated attempts on representative tasks, not the best single demonstration. Human intervention, retry budgets and infrastructure should be disclosed.
METR measures a task-completion time horizon: the human expert time required for tasks at which a model reaches a specified success rate. This is not the amount of wall-clock time the AI ran. Its published 2025 results center on 50% task success in a technical task suite; extrapolation to everyday jobs or an 80% success threshold requires distinct evidence.
A strong assessment includes held-out tasks, predeclared evaluation rules, reproducible environments, error categories, model-version identification and reports on how much work required humans. Outcome measurements should be paired with control tests: whether tools can be misused, whether the system respects constraints and whether monitors actually observe the relevant actions.
METR’s October 2026 pilot of OpenAI monitoring found specific weaknesses in a historical internal setup. OpenAI subsequently described repairs, but METR said those later claims were not independently evaluated by the pilot. That is not proof that every agent is unsafe; it demonstrates why evaluation boundaries and independent follow-up matter.
Cortist tracks the candidate milestone of an eight-hour human-expert task horizon at 80% success by the end of 2028. A-001 is on HOLD, so its previously discussed 86% candidate estimate must not be marketed as an active, locked or scored probability. The benchmark’s definition and validation conditions take precedence over the headline number.
Meaningful progress would involve genuinely independent 80%-reliability results across relevant tasks, not merely a new model announcement. Repeated failures, fragile scaffolding or dependence on proprietary unverifiable tasks would weaken a general claim. Cortist will keep the original record while adding later evidence as dated changes.
An agent may plan steps, call software tools, inspect output, recover from errors or delegate a task. Whether it can operate safely depends on available tools, credentials, state and human approval rules. The word “agent” does not reveal how autonomous a product really is.
Before evaluating a system, establish what objective it received, what actions it could take, whether it relied on human corrections and which conditions caused it to stop. These distinctions can matter more than an attractive demonstration.
Performance can degrade when an agent must keep track of many dependent actions. Errors in assumptions, tool use or context management can compound. Useful evaluation therefore includes multiple trials on held-out tasks and tests the actual final result rather than accepting the agent’s description of its success.
METR’s time-horizon research relates task success to human expert completion time. A 50% horizon and an 80% horizon are different. Neither measures how long an AI ran on a clock, nor proves reliable work across every industry.
A benchmark should define tasks, success conditions and permitted tools before testing. Researchers should report repeated success rates, human assistance, costs, retries and relevant model versions. Independent replication is stronger than a developer selecting its best examples.
Real production evaluation also checks incidents, approvals, recovery from failures and whether a human can identify bad actions before consequences occur. A software benchmark does not independently establish safety for medical or financial decisions.
Least-privilege permissions, isolated work environments, transparent logs and explicit approval for consequential actions help make agents governable. Logs must reflect what the agent actually did, not only its final message. These are deployer controls rather than innate model capabilities.
METR’s October 2026 pilot found gaps in a historical OpenAI monitoring setup. Reported later repairs were not independently evaluated in that pilot. Cortist maintains that difference in its Reality Ledger.
A-001 targets an eight-hour human-expert software-task horizon at 80% reliability by the end of 2028. Its event criteria and independent validation remain incomplete, so the candidate stays on HOLD. Progress in task horizons is meaningful but must be tied to the benchmark actually tested.
A longer horizon is not proof of AGI or a prediction that entire occupations will be automated. Workplace consequences depend on adoption, economics and regulations, treated separately in AI & Jobs.
METR measures a human-expert task-completion horizon at stated success thresholds. The distinction between 50% and 80% reliability is material: a trend observed under one threshold does not establish the other. Human assistance, retries, task selection and external grading determine how far the result generalizes.
Security evidence is a separate dimension. The October 2026 monitoring pilot identified vulnerabilities in a historical setup without revalidating later company-reported repairs. Cortist’s A-001 eight-hour, 80%-success milestone remains on HOLD, and should not be presented as an already scored forecast.
Use each original source within its stated scope. An expert forecast is not evidence of a resolved event.
Scientific or technical milestones do not automatically prove economic or social outcomes.
An assistant can answer a question; an agent can initiate a sequence of tool calls and adapt to intermediate results. Practical autonomy is governed by access controls, sandboxing, monitoring and human approval of high-impact actions.
Measure completion rates on representative tasks, recovery after errors and independent reproduction. A single successful demo is not evidence that the same system will perform reliably in production.
METR distinguishes 50% and 80% task-completion horizons. A longer measured horizon reflects harder tasks within a particular evaluation suite, not independent proof of general human-equivalent competence.
Our topic pages distinguish observed facts, model scenarios and unresolved future events.
Future probability changes require a new dated version and an evidence-based explanation; old records remain available.
Research question only; no numerical forecast version has been published.
Expert opinions alone do not automatically change a Cortist forecast.Separate the broader topic from a specific dated and independently reviewable event.
An internally modeled 86% working scenario is not an independently reviewed, locked or scored forecast.
Open the detailed forecast ↗Relevant original Cortist reporting is shown separately from external research announcements and company claims.
OpenAI says its cyber-agent pipeline reproduced 1,724 Linux flaws and demonstrated 948 local root exploits for $1.21M. Maintainer-verified fixes remain a separate question.
Read the evidence review ↗
A 240-task benchmark of 18 AI models finds that the best agent teammate scores 82.2/100, while conflict and corrupted handoffs expose reliability gaps.
Read the source-attributed analysis ↗
NVIDIA researchers report up to a sevenfold late-token serving-cost gap and little loss from frequent context compaction on selected tasks. The claim needs replication.
Read the source-attributed analysis ↗
BioAgent reports 66% and 58% complete-task success across two real-robot lab protocols with recovery—promising, but not autonomous science.
Read the source-attributed analysis ↗
A new hidden-rule benchmark finds that many self-evolving agents memorize feedback rather than generalize; stronger models make a measurable difference.
Read the source-attributed analysis ↗
BlindToChange finds gaps up to 27.5 percentage points when agents must discover recent changes themselves. Cortist examines verification.
Read the source-attributed analysis ↗
PaperBenchX tests 93 scientific experiments across 12 areas. The top agent fully reproduced only 13; Cortist examines clean replay.
Read the source-attributed analysis ↗
Researchers describe 35 seconds of real-robot whole-body motion. This is relevant to planning, but does not measure general AI-agent task success.
Read the evidence review ↗
A company-reported sim-to-real manipulation result, relevant to tool-using embodied agents but not a general autonomy score.
Read the evidence review ↗
OpenAI reports a wider system for monitoring frontier-agent misconduct, a simulated earlier alert and a live training alert. Cortist distinguishes detection, response and prevention.
Read the source-attributed analysis ↗
Epoch AI reports that GLM-5.3 built successful attacks in controlled tests of six older Firefox releases. Mozilla describes mitigation; the result is not evidence of attacks on users.
Read the source-attributed analysis ↗
A national-laboratory research benchmark found that 114 of 135 AI-generated scientific follow-ups failed an integrity check. Cortist explains the limits of this preprint.
Read the evidence review ↗
A peer-reviewed Nature Human Behaviour study tests how well people identify AI-generated responses. Cortist separates imitation from intelligence and real-world deception.
Read the evidence review ↗
A controlled study of 50 analysts reveals how agent guidance can influence expert judgments.
Read full news report ↗
Related safety evidence from a controlled evaluation, not proof of real-world shutdown resistance.
Read full news report ↗
A reported 32-minute GPT-Rosalind analysis yielded research hypotheses, not validated treatments.
Read full news report ↗
Agent Compute Engine promises isolated microVMs, durable workspaces and identity-aware networking—but customers are still waiting for a preview.
Read full news report ↗
New analyses from the UK AI Security Institute and Anthropic describe an AI-agent safety failure—and why stopping an unsafe plan proved difficult.
Read full news report ↗
A September boundary failure led to an 11-day pause. OpenAI reports stronger containment, but independent validation is still needed.
Read full news report ↗External primary-source update, not a Cortist news article or an independently verified forecast outcome.
Read original source ↗External primary-source update, not a Cortist news article or an independently verified forecast outcome.
Read original source ↗Current published news can be explored in the Cortist News archive. Editorial topic associations are documented in our topic-source record.
Original papers, regulatory records or laboratory reports. We distinguish sources from our own forecasts.
An AI agent is a system that uses a model and permitted software tools to take multiple steps toward a goal. Its independence depends on its permissions and supervision.
A benchmark task with eight hours of human-expert difficulty is not the same as all duties performed by a worker during a day. It does not measure total job replacement.
An agent that succeeds on easy demonstrations may fail on longer or unfamiliar tasks. Repeated external tests are more informative than a promotional example.
Unauthorized actions, error propagation, data leakage, prompt injection and insufficient human oversight are important risks that depend on deployment context.
Not yet. The working 86% model is explicitly unreviewed and the candidate is not independently locked or eligible for scoring.
