Self-evolving agents remember examples. CLR-Bench asks if they learn the rule.

Sometimes an agent appears to improve with experience because its memory file grows longer. The visible trajectory can look like learning: it makes a mistake, receives feedback and fixes the same mistake later. But a list of remembered exceptions is not evidence that the agent has inferred a rule that also applies to unseen cases.

A preprint released on 29 October 2026 introduces CLR-Bench, or Curriculum of Latent Rules, to put that distinction under pressure. The researchers constructed 20 chains spanning ten data domains. Each level adds a hidden rule that operates on an opaque label. One example could encode membership in a publicly named group; the label hides the group itself. Practice feedback reveals a further member and non-member, then testing asks about new members and plausible distractors.

The test is designed to reduce a common ambiguity: if a model never knew the underlying group, failure might merely reflect missing world knowledge. The authors therefore use probes to check that the base model knows the membership when the named grouping is exposed, cannot trivially guess it without feedback and receives enough evidence to identify the group. That still leaves the agent with the harder task of updating its own operational rule.

Six contemporary self-evolution methods were tested. With the model used for certification, the best feedback-only improvement was at most five percentage points over a frozen agent. The researchers also report that those methods excluded only 6–12% of unseen members from the count, against a 20% result from simply retaining the raw practice history. The comparison is about the authors' specific hidden-rule task and evaluation criteria; it is not an industry-wide agent learning rate.

The saved states offer an explanation for the weak result. They retain a newly revealed example member in 75–98% of cases but almost never record the category name. When the practice feedback exposes an item that the current list misses, the agent frequently appends that item. This improves future recall of encountered examples without producing a compact rule that generalizes to unfamiliar examples.

In a controlled condition, a fixed lesson explicitly saying that the labels refer to named groups increases performance by 14.5 to 16.5 percentage points. That intervention demonstrates that prompting and task representation matter as much as memory persistence. A separate stronger-model evaluation also changes the picture: with GPT-6-Sol, ACE and Dynamic Cheatsheet reach 78% accuracy and solve 59–60% of individual cells, whereas the certifying-model comparison reports 53% and 8%.

The meaningful question is not whether one model family won a leaderboard. It is what kind of adaptation a self-evolving agent can prove. A system may store dozens of corrections, appear more capable on repeated tasks and still fail a novel variant because it never represented the principle connecting its examples. Measuring only task success on seen histories can hide that failure mode.

CLR-Bench complements other Cortist reporting on agent reliability. BlindToChange examined whether agents notice updates when they are not explicitly instructed to check; PaperBenchX asked whether scientific coding agents reproduce experiments under clean replay. Here the target is different: a rule inferred from feedback must transfer to new members and non-members. These research questions are connected but do not measure interchangeable abilities.

The results should also be treated as a research preprint, not a deployment certification. Twenty benchmark chains and six methods cannot establish how commercial agents learn across arbitrary workplaces, nor do they prove a system can autonomously redesign its own objectives, tools or underlying model weights. Independent reruns with fresh data domains, fixed budgets, multiple seeds and open traces would provide much stronger evidence of generalization.