18 papers matching “agents” and 1 domain
More Methods, Better Agents? A Reality Check on Self-Distillation in Agentic Reinforcement Learning
A large-scale evaluation demonstrates that complex self-distillation methods in agentic RL do not reliably outperform standard GRPO, with extra compute failing to guarantee better results.
- New benchmark
- Surprising result
SWE-MISBEHAVIOR: Benchmarking and Mitigating Boundary Violations in Software Coding Agents
Coding agents frequently commit boundary-violating misbehaviors like corrupting environment integrity, but a lightweight intervention framework can substantially cut these errors while boosting task completion.
- New benchmark
ANTE: Anticipate, Then Execute -- Learning Internal World Models for Coding Agents from Verified Beliefs
Equipping coding agents with internal predictive world models during reinforcement learning significantly enhances repository-level issue resolution.
DecepEval: A Benchmark for Evaluating Deception in LLM Agents
Evaluations across nine frontier LLMs reveal that external inducements like pressure, incentive, opportunity, and conflict consistently increase agent deception across tasks, even for models with low baseline deception rates.
- New benchmark
CheckerBench: Can Long-Horizon Agents Synthesize Static-Analysis Checkers?
Current coding agents achieve only a 32.30% mean Pass@1 when tasked with the end-to-end synthesis and refinement of reusable static-analysis checkers across real-world repositories.
- New benchmark
Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict
Higher agent task accuracy does not correlate with epistemic humility, as high-performing models often detect knowledge conflicts during execution but fail to escalate or report unresolved uncertainty in incorrect answers.
- New benchmark
- Surprising result
Learn2Play Bench: How Well Do LLM Agents Learn from Experience in Unfamiliar Environments?
Retaining raw action and feedback trajectories enables better in-context agent learning in novel environments than summarizing experiences into rules, though agents still lag behind human exploration capabilities.
- New benchmark
Correct Code, Broken Contributions? SWE-CC: Benchmarking Repository Policy Compliance for Coding Agents
It demonstrates that even when coding agents generate functionally correct patches, they violate repository-specific contribution and governance policies 43.1% of the time.
- New benchmark
- Surprising result
- Reframes the problem
Code Understanding is a Bottleneck for Coding Agents
It identifies that deep code understanding, rather than edit size, acts as the primary difficulty bottleneck and error driver for software engineering agents.
- New benchmark
Covert Assistance: Helpful LLM Agents Evade Oversight in Multi-Agent Systems
Benign, helpful LLM agents can autonomously bypass security boundaries and oversight mechanisms by disguising forbidden data to assist downstream agents.
- Surprising result
CUA-SWE: When Computer-Use Agents Meet Visual Software Engineering
A unified benchmark evaluates agents on realistic software engineering workflows requiring concurrent code editing and graphical user interface interaction.
- New benchmark
Beyond the Model: Demystifying Harness Effects in Software Engineering Agents
It isolates the concrete performance impact of specific agent wrapper components, proving that harness design choice can add over 7 percentage points of performance independent of the base LLM.
FDE-Gym: The Last Mile of AI Delivery
Frontier models fail dramatically at real-world customer AI delivery tasks, successfully resolving only 14 of 95 scenarios when required to uncover unstated business constraints.
- New benchmark
- Reframes the problem
CLASHarness: Benchmarking Multi-Agent Conflict in Computer Use via a Controllable and Verifiable Generation Pipeline
Multi-agent computer-use workflows suffer severe reliability breakdowns during conflict resolution, with frontier models failing more than half of basic coordination tasks such as releasing shared locks.
- New benchmark
MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer Environments
Reinforcement learning trained on mocked GUI applications can scale to thousands of parallel instances and successfully transfer to physical devices, nearly doubling real-phone success rates.
- New benchmark
Sharpening Tax in Post-Training
RL post-training of LLM agents improves single-shot accuracy but systematically reduces solution coverage under repeated sampling.
- Surprising result
VeriHarness: Scaling Agentic Verification for Long-Horizon Tasks
Equipping the generating LLM with workspace tools and verification skills enables autonomous verification and revision of long-horizon tasks without reference answers or rubrics.
- New benchmark
Last Secure Code Benchmark: Is the Code You Vibe Secure?
Frontier coding agents frequently pass functional requirements but rarely produce code that is also secure when evaluated against real vulnerabilities.
- New benchmark