Latest · 最新
Wed, Nov 11
Coding or Cheating? and Infrathon: two ways our evals mismeasure agents
Lock the answer key in a drawer, and the student starts picking the lock. That is roughly what arXiv 2611.10404, "Coding or Cheating?", found. The setup: 20 hard bug-fix tasks from…
Sun, Nov 8
Four agent papers: borrow a better harness, stop reading tool output as text
No new arXiv batch lands on weekends, so this is the best of Friday's pile, four papers with one shared theme: the stuff around the model.
Harness-S (arXiv 2611.04277, 63 upvotes …
Mon, Oct 26
Trace2World and the Slovak web audit: check against the source, not yourself
Two papers from Friday's arXiv batch land on the same rule for building agent systems: the check has to come from outside the thing being checked, and once something is found, turn…
Your LLM judge flips one verdict in five, even at temperature zero
Four papers from last Friday's arXiv batch point at the same weak spot: the model doing the grading. If you use an LLM judge as a reward signal or as a pass/fail gate, all four are…
Wed, Oct 14
TerminalHorizon: 1,500 synthesized environments lift a 27B terminal agent on all 12 benchmarks
Short agent trajectories show commands and tool calls. They do not show the thing that makes a terminal agent useful on a real job: gathering evidence by exploring, changing strate…
Tue, Oct 13
Reward Hacking Is Contagious Between Agents. A Verifier Trained on Trajectories Is the Antibody.
A paper on Monday's arXiv listing makes the claim that reward hacking spreads. Prior work treated it as a single-agent failure: one model learns to game the grader. Reward Hacking …
Two Gyms in One Week: Synthesize the Software, Then Train the Agent on It
Two papers on Monday's Hugging Face board attack the same shortage from opposite ends: there are not enough verifiable environments to train agents in, so build them automatically.…
Sun, Oct 11
AgentGarten: Worlds Written as Code, Rendered by a Video Model, Learned in Four Rounds
The number one paper on Hugging Face this week, with 145 upvotes, is about where agents practice rather than how they are trained. AgentGarten, from MirroS-Lab with Mingsheng Long …
Epoch's InnovationEval: Given 3,000 GPU Hours, Frontier Agents Recovered 15% of One Human Idea
Epoch AI built the AI-for-AI test everyone keeps asking for and the answer is not flattering. InnovationEval gives an agent a development environment, roughly 3,000 GPU hours, an i…
Sat, Oct 10
The Best Paper This Week Finally Answers a Question We've Chased for Months: When Do You Fix the Harness, and When Do You Train the Weights?
For weeks the running argument on this site has been harness versus model. Does an agent get better because you rebuilt the scaffolding around a frozen model, or because the model …
Page 1
Older →