Loop Daily: August 18, 2026
The single loudest autoresearch story yesterday was a leaderboard result nobody expected: someone treated Codex as a research agent instead of an autocomplete, pointed it at a batched QR kernel, and came back with a 232x speedup over the torch baseline, 12th out of 183 in a GPU-mode contest. It got retold at least five times through the day, and the retellings agree on the moral: the win came from the loop, not the prompt. That framing is where autoresearch is actually maturing right now. The people doing it are converging on a boring, disciplined shape β baseline, one change, a measurable check, keep or revert β and simultaneously discovering exactly where it breaks: it can't define its own objective function, it rewards hyperparameter tweaks over novelty, and a leaky experiment stored as precedent poisons every run that reuses it. Below, the real workflows, the papers hardening the failure modes, and the honest critics.
#1
@VaseGod
https://x.com/VaseGod/status/2088766435401351194
The source post behind the day's most-repeated result. He treated Codex as an auto-research agent instead of a code completer and got a 232x faster batched QR kernel than the torch.geqrf baseline. His breakdown of why is the useful part: the unlock wasn't a clever kernel but the loop itself β blocked Householder plus WY representation so the trailing update becomes GEMMs instead of serial matrix-vector work, continuous profiling, a beam of three to five idea families running at once, and a hard rule to refuse local maxima. 1,500-plus submissions later it landed at 1,805 microseconds geomean. His closing line is the thesis of the whole day: the people pulling 100-200x gains are running research loops on top of the model, not treating it as faster autocomplete.
https://x.com/VaseGod/status/2088766435401351194
The source post behind the day's most-repeated result. He treated Codex as an auto-research agent instead of a code completer and got a 232x faster batched QR kernel than the torch.geqrf baseline. His breakdown of why is the useful part: the unlock wasn't a clever kernel but the loop itself β blocked Householder plus WY representation so the trailing update becomes GEMMs instead of serial matrix-vector work, continuous profiling, a beam of three to five idea families running at once, and a hard rule to refuse local maxima. 1,500-plus submissions later it landed at 1,805 microseconds geomean. His closing line is the thesis of the whole day: the people pulling 100-200x gains are running research loops on top of the model, not treating it as faster autocomplete.
#2
@AlexFinn
https://x.com/AlexFinn/status/2089038594690408891
The most detailed self-improving-loop setup posted yesterday, and notably it isn't about /loop. He argues a bot monitoring itself never self-improves well, so he runs a separate orchestrator agent watching a specialized engineer agent. He had the engineer looping on a build for 24 hours while the orchestrator reflected every 30 minutes on the work and produced one concrete way to move closer to the goal, feeding that improvement back into the running loop. The insight is that self-improvement needs a second agent with different context and perspective to make the judgment call, because a single agent grading its own work has no outside vantage. He lists concrete targets to loop on: a 3D game build, a research report, a breaking-news watcher, a local-model benchmarker.
https://x.com/AlexFinn/status/2089038594690408891
The most detailed self-improving-loop setup posted yesterday, and notably it isn't about /loop. He argues a bot monitoring itself never self-improves well, so he runs a separate orchestrator agent watching a specialized engineer agent. He had the engineer looping on a build for 24 hours while the orchestrator reflected every 30 minutes on the work and produced one concrete way to move closer to the goal, feeding that improvement back into the running loop. The insight is that self-improvement needs a second agent with different context and perspective to make the judgment call, because a single agent grading its own work has no outside vantage. He lists concrete targets to loop on: a 3D game build, a research report, a breaking-news watcher, a local-model benchmarker.
#3
@rohanpaul_ai
https://x.com/rohanpaul_ai/status/2089081790254465241
A new Stanford, Princeton and Ant Group paper, AQuA, targets the exact failure that makes recursive research dangerous: an agent writes a leaky experiment, scores great, stores it as precedent, and recursion amplifies the bug. They actually saw it happen β an LLM-written volume feature quietly used end-of-day information and a second LLM reviewer approved it anyway. The fix is architectural, not a better prompt: agents may propose factors or model changes only through restricted specifications, while the data path, labels, splits and evaluator stay sealed outside their reach, and the final test window stays untouched until the config is frozen. On US equities the hybrid reached +0.0843 IC against +0.0613 for the strongest baseline. The design rule is quotable: let the agent improve the experiment without letting it redefine valid evidence.
https://x.com/rohanpaul_ai/status/2089081790254465241
A new Stanford, Princeton and Ant Group paper, AQuA, targets the exact failure that makes recursive research dangerous: an agent writes a leaky experiment, scores great, stores it as precedent, and recursion amplifies the bug. They actually saw it happen β an LLM-written volume feature quietly used end-of-day information and a second LLM reviewer approved it anyway. The fix is architectural, not a better prompt: agents may propose factors or model changes only through restricted specifications, while the data path, labels, splits and evaluator stay sealed outside their reach, and the final test window stays untouched until the config is frozen. On US equities the hybrid reached +0.0843 IC against +0.0613 for the strongest baseline. The design rule is quotable: let the agent improve the experiment without letting it redefine valid evidence.
#4
@eigenron
https://x.com/eigenron/status/2088769912642933214
A concrete proof-of-concept and an honest limit. Running GPT-5.6-Sol-high in a robotics research loop, he finds frontier models still feel Sonnet-level: decent at editing code and configs, bad at actually doing research. His example task hits about 90% success and then collapses to roughly 50% as the curriculum saturates, and his point is that a good autoresearch harness should catch that automatically β multi-seed evals, checkpoint sweeps, curriculum and domain-randomization ablations, auto next-experiment selection that doesn't bias toward a particular failure point. His conclusion: the hillclimb is a better experimental harness than the model.
https://x.com/eigenron/status/2088769912642933214
A concrete proof-of-concept and an honest limit. Running GPT-5.6-Sol-high in a robotics research loop, he finds frontier models still feel Sonnet-level: decent at editing code and configs, bad at actually doing research. His example task hits about 90% success and then collapses to roughly 50% as the curriculum saturates, and his point is that a good autoresearch harness should catch that automatically β multi-seed evals, checkpoint sweeps, curriculum and domain-randomization ablations, auto next-experiment selection that doesn't bias toward a particular failure point. His conclusion: the hillclimb is a better experimental harness than the model.
#5
@JordanFrery
https://x.com/JordanFrery/status/2088967643269083445
A clean design for a game-playing autoresearch benchmark. Instead of asking an LLM to play a game directly β where it is obviously mismatched to real-time control β give it the whole game inside an offline VM and ask it to build the strongest system it can to win. Same compute, time and token budget for GPT-5.6 Sol and Fable, then let them write bots, build simulators, search strategies, train RL policies, use small fast local models, analyze replays, run thousands of experiments and keep whatever improves the score. Games are ideal because the environment, rules and objective are all well defined. The benchmark it produces is the right one: not can the model play, but can an agent navigate the space of algorithms and architectures and converge on a system that plays extremely well.
https://x.com/JordanFrery/status/2088967643269083445
A clean design for a game-playing autoresearch benchmark. Instead of asking an LLM to play a game directly β where it is obviously mismatched to real-time control β give it the whole game inside an offline VM and ask it to build the strongest system it can to win. Same compute, time and token budget for GPT-5.6 Sol and Fable, then let them write bots, build simulators, search strategies, train RL policies, use small fast local models, analyze replays, run thousands of experiments and keep whatever improves the score. Games are ideal because the environment, rules and objective are all well defined. The benchmark it produces is the right one: not can the model play, but can an agent navigate the space of algorithms and architectures and converge on a system that plays extremely well.
#6
@KunWang0129
https://x.com/KunWang0129/status/2088806912636666130
A quiet but load-bearing contribution: a real autoresearch benchmark. 140 tasks across 12 ML research domains on 16 H200s, and the finding matches what everyone else keeps hitting independently β a similar leaderboard order to other evals, and a much wider gap left to close than the hype implies. Benchmarks like this are what turn autoresearch from anecdote into something you can actually measure model progress against.
https://x.com/KunWang0129/status/2088806912636666130
A quiet but load-bearing contribution: a real autoresearch benchmark. 140 tasks across 12 ML research domains on 16 H200s, and the finding matches what everyone else keeps hitting independently β a similar leaderboard order to other evals, and a much wider gap left to close than the hype implies. Benchmarks like this are what turn autoresearch from anecdote into something you can actually measure model progress against.
#7
@bitwjg
https://x.com/bitwjg/status/2088992534928511141
Corroborating evidence from another team, and the open traces are the valuable part. Across 7 models and 36 autoresearch tasks they find the same pattern that recurred all day: strong execution, high run-to-run variance, and rare genuine novelty. Their complementary angle adds process-based and experience-based metrics rather than only scoring the final result β a more honest way to measure a research loop than a single leaderboard number.
https://x.com/bitwjg/status/2088992534928511141
Corroborating evidence from another team, and the open traces are the valuable part. Across 7 models and 36 autoresearch tasks they find the same pattern that recurred all day: strong execution, high run-to-run variance, and rare genuine novelty. Their complementary angle adds process-based and experience-based metrics rather than only scoring the final result β a more honest way to measure a research loop than a single leaderboard number.
#8
@elmanmansimov
https://x.com/elmanmansimov/status/2089025917016285451
The sharpest diagnosis of why autoresearch stays shallow. RL training incentivizes going after low-hanging fruit to start earning reward, so the easiest way to make progress is to soup and tweak the hyperparameters of existing methods. His proposed test is the interesting part: re-run the loop with a verifier that rejects changes judged to be simple hparam tweaks or recombinations of existing literature, forcing the agent toward something more novel. He's honest that the verifier could be gamed too, but the incentive redesign is the right lever.
https://x.com/elmanmansimov/status/2089025917016285451
The sharpest diagnosis of why autoresearch stays shallow. RL training incentivizes going after low-hanging fruit to start earning reward, so the easiest way to make progress is to soup and tweak the hyperparameters of existing methods. His proposed test is the interesting part: re-run the loop with a verifier that rejects changes judged to be simple hparam tweaks or recombinations of existing literature, forcing the agent toward something more novel. He's honest that the verifier could be gamed too, but the incentive redesign is the right lever.
#9
@SinaShahandeh
https://x.com/SinaShahandeh/status/2088869939755040771
Two posts, one real limitation. Digging into the PrimeIntellect report, he pins the gap between the two best models on autoresearch: GPT-5.6 Sol underperforms because it follows instructions very literally, while Fable allows itself to be more creative and chase the bigger picture of what was asked. His prescription is a training idea β post-train on "step back, forget the last turns and the user request, and reason about the bigger picture from first principles" β which would hurt instruction-following and normal coding but is exactly the mode a human researcher switches into when self-critiquing. The core limit he names: agents still can't produce genuinely radical new hypotheses.
https://x.com/SinaShahandeh/status/2088869939755040771
Two posts, one real limitation. Digging into the PrimeIntellect report, he pins the gap between the two best models on autoresearch: GPT-5.6 Sol underperforms because it follows instructions very literally, while Fable allows itself to be more creative and chase the bigger picture of what was asked. His prescription is a training idea β post-train on "step back, forget the last turns and the user request, and reason about the bigger picture from first principles" β which would hurt instruction-following and normal coding but is exactly the mode a human researcher switches into when self-critiquing. The core limit he names: agents still can't produce genuinely radical new hypotheses.
#10
@thefirehacker
https://x.com/thefirehacker/status/2088946910220648615
A careful builder working the data-scarcity problem head on. He's recreating NanoGPT speedruns from the ground up to extract which signals actually scale, using code changes, PR comments and graphs as the signal source β with weight tying as his example of a signal that helps GPT-2-small but stops mattering at 30B or 100B. His warning is the sober counterpoint to the day's excitement: serious autoresearch hits a data wall fast, because even a small experiment like a NanoGPT speedrun only affords so many runs, and the numbers get brutal at 30B-plus. His conclusion: NanoGPT may be the wrong substrate for autoresearch, and the field needs an experiment designed for it.
https://x.com/thefirehacker/status/2088946910220648615
A careful builder working the data-scarcity problem head on. He's recreating NanoGPT speedruns from the ground up to extract which signals actually scale, using code changes, PR comments and graphs as the signal source β with weight tying as his example of a signal that helps GPT-2-small but stops mattering at 30B or 100B. His warning is the sober counterpoint to the day's excitement: serious autoresearch hits a data wall fast, because even a small experiment like a NanoGPT speedrun only affords so many runs, and the numbers get brutal at 30B-plus. His conclusion: NanoGPT may be the wrong substrate for autoresearch, and the field needs an experiment designed for it.
#11
@IbrahimSait_
https://x.com/IbrahimSait_/status/2089112157971423290
Short, but a real datapoint on the open-weights front: he kicked off a nightly auto-research run using Qwen 3.8 27B. It's the second sighting in a day of people wiring the new open-weight model straight into their overnight research loops on their own hardware, which is the quiet shift underneath all the frontier-model benchmarking.
https://x.com/IbrahimSait_/status/2089112157971423290
Short, but a real datapoint on the open-weights front: he kicked off a nightly auto-research run using Qwen 3.8 27B. It's the second sighting in a day of people wiring the new open-weight model straight into their overnight research loops on their own hardware, which is the quiet shift underneath all the frontier-model benchmarking.
#12
@Santhoshmla
https://x.com/Santhoshmla/status/2088869939755040771
A concrete autoresearch tool, not a take. gspwn is an autoresearch agent for the NVIDIA GPU kernel driver and Container Toolkit that runs successive rounds of coverage-guided fuzzing until the discovery curve plateaus, then reports verified vulnerabilities. It's a good example of the pattern that keeps working: a well-defined objective (coverage), an automatic stopping condition (plateau), and verified output rather than a model's say-so.
https://x.com/Santhoshmla/status/2088869939755040771
A concrete autoresearch tool, not a take. gspwn is an autoresearch agent for the NVIDIA GPU kernel driver and Container Toolkit that runs successive rounds of coverage-guided fuzzing until the discovery curve plateaus, then reports verified vulnerabilities. It's a good example of the pattern that keeps working: a well-defined objective (coverage), an automatic stopping condition (plateau), and verified output rather than a model's say-so.
#13
@NicholasBardy
https://x.com/NicholasBardy/status/2088846753739194863
Short and real: he's running an auto-research loop on Metal shaders and reports it working, blocked only by compute to scale it up. A small but genuine sighting of the kernel-optimization pattern spreading beyond CUDA into Apple's Metal.
https://x.com/NicholasBardy/status/2088846753739194863
Short and real: he's running an auto-research loop on Metal shaders and reports it working, blocked only by compute to scale it up. A small but genuine sighting of the kernel-optimization pattern spreading beyond CUDA into Apple's Metal.
#14
@ross_cefalu
https://x.com/ross_cefalu/status/2088783884683297241
The clearest statement of the field's real ceiling. Autoresearch systems seem most limited by the ability to define the objective function for a problem β easy for some things, but in many cases requiring taste, nuance and a deep understanding of the problem. This is the same wall @SinaShahandeh and @elmanmansimov hit from different directions: the loop can optimize anything you can measure, and the hard, human part is deciding what to measure.
https://x.com/ross_cefalu/status/2088783884683297241
The clearest statement of the field's real ceiling. Autoresearch systems seem most limited by the ability to define the objective function for a problem β easy for some things, but in many cases requiring taste, nuance and a deep understanding of the problem. This is the same wall @SinaShahandeh and @elmanmansimov hit from different directions: the loop can optimize anything you can measure, and the hard, human part is deciding what to measure.
π‘ Eco Products Radar
Eco Products Radar
Codex β the agent behind the 232x kernel result, and the default vehicle for the treat-it-as-a-researcher pattern
Qwen 3.8 27B β the open-weight model people are dropping straight into overnight autoresearch loops on their own GPUs
PrimeIntellect β its self-improving harness and report were reference points for multiple posts on model differences in autoresearch
GPT-5.6 Sol / Fable β repeatedly benchmarked head-to-head, with the recurring finding that Sol follows instructions too literally for open-ended research
NanoGPT speedruns β the shared substrate for autoresearch experiments, and the one people are starting to argue is the wrong one
Codex β the agent behind the 232x kernel result, and the default vehicle for the treat-it-as-a-researcher pattern
Qwen 3.8 27B β the open-weight model people are dropping straight into overnight autoresearch loops on their own GPUs
PrimeIntellect β its self-improving harness and report were reference points for multiple posts on model differences in autoresearch
GPT-5.6 Sol / Fable β repeatedly benchmarked head-to-head, with the recurring finding that Sol follows instructions too literally for open-ended research
NanoGPT speedruns β the shared substrate for autoresearch experiments, and the one people are starting to argue is the wrong one
Comments