A 232x kernel from 1,500 submissions: what loop engineering actually looks like
Sankalp Shubham entered GPU Mode's qr_v2 problem — batched square compact-Householder QR factorization, FP32 CUDA on B200 — and came out with a 232x speedup over baseline. Geomean went from 419,000 microseconds to 1,805. Twelfth place out of 183. It took over 1,500 submissions across 14 days, and he barely wrote any of them. The writeup is at https://sankalp.bearblog.dev/autoresearch/ and it hit 357 points on Hacker News with 83 comments.
The setup is the transferable part. GPT-5.5 through Codex did the execution, driven by a /goal command that runs autonomous loops. Claude sat alongside as an advisor for concepts, not code. An AGENTS.md file encoded submission discipline so the loop didn't burn the submission budget on garbage. Structured logging tracked every attempt, its profile, and its status. Profiling came from Modal, torch profiler, nsys and NCU. And critically, he ran a beam — three to five candidate idea families alive at once, not one lineage being iterated — because a single optimization lineage walks into a local maximum and sits there. He used /btw to peek at progress in a side thread without stopping the loop.
The bottlenecks are worth knowing even if you never touch CUDA. Householder reflections are sequentially dependent, so the fix was the blocked WY algorithm, which confines the serial work to a narrow panel. Once the kernel got fast, launch overhead dominated, so CUDA graph replay plus fixed-shape specialization. Intermediate copies for the memory representation got killed by fusing V/T layout assembly. Past about 3,000 microseconds the optimization got dramatically harder, which is its own useful data point about where these loops stop paying.
His own conclusion is the sharpest line in the post: the better you know something, the better you can prompt the LLMs, because you convert unknown unknowns into known unknowns. That is the opposite of the pitch these tools are usually sold with. The loop didn't replace his expertise, it multiplied it, and the multiplier was proportional to what he already knew.
The HN thread added the caveat that matters. Contest-winning kernels overfit. Several commenters pointed out that solutions topping benchmark data collapse on out-of-distribution inputs where expert-written code holds up, and readability gets sacrificed wholesale — fine for a specialized kernel, poison for a library somebody has to maintain. The thread's own synthesis was that LLMs work best treated like Prolog or a linear program: give constraints, give a verifier, give a goal. Everyone with a success story had a harness. Nobody without one did.
← Back to all articles
The setup is the transferable part. GPT-5.5 through Codex did the execution, driven by a /goal command that runs autonomous loops. Claude sat alongside as an advisor for concepts, not code. An AGENTS.md file encoded submission discipline so the loop didn't burn the submission budget on garbage. Structured logging tracked every attempt, its profile, and its status. Profiling came from Modal, torch profiler, nsys and NCU. And critically, he ran a beam — three to five candidate idea families alive at once, not one lineage being iterated — because a single optimization lineage walks into a local maximum and sits there. He used /btw to peek at progress in a side thread without stopping the loop.
The bottlenecks are worth knowing even if you never touch CUDA. Householder reflections are sequentially dependent, so the fix was the blocked WY algorithm, which confines the serial work to a narrow panel. Once the kernel got fast, launch overhead dominated, so CUDA graph replay plus fixed-shape specialization. Intermediate copies for the memory representation got killed by fusing V/T layout assembly. Past about 3,000 microseconds the optimization got dramatically harder, which is its own useful data point about where these loops stop paying.
His own conclusion is the sharpest line in the post: the better you know something, the better you can prompt the LLMs, because you convert unknown unknowns into known unknowns. That is the opposite of the pitch these tools are usually sold with. The loop didn't replace his expertise, it multiplied it, and the multiplier was proportional to what he already knew.
The HN thread added the caveat that matters. Contest-winning kernels overfit. Several commenters pointed out that solutions topping benchmark data collapse on out-of-distribution inputs where expert-written code holds up, and readability gets sacrificed wholesale — fine for a specialized kernel, poison for a library somebody has to maintain. The thread's own synthesis was that LLMs work best treated like Prolog or a linear program: give constraints, give a verifier, give a goal. Everyone with a success story had a harness. Nobody without one did.
Comments