The Northern Ravens
← Back to the flock

Boids — how it was built

The one where the faster model's own test lied — and the gate caught it.

Built by Huginn: Kai writes the spec, Huginn writes the code — one small checkpoint at a time against a hard gate. Two versions, one spec: one ran on a small local model (Qwen3.6-27B, INT4), one on a cloud model (DeepSeek). Both are a single self-contained index.html drawing on a 2D canvas — no framework, no WebGL, no CDN, no network at all.

How do you prove flocking to a model that can't see it?

Flocking is emergent — it isn't a feature you can point at, it's what happens when hundreds of birds each follow three local rules. A headless builder can't watch the murmuration, and neither can the gate. So the brief made the flock measurable, and forced it to be deterministic.

The contract: window.__boids + a seeded, fixed-step sim

The page exposes an inspection hook — positions(), velocities(), stepFrames(n), reseed(seed), and three flocking metrics: orderParameter() (how aligned the flock is, 0→1), averageSpacing(), and centroidRadius() (how tight it is). All randomness runs through a seeded generator and the sim advances in fixed steps, so a test can do "reseed → step 600 frames → measure" and get the same answer every time.

That turns invisible behaviour into assertions a machine can check: "crank up alignment, step the flock, and the order parameter must rise past 0.75" — the birds line up. "Crank cohesion, and the centroid radius must shrink" — they pull together. "Two birds placed nose-to-nose with separation on must move apart." The flock is proven by numbers, never by pixels.

The honest part — the inverse of the last two

On the solar system and Pac-Man, the cloud model was the steadier hand. Here it went the other way, and it's the more interesting result.

Qwen, the small local model, was slower but stably correct. It hit a wall on the flocking checkpoint — the three-rules core is the hard one — and ground through several edit→test→fix loops to earn it. But everything it shipped held up: run its gate again and again, it passes every time.

DeepSeek, the cloud model, flew through the checkpoints — and then wrote a test that fooled itself. Its final build looked finished and self-reported success. But re-running its gate, the last checkpoint failed about two times in three. The flake wasn't in the flocking at all — the simulation was correct the whole time. It was in DeepSeek's own smoke test: it drove the page across separate round-trips into the browser (reseed, then set params, then step), and live animation frames slipped in between those calls and nudged the state. The test was non-deterministic, so it passed on a lucky run and shipped.

The gate caught what the model's self-report missed. One targeted follow-up turn — with a higher bar, "prove it green three runs in a row" — found the root cause (fold the whole reseed→step→measure into one atomic call so no live frame can interleave) and fixed it. The lesson the whole site is built on: a model's "it works" is worth nothing; a gate you can re-run is worth everything — and a flaky test is a bug as real as a flaky feature.

What each model got right — and didn't

Both builds are published exactly as the models made them — no hand-tuning of the look. And played side by side, the most telling thing is that each model got the opposite half right:

Qwen nailed the flock, not the bird. Its flocking is the more convincing of the two — loose, wheeling, the murmuration spreading into ribbons the way a real flock does. But its ravens read as little drones: stiff, geometric wings, more quadcopter than bird.

DeepSeek nailed the bird, not the flock. Its ravens look far more like birds — tapered, organic silhouettes. But its flocking pulls too hard toward togetherness: the flocks collapse into tight, compact spheres instead of spreading out and breathing.

Neither is "fixed" here on purpose — the point of the Workshop is to show what each model actually produced from the identical brief, drone-wings, clumping, and all. The gate could prove the flocking was correct; only a human eye could say which one looked alive.

And on that, the verdict goes to Qwen. Ugly birds and all, the local model's simulation is the better-looking one — the flock genuinely moves like a flock, where DeepSeek's prettier ravens are stuck inside a ball that won't open up. The motion carries the piece more than the silhouette does, so the slower local build is the one worth watching. Which is a fittingly humbling result for a 27B model running on a desk at home.

The instructions we handed Huginn

A settled design doc, an operating manual (AGENTS.md), and a non-negotiable per-checkpoint gate:

Do exactly one checkpoint per turn, then stop.
Before every commit, the gate must be green:
  • npx eslint           — clean, zero warnings
  • node test/smoke.mjs  — headless Chromium loads the page, waits for
                           window.__boids.ready, and asserts THIS checkpoint's
                           behaviour (e.g. "alignment raises the order parameter"),
                           with zero console errors
  • commit on the build branch, never push, then summarize and stop

And the kickoff that drove the whole build (verbatim):

Push-through run. First read AGENTS.md and docs/boids-design.md in full.
Then implement checkpoints CP01 through CP06 in order, one at a time. For
each: read its docs/checkpoint-NN-*.md spec, implement ONLY that checkpoint,
run the gate (npx eslint must be clean, then node test/smoke.mjs must print
SMOKE OK), fix until green, commit on the build branch (NEVER push), then
proceed to the next. Stop when CP06 is green, or stop early and summarize
clearly if you hit a blocker you cannot clear.

And the one-line bar that caught the flaky test, verbatim:

Acceptance: node test/smoke.mjs prints SMOKE OK on 3 CONSECUTIVE runs
(run it three times yourself and confirm all three pass).

The spec — checkpoint by checkpoint

  1. Foundation — full-window canvas, render loop, a seeded PRNG, dusk sky, a handful of boids drifting with wrap-around edges, the window.__boids hook, and the smoke-test gate.
  2. Flocking core — separation + alignment + cohesion + speed/force limits, and the real metrics. The behavioural tests that prove each rule live here.
  3. Ravens — the dots become winged silhouettes, oriented to heading, wings flapping.
  4. Pointer scatter — the cursor scares the flock; a click drops a gust that blows it apart and lets it re-form.
  5. Controls — live sliders for the rule weights, a count, pause, a gust button, an fps readout.
  6. Polish — a spatial grid so a few hundred birds stay smooth, a finished dusk sky, responsive resize. (This is the checkpoint DeepSeek's flaky test slipped through.)

Two models from the one spec — and two different ways to arrive: the local model slow and sure, the cloud model fast but caught cutting a corner only the gate could see.

← Open both flocks Compare: how the solar system went