The Northern Ravens
← Back to Huginn

Huginn — Roadmap

A self-hosted, model-agnostic coding agent. This is the canonical plan; it supersedes ad-hoc "Slice N" notes (CP0/CP1 below were the original Slice 0/1).

Guiding principles

  • Harness over weights. Models are swappable batteries. The durable, owned, learnable value is the harness. Build the model-agnostic layer; let every model improvement (local and frontier) be a free upgrade.
  • Model-agnostic. Any OpenAI-compatible endpoint + optional API key. Local (the GPU server/llama.cpp) by default; frontier on escalation.
  • Borrow ideas, not code. Build the spine ourselves to understand every bone; study the field and re-implement the best ideas. Sources tracked per item.
  • The three pillars (where coding-agent quality is actually won — confirmed by 5 OSS projects + the r/LocalLLaMA consensus): (1) the edit tool, (2) the context strategy, (3) the verify loop — not the model.
  • Local-first reality. Optimize for weaker local models: reliable edits without text reproduction, lean context, harness compensates for model limits.

Decisions pending

  • Dedicated coder model (CP7): which box (the GPU server spare GPUs vs a self-hosted host AI server) and which model (Qwen2.5-Coder-32B safe pick vs a Qwen3-Coder MoE). Needs Oz.
  • Memory (CP8): standalone vs reuse Muninn's memory spine.
  • TUI framework (CP9): ratatui vs keep simple line REPL longer.

Checkpoints

CP0 — Spine ✅ (shipped, commit d1dda6b)

  • Agent loop with native OpenAI tool-calling
  • Pluggable model client (any OpenAI-compatible API + optional Bearer key, ${ENV} expansion, env overrides)
  • Tools: read_file, list_files, search, edit_file, write_file (path-sandboxed)
  • Search/replace edit core: exact + whitespace-flexible fallback (unit-tested)
  • One-shot CLI + interactive REPL

CP1 — Verify loop ✅ (shipped, commit 1448828)

  • run_command tool (build/test/lint), exit code + tail-truncated stdout/stderr, timeout-wrapped
  • System prompt requires verifying via build/tests before claiming success
  • Validated E2E on local the GPU server (fixed a broken cargo project, gated on exit 0)

CP2 — Reliable edits ✅ (core complete)

Goal: make the core bulletproof on weak/local models. Highest ROI change we have.

  • Emit per-line content-hash anchors in read_file (LINE#HASH| code)
  • edit by anchor instead of reproducing search text — Hashline (preferred tool)
  • Reject stale/out-of-range anchors with a clear re-read message (8 unit tests; E2E on the GPU server)
  • Keep search/replace (edit_file) as fallback
  • Auto-run the language formatter after edits (rustfmt/prettier/black/gofmt; format_on_edit config). Resolved the indentation finding: anchored edit solves location/staleness, but replace is whole-line so the model must supply indentation (Qwen miscounted 3 vs 4). Format-on-edit normalizes it deterministically — verified E2E: model typed 3 spaces, rustfmt corrected to 4. Best-effort: missing/failing formatter is skipped, never fails the edit.
  • Per-task git checkpoint so a whole task is revertible — git stash create snapshot before each task + REPL /undo; non-destructive (never auto-commits to the branch); untracked new files are left in place. Verified E2E.
  • [→] Tree-sitter syntax validation before writing — moved to CP3 (where tree-sitter is introduced; avoids pulling the dep twice).
  • [→] Diff staging sandbox (stage/preview/accept/rollback) — deferred to CP10 (pairs with the run_command sandbox/approval gate).
  • Done when: multi-edit tasks on Qwen-27B land first-try far more often; no silent corruption; indentation always correct. ✅ met for the core; syntax-validate + sandbox tracked under CP3/CP10.
  • Borrow: oh-my-pi (Hashline, ast_edit), oh-my-openagent (hashline), plandex (diff sandbox, syntax+logic validation, fallback layers)

CP3 — Repo map & context ✅ (core complete)

Goal: handle a real multi-file repo without losing coherence (the community-confirmed ceiling).

  • Tree-sitter structural parse → symbols — Rust (repomap.rs::outline_rust); extensible per-language
  • gitignore-aware workspace walker (ignore crate, ripgrep's engine)
  • repo_map tool — each file → top-level defs + line numbers; verified E2E (oriented in Huginn's own source without reading files)
  • More language grammars: Python, TypeScript/JS added (repo_map + syntax-guard). C# deferred (tree-sitter-c-sharp 0.23 fails set_language at runtime vs tree-sitter 0.24 — needs a compatible grammar version); Kotlin deferred (community grammar ABI).
  • Tree-sitter syntax-validate edits before writing (moved from CP2) — syntax_guard rejects edits that regress a parsing file; only regressions gated.
  • Auto-load AGENTS.md / CLAUDE.md / .huginn.md at the workspace root into the system prompt (verified E2E); hierarchical per-dir discovery later
  • [→] Build a resolved importMap during the scan — deferred (optimization)
  • [→] mtime/fingerprint incremental cache — deferred (perf; matters at repo scale)
  • read_file summarized mode: large files (>600 lines) without a range return their structural outline instead of full contents (reuses outline); verified E2E. Unsupported large files capped + truncation note.
  • [→] Persist the repo map + inject a compact map at startup — deferred (on-demand repo_map covers orientation; revisit with CP9 headless core)
  • [→] Context budgeter: load only what each step needs — deferred (revisit when tasks span many files)
  • Done when: a task touching 5+ files in a real repo completes coherently.
  • Borrow: aider (repo-map docs), oh-my-pi (summarize, fs_cache, workspace walker), Understand-Anything (deterministic+semantic split, incremental fingerprints), plandex (project maps, 2M effective context)

CP4 — LSP / code intelligence ✅

Goal: give the agent IDE-grade truth, per language.

  • LSP client — lsp.rs: sync JSON-RPC over stdio (Content-Length framing), a background reader thread that demuxes responses (by id) / notifications / server-initiated requests (answers workspace/configuration, client/registerCapability, progress-token creation so rust-analyzer doesn't stall), $/progress load tracking, and pushed publishDiagnostics. Blocking request/notify via a Condvar.
  • Lsp manager: lazy spawn one server per language, reused; on each open() syncs current disk content (didOpen first, then didChange+didSave) so diagnostics reflect Huginn's own edits; binary-presence probe with a clear "not installed" error.
  • Tools: lsp_diagnostics, lsp_hover, lsp_goto_definition, lsp_find_references. Position input is model-friendly — pass the 1-based line + the symbol name and the harness finds the column (a small model can't count columns reliably).
  • Diagnostics double as a per-file verify step (richer/faster than a full build).
  • Servers configured: rust-analyzer (.rs) ✅ installed & E2E-validated; typescript-language-server (.ts/.tsx/.js), pyright-langserver (.py), gopls (.go) wired but not installed here.
  • Validated E2E on local the GPU server (Qwen-27B): broken c.bump() → lsp_diagnostics surfaced rust-analyzer's "no method named bump" → anchored edit → cargo build exit 0. Live #[ignore] test (lsp::tests::live_*) drives real rust-analyzer against Huginn itself.
  • lsp_rename — workspace-wide symbol rename: parses the server's WorkspaceEdit (documentChanges or legacy changes), applies each file's TextEdits via a tested edit::apply_lsp_edits primitive (0-based line + UTF-16 char → byte offsets, applied back-to-front, overlap-guarded), sandboxes every target under root, then formats. Validated E2E on local the GPU server: renamed greet→salute across def + 2 call sites in 2 files in one call, cargo build exit 0. Also hardened wait_for_load with an 800ms idle-debounce so cold cross-file queries don't race rust-analyzer's multi-phase indexing (was: first rename returned null, model retried; now: succeeds first call).
  • [→] lsp_code_action (quick-fixes/refactors: resolve + apply edits) — deferred, next opportunity.
  • [→] Safe file rename via willRenameFiles (updates imports/re-exports) — deferred (file rename, distinct from symbol rename).
  • [→] More servers installed + C#/Kotlin — as the languages come up.
  • Done when: the agent fixes an error using type info it couldn't get from text alone. ✅ met (diagnostics-driven fix + workspace rename).
  • Borrow: oh-my-pi (13 lsp ops), oh-my-openagent (lsp tool surface)

CP5 — Planning discipline ✅ (core complete)

Goal: plan before code on large tasks; don't drift.

  • PLAN.md generation for multi-step tasks — plan.rs (Plan/Phase + transitions
    • markdown render), plan_create tool writes an ordered phased plan to PLAN.md and activates phase 1.
  • Phase-by-phase execution, a git commit per phase — plan_advance marks the active phase done (with a summary), updates PLAN.md, and git.rs::commit_all makes a huginn: <phase> commit (only if there are changes). Config plan_commit_per_phase (default true).
  • Todo/phase tracking with enforcement (no idling mid-plan) — live plan state held in ToolContext; the agent loop, when the model tries to stop with phases still pending, pushes it to continue (capped at 3 nudges/turn). REPL /plan shows the plan.
  • Validated E2E on local the GPU server (Qwen-27B): "add a math module, wire it in, verify" → 4-phase plan → executed phase-by-phase → 4 per-phase commits → PLAN.md checked off → build+run correct. 3 unit tests on plan transitions/markdown.
  • [→] Plan review/iterate loop (interactive guards before execution) — deferred (the plan is visible via PLAN.md and /plan; a true approve/edit-before-run gate pairs with CP9 frontends).
  • [→] Interview-mode scoping for ambiguous requests — deferred (needs interactive turns; CP9).
  • Done when: a large feature is decomposed into a plan and executed phase-by-phase. ✅ met.
  • Borrow: r/LocalLLaMA u/ali0une (PLAN.md workflow), oh-my-openagent (Prometheus planner, Ralph loop, Todo enforcer)

CP6 — Self-review / critic ✅ (core complete)

Goal: quality beyond green tests — catch "compiles but bad."

  • Post-verify critic pass over the diff — review.rs + agent review_gate: when the model finishes, diff the turn (git::diff_since vs the start-of-turn snapshot; tracked changes + untracked new files; excludes PLAN.md/lockfiles/build-artifact dirs; capped 16KB) and run a review-hatted LLM call (no tools) over it.
  • Per-language quality rubric, loaded progressively by file extension — rubric_for emits only the checklists for languages present in the diff (Rust/Python/TS-JS/Go); borrowed from code-review-skill's progressive disclosure.
  • Severity-ranked findings (P0–P3) — lenient parse of P<n> file:line — issue lines
    • a VERDICT: line; blocking (P0/P1) always overrides a contradictory ship verdict.
  • Four review lenses in one pass (correctness / security / performance / maintainability) baked into the critic prompt. (True parallel multi-call adversarial verify — separate calls per lens — left for when a cheaper/dedicated critic model lands; single-pass is enough on local.)
  • Auto-fix P0/P1; surface the rest with a ship/no-ship verdict — blocking findings are fed back to the agent to fix (re-verify), looping up to MAX_REVIEW_ROUNDS=2; then the final ── self-review: SHIP ✓ / NO-SHIP ✗ verdict + remaining findings is appended to the answer. Config agent.self_review (default true). 6 unit tests (parse/rubric/lang-detect/filter).
  • Validated E2E on local the GPU server (Qwen-27B): (a) load_port() written with unwrap() → critic flagged the panic-on-bad-env edge case (P0/P1) → auto-fixed over 2 rounds into a proper Result<u16, PortError> with a custom error enum → shipped clean; (b) clean double(x) → shipped with only a surfaced P2 (i64 overflow), not over-fixed. Found+fixed a real bug in testing: untracked target/ leaked into review (no .gitignore) → added reviewable_untracked filter.
  • Done when: the critic catches and fixes an N+1 / missing SAFETY comment / edge case. ✅ met (unwrap edge-case).
  • Borrow: code-review-skill (per-lang rubrics MIT, severity labels, 4-phase), oh-my-openagent (hyperplan hostile critics), oh-my-pi (/review P0–P3 verdict)

CP7 — Model strategy & routing ⊘ (descoped by decision, 2026-06-09)

Decision (Oz): keep Qwen3.6-27B as the single workhorse for everything — "it is better than any other coding model we could run," and no dedicated coder model is wanted. That resolves CP7 rather than deferring it: the model-agnostic endpoint config already shipped in CP0 (any OpenAI-compatible base_url + optional key + HUGINN_* env overrides) already provides full model swappability with zero code changes, so there is no routing infrastructure worth building speculatively.

  • Model-agnostic swappability — delivered by CP0 config (sufficient for the chosen strategy).
  • [⊘] Dedicated coder model / its own llama-server — not wanted (27B is the pick).
  • [⊘] Category routing / per-role models / fallback chains — unnecessary with one model.
  • [→] Frontier-escalation policy (local-default, escalate only the hard turns) — the one piece that could still add value later (the "director" pattern); revisit only if Oz wants it. Trivial to add on the existing config (a second ModelConfig + an escalate signal).
  • Borrow (if ever revisited): oh-my-openagent (quick/deep/ultrabrain), oh-my-pi (roles, fallback chains); [[project_persona_model_routing]]

CP8 — Memory ✅ (core complete)

Goal: remember the codebase between sessions.

  • Project-scoped fact store (remember / recall) — memory.rs: plain-markdown <root>/.huginn/memory.md, dedup on append, keyword-filtered recall. Human-readable and commit-able. 2 unit tests.
  • Mental model loaded on the next session's first turn — at startup the remembered facts are injected into the system prompt (capped 4000 chars), so a fresh process starts already knowing them; the system prompt also tells the agent to remember durable facts.
  • [decision] standalone vs reuse Muninn's spine → standalone (no coupling, fully reversible; a Muninn-memory bridge stays a possible additive step later).
  • Validated E2E on local the GPU server (Qwen-27B), two separate processes: session 1 remembered "tests run via make verify (needs Postgres), not cargo test"; session 2 (fresh process) loaded it at startup and answered correctly, citing it as recalled from a prior session.
  • [→] Auto end-of-session summary (vs the agent remembering facts as it goes) — deferred; the remember-as-you-learn path covers the "done when". .huginn/ is excluded from review diffs.
  • Done when: the agent recalls a codebase fact learned in a prior session. ✅ met.
  • Borrow: oh-my-pi (Hindsight), Muninn memory

CP9 — Frontends & Muninn integration ✅ (Muninn delegation LIVE + TUI shipped)

Goal: reachable from anywhere; one core, many faces.

  • Separate the headless core from frontends — main.rs::build_agent() is the single construction path (resolve root + system prompt + project context + remembered facts + registry/LSP/plan + policy), shared by one-shot CLI, REPL, and the worker. One agent per job.
  • Muninn code_task tool — delegate over NATS (async, from phone/Matrix). Huginn side: huginn --worker = a JetStream work-queue consumer (worker.rs) that pulls jobs, builds an agent for the named workspace, works under the judge policy, and publishes the result back to the job's reply subject. Decoupled: the publisher only speaks NATS. Safety: ack-on-receipt + one-at-a-time (a coding job mutates a repo, so at-most-once beats double-run); workspace resolves strictly under workspaces_root (escape refused); optional shared token guards the auth-less LAN NATS. Muninn side: admin-only code_task tool (in coabai-muninn) publishes the job + spawns a reply waiter that posts the outcome into the room. [worker] (Huginn) / [tools.huginn] (Muninn) config. Validated E2E against the live JetStream (a self-hosted host): published a job → worker built an agent, created the requested file, self-reviewed (SHIP), published the result → received on the reply subject. Both crates committed.
  • Progress/result reporting back to Muninn (ping when done) — result envelope (ok/summary/error) on the reply subject; Muninn's waiter posts ✅/❌ to the room. (Live progress streaming during the job deferred; "ping when done" is met.)
  • Deployed & E2E-confirmed from Matrix (2026-06-09). huginn --worker runs as a systemd user unit on the workstation (~/.local/bin/huginn, ~/.config/huginn/config.toml, workspaces under ~/code, shared-token auth, linger on); Muninn (a self-hosted host) has [tools.huginn] enabled with the matching token. Full live round-trip: Oz messaged Muninn from Matrix → sub-agent routed to code_task → job over NATS → worker built an agent, wrote the file, self-reviewed (SHIP ✓) → result posted back into Oz's room ("✅ Huginn finished the task in huginn-test"). The done-when, met in production. (One mis-route en route: the sub-agent first tried local shell to find the repo; fixed by sharpening code_task's description — it's the ONLY path to a named repo and those repos aren't on this host. ~/code/huginn-test left as a safe E2E target.)
  • TUI shipped (ratatui). Decision: ratatui — it's Oz's main driver at the PC ("the way I interact with you"); Muninn-from-phone is for comfort/small tasks. First decoupled the agent from stdout — it emits AgentEvents over a channel (event.rs), consumed by either the console driver (one-shot/REPL/worker, byte-for-byte same dim log) or the TUI. huginn --tui (tui.rs): full-screen — status bar (model/workspace/ spinner), scrollable wrapped transcript with colored tool cards (cyan ⚙ start, red on error/blocked, green answer), bordered input, and a modal approval picker (←/→ or y/n/a) for Ask-policy commands. Agent runs in its own task; UI sends commands (turn/ plan/undo) in, consumes events + a turn-end signal out; approvals carry a oneshot the UI answers. pty-tested: renders all chrome, drives a full turn (input→agent→answer→ ready), exits clean. Binary at ~/.local/bin/huginn (release); worker re-verified on it.
  • [→] TUI v2 (later): syntax-highlighted edit/diff preview cards, collapsible cards, token meter, mouse, themes. v1 is the working cockpit.
  • [→] (Later) ACP editor mode.
  • Done when: "hey Muninn, fix X in dlejos" from the phone → Huginn works → ready ping ✅ (live in production), AND a real TUI cockpit for direct work ✅. CP9 met.
  • Borrow: oh-my-pi (4 entry points: TUI/one-shot/RPC/ACP), [[reference_nats_jetstream_queue]], [[project_muninn_coder]] (delegation handshake)

CP10 — Hardening & eval ◑ (run_command gate done; eval/recovery/packaging open)

Goal: safe unattended operation, measured progress.

  • run_command permission model — policy.rs: deterministic classifier (Safe allowlist of read-only/build/VCS-inspect tools, scanning EVERY &&/|/; segment so cargo build && rm -rf / is caught; Dangerous denylist incl. pipe-to-shell; else Unknown). Authorization lives in the AGENT pipeline (it has the model client + task context), gating run_command before dispatch. Four policies: auto (all), ask (default — Safe auto, else interactive approval with y/N/all, denied if no TTY), allowlist (Safe only), and judge — an LLM safety judge decides non-safe commands autonomously (Oz's Muninn pattern: deterministic fast-path → LLM judge → run; fail-safe to block). Config agent.command_policy. 7 unit tests. Validated E2E on local the GPU server: judge mode ALLOWed an in-task mv (Unknown) and ran it; headless ask mode BLOCKED mv/rm and the model fell back to the path-sandboxed file tools — no shell mutation leaked. Closes the long-flagged gap.
  • [→] True sandbox (containers/namespaces) — the classifier is heuristic defense, not isolation; real sandboxing is future work.
  • Session recovery (context-limit + API-failure auto-recover) — llm.rs: a single turn is dozens of round-trips against a local llama-server that can stall/restart/503, so chat retries transient failures (network / 429 / 5xx) with exponential backoff (4 attempts, 400→1600ms; terminal 4xx and parse errors don't retry). Context-window overflow is classified by error body not status (llama.cpp returns it as a 400 exceed_context_size_error) into a downcastable ContextOverflow; the agent's chat_recovering catches it and calls compact_messages to drop the oldest tool exchange — extending an assistant-with-tool_calls block over its tool results so a result is never orphaned (API invariant), protecting the system prompt + current task
    • last 4 messages — then retries, looping until it fits or nothing is safely droppable. 6 unit tests (overflow detection vs real llama.cpp/OpenAI phrasings, retry classification, downcast, compaction pairing/convergence/floor). Validated E2E: transient retry fired 4× with backoff against a dead endpoint then failed cleanly; the overflow heuristic confirmed against the GPU server's real wire error ("exceeds the available context size (131072 tokens)", matched 3 ways).
  • SWE-bench eval harness; baseline score on the local model, tracked over time — ✅ run via the official swebench harness driving Huginn (one-shot / --job headless path). SWE-bench Verified: 68.2% (341/500) on a local quantized Qwen3.6-27B, statistically tied with mini-SWE-agent on the same model (union 76.6%); harness-lever ablations measured on a frozen 39-instance subset. Full writeup in the README Benchmarks section; now also the exam that gates the L4 self-improvement loop.
  • Packaging / install / docs — scripts/build-static.sh produces a fully-static musl binary (ldd: statically linked, 15M stripped, drops onto any Linux x86-64 with no runtime deps; CC_x86_64_unknown_linux_musl=musl-gcc + the musl target). README rewritten: install (static / from-source), the model-endpoint requirement + HUGINN_* env overrides, the optional-tools table (git/LSP/formatters degrade gracefully), all four run modes, TUI keys, safety policy, workspace=cwd. `/.local/bin/ huginn` installed. Huginn confirmed fully standalone — Muninn/NATS are opt-in.
  • Done when: runs unattended safely and has a SWE-bench number trending up. (Safe-unattended ✅ via judge/allowlist; recovery ✅; packaging ✅; only the SWE-bench number is open.)
  • Borrow: plandex (controlled execution, rollback), SWE-agent (SWE-bench, ACI design), oh-my-pi (session recovery)

Reference implementations to read as we build

  • oh-my-pi github.com/can1357/oh-my-pi — ~27k-line Rust-core coding agent (MIT); our nearest blueprint. Read per-module when building CP2/CP3/CP4.
  • Pi / pi-mono github.com/badlogic/pi-mono — the leaner upstream of oh-my-pi.
  • aider aider.chat/docs — repo-map + edit-format leaderboard (read the docs, not the README).
  • mini-swe-agent github.com/SWE-agent/mini-swe-agent — minimalist reference matching our lean ethos.

Full idea catalogue with sources: kept in Kai memory reference_huginn_harness_ideas.