Muninn

A personal AI agent that lives in your house, on your hardware, on your terms.

Muninn is the brain of a self-hosted assistant: persistent memory, multi-step tool reasoning, identity-awareness, and proactivity — running as a single Rust binary against your own local LLM. It talks over Matrix (end-to-end encrypted), a terminal UI, a browser, or voice. No cloud, no data leaving your network, no account with anyone.

Named for the GPU server's raven — the one that flies out, gathers what's happening, and brings the knowledge home.

The idea: brain, body, ears & mouth

Most "smart home + AI" projects try to cram an assistant into the home-automation platform. Muninn takes the opposite stance: keep each layer doing the one thing it's great at, and put the thinking somewhere it can actually grow.

                      ┌──────────────────────────────┐
                      │   MUNINN  ·  the brain        │
                      │   cognition · memory ·        │
                      │   identity · tool orchestration│
                      │   · proactivity               │
                      └───────┬───────────────┬───────┘
                  controls    │               │   speaks / listens
                              ▼               ▼
        ┌─────────────────────────┐   ┌──────────────────────────────┐
        │  HOME ASSISTANT · body  │   │  VOICE STACK · ears & mouth   │
        │  device state · control │   │  wake · STT · TTS · satellites│
        │  the deterministic floor│   │  (Wyoming / LVA / your own)   │
        └─────────────────────────┘   └──────────────────────────────┘

Three rules keep the boundary honest:

1. The body works without the brain. A light still turns off through plain Home Assistant even if Muninn (or the LLM) is down. The brain is augmentation, never the only path.
2. The brain owns no I/O it can borrow. If a capability would make sense in someone else's house with a different brain, it belongs in Home Assistant or the voice layer — not here. Muninn only holds what's Muninn thinking.
3. One boundary per limb, swappable. Control, voice, and spatial context each cross a clean interface, so any layer can be replaced without touching the others.

The result is an assistant that remembers, recognizes who's talking, and notices — without being a single point of failure for your house, and without shipping your life to anyone's cloud.

What it does

Full inventory in FEATURES. Highlights:

Many tools are optional integrations — they only register when their config block is present and their backing service (Home Assistant, Bazarr, Jellyseerr, a Mastodon instance, a self-hosted OSM maps stack, Prometheus/Alertmanager, etc.) is reachable. The core (memory, sub-agent loop, web search, shell, file/code tools) needs none of them.

• Four surfaces, one memory — Matrix (E2EE bot), TUI (terminal), Web (axum + OIDC), and a Voice HTTP receiver, all sharing one store.
• Persistent memory — SQLite + FTS5 + sqlite-vec (schema v31): hybrid retrieval (RRF + 1-hop graph), Ebbinghaus decay, episodic summaries, fact consolidation, correction archival, rolling conversation summaries, per-user dialectic profiles (Honcho-lite), an optional reflective "dream" pass, per-user / per-topic scoping.
• Sub-agent architecture — the main LLM sees only one tool (agent, ~150 tokens); the real tools run in an ephemeral sub-context. ~70% fewer tool-definition tokens per turn, and tools scale without bloating the conversation.
• Identity-aware — a registry bridges Matrix IDs, voice speaker-IDs, and devices into one canonical user, with role-based tool permissions and an onboarding/audit path for unknown speakers.
• Tools — weather, home query/action, music, photos (Immich slideshow), presence (who/what is where), calculate, web search, shell (2-gate safety), camera, image generation, speak, transcribe, file read/write/list, code run, project, memory lookup, scratchpad (per-task working memory), progress, schedule, email, calendar, contacts, youtube, research, subtitles (Bazarr), mastodon, geocode / route (self-hosted maps), media_request / music_request, code_task (delegate coding to Huginn), and self-modification tools (identity / workspace_edit / self_restart).
• Music — full Music Assistant control: play-by-name, favorites, discover, radio, genre/mood/era, volume, shuffle, like/dislike, room targeting, synced multi-room. See MUSIC.
• DJ mode — opt-in radio host: a background worker speaks a short, language-matched intro over each new song (ducked, not muted), grounded in a quick web search for real artist facts; personas + rotating "vibes" live in an editable markdown file. Needs Music Assistant + TTS.
• Email — IMAP read/search/flags + SMTP send over one or more mailboxes (imap/mail-parser/lettre, default gateway a Mailcow server): list, read, search, send, threaded reply (reply-all), archive, delete, plus extract_events — scan mail for appointments and propose calendar events (confirm-first, never writes the calendar itself; Message-ID dedup ledger skips already-processed mail). Admin-only.
• Calendar — appointments with timed reminders Muninn delivers to the conversation (default 1 day + 1 hour before, overridable). One shared calendar with per-event owner + public/private visibility; recurring events (iCal RRULE, DST-safe). Local SQLite is the source of truth, best-effort mirrored to SOGo CalDAV so events show in real calendar clients. Admin-only.
• Contacts — CardDAV name→email/phone lookup + add over the SOGo address book (so "email a contact" resolves the address): search/list/add, hand-rolled addressbook-query REPORT + vCard parse. Admin-only.
• YouTube — summarize a video from its transcript: yt-dlp pulls auto-captions, a VTT cleaner produces a transcript, an LLM writes a TL;DR + key points. URL host-validated to YouTube. admin/adult.
• Research — deep research in one call → a cited markdown brief: plan → SearXNG search → fetch (SSRF-guarded) → synthesize, bounded and backgroundable via !bg. admin/adult.
• Presence & family map — a co-location engine fuses everyone's HA device_trackers (with GPS-freshness so stale fixes don't ghost), answers "is anyone home? / where is X? / are X and Y together?", and feeds a live people layer + custom drop-pin places to the self-hosted map.
• Open-house mode — opt-in ([tools] open_house): any voice — recognized, guest, or unknown — can control the house (lights, music, photos, presence), while the family-private surfaces (camera, memory) stay gated.
• Subtitles — subtitles drives Bazarr to fetch subs for a movie or TV show on demand (re-checks what actually landed), then triggers a Jellyfin scan; a whole-season ask runs in the background and posts results back to the room.
• Maps — geocode (worldwide, Photon) + route (car/bike/foot, North & Central America + Europe) over the self-hosted OSM stack; route attaches a traced map snapshot of the journey.
• Mastodon — native tool that generates an image and posts the daily "On This Day in History" status (3×/day via the scheduler).
• Delegate coding to Huginn — code_task hands runnable code jobs to the Huginn builder over NATS (new workspace or existing repo) with live progress + mid-flight steering, opening an interactive child tab in the Hermes console.
• Resilient brain — DeepSeek primary with a fallback brain on a separate host and round-robin peer pools, so a single LLM box going down doesn't make Muninn brain-dead.
• Learns from failure — every sub-agent run is recorded to an outcome ledger; an opt-in nightly review distils recurring failures into "constraint" memories that surface back through retrieval (!failures / !review-failures).
• Infra alerts + self-heal — an Alertmanager → Matrix relay with generic, config-driven auto-remediation (first use: re-kicking a wedged Onkyo AVR off WiFi).
• MCP client — stdio + HTTP / streamable-http transports; admin-gated.
• Observability — ⏱ timing logs, credential scrubbing, output garbage-guard, loop detector, per-tool progress lines, !stop/steer mid-task, trajectory export + replay (muninn replay re-runs a captured turn so you can A/B a prompt change against the real LLM).

Sub-agent architecture

The main LLM sees one tool definition (agent). When it needs to act, it delegates to a sub-agent that runs in a throwaway context with the full toolset. This saves ~70% on tool tokens per request, scales as tools are added, and isolates tool execution from the conversation.

Try it in 5 minutes (lean core)

You don't need Matrix, Home Assistant, or a voice stack to kick the tires — just an OpenAI-compatible LLM endpoint (Ollama, llama.cpp, vLLM, anything). The TUI runs the same brain locally with bundled SQLite memory:

# Prereqs: Rust toolchain + libsqlite3-dev libssl-dev pkg-config (+ python3 for the calculate tool)
cp muninn-tui.toml.example muninn-tui.toml
$EDITOR muninn-tui.toml          # set [llm] api_url + model

cargo run --release -- --tui

That's the whole lean core: an LLM endpoint and a SQLite file. Everything below — Matrix, Home Assistant, voice, image generation, web search — is an opt-in config section. Leave a section out and that limb stays dark.

Embeddings (Ollama) are optional — without them, memory retrieval degrades gracefully to FTS5 full-text search.

Full setup

Muninn ships three surfaces in one daemon — Matrix (always-on bot), TUI (terminal chat), Web (browser, behind OIDC) — plus an optional voice receiver. Templates live in the repo; copy each, drop the .example suffix, fill in secrets. None of the real configs are tracked (all gitignored to keep credentials and your topology out of git).

Daemon (Matrix + Web)

cp muninn.toml.example muninn.toml.local
$EDITOR muninn.toml.local          # Matrix creds, LLM URL, optional [tools]/[web] blocks

cargo build --release
MUNINN_CONFIG=muninn.toml.local ./target/release/coabai-muninn   # foreground test

# or as a user service:
cp coabai-muninn.service.example ~/.config/systemd/user/coabai-muninn.service
systemctl --user daemon-reload && systemctl --user enable --now coabai-muninn
journalctl --user -u coabai-muninn -f

# subsequent rebuild + restart (set MUNINN_HOST in deploy.env first):
./deploy.sh

Web (optional)

The [web] block adds a browser surface inside the daemon — bearer auth against any OIDC issuer with a userinfo endpoint, bound to 127.0.0.1:7792 by default. Flip the bind to 0.0.0.0 or a VPN IP once you trust your auth setup.

Configuration

See muninn.toml.example for the annotated reference (and the docs server at /config).

LLM sampling (per-request)

Both need llama.cpp --jinja for tool-template parsing.

Commands

An embedded docs server (axum) starts automatically at http://localhost:7790.

Status

This is a personal project being prepared for a wider audience. It has a lot of moving parts by design — that depth is the point — but the lean core (TUI + an LLM + SQLite) runs on its own, and every integration is opt-in. Expect rough edges in setup ergonomics; issues and notes welcome.

License

Not yet chosen for public release — see OPEN_SOURCE_PREP. Currently private.