Run a fully local multilingual voice assistant for Home Assistant

A Redditor runs a home-built voice agent on one RTX 3060 that answers Home Assistant commands in 500 to 1,000 ms, switches language on its own, and hands harder requests like playing a song to a local LLM.

Done withCustom agentinHome Assistant· Nov 13, 2026

What they did

u/liampetti built Fulloch, a hobby voice assistant that runs entirely on a home PC with an RTX 3060 and 12GB of VRAM, and shared a video of it working with Home Assistant. Speech is transcribed live by the Orukeet model. A spoken command first goes through a regex fast path, then Home Assistant's HassIL intent matcher, and only if both miss is it passed to a local 9B LLM that can call tools. A language detector picks the language of what was said and swaps the Pocket TTS voice weights to answer in it. The satellites are a Raspberry Pi 3 with a conference speaker and an ESP32-S3 screen fitted into the shell of an old Echo Dot. The code is open on GitHub.

How it went

Commands caught by the fast path act in 500 to 1,000 ms after he stops speaking, depending on transcription timing and network. Seven languages overlap across the speech and voice models in his setup, and ten with a full Qwen3 pipeline on 16GB of VRAM. A request to play music was slow in the video: it missed the fast path, so the LLM searched Spotify, took the track link and sent it to his Sonos speaker through Home Assistant, while also having to re-authenticate because nothing had played that day.

Worth knowing

Anything that falls through to the LLM is much slower than the fast path, so common commands are worth adding as patterns. If the language is unknown it falls back to the default. New voice weights download on first use, so the first reply in a new language uses the default voice. He priced the hardware in a separate blog post.

Try it

With Custom agent· 5 parts to fill in

Help me set up a local voice assistant for [my Home Assistant instance] on [my GPU and VRAM]. Use a fast path for common commands like [lights, timers, scenes] first, then Home Assistant intent matching, and only pass what both miss to a local LLM that can call Home Assistant services. Detect which of [my household languages] was spoken and reply in it. Target under [number] ms for fast-path commands and measure it. Stop and ask before exposing anything to the internet or changing existing automations.

Square brackets are the parts to fill in. The prompt is ours; the story is theirs.

Where they said it

Related