Notes from the team.
Technical deep dives, open-source releases, and developer guides for on-device AI.
Barge-in and interruption handling for on-device voice agents
Barge-in is the hardest part of a voice agent to get right: hearing yourself over your own TTS, deciding an interruption is real, and stopping in a couple of audio frames. Why the DIY Whisper + llama.cpp + Piper stack breaks it, and how an on-device C++ loop fixes it.
Read→Time-to-first-token: why voice agents live or die on latency
A latency budget for real-time voice agents: the 200ms human turn-taking rule, where cloud pipelines spend 600ms–1.7s, and how on-device buys it back.
Read→How to build an offline voice assistant on NVIDIA Jetson Orin
A builder's guide to running a fully offline voice agent on Jetson Orin — local STT, an SLM, and TTS, with real latency numbers and where the DIY Whisper + llama.cpp + Piper stack breaks down.
Read→Why we ripped cloud voice out of our robots: from Deepgram + OpenAI + ElevenLabs to on-device
We started building robot voice the easy way — Deepgram, OpenAI, and ElevenLabs glued together in Python. It demoed great and fell apart in the real world. Here's everything that went wrong, and why we're rebuilding the whole stack on-device.
Read→turboquant.cpp: near-optimal vector quantization in 400 lines of C++, no training required
We open-sourced turboquant.cpp, a C++23 implementation of TurboQuant: compress embeddings to 1-4 bits per coordinate with provable distortion bounds — no training, no codebooks.
Read→