2026.02.14
Building HOWL, a voice agent for the G1.
What it took to make a voice interface feel like a tool for robotics work rather than a demo layered on top.
HOWL started as a demo — 'look, you can talk to the robot' — and became a tool we actually use during lab sessions. The line between demo and tool is whether it saves you time when something is going wrong. HOWL does.
What HOWL does
- Voice-activates a command set for G1 state queries (joint positions, battery, error codes)
- Transcribes spoken experiment notes into structured JSON logs
- Reads sensor values aloud when both hands are occupied with the hardware
- Confirms motion plans before execution ('Play back episode 3, confirm?')
Architecture
# STT: Whisper base (local, ~150ms latency on M3)
# Intent parsing: GPT-4o-mini with function calling
# TTS: ElevenLabs (low-latency streaming mode)
# G1 bridge: ROS2 topic subscriber + action server
Mic → Whisper → GPT-4o-mini → ROS2 action → TTS responseThe constraint that shaped the design: the lab is loud. Wheeled carts, servo hum, AC. Wake-word detection needs to be aggressive about false positives in that environment, or you end up triggering robot motion during casual conversation.
What didn't work
- Cloud STT (Deepgram) — too slow for conversational latency over lab wifi
- LLM routing for all commands — over-engineered; keyword parsing is faster for ~20 known commands
- ElevenLabs streaming on ROS bridge machine — audio card conflicts, ended up running TTS on a separate Pi 5
- Push-to-talk — hands are always occupied; true wake-word was the only viable UX
The name: HOWL stands for Hands-Occupied Workflow Layer. For robotics-adjacent voice interfaces, reliability beats latency. Users tolerate 300ms lag; they won't tolerate a command being missed when the robot is moving.
↳ Reliability beats latency. Users tolerate 300ms lag; they won't tolerate a missed command when the robot is moving.