Claude Voice

There have been a few moments in the development of AI that caught the attention of the whole world. Stable Diffusion making unbelievable images out of a sentence. ChatGPT shipping on GPT-3 and finally answering useful questions. Claude Code arriving and unleashing agentic coding. Every one of those I watched happen. There's one landmark I got to build for myself: a truly agentic voice agent.

The peak of the AI revolution caught me working on a farm, farther from a computer than I had ever been. Everyone in tech was obsessed with Claude Code. So obsessed that they were driving remote agents from their phones, firing off coding tasks and coming back to shipped work. I felt disconnected to the point of depression, because my work meant my eyes and my hands were occupied with the task in front of me. I was covered in dirt. I was grimy. I couldn't just pull out my phone. What I did have was my ears and my voice, and I needed a way to work with an agent that used only those.

The obvious answer already existed, and it wasn't close to enough. Claude and OpenAI both had voice agents by then, and they were pretty good at holding a conversation. But their capabilities were neutered. They couldn't use a computer, they were bad at search, and they did very little thinking. They were built to be conversational, not complex and capable. Contrast that with Claude Code and Codex, which were incredibly capable and had no voice at all. What I needed was a bridge between the two.

The idea came into focus when I came across Claude channels, a way to pipe asynchronous data into a conversation with a Claude agent. What if I used speech-to-text to talk conversationally, in a persistent conversation, and fed that straight into a live Claude Code session? Channels ended up being the inspiration rather than the implementation. They weren't stable enough yet to build on. But they gave me the design, and I set out to build it.

The first thing I tried was a channel and a little web app to pass voice through, just to see if I could get something decent. I quickly realized that building my own voice bot and my own networking to carry audio down to Claude was a losing battle. So I asked a better question. What tool already solved the problems I didn't want to solve? What could hop into a persistent call and talk back and forth on every device I own? Voice chat apps. Discord was perfect for this. It had already handled audio capture, networking, identity, and presence. All I had to do was write a bot that sits in the middle of the conversation, catches my voice and turns it into text, and does the opposite every time Claude sends a response back.

I cannot explain how exhilarating it was the first time a conversation came through and it just worked. I joined a Discord channel and my bot joined alongside me. I'd named it Gary, for giggles. I started talking. The server picked up my speech, sent it to Whisper, got back the transcript, and handed it to a real Claude Code session on my machine at home. Claude did its thing. The response went through text-to-speech, and Gary's audio piped into my ear. Nothing to install, nothing to open. Now, I won't say this is perfect, and the biggest reason is that it isn't conversational. It doesn't feel like sitting in a chatbot, and it shouldn't, because Claude goes out and does useful things. Those things sometimes take a while. Then it comes back and tells me what it did, or comes back with a question. Two workers, each with their own job, checking in over a shared radio. It did exactly what I wanted, and it was so exciting.

Getting it to work well was harder than getting it to work. Voice forces problems a screen never shows you. When Claude spends four minutes refactoring, a chat window shows a spinner. A voice call just goes silent, and sitting in silence for five minutes not knowing whether an error had taken the bot down was anxiety-inducing. So I built ways to check its pulse. The server streams Claude's tool activity and a small model compresses it into one spoken sentence under twelve words, "running tests and fixing a config file," on an interval I control. For state changes too fast to narrate I synthesized audio cues in code, raw sine waves: a rising three-note chime for ready, a heavy octave drop for down, a single bell as an optional heartbeat. You learn the system's health by ear, which is the right sense to design for when the whole premise is that your eyes are busy. I added slash commands in the Discord channel to switch which project we're working in, change the model, set the level of effort, adjust the speaking speed, and toggle the narration. And I set up a dedicated text channel as a running log of everything, so a message I missed can be read back later and a failure can be debugged after the fact.

The problem that took the most thought wasn't audio. It was continuity, because I reach these conversations from several directions: Gary by voice from the field, a browser client from my phone, the plain CLI at my desk. Early on, each interface tracked its own session and resumed its own conversation, and they drifted apart. The voice bot would pick up a stale thread while the terminal was three exchanges ahead. The fix became a principle I wrote into the project docs: no interface owns the conversation. Claude Code already persists every session to disk, and its continue behavior picks up whatever was touched last, no matter who touched it. So I deleted the per-interface bookkeeping and made every window, voice or text, follow the most recent thread. Walk away from the keyboard mid-task, get in the truck, say "keep going," and Gary picks up the exact sentence the desk left hanging.

The largest problem, and the one that's still not solved, was faults. Keeping the network consistent and recovering from failures was the hardest part of the build, because the operator is two miles from the keyboard. If the server dies while I'm in the field, the project is worthless. So the server supervises itself. A scheduled task acts as a watchdog and restarts it within five minutes if it ever dies. It watches its own source file and gracefully restarts between conversations when the code changes, which matters because I routinely have Claude edit the server through the server, by voice. And if a bad edit crash-loops it, twice in under ten seconds, the supervisor restores the last known-good copy of the code and beeps. That restore isn't hypothetical. It fires constantly, and the evidence points at Discord more than at Claude. The worst days were the ones where the bot terminated its own processes, or failed in a way I couldn't recover from without walking back to the house.

It's still a little rough, and I'm genuinely surprised none of the major labs has launched a feature like this. But it truly is magical to talk to an agent through voice alone and have it go do the work. Since the original build I've swapped in better voice models, xAI's among them, and it is staggering how fast and pleasant these conversations have become. The payoff ran through the content program on the farm's website. By the end of that push I had dictated hundreds of thousands of words of bespoke site content, drafted by Claude and edited line by line by voice, without ever sitting at a computer.

I don't know what the future of AI looks like, or what the user experience of it will be. I can tell you it's going to be different. Conversation as the primary interface is going to change how we do everything. It will change the form factors of our hardware. It will reduce the need for complex visual interfaces. And it will let us do work we couldn't do before, because our hands were greasy and grimy, or because we were somewhere it would have been dangerous to look at a screen and touch a button. I know, because I've been doing that work from a field.