Voice Coding: Talk to Agents, Not Your IDE

Search voice coding and page one is dictation software. Turn on a microphone, speak, watch words land in a text box. That is a real category with a long history, and it is not what the people building coding agents mean when they use the phrase now.

Voice coding today means speaking a task to an AI coding agent in plain language and reviewing the code it produces, rather than speaking code into an editor. You are not dictating syntax. You never spell out punctuation. You describe the change in the same English you would use with a contractor, and the typing happens somewhere else.

Charlie Holtz, co-founder and CEO of Conductor and one of a wave of Y Combinator companies building AI coding tools, walked through a setup built entirely this way on Y Combinator's channel. Asked directly whether he still writes code today, his answer is one word: "No." He allows one exception, that "Very occasionally I will like edit Tailwind classes" or open an editor to change an environment file. Everything else is spoken.

Two things to hold while reading his setup. He is the CEO of the company that sells the app he is demonstrating, so the demo is also a sales argument. And one founder's workflow is an existence proof, not evidence about your team. What follows separates the parts that transfer from the parts that are him.

What voice coding means now

The distinction between the two meanings is the whole story, because they have opposite failure modes.

The older meaning is real and predates all of this. It grew out of accessibility work, largely for programmers with repetitive strain injuries, and it solved a genuinely hard problem: mapping speech onto a grammar of keystrokes, symbols, and casing conventions so a person could produce syntactically valid code without a keyboard. It works, and it is demanding, because programming languages are hostile to the mouth. Punctuation carries meaning, whitespace carries meaning, and identifiers are compressed words with no spaces in them. People who work this way are trained, not slow.

The new meaning inverts the problem. You are no longer producing code with your voice. You are producing an instruction, and instructions are already English. Asking an agent to look at the latest issue in your tracker and give you a rough pass at how it would solve it is not hostile to speech, because it is just a sentence. Voice coding works now, after two decades as a specialist skill, mainly because the target changed from a programming language to a natural one.

Holtz's framing of the trend is blunt: "We are all trying to talk to our computers more." The setup detail that makes it concrete is a gooseneck microphone he says he cannot live without, bought for twenty dollars, and the reason is not audio quality. It is that an open-plan office makes talking to your computer socially expensive. A close microphone means you can "lean over and whisper into Claude" without broadcasting to the room. His example of what he whispers is exactly as mundane as it should be: "Please merge PR 3475."

That is the most useful signal in the whole video. The unit of voice work is not a paragraph of specification. It is a short imperative sentence that would otherwise have cost four clicks and a context switch.

Why voice coding beats typing the same prompt

There is real evidence that speech input beats keyboard input on speed. A 2016 study by researchers at Stanford, the University of Washington, and Baidu measured English speech input on a smartphone at 161 words per minute against 53 for the keyboard, about three times faster, with a lower error rate.

Take that as directional and not as a finding about your workflow. It measured short English messages typed on a phone keyboard, using speech recognition from a decade ago, not prompts written on a mechanical keyboard by someone who types for a living. If you type at 100 words per minute, the raw throughput gap is much smaller than three to one.

The speed that matters is a different one. Speaking lowers the cost of starting a task, and in an agent workflow the number of tasks you start is the thing that compounds. Holtz's loop is a keyboard shortcut to open a new chat, a sentence spoken into the machine, and enter. Then he moves to another workspace while the first one runs. The prompt is not long. What changed is that the distance between having the thought and dispatching the thought went to nearly zero, so thoughts that were not worth typing out become worth saying.

That is also why the phone matters. He demonstrates speaking a feature request into his phone while away from his desk: "Let's add a new feature where I can change the theme to hacker mode." He taps a button, and "my computer starts working on it". Text on a phone keyboard is bad enough that nobody dispatches real work that way. Speech is good enough that they do.

The same logic explains the small edits. Rather than opening a file to adjust a component, he highlights the thing on screen and says what is wrong with it: "That button looks a little too wide." The correction is faster to say than to locate in code, which is the general shape of the win. Voice is best where describing the outcome is cheaper than specifying the change.

None of this requires exotic tooling. Holtz's dictation runs on a single keyboard shortcut and, in his description, "It's actually running a local model. It's running parakeet." Parakeet is NVIDIA's family of open speech recognition models, released under a permissive license and built to run on your own hardware rather than in someone else's data center. The rest of his setup is deliberately plain: he says "I always use fast mode. That's not a default." and that he uses a documentation server for library references, and otherwise runs things as they ship.

Where voice coding loses to the keyboard

The honest version of this workflow includes an escape hatch, and Conductor shipped one with a name that tells you how the team feels about it. Holtz shows a mode "we call caveman mode" that lets you type with your keyboard and edit a file directly, and adds that "it's called caveman mode for a reason".

The joke is doing work, but the escape hatch is real, and knowing when you need it is most of the skill. Three cases where the keyboard is simply better:

Exact strings. File paths, environment variable names, an identifier in a specific module, a version number. These are precisely what dictation gets wrong, because they are not words. The failure is also the dangerous kind: a misheard sentence usually reads as a strange instruction and you catch it, but a misheard identifier reads as a perfectly ordinary instruction pointing at the wrong thing. Saying a variable name out loud and hoping the transcription lands is slower than typing eleven characters, and riskier.

Single-character changes. A boolean flipped, a port number, a version pinned. Describing it takes longer than doing it, and involving an agent adds a review cycle to a change you could have verified by looking at it.

Anything you are still thinking about. Speech is a committing medium. You say the sentence and it is sent, with no draft state and no cursor to go back to. Typing lets you revise mid-thought, which matters when the task is genuinely unclear. If you cannot say what you want in one sentence, that is usually a signal the task needs decomposition rather than dictation, which is the argument for writing it down first that we made in spec-driven development.

The practical version of all three is a rule about sentence shapes rather than about tools. Say the outcome, type the coordinates.

The guardrails voice coding needs

Running agents in parallel needs guardrails whatever your input method, and the full set is the subject of AI agent orchestration: isolation per agent, human-owned architecture, marked no-fly zones, review before merge. Speech does not change that list. It changes the weight of three items on it.

The branch gate stops being a nicety, because nothing else catches a misheard sentence. You did not proofread the prompt; there was no prompt to proofread. Conductor enforces the gate in the product rather than by discipline: by design "you can't edit files directly", so every change becomes a branch, then a pull request, then a merge you approve. Holtz also says "we always run Claude in dangerously accept all permissions", which is the tool's default. That is defensible inside an isolated worktree whose only exit is a reviewed pull request, and indefensible without it. The isolation is what buys the permission setting, not the other way around, and copying the second half of that configuration without the first hands a fast, tireless process unreviewed write access to your main branch.

The boundary has to live in the file, because a spoken task carries less context. Typed prompts drift long. Spoken ones stay short, so everything the agent needs but you did not say has to already be written down. Holtz's team marks lines in the repository "do not touch if you are an AI" and "this is for human eyes only" rather than relying on a reminder in a prompt, and he states the general rule as "don't let the AI be your architect." His justification for the markers is a feedback-loop argument rather than a code-quality one, and it is the more interesting version: if you are careless, "it sees bad code and then it writes more bad code as a result." A codebase is also a prompt. He volunteers that they would do one thing differently, building the core of the app around human-written APIs and contracts that agents contribute to less, and admits the boundaries today are murkier than he would like. Drawing the line is ongoing work, not a one-time configuration.

The microphone is a disclosure surface, and this guardrail belongs to voice alone. As a CISSP, I would want two decisions written down before anyone on your team runs this way. An always-listening microphone in a shared office hears the hallway conversation as well as the sentence you meant it to hear, so use a key you hold. And if transcription happens on a third party's server, the thing you whispered about an unannounced feature is now in a log you do not control, so prefer a model that runs locally. Neither decision costs anything if you make it on day one. Both are awkward to retrofit.

What voice coding does not fix

The bottleneck is not input. It is review, and voice makes review harder before it makes anything easier.

Watch what Holtz actually does between dictated tasks. He opens a finished workspace, reads the diff, and where the agent has missed he leaves a comment on it the way he would on a colleague's pull request. His description of the hit rate is not triumphant: "quite often Claude won't get things exactly right". That reading and commenting is the work, and speaking does not touch it.

So the arithmetic gets worse before it gets better. Dispatching work gets much cheaper. Reviewing it gets no cheaper at all. The rate at which you can generate pull requests rises while the rate at which you can responsibly approve them stays exactly where it was, so the queue grows and the temptation to skim grows with it. The failure arrives quietly, and the first symptom is approving a diff you understood the summary of rather than the code of.

There is a second-order version worth naming. A workflow that makes it trivially easy to request work is a workflow that makes it trivially easy to generate code nobody needed. Holtz's team is deliberate about exactly this: "we're not big on lines of code" and "we try and keep the lines of code minimal". They spend on inference without apology, and he puts a number on his own peak, twenty two thousand dollars of tokens in one month, back in July 2025 when they were starting Conductor and on a previous generation of models. What they will not spend is diff.

The honest response is to cap yourself rather than the tool. How many agents you can hold and review is a working-memory question before it is a tooling one, and voice raises the pressure on that number without raising the number. Deciding it deliberately is the same discipline that separates agentic engineering from vibe coding, and the same reason keeping agents inside a blast radius you chose is the core of building software with AI agents.

The last thing voice does not fix is the quality of your standing instructions. Holtz's team spent real time on their agent instruction files, which run to a few hundred lines, and one line from theirs does more work than most style guides: "We're a startup. You're probably used to writing enterprise code, but that's not how we do things around here." Because a spoken task is short, the file is now carrying most of the context, which raises the return on getting it right. That is worth reading alongside what actually belongs in a system prompt.

Where this is heading, and what to discount

Holtz's larger claim is that the artifact is changing. In his framing, "really what matters is your prompts", because when better models arrive you rerun the prompt and get new code, so the old code mattered less than it seemed to. He expects that "the agents are going to run for 10 times longer" without intervention, and on machines you do not own.

Discount the specific multiplier. The direction is more defensible than the number, and it has an implication you can act on now: if the durable asset is the instruction rather than the output, then the things worth version-controlling and reviewing carefully are your specs, your instruction files, and your architectural decisions, while the generated implementation is more replaceable than your instincts say. That is a real change in where to spend attention, and it does not require the timeline to be right.

One counterweight, from Holtz himself, on why he does not just run all of this in a terminal: "I think humans are spatial visual creatures". The interface still matters because you are the constraint. A workflow where you speak into the void and receive a wall of text is not an improvement over typing. A workflow where you speak, and can see at a glance what several agents are doing and which one needs you, is.

What to do this week

1. Install a dictation tool bound to a key you hold, and prefer one running a local speech model. Not an always-on microphone, for the office-noise reason and the disclosure reason both.
2. Spend three days speaking only task starts. Do not try to dictate code or long specifications. Say the one-sentence version, send it, and go do something else. This is the habit that produces the gain.
3. Measure your own error rate before you trust it. Dictate ten real task sentences and count how many needed a correction before you would have been willing to send them. That number, not a benchmark, tells you where your line between speaking and typing sits.
4. Write down the shapes you will always type. File paths and identifiers, config values, anything where being one character off produces a plausible-looking instruction pointing at the wrong thing.
5. Confirm a misheard sentence cannot reach your main branch. Every agent on its own branch or worktree, every change arriving as a pull request you read. If an agent can write to main today, fix that before you make input three times faster.
6. Move the sentence you keep re-saying into your instruction file. Whatever you find yourself repeating out loud on every run is the thing that should be written once and read automatically.

The pattern under all six is that changing the input method only pays off if the review gate holds, which makes this an operating question rather than a tooling one. Deciding those defaults deliberately, across engineering and everything else, is what the AI Operating System for Startups is built around, and the engineering half of that picture starts with our pillar on AI for startups.

Sources

• Conductor CEO Charlie Holtz Walks Us Through His AI Coding Setup (Y Combinator), the walkthrough this article distills.
• Charlie Holtz, co-founder and CEO of Conductor, which he started with Jackson de Campos and which announced a $22 million Series A co-led by Spark Capital and Matrix.
• Speech versus keyboard input speed: Stanford's report on the 2016 Stanford, University of Washington, and Baidu study and the paper itself, which measured 161.20 words per minute for English speech against 53.46 for the keyboard on a smartphone.
• NVIDIA's technical blog on the Parakeet speech recognition models, for what the local model in that setup is.