How to Build a Voice-to-Task Agent for a Service Writer

A service writer has both hands full and about nine seconds.

The customer is walking to the waiting room. The next car is already on the drive. Something just got said that has to be remembered by this afternoon, and there is no keyboard in reach and no time to find one.

That gap is the whole problem. It is also the best first agent an operator can build.

I sell cars for a living. Twenty years in automotive retail, no tech background, no computer science degree. The service drive is fifty feet from where I stand, and the writers over there drop the same small handoffs I used to drop on the sales side. So I built a voice-to-task agent for that exact moment. This is the build log. The stack, the bugs, and the fixes.

What does a voice-to-task agent actually do for a service writer?

It closes the gap between a thing being said and that thing being written down.

Nothing more than that. It does not talk to the customer. It does not diagnose the car. It does not replace the writer's judgment about what matters. It listens to the writer, for two seconds, and turns what was said into a task that shows up where the writer will see it later.

Here is the real moment it is built for. A customer mentions on the way out that they will be back Thursday for the tires they passed on today. The writer cannot stop to log that. So today it lives in the writer's head until it does not, and Thursday the customer shows up and nobody is ready.

The agent replaces the head with a five-second voice note. Tap, speak, done. The task lands in a follow-up list with the customer, the car, and Thursday already pulled out of the sentence.

That is capture. A voice-to-task agent is a capture-and-route tool. If you have read anything else I have written, you have heard me say this. A chatbot is a conversation. A capture is different work, and for a floor operator it is usually the higher-leverage build. I laid out the difference in how to evaluate an AI agent for your sales floor.

What is the stack, conceptually?

Four pieces. Each one does exactly one job. That is the whole design.

Mic capture (phone web app)

→

Transcribe audio to text

→

LLM parse intent + fields

→

Route to destination

→

Writer sees it later

Mic capture. A simple web app the writer opens on a phone. One button. Tap it, it records, tap again, it stops and sends the audio up. No app-store install, no login gymnastics on the drive. Just a page with a mic.

Transcription. The audio goes to a speech-to-text step that returns plain text. This is a solved problem now. You send audio, you get a string back. The browser's own recording APIs and the codecs behind them are documented well at the MDN MediaRecorder reference, which you will end up reading more than you expect.

LLM parse. This is the one interesting step. You hand the transcript to a model with a tight instruction: pull out the intent and the fields. Intent is what kind of task this is, a follow-up or a parts note or a callback. Fields are the specifics, the customer, the car, the date, the thing to do. The model returns structured data, not prose.

Router. The structured task goes to its destination based on the intent. A follow-up goes to the follow-up list. A parts note goes to the parts queue. The router is a small switchboard. It is also, quietly, the piece that decides whether the whole thing is trustworthy.

You can stand all four up on free and hobby tiers to start. Once it is live for one writer it runs on less than fifty dollars a month in hosting. That is not a teaser number, it is what a single-operator agent actually costs to keep on.

Why build the routing layer before the cool part?

Because the cool part is useless if the task lands in the wrong place, and the router is where landing is decided.

The mic and the transcription feel like the product. They are not. They are the front door. The value only shows up when a captured note reliably becomes a task the writer trusts enough to stop keeping in their head. The moment one note goes to the wrong list, the writer stops trusting the whole thing, and an untrusted capture tool is just a slower notepad.

So I build auth and the router first. Before the mic looks good. Before the parse is smart. I make sure that a known-good structured task can travel from the router to the correct destination and show up where a real person will see it, every time, with a record of it having happened. Only then do I put the voice on the front.

I wrote a whole build log on why the routing layer comes first, across all my agents, in why I built the routing layer before any of the agents. The short version is that the plumbing is the product and the faucet is the demo.

The mic is the demo. The router is the product. Build the part nobody claps for first.

What breaks first, and how do you fix it?

Three failures, in the order they hit me.

One. Mobile audio breaks the moment a real phone opens it. On my laptop the recorder worked on the first try. On an actual phone in the service drive it threw a codec error, or the mic permission never resolved, or the recording came back empty. Desktop and mobile browsers do not agree on which audio formats they will record, and a phone in a loud bay is a different machine than the one you tested on. The fix is unglamorous. You detect what the device can actually record instead of assuming, you handle the permission prompt as a real state and not an afterthought, and you test on the oldest phone you can find, not the newest. The MDN docs linked above are where you go to learn which format the device in your hand will actually give you.

Two. The intent parser mis-reads the note. The writer says "hold the tires for the blue Tahoe, customer is back Thursday," and the parser files it as a callback instead of a parts hold, or drops the Thursday. Language is messy and a service drive is loud, so the transcript arrives imperfect and the model has to work with it. The fix is two-part. Tighten the parse instruction so the model returns a confidence signal and a normalized set of fields, and when confidence is low, do not guess. Route the note to a review lane the writer checks once, rather than silently sending a wrong task with full confidence. A wrong task with confidence is worse than a task the writer confirms.

Three. The silent lost note. This is the one that will actually hurt you. A note gets captured, the transcription succeeds, and then something downstream fails, and the task never lands anywhere. No error. The writer thinks it is handled. It is gone. This is the failure that kills trust for good, because the writer only finds out when a customer is standing there Thursday. The fix is to make every step confirm the next one, and to make the last step tell the writer it happened. A note is not "captured" when the audio uploads. It is captured when the task is visibly sitting in its destination with a record behind it. Until then it is in flight, and anything in flight can be lost.

How do you keep the build small enough to actually ship?

You ship one intent first.

Not every note a service writer might ever speak. One. The follow-up. Capture a spoken follow-up, parse the customer and the date, route it to one list, and confirm it landed. That is the whole v1. It is small, it is painful when it is missing, and it happens every single day. Those are the three criteria I hold every first build to.

Then you run it for a week without adding anything. A week of a real writer using it in a real drive tells you which of your imagined features were real and which were you gold-plating at your kitchen table. Almost always the thing that matters is not a new intent. It is the confirmation that the note landed. That is what turns a demo into a tool someone relies on.

Only after that week do you add the second intent. Then the third. The router you built first makes each new intent cheap, because adding one is just a new branch on a switchboard that already works. This is the compounding that makes an operator agent stack grow. Five rivers, twenty-two agents, all built the same way, one small trusted piece at a time. If you want the starting sequence for your own first build, I wrote it out in how to start building AI agents if you have a day job.

Does a service writer need to know how to code to run one?

No. The writer taps a button and speaks. That is the entire interface.

The person building it needs to be able to follow a build, modify code a model writes, and read enough of it to fix the three failures above when they show up. That is closer to running a complicated spreadsheet than to being a software engineer. The routing piece leans on the same boring plumbing every multi-tool agent needs, the kind of auth-and-connect layer the open Model Context Protocol was built to standardize. You do not have to master it to ship a first version. You have to respect that it is where the reliability lives.

The reason to build this one first, if you are an operator picking a starting agent, is that it is small, the failures teach you everything, and the moment it works you have removed a class of dropped balls from a real person's day. That is the only scoreboard that matters here. Fewer notes lost. Fewer Thursdays nobody was ready for.

If you want the full breakdown of how I built all of mine, what each one cost, and what each one does, the playbook is in the 10 agents lead magnet. The community where we ship these in public, bugs and all, is at skool.com/agent-empire-4291. Free.

While I sell cars for a living.

Michael