Voice Agents Need Two Brains

One keeps the conversation moving. The other finishes the job.

On 10 September, OpenAI launched GPT-Live-1 with an explicit handoff to a backend agent for deeper reasoning and tools. Five days later, Google announced Gemini 3.8 Live, which can keep a conversation going while tools run in the background. Its Extended Thinking version can reason and speak at the same time.

Both companies are selling a more natural voice experience. The product change underneath it is more important: talking and doing have started to run on different clocks, and PMs now have to design how those clocks meet.

This week in AI News

Three product moves this week show why AI roadmaps now have to cover the system around the model:

• Google Wants to Catch Agents That “Succeed” the Wrong Way — Google’s new audit layer looks for unusual reasoning and tool use across an agent session. Task completion alone is no longer enough; teams also need behavioral boundaries and an incident path.
• OpenAI Turns ChatGPT Ads Into Conversations — Sponsored Agents turn an ad into a guided question-and-handoff flow. PMs should judge the whole journey, not just the click-through rate.
• Salesforce Koa Turns Workflow Knowledge Into a Model — Salesforce trained an open model on synthetic CRM workflows. The useful question is whether the durable asset is the model or the workflow specification behind it.

One agent, two very different jobs

A voice agent has to stay present in a live conversation. It must notice interruptions, recover from half-finished sentences and respond quickly enough that the user doesn’t repeat themselves.

It may also have to search records, check policy, call several APIs, ask for approval and change a real system. That work can take seconds or minutes. It needs reliable state, permissions, retries and evidence that the outcome actually happened.

Starting with one live model and a few tools is reasonable. It keeps the architecture small, preserves shared context and avoids another handoff. For quick, low-risk tasks, that may be the right production design.

The first job rewards speed and flexibility. The second rewards care and verification.

As the work gets slower or more consequential, two common failure modes appear. The agent goes quiet while tools run, or it keeps talking before the work is complete.

Once work can outlive a conversational turn, be revised after dispatch or create a costly state change, separate the Talk, Work and Commit responsibilities. These are contracts, not necessarily separate models or services. Keep one loop when it meets the guardrails; add an asynchronous execution boundary only when it does not.

The conversational side is its own discipline. The team behind Open Yap 1K says its 1,000-hour dataset preserves overlaps, hesitations, backchannels and interruptions that clean chat transcripts remove. The dataset is evidence about how human conversation behaves, not proof that any voice model handles those moments well.

OpenAI makes the separation explicit: GPT-Live handles the live exchange, while backend reasoning and business logic cross an asynchronous boundary. Google compresses more capability into one live model and session. Yet Gemini still has foreground conversation and background work.

The implementations differ, but both leave product teams with the same coordination problem.

The hard part begins with “Actually…”

Imagine a customer asks a service agent to move an appointment from Thursday to Friday.

The agent says, “I’m checking Friday,” and the backend starts the lookup. Before it returns, the customer says, “Actually, make it Monday.” A moment later, the Friday result arrives.

A demo may treat this as a small conversational correction. The production system has several decisions to make.

Is the Friday task cancelled or merely superseded? Can its result still be mentioned? Is the earlier confirmation valid? What if the change has already started? Which version of the request should the booking system trust?

OpenAI’s migration guidance makes the requirement concrete: when the user revises a pending task, the application should update the task revision, invalidate stale confirmation and check that the final action still matches the latest request.

An interruption may stop the agent’s speech without stopping the work behind it. A user may ask an unrelated question while the original task continues. They may hang up before it finishes. Every case needs a policy for cancellation, continuation and notification.

Product teams have to choose and document those rules. A better model cannot decide which unfinished business process the company should continue after a customer hangs up.

What the agent can safely claim

Spoken responses can blur the status of a task.

“I heard you” means the request was understood.

“I’m checking” means work has started.

“Friday is available” means a lookup returned a result.

“Your appointment is now Friday” should mean the authoritative system recorded the change.

Those are four different states. A fluent agent can move between them so smoothly that users, and teams reviewing a demo, assume more happened than actually did.

The transcript is not enough to prove that an action happened. The application still needs to own identity, permissions, the latest version of the task, confirmation, retries and the committed outcome.

A useful acceptance test is simple: did the outcome described to the user match the outcome recorded by the system of record?

A PM framework: Talk, Work, Commit

PMs don’t need to prescribe a specific model architecture. They can start by assigning three responsibilities.

Talk owns listening, speaking, clarification, interruption and progress updates. It can abandon a sentence. It cannot invent a result.

Work owns reasoning, retrieval and tool coordination. It may continue while the conversation moves on, but every task needs explicit timeout, revision, retry and cancellation behaviour.

Commit is owned by the application. It checks identity, permission, the latest task revision and confirmation before changing state. If a write times out, the status stays unknown until the application reconciles it; a retry must not create a duplicate action.

This also gives teams a better evaluation plan. Test conversational quality and task execution separately, then test whether they agree. A voice agent can sound excellent while acting on stale intent. It can also complete the task correctly while leaving the customer unsure whether anything happened.

For a pilot, pick one state-changing workflow. Measure how often the spoken outcome matches the system of record, with maximum tolerances for stale and duplicate actions. Compare it with a single-loop baseline. Keep the split only if the gain justifies the added delay and maintenance; stop if it breaches a guardrail.

When one brain is enough

A separate execution loop can become architecture theatre. A low-risk FAQ, a quick lookup or a safely repeatable action may work well inside one live model with synchronous tools. Gemini 3.8 Live is evidence that the boundary can move further inside the model without recreating a traditional speech-to-text pipeline.

The split becomes useful when work is slow, revisable, state-changing or expensive to get wrong.

Model naturalness is only one buying criterion. PMs also need to ask what happens to unfinished work when the human changes their mind.

Before approving a voice-agent roadmap, PMs should be able to point to the owner of the live exchange, the owner of unfinished work and the system that authoritatively records completion. If all three answers are “the conversation history,” the product is not ready for consequential actions.