Voice conversation

Installation

npx assistant-ui@latest add elements-voice-conversation

The CLI reads react-native from your package.json and installs from the native registry tree. The element takes the same props as the React one; the React Native elements guide covers setup and what changes on a phone.

npx shadcn@latest add "@assistant-ui/voice-conversation"

First time? Set up a runtime

Runtime components read their state from an assistant-ui runtime. Add one to an existing project:

npx assistant-ui@latest init

Then wrap your app in a runtime provider:

import { AssistantRuntimeProvider } from "@assistant-ui/react";
import { useChatRuntime, AssistantChatTransport } from "@assistant-ui/ai-sdk";

export default function App() {
  const runtime = useChatRuntime({
    transport: new AssistantChatTransport({ api: "/api/chat" }),
  });

  return (
    <AssistantRuntimeProvider runtime={runtime}>
      {/* your components */}
    </AssistantRuntimeProvider>
  );
}

The installation guide covers new projects, templates, and API routes.

A voice conversation is a live call laid over the thread: an orb that tracks who's talking, a caption naming the turn, and the transcript filling in as it goes. With a runtime the orb, the caption, and the transcript all read straight off the thread's voice session; standalone you drive mode, amplitude, and the turns yourself.

Getting started

assistant-ui models a realtime voice call as s.thread.voice, populated once you call connectVoice() against a runtime configured with a realtime voice adapter. /elements/orb covers wiring that adapter and its own states in full; this page covers the surrounding call screen.

Render the bound call screen

app/voice-call.tsx

"use client";

import { AuiIf } from "@assistant-ui/react";
import { VoiceConversation } from "@/components/assistant-ui/elements/voice-conversation.aui";

function LiveCallScreen() {
  return (
    <AuiIf condition={(s) => s.thread.capabilities.voice}>
      <VoiceConversation />
    </AuiIf>
  );
}

VoiceConversation reads the session, volume, controls, and the current session's transcript itself. The capability condition keeps the call screen out of runtimes that do not configure a voice adapter.

Build a custom call screen

Build from the same state when you need a different call layout. Voice turns are ordinary messages that stay in the thread after the call ends, so useVoiceTranscript from the bound element returns only the turns spoken since the current session connected:

import {
  type VoiceSessionState,
  useVoiceControls,
  useVoiceState,
  useVoiceVolume,
} from "@assistant-ui/react";
import { VoiceConversation, type VoiceMode } from "./voice-conversation";
import { useVoiceTranscript } from "./voice-conversation.aui";

const toMode = (voice: VoiceSessionState): VoiceMode => {
  if (voice.status.type === "starting") return "connecting";
  return voice.mode;
};

function CustomVoiceConversation() {
  const voice = useVoiceState();
  const amplitude = useVoiceVolume();
  const { mute, unmute, disconnect } = useVoiceControls();
  const transcript = useVoiceTranscript();

  if (voice === undefined || voice.status.type === "ended") return null;

  return (
    <VoiceConversation
      mode={toMode(voice)}
      amplitude={amplitude}
      transcript={transcript.slice(-2)}
      muted={voice.isMuted}
      onToggleMute={voice.isMuted ? unmute : mute}
      onEnd={disconnect}
    />
  );
}

useVoiceTranscript records where the thread ended when the session connected and maps the voice messages after that point to VoiceTurn values, so a redial does not open with the previous call's turns. The call screen has no scroll region, so pass the tail you want on screen. The adapter only reports "listening" and "speaking" for mode; infer a separate "thinking" phase if your screen needs one.

Standalone, every part of the call is a prop: which mode it's in, how loud the input is, and the turns said so far.

Drive the call from state

app/call.tsx

"use client";

import { useState } from "react";
import { VoiceConversation, type VoiceMode, type VoiceTurn } from "@/components/assistant-ui/elements/voice-conversation";

export function Call() {
  const [mode, setMode] = useState<VoiceMode>("connecting");
  const [muted, setMuted] = useState(false);
  const [transcript, setTranscript] = useState<VoiceTurn[]>([]);

  return (
    <VoiceConversation
      mode={mode}
      amplitude={mode === "listening" ? 0.6 : 0}
      transcript={transcript}
      muted={muted}
      onToggleMute={() => setMuted((m) => !m)}
      onInterrupt={mode === "speaking" ? () => setMode("listening") : undefined}
      onEnd={() => setMode("connecting")}
    />
  );
}

Only show interrupt while speaking

The orb's click target, labeled "Interrupt the assistant", only fires while mode === "speaking" and onInterrupt is set; pass undefined the rest of the time and the button disables itself rather than doing nothing on click.

Anatomy

<div data-slot="voice-conversation">
  <button aria-label="Interrupt the assistant">{/* orb rings, amplitude-scaled */}</button>
  <div>{/* caption plus hint */}</div>
  <div>{/* transcript, user or assistant tagged */}</div>
  <div>
    <button aria-pressed={/* muted */}>{/* mic */}</button>
    <button aria-label="End the call" />
  </div>
</div>

The orb's rings scale with amplitude (clamped to 0..1) only while mode is "listening" or "speaking"; in "connecting" and "thinking" they hold a fixed smaller scale and the center dot pulses instead. The caption under the orb reads "Mic off" whenever muted is true, taking priority over the interrupt hint even while speaking.

Examples

Every mode

<VoiceConversation mode="connecting" amplitude={0} transcript={[]} />
<VoiceConversation mode="listening" amplitude={0.4} transcript={[]} />
<VoiceConversation mode="thinking" amplitude={0} transcript={[]} />
<VoiceConversation mode="speaking" amplitude={0.7} transcript={[]} onInterrupt={() => {}} />

Ending the call

useVoiceControls().disconnect() is the only way to end a session; there's no separate hang-up action distinct from disconnecting, so wire it straight to onEnd:

const { disconnect } = useVoiceControls();

<VoiceConversation onEnd={disconnect} /* ... */ />

API reference

Bound VoiceConversation

Input	Type	Description
`className` prop	`string \| undefined`	Merged onto the call screen root.
Voice session	`s.thread.voice`	Maps `starting` to `connecting`, otherwise reads the session mode and mute state.
Voice volume	`useVoiceVolume()`	Drives the live input amplitude.
Transcript	`useVoiceTranscript()`	The voice messages committed since the session connected, mapped to `VoiceTurn`; the bound screen shows the latest two turns.

Voice hooks

Hook	Returns	Description
`useVoiceState()`	`VoiceSessionState \| undefined`	`{ status, isMuted, mode }`. `undefined` before `connectVoice()` is called or after the session ends.
`useVoiceVolume()`	`number`	Live input amplitude while a session is connected.
`useVoiceControls()`	`{ connect, disconnect, mute, unmute }`	Each calls the matching `aui.thread.*Voice()` method.

Voice session state

Field	Type	Description
`status.type`	`"starting" \| "running" \| "ended"`	`"starting"` maps to the `"connecting"` mode shown here.
`mode`	`"listening" \| "speaking"`	No `"thinking"` value; infer that gap yourself if you want to show it.
`isMuted`	`boolean`	Reflects `mute()` and `unmute()`.

VoiceConversation

Prop	Type	Default	Description
`mode`	`"connecting" \| "listening" \| "thinking" \| "speaking"`	required	Drives the orb, caption, and hint text.
`amplitude`	`number`	required	Clamped to `0..1`; scales the orb's rings while listening or speaking.
`transcript`	`readonly VoiceTurn[]` (`{ id, role, text }`)	required	Rendered in order below the orb.
`muted`	`boolean`		Shows the muted mic icon and the "Mic off" caption.
`onToggleMute`	`() => void`		Called by the mic button; the button disables without it.
`onInterrupt`	`() => void`		Enables the orb's click target; only meaningful while `mode === "speaking"`.
`onEnd`	`() => void`		Called by the end-call button; the button disables without it.
`className`	`string`		Merged onto the root.

All other div props are forwarded to the root.