The Agent UX Patterns Catalog

Source: Server-Sent Events (SSE) over HTTP; WebSockets; vendor SDKs (Anthropic Messages API streaming, OpenAI Responses API streaming)

Intent

Reduce perceived latency for agent responses by streaming tokens to the UI as they generate, producing the feeling of an agent typing in real time rather than waiting silently and then dumping a complete response.

Motivating Problem

Agent responses can take seconds to a minute for typical operations. Waiting silently for the complete response, then displaying it all at once, produces a worse user experience than showing the response as it generates, even when the total wall-clock time is identical. Users tolerate streaming latency much better than silent latency: the visible token-by-token generation reads as progress; the silent wait reads as a frozen application. Token streaming was an immediate consensus pattern in 2022—2023 and remains the baseline for any agent UI with response times longer than a second.

How It Works

Server-side: the agent’s response is generated token-by-token (the way LLMs natively work) and pushed to the client over a streaming connection. Server-Sent Events (SSE) is the standard transport for HTTP-based streaming --- a long-lived HTTP connection where the server can push events to the client. WebSockets is the alternative for bidirectional communication.

Client-side: the UI receives tokens as they arrive and appends them to the rendered response. The user sees the response building up character-by-character (or word-by-word, depending on the granularity). Markdown formatting is applied progressively (a streaming markdown renderer handles incomplete markdown gracefully).

Vendor SDK support: Anthropic’s Messages API and OpenAI’s Responses API both support streaming as a first-class mode; OpenAI’s Responses API streams typed semantic events rather than bare text deltas. The SDKs (anthropic-sdk-python, openai-python, ai (Vercel’s SDK)) handle the streaming protocol details; application code typically calls a streaming method and iterates over the resulting async stream.

Granularity choices: streaming at the token level produces a fast-typewriter effect; streaming at the word boundary produces smoother output; streaming at the sentence boundary trades the typewriter feel for cleaner partial output. Most production UIs stream at the token level, debounced or smoothed in the UI to avoid jitter.

Edge cases: errors mid-stream require graceful handling (the user has seen partial output and the error means more won’t arrive); cancellation requires the client to be able to close the stream and the server to detect the closure; reconnection after network interruption is rarely implemented because most cases are short enough that retry-from-scratch is acceptable.

When to Use It

Essentially all user-facing agent interfaces. The pattern is so universally expected that its absence reads as a defect. Even applications that primarily produce structured output (generative UI, tool outputs) typically stream the natural-language portions of responses.

Alternatives --- non-streaming responses where the response is structured data rendered atomically (a generated JSON object, a complete chart) without intermediate text. Asynchronous patterns (Section E) where the response time is long enough that the UI shifts to background-task mode entirely.

Sources

• platform.claude.com/docs/en/build-with-claude/streaming

• developers.openai.com/api/docs/guides/streaming-responses

• ai-sdk.dev/docs/ai-sdk-ui/chatbot (Vercel AI SDK streaming primitives)

Example artifacts

Code.

// Token streaming with the Vercel AI SDK (React, AI SDK 5 and later)

'use client';

import { useChat } from '@ai-sdk/react';

import { useState } from 'react';

export default function ChatComponent() {

const [input, setInput] = useState('');

// useChat posts to /api/chat by default

const { messages, sendMessage, status } = useChat();

return (

<div className="chat">

{messages.map(m => (

<div key={m.id} className={`message \${m.role}`}>

{/* text parts grow token-by-token as the response streams */}

{m.parts.map((part, i) =>

part.type === 'text'

? <span key={i}>{part.text}</span>

: null)}

</div>

))}

<form onSubmit={e => {

e.preventDefault(); sendMessage({ text: input }); setInput('');

}}>

<input value={input}

onChange={e => setInput(e.target.value)} />

<button type="submit" disabled={status !== 'ready'}>

Send

</button>

</form>

</div>

);

}

// Server-side (Next.js Route Handler):

// app/api/chat/route.ts

import { anthropic } from '@ai-sdk/anthropic';

import { convertToModelMessages, createUIMessageStreamResponse,

streamText, toUIMessageStream, UIMessage } from 'ai';

export async function POST(req: Request) {

const { messages }: { messages: UIMessage[] } = await req.json();

const result = streamText({

model: anthropic('claude-sonnet-5-5'),

messages: await convertToModelMessages(messages),

});

return createUIMessageStreamResponse({

stream: toUIMessageStream({ stream: result.stream }),

});

}

Source: Anthropic Claude thinking blocks; OpenAI reasoning visibility; LangChain/LangGraph streaming events; custom implementations

Intent

Show users what the agent is doing during the response generation --- not just the final answer, but the reasoning, tool calls, retrieved sources, and other process steps --- in a way that supports trust and debugging without overwhelming with cognitive load.

Motivating Problem

Pure token streaming shows the final response building up but not the process behind it. Users wait through silent gaps when the agent is thinking, retrieving, or calling tools. The gaps erode the streaming UX’s benefit and leave users uncertain whether the agent is making progress. Intermediate state display fills these gaps: the UI shows thinking tokens, tool call invocations, retrieved documents, evaluation steps as they happen, alongside the final response that streams when it’s ready. The design challenge is presenting the intermediate state without overwhelming the primary response --- progressive disclosure (Chapter 4) is the dominant pattern.

How It Works

Thinking blocks (Anthropic’s pattern): the agent generates reasoning tokens explicitly marked as thinking. The UI renders these in a visually distinct way (typically collapsed by default, in a muted color when expanded, with a label like “Thinking” or “Reasoning”). The thinking is part of the streaming response --- it streams in real time --- but it’s distinguished from the final answer the user is meant to consume.

Tool call visibility: when the agent calls a tool, the UI shows the tool invocation (which tool, with what arguments) and the tool’s result. Different products handle this differently. ChatGPT shows tool calls as expandable cards inline with the response. Claude.ai shows tool calls as smaller text in the response stream with the result inline. Cursor and other coding assistants show file-system operations and code edits as their own UI elements distinct from the chat. The right pattern depends on the audience and the tool’s nature.

Retrieved source display (RAG agents): when the agent retrieves documents, the UI shows what was retrieved --- typically a card or list of retrieved sources with titles, snippets, and relevance scores. The sources are then cited in the response (Section D). The display lets users verify the retrieved content makes sense for the question and gives them a starting point for digging deeper.

Plan display: for agents that plan before executing, the UI shows the plan --- the sequence of steps the agent intends to take --- before executing them. Users can review and override; the agent updates the plan as execution proceeds. The pattern is most useful for multi-step tasks where the plan has meaningful structure; it’s overhead for simple single-step tasks.

Streaming event protocols: the Vercel AI SDK’s UI message stream (typed message parts for text, reasoning, and tool calls), OpenAI’s Responses API streaming events, Anthropic’s streaming Messages API all support emitting structured events alongside the text stream. The events carry typed information (tool_use, thinking, citation, error) that the UI uses to render the appropriate intermediate state element. AG-UI (next entry) defines the same kind of event stream independently of any one vendor or framework.

When to Use It

Production agent UIs where users benefit from understanding the agent’s process. Coding agents (showing file operations matters). Research agents (showing retrieved sources matters). Complex multi-step agents (showing plans matters). High-stakes agents where trust depends on inspectability.

Alternatives --- hidden intermediate state for simple consumer applications where the user wants just the answer (a recipe suggestion, a quick lookup). The trade-off is users who later want to debug why the agent gave a wrong answer have less to work with; production deployments often expose intermediate state as an opt-in (developer mode, debug view) rather than a default.

Sources

• platform.claude.com/docs/en/build-with-claude/extended-thinking

• ai-sdk.dev/docs/ai-sdk-ui/streaming-data

• developers.openai.com/api/docs/guides/reasoning

Source: github.com/ag-ui-protocol/ag-ui (open source, MIT; npm: @ag-ui/core, version 1.0); docs.ag-ui.com

Intent

Give agent backends and frontends one shared, event-based wire format for streaming text, tool calls, reasoning, shared state, and human-in-the-loop interrupts, so a frontend built once can render runs from any compatible agent framework.

Motivating Problem

Every agent framework streams its runs in its own shape. A frontend that renders LangGraph runs cannot render a CrewAI or Microsoft Agent Framework run without a second adapter, and each adapter re-implements the same intermediate-state, approval, and shared-state UX. AG-UI came out of CopilotKit’s work with LangChain and CrewAI as a common protocol for this layer. Its README places it beside the other two agent protocols: MCP gives agents tools, A2A lets agents talk to agents, AG-UI brings agents into user-facing applications.

How It Works

Event stream: during a run the backend emits events from a fixed set of about sixteen standard types (the README’s count; the docs list more as the protocol grows). Lifecycle events (RunStarted, RunFinished, RunError, StepStarted, StepFinished) frame the run. Text message events (TextMessageStart, TextMessageContent, TextMessageEnd) carry the streamed answer. Tool call events (ToolCallStart, ToolCallArgs, ToolCallEnd, ToolCallResult) carry tool use. Reasoning events carry thinking output, and subagent events track nested delegation.

Shared state: StateSnapshot sends the full agent state; StateDelta sends incremental changes as JSON Patch operations (RFC 6902). The frontend and the agent keep one state model in sync, which is what lets the UI show a live plan, a form the agent is filling in, or a document both sides edit.

Interrupts and frontend tools: the protocol covers human-in-the-loop interrupts (pause, approve, edit, retry mid-run without losing state) and frontend tool calls, where the agent asks the UI to run an action in the browser and gets the result back.

Transport: any event transport works (SSE, WebSockets, webhooks); the project ships a reference HTTP implementation and a middleware layer that tolerates loose event formats. A2UI payloads (Section B) can travel over AG-UI.

Adoption: CopilotKit’s v2 React API is built on AG-UI (Section F). The README lists integrations for LangGraph, CrewAI, Microsoft Agent Framework, and others. Thoughtworks’ Technology Radar moved AG-UI to Trial in April 2026, with the caveat that MCP-based UI (MCP Apps) may reduce the need for a separate UI protocol layer.

When to Use It

Frontends that need to work with more than one agent backend, or teams that want the backend and frontend to evolve separately. Applications with shared agent-user state (live plans, co-edited documents, forms) and in-run approvals.

Alternatives: a single vendor’s stream format (the AI SDK UI message stream, OpenAI Responses events) when the stack is fixed. MCP Apps (Section B) when the UI should ship with the tool on an MCP server rather than live in the application.

Sources

• github.com/ag-ui-protocol/ag-ui

• docs.ag-ui.com/concepts/events

• thoughtworks.com/radar/platforms/ag-ui-protocol

Source: Claude on the web, Claude Desktop, Claude Code, and the Claude mobile apps (product feature; preview with Claude 3.5 Sonnet in June 2024, generally available August 2024)

Intent

Provide a dedicated UI surface for substantial work products the agent generates --- documents, code, interactive HTML, diagrams --- displayed alongside the chat where the user iterates on them, distinct from the conversation history where chat-style interaction happens.

Motivating Problem

Generated content like documents, code, and visualizations doesn’t fit the linear chat format. Users want to see the work artifact prominently, iterate on it, copy it, edit it; users don’t want to scroll through the chat history to find it after a few more turns. The artifact pattern resolves this: the chat stays on one side for the conversation; the artifact stays visible on the other side, updated as the user requests changes, with affordances for preview, code view, copy, and download. The pattern has been widely copied since Anthropic shipped it in 2024. OpenAI Canvas was the best-known second implementation; OpenAI removed it from its GPT-5.5 models in May 2026 in favour of writing and code blocks inside the chat, so it now stands as a retired variant.

How It Works

Trigger: the model decides when content warrants an artifact. Anthropic’s help center gives the criteria: content that is significant and self-contained (typically over 15 lines), likely to be edited, iterated on, or reused outside the conversation, understandable without the surrounding chat, and likely to be referred back to. Documents, code, single-page websites, diagrams, dashboards, and small interactive tools all qualify. In applications built on a model API, the system prompt sets the threshold.

Display: the artifact opens in a panel beside the conversation while the chat remains in a narrower column. The artifact is the focus when generated; the chat continues below the message that produced it without re-displaying the artifact content inline.

Iteration: subsequent turns can update the artifact. The agent regenerates it or applies targeted edits. The artifact panel shows the latest version; version history may be available depending on the product. The user can edit the artifact directly in some cases (text documents, code) or only through chat in others. Canvas added direct editing and inline suggestions the user accepts one at a time; those ideas carry over to any implementation of the pattern.

Preview vs source view: artifacts that render visually (HTML, SVG, React components) show in preview mode by default with a toggle for source view. Code artifacts show source by default with a copy button. Document artifacts render with formatting.

Sharing and persistence: artifacts can be published and shared by link, and Claude also offers templates (Docs, Slides, Design, and Dashboards in beta) for artifacts meant to be shared with a team. The artifact becomes a first-class object beyond the ephemeral chat conversation.

Review surface: the artifact is often where the human and the agent collaborate (reviewing a draft document, refining a UI mock, checking a structured output), which is why Volume 7 treats it as a human-in-the-loop surface.

Building the pattern yourself: the Claude API provides the model, not the Artifacts UI. Applications that want the pattern have the model emit the artifact (HTML, component code, SVG, a document) as structured output or a tool call, and render it in their own panel beside the chat.

When to Use It

Substantial generated content the user wants to iterate on or use elsewhere: documents to copy into other tools, code to integrate into a project, interactive demonstrations, structured outputs. Conversational AI products where chat is the right primary interface but generated content needs its own home.

Alternatives --- inline content within the chat for short generated outputs that fit naturally there (OpenAI’s choice after it removed Canvas). Generative UI (Vercel AI SDK UI, MCP Apps, A2UI) where the interface elements appear in the conversation flow rather than a fixed chat-plus-side-panel layout. Embedded agents (CopilotKit pattern) where the AI lives inside the existing application rather than producing artifacts in a separate AI interface.

Sources

• support.claude.com/en/articles/17153992-what-are-artifacts-and-how-do-i-use-them

• anthropic.com/news/claude-3-5-sonnet (launch)

• help.openai.com/en/articles/9624314-model-release-notes (Canvas removed from GPT-5.5 models, 28 May 2026)

• Volume 7, Section F (artifacts as a review surface; points here)

Source: modelcontextprotocol.io/extensions/apps/overview (MCP Apps, official MCP extension since January 2026); github.com/a2ui-project/a2ui (A2UI, Apache 2.0, created by Google)

Intent

Let an agent or tool return interactive UI (forms, dashboards, charts, approval screens) that works across hosts, either by shipping a sandboxed web app with the tool (MCP Apps) or by describing the UI as data the host renders from its own trusted components (A2UI).

Motivating Problem

Framework generative UI (Section F) and product features like Artifacts tie the UI to one application or one host. A tool published on an MCP server is used from many hosts, and a remote agent may sit across a trust boundary, where running code it generated is unsafe. Text answers are not enough either: a tool that returns hundreds of database rows forces the user to ask again for every sort, filter, or drill-down. Both protocols give agents a portable way to return UI; they differ in who owns the rendering.

How It Works

MCP Apps: a tool declares a UI resource (a ui.resourceUri entry in its _meta field) pointing to bundled HTML and JavaScript served under the ui:// scheme. The host fetches the resource, renders it in a sandboxed iframe, and talks to it over JSON-RPC on postMessage. Through the App class in the @modelcontextprotocol/ext-apps package the UI receives tool results, calls server tools, sends follow-up messages, and updates the model’s context, so the model sees what the user did. Security layers: iframe sandboxing, templates the host can review before rendering, loggable messages, and optional user approval for UI-initiated tool calls; hosts without UI support fall back to text. MCP Apps launched on 26 January 2026 as the first official MCP extension, built on MCP-UI and the OpenAI Apps SDK and developed with OpenAI and the MCP-UI maintainers. ChatGPT, Claude, Goose, and Visual Studio Code support it.

A2UI: the agent sends declarative JSON describing a component tree and its data model. The client keeps a catalog of trusted, pre-approved components (Card, Button, TextField, and its own custom ones) and the agent can only request components from that catalog, so the payload is data, not executable code. The tree is a flat list of components with ID references, which models can generate incrementally and the client can render progressively. The same payload renders on different clients; renderers exist for Lit, Angular, and React, with Flutter and SwiftUI renderers in progress. Payloads travel over A2A or AG-UI (Section A). A2UI is an early-stage public preview: v0.9 is the stable protocol family and v1.0 is a release candidate. Google created it, with contributions from CopilotKit.

Choosing between them: the A2UI project’s own comparison puts it this way: choose MCP Apps when the server should own the full UI experience; choose A2UI when the UI should be dynamic, cross-platform, and look native to the host application. The two combine: Google and the MCP Apps co-creators have documented carrying A2UI over MCP and embedding MCP Apps in A2UI hosts.

Design cost: as with any generative UI, someone has to design the components the agent can use. MCP Apps puts that work on the tool author; A2UI puts it on the host’s component catalog.

When to Use It

MCP Apps: tools that need rich interaction (data exploration, configuration wizards, document review, live monitoring) and should work in whichever MCP host the user runs. A2UI: multi-agent and cross-organization setups where a remote agent must present UI without running code in the host, and products that render the same agent output on web and mobile.

Alternatives: framework generative UI (Vercel AI SDK UI, CopilotKit) when the agent and the frontend ship together in one application. Side-panel artifacts for substantial standalone work products. Plain text when the added complexity is not warranted; Thoughtworks placed MCP Apps in Assess (April 2026) with that caution.

Sources

• blog.modelcontextprotocol.io/posts/2026-01-26-mcp-apps/

• modelcontextprotocol.io/extensions/apps/overview

• thoughtworks.com/radar/platforms/mcp-apps

• github.com/a2ui-project/a2ui; a2ui.org

• a2ui.org/introduction/agent-ui-ecosystem/ (A2UI compared with MCP Apps and AG-UI)

• developers.googleblog.com/en/a2ui-and-mcp-apps/

Source: Pattern documented across many production agent UIs; engineering substrate covered in Volume 7

Intent

Provide users with visibility into what the agent is about to do, with explicit opportunity to confirm, modify, or cancel before the action executes, scaled to the operation’s significance and frequency.

Motivating Problem

For operations the agent shouldn’t execute without user knowledge --- sending external communications, modifying important data, making purchases, deleting content --- the user needs an opportunity to review what the agent intends before it happens. The naive design (confirm every action) produces friction that destroys the agent’s value proposition. The more nuanced design scales approval to the operation’s significance: routine read operations execute without approval; consequential operations get explicit review; sensitive operations may require step-up authentication. The design discipline involves deciding which operations get approval gates and what those gates look like.

How It Works

Pre-action confirmation: the agent describes what it’s about to do (“I’ll send this email to alice@example.com with this subject and body”); the user clicks confirm to proceed or modify to revise. The pattern is clearest for irreversible operations with high blast radius. Most production agent UIs use this pattern for at least some operations.

Batched approval: instead of confirming each action separately, the agent presents N proposed actions together and the user approves the batch. The pattern reduces approval fatigue for medium-frequency operations where one-at-a-time confirmation would be tedious. Cursor’s code edit approval shows multiple file changes in a diff view the user accepts together. The pattern works when the actions are independent enough that batch approval is meaningful and not so consequential that they each need individual review.

Progressive trust: the agent auto-approves patterns the user has explicitly approved in prior sessions; new patterns trigger approval. The pattern adapts to the user’s growing trust over time. ChatGPT’s memory feature uses something similar: facts the user has explicitly let the AI remember don’t require re-confirmation in future sessions. The design challenge is making the trust adjustments visible (the user can see what’s been auto-approved) and revocable (the user can withdraw trust).

Risk-scaled approval: only sensitive operations require explicit approval; routine operations execute without it. The agent classifies operations by risk (often using a separate classifier or rule-based heuristics); the UI shows the classification (a badge or label indicating sensitivity) so the user understands why approval was or wasn’t required. The pattern works when the risk classification is reliable; misclassification of sensitive operations as routine creates the same risk as no approval gates.

Approval interface design: the approval UI itself matters. Showing what will happen, in concrete detail, is more useful than abstract descriptions (“Send email to alice@example.com” vs. “Send email”). Showing what the user can change in the approval step (“edit the message before sending”) reduces the friction of “approve or restart entirely.” Showing the consequences of approving and not approving (“if approved, the message goes; if denied, nothing happens”) makes the choice concrete.

When to Use It

Irreversible operations with significant blast radius. Operations affecting external systems where the user can’t easily undo through the agent. Sensitive operations (financial transactions, content with privacy implications). High-stakes operations where the cost of a wrong action exceeds the cost of friction.

Alternatives --- undo-first design for reversible operations. Auto-execution for low-stakes routine operations. The discipline is matching the pattern to the operation, not picking one pattern for everything.

Sources

• Production agent UIs (Cursor, Claude Code, ChatGPT, Claude.ai documented patterns)

• Volume 7 (engineering substrate)

Source: Pattern documented in agent and non-agent software design; cited as alternative to approval-first in agent UX literature

Intent

Let the agent act on routine operations without explicit approval, with a clear path to undo if the action is wrong. Reduce friction for the common case while preserving the user’s control over outcomes.

Motivating Problem

Approval-first design produces friction proportional to the frequency of operations. For high-frequency low-stakes operations, requiring approval for each one destroys the agent’s productivity benefit; users learn to click confirm reflexively (which defeats the purpose of confirmation) or stop using the agent for these operations. Undo-first design takes the opposite approach: act fast, allow rollback, design the system so reversal is easy and well-supported. The pattern produces better UX for operations where reversal is feasible; it requires engineering investment in the reversibility itself.

How It Works

Snapshot-and-restore: before performing a stateful operation, the agent captures the relevant state. After the operation, the user can choose to restore the previous state if the result wasn’t what they wanted. The pattern works for document edits (preserve the prior version), settings changes (capture the prior value), and other operations with capturable state.

Soft delete with grace period: destructive operations (delete, remove) are staged rather than executed immediately. The UI shows the item as removed but actually preserves it for a window (minutes to days) during which the user can restore. Email systems have used this pattern for decades (the “Undo Send” feature, the trash folder); agent UIs apply it to broader operations.

Action history with selective revert: the agent maintains a log of actions taken; the user can review the log and revert specific actions selectively. The pattern allows undoing not just the most recent action but earlier ones too, which is valuable when the user doesn’t notice the wrong action immediately.

Engineering investment: undo-first design requires the operations to be reversible, which is engineering work the team has to do. Some operations are inherently irreversible (external emails sent, payments made, content posted to public channels); for these, approval-first is the only viable pattern. The discipline is identifying which operations are reversible and engineering the reversibility well; the design follows from the engineering capability.

Combining patterns: most production agent UIs use undo-first for reversible operations and approval-first for irreversible ones, with the classification often visible to users (“you can undo this” vs. “this cannot be undone”). The combination handles both cases well; using either pattern exclusively produces failures at the boundary.

When to Use It

Reversible operations where the engineering of reversibility is feasible. High-frequency operations where approval friction would destroy the agent’s value proposition. Document editing, settings management, content creation where the user iterates and doesn’t want every iteration to require confirmation.

Alternatives --- approval-first design for irreversible operations. Hybrid patterns where the action executes immediately with a brief undo window before becoming permanent.

Sources

• Jef Raskin, The Humane Interface (the original undo-first articulation)

• Email Undo Send feature design retrospectives

• Modern agent UX literature citing the pattern as preferred for reversible operations

Source: Pattern across Anthropic Claude (citations in responses), OpenAI ChatGPT (sources in search results), Perplexity (citations as first-class UX), Google Gemini

Intent

Make the sources of agent claims visible and verifiable, so users can check the agent’s work, understand which claims are supported by retrieved sources vs. model knowledge, and dig deeper into specific sources when needed.

Motivating Problem

Agent responses synthesize information from multiple sources: retrieved documents, training data, tool results. Without explicit citation, users can’t distinguish supported claims from hallucinated ones; can’t check the agent’s synthesis against the originals; can’t dig deeper into specific points. Citation makes the sources visible and verifiable. The design challenge is presenting citations without overwhelming the response --- inline footnote-style references, hover-to-preview source snippets, source lists with relevance indicators all serve the function with different trade-offs.

How It Works

Inline citation markers: the response includes superscript numbers or links at specific points; the citations resolve to the sources that supported the claim at that point. Perplexity’s UX is the canonical implementation; many other products follow similar patterns. The user can hover or click for source detail without leaving the response.

Source lists: the response is followed (or accompanied) by a list of sources with titles, snippets, URLs. The user can see at a glance what informed the response and explore specific sources. Anthropic’s citations in Claude API responses use this structured pattern; the application renders the list in whatever format fits the product.

Span-level citation: more granular than inline markers, span-level citation marks the specific text spans in the response that came from each source. The pattern is more accurate but visually denser; production UIs typically use it as an opt-in (toggle to see span-level citations) rather than a default.

Citation reliability: the citation’s accuracy depends on the agent’s ability to attribute claims correctly to sources. Modern APIs (Claude’s citations and search result content blocks, OpenAI Responses API url_citation annotations from web and file search) have improved reliability; older patterns where the model generated free-form citations often produced citations to documents that didn’t actually support the claim. The pattern works when the engineering produces reliable citations; it fails badly when the citations are wrong because users trust them.

Citation in retrieved content: the source’s position in the retrieved document matters. “Source X says Y” should ideally point to the specific passage in X that supports Y, not just X as a whole. The deep-link pattern (cite to the specific paragraph or passage) provides this; it requires the retrieval system to preserve location information.

When to Use It

RAG-based agents where the agent’s value comes partly from synthesizing retrieved sources. Research and analysis tools where source verification matters. Customer-facing agents where users need to trust the agent’s claims. Regulated domains (legal, medical, financial) where claims need provenance.

Alternatives --- no citation for purely conversational agents where the model’s general knowledge is the source. Source lists without inline citations for cases where the response is a synthesis from multiple sources rather than discrete claims attributable to specific ones.

Sources

• perplexity.ai (UX reference)

• platform.claude.com/docs/en/build-with-claude/citations

• platform.claude.com/docs/en/build-with-claude/search-results

• Academic citation practice (general background)

Source: Pattern across multiple products; emerging discipline less consolidated than citation

Intent

Communicate to users when the agent is uncertain about claims, decisions, or actions, so users can apply appropriate skepticism and verify or override as needed.

Motivating Problem

Agents often produce confident-sounding output regardless of whether the underlying reasoning is well-supported. Users learn to read all AI output as authoritative because the model’s confidence calibration in generated text doesn’t reliably distinguish certain claims from uncertain ones. Explicit uncertainty display --- “I’m not sure about this part”, “This is my best guess but I’d verify”, confidence levels for specific claims --- helps users apply appropriate skepticism. The pattern is less developed than citation; the engineering of reliable uncertainty estimation in LLMs is still maturing, and UX patterns for displaying uncertainty are correspondingly less consolidated.

How It Works

Explicit uncertainty language: the simplest pattern is the model expressing uncertainty in natural language (“I’m not certain, but…”, “my best guess is…”, “I don’t know”). The pattern depends on the model’s calibration --- if it expresses uncertainty randomly rather than tracking actual confidence, the cue is unreliable. Modern foundation models are reasonably well-calibrated for this pattern; system prompts can reinforce it.

Confidence scores: numeric confidence indicators alongside specific claims. The pattern requires the agent to produce confidence estimates, which is harder than producing uncertain-sounding language. Specific tooling (perplexity-based confidence, token-probability-based confidence, ensemble approaches) can produce numeric confidence; the UX displays it as a percentage, a scale, or a categorical indicator (“high confidence”, “medium”, “low”).

Disagreement display: when the agent has multiple plausible answers, showing the disagreement is more honest than picking one and presenting it as confident. Patterns include showing multiple alternatives (with their relative likelihoods), explicit disagreement statements (“sources differ on this”), or branching responses that let the user explore alternatives.

Refusal patterns: when the agent shouldn’t answer (insufficient information, safety constraint, out-of-scope question), the refusal itself is a UX element. Production patterns: clear acknowledgment of what the user asked, explanation of why the agent isn’t answering, suggestions for what the user could do instead. The pattern is well-established in safety-critical contexts (medical, legal); it’s less developed in general consumer agents.

Verification suggestions: when the agent is uncertain, suggesting how the user could verify --- “check the official documentation,” “confirm with your accountant,” “search for recent updates” --- turns uncertainty into actionable guidance. The pattern combines uncertainty acknowledgment with practical next steps.

When to Use It

Agents handling high-stakes claims where misplaced confidence has costs (medical advice, legal guidance, financial decisions). Research and analysis tools where users need to apply judgment to AI output. Customer-facing agents where overconfident wrong answers damage trust more than acknowledged uncertainty does.

Alternatives --- no explicit uncertainty for low-stakes conversational agents where the costs of wrong answers are bounded. Implicit uncertainty (the model’s natural language tends to express uncertainty when present) without explicit numeric or categorical indicators.

Sources

• Calibration literature in ML

• Production agent UX in regulated domains (medical, legal AI products)

Source: Pattern across deep research products (Anthropic Claude with web search and research, OpenAI Deep Research, Perplexity Pro), agent platforms (LangSmith Studio, formerly LangGraph Studio; agent dashboards)

Intent

Communicate progress on tasks that take longer than reasonable synchronous wait times, in ways that let users monitor without forcing constant attention and that surface relevant state without overwhelming with raw trace data.

Motivating Problem

When agent tasks take more than 10—20 seconds, the synchronous wait pattern breaks down. Users either stare at the screen unhappily or task-switch and forget about the agent. Status and progress UX bridges this gap: the user can monitor the task without watching it, switch contexts and return later, see what’s happening at the level of detail appropriate for the moment. The design challenge is choosing what level of detail to surface and how to communicate progress when the agent’s work doesn’t have a natural percentage-complete metric.

How It Works

Step-level status: the agent’s work is broken into named steps (planning, searching, reading, analyzing, writing); the UI shows which step is currently active and which have completed. The pattern works when the steps have clear boundaries and stable structure. Anthropic’s deep research feature and similar products use this pattern: “Researching… reading 5 sources… analyzing findings… writing summary…”.

Activity feed: a scrolling feed of what the agent is doing in real time --- tool calls, retrievals, decisions --- typically at higher granularity than step-level status. The pattern lets users dip in for detail; the feed isn’t the primary content but is available. LangSmith Studio (formerly LangGraph Studio) and developer-focused agent UIs use this pattern.

Progress estimates: when the task has predictable structure, the UI can estimate progress (“step 3 of 5”, “approximately 60% complete”, “estimated time remaining: 2 minutes”). The estimates are imprecise for agent tasks where the work depends on what the agent discovers; communicating the imprecision (“approximately”, “estimated”) is important to set expectations.

Streaming partial results: even when the final result isn’t ready, the agent can stream partial results as they become available. A research agent can stream early findings while continuing to research; a code agent can stream completed files while still working on others. The pattern keeps the user engaged with the work as it builds up rather than waiting for atomic completion.

Background mode: for tasks too long for any reasonable wait, the UI shifts to background mode. The task continues; the user can navigate elsewhere; the UI shows the task in a list of in-progress work; completion triggers notification. The pattern is essential for hour-scale tasks; it works less well for minute-scale tasks where the user would naturally wait.

When to Use It

Agent tasks taking longer than ~20 seconds. Deep research workflows. Complex code generation across many files. Multi-step automation tasks. Anywhere the synchronous wait pattern would damage UX.

Alternatives --- synchronous wait with streaming for tasks short enough that the user reasonably waits. Pure asynchronous (fire-and-forget) for batch tasks where no progress display is meaningful.

Sources

• Anthropic Claude research feature documentation

• OpenAI Deep Research product UX

• LangSmith Studio, formerly LangGraph Studio (docs.langchain.com/langsmith/studio)

Source: Pattern across consumer AI products (ChatGPT, Claude.ai, Gemini), professional tools (Cursor, agent platforms)

Intent

Let users start agent tasks and return to them later --- across sessions, across devices, after the task completes --- with the agent’s state preserved and the user’s context restored.

Motivating Problem

Volume 6 covered the engineering of memory and state persistence. The UX dimension determines whether users actually benefit from the persistence. Notification on completion lets users move on while a task runs and surfaces the result when ready. Conversation persistence lets users return to in-progress conversations across days. Resume-from-checkpoint lets users pick up after failures. Shareable session links support collaboration. The patterns transform agents from chat-with-now sessions into work-with-over-time tools.

How It Works

Completion notifications: when a long-running task completes, the user gets notified through their preferred channel --- in-app banner, push notification, email, sometimes SMS for high-stakes tasks. The notification surfaces enough information to know what completed without requiring the user to open the app. Production patterns: “Your research on X is ready”, “Claude finished refactoring 12 files”, “Your analysis completed with these findings.”

Conversation persistence: conversations stay available indefinitely; the user can return to a conversation from days or weeks ago and continue. The pattern depends on engineering (Volume 6) but the UX matters too: how is the conversation surfaced (sidebar, history page, search), how is it titled (auto-generated from content, editable by the user), how is it organized (chronological, by topic, by project)?

Resume-from-checkpoint: when a task fails or is interrupted, the user can resume from where it left off rather than starting over. The UX surfaces the partial state (“this task got through 6 of 10 steps before failing”) and offers explicit resume actions. The engineering depends on state persistence (Volume 6) and idempotent steps (Volume 1 patterns).

Shareable session links: a session can be linked, shared via URL, and continued by another user. The pattern supports collaboration (one team member starts research, another continues) and handoff scenarios. ChatGPT’s shared conversations and Claude.ai’s shared chats both implement this pattern. The design considerations: what gets shared (the conversation, the artifacts, the underlying tools), what permissions apply (read-only, fork, continue), how the recipient sees the shared content.

Cross-device continuity: starting a session on one device and continuing on another. The pattern is increasingly important as users move between phone, tablet, and desktop. The UX requires careful attention to interface adaptation (what works in the phone form factor may not match what works on desktop) but the underlying engineering is conversation persistence applied across devices.

When to Use It

Any agent product where users have tasks longer than a single session. Professional tools where work spans multiple work sessions. Consumer products with research, analysis, or creation workflows that benefit from across-session continuity. Collaboration scenarios where multiple users interact with the same agent session.

Alternatives --- ephemeral sessions for low-commitment use cases (quick lookups, casual queries). The trade-off is users can’t return to previous work; for the casual use cases that’s often fine.

Sources

• Production AI products (ChatGPT, Claude.ai, Gemini) documented patterns

• Cursor and Claude Code (professional agent tools) UX

Source: ai-sdk.dev (open source, Apache-2.0; npm: ai, @ai-sdk/react, @ai-sdk/vue, @ai-sdk/svelte, @ai-sdk/angular)

Intent

Provide React, Vue, Svelte, and Angular hooks for building AI applications, with first-class support for streaming, tool calling, generative UI (tool results rendered as components), and multi-turn conversations.

Motivating Problem

Building AI UIs requires implementing many patterns from scratch: streaming state management, tool call rendering, generative UI composition, message history persistence. The Vercel AI SDK’s UI package wraps these into framework-specific hooks: useChat for chat interfaces, useCompletion for single-turn completion, useObject for streaming structured objects. Each hook handles a category of AI UI work that would otherwise require significant from-scratch implementation. Generative UI is a feature of the same package: rather than only text, the agent’s tool calls come back as typed message parts the client renders as interactive components (forms, cards, charts, custom widgets), so the interface can match the task (Chapter 3) without building an artifact panel.

How It Works

useChat hook: the canonical primitive. Returns messages (each with typed parts for text, reasoning, files, and tool calls), sendMessage, status (‘submitted’, ‘streaming’, ‘ready’, or ‘error’), stop, regenerate, error, and helpers for tool results and tool approval requests. Input state belongs to the application. Since AI SDK 5 the hook is imported from @ai-sdk/react, talks to the server through a transport (by default a POST to /api/chat), and the server returns a UI message stream.

useCompletion hook: simpler variant for single-turn completions that don’t need conversation history. The pattern fits applications where the AI is consulted for specific tasks rather than carrying on conversations.

useObject hook: streams structured objects (validated against a Zod schema) rather than free-form text. The pattern fits cases where the AI’s output has known structure that the UI can render progressively as it streams.

Generative UI: server-side tools (defined with tool(), an inputSchema, and an execute function) return data, not JSX. Each tool call arrives in message.parts as a part typed after the tool (tool-showOrderChart for a tool named showOrderChart) with a state (input-available, output-available, output-error); the client maps the part to a React component. Generated components can use the same charting library and backend as the rest of the application, and interactive ones (forms that submit back, charts users drill into) trigger new AI turns. The older React Server Components path (streamUI, useUIState, useActions in @ai-sdk/rsc) is marked experimental; the docs recommend AI SDK UI for production and publish a migration guide.

Approval: tools can require human approval before they run; the client renders the pending tool part with approve and reject controls and answers through the hook’s tool approval response function. Volume 7 covers the approval gate itself.

Framework support: React (Next.js, plain React), Vue, Svelte, and Angular are official; SolidJS is a community integration. The hooks have parallel implementations with consistent APIs. The server side abstracts over providers (OpenAI, Anthropic, Google, Mistral, and others) behind one API.

Trade-off: generative UI requires more upfront design effort than chat. The developer has to define what components the AI can generate and what they do; the design of those components determines the user experience.

When to Use It

Next.js applications building AI interfaces (the SDK is Vercel-native and integrates particularly well). React, Vue, Svelte, or Angular applications wanting framework-native AI primitives. Applications where the AI’s outputs have structure that maps to UI components (charts, forms, lists, custom widgets). Cases where the application uses Vercel AI SDK’s server-side capabilities and wants matching UI capabilities.

Alternatives --- assistant-ui for cases where a more polished standalone chat experience is the goal. CopilotKit for embedded AI patterns. MCP Apps or A2UI (Section B) when the UI must work across hosts. Chat-with-artifacts (Section B) for products that are mainly conversational with occasional structured outputs. Direct integration with vendor SDKs (Anthropic, OpenAI) for cases where the SDK’s abstractions don’t fit. Volume 7 covers the same SDK from the human-in-the-loop side and points here.

Sources

• ai-sdk.dev/docs/ai-sdk-ui/overview

• ai-sdk.dev/docs/reference/ai-sdk-ui/use-chat

• ai-sdk.dev/docs/ai-sdk-ui/generative-user-interfaces

• ai-sdk.dev/docs/ai-sdk-rsc/overview (experimental RSC path)

• vercel.com/blog/ai-sdk-3-generative-ui (the 2024 introduction of generative UI)

• github.com/vercel/ai

Example artifacts

Code.

// Generative UI with AI SDK UI: the tool returns data and the

// client maps its typed tool part to a React component.

// ai/tools.ts

import { tool } from 'ai';

import { z } from 'zod';

export const showOrderChart = tool({

description: "Display a chart of the user's orders.",

inputSchema: z.object({

period: z.enum(['7d', '30d', '90d']),

}),

execute: async ({ period }) => fetchOrderData(period),

});

// app/page.tsx, inside messages.map(m => ...)

{m.parts.map((part, i) => {

if (part.type === 'text')

return <span key={i}>{part.text}</span>;

if (part.type === 'tool-showOrderChart') {

if (part.state === 'output-available')

return <OrderChart key={i} data={part.output} />;

if (part.state === 'output-error')

return <div key={i}>Error: {part.errorText}</div>;

return <div key={i}>Loading chart...</div>;

}

return null;

})}

Source: github.com/assistant-ui/assistant-ui (open source, npm: @assistant-ui/react)

Intent

Provide a polished, batteries-included React component library for building AI chat interfaces, with thread management, attachments, tool calls, message editing, and many other features that production chat UIs need.

Motivating Problem

For applications where AI chat is the primary interface, building all the features users expect (thread switching, message editing, attachment upload, tool call rendering, copy/regenerate/edit affordances) requires significant work. assistant-ui packages these features as React components that integrate with various AI backends through runtimes (Vercel AI SDK, LangGraph/LangChain, custom). The library is more opinionated about the chat experience than Vercel AI SDK’s primitives, providing more out-of-the-box at the cost of less flexibility for non-chat patterns.

How It Works

Composable components: the library is built as composable React components rather than a monolithic chat widget. Thread, ThreadList, Message, Composer, ActionBar, and other primitives compose into the chat UI; teams can customize specific components without rebuilding the whole interface.

Runtimes: the library connects to AI backends through runtimes. Runtimes are included for the Vercel AI SDK (useChatRuntime), LangGraph and LangChain, a data stream runtime, and custom backends; model providers (OpenAI, Anthropic, Google Gemini, Mistral, AWS Bedrock, and others) come through the AI SDK. The pattern decouples the UI from the specific AI provider.

Features: thread management (multiple conversations, switching, persisting), message editing (edit prior messages and regenerate), attachments (file upload, image input), tool call rendering (expandable cards showing tool invocations and results), copy/regenerate/like/dislike affordances on messages, markdown rendering with code highlighting, and many more.

Styling: the primitives are unstyled; the CLI copies a shadcn/ui starter theme into the project, in a Base UI (default) or Radix UI flavour. Teams can keep the starter theme, customize it, or replace styling entirely while keeping the component logic.

When to Use It

Applications where AI chat is the primary interface and the team wants a polished experience without building everything from scratch. Teams already using React and Tailwind. Cases where the library’s opinions about chat UX match what the product needs.

Alternatives --- Vercel AI SDK UI for more flexibility at the cost of more from-scratch UI work. AI Elements (shadcn) for shadcn-style composition. CopilotKit for embedded patterns. Custom implementations when the library’s opinions don’t fit.

Sources

• github.com/assistant-ui/assistant-ui

• assistant-ui.com

Source: ui.shadcn.com/docs/ai-elements (AI Elements, now part of shadcn/ui); github.com/CopilotKit/CopilotKit (MIT)

Intent

Cover two complementary substrates: AI Elements as shadcn-style composable AI components for teams already using shadcn, and CopilotKit as a framework for embedding AI inside existing applications rather than as standalone chat.

Motivating Problem

Different applications need different patterns. Teams using shadcn want AI components that follow shadcn’s composition style: copy the component code into your project, customize freely. Teams with existing applications want to embed AI as a sidekick or co-pilot inside the application, not as a separate chat interface. AI Elements and CopilotKit serve these different patterns; they’re documented together because the choice between them is one of the foundational architectural decisions in agent UX.

How It Works

AI Elements pattern: shadcn-style component library where users copy components into their project and modify them, rather than installing a package and using opaque components. The library covers AI-specific components (Message, Tool, InlineCitation, Sources, PromptInput, Reasoning, Plan, Confirmation) in the shadcn aesthetic. AI Elements started as a Vercel project and now lives in the shadcn/ui docs. The benefit is full control over component code; the cost is more code to maintain in the user’s project. Fits teams already using shadcn for the rest of their UI.

CopilotKit pattern: a React framework for embedded AI rather than standalone chat. The agent runs inside the existing application; AI features manifest as side panels, inline suggestions, command palettes, autocomplete-with-AI rather than a dedicated chat screen. The pattern fits applications where AI is augmenting existing functionality rather than being the primary interface.

CopilotKit specifics: the v2 React API (@copilotkit/react-core/v2) is built on the AG-UI protocol (Section A) and provides hooks for embedding AI capabilities: useFrontendTool lets the agent invoke application functions, useAgentContext shares application state with the agent, useAgent connects to the backend agent, useHumanInTheLoop renders approval or input UI mid-run, and useRenderToolCall renders tool calls as components. Prebuilt CopilotChat, CopilotPopup, and CopilotSidebar components provide the in-app conversational surface. The v1 hooks (useCopilotAction, useCopilotReadable, useCopilotChat) are deprecated. The combination supports AI that knows about the user’s context (what page they’re on, what data is displayed) and can take actions in the application directly. Because the backend speaks AG-UI, the agent can run on LangGraph, CrewAI, Microsoft Agent Framework, or another framework on the team’s own infrastructure.

Comparison: AI Elements is for chat-shaped AI products where the team wants component-level control. CopilotKit is for AI-augmented existing products where AI lives inside the application surface. assistant-ui sits between them as a polished chat library. Vercel AI SDK UI primitives sit beneath all of them as lower-level building blocks.

When to Use It

AI Elements: teams using shadcn for their UI who want AI components in the same style. Cases where component-level control matters more than installable convenience.

CopilotKit: applications where AI augments existing functionality rather than being the primary interface. SaaS products adding AI to existing surfaces (Notion-style AI in document editors, AI in dashboards, AI in CRMs).

Alternatives --- the other libraries in this section for cases where the patterns fit better. Custom implementations when no library matches the application’s needs.

Sources

• ui.shadcn.com/docs/ai-elements

• github.com/CopilotKit/CopilotKit

• copilotkit.ai

• docs.copilotkit.ai/reference/v2 (v2 React API)

• Volume 7, Section F (CopilotKit’s human-in-the-loop hooks; points here)

Source: Patterns across LangSmith Studio (formerly LangGraph Studio), agent observability platforms, multi-agent products (Manus, now owned by Meta; Devin; others)

Intent

Provide users with visibility into multi-agent systems --- what each agent is doing, how they’re coordinating, where intervention is possible --- and intervention surfaces appropriate to the system’s complexity.

Motivating Problem

Multi-agent systems multiply the visibility problem of single agents. Where a single agent’s tool calls and reasoning can be shown sequentially in chat, multi-agent activity happens in parallel, with agents communicating with each other and producing intermediate state that affects other agents. The chat-as-UI pattern breaks down entirely. Visualization patterns --- graphs of agent activity, swim lanes showing parallel work, dashboard views of agent state --- emerge as the working alternatives. The patterns are still evolving; what works for two coordinated agents may not work for ten.

How It Works

Agent dashboards: a panel showing each active agent with its current state, recent activity, and pending tasks. The dashboard updates in real time as agents work. The pattern fits cases with a small number of agents (2—5) where each agent has meaningful identity from the user’s perspective.

Graph visualization: agents and their interactions rendered as a graph. Nodes are agents; edges are messages or task delegations between them. The graph updates as the system runs. The pattern fits complex coordination patterns where the relationships between agents matter; tools like LangSmith Studio (formerly LangGraph Studio) implement variations on this pattern.

Swim lanes: parallel work streams shown as horizontal lanes with time flowing left to right. Each lane corresponds to an agent (or a task within the multi-agent system); events are placed on the lane at the time they occurred. The pattern fits cases where temporal coordination matters and users want to see what happened when.

Aggregate progress: when there are many agents and individual visibility would overwhelm, an aggregate progress view shows the overall state (“5 of 12 agents complete, 4 in progress, 3 pending”) without per-agent detail. The pattern fits cases with many similar agents (parallel processing patterns) where individual identity doesn’t matter from the user’s perspective.

Intervention surfaces: where can the user intervene? Per-agent pause/cancel buttons, ability to send messages to specific agents, ability to override the coordinator’s decisions, ability to add new agents to the system. The intervention design depends on how much agency the user wants to retain; many multi-agent products limit intervention significantly because allowing arbitrary intervention can break the coordination patterns.

Audit and replay: the ability to scroll back through what happened, replay the system from a checkpoint, or inspect specific message flows. The pattern is essential for debugging when the multi-agent system produces unexpected outcomes; the engineering depends on full state capture which Volume 7 and Volume 12 cover.

When to Use It

Multi-agent products where users have legitimate reasons to monitor the multi-agent activity. Debugging-oriented UIs for developers building multi-agent systems. Operations dashboards for production multi-agent deployments. Use cases where users need to trust the coordination they can’t fully observe.

Alternatives --- hidden multi-agent (the user sees only a single conversational interface; the multi-agent nature is implementation detail) for cases where multi-agent is an internal architectural decision rather than a user-visible feature. The trade-off is users can’t intervene effectively when something goes wrong; production deployments often start with hidden multi-agent and expose more visibility as users encounter situations where they need it.

Sources

• LangSmith Studio, formerly LangGraph Studio (developer observability)

• Multi-agent product UIs (Manus, now owned by Meta; Devin; others) for working examples

Source: Speech-to-speech model APIs (OpenAI Realtime API, Gemini Live); open-source voice agent frameworks (LiveKit Agents, Pipecat)

Intent

Make a spoken conversation with an agent feel natural: the agent knows when the user has finished speaking, stops talking when the user interrupts, remembers only what the user actually heard, and answers fast enough that the pause sounds like thinking rather than a dropped call.

Motivating Problem

A voice agent built by chaining a text agent between speech-to-text and text-to-speech inherits none of the conversational behaviour people expect. It waits for a fixed silence before answering, so it either cuts users off mid-thought or leaves long gaps. It keeps talking when the user tries to correct it. After an interruption, its conversation history claims it said things the user never heard. Network jitter that a text UI hides turns into stutter. Each of these needs an explicit design decision; none of them exists in the text patterns earlier in this volume.

How It Works

Model shape: speech-to-speech realtime models (the OpenAI Realtime API, Gemini Live) take audio in and produce audio out in one session, which cuts the latency of a speech-to-text, LLM, text-to-speech chain. Frameworks such as LiveKit Agents and Pipecat support both shapes, and can pair a realtime model’s speech understanding with a separate text-to-speech voice.

Turn detection: deciding when the user’s turn has ended. Silence-based voice activity detection (VAD) waits for a configured gap (for example, 500 ms of silence with a 300 ms prefix padding in the OpenAI server VAD settings LiveKit documents). Semantic turn detection predicts the end of a turn from the meaning of what was said as well as its sound; OpenAI offers a semantic VAD with an eagerness setting, and LiveKit’s default turn detector model combines meaning and acoustics.

Interruption (barge-in): when the user starts speaking, the agent stops. LiveKit pauses the agent’s speech when it detects user speech, and truncates the conversation history to the portion of the reply the user heard before the interruption, so the model does not assume it delivered the rest. False interruptions (a cough, background noise, a VAD trigger with no words) are handled by resuming where the agent left off after a short timeout.

Transport: browser and mobile clients connect over WebRTC rather than WebSocket. Pipecat’s documentation gives the reason: WebSocket runs on TCP, whose retransmission holds up later audio behind a lost packet, while for speech a short gap sounds better than a stutter, and the Opus codec’s loss concealment is built for UDP. Pipecat recommends WebSocket only for server-to-server and text-only connections.

Visible companions: many voice agents still have a screen. Live transcripts, a speaking or listening indicator, and on-screen confirmation for consequential actions carry over the intermediate state and approval patterns from Sections A and C, since an approval spoken aloud is easy to mishear.

When to Use It

Phone and contact-center agents, in-car and hands-busy use, accessibility for users who cannot easily type or read, language practice and coaching, and any product where speaking is faster than typing for the user.

Alternatives: push-to-talk voice input into a text agent when natural turn-taking is not needed and latency is tolerable. Text chat with optional read-aloud when users need to scan, copy, or review what the agent said.

Sources

• docs.livekit.io/agents/models/realtime/openai/

• docs.livekit.io/agents/logic/turns/ (turn detection and interruptions)

• docs.pipecat.ai/api-reference/client/js/transports/openai-webrtc

• Volume 14 (voice agent platforms as products)

Appendix A --- Pattern Reference Table

Cross-reference of the major patterns covered in this volume, what each solves, and representative implementations.

Pattern	Solves	Representative implementations	Section
Streaming UX	Latency feedback	Token streaming, SSE/WebSockets	Section A
Intermediate state display	Process transparency	Thinking blocks, tool call visibility	Section A
Agent-UI event protocol	Backend-neutral streaming	AG-UI events, shared state, interrupts	Section A
Side-panel artifacts	Persistent work artifacts	Anthropic Artifacts (OpenAI Canvas, retired)	Section B
Generative UI	Task-shaped interfaces	MCP Apps, A2UI; AI SDK tool parts (Section F)	Section B
Pre-action approval	Reversibility safeguards	Confirmation gates for irreversible operations	Section C
Undo-first design	Low-friction reversibility	Snapshot-and-restore, soft delete	Section C
Inline citation	Source attribution	Perplexity-style citations, Claude citations	Section D
Confidence display	Uncertainty communication	Calibrated language, confidence indicators	Section D
Step-level status	Long-task visibility	Named steps with progress indicators	Section E
Conversation persistence	Across-session continuity	Conversation history, resume from checkpoint	Section E
Multi-agent dashboards	Multi-agent visibility	Per-agent state, graph visualization	Section G
Realtime voice	Spoken turn-taking	Turn detection, barge-in, WebRTC transport	Section H

Appendix B --- The Series

This catalog joined the twelve prior volumes to form a thirteen-layer vocabulary for agentic AI; the series has since grown to nineteen volumes.

• Volume 1 --- Patterns of AI Agent Workflows --- the timing of agent runs.

• Volume 2 --- The Claude Skills Catalog --- model instructions in packaged form.

• Volume 3 --- The AI Agent Tools Catalog --- the function-calling primitives.

• Volume 4 --- The AI Agent Events & Triggers Catalog --- the activation layer.

• Volume 5 --- The AI Agent Fabric Catalog --- the infrastructure substrate.

• Volume 6 --- The AI Agent Memory Catalog --- the state and context layer.

• Volume 7 --- The Human-in-the-Loop Catalog --- HITL engineering for human-supervised agents.

• Volume 8 --- The Evaluation & Guardrails Catalog --- LLM-internal safety mechanisms.

• Volume 9 --- The Multi-Agent Coordination Catalog --- the agent-to-agent communication layer.

• Volume 10 --- The Retrieval & Knowledge Engineering Catalog --- finding the right information in a corpus.

• Volume 11 --- The AI Compliance & Regulatory Catalog --- compliance-facing governance.

• Volume 12 --- The AI Infrastructure Security Catalog --- security around the AI system.

• Volume 13 --- The Agent UX Patterns Catalog (this volume) --- design discipline for agent interaction.

The series now has four adjacent-to-engineering volumes: Volume 8 covers LLM-internal safety as engineering, while Volumes 11, 12, and 13 cover three complementary disciplines that consume engineering outputs --- compliance, security, and design. The pattern of these adjacent volumes is consistent: each documents a discipline with its own audience, artifacts, and vocabulary, complementary to but distinct from the engineering substrate it sits alongside. The catalog’s growth into these adjacent volumes reflects production reality: shipping successful agent products requires the engineering plus the adjacent disciplines done well, and the disciplines deserve explicit vocabulary.

The series can be read as engineering layers (Volumes 1—10 with Volume 8 as the engineering-of-safety layer) plus three discipline complements (Volumes 11—13). The compliance discipline (Volume 11) reads outputs from Volume 8 to produce documentation auditors and regulators accept. The security discipline (Volume 12) protects the infrastructure all the engineering layers run on. The design discipline (Volume 13) turns the engineering into interfaces end users experience. Together the first thirteen volumes describe what shipping agentic AI products entails as of mid-2026.

Appendix C --- The Eight Agent UX Anti-Patterns

Eight recurring mistakes that distinguish thoughtful agent UX from improvised AI interfaces. Avoiding these is most of the practical wisdom in the field:

1. Chat as the default interface for every task. Chat works for some tasks and fights others. Tasks with structure (tables, diagrams, forms, persistent reference) suffer in chat’s linear format. Choose the interface shape from the task; generative UI and side panels are working alternatives.

2. Hiding the agent’s process entirely. Users can’t trust outcomes they can’t inspect. Show summaries always; make reasoning and tool calls available on demand. The progressive disclosure pattern --- collapsed details that users can expand --- resolves the transparency-vs-cognitive-load trade-off for most users.

3. Showing the entire trace by default. Power users want full traces; general users are overwhelmed by them. The dominant pattern in 2026 is summary primary with on-demand detail, not full trace as default. Save full traces for debug modes and power users who explicitly request them.

4. Approval-first design for every operation. Confirmation fatigue destroys agent productivity. Use approval for irreversible high-stakes operations; use undo-first design for reversible routine operations; use risk-scaled approval where the agent classifies operations and only sensitive ones gate.

5. Citing without making citations verifiable. Citation that just lists sources without letting users check whether the source actually supports the claim erodes trust when the citations turn out to be wrong. Span-level citation, hover-to-preview snippets, clickable source links are the patterns that make citations useful rather than performative.

6. Synchronous wait for long-running tasks. Tasks longer than 20 seconds need different UX than tasks shorter than 20 seconds. Status indicators, partial result streaming, notification on completion, background mode --- the patterns from Section E. Forcing users to stare at loading spinners for minutes is a UX failure.

7. Ignoring conversation persistence. Sessions that disappear when the user closes the tab feel disposable. Conversation history, resume across sessions, shareable links --- these patterns transform agents from chat-now sessions into work-over-time tools. Users notice when the persistence is missing.

8. Building all the patterns from scratch. Component libraries and SDKs (Section F) package the working patterns. Building everything from scratch produces UI that’s less polished, takes longer to ship, and lacks the affordances users expect from agent products. Use the libraries unless the application has specific reasons not to.

Appendix D --- Discovery and Standards

Resources for tracking agent UX patterns as they evolve:

• Anthropic blog (anthropic.com/news) --- design rationale for Claude, Artifacts, Computer Use.

• OpenAI blog (openai.com/blog) --- design and product posts for ChatGPT and related products (Canvas, now removed from current models).

• Vercel AI SDK documentation (ai-sdk.dev) --- reference implementations of many patterns documented here.

• Agent-to-UI protocols: AG-UI (docs.ag-ui.com), MCP Apps (modelcontextprotocol.io/extensions/apps/overview), A2UI (a2ui.org) - the open standards for agent event streams and agent-returned UI.

• Nielsen Norman Group AI coverage (nngroup.com) --- design research perspective on AI UX.

• CHI conference proceedings --- academic HCI research on AI interaction.

• Component library documentation: assistant-ui, AI Elements, CopilotKit --- working pattern documentation.

• Product design retrospectives published by major AI products as they iterate --- practitioner perspective on what worked and didn’t.

• Design community discussions: AI-focused Slack and Discord communities, design Twitter, design conferences with AI tracks.

• AI UX design resources and community tracking - a reading list rather than a pattern: vendor design posts, Nielsen Norman Group, A List Apart and Smashing Magazine, design conferences (IxDA Interaction, CHI), and the CHI and TOCHI research literature (nngroup.com; chi.acm.org).

Two practical recommendations. First, read the design system documentation of major AI products you don’t build for. Anthropic Claude’s design decisions inform OpenAI ChatGPT’s alternatives; both inform what your application should consider. The cross-pollination across products accelerates the discipline’s maturation. Second, prototype patterns before committing. Agent UX patterns are non-obvious; what reads well on paper may feel wrong in practice; what feels wrong in early prototypes may turn out to be the right pattern after iteration. Prototyping fidelity matters more for AI UX than for traditional UX because the interaction qualities (streaming feel, response latency, decision visibility) only manifest when actually using the prototype.

Appendix E --- Omissions

This catalog covers 16 entries across 8 sections. The wider agent UX landscape is larger; a non-exhaustive list of what isn’t here:

• HITL engineering (approval queues, intervention APIs, observability). Covered in Volume 7.

• Visual design fundamentals (typography, color theory, layout grids). The established design literature covers these.

• Accessibility for AI specifically (screen reader behavior for streaming responses, keyboard navigation for chat interfaces, cognitive accessibility for AI). Important and emerging; deserves dedicated treatment.

• Brand and personality design for AI products (voice and tone, persona design, anthropomorphization choices). Cited briefly; not the focus.

• Mobile-specific agent UX (touch interactions, keyboard considerations, small-screen layouts for AI). The patterns largely transfer; mobile-specific dimensions warrant separate treatment.

• Multimodal interfaces beyond text and voice (video input, multimodal output). Emerging but not yet stable enough for catalog treatment. Realtime voice has one entry (Section H); voice agent platforms are in Volume 14.

• Embodied AI UX (robots, AR/VR, ambient computing with AI). Different design discipline with overlap; out of scope for this volume.

• Specific product analysis (deep dives on individual products) beyond the substrate-defining entries. Many products implement variants of the patterns; comprehensive coverage isn’t feasible.

• Design system documentation conventions for AI components. Emerging but not standardized.

Appendix F --- A Note on a Young Discipline

Agent UX is a young discipline. The patterns documented in this volume were largely invented between 2023 and 2026. Token streaming as a first-class UX pattern is roughly four years old. Generative UI as a coherent pattern is roughly two years old. Side-panel artifacts emerged in 2024. Open protocols for agent-to-UI traffic (AG-UI, MCP Apps, A2UI) arrived in 2025 and 2026. Multi-agent UX is still being figured out as multi-agent products ship. The component libraries that package the patterns (Vercel AI SDK UI, assistant-ui, AI Elements, CopilotKit) have all been released or substantially developed within the past 18 months. Compared to the engineering disciplines documented in Volumes 1—10, the UX discipline is recent and the patterns are correspondingly less stable.

What this means for how to use this volume: the patterns documented here represent the working state of practice in mid-2026, not a settled discipline. Some patterns will mature into long-lasting conventions (token streaming probably; progressive disclosure probably). Some patterns will be replaced by better alternatives as the field learns (specific component-library APIs probably; specific multi-agent visualization techniques probably). The structural insights --- nondeterminism requires uncertainty UX, agents have agency that needs boundary communication, chat is wrong for tasks with structure, transparency and cognitive load trade off via progressive disclosure --- should hold up better than the specific implementations. Treat the volume as a starting point that anchors current practice while remaining open to the patterns evolving.

The catalog’s series-level reflection from Volume 12’s Appendix F applies here too: the series could continue. Adjacent areas where comparable treatment would be valuable include cost engineering for AI (the operational discipline of managing inference costs at scale); model lifecycle management beyond evaluation (versioning, deprecation, replacement); the integration patterns between AI systems and existing enterprise systems (ERP, CRM, SaaS, identity, communication). Each could be a volume; none is currently a major gap; each would add structural vocabulary to the catalog’s existing layers. Whether to extend further is a judgment about diminishing returns. Thirteen volumes covers what an architect needs to design, build, operate, and ship agentic AI products responsibly across the engineering, governance, security, and design dimensions as of mid-2026.

Thirteen volumes at the time of writing (the series has since grown to nineteen). Patterns, Skills, Tools, Events, Fabric, Memory, Human-in-the-Loop, Evaluation & Guardrails, Multi-Agent Coordination, Retrieval & Knowledge Engineering, AI Compliance & Regulatory, AI Infrastructure Security, Agent UX Patterns. The structural vocabulary covers the working design space of agentic AI as of mid-2026. The products and patterns will keep evolving; the structural understanding should hold up; that’s the catalog’s value proposition. Thirteen volumes in, the proposition still holds.

--- End of The Agent UX Patterns Catalog v0.1 ---

— The Agentic AI Series —