The full index
All 331 Patterns
Every pattern in the catalog, by volume — name, intent, and a link straight to the full entry. Press / to search the full text of every volume.
Augmented LLM Wrap a base model with retrieval, tools, and memory before composing it into anything else. Prompt Chaining Decompose a task into a fixed sequence of LLM steps, where each step consumes the output of the previous one. Routing Classify the input and dispatch it to a specialized downstream handler. Cascade / Fallback Try a cheaper or faster model first; escalate to a stronger one only when the cheaper attempt is unconfident or fails verification. Parallelization — Sectioning Split a task into independent subtasks, run them concurrently, and aggregate. Parallelization — Voting Run the same task N times and combine the results to get a more confident answer. Map-Reduce over Corpus Apply the same LLM operation to each document in a corpus in parallel, then combine the results in a final reduce call. Orchestrator–Workers A central LLM dynamically decomposes the task into subtasks, delegates each to a worker LLM, and synthesizes their outputs. Hybrid Pipeline Compose a workflow as a sequence of stages, where one or more stages internally fan out in parallel. Evaluator–Optimizer One LLM generates a candidate; a second LLM evaluates it and produces feedback; the generator revises. Loop until the evaluator accepts or a budget is exhausted. Generator–Verifier Generate a candidate; check it with a binary Pass/Fail verifier; on Fail, sample again. The verifier emits no feedback; the generator simply produces a fresh sample. Reflexion Evaluator–Optimizer extended with persistent verbal lessons: the agent reflects on each failure in writing, stores the reflection, and consults the store on the next attempt. Autoresearch (Ratchet Loop) Propose a change, run an experiment, measure a metric, keep the change only if the metric improved; otherwise roll back. Repeat indefinitely. ReAct Interleave the model's reasoning ("Thought") with tool calls ("Action") and tool results ("Observation") in a single rolling context until the task is done. Plan-and-Execute Produce a full step-by-step plan up front; then execute each step, often with a smaller and cheaper model. ReWOO Produce a plan with explicit data dependencies, execute the independent steps in parallel without intermediate model invocations, and combine the results in a final synthesis step. Code-as-Action (CodeAct / Code Mode) Let the agent write and run a short program that calls tools, instead of emitting one structured tool call per turn, so loops, filtering and intermediate results stay in a sandbox and out of the model's context. Tree-of-Thoughts Generate multiple reasoning branches at each step, evaluate the partial branches, expand the most promising, backtrack when a branch fails. LATS Combine Tree-of-Thoughts (branching search), ReAct (tool use), and Reflexion (verbal lessons) into a unified deliberation loop. Multi-Agent Debate Multiple solver agents independently answer a question; they then exchange and critique each other's answers across one or more rounds; an aggregator synthesizes the final answer. Hierarchical Supervisor A top-level supervisor agent delegates to mid-level supervisors, which in turn delegate to specialist workers, forming a tree of responsibility. Autonomous Agent An LLM operates a tool-use loop with no predetermined number of steps, deciding its own actions, observing the environment, and terminating when it judges the task complete. Long-Running Agent Harness Carry a task across many context windows by having a first session write a feature list, a progress log and a start-up script, and having every later session read them, finish one increment, test it, commit it and update the log. Human-in-the-Loop Checkpoint Pause an otherwise-autonomous workflow at defined points to obtain human approval, correction, or judgment before proceeding. Deep Research Agent Decompose an open-ended research question into parallel sub-investigations; each sub-investigation runs its own ReAct-style loop over external sources; a synthesizer combines the findings into a long-form report. Goal Setting and Monitoring Give an agent explicit, measurable objectives and a monitoring loop that scores progress against them while the task runs, so drift is corrected mid-flight instead of discovered at the end. Prioritization Rank competing tasks by value, urgency, effort, and dependency-readiness before committing agent capacity, and re-rank as conditions change. Exception Handling and Recovery Detect failures in LLM calls, tools, and downstream services; retry what is transient, repair what is malformed, fall back when a dependency is down, and checkpoint state so long tasks resume instead of restarting. Resource-Aware Optimization Make cost, latency, and token budget explicit decision variables at runtime — choosing model, cache, batch, and context size per request — instead of paying the peak price for every call. Learning and Adaptation Improve an agent across episodes — not within one task — by turning outcomes, corrections, and feedback into durable updates: memories, instructions, skills, and only rarely weights. Exploration and Discovery Direct an agent to map an unknown space — generating hypotheses, probing, and accumulating a picture of what exists — rather than executing a known task toward a known end state.
skill-creator Guide Claude through creating, validating, and optimizing new SKILL.md files for any domain. claude-api Build, debug, and optimize Claude API and Anthropic SDK applications, including migration between model versions and adoption of features like prompt caching, tool use, thinking, and Managed Agents. Document creation skills — docx / pdf / pptx / xlsx Create, edit, and analyze Microsoft Office and PDF documents from Claude, with full fidelity for tracked changes, comments, formulas, layouts, and binary attachments. frontend-design Instruct Claude to avoid generic "AI slop" aesthetics and make distinctive, considered visual decisions when building new UI or reshaping an existing one. mcp-builder Guide Claude through producing high-quality MCP servers — the integration layer between Claude (or any MCP-compatible agent) and external APIs or data sources. webapp-testing Test local web applications using Playwright for UI verification, debugging, and visual diffing. critical-code-reviewer Conduct rigorous, adversarial code reviews that name security holes, lazy patterns, edge-case failures, and bad practices across Python, R, JavaScript/TypeScript, SQL, and front-end code. describe-design Research a codebase and create architectural documentation describing how features or systems work, with Mermaid diagrams and stable code references. r-testthat Write idiomatic R package tests using testthat 3+, with the appropriate use of fixtures, snapshots, mocking, and BDD-style describe/it blocks. release-post + create-release-checklist Streamline package releases: a changelog-driven release blog post for R or Python packages, and (for R packages) a per-release checklist filed as a GitHub issue. alt-text Generate accessible alt text for figures in Quarto documents, R packages and pkgdown sites using Amy Cesal's three-part formula: chart type, data description, key insight. r-cli Use the cli R package well: semantic messaging, inline markup, progress indicators, theming. sql-optimization-patterns Optimize SQL by reading the query plan first — parse EXPLAIN output, identify the actual bottleneck, then make targeted changes, rather than reaching for indexes or rewrites on instinct. stripe-integration + pci-compliance Integrate Stripe (or similar payment processors) with correct webhook validation and PCI-compliance discipline. stride-analysis-patterns Apply STRIDE threat modeling (Spoofing, Tampering, Repudiation, Information disclosure, Denial of service, Elevation of privilege) to an architecture or design. attack-tree-construction Construct an attack tree for a target asset: the attacker's goal as root, the strategies that achieve it as branches, the sub-steps as leaves, with cost/feasibility estimates per path. context-driven-development Apply Context-Driven Development methodology: maintain product context, write specifications, and proceed in phased planning rather than ad-hoc code production. openskills CLI Install, sync, and read SKILL.md folders across any AI coding agent that reads AGENTS.md, with progressive disclosure preserved. skills CLI and skills.sh directory Install SKILL.md folders from any repository into whichever coding agents are present, with one command and a public directory for finding them. Plugin marketplaces as skill distribution Ship skills to Claude Code users as plugins listed in a marketplace.json catalog, so they install, update and uninstall as named units alongside the commands, agents, hooks and MCP servers that go with them. denotecli Teach Claude the Denote file-naming convention and give it a CLI to search, read and traverse the link graph of a Denote vault, so it can navigate thousands of notes without reading them all. tdd Enforce strict red-green TDD with vertical slicing only: one test, one slice of behavior, one piece of implementation at a time. No horizontal slicing. grill-me Force the agent to ask detailed clarifying questions before starting work, surfacing the misalignment between what the user said and what they meant. diagnosing-bugs Run a structured debugging loop: first build a feedback loop that reproduces the bug deterministically, then minimise, form hypotheses, and fix behind that loop. improve-codebase-architecture Audit a codebase for architectural problems — shallow modules, exposed complexity, drift from the ubiquitous language — and recommend targeted refactors. Rosetta MCP server (ims-mcp) Serve Rosetta's instructions, skills, and workflows to any MCP-compatible agent on demand, with progressive disclosure and source-code isolation, for deployments that do not use the IDE plugins. The five-phase workflow (Prepare → Research → Plan → Act → Validate) Impose a consistent five-phase structure on agent work: load context, search for guidance, produce a reviewable plan, execute, validate. superpowers Give a coding agent a complete, enforced development method (brainstorm the design, plan in small tasks, execute with test-driven development and review, then finish the branch) as a set of skills that trigger on their own.
web_search Search the live web from inside a Claude turn, returning ranked results with snippets and source attributions. web_fetch Fetch the full contents of a specific URL, returning the page's text content to the model. code_execution Run Bash commands and Python in a sandboxed container with file operations and pre-installed libraries. The container has no internet access. bash_tool Execute shell commands in a Linux environment. Used heavily by Claude Code and by Anthropic-hosted agent containers. text_editor (view / str_replace / create) Read, modify, and create files via a focused editing API that maps naturally to how the model reasons about code changes. computer use Control a computer's screen, keyboard, and mouse as a human would — move the cursor, click, type, take screenshots, scroll. memory tool Give an agent durable cross-conversation memory — write notes, recall them in future sessions, and apply context editing to keep long-running conversations manageable. tool search Let Claude work with hundreds or thousands of tools by dynamically discovering and loading them on demand, instead of loading every tool definition into context upfront. advisor Let a faster, lower-cost executor model consult a higher-intelligence advisor model mid-generation for strategic guidance, without breaking out of the single request boundary. Tavily / Exa / Perplexity / Brave — the search backends Live web search via four distinctively different backends, each surfacing different parts of the web with different trade-offs. Firecrawl Industrial-strength web scraping — search, scrape, crawl, and extract structured data from JavaScript-heavy sites, returning clean markdown. Context7 Pull current, version-pinned documentation for 9,000+ libraries directly into the agent's context, so the model writes code against real APIs rather than from training-cutoff memory. E2B Open-source secure sandboxes for AI-generated code, running on Firecracker microVMs with sub-second boot times. Modal sandboxes General-purpose serverless compute that scales to AI sandbox workloads: GPUs, large memory, custom Docker images. Daytona / Blaxel — long-lived sandboxes Sandbox products built for long-lived state and fast resume rather than for ephemeral microVMs. Filesystem MCP Secure file operations — read, write, list, search — against a configurable directory tree, with access controls. Git MCP Read, search, and manipulate local Git repositories: status, diff, log, branch, commit. GitHub MCP (vendor-maintained) Access the GitHub API as tools — search repos and code, read PRs and issues, create issues and PRs, comment on threads, get commit info. Playwright MCP Drive a browser through structured DOM operations — navigate, click elements, fill forms, take accessibility snapshots — rather than via screen coordinates. Chrome DevTools MCP Let a coding agent control and inspect a live Chrome browser through Chrome DevTools: record performance traces, read network requests and console messages, and automate the page. browser-use Browser automation built specifically for AI agents, with an agent-loop API rather than a low-level browser-control API. Gmail Read and search Gmail messages and threads, draft replies, and manage labels. Linear / Jira — issue trackers Manage issues, projects, sprints, and comments in Linear or Jira via natural-language agent interactions. Google Calendar Read, search, create, and modify calendar events; suggest meeting times; respond to invitations. Cloudflare MCP Operate Cloudflare resources — Workers, KV, R2, D1, DNS, Pages, Analytics — from inside an agent. AWS Knowledge Base / AWS MCPs Retrieve from AWS Bedrock Knowledge Bases via MCP; access broader AWS services through the AWS Labs MCP project. Kubernetes / kubectl MCP Inspect cluster state, read pod logs, apply manifests, manage deployments — the standard kubectl operations as agent tools. Sentry MCP Give a coding or incident-response agent direct access to error tracking and observability data: issues, stack traces, events, traces, logs, releases and monitors. PostgreSQL MCP Query PostgreSQL databases from inside an agent — schema introspection, table queries, and (with care) limited mutations. MCP Toolbox for Databases One open-source MCP server that connects agents to many databases, with prebuilt generic tools for exploration and a configuration file for defining narrow, safe production tools. DuckDB MCP Analytical SQL over local files (CSV, Parquet, JSON, Excel) or remote sources, using DuckDB's friendly SQL dialect. Tool descriptions and model selection Write a tool's name and description so the model reliably selects the right tool for a request — and declines the wrong ones — given only the definitions it sees, never the implementation behind them. Tool input schemas as contracts Design a tool's JSON input schema — its parameter names, types, required set, and constraints — as a contract that guides the model to generate valid arguments and rejects invalid ones, rather than as a loose shape validated only after the call. Tool results and error responses for recovery Shape what a tool returns — both success results and errors — so the model can decide what to do next: trim results to the signal the model needs, and return errors as structured, actionable responses rather than raw stack traces. Code Mode / programmatic tool calling Let the model write a short program that calls tools as functions inside a sandbox, so that many calls, loops and filtering happen in one step and only the final result enters the model's context. CLI + skills instead of an MCP server Give a coding agent a concise command-line tool plus a skill that teaches it the commands, instead of an MCP server, when the agent already has a shell and context is the scarce resource.
CrewAI Flows Provide explicit execution paths, conditional branching, and clean state management for event-driven agent workflows — the deterministic counterpart to CrewAI's autonomous, role-based Crews. LlamaIndex Workflows Build event-driven agent workflows as steps that emit and listen for typed events, with the runtime handling dispatch, parallelism, and durability. Inngest Make long-running, event-driven workflows reliable by abstracting the durability concerns (queue, retries, idempotency, fan-out, scheduling) behind a TypeScript/Python function-style API. Trigger.dev Run long-running TypeScript tasks and agents triggered by events, schedules or other tasks, with retries, queues, waits and human approval points, and without the execution time limits of serverless functions. Temporal Run long-lived, fault-tolerant workflows across machines and deployments, with deterministic replay as the central guarantee. Restate Combine durable functions, durable communication, and durable state in one engine, with the operational footprint of a single binary rather than a cluster. Durable execution as a library (DBOS, Vercel Workflow) Make ordinary application functions durable by checkpointing their steps into storage the application already has (Postgres for DBOS, a pluggable backend for Vercel's Workflow SDK), so an event-triggered agent loop survives crashes without a workflow cluster to run. Serverless durable runtimes (Cloudflare Workflows + Agents SDK, AWS Lambda durable functions) Run event-triggered, long-lived agent work on the serverless platform the application already uses, with durable steps, sleeps, event waits and per-agent state provided by the platform instead of a separate orchestrator. Orchestrator.inc Run parallel AI coding agents (Claude Code, Codex, Cursor, OpenCode and 30+ others) on a single repo, each in its own git worktree, and route CI failures, review comments, and PR state changes back to the agent that owns the work. GitHub Actions as agent runner Use GitHub Actions' native event triggers (push, pull_request, schedule, workflow_dispatch, repository_dispatch) to invoke AI coding agents on a repository. Background coding agent platforms Provide background coding agents that watch a repository, react to issues / PR comments / mentions, and produce PRs without the team operating the orchestrator themselves. Webhook ingress (signature verification, idempotency, replay) Accept inbound webhooks so that only authentic, fresh, first-time events start agent runs, and failed deliveries can be replayed instead of lost. n8n Build event-driven workflows with a visual graph editor and a large library of pre-built integrations, including LLM and agent nodes that work as first-class steps. Zapier (with AI Agents) Connect SaaS apps via "Zaps" (trigger → actions) with a vast connector library, now extended with AI Agents that run multi-step LLM workflows. Pipedream / Activepieces Sit between Zapier (no-code, polished) and n8n (self-hosted, code-friendly): code-first workflow steps with managed infrastructure, full webhook surface, and a generous free tier. AWS EventBridge Route events between AWS services, SaaS apps, and custom applications via declarative pattern rules, with a schema registry that knows the shape of every event source. Apache Kafka Carry millions of events per second across many producers and consumers, with durable retention measured in days or weeks, and the strongest delivery semantics of any of the options in this section. Redis Streams / NATS Provide event streaming and pub/sub with much lower operational footprint than Kafka, suitable for agent systems that don't need Kafka's scale. Schedule and cron triggers Fire events on a recurring schedule or at a specific future time, so agents can run periodic work without an external system to nudge them. File-system and storage triggers Fire events when files or storage objects change, so agents can react to incoming documents without polling. Conversation-internal (topic) triggers Standardize the inventory of in-conversation events an agent should be able to react to: "user said something," "planner picked a topic," "user has been silent," "plan completed," "typed activity received." Policy and governance triggers Fire events when a permission gate, budget threshold, or governance rule is crossed, so an agent's action can be paused for review, escalated, or routed differently. AgentOps Provide observability for agents built with any framework — LangGraph, CrewAI, AutoGen, OpenAI Agents SDK, custom — with the agent run as the central unit.
AWS Sample Agentic Fabric ("Arbiter") Demonstrate a complete governance-first agent fabric on AWS, with deterministic policy evaluation on every dispatch, dynamically-generated worker agents, and explicit authority modeling. AWS Sample Agentic Platform Demonstrate the operational shape of an agent platform on AWS across two compute paths, EKS with Helm or Bedrock AgentCore Runtime with an ECS/Fargate gateway layer, organized so that coding agents can navigate the repo. Microsoft Agent Framework Provide Microsoft's official building blocks for orchestrating and safely deploying production multi-agent workflows on the Azure stack. AWS Bedrock AgentCore Provide a managed, serverless runtime for AI agents on AWS, with a dedicated microVM per session and managed identity, memory, tool gateway and observability services, so teams run agents without operating the substrate. GitHub Agentic Workflows substrate Contain a coding agent running inside GitHub Actions by isolating it in its own container on the runner VM, routing its network, tool and model traffic through trusted proxies, and letting it change the repository only through vetted outputs. Strata Agent Fabric (Identity Orchestration) Provide an identity security control plane purpose-built for AI agents: discovery, registry, OAuth scope auditing, IDP binding, federated trust across clouds and platforms. Service mesh for agents (Istio / Linkerd) Use a standard service mesh (Istio, Linkerd, Consul Connect) to provide mTLS, traffic management, and observability for agent-to-agent and agent-to-service communication. Agent gateway (agentgateway) Put a proxy that understands MCP and A2A between agents, tools and models, so authentication, authorization, rate limiting and tracing apply to agent traffic without being coded into each agent or MCP server. Pilot Protocol (purpose-built agent networking) Issue agents encrypted tunnels and virtual addresses so they can communicate across networks and clouds without exposing routable IPs or requiring traditional service-discovery infrastructure. Agent fabric registry (private) Maintain a source of truth for which agents exist in the deployment, their identity bindings, scopes, intent, TTL, audit trail, and risk level. KEDA (Kubernetes Event-driven Autoscaling) Scale Kubernetes deployments up and down in response to external event sources (queue depth, message backlog, HTTP request rate) rather than CPU/memory metrics alone. Kubernetes Agent Sandbox (kubernetes-sigs/agent-sandbox) Give each agent or code-execution session its own isolated, stateful pod with a stable identity and persistent storage, managed declaratively through a Sandbox custom resource. kagent (Kubernetes-native agent runtime) Run AI agents, their tools and the MCP servers they use as Kubernetes custom resources, so agents are defined in YAML, reconciled by a controller and managed with the same kubectl and GitOps workflows as the rest of the platform.
LangGraph state and checkpointers (cross-reference) Model agent execution as a graph of nodes operating on a typed state object, with the framework managing state persistence, replay, and human-in-the-loop interrupts. Redis-backed chat history Persist conversation threads in Redis for sub-millisecond reads and writes, with TTL-based expiration and the option to back it with disk-persistent Redis variants for durability. Chroma Provide a vector database that runs in-process inside a Python application for fast prototyping, with a path to a client/server deployment when scale demands it. Qdrant Provide a production-grade vector database written in Rust, with distributed deployment, rich payload filtering, hybrid search, and the operational maturity to back agent memory at scale. Pinecone Provide vector search as a pure SaaS, with no operational responsibility for the deployer: API key in, vectors in, queries out, with serverless scaling and global availability. Weaviate Provide vector search with first-class support for multi-vector objects, multimodal data (text + image + audio embeddings on the same object), and graph-like relationships between objects. LanceDB Provide a vector store that runs inside the application process and keeps its data as files in the Lance columnar format, on local disk or object storage, with no server to operate. Mem0 Provide a memory layer that integrates with any agent framework, automatically extracts and stores user-relevant facts, retrieves them with hybrid scoring, and exposes a uniform add()/search() API. Letta (formerly MemGPT) Make memory something the agent manages itself: it decides what to keep in context, what to write down, and what to look up, starting from MemGPT's OS-style memory hierarchy and now implemented as memory files the agent edits. Zep Serve as a managed long-term memory and context store for AI assistant apps: build a temporal knowledge graph from conversations and business data, and assemble the right context from it for each turn. LangGraph long-term memory Store and LangMem Give LangGraph agents memory that outlives a single thread: a store keyed by namespace (typically the user), with optional semantic search, plus LangMem tools that let the agent record and search memories and a background manager that extracts and consolidates them. Managed cloud memory services (Amazon Bedrock AgentCore Memory, Google Cloud Memory Bank) Buy agent memory as a cloud service: the provider stores session events, runs LLM extraction to turn them into long-term memories, and serves them back, under the same IAM, encryption and compliance controls as the rest of the account. CrewAI memory abstractions Provide built-in, opinionated memory for multi-agent crews: a single unified Memory class shared by the crew, with hierarchical scopes, LLM-driven consolidation of similar records, and recall ranked by similarity, recency and importance. Supabase (Postgres + pgvector + auth + realtime) Provide Postgres plus pgvector plus auth plus real-time subscriptions as a coherent backend, suitable as the unified state-and-memory store for AI agent applications. pgvector (the Postgres extension itself) Add vector data types and similarity search operators to Postgres, so any Postgres deployment can serve as both a relational store and a vector memory store. Graphiti (temporal knowledge graph) Build a temporal graph of entities, relationships and facts from conversations and business data, where every fact has a validity window, so an agent can ask what is true now and what was true at any earlier point. Agent memory benchmarks: LoCoMo, LongMemEval, BEAM Measure whether a memory system can recall, update and reason over information spread across many sessions, using shared datasets so results from different systems can be compared.
LangGraph interrupts (static and dynamic) Let the agent author mark places in the graph where execution should pause for human input, and let the runtime persist the paused state for resumption when input arrives. CrewAI human_input Let the crew author mark specific tasks as requiring human review of the output before the crew advances. Declarative per-tool approval in agent SDKs Mark individual tools as requiring human approval when the agent is defined, and have the SDK stop before running such a tool, return the pending calls, and accept an approve or reject decision that resumes the same run, even in a different process. Temporal (Signals for HITL) Provide durable execution for long-running workflows, with Signals as the mechanism by which external events (including human approvals) deliver into a running workflow regardless of how many process restarts have happened. AWS Step Functions wait-for-callback Provide AWS-native durable workflow orchestration, with the task-token callback pattern as the mechanism for pausing a workflow until an external event (including human approval) triggers resumption. LangSmith Provide the observability story for LangChain and LangGraph applications: granular nested traces of every agent run, dataset-based evaluation, prompt versioning, human-feedback capture, and the audit surface for production agent systems. Arize Phoenix Provide LangSmith-class observability features as source-available (ELv2) software that is free to self-host, with OpenTelemetry as the trace transport and the option to run entirely on-premises or in a local notebook. Langfuse Provide an open-source observability and evaluation platform with first-class human-feedback capture (thumbs up/down, custom scores), prompt management, and dataset-based evaluation, deployable as self-hosted or as Langfuse Cloud. OpenInference Define a vendor-neutral standard for representing LLM calls, tool calls, agent steps, and retrieval operations as structured trace spans, with attribute conventions that make the resulting traces portable across observability backends. OpenTelemetry GenAI semantic conventions Extend OpenTelemetry's semantic conventions with LLM-specific attributes (model name, token counts, prompt and completion contents, agent step types) so that agent telemetry flows through the existing OpenTelemetry infrastructure used for general application observability. Streamlit as agent UI Serve as the agent UI when the broader Streamlit application surface (dashboards, multi-page apps, non-chat UI) is valuable and the chat-component subset is sufficient for the agent interaction. Anthropic API tool_use ↔ tool_result loop Use the tool_use / tool_result handshake as the foundational pause-and-resume mechanism: when the model emits a tool_use, the application has full control over when and how to return a tool_result, including indefinite delays for human approval. MCP elicitation Let an MCP server pause a request and ask the user, through whatever client the user is in, for structured input or a confirmation, then continue with the answer, without the server knowing anything about the client's UI.
NVIDIA NeMo Guardrails Provide a programmable execution engine that enforces hard boundaries around conversation topics, prevents prompt injection, blocks toxic content, and steers conversations through canonical dialog flows defined in a domain-specific language called Colang. Guardrails AI Provide a Python framework where input and output validation is composed from reusable validators (Pydantic-style schema checks, PII detection, hallucination detection, profanity filtering, structural constraints), executed against LLM I/O at runtime. Meta Llama Guard Provide a fast, classification-focused safety model that classifies inputs and outputs as safe or unsafe across a defined taxonomy of harm categories, deployable in front of a main agent LLM at sub-100ms latency. Managed classifier guardrails (Azure, AWS, OpenAI) Provide the Llama Guard pattern as a managed service: an API endpoint that classifies content against a vendor-defined safety taxonomy, with no model weights to download or infrastructure to run. Prompt-injection classifiers and agent firewalls Run a small, fast classifier on user input and on untrusted content (web pages, emails, tool results) before the agent acts on it, to flag text that tries to override the agent's instructions, as a layer distinct from the harm classifiers that judge whether content is unsafe. DeepEval Provide a Python evaluation framework that feels like pytest, with assertions over LLM-specific metrics (hallucination, faithfulness, contextual relevancy, tool correctness, bias, toxicity), runnable in CI/CD pipelines, with first-class support for tracking metrics over time and detecting regressions. Ragas Provide the canonical evaluation framework for retrieval-augmented generation systems, with RAG-specific metrics (faithfulness, answer relevancy, context precision, context recall, answer correctness) computed correctly and with thoughtful handling of the multi-component nature of RAG pipelines. Promptfoo Provide a CLI tool and library for evaluating LLM outputs through side-by-side comparison of prompts, models, and parameters, with red-teaming capabilities for security testing and a web UI for inspecting results. OpenAI Evals Provide an open framework for creating, sharing, and running LLM evaluations, with a registry of community-contributed evaluations covering benchmarks, behavioral checks, and capability tests. HELM (Holistic Evaluation of Language Models) Provide a comprehensive, methodologically rigorous evaluation framework that compares language models across dozens of scenarios and metrics, with public leaderboards tracking model performance over time. Inspect AI and inspect_evals Provide an open framework for writing and running model and agent evaluations from composable parts (datasets, solvers or agents, tools, scorers), with sandboxed execution for agentic tasks and a maintained library of pre-built benchmarks that run against any supported model. Agent task benchmarks (tau-bench, Terminal-Bench, AgentDojo) Evaluate an agent by giving it a realistic task in a simulated environment with tools, then checking the state the environment ends in and how consistently the agent succeeds across repeated trials, rather than grading a single text answer. LangSmith Evaluators Provide built-in evaluation capabilities tied to LangSmith traces: capture production runs into datasets, run evaluators against them, track metric trends over time, and integrate with the same UI that surfaces traces and prompt versions. Phoenix Evals (Arize) Provide LLM evaluation capabilities tightly integrated with Phoenix's trace UI, with built-in evaluators (hallucination, relevance, toxicity, code QA) and the workflow of capturing traces, running evals against them, and inspecting failures in the same notebook or self-hosted UI. NVIDIA Garak Provide a vulnerability scanner for LLM endpoints, with a catalog of probes that test for specific failure modes (prompt injection, jailbreaks, data leakage, encoding attacks, toxicity, malicious code generation), runnable from the CLI with structured reports. Microsoft PyRIT Provide a Python framework for composing and running adversarial attacks against LLM systems, with built-in attack strategies (jailbreaks, prompt injections, prompt converters), pluggable target endpoints, and scoring mechanisms to identify successful attacks. OWASP Top 10 for LLM Applications (2025) Identify the ten most critical security risks in LLM applications, with descriptions, example attack scenarios, and prevention strategies, updated periodically (2023 and 2025 editions) based on community input and real-world incidents. OWASP Top 10 for Agentic Applications (2026) Identify the ten most critical security risks of autonomous and agentic AI systems (agents that plan, act, use tools, keep memory and delegate to other agents) as a peer-reviewed checklist that sits beside the LLM Top 10. Per-field confidence and selective routing Have the model emit a confidence score for each field it produces, then auto-accept the confident fields and route only the low-confidence ones to human review, rather than accepting or reviewing a whole record on a single coarse score. Confidence calibration Verify that a model's stated confidence matches how often it is actually right — and set routing thresholds from the measured relationship — so that a "0.9" means roughly ninety-percent accuracy rather than an arbitrary number the model emitted. Stratified sampling for human review Choose which outputs a human reviews by stratifying the stream — oversampling low-confidence, rare-category, and high-stakes outputs — rather than sampling at random, so a fixed reviewer budget catches the most errors and the most consequential ones.
MCP — Model Context Protocol Provide a JSON-RPC-based protocol for connecting LLM agents to external tools, data sources, and (increasingly) other agents, with a standardized server interface that any compatible client can consume. MCP primitives — tools, resources, and prompts Model an MCP server's surface as three distinct primitives — tools, resources, and prompts — distinguished by who decides when each is used, so that a capability is exposed through the primitive that matches its control model rather than defaulting everything to a tool. MCP transports — stdio and Streamable HTTP Choose how an MCP client and server exchange messages — stdio for a local subprocess, Streamable HTTP for a remote service — matching the transport to whether the server runs beside the client and whether it must serve multiple users. A2A — Agent-to-Agent Protocol Provide a protocol with JSON-RPC, gRPC and HTTP+JSON bindings, designed specifically for agent-to-agent communication, with first-class agent discovery (via Agent Cards), capability advertisement, task delegation with streaming, and a well-defined task lifecycle. Agent Client Protocol (editor-to-agent) Standardize how a code editor talks to a coding agent, so that any agent implementing the protocol works in any editor that supports it, the way the Language Server Protocol decoupled language tooling from editors. Agent payment protocols (AP2, Agentic Commerce Protocol, x402) Let an agent buy something for a user in a way the merchant, the payment network and the user can all verify: proof of what the user authorized, a checkout the merchant controls, and a payment credential the agent never sees in the clear. LangGraph (multi-agent) Provide a state-graph framework where multi-agent systems are designed as explicit graphs with named agent nodes, typed state, and named transitions, with built-in patterns for supervisor (one manager dispatches to workers), hierarchical (managers of managers), and network (peer-to-peer) topologies. Google Agent Development Kit (ADK, multi-agent) Build multi-agent systems from one agent type tree in which some agents follow fixed control flow (sequence, parallel, loop, graph), others delegate to sub-agents by LLM decision, and any agent can be exposed to or consume a remote agent over A2A. AutoGen / AG2 Provide a framework where multi-agent systems are designed as conversations among agents, with configurable conversation patterns (sequential, group chat, hierarchical), termination conditions, and the ability to mix LLM agents, code-executor agents, and human-input agents in the same conversation. CrewAI Provide a framework where multi-agent systems are designed as crews of role-defined agents executing structured tasks, with explicit roles, goals, and backstories per agent and clear task definitions including expected outputs. OpenAI Agents SDK Provide a lightweight framework where multi-agent systems are designed as networks of agents connected by hand-offs (agent-to-agent transitions implemented as tool calls), with the framework handling routing and the agents themselves being simple functions. Microsoft Magentic-One Provide a research-grade implementation of the manager-plus-specialists hierarchical pattern, with concrete specialists for web browsing (WebSurfer), file operations (FileSurfer), code execution (Coder, ComputerTerminal), and an Orchestrator that manages the team. Agents-as-tools Let an orchestrator agent delegate a sub-task to a specialist agent through an ordinary tool call, receive the specialist's answer as the tool result, and keep ownership of the conversation, instead of transferring control to the specialist. The blackboard pattern Provide a structured shared workspace where each agent contributes typed artifacts and agent activation depends on which artifacts are present, supporting opportunistic and emergent coordination patterns. Multi-agent tracing across LangSmith, Phoenix, Langfuse Extend the trace-tree mental model from Volume 7 to multi-agent systems by adding agent identity to spans, correlating cross-agent calls, and capturing inter-agent metrics, with the major observability platforms providing native support through OpenInference's multi-agent attribute conventions. AGNTCY Agent Directory Let agents be published, found and checked across organizations through a shared directory of signed, structured records, so a client can find an agent by what it can do rather than by a URL it already knows.
OpenSearch Provide a fully open-source search and analytics engine that supports BM25, dense vector retrieval (k-NN), learned sparse retrieval, learning-to-rank, and the orchestration to combine them into hybrid search pipelines. Elasticsearch Provide the original Elasticsearch implementation under Elastic's licensing, with first-class support for the full hybrid retrieval stack: BM25, dense vectors, ELSER learned sparse retrieval, RRF fusion, and the broader Elastic Stack (Kibana, Logstash, Beats, observability) for operational visibility. Vespa Provide a search platform with deeper programmability than Elasticsearch or OpenSearch: tensor-native data model, phased ranking with custom expressions at each phase, first-class support for late-interaction (ColBERT-style) retrieval, and the operational maturity of having run Yahoo's personalized news ranking at production scale for over a decade. Proprietary embedding APIs Provide high-quality embedding models as managed API services, eliminating the operational burden of self-hosting embedding model inference at the cost of vendor lock-in and per-token API charges. Open-weights embedding models Provide downloadable embedding model weights that run on local infrastructure, eliminating per-token API charges and external dependencies at the cost of operational responsibility for serving the model. Contextual chunk embedding (Contextual Retrieval, late chunking, contextualized-chunk models) Embed and index each chunk together with the context of the document it came from, so a chunk that says "revenue grew by 3%" still retrieves for a query that names the company and the quarter the chunk never mentions. Cross-encoder rerankers Provide cross-encoder rerankers that score query-document pairs with full transformer attention, producing higher-quality ranking over the candidate set returned by first-stage retrieval. ColBERT and late-interaction models Provide a middle ground between bi-encoders (single-vector embeddings, fast but limited representation) and cross-encoders (full cross-attention, expressive but slow): per-token embeddings with MaxSim aggregation, document representations precomputable and indexable, query-time computation lighter than cross-encoders. Unstructured.io Provide a unified document-processing library that handles PDFs, Word documents, HTML, images, emails, presentations, and many other formats, producing a structured element stream (Title, NarrativeText, Table, ListItem, etc.) that downstream chunking and indexing can consume intelligently. LlamaParse and the LLM-driven document parsers Provide LLM-driven document parsing that handles complex layouts — multi-column documents, nested tables, mathematical content, scientific papers, financial filings — with fidelity that heuristic-based parsers struggle to match. LlamaIndex Provide a Python framework whose core abstractions are organized around retrieval: documents become nodes; nodes feed indexes; indexes produce retrievers; retrievers compose into query engines; query engines compose into agents. The framework's opinionation matches the shape of production RAG. LangChain retrievers Provide a retriever abstraction (BaseRetriever) that composes uniformly with the rest of LangChain's ecosystem: any vector store can expose a retriever, retrievers compose via LCEL into chains, the chains plug into agents, and the agents plug into the broader LangChain runtime. Managed file-search RAG (OpenAI file_search, Gemini File Search, Bedrock Knowledge Bases) Hand the whole retrieval stack to the model provider: upload files to a hosted store, and let the provider parse, chunk, embed, index, retrieve, and cite them when the model answers. Query transformation patterns (implementable across frameworks) Improve retrieval quality by transforming user input before retrieval: cleaning noisy inputs (rewrite), broadening vocabulary (expansion), bridging vocabulary mismatch (HyDE), breaking compound questions (decomposition), or hedging against any single phrasing being wrong (multi-query). Classical IR techniques in the LLM era Cover the classical information retrieval techniques — query parsing, synonym expansion, learned-to-rank reranking, faceted filtering, intent classification — that predate LLM-driven retrieval and remain valuable in domains where structured queries and curated synonyms outperform LLM transformations. Agentic search (tool-driven iterative retrieval) Let the model drive retrieval: give it search tools (grep, file listing, file reads, a keyword or vector search endpoint) and let it plan queries, read the results, and search again until it has what it needs, instead of running one fixed retrieve-then-generate pass. Neo4j with vector indexes Represent the corpus as a property graph (nodes, relationships, properties), index node content with vector embeddings via native vector indexes, and retrieve via Cypher queries that combine graph traversal with vector similarity. Microsoft GraphRAG Automatically extract entities and relationships from an unstructured corpus into a knowledge graph using an LLM, identify communities of densely-related entities, produce hierarchical summaries for each community level, and retrieve via the community structure for queries that span the whole corpus rather than matching specific chunks. MTEB, BEIR, and benchmark resources Provide standardized measurement of retrieval components — MTEB for embedding models (expanded in 2025 as MMTEB to 500+ tasks in 250+ languages, with the leaderboard split into named benchmarks), BEIR for retrievers on a curated set of zero-shot retrieval benchmarks, RAGAS for end-to-end RAG quality — with public leaderboards that update as new models and methods are released.
NIST AI Risk Management Framework Provide a structured, voluntary framework for managing risks of AI systems across their lifecycle, organized around four core functions (Govern, Map, Measure, Manage) with detailed playbooks, profiles for specific contexts (Generative AI, Government Use), and crosswalks to other frameworks. ISO/IEC 42001 Provide an international standard specifying requirements for establishing, implementing, maintaining, and continually improving an AI management system within an organization, structured to align with other ISO management system standards (27001, 9001) and certifiable through accredited third-party audit. ISO/IEC 42005 AI system impact assessment Give organizations a structured, repeatable process for assessing how an AI system affects individuals, groups, and society across its lifecycle, documenting the potential benefits and harms and the mitigations, so the impact-assessment duties in regulation, procurement, and ISO/IEC 42001 have a common template. EU AI Act (Regulation 2024/1689) Provide a binding legal framework for AI systems placed on the market or put into service in the EU (or whose output is used in the EU), with risk-tiered obligations scaling from prohibited practices to minimal obligations, ex-ante conformity assessment for high-risk systems, post-market monitoring requirements, and significant fines for non-compliance. GPAI Code of Practice Provide voluntary implementation guidance for providers of General-Purpose AI models to demonstrate compliance with Articles 53-55 of the EU AI Act, developed through a multi-stakeholder process under the AI Office's coordination per Article 56 of the Act. Colorado: SB 24-205 and its SB 189 replacement Originally to impose comprehensive risk-management and impact-assessment obligations on developers and deployers of high-risk AI systems making consequential decisions affecting Colorado residents (SB 24-205); replaced in May 2026 by a narrower disclosure-and-consumer-rights framework (SB 189) ahead of the original law's effective date. NYC AEDT, California AB-2013, and other state AI laws Cover the patchwork of US state and local AI laws that, taken together, form the working US state regulatory environment for AI in specific domains (employment, transparency, training data, generative AI disclosures). California CCPA automated decision-making technology (ADMT) regulations Give California consumers notice, an opt-out, and access rights when a business uses automated decision-making technology (ADMT) to make a significant decision about them, and require a documented risk assessment before such processing, under regulations issued by the California Privacy Protection Agency (CPPA) under the CCPA. Frontier model transparency laws (California SB 53, New York RAISE Act) Require developers of the largest foundation models to publish how they assess and mitigate catastrophic risk, publish a transparency report at each model release, and report critical safety incidents to the state within fixed deadlines (15 days in California, 72 hours in New York). FDA AI/ML guidance for medical devices Establish how the FDA regulates AI/ML-enabled medical devices throughout their lifecycle, with specific guidance on premarket submission, software changes (Predetermined Change Control Plans), and post-market monitoring of AI/ML medical devices. Financial services Model Risk Management (SR 26-2, successor to SR 11-7) Establish supervisory expectations for managing risk associated with quantitative models used by financial institutions, originally focused on credit and market risk models and applied by banks to AI/ML models. SR 11-7 predated modern AI by over a decade; its April 2026 replacement keeps the core discipline (model development, validation, ongoing monitoring, governance), which maps onto traditional AI/ML model management directly, but excludes generative and agentic AI from its scope. HIPAA and AI in healthcare Apply existing HIPAA Privacy and Security Rule obligations to AI/ML systems that process protected health information (PHI), with additional considerations addressing AI-specific risks (training data PHI exposure, model memorization, output PHI disclosure). China generative AI measures and related frameworks Regulate generative AI services, algorithmic recommendation systems, and deep synthesis (synthetic media) provided to users in mainland China through specific, binding obligations enforced by the Cyberspace Administration of China and other regulators. UK, Singapore, Japan, and other emerging frameworks Cover the collection of national AI governance frameworks outside the EU, US, and China that shape industry practice in their markets through voluntary frameworks, sectoral guidance, or developing legislation. Singapore Model AI Governance Framework for Agentic AI Give organizations deploying AI agents, whether built in-house or bought, a government reference for doing it responsibly: bound the agent's risks up front, keep humans meaningfully accountable with defined approval checkpoints, apply technical controls across the agent lifecycle, and equip end users to use and oversee agents. Model Cards and Data Sheets Provide standardized documentation formats summarizing what AI models do (Model Cards) and what data they were trained on (Data Sheets), with enough structure to be machine-readable and enough flexibility to apply across model types. AI Bills of Materials (AIBOM) Provide a structured inventory of components, dependencies, and provenance information for AI systems, modeled on Software Bills of Materials (SBOMs) used in software supply-chain security, adapted to capture AI-specific components (foundation models, fine-tuning data, embedding models, prompts as code). SOC 2 with AI considerations and emerging AI assurance services Provide audit-attested evidence of AI governance practices through SOC 2 reports with AI-specific Trust Services Criteria considerations or through standalone AI assurance services, in formats that enterprise procurement accepts as third-party validation.
Workload identity systems (SPIFFE/SPIRE, cloud-provider patterns) Provide AI agents with cryptographic identity rooted in their deployment context (where they run, what image they run as, what cluster they're in) rather than in human-issued credentials, enabling fine-grained authentication and authorization without long-lived secrets. OAuth-for-agents and human-delegation flows Extend OAuth 2.1 patterns to AI agents acting on behalf of human users, providing scoped, time-limited, revocable delegation with explicit user consent for the specific operations the agent will perform. Verifiable agent identity on the web (Web Bot Auth, HTTP Message Signatures) Let an agent operator sign every outbound HTTP request with a published key, so that any website can verify which operator sent the request and apply its own policy, instead of guessing from the User-Agent string or IP ranges. Policy engines for agent authorization (OPA, Cedar, SpiceDB) Make authorization decisions for AI agent operations through externalized policy engines, with policies written in domain-specific languages, evaluated outside the application code, and auditable independently of the enforcing application. Secrets management adapted for AI agents (Vault, AWS Secrets Manager, Doppler) Provide centralized, audited, controlled storage and distribution of secrets that AI agents need to function, with automatic rotation, fine-grained access control, dynamic secret generation where appropriate, and immutable audit logging. Code execution sandboxes (E2B, Modal, Daytona, Cloudflare Sandboxes) Provide secure execution environments where AI agents can run arbitrary code (Python, JavaScript, shell commands, file operations) without compromising the host system, with each execution environment ephemeral, network-controlled, and bounded in resources. Browser automation sandboxes (Browserbase, Anthropic Computer Use, Anchor Browser) Provide isolated browser environments where AI agents can interact with web pages (clicking, scrolling, form filling, navigation) with the browser running in a controlled sandbox that bounds what the agent can access, what content can affect the host, and what state persists. MicroVM and container isolation substrates (Firecracker, gVisor, Kata Containers) Provide the substrate technology that production AI sandboxes are built on, with stronger isolation than container runtimes alone and lighter weight than full VMs, suitable for ephemeral per-task execution environments. OS-level sandboxing for local coding agents (Seatbelt, bubblewrap, sandbox-runtime) Confine the shell commands a local coding agent runs to an allowed set of directories and network destinations, enforced by the operating system, so the agent can work with fewer approval prompts without being able to write anywhere or reach any host on the machine's network. Model provenance and signing (Sigstore, Hugging Face security) Apply software supply chain provenance disciplines (signing, verification, attestation) to AI model artifacts — foundation model weights, fine-tuned models, training datasets — producing verifiable evidence of where models came from and what was done to them. Dependency scanning adapted for AI (Snyk, Aikido, Garak indirect) Extend dependency scanning beyond traditional software components to cover the AI-specific stack: model files, embedding models, RAG components, agent frameworks, prompt templates, and the broader AI dependency surface where vulnerabilities may originate from data-handling rather than code execution. MCP server and agent-skill supply-chain scanning (tool poisoning, rug pulls, pinning) Treat MCP servers and agent skills as third-party dependencies that can carry instructions as well as code: scan their tool descriptions and contents before use, pin the version that was reviewed, and detect any change made after approval. Immutable audit trails and SIEM integration for AI agents Provide tamper-resistant, regulator-grade audit logs for AI agent activity, integrated with SIEM platforms where security operations already work, capturing both AI-specific events (prompt injection attempts, jailbreak attempts, unusual tool-call patterns) and standard operational events (authentication, authorization decisions, data access). MITRE ATLAS Provide a comprehensive, structured catalog of adversarial techniques against AI systems with documented examples, mitigations, and detection strategies, modeled on the highly-adopted MITRE ATT&CK framework for general cybersecurity. OWASP AI Security and Privacy Guide and AI threat modeling patterns Cover the OWASP AI Security and Privacy Guide as a complementary resource to MITRE ATLAS, and the broader landscape of AI threat modeling patterns including STRIDE adaptations for AI systems. Prompt-injection containment design patterns (lethal trifecta, Dual LLM, Plan-Then-Execute, CaMeL) Structure an agent so that untrusted content it reads (documents, web pages, email, tool outputs) cannot choose or change the consequential actions it takes, by separating the component that plans and holds tools from the component that reads untrusted text, or by fixing the plan before untrusted text arrives.
Token streaming UX Reduce perceived latency for agent responses by streaming tokens to the UI as they generate, producing the feeling of an agent typing in real time rather than waiting silently and then dumping a complete response. Intermediate state display (thinking, tool use, retrieval) Show users what the agent is doing during the response generation — not just the final answer, but the reasoning, tool calls, retrieved sources, and other process steps — in a way that supports trust and debugging without overwhelming with cognitive load. AG-UI (Agent-User Interaction protocol) Give agent backends and frontends one shared, event-based wire format for streaming text, tool calls, reasoning, shared state, and human-in-the-loop interrupts, so a frontend built once can render runs from any compatible agent framework. Side-panel artifact workspace (Anthropic Artifacts) Provide a dedicated UI surface for substantial work products the agent generates — documents, code, interactive HTML, diagrams — displayed alongside the chat where the user iterates on them, distinct from the conversation history where chat-style interaction happens. Generative UI protocols (MCP Apps and A2UI) Let an agent or tool return interactive UI (forms, dashboards, charts, approval screens) that works across hosts, either by shipping a sandboxed web app with the tool (MCP Apps) or by describing the UI as data the host renders from its own trusted components (A2UI). Pre-action approval patterns Provide users with visibility into what the agent is about to do, with explicit opportunity to confirm, modify, or cancel before the action executes, scaled to the operation's significance and frequency. Undo-first design Let the agent act on routine operations without explicit approval, with a clear path to undo if the action is wrong. Reduce friction for the common case while preserving the user's control over outcomes. Citation patterns for AI responses Make the sources of agent claims visible and verifiable, so users can check the agent's work, understand which claims are supported by retrieved sources vs. model knowledge, and dig deeper into specific sources when needed. Confidence and uncertainty display Communicate to users when the agent is uncertain about claims, decisions, or actions, so users can apply appropriate skepticism and verify or override as needed. Status and progress UX for long-running agent tasks Communicate progress on tasks that take longer than reasonable synchronous wait times, in ways that let users monitor without forcing constant attention and that surface relevant state without overwhelming with raw trace data. Notification and resume patterns for sessions Let users start agent tasks and return to them later — across sessions, across devices, after the task completes — with the agent's state preserved and the user's context restored. Vercel AI SDK UI and generative UI Provide React, Vue, Svelte, and Angular hooks for building AI applications, with first-class support for streaming, tool calling, generative UI (tool results rendered as components), and multi-turn conversations. assistant-ui Provide a polished, batteries-included React component library for building AI chat interfaces, with thread management, attachments, tool calls, message editing, and many other features that production chat UIs need. AI Elements (shadcn-style) and CopilotKit Cover two complementary substrates: AI Elements as shadcn-style composable AI components for teams already using shadcn, and CopilotKit as a framework for embedding AI inside existing applications rather than as standalone chat. Multi-agent UX patterns and visualization Provide users with visibility into multi-agent systems — what each agent is doing, how they're coordinating, where intervention is possible — and intervention surfaces appropriate to the system's complexity. Realtime voice agent UX Make a spoken conversation with an agent feel natural: the agent knows when the user has finished speaking, stops talking when the user interrupts, remembers only what the user actually heard, and answers fast enough that the pause sounds like thinking rather than a dropped call.
Provider-native agent offerings Provide turnkey agent capabilities on top of foundation models, with deep integration that third-party products typically can't match, covering coding (Claude Code, Codex, Gemini Code Assist, Jules, Antigravity), computer use (Anthropic's computer use tool and Claude in Chrome, ChatGPT agent mode), and general assistance (ChatGPT, Claude.ai, Gemini). Claude Code, Cursor, Windsurf, GitHub Copilot — the supervised pair-programming tier Provide coding assistance that augments developer productivity through inline suggestions, multi-file edits, agentic task completion, and codebase-aware reasoning, with the developer reviewing each suggestion or task in real-time supervision mode. Devin, OpenAI Codex, Replit Agent — the autonomous async tier Provide async coding capability where the developer delegates entire tasks (a ticket, a bug fix, a feature implementation) and the agent plans, implements, tests, and submits a PR with minimal supervision, fitting workflow patterns where the alternative would be a junior engineer. Open-source coding agents (Aider, Cline, Continue, OpenCode) Provide credible open-source alternatives to commercial coding agents for teams or individuals who prioritize cost, inspectability, or customization over commercial polish. Prompt-to-app builders (Lovable, Bolt.new, v0, Replit) Let a person describe an application in chat and get back a working, hosted app with frontend, database, authentication and deployment, without opening an editor or reading the code. LangChain and LangGraph Provide a comprehensive Python and TypeScript framework for building LLM-powered applications and agents, with LangChain as the broader application framework and LangGraph as the specific orchestration layer for multi-step agent workflows. AutoGen, CrewAI, PydanticAI, Vercel AI SDK — the alternatives Cover the agent frameworks that hold meaningful positions in specific use cases or for specific developer preferences, alongside the dominant LangChain and provider-native options. Customer support agents (Decagon, Sierra, Intercom Fin) Automate customer support interactions through AI agents that handle inquiries autonomously, assist human agents with information retrieval and response drafting, and integrate with existing customer support infrastructure (Zendesk, Salesforce, custom ticketing). Voice agent platforms (ElevenLabs Agents, Vapi, Retell) Provide the speech, turn-taking and telephony layer for agents that answer and place phone calls, so a team can configure a voice agent's prompt, tools and voice instead of building the audio pipeline itself. Legal, research, and other vertical agents (Harvey, Hebbia, Glean, Perplexity) Cover three additional vertical categories with established players: legal AI dominated by Harvey for large-firm work, enterprise research dominated by Hebbia and Glean for internal knowledge synthesis, and Perplexity for consumer and enterprise research with web-search emphasis. Anthropic Computer Use, OpenAI Operator, and Browserbase Enable AI agents to operate computers (clicking, typing, scrolling, navigating, executing tasks in arbitrary applications) and browsers (visiting URLs, filling forms, extracting data, interacting with web applications), with appropriate sandboxing and human oversight. Enterprise agent platforms (Salesforce Agentforce, Microsoft Copilot Studio, Google Gemini Enterprise, OpenAI Frontier) Give an enterprise one governed place to build, connect, deploy and monitor many agents on its own data and systems, so business teams can create agents without each one becoming a separate engineering project with its own permissions and audit trail. Agent observability platforms (LangSmith, Phoenix, Langfuse, Helicone, Braintrust, Galileo) Provide the operational substrate for production agents: tracing of agent behavior, evaluation pipelines, prompt management, cost tracking, performance monitoring, and the operational tooling that distinguishes production deployments from prototypes.
System prompt design patterns Design system prompts that produce reliable agent behavior across the range of user inputs by establishing role, capabilities, constraints, and output guidelines in a structure the model handles well. Few-shot and N-shot prompting patterns Improve model performance on specific tasks by including 1–N examples of desired input→output pairs in the prompt, allowing the model to infer patterns from the examples rather than relying on instruction-following alone. Chain-of-thought and reasoning patterns Improve performance on complex reasoning tasks by prompting the model to think step-by-step before producing the final answer, and understand how reasoning-trained models have shifted the technique's applicability. Reasoning effort control Trade answer quality against latency and token cost per call by setting the model's effort level (Anthropic effort, OpenAI reasoning.effort, Gemini thinking_level), instead of prompting for more or less reasoning or setting a fixed thinking budget. Context selection and ordering at production scale Determine what content goes into the context window and in what order to maximize the model's ability to use it effectively, recognizing that larger context windows have not eliminated the need for this engineering. Context compression patterns Compress context content when the relevant material exceeds the available context budget, using summarization, extraction, hierarchical retrieval, or compression-specific models, with explicit awareness of the trade-offs each compression approach introduces. Server-side compaction Keep a long-running agent inside its context window by having the API replace older turns with a provider-written summary, triggered on demand or at a token threshold, instead of writing and maintaining a summarization loop in the application. Structured output prompting and vendor features Get reliable, parseable structured output (JSON, XML, custom formats) from LLMs using vendor-provided structured output features where available, falling back to prompt engineering and validation patterns where they aren't. Sampling parameters — temperature, top-p, and top-k Control the randomness and diversity of model output at generation time through the decoding parameters — temperature, top-p (nucleus sampling), and top-k — trading determinism against exploration to match the task, rather than accepting whatever the defaults produce. On current frontier APIs these parameters are deprecated or rejected, so the pattern now applies mainly to older models, open-weights models and self-hosted inference. Prompt registries and versioning (LangChain Hub, PromptLayer, vendor playgrounds) Treat prompts as versioned artifacts with explicit history, rollback capability, and lifecycle management, using prompt registry infrastructure that separates prompts from application code while maintaining the connection between specific prompt versions and specific deployment versions. Prompt testing and evaluation patterns Apply systematic testing discipline to prompts: verify specific behaviors against test cases, detect regressions when prompts or models change, compare prompt variants quantitatively for A/B testing decisions. Model-specific prompting conventions across major providers Use the prompting conventions each major foundation model handles best, recognizing that cross-model portability is partial and production deployments need to be aware of which conventions they're using. Meta-prompting and prompt optimization patterns Use foundation models to generate, optimize, or refine prompts for other AI tasks, leveraging the model's knowledge of effective prompt patterns to accelerate prompt development.
Per-task model selection patterns Match each subtask in an agent to the foundation model that best fits its specific requirements — capability, cost, latency, context window — rather than defaulting to one model for all subtasks. Prompt caching across providers (Anthropic, OpenAI, Google) Reduce inference costs and latency by reusing previously-processed prompt content across requests, using provider-native caching features that bill cached content at substantially reduced rates. Semantic response caching Skip the model call entirely when a new request means the same thing as one already answered, by matching it against stored requests on embedding similarity and returning the stored response. Latency engineering patterns for agent deployments Reduce the latency of agent responses through streaming (improves perceived latency), parallelization (reduces total latency for multi-step work), and batching (improves throughput for high-volume use cases). AI gateway (LLM proxy) Put one service in front of every model provider so that provider abstraction, fallback, rate limits, per-key budgets, caching and cost tracking are configured once, outside agent code, instead of being rebuilt in each agent. Fine-tuning techniques and when to use each Choose the right fine-tuning technique for the specific use case when fine-tuning is genuinely the right answer, recognizing that different techniques have different cost, complexity, and outcome profiles. Self-hosting vs. API economics Evaluate whether self-hosting open-weight models or consuming cloud-hosted foundation model APIs is more appropriate for a specific deployment, considering volume economics, deployment constraints, capability requirements, and operational capacity. Model versioning and migration patterns Handle the recurring cycle of model changes — new versions, deprecations, behavior drift — with patterns that minimize production disruption and maintain quality as models evolve.
TCO modeling for AI deployments Build a working model of all costs attributable to an AI deployment — inference, infrastructure, people, data, migration, risk — so that strategic decisions reflect the full cost picture rather than only the visible inference bill. Procurement strategy patterns for foundation model providers Move from commodity pay-as-you-go consumption to strategic procurement relationships when deployment scale justifies it, capturing volume discounts, negotiated terms, and provider commitments that improve TCO and reduce strategic risk. Build-vs-buy decision framework for AI stack layers Make conscious build-vs-buy decisions at each layer of the AI stack — application, agent logic, framework, foundation model, infrastructure — with rationale based on strategic value, differentiation potential, and capability fit rather than uniform defaulting. Modeling the latency-cost-quality frontier Make the latency-cost-quality trade-off explicit through modeling rather than implicit through engineering intuition, so deployment decisions about model selection, routing, and infrastructure reflect the actual frontier rather than assumed trade-offs. Performance benchmarking methodology for AI procurement Build benchmarks that answer procurement questions — which model represents best value for this use case, what are the strategic risks of each provider, how do the alternatives compare across all relevant dimensions — rather than benchmarks that only optimize engineering parameters. FinOps for AI — the organizational practice Apply FinOps practice to AI deployments — cross-functional accountability for cost, continuous visibility into spending, optimization driven by business value rather than engineering preference — with the discipline adapted for AI-specific cost patterns. FOCUS-normalized AI cost data Normalize every provider's billing export (model APIs, cloud, SaaS, GPU clouds) into the FOCUS schema before allocating or reporting it, so showback, chargeback and anomaly detection run on one dataset instead of one parser per vendor. Capacity planning patterns for AI workloads Provision AI capacity — API rate limits, reserved capacity, infrastructure for self-hosted models, GPU allocations — in ways that handle realistic demand variability without over-provisioning waste or under-provisioning failures. Service-tier selection (Priority / Standard / Flex / Batch) Route each workload to the cheapest service tier whose latency and availability it can tolerate, since the major providers now sell the same model's tokens at several price points chosen per request. Outcome- and action-based pricing for agent products Price an agent product per unit of work it completes (a resolved conversation, an action taken) so that revenue scales with the same activity that drives inference cost, and the customer pays for results rather than access.
CLAUDE.md and the memory hierarchy Give Claude Code durable, project-aware context by placing instructions in CLAUDE.md files across a defined hierarchy (enterprise, user, project, personal-local, and subdirectory) that load together into context, so the agent behaves consistently without being re-briefed each turn. Writing effective CLAUDE.md instructions Write CLAUDE.md content that changes agent behavior reliably: specific, actionable, concise directives that earn their place in the context window, rather than a verbose document the agent skims and half-follows. Bootstrapping with /init Generate a first-draft CLAUDE.md automatically by having Claude Code analyze an existing repository with the /init command, then refine the draft rather than writing standing instructions from a blank page. Path-scoped rules and auto memory Split standing context so each piece loads only when it applies: topic rule files in .claude/rules/, optionally scoped to file paths so they load only when Claude works on matching files, and auto memory, a per-repository set of notes Claude writes and reads back across sessions. Custom slash commands Package a repeatable instruction as a markdown file that becomes a slash command, so a common task — a review, a scaffold, a fix routine — is invoked by name with arguments instead of re-typed each time. Project subagents Define specialized subagents as files in the project so Claude Code can delegate a bounded job — a code review, a codebase search, a test-writing pass — to a separate agent with its own context window, its own instructions, and a restricted tool set. Plugins Bundle a team's Claude Code configuration (skills and commands, subagents, hooks, and MCP servers) into a plugin that installs as one unit from a marketplace, so the same setup reaches every repository and every developer without copying files by hand. The permission model Bound what Claude Code may do without asking through permission rules — allow, ask, and deny lists matched against tool calls — so routine actions run friction-free, sensitive ones prompt for confirmation, and forbidden ones are blocked outright. Auto mode (classifier-reviewed permissions) Let Claude Code work through long tasks without routine permission prompts by routing each tool call through a separate classifier model that blocks anything irreversible, destructive, beyond the request, or aimed outside the trusted environment, while deny and ask rules still apply first. Lifecycle hooks Run your own commands at defined points in Claude Code's lifecycle — before and after tool calls, on session and turn boundaries, on prompt submission — so policies and side effects execute deterministically rather than depending on the model choosing to perform them. Plan mode and approval workflows Separate deciding from doing by running Claude Code in a read-only plan mode: the agent investigates and proposes a plan, a human reviews and approves it, and only then does execution begin — so consequential work is inspected before any file is touched. Headless mode and CI integration Run Claude Code non-interactively — one prompt in, structured output and an exit code out — with the print flag, so a script or a CI job can drive the agent unattended as a pipeline step. The Claude Agent SDK Embed Claude Code's agent loop inside an application through the Agent SDK, constructing sessions programmatically — with the same tools, permissions, and CLAUDE.md context as the CLI — so the agent becomes a component of a larger system rather than a tool a human drives. Spec-driven development (GitHub Spec Kit, Kiro, Tessl) Write a structured, behavior-oriented specification in natural language before any code, then have the coding agent derive a plan and a task list from it and implement task by task, with a human reviewing each artifact, so the spec becomes the shared source of truth for the human and the agent.
Turn-Based Loop Run the agentic loop at human pace, with the user approving each turn before the next begins. Claude gathers context, acts, checks its work, repeats if the turn is not yet complete, and responds — and the human decides whether to continue, redirect, or stop. The stop decision never leaves the person at the keyboard. Goal-Gated Loop Hand the stop decision off from the human to an evaluator. Each time the agent tries to stop, an evaluator checks the run against a stated condition; if the condition is unmet, it sends the agent back to work; if it is met, the run ends. The human states the goal once and steps away from the per-turn approval. This rung is how Volume 01's Goal Setting and Monitoring pattern gets the stop decision: that pattern defines and tracks the objective, this one acts on it. Interval Loop Re-run a prompt on a fixed local interval so the agent reacts to an external system that changes on its own. The clock, not a human and not an evaluator, invokes each iteration; the run continues as long as the local session is alive. Local Scheduled Loop Run a prompt on a schedule from a scheduler on your own machine, so the routine outlives any one session while keeping direct access to your local files and tools. Each firing starts a fresh session; no terminal has to stay open, but the machine has to be on. Proactive / Scheduled Loop Move the routine off your machine and into the cloud, where it persists without your session. A schedule or an external event invokes each run; the routine executes whether or not you are at your desk, and no human is present in real time to end it. Loop Composition Build long-running autonomy by composing the primitives rather than choosing one. A schedule to wake the run, a goal to know when to stop, parallelism to explore, and permission to proceed unattended combine into a single durable operation that no one rung could deliver alone.
No patterns match that filter.