Skip to content
All posts
1 min read

Designing reliable AI agents for production

  • AI Agents
  • LLM
  • Architecture

A demo agent and a production agent can share the same prompt and still behave completely differently. The difference is everything around the model: how tools are defined, how state is stored, how failures are retried, and how humans stay in the loop.

Treat the model as an unreliable dependency

I design agent systems the same way I design integrations with a flaky third-party API. Every model call has a timeout, a retry budget, structured output validation, and a fallback path. If the output does not match the schema, it never reaches the rest of the system.

const result = await withRetry(() => model.generate(prompt), { attempts: 3 });
const parsed = PlanSchema.safeParse(result);
if (!parsed.success) return fallbackPlan(input);

Keep tools small and explicit

  • One tool, one responsibility, with typed inputs and outputs
  • Side-effecting tools require an explicit confirmation step
  • Every tool call is logged with its arguments and result

Make every run observable

Traces are the debugger for agents. Capturing the full sequence of prompts, tool calls, and decisions turns "the agent did something weird" into a reproducible bug report you can actually fix.

Reliability is not a property of the model. It is a property of the system you build around it.

Building something similar? Let's talk.

Get in touch