Infrastructure Patterns for Agentic Applications
· 20 min read
Most teams start building agents the same way they build any other web feature: wrap the model in a route handler, parse the request, and wait for the response.
app.post("/agent", async (req, res) => {
const context = await memory.load(req.body.userId)
const result = await agent.run({ prompt: req.body.prompt, context })
res.json(result)
})This is fine for a demo. It is also a trap.
Agents are long-running, stateful, and non-deterministic — not fast, stateless, or deterministic. They call tools, wait for external APIs, branch into subtasks, hit rate limits, and crash halfway through multi-step sequences. If your agent is tied to a single HTTP request, you've coupled your application's reliability to the wall-clock time of a non-deterministic loop.
Moving past the demo means breaking that coupling. This post walks through the three core patterns teams use to turn fragile agent scripts into scalable, resilient production systems.

Walking through these patterns live at Localhost 2026.
##The Nature of Agents
At the core of an agent is a loop: receive a goal, decide what to do next, call a model or tool, observe the result, update state, repeat until done. That loop has a few properties that make it hostile to naive web architectures.
Agents are long-running. A normal API request should finish quickly. Agents often don't, sometimes taking hours or even days to complete.
Agents are stateful. A run isn't one function call. It's a goal, a plan, tool calls and outputs, retries, errors, decisions, and a final output. If the process crashes, you need to know what already happened. If not, you either lose progress or rerun work blindly. Neither is great, especially for long-running agents.
Agents are non-deterministic. Traditional workflow code says:
const user = await db.users.find(userId)
const invoice = await billing.createInvoice(user)
await email.sendInvoice(user.email, invoice)
await db.invoices.markSent(invoice.id)Agent code says:
while (!run.complete) {
const context = await memory.load(run.id)
const action = await agent.decide({
goal: run.goal,
context,
tools: toolRegistry.availableTools(),
})
if (action.type === "finish") {
await runs.markComplete(run.id, action.answer)
break
}
const observation = await tools.execute(action)
await memory.append(run.id, {
action,
observation,
})
}The model chooses the next action at runtime, which makes recovery, replay, and debugging much harder.
A run may take three steps or thirty. It may fan out across several tools, wait for a human, produce large intermediate artifacts, or stop early. Infrastructure has to set boundaries around that unpredictability with timeouts, budgets, checkpoints, approvals, and explicit termination conditions.
Taken together, these properties mean agents need more than model calls wrapped in application code. They need architecture that decouples runs from requests, and infrastructure that preserves progress, records decisions, controls side effects, and manages artifacts separately from model context.
##Pattern: Background Worker + Queue
The first production pattern is to stop running the whole agent inside the web request. A request is the wrong lifetime for an agent run. It should create a durable run record, enqueue the work, and return immediately.
That gives the application a simple boundary:
`API → Queue → Worker → Database
`
The API starts the run. The queue stores the work durably. The worker executes the run. The database records progress.
// api.ts
app.post("/runs", async (req, res) => {
const run = await db.agentRun.create({
data: { status: "queued", input: req.body },
})
await queue.send({ type: "run_agent", runId: run.id })
res.status(202).json({ runId: run.id, statusUrl: `/runs/${run.id}` })
})
// worker.ts
queue.process("run_agent", async (job) => {
const { runId } = job.data
await db.agentRun.update({
where: { id: runId },
data: { status: "running" },
})
try {
const run = await db.agentRun.findUniqueOrThrow({ where: { id: runId } })
const result = await runAgent(run.input)
await db.agentRun.update({
where: { id: runId },
data: { status: "completed", result },
})
} catch (error) {
await db.agentRun.update({
where: { id: runId },
data: { status: "failed", error: serializeError(error) },
})
throw error
}
})The client gets a run ID immediately. The agent runs somewhere else. The user checks status, subscribes to updates, or receives a callback when the run completes.
A queue is a durable buffer between the thing that creates work and the thing that performs it. This is the default starting point when runs are longer than a normal request, tasks are mostly independent, and you need worker scaling, retries, or burst absorption.
The trap: a queue knows a job exists, but it doesn't know the logical process that job belongs to. One background job is fine. Twenty dependent steps, three retries, two branches, and a human approval pause is not something a queue manages for you. You'll build that logic yourself. As we'll see later, that's where workflow engines come in.
##Reliability: Safe Retries and Partial Failure
Moving work to a queue solves the lifetime problem. It does not solve for durability of execution.
Most production queues are at-least-once: a job may run more than once. That's a deliberate tradeoff to avoid losing work, and your code has to survive it. Once an agent can call tools, write records, send messages, or provision resources, retry behavior becomes part of the application's correctness model.
Two disciplines matter most. Idempotency answers what happens when a single step runs twice. Compensation answers what happens when a run stops partway through a sequence of steps. Retries make the first a requirement. Permanent failures make the second necessary for production. Agents that touch remote services need both.
###Idempotency: Surviving a Step That Runs Twice
Bad worker code assumes no crash between steps:
await sendEmail(user.email, message)
await markEmailSent(user.id) // crash here → duplicate email on retryBetter code creates an idempotency boundary:
queue.process("send_email", async (job) => {
const email = await db.email.findUniqueOrThrow({ where: { id: job.data.emailId } })
if (email.status === "sent") return
await emailProvider.send({ to: email.to, subject: email.subject, body: email.body, idempotencyKey: email.id })
await db.email.update({ where: { id: email.id }, data: { status: "sent" } })
})For agents, every side-effecting tool call needs the same treatment: check for a completed record before calling the tool, upsert a "running" record, call the tool, mark it complete.
async function executeToolCall(runId: string, toolCall: ToolCall) {
const existing = await db.toolExecution.findUnique({
where: { runId_toolCallId: { runId, toolCallId: toolCall.id } },
})
if (existing?.status === "completed") return existing.result
await db.toolExecution.upsert({
where: { runId_toolCallId: { runId, toolCallId: toolCall.id } },
update: {},
create: { runId, toolCallId: toolCall.id, toolName: toolCall.name, status: "running" },
})
const result = await tools.call(toolCall.name, toolCall.args)
await db.toolExecution.update({ where: { runId_toolCallId: { runId, toolCallId: toolCall.id } }, data: { status: "completed", result } })
return result
}Retry is not a recovery strategy unless the retried operation is safe.
###Compensation: Surviving a Run That Stops Halfway
Consider an agent that completes three of five steps: it charges a card, provisions a resource, and sends a confirmation email. Step four then fails permanently. None of the first three steps can be rolled back with a database transaction, since a charge is not undone by deleting a database row; the money has already moved. Restarting the run from step one would recharge the card and resend the email, recreating the exact duplicate-side-effect problem idempotency was meant to prevent.
Distributed systems have a standard answer to this: the saga pattern. For every side-effecting action an agent can take, define a compensating action that reverses the effect. A charge is reversed with a refund. An email is reversed with a correction message. When a run fails permanently, the orchestrator walks the completed steps in reverse order and runs their compensations.
const compensations: Record<string, (ctx: StepContext) => Promise<void>> = {
charge_card: async (ctx) => refund({ chargeId: ctx.result.chargeId }),
provision_resource: async (ctx) => deprovision({ resourceId: ctx.result.resourceId }),
send_email: async (ctx) => sendCorrection({ originalMessageId: ctx.result.messageId }),
}
async function compensate(runId: string) {
const completed = await db.stepExecution.findMany({
where: { runId, status: "completed" },
orderBy: { completedAt: "desc" },
})
for (const step of completed) {
await compensations[step.name]?.(step)
await db.stepExecution.update({ where: { id: step.id }, data: { status: "compensated" } })
}
}Compensations must be idempotent in the same way forward actions are. Compensations can also fail on their own; a refund API can be unavailable just as easily as a charge API. A compensation chain needs a bounded retry, a dead-letter path for compensations that will not complete, and a manual escape hatch. Without these, a failed run can end up half-undone instead of half-done, which is not an improvement.
Compensation is not always the right response to a partial failure. If four of five parallel sub-orders succeeded and one failed, completing the four and dropping the fifth is often the better outcome. This avoids unwinding work that succeeded. Compensation is worth building when partial success is unacceptable, not as a default response to any failure.
Idempotency covers a step that runs twice. Compensation covers a run that stops halfway through. Both are required whether the run is executed by a simple worker or by a workflow engine.
##Pattern: Workflows
A queue distributes work, but it does not remember the shape of a run. It knows that a job exists. It does not know that step three depends on step two, that a human approval is pending, that two branches need to join before synthesis, or that compensation should run if the final step fails.
Once the hard part is no longer where should this job run? but what should happen next, given what already happened?, you have moved from queueing into orchestration.
A workflow engine provides that orchestration layer. It stores the history of a run: which steps started, completed, failed, retried, timed out, or waited for external input. In a workflow, a crash doesn't mean starting over.
The code splits into two layers: a workflow function that decides what happens next, and steps that do the actual work — model calls, tool executions, record writes. Engines differ on naming (Temporal calls them activities; others use steps or tasks), but the split is the same everywhere.
`Workflow → Steps → Durable history → Replay/resume
`
export async function agentWorkflow(input: AgentInput) {
const run = await steps.createRun(input)
const plan = await steps.callModel({ runId: run.id, purpose: "plan", input })
for (const step of plan.steps) {
const result = await steps.executeTool({ runId: run.id, step })
await steps.recordStepResult({ runId: run.id, stepId: step.id, result })
}
const final = await steps.callModel({
runId: run.id,
purpose: "synthesize",
input: { originalGoal: input.goal, stepResults: await steps.loadStepResults(run.id) },
})
await steps.completeRun({ runId: run.id, result: final })
return final
}Workflow code should stay deterministic: no direct LLM calls, external API calls, random numbers, or wall-clock reads in the orchestration function itself. Those belong in steps. The determinism requirement comes from how recovery works. After a crash, the engine rebuilds a run by re-executing the workflow function from the top, substituting recorded results for steps that already completed. Replay only lands in the same place if the code makes the same decisions every time; an LLM call, a clock read, or a random number inside the orchestration function can return a different answer the second time through, and the resumed run silently diverges from its own history. Steps don't have this constraint because their results are recorded the first time they run and replayed from history afterward. Durable execution tells you where a run was when it crashed. It does not remove the need for idempotency, because steps can still run more than once. This is why a workflow engine is the natural home for compensation logic: it already tracks which steps completed, which is the record a saga needs to walk backward.
###Scaling Workflows: Fan-Out, Fan-In, and Contention
Workflows become especially useful when a run branches. Fan-out is the common shape: a lead agent breaks a goal into independent subtasks, sends each to a worker or subagent, and then synthesizes the results.
`Orchestrator → N worker jobs → Result store → Orchestrator synthesis
`
This buys parallelism and fresh context windows, which is useful for research, document analysis, and other breadth-first work. But the hard part is not dispatching the workers. It is coordinating the fan-in: what happens if one branch fails, how many failures are tolerable, whether to synthesize partial results, and how to avoid waiting forever. A queue can run the workers. A workflow decides what happens next.
Fan-out is worth using when subtasks are independent, mostly read-only, and parallelizable. It works poorly when subtasks are tightly coupled or need to share state that keeps changing. Adding agents also adds cost and coordination overhead, so the question of how many agents to use matters more than the mechanics of spawning them.
Fan-out also introduces contention. A system can work with three parallel subagents and fail at eleven, even when there is plenty of aggregate provider capacity. Uncoordinated calls can collide on the same rate limit, exhaust the same connection pool, or retry at the same time. Treat providers, databases, and model APIs as shared resources: cap concurrency, use token buckets where needed, and back off when the shared resource pushes back.
###Putting Patterns Together: Workflow + Queue
At scale, a workflow coordinates the process and a queue distributes execution. They are not competitors; they operate at different layers.
`Workflow engine → Queue → Specialized workers → Workflow result
`
export async function multiAgentWorkflow(input: AgentInput) {
const run = await steps.createRun(input)
const plan = await steps.planRun({ runId: run.id, goal: input.goal })
// Fan out: dispatch each subtask to the queue; a worker pool executes them in parallel.
const settled = await Promise.allSettled(
plan.tasks.map((task) => steps.runSubagentTask({ runId: run.id, task }))
)
// Fan in: the part a queue won't do for you.
const results = settled
.filter((s) => s.status === "fulfilled")
.map((s) => s.value)
const failures = settled.filter((s) => s.status === "rejected")
// Decide what "enough" means before synthesizing.
if (results.length < plan.minResults) {
await steps.failRun({ runId: run.id, failureCount: failures.length })
throw new Error(`only ${results.length}/${plan.tasks.length} subtasks succeeded`)
}
const final = await steps.synthesize({ runId: run.id, results })
await steps.completeRun({ runId: run.id, result: final })
return final
}Two details carry the weight here. `Promise.allSettled` keeps one failed branch from poisoning the entire fan-in, and the success threshold makes the partial-failure policy explicit: the workflow, not the queue, decides whether the run can proceed with what came back.
This is the mature pattern for high-scale agentic systems: durable process state, multiple worker pools, rate-limited dispatch, audibility. The cost is real operational complexity. You now have an API plane, an orchestration plane, and an execution plane. This is worth it when the workload demands it.
##Making It Concrete
The patterns above are platform-agnostic. You can assemble them from any queue, any workflow engine, and any worker fleet. The question is how much of that assembly you want to own and operate.
On Render, the three planes map directly onto managed primitives. A Web Service is the API plane. Workflows is the orchestration and execution plane: TypeScript or Python tasks running in on-demand containers, with queueing, retries, chaining, logs, and per-run status handled by the platform. Cron Jobs cover scheduling, Key Value covers shared state and rate limiting, and Postgres holds the run record. There is no queue to provision, no worker fleet to keep warm, and no orchestrator cluster to operate.
The key question is still the same: which layer starts the run, which layer coordinates progress, and which layer executes work? Render collapses the plumbing between those layers, not the boundaries. The rest of this section walks the same architecture, primitive by primitive.
####Triggering a run
Use a Render Web Service for the API. It creates a run record, starts the workflow, and returns immediately:
// api.ts — Render Web Service
import { Render } from '@renderinc/sdk'
const render = new Render()
app.post('/runs', async (req, res) => {
const run = await db.agentRun.create({ data: { status: 'queued', input: req.body } })
const startedRun = await render.workflows.startTask(
'agent-workflow/runAgent',
[run.id, req.body]
)
await db.agentRun.update({
where: { id: run.id },
data: { renderTaskRunId: startedRun.taskRunId },
})
res.status(202).json({ runId: run.id, statusUrl: `/runs/${run.id}` })
})
app.get('/runs/:id', async (req, res) => {
res.json(await db.agentRun.findUniqueOrThrow({ where: { id: req.params.id } }))
})`startTask` creates the task run and returns immediately with a `taskRunId`. If the caller needs to block until completion, the TypeScript SDK also provides `runTask`; for an API request boundary, `startTask` is the right fit.
Notice what is missing: there is no queue to stand up, no message schema to design, no consumer to configure. `startTask` is the enqueue. The platform stores the work durably, dispatches it to an isolated container, and exposes its status — the entire API → Queue → Worker boundary from the first pattern, in one SDK call.
####Defining the workflow
The Workflows service owns task-run execution and chaining. Your database still owns the agent run's application state. `runAgent` is the entry point; `executeStep` is the fan-out task and idempotency boundary:
// workflow.ts — Render Workflows service
import { task } from '@renderinc/sdk/workflows'
const runAgent = task(
{ name: 'runAgent', timeoutSeconds: 3600 },
async function runAgent(runId: string, input: AgentInput) {
await db.agentRun.update({ where: { id: runId }, data: { status: 'running' } })
const plan = await planSteps(runId, input)
// Calling a task function from another task starts chained task runs.
const results = await Promise.all(
plan.steps.map((step) => executeStep(runId, step))
)
const final = await synthesize(runId, input.goal, results)
await db.agentRun.update({ where: { id: runId }, data: { status: 'completed', result: final } })
return final
}
)
const executeStep = task(
{ name: 'executeStep', retry: { maxRetries: 3 } },
async function executeStep(runId: string, step: PlanStep) {
const existing = await db.toolExecution.findUnique({ where: { runId_stepId: { runId, stepId: step.id } } })
if (existing?.status === 'completed') return existing.result
await db.toolExecution.upsert({
where: { runId_stepId: { runId, stepId: step.id } },
update: {},
create: { runId, stepId: step.id, status: 'running' },
})
const result = await tools.call(step.tool, step.args)
await db.toolExecution.update({ where: { runId_stepId: { runId, stepId: step.id } }, data: { status: 'completed', result } })
return result
}
)`Promise.all` fans out chained task runs. In the TypeScript SDK, `executeStep(...)` is a task-chain invocation because `executeStep` is the wrapped function returned by `task(...)`; it is not an ordinary local function call. Each task can use its own instance size, so planning, tool execution, and synthesis do not need the same compute profile.
That per-task sizing is hard to get from a homogeneous worker fleet, where every worker must be provisioned for the largest task it might receive. Here, a lightweight planning step and a memory-hungry synthesis step each declare their own profile and bill only for the seconds they run. Notice also what the snippet does not contain: no queue client, no polling loop, no worker registration. Two `task()` wrappers are the entire execution plane.
The idempotency check is required. If `runAgent` retries, it may re-issue `executeStep`. Workflows gives you durable runs and retries; it does not give you exactly-once side effects.
####Scheduling and approval
For scheduled agents, use a Render Cron Job to call `startTask` or the Render API's Run task endpoint.
For approval gates, split the workflow: one task writes a pending approval record, the web service handles approve/reject, and approval triggers the side-effecting task. Today this is an application-level pattern; native approval gates are on the Workflows roadmap (more on that below).
####Rate limiting and shared state
Recall the contention problem from the fan-out section: parallel subagents colliding on the same provider rate limit, connection pool, or retry window. Render gives you both halves of the fix on the same platform.
For limits, use Render Key Value for tenant-, provider-, or model-level token buckets. It is a managed Redis-compatible store on your private network, so admission control is a few atomic counter operations — not a new piece of infrastructure with its own vendor, auth story, and failure modes.
For pacing, place a small, always-on Background Worker in front of Workflows as a metering gateway. The web service enqueues work into Key Value immediately; the worker drains that queue at a controlled rate, calling `startTask` only as fast as the downstream provider can absorb it. The background worker does no agent work itself, it only meters the rate at which work reaches your Workflows, which still handle the actual multi-step processing with their own isolation, retries, and per-task compute sizing.
This is where composing primitives on one platform pays off. The metering gateway is a few dozen lines deployed from the same repo, talking to Key Value over private networking, visible in the same dashboard as the workflows it protects. Assembling the equivalent from a standalone queue, a separate Redis, and self-managed workers is a project; here it is an afternoon.
####Artifacts
Task runs are isolated, ephemeral containers — a feature, not a limitation, because no run can corrupt another's state. Store large outputs in Render Postgres or external object storage, then pass artifact IDs between tasks: pointers, not payloads. State that must outlive a task goes in a store; state that must not leak between runs dies with the container.
####Observability
Render gives you task status, retries, and logs per run. Store the Render `taskRunId` returned by `startTask` alongside your own `runId`, then thread both through task arguments, database rows, and logs.
####What this doesn't solve (yet)
Workflows removes queue and worker plumbing. Today, it does not remove application-level correctness work: idempotency, compensation, artifact storage, trace propagation, approval boundaries, cost controls, and rate limits are still yours to build with the patterns in this post.
That boundary is moving, though. Much of this list is exactly where Workflows is headed: stronger durability guarantees, first-class idempotency, and human-in-the-loop primitives like native approval gates are all on the roadmap. The patterns in this post will stay the same, but expect more of them to ship as platform features instead of application code you maintain.
Workflows is also in public beta, with current limitations around supported languages, scheduling, Blueprint support, and HIPAA-compatible hosts.
What no platform can remove is the design burden of orchestration. You still have to decide where state lives, where retries are safe, where approval happens, and how a run is traced end to end. The platform's job is to make those decisions cheap to implement, not to make them for you.
##Conclusion
The next generation of agentic applications will be built like distributed systems, whether the underlying infrastructure is hand-rolled or managed.
Remember that a queue asks: where should this work run? A workflow asks: what should happen next, given what has already happened? An agent asks: what should I do next to accomplish the goal?
Render Workflows can make the execution layer simpler. It can start task runs, queue them, retry them, and show their status. But the application still owns the semantics of the run: what counts as progress, which side effects are safe to retry, where artifacts live, when a human must approve, and how partial failure is repaired.
That is the real boundary. Infrastructure moves work. Workflows preserve progress. Agents choose actions. Reliable agentic applications come from keeping those responsibilities separate, even when the platform makes them easy to deploy together.