Stage 11 of 12 · about 3.1 h

Deploying agents

Serve Pocket's agent, box it in, watch it run, and roll it out safely.

Turn Pocket's agent into a service other people can rely on. You serve it as background jobs with idempotent submission and cancellation, run model-written code in a sandbox and keep secrets out of the model's context, trace every run as spans with budgets, rate limits and a circuit breaker, and release changes through an offline gate, shadow traffic and a hash-bucketed canary that rolls itself back.

Before you start

You can run a bounded agent loop with guarded tools as in stage 6, and trace, price and gate Pocket on a golden set as in stage 7.

When you finish

pocket/stage11/ holds serve.py, sandbox.py, observe.py and rollout.py: Pocket's agent served as background runs with events, idempotent retries and cancellation, model-written code run under limits with secrets kept in the tool layer, every run traced as spans with per-user rate limits, budgets, a circuit breaker and alerts, and releases rolled out through a canary with an automatic rollback rule and a kill switch. This is the end of the path. You trained a model with gradient descent, built a small transformer, built an LLM app with structured output, added retrieval and made it production RAG, gave it tools, built agents and agentic patterns, tested and evaluated them, and deployed an agent as a service you can watch, limit and roll back.

1. Lesson 1 · 45 min

   Serving an agent: runs, workers and events

   You can serve an agent as a background job: POST /runs returns a run id at once, a worker runs the loop and records an event per step, the client polls with a cursor, and retries and cancels are safe.

2. Lesson 2 · 45 min

   Sandboxes and least privilege

   You can run model-written code in a subprocess with a wall-clock timeout, CPU and file-size limits, a scrubbed environment and a temporary directory, say what only a container or VM adds, keep secrets inside the tool layer, and gate tools with a default-deny policy table.

3. Lesson 3 · 45 min

   Production observability and cost control

   You can trace an agent run as a tree of spans, enforce per-run and per-user budgets, rate-limit each user with a token bucket, wrap a failing tool in a circuit breaker, and alert on error rate and p95, all tested with a fake clock.

4. Lesson 4 · 50 min

   Rollout and operations

   You can version a release as prompt, model, tools and config together, move it through an offline gate, shadow traffic and a hash-bucketed canary, compare canary and baseline with a written rollback rule, and run the service with a kill switch, a review queue and an incident runbook.