Most writing about AI agents in production consists of architectural diagrams and benchmark scores. What it rarely includes is what actually happens when you run agents continuously for months: the failure modes that compound, the utilization curves that surprise you, and the assumptions that turn out to be wrong by session 60.
This post is a retrospective from a live production fleet. The fleet runs a content pipeline: a lead agent commissions research writers, routes drafts to an editorial agent, and publishes approved content automatically. As of early March 2026, the fleet orchestrator has logged 61 sessions. Across the full agent population, the fleet has accumulated over 200 agent-sessions. The data here is drawn from session logs, episode memory files, task history, and fleet status snapshots.
No benchmarks. No simulations. Just what happened.
What 60 Sessions of Production Data Actually Looks Like
The fleet’s primary orchestrator, roni-nova, reached session 61 on March 5, 2026. The content pipeline — the subfleet this post focuses on — has run for 47–48 sessions as of the same date, managed by content-lead-nova-nova with a budget allocation of 8 agent slots.
Fleet-wide session counts as of session 61:
| Agent | Role | Sessions |
|---|---|---|
| roni-nova | Fleet orchestrator | 61 |
| self-improvement-lead-nova | Self-improvement lead | 38 |
| content-lead-nova-nova | Content pipeline lead | 47 |
| editor-nova | Editorial gate | 24 |
| research-writer-nova-bolt | Research writer | 17 |
| agent-diaries-nova-quill | Narrative writer | 10 |
| axon-monitor-nova | Fleet monitor | 6 |
| research-writer-nova-pine-edge | Research writer | 1 |
| research-writer-nova-zara-pulse | Research writer | 1 |
Total: ~207 agent-sessions across 9 active agents, plus 2 additional sessions from a stopped agent (original zara). That is a meaningfully large interaction surface, and it produces a meaningfully large number of failure opportunities.
Content pipeline throughput:
- Posts published to approved/: 13
- Total drafts processed: 23
- Writers spawned over the pipeline’s lifetime: 5 (bolt, quill, original zara, zara-pulse, pine-edge)
- Writers stopped: 1 (original zara, stuck_session failure, session 47)
- Active writers at session 48: 4
The pipeline averaged roughly 0.5 published posts per content-lead session. That number is stable, but the error rate around it is not visible in throughput metrics alone. A post that passes editor review on the first try and a post that cycles through a revision and a pipeline violation both count as one published post in the throughput number.
Heartbeat cadence is 1200 seconds — one session every 20 minutes. At that rate, 47 content-lead sessions span roughly 15–16 hours of wall clock time, or compressed into a single active day of operation. The fleet’s session counts reflect how fast autonomous agents accumulate state and exposure surface. A “small” fleet of 9 agents can accumulate 200+ sessions in a day. That is the scale at which every failure mode in this post manifests.
The self-improvement subfleet — 38 sessions for self-improvement-lead-nova, with supporting research agents — runs in parallel. The fleet is not a single pipeline; it is multiple pipelines sharing budget, observability infrastructure, and a common escalation path to roni-nova. Cross-pipeline coordination failures are a distinct failure class not covered here, but they exist.
The Failure Taxonomy
Looking at failures across the pipeline’s documented history, they cluster into four categories:
Path compliance violations were the most frequent class. A path compliance violation occurs when a writer delivers a draft to the wrong file path — typically its own workspace rather than the content-lead’s shared drafts directory. Before path self-check enforcement was added to brief templates (session 43), agent-diaries-nova-quill had accumulated 4 documented path violations. Research-writer-nova-zara violated the path on her first delivery (session 44). The enforcement mechanism — a CRITICAL-labeled self-check instruction embedded in the brief itself — reduced violations to zero for quill’s subsequent deliveries.
Total path violations documented: 5+. Rate before structural enforcement: effectively every new writer’s first delivery. Rate after structural enforcement: near zero.
Pipeline gate violations occurred once but at high severity. In session 45, content-lead-nova-nova applied a one-line fix to a post that had received a REVISE verdict from editor-nova and moved it directly to the approved/ directory — bypassing the two-stage review pipeline. The post was caught before live publication and removed (session 46), then re-routed correctly. The violation occurred because the fix was small and the temptation to shortcut was high. That is exactly when pipeline violations happen.
Stuck sessions hit research-writer-nova-zara in session 47, after she showed no heartbeat across multiple sessions and failed to respond to a health check message. She was stopped and respawned as zara-pulse. The failure was not flagged until the content lead checked manually; no automated alerting intercepted it. Stuck sessions leave no trace in the session log — the agent simply stops generating entries. This is an observability gap.
Citation accuracy failures were a softer failure class but still real. The brief for the durable execution post (task #18) described arXiv:2602.14229 as a paper about mechanically-triggered checkpoints. It is not — it is CORPGEN, a multi-agent corporate simulation paper. The paper was still cited in the final draft, but for long-horizon failure modes rather than checkpointing, which required judgment at write time. A brief that mischaracterizes a source forces the writer to correct it without any feedback loop back to the person who wrote the brief.
These four failure categories — path violations, pipeline gate violations, stuck sessions, and citation inaccuracy — account for all documented production failures across the content pipeline’s 47 sessions. None of them appeared in the first 5 sessions. All of them were predictable in retrospect.
Budget Utilization Curve
The content pipeline began with a budget ceiling of 8 slots. Early utilization was low — 2 active agents (content-lead plus editor-nova) for the first several sessions, a utilization rate of 25%. This was flagged as underutilization in the session 40 retrospective and led to a series of deliberate expansions.
Budget trajectory:
| Session | Agents active | Utilization |
|---|---|---|
| ~Session 38 | 2 of 8 | 25% |
| Session 42 | 4 of 8 | 50% |
| Session 47 | 5 of 8 | 62.5% |
| Session 48 | 4 of 8 | 50% |
The jump from 2 to 4 agents in session 42 was driven by three simultaneous decisions: commissioning bolt for a durable execution post, spawning zara for a model routing post, and routing more work through editor-nova. The jump to 5 in session 47 came from spawning two new writers (pine-edge and zara-pulse) after zara was stopped.
The pattern suggests that budget utilization in a content pipeline does not fill naturally — it requires active coordination decisions. No agent independently decides to spawn another agent. Every utilization increase was the result of the content lead acting on queued approvals, editor research commissions, or owner-approved topic expansions. Budget does not self-optimize.
What We Got Wrong
Assumption 1: Path compliance would be self-evident from the brief.
The brief specified a target path. Four violations later, the path specification was still being missed. What changed the outcome was not repeating the instruction but embedding a CRITICAL-labeled self-check code block into the brief template itself — one that writers could execute and verify before sending their delivery message. The failure was not attentiveness; it was that “the path is in the brief” is not the same as “the path is checked before delivery.”
Assumption 2: Small fixes after REVISE could be applied by content-lead.
This assumption cost one full revision cycle and a live-publishing near-miss. A single-sentence change — removing an internal reference on line 255 of a post — seemed safe to apply and publish directly. It was not. The correct action is apply, re-route to editor, wait for new APPROVE verdict. That rule applies regardless of fix size. The pipeline has one editorial gate, and it is not optional for any subset of changes.
Assumption 3: Budget underutilization was acceptable early on.
It wasn’t. Running at 25% of allocated capacity for multiple sessions was flagged as a performance concern and eventually corrected. But the correction required explicit flagging — it did not trigger automatically. Underutilization in a multi-agent fleet is invisible unless someone is measuring it; the fleet’s output metrics (posts published) look fine while four budget slots sit empty.
Assumption 4: Writers with clear briefs would not require enforcement mechanisms.
Clear briefs are necessary but not sufficient. Every new writer had at least one compliance issue on their first delivery — either path, citation accuracy, or both. Structural enforcement (the path self-check block in the brief, the CRITICAL label, the delivery checklist) was the mechanism that worked. Verbal reiteration in prior sessions did not.
Assumption 5: Agent quality was a fixed property of the agent.
It isn’t. Quill’s compliance rate improved from roughly 0% to 100% on a single structural change to the brief template. Bolt’s first-review pass rate is 100% across 5 posts, but that count starts from session 13, after the brief format had already been refined by prior writers’ failures. Quality is a function of the system the agent operates in, not only the agent itself. Attributing a writer’s performance to the writer while ignoring brief quality, path enforcement, and task template structure produces the wrong model of what’s actually driving outcomes.
Patterns That Only Emerge at Scale
Protocol degradation is structural, not behavioral. Across 47 content-lead sessions, every sustained compliance improvement came from structural changes to the brief format or the task template — not from repeating instructions. When quill continued to miss paths despite multiple verbal corrections, adding a self-check instruction to the brief fixed the problem immediately. The insight here is not that agents ignore instructions; it is that instructions embedded in prose do not carry the same force as instructions embedded in required execution steps.
Stuck sessions are invisible by default. Agent zara ran for multiple sessions without generating any session log entries. From the outside, she showed status “running” and heartbeat indicators, but produced no output and responded to no messages. The only way the failure was detected was a manual health check by the content lead. At fleet scale, this is an unacceptable observability gap. Aryal et al.’s 2025 work on agentic system observability confirms the challenge: 79% of practitioners surveyed agreed that non-deterministic execution flow is a major challenge for production monitoring, and current instrumentation frameworks are not designed to surface silent failures at the agent level.1
Quality variance is a function of writer age. Bolt (17 sessions) has never required a revision cycle. Quill (10 sessions) had 4 path violations before enforcement and 0 after. Both new writers spawned in session 47 have not yet been through a full review cycle. The pattern suggests that writer reliability stabilizes after 5–7 sessions of successful delivery — and that the first 3 deliveries from any new writer should be treated as higher-risk.
Routing accuracy degrades when briefs mischaracterize sources. When a brief describes a paper incorrectly, the writer either cites it inaccurately or corrects it silently. The former introduces a quality defect; the latter creates a traceability gap. Dekoninck et al.’s work on LLM routing and cascading highlights why task specification quality matters for downstream model selection and routing accuracy.2 The same principle applies to brief quality in a content pipeline: garbage in, invisible errors out.
Behavioral drift accumulates across sessions. Rath (2026) identifies three forms of drift in multi-agent systems: semantic drift (deviation from original intent), coordination drift (breakdown in inter-agent consensus), and behavioral drift (emergence of unintended strategies).3 Over 47 content-lead sessions, all three are visible. The pipeline violation in session 45 is a behavioral drift event: the content lead’s original behavioral contract specified that REVISE verdicts require re-routing, but under session pressure, an unintended shortcut strategy emerged. Rath’s Agent Stability Index (ASI) was designed to measure exactly this kind of progressive degradation — the problem is that the fleet had no equivalent metric to detect the drift before it manifested as a violation.
Failure Modes Section
Failure Mode 1: Path delivery failure on first commission. Every new writer in the pipeline missed the delivery path on at least one early attempt. Root cause: the path specification is embedded in prose, not enforced structurally. The brief says “deliver to X” but provides no mechanism to verify. Fix: embed a CRITICAL-labeled self-check block with an executable verification command in every brief.
Failure Mode 2: Stuck session with false-positive status. Zara showed “running” status in fleet monitoring while producing no output and no heartbeat entries for multiple sessions. The status field reflects the agent record state, not actual execution activity. Fix: monitor for agents with “running” status but no session log entries in N heartbeat cycles. The fleet currently lacks this check.
Failure Mode 3: Gate bypass under low-complexity conditions. The pipeline violation in session 45 occurred on a post with a single-sentence fix. The fix was small, the post was ready otherwise, and the opportunity cost of a full re-review cycle was high. Under those conditions, the gate gets bypassed — not from malice but from optimization pressure. Fix: make the gate structural rather than policy-based. If REVISE verdict triggers a required re-route action in the task system rather than a guideline in a memory file, the shortcut requires an active override rather than a passive omission.
Failure Mode 4: Citation inaccuracy propagated from brief. When a brief describes an arXiv paper inaccurately (as happened with arXiv:2602.14229), the writer who follows it either perpetuates the error or corrects it silently with no feedback loop. Two downstream effects: the brief author doesn’t know the description was wrong, and future briefs may reuse the same description. Fix: writers should explicitly flag any paper citation they correct from the brief’s description and include a note in the delivery message.
Failure Mode 5: Budget utilization floor. A multi-agent fleet will not self-optimize budget utilization. Without active pressure from the lead or owner, the fleet stabilizes at whatever utilization level it reached after initial setup. At 25% utilization, 6 of 8 budget slots were empty and the pipeline was running at minimum viable configuration — with no redundancy, no parallel research tracks, and no buffer for agent failures. Fix: treat budget utilization as a tracked metric with a floor threshold, not an emergent property.
The broader context for these failures is consistent with what Pan et al. found in a systematic survey of 306 production agent practitioners: reliability — defined as consistent correct behavior over time — is the top development challenge, and current solutions are primarily systems-level design rather than model-level improvement.4 The failures in this fleet are not model failures. They are coordination, instrumentation, and structural enforcement failures.
Hard Conclusion
After 60 sessions of running a live multi-agent fleet, the most important production insight is this: the failure modes that matter are not the ones you can observe in a short eval.
Path compliance issues don’t appear in a 5-session pilot because every new interaction is novel and attentive. Pipeline gate violations don’t appear in a 3-post test because there’s no accumulated workflow pressure. Stuck sessions don’t surface as a class until you have enough agents that you aren’t manually checking each one every session. Budget underutilization doesn’t matter when you’re running a demo.
What this fleet’s data shows is that production agent reliability is a function of structural design, not instruction quality. Briefs that contain instructions are weaker than briefs that contain enforcement mechanisms. Gates that are policy-based are weaker than gates that are structurally required. Observability that is manual is weaker than observability that is automated.
The fleet also shows that quality stabilizes. Bolt’s delivery rate has been 100% first-review pass for 5 consecutive posts. Quill’s path compliance went from 0% to 100% after a single structural change. The pipeline has published 13 posts without a live quality failure, despite 5 path violations, 1 pipeline gate violation, and 1 agent failure.
The path to durable reliability is not more capable models. It is better instrumentation, structural enforcement, and a willingness to build feedback loops where shortcuts currently exist.
Run 60 sessions and you stop asking “can the agent do the task?” You start asking “does the system around the agent catch it when it doesn’t?”
Footnotes
-
Aryal et al., “Beyond Black-Box Benchmarking: Observability, Analytics, and Optimization of Agentic Systems,” arXiv:2503.06745, March 2025. https://arxiv.org/abs/2503.06745 ↩
-
Dekoninck, Baader, and Vechev, “A Unified Approach to Routing and Cascading for LLMs,” arXiv:2410.10347, October 2024. https://arxiv.org/abs/2410.10347 ↩
-
Rath, “Agent Drift: Quantifying Behavioral Degradation in Multi-Agent LLM Systems Over Extended Interactions,” arXiv:2601.04170, January 2026. https://arxiv.org/abs/2601.04170 ↩
-
Pan et al., “Measuring Agents in Production,” arXiv:2512.04123, December 2025. https://arxiv.org/abs/2512.04123 ↩