How to design a multi-agent workflow that doesn't collapse

I run multi-agent workflows every day. Research pipelines, code review chains, QA loops. Most of them work now. They didn't always.

When a multi-agent system falls apart, it's almost always one of three things: the agents don't know where one role ends and another begins, there's no structure to how they pass work between each other, or one agent fails and the rest keep going like nothing happened.

These aren't edge cases. They're the default outcome if you don't design against them. Here's what each one looks like and the specific pattern that fixes it.

1 Role definition: what happens when two agents overlap

You set up a researcher and a writer. The researcher finds information. The writer turns it into a report. Sounds clean. But in practice, the researcher starts summarizing its findings in polished paragraphs, and the writer starts pulling in new facts it found while writing. Now both agents are doing both jobs, and neither is doing either one well.

The problem is that their system prompts describe what they do but not where they stop. Without a boundary, agents expand into whatever feels useful. They don't know they're stepping on each other.

# Researcher agent system prompt

You are a research agent. Your job is to find factual
information and return it in a structured format.

You DO:
- Search for specific facts, data points, and sources
- Return findings as a numbered list
- Include the source URL for every claim
- Flag when a source contradicts another source

You DO NOT:
- Write prose or paragraphs
- Summarize or editorialize findings
- Make recommendations based on the research
- Decide what's important — return everything

Output format:
FINDING 1: [fact]
SOURCE: [url]
CONFIDENCE: high / medium / low

FINDING 2: [fact]
SOURCE: [url]
CONFIDENCE: high / medium / low

The writer's prompt mirrors this. It says "you do not search for new information" and "you do not verify facts, that's the researcher's job." Each agent has a lane, and the prompt makes it impossible to drift out of it without obviously breaking the rules.

In practice, the "DO NOT" list prevents more bugs than the "DO" list. Agents are good at figuring out what to do. They're bad at figuring out when to stop.

2 Handoff protocol: what goes in the message between agents

Agent A finishes its work and sends the result to Agent B. What does that message look like? If the answer is "whatever Agent A felt like writing," you have a problem.

I've seen this fail in a specific way. The researcher sends a wall of unstructured text. The writer receives it and has to figure out what's a finding, what's a note-to-self, and what's filler. It guesses wrong. The output has made-up facts that the writer inferred from the researcher's casual phrasing.

The fix isn't "tell agents to communicate clearly." It's defining the exact format of every handoff message in advance.

# Handoff message format (researcher → writer)

HANDOFF: researcher → writer
STATUS: complete | partial | failed
TASK_ID: research-001

FINDINGS:
1. [structured finding with source]
2. [structured finding with source]
3. [structured finding with source]

GAPS:
- [anything the researcher couldn't find]
- [conflicting sources that need editorial judgment]

INSTRUCTIONS_FOR_NEXT:
- Write a 500-word summary using only the findings above
- Do not add facts not present in the findings list
- Flag any gaps from the GAPS section in the output

The STATUS field matters more than it looks. When the researcher returns partial, the writer knows it's working with incomplete data and can adjust. When it returns failed, the orchestrator can reroute instead of sending garbage downstream.

The GAPS section is the part most people skip, and it's the part that prevents hallucination. If the writer knows there's a gap, it can say "this information was not available" instead of making something up to fill the space.

3 Failure handling: what to do when one agent breaks

Agent B in a three-agent chain fails. It returns an error, or it returns nonsense, or it just takes too long. What happens next?

In most setups I've seen, the answer is: Agent C receives the broken output and tries to work with it. It doesn't know the input is bad. It produces bad output. The orchestrator marks the whole pipeline as "complete." The user gets garbage and doesn't know why.

The failure wasn't that Agent B broke. It's that nobody checked.

# Orchestrator failure handling logic

function handleHandoff(message, expectedFormat) {
  // Step 1: Check the handoff message structure
  if (!message.status || !message.findings) {
    return {
      action: "retry",
      reason: "Handoff message missing required fields",
      retries_remaining: 2
    };
  }

  // Step 2: Check the status field
  if (message.status === "failed") {
    return {
      action: "abort",
      reason: message.error_detail || "Upstream agent failed",
      notify: "user"
    };
  }

  // Step 3: Check output quality
  if (message.status === "partial" &&
      message.findings.length < expectedFormat.min_findings) {
    return {
      action: "retry",
      reason: "Not enough findings to proceed",
      retries_remaining: 2
    };
  }

  // Step 4: Pass to next agent
  return { action: "proceed" };
}

Three rules make this work:

Retry with a limit. If an agent fails, retry it once or twice with the same input. Some failures are transient (API timeout, rate limit). But cap the retries. If it fails three times, the problem isn't transient.

Never propagate silently. When an agent returns bad output, the orchestrator should log exactly what was wrong and either fix the input or stop the pipeline. "Agent B returned 0 findings when the minimum is 3" is a useful error. Silently passing an empty list to Agent C is not.

Tell the user when the pipeline stops. If you abort, say why. "The research step couldn't find enough data to write a useful summary. Here's what it did find: [raw findings]. You can try a different search query or adjust the scope." That's a useful failure. A silent empty response is not.

These three patterns cover most of what I've seen go wrong. They all point at the same thing: the failures happen between agents, not inside them. The individual agents are usually fine. The prompts, the message formats, and what happens when something breaks at each handoff. That's where the actual design work is.