Module 4 · Agents that ship · Chapter 28 · Free
Agent design patterns: tools, planning, reflection, routing and parallel work
- Most good agents are built from a few small patterns. Each one fixes one kind of problem.
- Every pattern adds wait time, cost, or both. Use the simplest one that works.
- If you can write the steps down in advance, a fixed workflow beats an open agent.
The one picture to remember: a kitchen. A fixed recipe for the usual dish, a head chef who sends each order to the right station, and three cooks working side by side. You do not hire a genius to boil an egg.
- He gives one agent twelve tools and says: "Handle every customer email."
- The demo is exciting. In real use, refund replies take 14 seconds.
- Sometimes it picks the wrong tool. Sometimes it loops. The bill is hard to predict.
An agent is a loop. You send the model a goal and a list of tools. The model either answers or asks to use a tool. Your code runs the tool, adds the result to the conversation, and calls the model again. This repeats until the model answers or a stop rule fires, such as a step cap. The chapters AI agents and agentic workflows and The agent loop from scratch teach this in full. You only need the idea above to read on.
- Step 1: tool use, chaining and routing. Watch one email travel through them.
- Step 2: parallel work, and a lead with helpers. Work out the wait time with sums.
- Step 3: planning, and draft-critique-revise. See what they cost.
- Step 4: a matching game. Match real situations to patterns.
- Step 5: a decision tree. Walk it to a pattern.
- Step 6: the rule that matters most: start simple.
After this chapter you can:
- Name seven agent patterns and say, in plain words, what each one is for.
- Work out how long three steps take one after another, and how long they take in parallel.
- Choose the simplest pattern for a new job, and say what the fancier pattern would cost.
Next: a few words you will see all the way through. Each has a plain meaning and a small example.
Words you'll meet
- Pattern
- A shape for how steps fit together that people reuse. Example: "route the question, then answer".
- Tool use
- The model asks your code to do something it cannot do alone. Example: look up an order number.
- Workflow
- A fixed path that your code controls. The model fills in each step but does not choose the path.
- Routing
- Pick a path by the type of question. Example: refunds go one way, product questions another.
- Step cap
- A hard limit on how many loops or calls are allowed. Example: at most 2 revise rounds.
You may also hear "agentic workflow". It usually means a workflow that uses some of these patterns. Other terms are defined where they first appear. Kit names such as Google's Agent Development Kit (ADK), CrewAI or LangGraph appear in the next chapters. This chapter uses plain words and pseudo-code only.
Next: the three simplest patterns. They cover more real jobs than you might expect, and they cost the least.
Step 1: The napkin plans: tools, chains and routers
Think of a shop counter. Some jobs follow a fixed routine. Some need a lookup. Some go to a different desk.
Pattern 1: tool use. The model cannot see your order system. So it asks your code to look something up.
- Picture: a clerk who phones the stockroom.
- Use when: the answer lives in a system the model cannot reach.
- Hurts when: you hand over too many tools. The model must pick from all of them, and it can pick wrong. Keep each tool small and clearly described.
Pattern 2: prompt chaining. (A prompt is the instruction you send the model.) Fixed steps, in order. Each step gets the last step's output.
- Picture: an assembly line.
- Use when: you know the steps and the order every time.
- Hurts when: a real case needs a different path. A chain cannot bend. It also adds one wait per step.
Pattern 3: routing. One quick step decides the type of request. Then code sends it down the right path.
- Picture: the front desk that says "refunds, second door on the left".
- Use when: different kinds of requests need different tools, rules or prompts.
- Hurts when: the router guesses wrong. A refund sent to the product desk gets a wrong answer. Add a catch-all path, such as "hand to a person".
A router is only a few lines. This is pseudo-code (plain-words code, not a real library):
kind = ask_model("Is this email a refund, a product question, or other (unsure or upset)?")
if kind == "refund": reply = run_refund_chain(email)
elif kind == "product": reply = run_manual_search(email)
else: reply = hand_to_person(email)
What you are looking at: one customer email entering a router. The three middle boxes are three different paths. The amber box is the safe exit. Press next to follow the email.
On a phone, swipe the picture sideways to see all of it. Tap Next and swipe to follow each step.
- "I want to return my tent. The zip broke." Nothing has decided yet what kind of job this is.
- One short model call reads the email and names its type. Why: a cheap early decision saves a long, costly one later. Your code, not the model, then follows the path.
- Three fixed steps in order: look up the order (a tool), check the return rule, write the reply. Each step feeds the next. Why: the steps are known, so no agent is needed.
- A question like "is this tent waterproof?" needs one search of the product manual. A single tool call is enough.
- If the router is unsure, or the email is anything else (for example, an upset customer), a person takes over. Why: every router needs a path for what it cannot sort.
- Each path ends in a reply or a hand-off. Notice that no part of this picture needed an open-ended agent.
Where the picture breaks down:
- A front desk worker knows the customer's mood. A router only reads text, and it can guess wrong. So test it on real emails.
- The arrows look free. The router is a model call, and each chain step is another one. Each adds wait.
- A desk clerk can improvise. A fixed chain cannot. If a case does not fit, it fails or goes to the safe exit.
Email: "Hi! The tent arrived fast and the setup video was lovely,
thank you. One small thing: I was charged twice. 86 dollars came
out of my account two times. Could someone look into it when
they have a moment? No rush at all."
Router filed it as: product question
(polite, thanks the team, mentions the product,
no angry words, no word "refund")
Reply sent: "Thanks for the kind words! Glad the setup video
helped. Your tent is fully waterproof, so you are ready for
wet weekends."
The customer asked about a double charge. The reply answered a question nobody asked, and never mentioned the money. This email fit no bucket well, which is exactly the case the catch-all path exists for. If "unsure" is not a bucket of its own, the router still picks one, confidently.
Next: chains are safe but slow, because they wait in line. When steps do not depend on each other, we can run them side by side.
Step 2: Do more at once: parallel work, and a lead with helpers
Say the refund path needs three facts. Look up the order, read the return policy, and check stock for a swap. None needs the others' answers.
Doing them one after another wastes time. A person would ask three colleagues at the same time.
Pattern 4: parallel work. Start the independent steps together. Wait for all of them. Then combine. The join is the step that combines the results of parallel jobs. It is one more model call that writes the combined answer, so it takes time and money.
- Picture: three cooks, then one plate.
- Use when: steps do not need each other's results, and a person is waiting.
- Hurts when: a step needs another's answer. Then it cannot start early. Also, you still pay for every call. Parallel work saves time, not money.
Pattern 5: orchestrator and workers. A lead model decides at run time how to split a big task. Helpers do the parts. The lead joins the results.
Hold this pattern against Pattern 4 before going further. In parallel work, you wrote the split, in advance, in code. Here the model chooses the split, at run time. Who decides the split is the whole difference between the two patterns.
- Picture: a project lead who hands out three contracts to read.
- Use when: you cannot know the parts in advance. The task decides the split.
- Hurts when: the parts depend on each other, or the lead splits badly. It also adds the lead's calls on top of the helpers' calls.
The difference from plain parallel work: in parallel work, you wrote the split. In orchestrator and workers, the model chooses it.
What you are looking at: two timelines for the same three jobs. Each bar's length is its time. The numbers are made up. One second is the same width on every bar. Press next to compare.
On a phone, swipe the picture sideways to see all of it.
- Look up the order takes 4 seconds. Read the policy takes 5. Check stock takes 3. None needs another's answer. (All made-up numbers.)
- Each job waits for the one before it. The waits add up: 4 + 5 + 3 = 12 seconds. A 2-second join step makes 14 seconds. The customer waits the whole time.
- All three start at once. Job A is done at 4 seconds and job C at 3. They sit idle while B finishes.
- The join cannot start until every job is back. So the wait is the longest job, not the sum. Here that is 5 seconds.
- The join is one more model call that writes the combined answer. It takes 2 seconds, so the total is 5 + 2 = 7 seconds. Why: the join needs all three answers, so it cannot go earlier.
- 14 - 7 = 7 seconds saved, half the wait. But you still made the same calls, so the cost did not fall. Parallel work buys speed, not savings.
Where the picture breaks down:
- Real jobs do not all take their exact time. The slowest one in a run sets the wait, and it changes every run.
- The bars look free to start. Some services limit how many calls you can make at once. Check your limits.
- One failed job can block the join. Decide in advance: wait, retry once, or answer with a gap.
Worked example, slowly. Kavish times the three jobs on a normal day. The numbers are made up.
- Order lookup A: 4 seconds. Policy read B: 5 seconds. Stock check C: 3 seconds. Join: 2 seconds.
- One after another: 4 + 5 + 3 = 12 seconds. Add the join: 12 + 2 = 14 seconds.
- In parallel: the longest of 4, 5 and 3 is 5 seconds. Add the join: 5 + 2 = 7 seconds.
- Saved: 14 − 7 = 7 seconds. Share saved: 7 ÷ 14 = half.
- Cost: three calls plus the join, which is also a model call, so 4 calls either way. If each call costs 0.002 dollars (made up), that is 4 × 0.002 = 0.008 dollars in both cases.
So the answer is: parallel work cuts the wait from 14 to 7 seconds in this made-up case. It does not cut the cost.
Next: speed is one goal. Quality is the other. Two patterns trade extra calls for better, more reliable answers.
Step 3: Get better answers: planning and reflection
Some jobs are long, or the first draft is rarely good enough. Two patterns help here.
Pattern 6: planning. The model writes a numbered plan first. Then it follows the plan, and updates it as it learns.
- Picture: a checklist on a clipboard.
- Use when: a task has many steps and the order matters, such as planning a customer migration.
- Hurts when: the task is short. The plan costs a call and adds wait. A bad plan sends every later step the wrong way. So let the model fix the plan when a step fails.
plan = ask_model("Write a numbered plan for: " + goal)
for step in plan:
result = do(step) # may use tools
plan = ask_model("Update the plan given: " + result)
That is pseudo-code (plain-words code, not a real library). Notice that the plan is rewritten after each step. It is a living list, not a contract.
Pattern 7: reflection. One call drafts. A second call critiques the draft against a short list. A third revises. Repeat a few times at most.
- Picture: an editor who marks up your page.
- Use when: you can say what "good" looks like, such as a checklist, and a first draft often misses it.
- Hurts when: the critic is weak or agrees with everything. Then you pay for rounds that change nothing. Always cap the rounds.
What you are looking at: a reflection loop. The dashed loop is the second trip round. A passing draft exits straight to Done. Press next to see why the cap matters.
On a phone, swipe the picture sideways to see all of it.
- The model writes a reply to the customer. First drafts often miss a rule, such as the refund window.
- A second call checks the draft. The list: names the order, states the refund window, says what happens next. Why: a short, clear list gives a sharper critique than "is this good?".
- The critic says the window is missing. The model adds it. Why: the critic's notes tell it exactly what to fix.
- The new draft is checked again. If it still fails, it returns for another revise. Each trip is two more model calls.
- The cap here is 2 revise rounds, a made-up value. Without it, a harsh critic and an eager writer can go on for ever.
- The draft passes (even on the first check), or the cap is hit. Send it, or send it to a person with the critic's notes.
Where the picture breaks down:
- A human editor has fresh eyes. A model critic is often the same model with the same blind spots, so it may praise its own mistakes.
- A person gets tired and stops. A loop does not, so you must set the cap.
- Each arrow is a model call. The picture hides the wait.
Worked example, slowly: what does reflection cost? The numbers are made up.
- A draft takes 3 seconds. A critique takes 2 seconds. A revise takes 3 seconds.
- With no reflection: 3 seconds.
- One revise round: draft 3, critique 2, revise 3, then a final critique 2. Total: 3 + 2 + 3 + 2 = 10 seconds, and 4 calls.
- Two revise rounds: 3 + (2 + 3) + (2 + 3) + 2 = 15 seconds, and 6 calls. Term by term: the first 3 is the draft. Each (2 + 3) is one round: a critique (2), then a revise (3). The last 2 is a final critique, checking the second revise. So the 6 calls are: draft, critique, revise, critique, revise, final critique.
So the answer is: two rounds took this reply from 3 seconds to 15 seconds, and from 1 call to 6. That is 5 times the wait and 6 times the calls. Use it where an error is costly. Skip it for a casual reply.
One anchor in the real world: the made-up 0.002 dollars per call used earlier in this chapter is almost exactly a real call. On Claude Haiku 4.5 (prices as of November 2026: 1.00 dollar per million input tokens, 5.00 dollars per million output tokens), a typical call of about 1,000 input tokens and 300 output tokens costs about 0.0025 dollars.
So this 6-call reflection setup costs about 1.5 cents per reply, which is about 15 dollars per 1,000 replies. Small for a letter that must be right. A real budget line if every casual reply pays it.
Next: you now have seven patterns. The skill that matters is picking the right one. Test yourself on real situations.
Step 4: Try it: match each situation to a pattern
Why this exercise: to build the habit of reading a situation and naming the lightest pattern that fits.
The rules of this game:
- Each card is a made-up situation. Click a card, then click the pattern bucket you think fits best.
- Pick the simplest pattern that does the job, not the most impressive one.
- A wrong answer shows why, and a hint is there if you are stuck.
Do this:
- Read all seven cards first. Underline words like same order, unknown size, checklist, waiting.
- Place the easy ones. Then go slowly on the cards about speed and about quality.
- When wrong, read the "why". Notice which word in the card you missed.
- Chain, routing and parallel work are workflows where code owns the path. Tool use needs no open agent either.
- Parallel work is about wait time. Reflection is about quality. They solve different problems.
- Words in the card gave it away: "same order", "unknown size", "checklist", "customer is waiting".
Next: the matching game gave you the answer in a bucket. Real projects are less tidy. A decision tree helps you reach the same answer by asking questions.
Step 5: Try it: which pattern do I need?
Why this exercise: to practise the most useful habit in agent work: ask the cheap questions first.
How the tree works:
- It asks yes-or-no style questions about a project you have in mind.
- It ends with a recommended pattern, why it fits, and what a fancier pattern would cost you.
- It is a teaching guide, not a law. Your own tests have the last word.
Do this:
- Think of one real project, or use the Northfield email helper.
- Walk the tree once with honest answers.
- Press the Back button under the question. At the first question pick "No, it needs more than one step". At the second pick "No, the steps depend on what comes back". See how fast the advice gets more careful.
- The very first question asks whether you need any pattern at all. Often you do not.
- Only the "steps depend on what comes back" branch leads to open-ended patterns. Reflection can also be added to a known path, as in the Greenleaf case below.
- Tool use is not in the tree because it fits inside every path. A one-call answer can still use a tool.
Next: two worked cases, one at a customer. Then the rule that ties the chapter together.
Step 6: The rule that beats every pattern: start simple
Back to Kavish. Amara sat him down and drew a ladder.
- Rung 1: one model call, with the right background material.
- Rung 2: add a tool if the answer lives in a system.
- Rung 3: a fixed chain or a router, if there are several known steps or types.
- Rung 4: parallel steps, if a person waits and steps are independent.
- Rung 5: reflection or planning, if quality or length demands it.
- Rung 6: an open agent or a lead with helpers, only if the path truly cannot be known.
Climb one rung at a time. Stop as soon as your real test questions pass. Each rung costs more wait, more money and more ways to fail.
This is one widely shared view, not a law of nature. Test it on your own cases.
- Each letter has a known layout and a short list of rules: right name, right date, no medical promises.
- First drafts miss one rule about one time in four (a made-up figure).
- Kavish builds a fixed chain with a reflection step against the rule list, capped at 2 rounds. Chain for the known steps, reflection because first drafts often miss a rule.
- Kavish checks the logs. Three lookups ran one after another, and none needed another's result.
- He runs them in parallel. The wait drops to 7 seconds, as in the worked example.
- The cost per reply stays the same, so the finance team notices no change.
Patterns stack. Look again at the two scenarios. The letter checker is a chain with a reflection step inside it. The refund path sits behind a router, and runs its lookups in parallel inside one step. That is the normal case, not the exception: real systems are a router in front of fixed chains, parallel work inside a step, and reflection only on the one chain where errors are costly. Learn the patterns one at a time. Then combine the ones that earn their place.
Next: a prediction, a second quick check, and then the common mix-ups.
A critique call is paid even when the draft is fine. Use reflection where errors are costly or common. Or run it only when a cheap check says the draft looks wrong.
Mix-ups to avoid
- "Parallel work saves money." No. It saves wait time. You still make every call.
- "More agents means a smarter system." No. Each extra agent adds calls, wait and places to go wrong.
- "Reflection always improves the answer." No. A weak critic adds cost and changes nothing. Give it a list, and cap the rounds.
- "A plan is a promise." No. A plan is a living list. Update it when a step fails.
- "An agent is the modern way, so a workflow is old-fashioned." No. When the steps are known, a workflow is cheaper, faster and easier to test.
Recap
- Seven patterns: tool use, chaining, routing, parallel work, orchestrator and workers, planning, and reflection.
- Parallel work cuts wait to the longest step plus the join. Reflection buys quality with extra calls.
- Climb the ladder one rung at a time. Start with the simplest pattern that passes your tests.
- She reads the logs. A single agent loop makes eleven model calls (a made-up figure) for a common question.
- Most of those questions are one of three types with known steps.
- She adds a router and fixed chains for the three types. Only the odd cases still enter the open loop.
Next: a four-question quiz, then the next chapter.
Review this chapter in 5 minutes
Watch the whole chapter recapped: the seven patterns, what each one costs, and the ladder for choosing.
Chapter quiz
Four questions. Score 70% or more to mark this chapter complete.
Review
Next: the next chapter, "Choosing an agent framework: ADK, CrewAI, LangGraph, or plain code". You now know the shapes. A framework is a ready-made kit for building them. The next chapter shows what a kit gives you, what it costs, and when plain code is the better choice. Then comes a look at teams of agents, and when not to use more than one.