Designing Your Agent Architecture

Before You Build

Do not start coding yet. Design first. A good design prevents wasted work.

Step 1: Pick a Use Case

What problem will your agent solve? Be specific.

Good use cases:

• Research assistant: user asks a question, agent searches the web, reads articles, summarizes findings
• Data pipeline: agent processes CSV files, calls APIs, cleans data, writes results
• Customer support triage: agent reads customer questions, looks up account info, decides if human needed or if it can help
• Content workflow: agent takes user topic, searches for info, generates outline, writes draft

Bad use cases:

• Too vague ("general AI assistant")
• Too ambitious ("solve any problem")
• No clear success metric (how do you know if it worked?)

Exercise: Write one sentence describing your use case.

Example: "My agent helps writers research a topic and generate a blog post outline in 5 minutes."

Step 2: Map Out Which Tools You Need

What actions will the agent take? What does each action require?

For each action, ask:

1. What tool does it need?
2. When should it call this tool?
3. What does the tool return?
4. How will the agent use the result?

Research assistant example:

Code template:

tools_needed = [
  {
    "name": "web_search",
    "description": "Search the web for current information",
    "endpoint": "https://api.example.com/search",
    "parameters": ["query", "num_results"]
  },
  {
    "name": "fetch_webpage",
    "description": "Fetch and parse the full text of a web page",
    "endpoint": "https://api.example.com/fetch",
    "parameters": ["url"]
  }
]

Step 3: Choose an Orchestration Pattern

How will the agent decide which tool to call when? There are three main patterns.

Pattern 1: Agentic loop (most flexible) The model decides what tool to call at each step.

Flow: Model thinks -> Model picks tool -> Call tool -> Model reads result -> Repeat

Best for: Open-ended tasks where you cannot predict the steps.

Pattern 2: Predefined workflow (most predictable) You define the exact order of tools. The agent follows your plan.

Flow: Step 1 (search) -> Step 2 (fetch) -> Step 3 (summarize) -> Done

Best for: Tasks with a known sequence.

Pattern 3: Conditional logic (hybrid) The agent follows a workflow, but can branch based on results.

Flow: Search -> If results found, fetch -> If good sources, summarize. Else, ask for more context.

Best for: Tasks with some structure but some variability.

For your use case, which pattern fits best?

Research assistant: Agentic loop (you do not know how many sources you will need). Data pipeline: Predefined workflow (you know the steps: read -> validate -> process -> write). Customer support: Conditional (if question is in FAQ, answer it; else route to human).

Step 4: Define the State Your Agent Tracks

What information needs to persist across tool calls?

Example: Research assistant

Agent state:
- Original query (user's question)
- Sources found so far (list of URLs)
- Facts extracted (key information from each source)
- Response in progress (summary being built)
- Iteration count
- Total cost

Why track state?

• The model needs context to know what it has done
• You need to know if progress is being made
• You can detect if the agent is looping (state not changing)

Code template:

class AgentState:
  def __init__(self, query):
    self.query = query
    self.sources = []
    self.facts = {}
    self.response = ""
    self.iteration_count = 0
    self.total_cost = 0.0
  
  def add_source(self, url, content):
    self.sources.append(url)
    self.facts[url] = content
  
  def add_to_response(self, text):
    self.response += text + " "
  
  def update_cost(self, cost):
    self.total_cost += cost

Step 5: Draw the Flow Before Writing Code

Sketch out your agent as a flowchart or state diagram.

Example: Customer support agent

Start
  |
  v
Read customer question
  |
  v
Search FAQ
  |
  +--- Found exact match? ---> Return FAQ answer ---> End
  |
  +--- No match? ---> Look up customer account
         |
         v
      Search company knowledge base
         |
         +--- Found solution? ---> Send to customer ---> End
         |
         +--- No solution? ---> Route to human agent ---> End

Benefits of drawing first:

• You see edge cases (what if FAQ returns multiple matches?)
• You identify missing tools (do we have an account lookup tool?)
• You spot loops (does the agent ever get stuck?)
• You can explain the design to others

Agent Design Doc Template

Use this template to document your design:

# Agent Design: [Your Agent Name]

## Use Case
[One sentence describing what the agent does]

## Success Criteria
[How do you measure if the agent works? Examples: "Completes task in under 30 seconds", "At least 80% accuracy", "Costs less than $0.10 per run"]

## Tools
[List each tool with endpoint, parameters, and purpose]

## Orchestration Pattern
[Agentic loop / Predefined workflow / Conditional]

## State Tracked
[What information persists across tool calls]

## Flow
[Describe the steps, or include a diagram]

## Error Handling
[What happens if a tool fails? How does the agent recover?]

## Guardrails
[Max iterations, cost cap, rate limits, approvals needed]

## Test Cases
[3 happy path examples, 2 edge cases, 2 failure cases]

Checklist: Before You Code

• Use case is specific and measurable
• Each tool is mapped to an action
• Orchestration pattern is chosen
• State tracking is defined
• Flow is drawn (diagram or text)
• Design doc is complete
• Team agrees on the design

Do not skip this step. A good design saves hours of debugging later.