← Back to Table of Contents

Chapter 37 — Agents & Tool Use

“A language model that can only generate text is a brain in a jar. An agent is a brain with hands.”

For a comprehensive deep dive into AI agents, multi-agent systems, and frameworks, see the companion guide: Agents Guide.

The Agent Loop

An agent is an LLM that can observe, reason, act, and observe again in a loop:

Function Calling

The foundation of tool use — LLMs generate structured function calls instead of (or alongside) natural text:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
# OpenAI function calling
import openai

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string", "description": "City name"},
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                },
                "required": ["location"],
            },
        },
    }
]

response = openai.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What's the weather in Tokyo?"}],
    tools=tools,
    tool_choice="auto",
)

# Model responds with:
# tool_calls: [{"function": {"name": "get_weather", "arguments": '{"location": "Tokyo"}'}}]

The model doesn’t execute the function — it generates a structured request. Your code executes it and feeds the result back:

Model Context Protocol (MCP)

MCP (Anthropic, 2024) standardizes how LLMs connect to external tools and data sources:

MCP provides a universal interface so that:

• Any LLM application can connect to any MCP server
• Tools are described once and discovered automatically
• Context (resources) can be exposed alongside tools

Coding Agents

Agents that write, debug, and modify code:

Computer Use Agents

Agents that interact with graphical interfaces:

Examples: Claude Computer Use, OpenAI Operator, browser-use agents.

ReAct Pattern

ReAct (Reasoning + Acting) interleaves thinking and tool use:

1
2
3
4
5
6
7
8
9
10
11
12
Question: What is the population of the capital of France?

Thought: I need to find the capital of France, then its population.
Action: search("capital of France")
Observation: The capital of France is Paris.

Thought: Now I need the population of Paris.
Action: search("population of Paris 2024")
Observation: The population of Paris is approximately 2.1 million (city proper).

Thought: I have the answer.
Answer: The population of Paris, the capital of France, is approximately 2.1 million.

Multi-Agent Systems

Complex tasks can be decomposed across specialized agents:

Frameworks: LangGraph, CrewAI, AutoGen, Semantic Kernel, OpenAI Swarm.

Agent Challenges

What’s Next

We’ve now covered the full stack: from tokenization to training, inference, hardware, and applications. The final chapter looks at the frontier — where the field is heading and what remains unsolved.

← Previous: Chapter 36 — Retrieval-Augmented Generation · Next: Chapter 38 — The Frontier →

Last updated: April 2026