Welcome to the NZRT Wiki Podcast. Today we’re looking at Tool Use & Agents.

Let’s start with the concept of tool use, sometimes called function calling. At its core, this is the ability for a large language model to reach out and interact with the world beyond its own knowledge. When you send a prompt to an AI, rather than just generating a text answer, the model can recognise that it needs to call an external tool to get the job done. It does this by producing a structured piece of data, essentially a description of what tool it wants to use and what inputs it needs. Your application picks that up, runs the actual tool, whether that’s an API call, a database query, or a file read, and then passes the result back to the model, which uses it to form its final response.

To make that concrete, imagine Claude deciding it needs to check the weather. It would output something that says: call the get weather function, with Auckland, New Zealand as the location. Your app runs that, gets the forecast, hands it back, and Claude says something sensible like “It looks like it’ll be cloudy in Auckland today.” The model never directly calls anything itself. It generates the instruction, and your host application does the actual work.

Next up is Retrieval-Augmented Generation, or RAG. This is a technique for grounding a model’s responses in real, up-to-date information rather than relying purely on what it learned during training. The idea is that when you ask a question, your system first converts that question into a mathematical representation called an embedding, then searches a database of similarly encoded document chunks to find the most relevant pieces of content. Those chunks get inserted into the prompt as context, and the model generates its answer based on that retrieved material rather than guessing.

There are a few moving parts to know about here. First, chunking: your source documents get split into overlapping segments, typically somewhere between two hundred and five hundred tokens each, so they’re a manageable size for retrieval. Second, an embedding model converts both your query and your document chunks into vectors, which are essentially lists of numbers that capture meaning. Models like text-embedding-3-small are commonly used for this. Third, a vector store holds all those embeddings and lets you search them by similarity. Popular options include Pinecone, Chroma, pgvector, Qdrant, and Meilisearch. And finally, a reranker can take the top results from that search and re-score them more carefully before they go into the prompt, improving quality even further.

Now let’s talk about agents. An agent is a large language model running in what’s called an agentic loop: it perceives a goal, reasons about what to do, takes an action using a tool, observes the result, and then repeats that cycle until the task is complete or it hands off to something else. This is what separates an agent from a simple chatbot. It’s not just answering your question once, it’s working through a problem across multiple steps.

There are several common patterns for how agents are designed. The ReAct pattern, short for Reason and Act, has the model interleave its thinking with its tool calls, reasoning out loud before each action. Plan-and-Execute agents generate a full plan upfront and then carry out each step in sequence. Reflection agents go a step further and critique their own output, retrying if something doesn’t look right. Multi-agent setups have an orchestrator model delegating subtasks to specialist subagents, each with their own focus area. And Human-in-the-Loop agents pause at key decision points and wait for a human to approve before continuing. Each pattern suits different use cases depending on how much autonomy and reliability you need.

One important standard in this space is MCP, the Model Context Protocol, released by Anthropic in 2024. MCP provides a unified way for language models to connect to tools and data sources. Think of it as a common language that lets Claude talk to a wide range of external systems without needing custom integration code for each one. In an MCP setup, you have the model itself, then an MCP host like Claude Code or Claude Desktop, and then MCP servers that expose the actual capabilities. Those servers can provide access to filesystems, databases, APIs, browsers, and more. MCP supports two transport modes: stdio for running things locally, and server-sent events over HTTP for remote connections. The things MCP servers can expose fall into three categories: tools, which are functions the model can call; resources, which are data the model can read like files or database rows; and prompts, which are reusable templates. MCP servers can be written in Python or TypeScript using the official SDK.

Finally, a quick word on memory. Language models are stateless by default, meaning each new conversation starts completely fresh. But there are patterns to work around this. You can keep prior turns or summaries in the prompt itself. You can store facts in a database and retrieve them by key. You can use semantic memory, embedding past interactions and retrieving them by similarity. You can log full conversation histories and summarise them on demand. Or you can encode persistent rules and persona directly into the system prompt, which is called procedural memory. Each approach has trade-offs in cost, speed, and fidelity, and many real-world systems combine several of them.

That’s it for this episode of the NZRT Wiki Podcast. Thanks for listening.