Station 2 · AI Agents 101
Every agent is built on top of a large language model. You don't need to be an LLM expert to build agents — but you do need just enough to know why an agent behaves the way it does. This is that recap.
What you'll walk away with
An agent is an LLM + tools + a loop. If the LLM is the engine, this page is the quick "how the engine works" briefing before you start driving. This is a recap for builders — the full deep dive lives in Level 1 · Foundations.
Concept 1
An LLM doesn't read letters or whole words. It chops text into tokens — little chunks, roughly ¾ of a word each — and works with those. "Agents are amazing" isn't 3 words to the model; it's a handful of tokens.
# The sentence you type... "Agents keep working until the job is done." # ...is split into tokens (≈ chunks the model reads): ["Agents"] [" keep"] [" work"] ["ing"] [" until"] [" the"] [" job"] [" is"] [" done"] ["."] Count → 10 tokens in, more tokens out Bill → you pay per token — input + output both count
The context window is how much text the model can "see" at once — everything in the current conversation: your instructions, the history, tool outputs, and its own reply-in-progress. In 2026 that ranges from tens of thousands of tokens to well over a million, depending on the model.
The context window is the model's desk, not its memory. Only what fits on the desk right now can be used. Pile on too much — a giant search result, a whole log file — and older stuff slides off the edge and is forgotten. The model has no "filing cabinet" it quietly remembers from; if it's not on the desk, it's gone.
Why this matters for agents especially: agents call tools, and tools return text — sometimes a lot of it. A single web page or database dump can swallow a big slice of the window. Manage what you feed in, or the agent runs out of room (and burns tokens) before it finishes the job. That pressure is exactly why memory (Station 6) exists.
Concept 2
When a model writes, it's really predicting the next token over and over — and it doesn't always pick the single most likely one. A few dials control how it chooses, and they change how an agent behaves.
Low (near 0) = focused and predictable. High (near 1+) = more varied and surprising. For reliable tool use, keep it low.
"Nucleus sampling" — only consider the most likely tokens that add up to p of the probability. Another way to widen or narrow choices.
A hard limit on how many tokens the model may generate. Stops runaway output and caps your cost per call.
"Stop sequences" — tell it to halt the moment it writes a certain string, so replies end cleanly where you want.
Two more you'll see occasionally: frequency penalty discourages repeating the same tokens, and presence penalty nudges the model toward new topics. Handy for creative writing, rarely critical for agents.
For an agent that has to call tools and produce clean, parseable output, you usually want low temperature. You're optimizing for reliability and repeatability, not surprise. Crank the creativity up only when the task is brainstorming or writing.
Concept 3
Not all models are reached the same way. The big split is whether you can download the model itself or only talk to it over the internet.
The trained weights are published — you can run them on your own hardware. Families like Llama and Mistral. More control and privacy; you supply the compute.
You send requests to the provider's servers and get answers back. GPT (OpenAI), Claude (Anthropic), Gemini (Google). Easiest to start; you're renting, not owning.
Two labels worth knowing when you pick one:
Open-weight doesn't mean anything-goes. Each family ships a licence with real terms — some restrict commercial use or large-scale deployment. Read it before you build a product on top. And "open-weight" usually means the weights are shared, not the training data or full recipe.
Concept 4
Here's the trick that lets an agent "look things up by meaning" instead of exact keywords — and it's the foundation of agent memory.
An embedding turns a piece of text into a long list of numbers that captures its meaning. Two texts that mean similar things end up with similar numbers — even if they share no words. "How much does it cost?" and "What's the price?" land right next to each other.
# Each sentence becomes a vector (shortened here): "What's the price?" → [0.82, 0.11, 0.74, ...] "How much does it cost?" → [0.80, 0.09, 0.77, ...] # very close! "What time is sunset?" → [0.05, 0.91, 0.22, ...] # far away Search → compare vectors, return the closest meanings
RAG — Retrieval-Augmented Generation — is the pattern that puts this to work. Before the model answers, you retrieve the most relevant chunks (via vector search) and paste them into the context window, so the model answers from real, specific documents instead of only its frozen training.
A plain LLM takes a closed-book exam — it answers from memory and can misremember. RAG hands it the open book, flipped to the right page, right before it answers. That's how an agent "remembers" your notes, docs, or past chats (you'll build this in Station 6).
One last builder's choice: when a model doesn't behave how you want, you rarely need to fine-tune it (retrain it on your data — expensive and slow). Most of the time, better prompt engineering — clearer instructions and good examples in the context — gets you there. Start with the prompt; reach for fine-tuning only when prompting truly can't.
Checkpoint
Your agent calls a tool that returns a giant 40-page document, and suddenly it "forgets" your earlier instructions. What most likely happened?
The context window is finite. A huge tool result eats up the available tokens, and once the window is full, the oldest content (your early instructions) falls out of view. The fix is to trim or summarize what you feed in — or use retrieval/memory to pull back only the relevant bits. Temperature has nothing to do with it.
See it yourself — 5 minutes
Two quick experiments that make these ideas concrete:
Recap. Agents ride on an LLM. Text becomes tokens you pay for, and they only fit inside a limited context window. Generation controls like temperature tune how the agent acts — keep it low for reliable tool use. Models are open-weight (download) or closed-weight (API). And embeddings + vector search + RAG let an agent look things up by meaning — the seed of memory. Next up: the loop that ties the brain and the tools together.