Tokens
What the model actually reads, and why it explains odd behavior.
What you’ll learn
Four ideas, one module.
Embeddings and attention
How each token’s meaning is shaped by everything around it.
Context windows
A hard budget. Fitting inside it isn’t the same as being used well.
Token cost
Why every token costs money and time, on every single call.
Applied exercise
Predict the strongest context
Predict which of three differently structured contexts will produce the strongest answer—and explain why.
- A: The whole handbook
- B: Just the question
- C: Curated context
Cheat sheet · for your projects
Every answer is shaped by what’s in the context: choose it, measure it, pay for it.
Ask on every project
- What is in the context on a typical call, and how many tokens does each part take?
- Which parts of that context does this answer actually need?
- Where do the key rule and facts sit: at the start, buried mid-way, or by the question?
- What gets trimmed when the window fills, and would anyone notice?
- Do any sources in the context contradict each other or include outdated versions?
Rules of thumb
- Count tokens with the model’s own tokenizer; don’t estimate from word counts.
- Send only what this request needs; cut stale versions and unrelated history.
- Keep critical rules in instructions sent every call; restate them near the question.
- Estimate tokens per call × calls per day before deciding how much context to send.
Checklist
Context budget check
- List every source in the context: instructions, history, documents, tool output, question.
- Count tokens per source with the real tokenizer and note the biggest share.
- Cut what this request doesn’t need: stale versions, unrelated turns, noisy tool output.
- Keep key rules in instructions, label key facts, and put the question near the end.
- Check what gets trimmed near the window limit, and leave room for the answer.
- Multiply tokens per call by calls per day, and time a long request end to end.
Leave at the door
Five myths this module breaks.
| Myth | What’s actually true |
|---|---|
| “The model reads words.” | It reads tokens: pieces of text mapped to numbers. |
| “A bigger context window fixes it.” | Fitting in the window doesn’t mean being used well. |
| “More context means better answers.” | Irrelevant context dilutes the relevant part. |
| “It remembers earlier in the chat.” | The system re-sends the history on every call. |
| “Tokens are just a billing detail.” | They set cost, speed, limits and odd behavior. |
Next · Module 02