Token probabilities
Each step is a spread of likely next tokens, not a looked-up answer.
What you’ll learn
Four ideas, one module.
Sampling and temperature
How one token gets picked, and why the same prompt can differ.
Hallucination
Why fluent, confident text can still be false.
Reasoning and verification
More thinking improves the odds. Checks decide what ships.
Applied exercise
Where is probability acceptable?
Decide where probability is acceptable in six product scenarios and where verification must be deterministic.
- Writing for people
- Data into systems
- Money and actions
Cheat sheet · for your projects
Treat every output as a likely guess until something outside the model checks it.
Ask on every project
- What does one wrong output cost here, and who pays for it?
- Which parts are exact values: numbers, names, dates, IDs or citations?
- What catches an error before a user sees it, and does it depend on the model?
- Where do we need the same output twice, for tests, audits or caching?
- Would more reasoning help, or is the model missing a fact nobody gave it?
Rules of thumb
- Get numbers, prices and policy values from source systems; let the model only phrase them.
- Store outputs you must reproduce; never count on rerunning the prompt to get them back.
- Tune temperature for variety, not accuracy: a low setting can repeat a wrong answer.
- Spend reasoning on math, code and planning; for missing facts, add sources, not thinking.
Decision guide
Probabilistic or verified?
- If variety is the point (names, drafts, ideas)→ leave it probabilistic; test with real users
- If a person reads it next to the source→ probabilistic is OK; show or link the source
- If code consumes it→ validate the schema and the values; reject, retry or fall back
- If it states a number, price, date or citation→ fetch it from a source system; don’t generate it
- If it triggers an action or moves money→ code computes and permits it; a person approves risky ones
- If an error can’t be caught or undone→ don’t automate it; keep a person deciding
Leave at the door
Five myths this module breaks.
| Myth | What’s actually true |
|---|---|
| “Temperature 0 makes it deterministic.” | It reduces variation. Repeats aren’t guaranteed. |
| “It sounds sure, so it’s right.” | Confident tone is generated, not measured. |
| “Hallucination is a bug they’ll patch.” | It follows from generating plausible text. |
| “Reasoning models don’t hallucinate.” | They think longer and can still be wrong. |
| “Ask it to check itself and you’re safe.” | Self-checks help. Deterministic checks decide. |
Next · Module 04