From scores to a chosen token
The bars show the probability of each candidate after temperature, top-k, and top-p. Then one token is drawn at random.
The capital of France is?
| Token | Probability | p | Draws |
|---|---|---|---|
| ␣Paris | 77.9% | ||
| ␣a | 7.1% | ||
| ␣the | 5.2% | ||
| ␣located | 2.6% | ||
| ␣known | 1.7% | ||
| ␣home | 1.4% | ||
| ␣one | 1.2% | ||
| ␣not | 0.9% | ||
| ␣also | 0.7% | ||
| ␣called | 0.6% | ||
| ␣beautiful | 0.5% | ||
| ␣Lyon | 0.3% |
after your settings at temperature 1, before any cuts
Tokens still in play12 of 12
Uncertainty (entropy)1.39 bits
Most likely␣Paris 77.9%
Try this
- Compare the Fact and Story prompts at temperature 1. Which has the higher entropy, and why does that match how many good continuations each prompt has?
- Set temperature to 0.3 and draw 100 times. Then try 1.8. Watch how the counts follow the bars.
- With top-p at 0.8, switch between prompts and count how many tokens survive each time.