The temperature parameter is a sampling parameter that controls how random a large language model's output is. It hides in the advanced settings and API parameters of most AI chat products. You rarely see it directly, but it determines whether a model's answers are tight and convergent, or free-wheeling and imaginative.
The key idea fits in one sentence: every time a model generates a word, it first computes a probability distribution over all candidate words, then "draws" one word from it. The temperature parameter reshapes that distribution — raise the temperature and the distribution flattens, giving long-shot words a chance to be picked; lower it and the distribution steepens, so the model almost always picks the most probable word; at temperature 0, the model always picks the top word and output becomes nearly deterministic.
Temperature changes "how words are picked," not "how much the model knows"
Temperature operates at the decoding stage. It never changes a model's knowledge or capability, only its word-selection strategy. Crank it up and the model doesn't get smarter; turn it down and it won't suddenly know things it never learned. That is the most common misconception: temperature is a randomness knob, not an intelligence knob.
Temperature and top-p are not the same thing
Many people conflate temperature with top-p, but they do different jobs: temperature reshapes the whole probability distribution, stretching or compressing it; top-p truncates the candidate list by cumulative probability, keeping only the top words whose probabilities sum to p — in other words, it draws the candidate pool. The two can be combined: use top-p to define the pool, then temperature to decide how randomly to pick within it. Remember "one shapes, one scopes" and you'll never mix them up again.
Three common misconceptions
The first is "higher temperature means more creativity." Raising temperature moderately (say, from 0.7 to 1.2) does bring more varied expression — but push it too high and the model starts rambling and hallucinating; "creativity" turns into "nonsense." Real creativity is varied yet coherent, and temperature only controls the varied half.
The second is "temperature 0 guarantees perfect reproducibility." At 0, sampling collapses into greedy decoding and output is highly stable for the same input, yet hardware differences, inference frameworks, and batch scheduling can still introduce tiny variations. For tasks demanding strong consistency, temperature 0 is necessary but not sufficient.
The third is "the default temperature is fine for everyday use." It depends on the task: for coding, math, and factual questions, lowering temperature (0 to 0.3) noticeably cuts down confident-sounding fabrications; for brainstorming, copywriting, and storytelling, raising it to 0.8–1.2 yields more surprises. The people who can actually turn this knob are mostly API users and those running models locally — web chat products usually fix it for you.
Suggested settings by task
A practical reference range (scales vary slightly by model; the common range is 0 to 2): use 0 to 0.3 for code, math, and factual Q&A to prioritize accuracy and reproducibility; around 0.7 for everyday conversation, summarization, and rewriting to balance naturalness and stability — also the default in most products; 0.9 to 1.3 for brainstorming and creative writing to maximize variety, with human review; above 1.5, only for experiments, since output quality degrades noticeably.
Temperature is fundamentally a trade-off between "steady" and "lively." Next time an AI's answer feels too rigid or too wild, check where the temperature knob is set — often the problem isn't the model, but this small parameter.
