Chain-of-Thought (CoT) is a prompting technique: instead of letting the model answer directly, you ask it to write out its intermediate reasoning steps first and give the conclusion last. When solving a math problem, the model lists "define variables, set up equations, solve" before stating a number. This "show your work first" constraint often noticeably improves accuracy on complex problems.
The core move: steps first, answer last
Large language models generate tokens one at a time, and earlier output becomes context for what follows. Chain-of-thought exploits exactly this: once intermediate steps are written down, they become the basis for the final answer. Solid steps make it harder for the answer to "skip ahead" into an error.
From worked examples to a one-line trigger
The original CoT was few-shot: include two or three "question + worked solution" examples in the prompt, and the model follows that format for new questions. In 2022, Kojima et al. showed in "Large Language Models are Zero-Shot Reasoners" that examples are not required — appending "Let's think step by step" to a question makes the model lay out its own reasoning. That is zero-shot CoT, which lifted accuracy on the GSM8K math benchmark from about 10% to about 40%. Use examples for tasks with unusual formats or heavy notation; use the one-line trigger when you have no examples handy and want to move fast.
Winning by vote: self-consistency decoding
Self-consistency is CoT's extension: have the model solve the same problem many times via chain-of-thought, each attempt possibly taking a different path, then take the most frequent answer. The chance that several independent chains make the same mistake is low, so voting filters out flukes. The cost is multiplied compute.
It is not the same thing as a "reasoning model"
Chain-of-thought is a prompting method that works with any large language model; "reasoning models" (like o1 or DeepSeek-R1) are specially trained model types for which step-by-step thinking is second nature. Ordinary models need CoT to remind them to think step by step, while reasoning models do it natively — adding CoT prompts on top of them usually gains little.
Four limitations to know before using it
First, high token cost: every step must be generated, and hard problems can cost several times more tokens. Second, reasoning hallucinations: steps can look impeccable while the conclusion is wrong — models are good at producing text that looks like reasoning, so key steps still need human checking. Third, limited gains on small models: the effect depends heavily on the model's underlying capability; tiny models still err often even with steps. Fourth, error accumulation in long chains: more steps mean more chances for a mistake, and one wrong step can derail everything after it.
Two common misconceptions
Misconception one: with CoT, the model is "really reasoning." Not quite — CoT only guides the model to generate step-by-step text; it still predicts the next token by probability, without a human-like logic-checking mechanism. Misconception two: the displayed chain reflects the model's "true thoughts" and can serve as explainability evidence. This is the dangerous one — the steps shown are text generated for the user, not the model's actual internal computation. Research has found models sometimes settle on a conclusion first and then invent steps to justify it; the steps are post-hoc rationalization.
Where it is worth trying
Math, logical reasoning, multi-step planning, and Q&A that must cite sources are CoT's home turf. Creative writing, chit-chat, and simple factual lookups mostly waste tokens with it: the tighter the derivation a question demands, the more CoT pays off; the more open-ended the answer, the less it is needed.
Chain-of-thought turns "think before you speak" into a repeatable operation. Remembering its limits and misconceptions often matters more than remembering how to use it.
