Self-consistency is a decoding strategy, not a new model, and it needs no extra training. It has the model sample several different reasoning paths for the same problem using chain-of-thought, then takes a majority vote over the final answers of each path, and the answer with the most votes is the final answer. For tasks whose answers can be compared, such as maths and multiple-choice questions, it is often more accurate than a single pass — but it picks the answer the model itself gives most often; it is not external verification.
Sampling several times, then voting: how do the three steps work?
First the model is prompted with chain-of-thought examples, then greedy decoding is replaced with temperature sampling so that diverse reasoning paths are sampled from the decoder, and finally the final answers of all paths are tallied and the most frequent one is taken. In the paper, PaLM-540B was tested with 40 sampled paths: on GSM8K, a grade-school maths benchmark, accuracy rose from 56.5% to 74.4%, up 17.9 percentage points; SVAMP improved by 11.0 points, StrategyQA by 6.4 points, and ARC-challenge by 3.9 points. Correct paths often reach the same answer by different routes, while wrong paths go wrong in different ways, so the most frequent answer is an effective selection signal.
How is it different from chain-of-thought, self-reflection, and Best-of-N?
Chain-of-thought generates only one reasoning path and settles everything in a single greedy pass; self-consistency builds on chain-of-thought by computing several times and then voting. Self-reflection checks the answer afterwards and rewrites it when it finds an error — it changes the answer; self-consistency changes none of the paths and only picks one by vote, with no feedback loop. Best-of-N with a verifier needs a separately trained scoring model to score each path; self-consistency needs no scoring model, no extra model, and no human annotation — it just counts votes. The paper verified this approach on UL2-20B, GPT-3 175B, LaMDA-137B, and PaLM-540B.
When does it work, and when should you not rely on it?
It works only when two conditions hold at once: the final answer can be extracted and compared, and the paths are genuinely diverse. Open-ended writing and creative tasks have no single answer to vote on. The cost is also direct: sampling 40 paths costs about 40 times as much as a single path, latency and token spend rise with the number of samples, more samples bring larger gains, but with diminishing marginal returns. There is one easily overlooked limit: if the model is systematically wrong and most paths share the same misunderstanding, voting amplifies the wrong answer instead of correcting it. Diversity is a precondition — sampling must use temperature; greedy decoding gives the same result every time, making voting meaningless. When deciding, first ask: does this problem have one comparable answer, and could the model reach a different answer by another route? Only if both answers are yes is it worth computing several times.
