Emergent abilities are task abilities that small models can barely perform at all, but that suddenly become possible once a model's scale crosses a certain threshold. This is not ordinary, gradual improvement, but an apparent jump from nothing to something: when a model is small, it gets multi-step arithmetic almost entirely wrong and cannot learn a new format even when given several examples; once it is large enough, it suddenly solves the same problems correctly, works through complex reasoning step by step, and can learn a new task on the spot from examples. This ability to learn from examples is called in-context learning, and it is the most frequently cited example of emergence.
Why overall progress is smooth, but individual abilities jump
Emergence cannot be discussed without scaling laws. Scaling laws describe how, as the number of parameters, the amount of data, and compute grow, a model's overall prediction loss on large amounts of text falls smoothly and predictably. The catch is that smooth overall loss does not mean every individual task improves smoothly. Tasks such as multiple-choice questions and multi-step word problems require a whole chain of steps to be right; one wrong step makes the final answer wrong. A small model is in fact inching closer, but while it is still a few steps short of fully correct, its score sheet shows nothing but zeros. Once the accuracy of each step crosses a critical point, the whole problem suddenly comes out right, and the curve shows a jump.
The debate: some emergence may be an artifact of evaluation
Whether emergence means something inside the model has truly changed qualitatively is still debated. Some research points out that many reported jumps are tied to how the evaluation metric is chosen: if you only look at an all-or-nothing metric such as whether the final answer is right, progress gets compressed into steps; switch to a more continuous metric, such as scoring how close each token prediction is or giving partial credit for correct steps, and the curves for the same models become much smoother, with far less sense of a jump. This is a reminder that when an ability appears to pop up suddenly, we cannot conclude directly that some qualitative change must have occurred inside the model; part of the effect may be amplified by the measurement method. That does not disprove emergence, either; it only shows that we need to distinguish changes in the model from changes in the metric.
The limits: it cannot be precisely predicted, and parameters are not the only variable
The most practical limit of emergence is that it cannot be precisely predicted. No one can calculate in advance at what scale a given ability will appear; it can only be confirmed in hindsight, after the model has been trained and tested. Nor does every ability emerge: some tasks improve only slowly with scale, and some remain poor no matter what. The number of parameters is not the only variable, either. Data quality, data composition, and training methods also change when an ability appears, and two models with similar parameter counts can perform very differently. So it is fair to say scale is a threshold for emergence, but not fair to say that piling up parameters will necessarily buy a particular ability.
Two common misconceptions also need clearing up. Emergence does not mean awakening consciousness. It describes a change in task performance, a measurable and reproducible phenomenon, which is a different matter from whether a model has subjective experience. Emergence also does not mean that bigger means all-powerful. Jumps happen only on some tasks, and the same model may still make frequent mistakes on others. Understanding emergence as a limited, not fully controllable change in ability brought about by scale is closer to what the term actually means in research than imagining it as some mysterious awakening.
