Machine unlearning tackles a very specific problem: a model has already been trained on a certain set of data, and now it is supposed to behave as if it had never seen that data. Deleting the original data files does not achieve that, because the influence of the data already lives in the model's parameters, and the model may still repeat or use it.
There are currently two routes. Exact unlearning removes the data to be forgotten from the training set and retrains from scratch. By definition, that gives the cleanest result, but the compute cost is often hard to bear. Approximate unlearning adjusts the parameters of the existing model directly so its behavior comes as close as possible to a retrained model. That saves compute, but how close "close" is has to be checked against a retrained reference model.
It is not the same as catastrophic forgetting
Catastrophic forgetting means a model loses old skills while learning a new task. It is a side effect of training, and nobody specifies which data should be forgotten. Machine unlearning, by contrast, follows an explicit deletion request: the influence of a specific set of data is to be removed, and the goal and the test criteria are written into the request. Both involve "forgetting," but they point in completely different directions.
Why "forgotten" is so hard to verify
It is not enough to check that the model no longer outputs something: it may simply be suppressed and answer again when asked differently. Research therefore often compares the unlearned model with a retrained reference model, checking whether its behavior on the forgotten data and on the retained data is close enough. One compute-saving method trains the data in shards (SISA) and retrains only the affected shard. In an audit, what can usually be checked is which algorithm was run and whether the process can be reviewed — not whether a promise such as "already forgotten" is true.
So when someone says a model has forgotten certain data, ask three questions first: which specific set of data? Exact retraining or an approximate method? Checked against what reference? Anyone who cannot answer those is offering, at most, "no longer actively outputs it," not a removal from inside the model. For ordinary users, this also means: deleting a sentence from a chat history or clearing a conversation is a different thing from making a model unlearn training data. The first concerns the records in front of you; the second concerns the influence inside the model.
