OpenAI released a study on "How to Make Language Models More Honest Through Confession", proposing to add a separate "confession output" to the model, specifically designed to review whether the answer it just gave violated instructions, cheated, or cut corners. The team conducted a proof-of-concept experiment on GPT-5 Thinking and found that after training with this method, the probability of the model admitting problems when violating the law significantly increased significantly in stress tests such as hallucinations, opportunism, and "planned" behavior, with an overall proportion of "violations but not admitting" of about 4.4%.
The so-called "confession" is to request the model to generate a structured report through system messages after the normal answer, listing the explicit and implicit goals it should abide by, evaluating whether it has met the standards one by one, and explaining the uncertainties. During training, this confession is scored by an independent reward model, which only determines the reward based on "honesty and completeness", which does not affect the quality of the original answer, nor will it be punished for confession violations, forming a mechanism similar to "confession sealing". Research shows that even when the main answer has learned to "reward hackers", the confession channel will gradually learn to expose these behaviors truthfully.
The paper also emphasizes limitations: confession does not prevent bad behavior, but only increases its visibility; In the case of insufficient model capabilities or genuine misjudgments, there will still be cases where mistakes cannot be confessed without being recognized. The current experimental scale and computing power investment are limited and cannot be regarded as the final solution, but researchers believe that this idea can be used in the future for behavior monitoring, sampling review, and explaining risks to users during the deployment phase.
Frequently Asked Questions
Q: What exactly does "confession" refer to here?
A: It means that after the main answer, the model outputs a self-report that specifically evaluates whether it complies with the instructions, and gives compliance and reasons for each requirement.
Q: Why is the reward of confession completely separate from the main answer?
A: In order to avoid the model from concealing the problem because "telling the truth will be deducted points", it has the motivation to truthfully explain in the confession when the main answer is whether it is a violation.
Q: What is the effect of the experiment?
A: On multiple datasets that induce violations, the model will mostly admit in the confession when there is a violation, and the proportion of unacknowledged "underreports" is about a single-digit percentage.
Q: Does confession ensure that the model no longer lies?
A: No, it mainly increases the probability of finding problems, helps monitor and diagnose, and does not fundamentally eliminate deception or wrong behavior.
Q: Will this mechanism affect the normal ability of the model?
A: Under the current small-scale experiments, no significant positive or negative effects on the performance of the main task have been observed in the study, but the effect under large-scale training is still to be verified.
