A reward model is the judge that scores AI answers: it does not answer questions itself, but ranks the answers a model produces and tells it which kind is worth learning. During training, people first rank several answers to the same question; the reward model learns those preferences and can then score thousands of answers automatically, so the model adjusts toward higher scores.
What does it score?
It mainly learns three things: whether the answer actually solves the problem, whether it is clear and complete, and whether it crosses a safety boundary. Pairwise comparison is a common method: two answers to one question are shown, an annotator picks the better one, and the model generalizes from many such choices. Scores have no absolute unit; only relative ranking matters, and a different group of annotators or a different rubric will shift the tendency.
It is not the final judge
The scoring model and the chat model are two roles: one scores, the other answers. The scorer can also be wrong, for example by favoring long, confident answers even when they are inaccurate, or by being overcautious on some topics and marking normal answers down. It is more like a sparring partner during training than a quality inspector for every sentence after launch.
Three common misunderstandings
First, a high score does not mean the facts are right; a reward model measures what matches human preference, not what is true. Second, it does not replace human review; genuinely sensitive content still needs extra rules and human checks. Third, changing the scoring rubric does not instantly change a model's style; style changes come from further training, not from one tweak to the scorer.
Many things you notice in daily use are connected to it: answers getting more polite, more fond of bullet points, and quicker to refuse sensitive questions. These tendencies often come from the reward model repeatedly rewarding such behavior in training. Knowing this explains why a model sometimes prefers a cautious answer over a blunt one.
