Anthropic released the "How AI assistance impacts the formation of coding skills" study, which compares the skill mastery of developers after completing similar tasks under the conditions of "AI-assisted programming" and "handwritten code". The study concluded that the use of AI assistance leads to a statistically significant "decrease in mastery": in tests that cover concepts that have just been used, the AI group scores 17% lower than the handwriting group, which is equivalent to a gap close to two letter levels.
In terms of efficiency, the speed of the AI group to complete the task increased slightly, but the increase did not reach the statistically significant threshold. The study emphasizes that AI assistance not only affects the speed of output, but also changes the learning formation process: short-term understanding and memory are more likely to weaken when key reasoning and details are taken over by tools.
The report also suggests that the conclusions apply to the tasks and measurements they set, and that further validation is still needed when extrapolated to longer learning, tasks of different difficulty, or people with different proficiency levels. For individuals and teams, it is safer to use AI to speed up while retaining the necessary autonomous derivation, review and testing links to reduce the risk of "knowing how to use but not understand".
FAQs
Q: Which institution published this AI-assisted coding study?
A: The study was published by Anthropic and focused on the impact of AI assistance on coding skill formation.
Q: What are the core conclusions of Anthropic's research?
A: Studies have found that AI assistance significantly reduces real-time mastery, with an average reduction of 17% in relevant test scores.
Q: Has AI-assisted programming significantly improved development speed?
A: Studies have shown a slight increase in speed, but not at a statistically significant level.
Q: Does this study show that AI will definitely make programmers worse?
A: The research mainly points to the difference in learning under specific experimental tasks, which is not equivalent to absolute judgment of all scenarios.
Q: How can developers reduce the learning risks caused by AI assistance?
A: You can use AI while retaining independent derivation, code review, and targeted quizzes to avoid only copying and not understanding.
