How increased sparsity affects pre-training loss performance

From Black Box to Schematic: OpenAI Improves Neural Network Interpretability with Weight Sparse Transformers

On November 13, 2025, OpenAI released a research and paper titled "Understanding neural networks through sparse circuits", proposing to improve the me...