Model Compression and Quantization
The routes to smaller, faster models: how distillation moves capability into a smaller network, what low-precision quantization actually buys, and where small language models stop being enough.
Getting a model onto a phone or an entry-level GPU means three things: distillation, where a small network imitates a large one; quantization, which drops weights to four or eight bits; and pruning, which removes low-value connections. You trade memory, latency and power for accuracy, and the loss is uneven - long-form generation and math degrade first. This tag covers when each method is worth it and how to verify the result before shipping.
