Chaining pruning, distillation, and quantization, and why order matters
A real compression pipeline rarely uses just one of these techniques, it chains them, and the order is not arbitrary. A common sequence is distillation or pruning first, quantization last. Pruning removes weights based on some notion of importance, and that notion is most meaningful when computed on a model still operating in full precision, where small differences between weights are still visible; quantizing first would compress those differences into a handful of discrete levels, making many weights look equally important when they aren't, and corrupting the pruning decision. Similarly, if distillation is used, it's typically run so the student model is trained (or fine-tuned) toward its final, already-pruned architecture, so the student's capacity matches what will actually ship.
Quantization goes last because it's the technique most sensitive to exactly what it's given: quantizing a model and then pruning it risks the pruning criterion being distorted by quantization noise, and quantizing a model and then further training it (as in QAT) needs a settled architecture to simulate against, not one still being resized by pruning decisions. In practice this gives a pipeline shape like: train or distill a right-sized student, prune it (structured, unstructured, or both) with optional fine-tuning to recover accuracy after each pruning step, and quantize the result last, often with QAT if the accuracy budget is tight. Running these out of order doesn't just underperform, it can actively work against itself, since each technique's decisions are only as good as the precision and structure of the model it's applied to.