Quantization

Lowering numeric precision to shrink a model.

Weights are stored at lower precision such as 4 bits instead of 16, cutting memory and compute. With small accuracy loss it is common for local use.