Tag
Quantization
Every published article tagged with Quantization.
Why Local LLM Execution Yields Subpar Reasoning Output
Aggressive quantization, small context windows, bad samplers explain why local llm dumber. Adjust parameters to restore reasoning.
AirLLM: Running 70B Parameter Models on a Single 4GB GPU
AirLLM claims you can run 70B models on consumer GPUs with just 4GB VRAM. Here's how it works, where it breaks, and whether it's actually useful for real workloads.
How Post-Training Quantization Shrinks LLMs to Run on Laptops
Under the hood of post-training quantization. Learn how mapping FP16 weights to INT4 shrinks LLMs, reduces memory bandwidth, and enables local AI execution.
4 min read