Nf4.rar ⟶

: RNF4 mediates the degradation of the PML-RARα fusion protein.

💡 : If you are looking for the software/machine learning paper, search for "QLoRA" or "4-bit NormalFloat" on arXiv .

The term "NF4" is central to this "long paper" which revolutionized how large language models (LLMs) are fine-tuned on consumer hardware. NF4.rar

: A feature to handle memory spikes during training by offloading to CPU RAM. 🔬 Key Technical Details

: An information-theoretically optimal data type for normally distributed weights. It uses 16 quantization levels based on the quantiles of a standard normal distribution. : RNF4 mediates the degradation of the PML-RARα

The paper explains why NF4 is superior to standard 4-bit integers (Int4) or floating-point (Float4) formats:

: Compresses 16-bit weights to 4 bits, effectively reducing VRAM usage by ~75% (e.g., a 65B parameter model can be loaded with ~35GB instead of ~130GB). : A feature to handle memory spikes during

: To reduce the memory footprint of LLMs (like Llama) enough to fit on a single GPU (e.g., a 24GB RTX 3090) while maintaining full 16-bit performance.