Story details

Researchers introduce PACT, a system for LoRA fine-tuning on consumer GPUs that uses native NVFP4 matrix operations for frozen projections. The paper also addresses memory spent on activations and the cost of quantizing and dequantizing data.

The work targets computation and memory bottlenecks during fine-tuning. It is a preprint describing an evaluated system, not a new GPU or a guarantee that every consumer setup will achieve the same results.