Full Fine-Tuning

Fine-tuning that updates every parameter of a pretrained model, rather than training just a small subset of them.

Fine-tuning means continuing to train a pretrained model on data for a downstream task. Full fine-tuning updates every weight; the alternative is parameter-efficient fine-tuning (PEFT), which trains only a small number of new or selected parameters, such as LoRA (low-rank adaptation). Full fine-tuning has the most freedom to adapt, and when data is plentiful and the target task differs a lot from pretraining, it usually performs best — but it's expensive: every parameter needs its own stored gradient and optimizer state, pushing memory needs to several times that of plain inference, and a full copy of the model must be saved per task; with little data it's also more prone to overfitting or catastrophic forgetting, where the model loses old abilities while learning a new task. When adapting a VLA to a new robot or task, full fine-tuning and LoRA are the two most common choices, and picking between them comes down to available GPU memory and how much data there is.

ExampleThe OpenVLA (7B) paper reports full fine-tuning needing 8 A100s running 5–15 hours, reaching 69.7% success; LoRA trains only 1.4% of the parameters, fits on a single A100, and reaches 68.2%. The openpi repository lists π0 full fine-tuning as needing over 70 GB of memory versus about 22.5 GB for LoRA.

Also called

Full-parameter Fine-Tuning

Related

Fine-tuning · Parameter-Efficient Fine-Tuning · LoRA · Catastrophic Forgetting · Backbone Freezing · Supervised Fine-Tuning

Sources

LoRA: Low-Rank Adaptation of Large Language Models
OpenVLA: An Open-Source Vision-Language-Action Model
openpi GitHub README（显存需求表） (Chinese)

As of

2025