Knowledge Insulation

A VLA training trick that blocks gradients from the action expert back into the VLM backbone, protecting its pretrained knowledge.

Knowledge insulation is a VLA training method from a May 2025 Physical Intelligence paper by Danny Driess, Sergey Levine, and colleagues. Models such as π0 attach a newly initialized action expert to a VLM backbone and use flow matching to output continuous actions, but gradients flowing back from the action expert into the backbone disrupt it, which slows training and hurts language understanding. The fix has three parts: a stop-gradient cuts off gradients from the action expert to the backbone; the backbone instead learns actions through next-token prediction on discretized FAST action tokens, while co-training on ordinary VLM data such as visual question answering; and at inference time, the action expert still generates continuous action chunks quickly from the backbone's features.

ExampleOn a table-bussing task, the paper reports that the knowledge-insulated model converges about 7.5 times faster than a π0 trained purely with flow matching, while keeping better language-instruction-following ability.

Also called

KI, Knowledge Insulating VLA

Related

Stop-Gradient · Action Expert · π0.5 · π0-FAST · Catastrophic Forgetting · Co-training

Sources

Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Better (arXiv:2505.23705)
论文 HTML 版（方法与实验细节） (Chinese)

As of

2025-05