Auxiliary Loss / Auxiliary Task

An extra prediction task and loss term added alongside the main training objective, to help the model learn better features.

An auxiliary task means having the same network make additional, related predictions beyond the main training objective (such as outputting an action or maximizing reward); each prediction has its own auxiliary loss, weighted and added into the total loss to optimize together. These tasks share the backbone (the part of the network responsible for extracting features) with the main task, and the extra supervisory signal helps features get learned faster and more stably; the extra branches are usually simply discarded at inference. In reinforcement learning, where rewards are often sparse and the learning signal weak, auxiliary tasks are especially useful: DeepMind's UNREAL agent (2016) added pixel-control, reward-prediction, and value-replay auxiliary tasks, learning roughly 10 times faster in the 3D maze environment Labyrinth. Robot imitation learning and VLAs also commonly have the model predict future images, object locations, or sub-task text on the side. If an auxiliary loss is weighted too heavily, it competes with the main task for model capacity, so the weight needs tuning.

ExampleDeepMind's 2016 navigation agent, while learning to find goals in a 3D maze, also had the same network predict scene depth and judge whether it had returned to a previously visited location (loop-closure classification); these two auxiliary losses noticeably improved navigation performance.

Also called

Auxiliary Objective

Related

Loss Function · Representation Learning · Multi-Task Learning · Reinforcement Learning · Sparse Reward · Backbone Network

Sources

Reinforcement Learning with Unsupervised Auxiliary Tasks (UNREAL, arXiv 1611.05397)
Learning to Navigate in Complex Environments (arXiv 1611.03673)