Vectorized Environments

Running many independent copies of an environment at once to collect interaction data in batches.

Vectorized environments are a way to speed up sampling in reinforcement learning: the same environment is duplicated N times (N is the num_envs value in code), and each step sends in N actions together and gets back N observations, rewards, and done flags — exactly forming one batch to feed a neural network. Gymnasium provides a sequential SyncVectorEnv and a multi-process AsyncVectorEnv, and automatically resets any copy whose episode ends without waiting for the others. GPU simulators such as Isaac Gym, Isaac Lab, and MJX go a step further, computing thousands of environments at once on a single GPU and handing the data to the training code directly as PyTorch tensors; the Isaac Gym paper claims speedups of two to three orders of magnitude over CPU simulation. Rudin and colleagues used this in 2021 to get an ANYmal quadruped walking on flat ground in under 4 minutes.

ExampleIsaac Lab's velocity-tracking task for legged robots defaults to num_envs=4096, simulating 4,096 robot copies at once on a single GPU while training a walking policy with PPO.

Also called

num_envs, Vector Env, Parallel Environments

Related

GPU-Accelerated Parallel Simulation · Massively Parallel Reinforcement Learning · NVIDIA Isaac Lab · Isaac Gym · Simulation Throughput (Steps / Frames Per Second) · Proximal Policy Optimization

Sources

Gymnasium Documentation: Vector environments
Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning (arXiv 2108.10470)
Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning (arXiv 2109.11978)

As of

2026-09