DL0151 Diffusion Policy and pi0 Flow Matching

Neither model emits one action per forward pass. Both treat a whole chunk of future actions as a single high-dimensional sample from a conditional generative model, and they produce it by starting from Gaussian noise and running an iterative sampler conditioned on the current observation. Diffusion Policy does this with a DDPM: a 1D temporal U-Net (or a transformer variant) predicts the noise inside a noisy action sequence, and denoising steps turn into an executable chunk, of which only the first actions are executed before replanning. π0 keeps that output object and changes two things. The sampler becomes conditional flow matching along a straight noise-to-action path integrated with about 10 Euler steps, and the denoiser becomes a 300M-parameter action expert placed inside a PaliGemma 3B VLM as a second set of weights in one transformer. Images and language flow through the VLM weights, the robot state and the 50 noisy action tokens flow through the expert weights, and a single shared self-attention operation joins them, which is why the backbone is initialized from a VLM and fine-tuned rather than kept literally frozen.

(1) Chunks, Not Single Actions: the policy models over an matrix, which suppresses per-step jitter and makes long idle or contact phases survivable.
(2) Denoising Is The Policy: sampling replaces regression, so the network never has to collapse several valid demonstrated behaviors into their average.
(3) Multimodality Is Preserved: an MSE regressor asked to pass left or right of an obstacle outputs the mean of the two, which hits the obstacle; a denoiser draws one mode per rollout.
(4) Receding Horizon Closes The Loop: predict , execute , re-observe, resample. This is the only feedback mechanism the chunk has.
(5) Flow Matching Straightens The Path: a linear interpolation between noise and data gives an almost constant velocity field, so 10 integration steps suffice for a 50-step chunk at 50 Hz.
(6) Two Experts, One Attention: π0 routes tokens to modality-specific weights but keeps one attention operation, and the prefix KV cache is computed once per observation while only the small expert runs on every integration step.

Figure 1: The sampler operates on the entire chunk at once, so temporal smoothness is a property of the generated sample rather than something enforced by a filter. At the bottom, only the leading actions of each chunk are executed, so the replanning period sets the reaction latency to anything the model did not anticipate.

The reason to pay for an iterative sampler is the shape of the demonstration data. Teleoperated demonstrations are multimodal and idle-heavy: the same scene is solved in several ways, and a maximum-likelihood Gaussian head trained with MSE returns the conditional mean, which is frequently not a valid action. Discretizing each dimension independently avoids averaging but breaks cross-dimension coordination, and a joint discretization is exponential in . Diffusion Policy’s published recipe uses observation horizon 2, prediction horizon 16, execution horizon 8, 100 DDPM training steps with 10 DDIM inference steps, FiLM conditioning of the observation embedding into a 1D temporal convolutional U-Net, and end-effector position control rather than velocity control, reporting an average 46.9% relative improvement over prior behavior-cloning baselines across 15 tasks.

Diffusion Policy (DDPM formulation):

The same object, an chunk, is what π0 produces, but the generative process is a continuous-time flow rather than a discrete Markov chain. Training samples a noise vector and a time from a beta distribution that deliberately over-weights the noisy end of the path, forms the linear interpolant, and regresses the network onto the constant velocity that carries noise to data. Because the path is straight by construction, inference integrates with a fixed step from to , which is 10 network evaluations for a chunk of 50 actions at 50 Hz. Cross-embodiment training is handled crudely and effectively: every state and action vector is zero-padded to the largest action dimension in the mixture (18 in the released model), and robots with fewer joints simply ignore the padded slots.

Figure 2: π0 is a mixture of two experts inside one transformer: the token type decides which weight matrices are applied, while attention is computed jointly over the whole sequence. Only the small expert is re-run per integration step, so the reported cost of a chunk is one 3B prefill plus ten passes over 300M parameters, roughly 73 ms in the released report.

Flow-matching action expert:

Where:

• is the action chunk starting at time , one action, the prediction horizon (16 in Diffusion Policy, 50 in π0), and the padded action dimension.
• and are the conditioning observations: for Diffusion Policy a short stack of image features and proprioception, for π0 the image tokens, language tokens, and the state token.
• indexes discrete denoising steps and is the noise-prediction network; come from the noise schedule and is the injected sampling noise.
• is the continuous flow time, the noise endpoint, and the linear interpolant between them.
• is the target velocity field of the straight path, the action expert’s prediction of it, and the Euler step, giving 10 evaluations per chunk.
• is the control period at 50 Hz, so one chunk covers one second; and are the per-pass costs of the 3B prefix and the 300M expert.

Figure 3: The blockwise causal mask is what makes the cost structure possible. Because the image and text prefix never attends to the noisy action tokens, its keys and values do not depend on and stay valid across all 10 integration steps, while the action block attends bidirectionally within the chunk so every predicted timestep sees every other.