DL0148 Action Chunking Transformer (ACT)

ACT is the imitation-learning policy introduced with the ALOHA bimanual setup: a transformer encoder-decoder that takes the current multi-view images plus the 14-dimensional joint state and emits, in a single forward pass, a whole chunk of future joint-position targets rather than one action. It is trained as a conditional VAE (CVAE) with an reconstruction loss on the chunk plus a KL term on a small style latent , which lets one deterministic-at-test-time policy absorb the multimodality of human teleoperation instead of averaging it away. Chunking attacks drift arithmetically: at 50 Hz a 20-second task is 1,000 control steps but only chunk decisions, so the number of places where the policy can leave the training distribution shrinks by a factor of and the classic behaviour-cloning bound falls to roughly . Chunking also removes per-step temporal inconsistency, because the actions inside a chunk are generated jointly from one latent and one observation, so the policy cannot flip between two valid modes on consecutive 20 ms steps. To avoid a visible discontinuity when a new chunk starts, ACT still queries the network every step and blends the overlapping predictions with temporal ensembling.

(1) The Chunk Is The Output Unit: the policy models instead of , so correlated actions are predicted together as one object.
(2) Effective Horizon Divided By : the agent makes sequential decisions, and compounding error grows with the number of decisions, not with the number of motor commands.
(3) CVAE For Multimodality: a BERT-style encoder compresses the demonstrated chunk into a 32-dimensional latent during training and is discarded at test time, where gives one clean decisive mode.
(4) Non-Markovian Demos Stop Being Fatal: a single-step policy standing at a human pause sees a bimodal target (hold still or move) and can freeze forever; a chunk that contains the pause and the following motion resolves the ambiguity.
(5) Temporal Ensembling For Smoothness: re-query every step and average the overlapping chunk predictions with weights , which keeps reaction latency at one control step while removing chunk-boundary jumps.
(6) Over : the loss on absolute joint targets produces sharper, less smeared trajectories, which matters for millimetre-scale contact tasks.

Architecturally there is nothing exotic. Each of the four RGB streams (two wrist cameras, two static cameras at 480×640) goes through a ResNet-18 whose final feature map is flattened into roughly 300 tokens, the joint vector becomes one more token, and the style latent becomes one more; a 4-layer transformer encoder with width 512 mixes them, and a 7-layer decoder attends to that memory from fixed learned position embeddings, one per future step, so the output is produced non-autoregressively in about 0.01 s. The whole policy is roughly 80M parameters trained from scratch per task on about 50 demonstrations, which is the interesting part: the gain does not come from scale or pretraining, it comes from changing what a single prediction means.

Figure 1: ACT is a CVAE whose decoder is a chunk predictor. The dashed lane exists only during training: it sees the ground-truth action sequence and squeezes the demonstrator’s stylistic choice into . At test time is fixed to the prior mean, so the same network becomes a deterministic policy that outputs absolute joint targets in one pass.

Mathematical Formulation:

Where:

• is the predicted action chunk, here absolute target joint configurations in for a bimanual 6-DoF-plus-gripper arm pair.
• are the four camera images at time and the measured joint positions, together forming the only observation the chunk is conditioned on.
• is the style latent, sampled from the encoder posterior in training and set to at inference.
• indexes positions inside the chunk, each produced by its own learned decoder query.
• weights the KL term; a large collapses to noise while a small one lets the encoder leak the answer and hurts test-time behaviour.
• The norm is deliberate: it penalises large joint errors less quadratically than and therefore averages competing modes less aggressively.

The drift argument is worth writing down. Standard behaviour cloning with per-step error suffers covariate shift: a mistake moves the robot to a state the demonstrations never covered, the next prediction is worse, and the regret bound is quadratic in the horizon. Chunking does not make the policy immune, it reduces how many times the loop is closed. With decision points, each contributing error that persists over the remaining decisions, and each decision covering timesteps of cost, the accumulated term scales as . The same factor cuts inference calls, which is why a chunked policy can afford a much heavier network per decision.

Horizon Arithmetic At 50 Hz:

Figure 2: The bound is still quadratic in task length, so chunking does not abolish drift, it buys two orders of magnitude at . The practical reading is that a long-horizon task becomes as hard as a short one only if the chunk itself remains executable open-loop, which is exactly the assumption that breaks when the scene moves during the 2 seconds a chunk spans.

Naive open-loop execution of the chunk creates a new problem: every 2 seconds a fresh observation produces a fresh chunk that need not start where the previous one ended, and the robot jerks. ACT therefore runs the policy at the full control rate and aggregates. At timestep there are up to different chunks that predict an action for , one from each of the previous queries, and the executed command is their exponentially weighted mean with , where index is the oldest prediction. That value of keeps the weights nearly uniform (1.00, 0.99, 0.98, …), so new observations are folded in immediately while the average stays smooth; a larger leans on the oldest chunk and reacts more slowly.

Temporal Ensembling:

Figure 3: Chunks overlap because the policy is queried every step, so each control command is a vote of up to predictions made from different observations. This is what keeps ACT closed-loop at 50 Hz while its prediction horizon stays 2 seconds long, and it costs one extra forward pass per step rather than any extra training.