Tag: VLA

Vision-Language-Action models (robotic policies)

DL0157 VLA Zero-Shot and Few-Shot Transfer

How do you evaluate zero-shot and few-shot transfer of generalist VLA policies to novel physical environments and unseen manipulation objects, for a checkpoint such as Google DeepMind’s RT-2 or the open OpenVLA model?

Answer

Evaluating transfer is a measurement design problem, not a demo reel. The protocol must vary one generalization axis per evaluation cell, hold everything else fixed with a scripted scene reset and matched initial states, and report a per-axis success rate with a confidence interval instead of one aggregate number. Zero-shot cells run the frozen checkpoint at $k = 0$ demonstrations, while few-shot cells sweep $k \in \{1, 5, 10, 25\}$ target demonstrations and report a curve plus the retention on the original seen-task suite after adaptation. The trial budget sets the resolution of the entire experiment, because 20 trials per cell give a 95% interval of roughly $\pm 22$ points at $\hat{p} = 0.5$ , which cannot resolve the 10 to 15 point differences people actually argue about. Cheap simulation suites such as LIBERO and SIMPLER buy statistical power and reproducibility, and real-robot paired trials buy validity, so a credible report uses both and checks that they rank policies the same way.

(1) Factor-Isolated Axes: Split “novel” into unseen object instance, unseen object category, unseen scene and background, added distractors and camera pose, and rephrased language. A single mixed “hard eval” set cannot attribute a failure to any one of them.
(2) Matched Initial States: Every policy under comparison sees the same object poses, lighting, and clutter, logged from a reset script or a fixture template, which turns a noisy independent comparison into a paired test with far more power.
(3) Trial Budget Sets Resolution: Detecting a 10-point difference at 95% confidence needs roughly 190 trials per arm, so 20-trial evals should be reported as pilots, never as rankings.
(4) Progress Score, Not Only Binary Success: Score reach, grasp, transport, and place as staged partial credit in $[0, 1]$ , which exposes whether a novel object breaks perception or grasp geometry.
(5) Few-Shot Is A Curve: Report $\hat{p}(k)$ over several $k$ values and fit a demo-efficiency constant, because a single $k = 10$ number hides whether the policy needed 2 demos or 50.
(6) Retention And Blind Operation: Re-run the seen-task suite after fine-tuning to measure catastrophic forgetting, and keep the operator unaware of which checkpoint is running to remove reset and intervention bias.

Protocol diagram with five generalization axis lanes on the left (unseen object instances, unseen object categories, unseen scene, unseen distractors with camera pose shift, rephrased instructions) feeding a tall paired-trial harness box that specifies scripted resets, matched initial states, a blind operator, randomized trial order, N trials per cell and a staged progress score, which then splits into a zero-shot arm at k equals zero demos and a few-shot arm at k equals 1, 5, 10, 25 demos, each producing its own report block

Figure 1: One checkpoint, one factor per cell. The harness is the part people skip: without scripted resets and matched initial states, the operator becomes an uncontrolled variable, and the same policy can swing 20 points between two afternoons. Each cell also carries a seen-object control in the new scene, so a gap can be attributed to the object rather than to the room.

The protocol hygiene matters more than the model comparison. Each trial follows a fixed loop, reset → verify pose against a reference photo → run for a hard time limit → score stages → log the episode, and every episode is kept, including ones the operator considers unfair, because discarding “bad setups” is how a 55% policy becomes an 80% policy on paper. Interventions must be recorded rather than silently corrected, since a policy that needs one nudge per episode is not transferring. On the few-shot side, the demonstrations for a novel object must be collected under the same distribution the evaluation will use, and the evaluation objects must be held out from those demonstrations; reusing the same physical mug for teaching and testing measures memorization, not transfer. Finally, adaptation method is part of the result: LoRA on the action head, full fine-tuning, and fine-tuning with a replay mix of pretraining data produce very different retention curves at the same novel-task success.

Mathematical Formulation:
$\hat{p} = \frac{1}{N}\sum_{i=1}^{N} s_i$
$\mathrm{CI}_{95} = \hat{p} \pm 1.96\sqrt{\hat{p}(1-\hat{p})/N}$
$G_a = \hat{p}_{\mathrm{seen}} - \hat{p}_a$
$N \geq 2 z^2 \bar{p}(1-\bar{p}) / \Delta^2$
$\hat{p}(k) = p_{\infty} - (p_{\infty} - p_0)e^{-k/\tau}$

Where:

$\hat{p}$ is the estimated success rate of one evaluation cell and $s_i$ the outcome of trial $i$ , either binary or a staged progress score in $[0, 1]$ .
$N$ is the number of trials in that cell, the single quantity that decides whether the reported number can support a claim.
$G_a$ is the generalization gap on axis $a$ , measured against the seen-task control run in the same session and the same scene.
$z = 1.96$ at 95% confidence, $\bar{p}$ is the pooled rate of the two arms, and $\Delta$ is the minimum detectable difference you are willing to claim.
$k$ is the number of target-domain demonstrations, $p_0$ the zero-shot rate, $p_{\infty}$ the saturation rate, and $\tau$ the demo-efficiency constant in demonstrations, so smaller $\tau$ means faster adaptation.
The unpaired formula for $N$ is an upper bound; matched initial states reduce the variance of the difference and can cut the required $N$ by a factor of two or more.

Trial Budget For A 10-Point Claim:
$N \geq 2(1.96)^2(0.5)(0.5)/(0.10)^2$
$N \geq 192$

At roughly 90 seconds per real trial including reset, 192 trials per arm is about 5 hours of operator time for a single cell, and a five-axis grid with two policies multiplies that by ten. That arithmetic, not modelling taste, is why teams push generalization sweeps into simulation and reserve real hardware for a small number of decisive paired comparisons, and why crowd-sourced or autonomous evaluation has become an active research direction.

Two panel chart: left panel shows per-axis success rate bars for seen tasks, novel object instance, novel object category, novel scene, added distractors and rephrased instruction, each with a wide red 95 percent interval for 20 trials and a narrow black interval for 200 trials, showing that the 20 trial intervals overlap across axes; right panel plots success rate against number of demonstrations per novel task at k equals 0, 1, 3, 5, 10 and 25 for novel object and novel scene curves rising toward saturation, plus a declining dashed curve for retention on the original seen task suite after naive fine-tuning

Figure 2: Left, the same measurements at two trial budgets. With 20 trials per cell the intervals for novel instance, distractors, and rephrasing all overlap, so the axis ranking is unsupported; at 200 trials the ordering becomes real. Right, few-shot transfer is a curve with a second axis nobody reports: novel-task success climbs with $k$ while seen-task retention decays under naive fine-tuning.

Property	Simulation suites (LIBERO, SIMPLER, CALVIN)	Real-robot paired trials
Cost of 100 trials	Minutes of wall clock, fully parallel across workers	2 to 3 hours of operator time plus fixtures and resets
Axes varied cheaply	Object mesh and texture, lighting, camera pose, distractor count, instruction text	Anything physically buildable, but each new factor costs a scene rebuild
What it misses	Contact dynamics, deformables and liquids, actuator lag, real camera noise and exposure	Nothing by construction, since it is the deployment distribution
Achievable precision	Thousands of trials, interval within a few points	20 to 50 trials is typical, giving intervals of 15 to 22 points
Dominant validity risk	Rank inversion versus hardware, and policies tuned to the renderer	Operator drift, inconsistent resets, unlogged interventions, cherry-picked episodes
Right role	Wide axis sweeps, regression gates, hyperparameter and checkpoint selection	A few decisive head-to-head comparisons and the final transfer claim

August 19, 2026

DL0156 Force-Torque and Visual Fusion in VLA

How do force-torque sensor inputs and joint currents fuse with visual features inside VLA cross-attention layers to handle physical contact tasks?

Answer

Fusion begins with tokenization, not with attention: a 6-axis wrench sampled at 1 kHz and the per-joint motor currents are cut into short windows (typically 50-200 ms), pushed through a small 1D CNN or spectrogram encoder, pooled down to roughly one token per 10 ms, and projected by a learned matrix into the same $d$ -dimensional space that holds the visual patch tokens. Inside the block, the language and vision residual stream supplies the queries while the concatenated visual and proprioceptive tokens form the key/value memory, so softmax attention decides layer by layer how much contact evidence to read instead of averaging modalities with fixed weights. Because the pretrained VLM has never seen a wrench token, the cross-attention output is normally added through a zero-initialized tanh gate, which makes the new modality an exact no-op at initialization and lets it grow in during robot fine-tuning without destroying the language prior. The two streams also carry different information: vision says where and what, force says when and how hard, and force is the only channel that survives occlusion once the tool is inside the hole. What this fusion cannot do is close the contact loop, since one VLA forward pass costs 50-200 ms while an impact transient decays in a few milliseconds, so the policy emits setpoints, target wrenches, and impedance gains for a 1 kHz low-level controller that does the actual reacting.

(1) Window Then Tokenize: a 1 kHz stream cannot enter a 5 Hz backbone sample by sample, so it is windowed and pooled into about 10 tokens per 100 ms, which keeps the added sequence length near 2 percent of a two-camera observation.
(2) Shared Embedding Space: a per-modality encoder plus a linear projection is what makes wrench tokens, current tokens, and patch tokens legal keys in the same attention operation.
(3) Queries From The Stream, Keys From The Sensors: cross-attention is asymmetric on purpose, so the policy can ignore force during free-space motion and attend heavily to it during contact.
(4) Zero-Init Gating: $\tanh(\alpha)$ with $\alpha = 0$ preserves the pretrained VLM at step zero and is the standard fix for adding a modality that was absent from web-scale pretraining.
(5) Action Space Must Be Contact-Aware: position-only outputs cannot express compliance; the head has to emit a target wrench or stiffness and damping for an impedance or admittance law.
(6) Frequency Separation: the transformer supervises at 5-10 Hz and the 1 kHz controller reflexes, because policy latency is roughly 30 times longer than the impact transient it would need to catch.

Architecture diagram with three input lanes for the language instruction, wrist and scene cameras, and a 6-axis force-torque sensor with joint currents; each lane passes through its own encoder plus linear projection to the model width, the vision and force tokens form a shared key-value memory, the language and vision hidden states act as queries into a gated cross-attention block inside the pretrained VLM, and the policy emits an action chunk, a target wrench, and impedance gains

Figure 1: Only the encoders and projections are modality-specific. After projection, the wrench and current tokens are ordinary keys and values, and the zero-initialized gate controls how much of that memory reaches the residual stream. The head deliberately outputs gains and wrenches, not just poses, because a stiff position command in contact is how robots break parts.

Two practical details decide whether this works on hardware. The first is signal conditioning: a raw wrench includes tool gravity, payload inertia, and thermal bias drift, so the encoder must be fed a gravity-compensated and re-zeroed signal, otherwise the same physical contact produces different tokens after every payload change. Joint current is an even noisier torque proxy, since $\tau_j = k_t i_j$ holds only after subtracting harmonic-drive friction and stiction, which are velocity-dependent and hysteretic; current is therefore best used for coarse collision and jam detection across the whole arm rather than for fine wrench estimation at the tool. The second is bandwidth. Slip, chatter, and impact live in the 100-1000 Hz band, so pooling to 10 Hz destroys exactly the evidence you added the sensor for, which is why many implementations feed short-time spectral features or per-window statistics (peak, RMS, jerk) alongside the mean rather than a naive average.

Four stacked time plots over 400 milliseconds: the top shows a 1 kHz force-torque channel that jumps to 22 newtons at contact and rings down within about 4 milliseconds to an 8 newton steady value, the second shows camera frames as ticks every 100 milliseconds, the third shows VLA forward passes as 115 millisecond blocks repeating at 5 hertz, and the bottom shows the commanded stiffness as a staircase that only changes when a new policy output lands

Figure 2: The rate mismatch is the whole design constraint. The impact transient starts and finishes inside a single policy cycle, and the camera may not even sample it, so the VLA can only choose a compliance policy in advance and let the 1 kHz loop execute it. Anything that must react in under 10 ms cannot live in the transformer.

Mathematical Formulation:
$w_t = (f_x, f_y, f_z, \tau_x, \tau_y, \tau_z)$
$\tau_j = k_t i_j - \tau_{f}(\dot q_j)$
$U_{ft} = W_{ft}\, \phi_{ft}(w_{t-K+1:t})$
$M = [\, U_{vis} ; U_{ft} ; U_{cur} \,]$
$C = \mathrm{Attn}(h W_q,\; M W_k,\; M W_v)$
$h \leftarrow h + \tanh(\alpha)\, C$
$f_{cmd} = K(x_d - x) + D(\dot x_d - \dot x)$

Where:

$w_t \in \mathbb{R}^{6}$ is the wrench at time $t$ , with three forces $f$ and three moments $\tau$ expressed in the tool frame after gravity and payload compensation.
$i_j$ is the measured current of joint $j$ , $k_t$ the motor torque constant, and $\tau_f(\dot q_j)$ the friction and stiction term that makes current a biased torque estimate.
$\phi_{ft}$ is the window encoder over the last $K$ samples and $W_{ft}$ its projection to the backbone width $d$ ; $U_{ft}$ is the resulting small set of soft tokens.
$M$ is the concatenated key/value memory holding visual, force-torque, and joint-current tokens, and $h$ is the language and vision hidden state that provides queries through $W_q$ .
$\alpha$ is the learned scalar gate, initialized at $0$ so that $\tanh(\alpha) = 0$ and the block reduces to the pretrained VLM on the first step.
$K$ and $D$ are the commanded stiffness and damping matrices, $x_d$ and $\dot x_d$ the desired pose and velocity, and $f_{cmd}$ the wrench the 1 kHz controller actually applies.

Rate And Token Budget:
$K = 0.1 \times 1000 = 100$
$L_{ft} = 100 / 10 = 10$
$t_{vla} \approx 115\ \mathrm{ms}$
$t_{contact} \approx 4\ \mathrm{ms}$

A 100 ms window at 1 kHz gives 100 raw samples, pooled to 10 tokens, so about 100 force tokens per second ride alongside thousands of visual tokens. Meanwhile roughly 29 impact transients fit inside one policy forward pass, which is the quantitative reason the fusion is about situational awareness and gain scheduling rather than reflexes. The most common training pathology follows from the same asymmetry: proprioceptive inputs are low-dimensional and almost perfectly predict the next demonstrated action, so behaviour cloning happily learns to regress the previous command from state and stops looking at the camera. That is textbook causal confusion, and the usual defenses are proprioception dropout, delta actions instead of absolute targets, action chunking, and holdout episodes with the object moved.

Line chart of the share of cross-attention probability mass over normalized task progress for a peg insertion, with vision tokens starting near 0.62 and falling to about 0.30 during the shaded contact and insertion phase, force-torque tokens rising from 0.05 to about 0.56 across that same phase, and joint-current tokens staying below 0.18 throughout, with phase labels for reach, approach, contact plus insert, and retract

Figure 3: Illustrative attention-mass trace for a peg-insertion policy. During free-space reaching the wrench tokens are near zero and attract almost no mass, and once the peg is inside the hole vision is largely occluded so the wrench becomes the dominant evidence. The shares do not sum to one because language and history tokens hold the remainder.

Property	Early token concat (self-attention)	Gated cross-attention adapter	Hierarchical (force outside the model)
Where force enters	Appended to the input sequence, every layer sees it	As keys and values in an inserted cross-attention block	Only in the 1 kHz impedance loop, plus a scalar contact flag
Parameters touched	Full backbone fine-tune in practice	Encoder, projection, and adapter only; backbone can stay frozen	None; the controller is hand-designed
Robot data needed	Largest, since force was absent from pretraining	Moderate, the zero-init gate keeps the prior intact	Smallest, contact behaviour is not learned
Contact reaction latency	One policy cycle, 100-200 ms	One policy cycle, 100-200 ms	About 1-2 ms
Dominant failure mode	Proprioception shortcut and catastrophic forgetting of language grounding	Gate saturates near zero and the sensor is silently ignored	Cannot express task-dependent compliance or recover from a jam

August 19, 2026

DL0155 Crowdsourced Teleoperation Quality

How do you handle dataset quality variance in crowdsourced teleoperation data when training large VLA policies, for a collection like DROID or Open X-Embodiment?

Answer

Crowdsourced teleoperation data is a mixture over operators, rigs, and sessions, not a single distribution. DROID’s 76k trajectories were collected by about 50 operators across 13 institutions, and Open X-Embodiment pools 60 datasets over 22 embodiments, so per-episode quality varies more than the task labels do. The working assumption is that quality is a measured per-episode variable, not a binary success flag, so the first engineering step is a scoring pass that attaches provenance plus cheap kinematic proxies to every episode and stores them as metadata rather than deleting anything. Training then spends that metadata at three separate points: group weights over operator or site during pretraining, a quality token in the conditioning so the model learns the difference between clean and sloppy behaviour, and a short anneal on a curated gold subset that decides what the policy actually imitates at test time. Hard filtering is the last resort, because imitation error compounds as $O(\epsilon H^2)$ and most of that $\epsilon$ comes from states the policy has never visited, which is exactly what the messy tail supplies. Two mechanical fixes matter as much as the curation: percentile-based action normalization so one jerky operator cannot rescale the action space, and an expressive action head over action chunks so contradictory strategies are not averaged into a mean action.

(1) Quality Is A Variable, Not A Flag: score every episode continuously and keep the score, because a policy trained on a thresholded boolean cannot be re-mixed later without a second scoring pass.
(2) Cheap Automatic Proxies: action jerk, path-length ratio, idle-time fraction, regrasp and retry count, teleop latency spikes, and VLM-based verification of the success label and the language annotation.
(3) Provenance Is The Grouping Unit: operator, site, and rig ids define the groups used for reweighting and, critically, for held-out evaluation splits so a strong operator’s style does not leak across train and test.
(4) Coverage Versus Purity: the sloppy tail carries the recovery states and rare scenes that keep compounding error small, so aggressive filtering can lower success even while raising average demo quality.
(5) Pretrain Broad, Post-Train Narrow: the data-pyramid recipe used by recent VLA foundation models trains on nearly everything and then anneals on a small, verified, high-quality set.
(6) Quality-Conditioned Cloning: feed the tier as a token during training and sample at the gold setting at inference, which uses bad data as negative evidence instead of throwing it away.
(7) Robustness Plumbing: 1st/99th percentile action normalization as in OpenVLA, plus flow-matching or diffusion heads, so outliers and multimodality do not corrupt the regression target.

Pipeline diagram: three crowdsourced collection sites with per-operator episode counts feed an automatic episode scoring stage using VLM success checks, action jerk, path ratio, idle fraction, regrasps and latency spikes; the scorer assigns gold, silver and bronze tiers; all three tiers feed one training recipe box listing group-DRO pretraining weights, a quality token in the prompt, and a final anneal on the gold tier; the recipe trains a VLA policy that is evaluated on held-out sites, with a feedback arrow returning operator scorecards and re-collection requests to the sites

Figure 1: The topology is what matters: scoring is a metadata pass, not a delete pass, all three tiers reach the pretraining stage, and only the last stage is restricted to gold. The closing arrow is the part teams forget, since per-operator scorecards turn quality measurement into targeted re-collection rather than a one-time filter.

The scoring pass should stay cheap enough to run over every episode. Kinematic proxies are computed directly from the recorded action stream: mean squared jerk over the trajectory, the ratio of executed end-effector path length to the straight-line distance, the fraction of timesteps with near-zero commanded velocity, and the number of gripper open/close reversals as a regrasp counter. Semantic proxies need a model: a VLM watches the final frames and votes on whether the stated goal was reached, and a second pass checks that the language annotation matches the video, which catches the common crowdsourcing failure where the instruction and the demonstration disagree. These proxies are then aggregated into one score per episode and, separately, per operator. Reweighting uses the group statistic because per-episode weights are noisy and because the real covariate shift is at the operator and rig level, where calibration offsets, camera mounts, and latency are shared.

Mathematical Formulation:
$q_j = \sigma(\beta^{\top} \phi(\tau_j))$
$\mathcal{L}_g(\theta) = \mathbb{E}_{(o,a) \sim D_g}[\ell(a, \pi_{\theta}(o))]$
$\min_{\theta} \max_{w \in \Delta_G} \sum_{g=1}^{G} w_g \mathcal{L}_g(\theta)$
$J(\pi^{*}) - J(\hat{\pi}) \leq C \epsilon H^2$
$\tilde{a} = 2 (a - q_{01}) / (q_{99} - q_{01}) - 1$

Where:

$q_j \in (0,1)$ is the quality score of episode $\tau_j$ , with $\phi$ the proxy feature vector (jerk, path ratio, idle fraction, regrasps, VLM verdict) and $\beta$ fitted on a few hundred human-labelled episodes.
$\mathcal{L}_g$ is the loss on group $g$ , where a group is one operator, site, or source dataset, and $D_g$ its observation-action pairs with observation $o$ (images plus instruction plus proprioception) and action chunk $a$ .
$g \in \{1,\ldots,G\}$ indexes groups, $\Delta_G$ is the simplex of mixture weights, and the inner maximization is group DRO, which upweights whichever group is currently worst-fit instead of trusting a hand-tuned mixture.
$\pi_{\theta}$ is the policy, $\ell$ the per-sample cloning loss, $H$ the rollout horizon, and $\epsilon$ the per-step error under the expert’s state distribution; the $H^2$ factor is why coverage of off-nominal states is worth more than average demo neatness.
$q_{01}$ and $q_{99}$ are the 1st and 99th percentiles of each action dimension over the training mixture, so $\tilde{a} \in [-1,1]$ is a quantile-normalized target that a single spiking operator cannot stretch.

The trade-off curve is the part that surprises people. Sorting episodes by score and keeping only the top fraction improves the average target the policy regresses onto, but it also removes scenes, lighting conditions, object instances, and above all recovery segments: the moment where an operator overshot, re-approached, and succeeded is precisely the state a deployed policy will find itself in. Empirically the filter-only curve is an inverted U, peaking somewhere around half the data and then falling below the train-on-everything baseline once coverage collapses. Reweighting dominates filtering because it keeps every state in the support while shrinking the gradient contribution of noisy actions, and the gold anneal captures most of the remaining gain at almost no cost since it touches only the final few percent of optimization steps.

Line chart of real-robot success rate against the fraction of training data retained when keeping the highest-quality episodes first. The hard-filter curve is an inverted U peaking near 55 percent retained and dropping sharply below the baseline at 10 percent retained, while the reweight-plus-gold-anneal curve rises monotonically and is highest when all data is retained. A dash-dotted horizontal line marks the uniform train-on-everything baseline.

Figure 2: Two different shapes from the same scores. Hard filtering trades coverage for purity and therefore has an interior optimum you must search for, while reweighting plus a gold anneal keeps the full state support and improves monotonically as more data is retained. The gap at the right edge is the cost of deleting data you could have downweighted instead.

Quality variance also shows up as multimodality, which is a modelling problem rather than a data problem. When two operators route around the same obstacle in opposite directions, the conditional action distribution given the observation is bimodal, and any head trained with a plain mean-squared-error objective converges to $\mathbb{E}[a \mid o]$ , the average of the two modes. That average is often an invalid action, so the policy fails on the exact scenes where it had the most data. The fixes are an expressive head (diffusion or flow matching over an action chunk, as in the pi-zero family) which samples a single coherent mode, action chunking so a commitment persists for several timesteps instead of flip-flopping per step, and conditioning on style or operator id when the modes are genuinely operator-specific. This is also why “smooth” should never be the only quality criterion: a smooth demonstration that solves the task differently from the rest of the corpus still injects a competing mode.

Two panels sharing the same scene with a start point, a goal, and a rectangular obstacle between them. Left panel: one operator's demonstrations arc above the obstacle, another operator's arc below, and a red dashed straight line at the midline shows that the mean-squared-error head predicts the average of both modes and drives into the obstacle. Right panel: a flow-matching or diffusion head produces three sampled trajectories above and three below, each committing to a single side and clearing the obstacle.

Figure 3: Mixed-operator data is multimodal in action space. A unimodal regression head collapses two valid strategies onto their mean, which here is a straight line through the obstacle, while a sampling head over action chunks commits to one mode per rollout. No amount of filtering fixes this if both strategies are high quality.

Property	Hard filtering	Group reweighting + gold anneal	Quality-conditioned cloning
Mechanism	Drop every episode below a score threshold before training	Per-group loss weights during pretraining, then a short fine-tune on the gold tier	Tier token in the conditioning; sample at the gold setting at inference
Effect on coverage	Shrinks the state support, removes rare scenes and recovery segments	Full support retained; only the gradient share of noisy groups shrinks	Full support retained; bad data acts as contrastive evidence
Extra machinery	One threshold, tuned by expensive real-robot sweeps	Group ids, a weight optimizer such as group DRO, a second training stage	Score discretization, a token in the prompt, calibrated tier boundaries
When it wins	Labels are corrupt rather than merely sloppy, or the episode is unsafe to imitate at all	Large heterogeneous corpora where coverage is the scarce resource	Scores are reliable and the sloppy modes are still physically valid
Main failure mode	Proxy measures task difficulty, so filtering silently deletes the hard tasks	Weights overfit one noisy group; the anneal overfits and forgets the broad prior	Miscalibrated tiers make the gold token meaningless, and the policy ignores it

August 19, 2026

DL0154 Synthetic Data and Isaac Sim

How do synthetic data generators such as Isaac Sim and Omniverse produce realistic tactile, force sensor, and RGB-D depth outputs for VLA pre-training?

Answer

Nothing in the stack emits a realistic sensor reading directly. Each modality comes out of a different subsystem with its own fidelity ceiling, and realism is added afterwards as an explicit sensor model. RGB and depth are produced by the RTX renderer and read back through Omniverse Replicator annotators, where distance_to_image_plane returns per-pixel metric depth with no holes, no quantization, and no stereo shadows. Joint and contact forces are read out of the PhysX 5 reduced-coordinate articulation solver, so a “measured” wrist force is really a constraint impulse divided by the physics timestep. Tactile is not a first-class sensor at all: it is reconstructed from the signed-distance penetration between a gel collider and the object, then either rendered as a GelSight-style image or collapsed into a taxel normal-force map. Everything the simulator hands you is exact, and the actual engineering is deciding how to break it so a VLA trained on it survives a real RealSense and a real six-axis load cell.

(1) Ground Truth First, Noise Second: the renderer and the solver give error-free signals, so every realism claim rests on a post-processing layer that injects noise, quantization, dropout, bandwidth limits, and latency.
(2) RGB-D Is A Rendering Plus Annotator Problem: PBR materials and HDRI domes handle appearance, while tiled rendering makes many cameras affordable, but no renderer reproduces stereo matching failure on specular and transparent surfaces unless you model it.
(3) Forces Are Solver Readouts: get_measured_joint_forces and the contact reporter expose impulses at the physics rate, so magnitudes depend on timestep, substeps, solver iterations, and contact offset rather than on any sensor.
(4) Tactile Is Reconstructed Geometry: the penetration field against the object SDF drives both the rendered gel image and the taxel forces, with shear obtained by clipping tangential displacement to the Coulomb friction cone.
(5) Physics Calibration Sets Force Fidelity: friction, mass, inertia, restitution, and SDF colliders instead of convex hulls matter more to contact realism than any amount of visual polish.
(6) Scale Comes From Trajectories, Not Pixels: Replicator randomizes lighting, materials, poses, and per-environment physics while MimicGen-style augmentation turns a few dozen human demos into hundreds of thousands of contact-rich episodes.

Pipeline diagram in which a USD scene feeds both a PhysX 5 GPU solver and the RTX renderer, which in turn feed three sensor lanes for joint and contact forces, a tactile penetration field, and RGB-D annotators, all passing through a shared sensor realism layer of noise, quantization, invalid masks, filtering and latency before a dataset writer and VLA pre-training

Figure 1: Three sensor lanes, one scene graph. Only the RGB-D lane is a genuine sensor simulation; the force lane is a solver readout and the tactile lane is reconstructed from geometry, which is why the shared realism layer does most of the sim-to-real work.

Depth is where naive pipelines fail first. The annotator gives exact metric depth, so a policy trained on it learns to trust razor-sharp object boundaries, reads valid depth off glass and polished steel, and never sees the occlusion shadow that a stereo baseline creates on the left edge of every object. A usable depth channel is therefore built by emulating the device: convert depth to disparity, quantize it, add matching noise that grows as $z^2$ , invalidate pixels where the simulated left and right views disagree or where the material is specular or transparent, quantize to the 1 mm uint16 grid, and finally apply the frame delay and rolling-shutter smear of the real driver. Two annotator traps show up in review: distance_to_camera returns Euclidean range while distance_to_image_plane returns the $z$ component that RGB-D APIs expect, and a policy trained on the wrong one develops a radial bias that grows toward the image corners.

Line chart of injected axial depth noise in millimetres versus range in metres for stereo baselines of 50, 95 and 120 millimetres, all growing quadratically with range, plus a flat zero-error line for the simulator raw depth and a dash-dotted line marking the one millimetre uint16 quantization step

Figure 2: Depth realism is arithmetic, not rendering. With $f = 600$ px and $\sigma_d = 0.1$ px, a 50 mm baseline is submillimeter at 0.5 m and roughly 13 mm at 2 m, while the simulator’s raw depth sits on the flat zero-error line at every range.

Mathematical Formulation:
$z = \frac{f b}{d}$
$\sigma_z = \frac{z^2}{f b}\,\sigma_d$
$\tilde z = q\,\mathrm{round}(z/q) + \eta$
$F_{\mathrm{meas}} = \lambda_c / \Delta t$
$d(u,v) = \max(0,\, -\phi(x_{uv}))$
$\|f_t\| \leq \mu f_n$

Where:

$z$ is metric depth, $f$ the focal length in pixels, $b$ the stereo baseline, and $d$ the disparity in pixels.
$\sigma_d$ is subpixel matching noise, typically 0.05 to 0.2 px, and the $z^2$ factor is why one camera is millimeter-accurate up close and centimeter-accurate at 2 m.
$q$ is the depth quantization step (1 mm for uint16 output) and $\eta$ the residual additive noise; a separate invalid mask zeroes stereo-shadow, specular, and transparent pixels.
$\lambda_c$ is the constraint impulse the solver applied at a joint or contact and $\Delta t$ the physics step, so the reported force scales as $1/\Delta t$ and is rate and solver dependent rather than physical.
$\phi$ is the object’s signed distance function evaluated at gel sample $x_{uv}$ , and $d(u,v)$ is the penetration depth field on the tactile grid that drives both the gel image and the taxel normal force.
$f_n$ and $f_t$ are normal and tangential contact force with friction coefficient $\mu$ ; marker flow exists only inside the cone, and slip appears when the bound is reached.

On the force side the honest framing is that PhysX gives you a well-behaved dynamics readout, not a load cell. A stiff impact resolved in one 8.3 ms step at 120 Hz appears as a single enormous impulse, whereas a real 1 kHz sensor reports a damped ringdown shaped by its own mechanical resonance plus a slowly drifting bias and a few percent of cross-axis coupling. The practical recipe is more substeps or a smaller $\Delta t$ for contact-rich phases, a contact history buffer in the sensor wrapper, then a low-pass filter to the real sensor bandwidth followed by injected bias, drift, and colored noise. Tactile needs the same discipline plus a calibration step: the elastomer in simulation is rigid geometry, so penetration depth substitutes for deformation, and the mapping from $d(u,v)$ to gel pixel intensity has to be fitted against real presses on known indenters. Hysteresis, adhesion, and creep are simply absent, which bounds how far a purely simulated tactile channel can carry a slip-detection policy.

Two panels: a cross-section showing an object surface pressing 1.2 millimetres into a 3 millimetre gel layer with the penetration region shaded and the contact radius marked, and a heatmap of the resulting penetration depth field over a 12 by 12 millimetre tactile grid with radial shear arrows and a dashed contact boundary

Figure 3: Tactile output is reconstructed, not sensed. The penetration field $d(u,v) = \max(0, -\phi)$ against the object SDF supplies both the rendered gel image and the taxel normal-force map, while tangential flow is only valid inside $\|f_t\| \leq \mu f_n$ .

Property	RGB-D	Joint and contact force	Tactile
Source subsystem	RTX renderer plus Replicator annotators	PhysX 5 articulation and contact solver	SDF penetration query plus a gel renderer (TacSL, TACTO)
Native output	Exact metric depth, segmentation, normals, no holes	6D joint reaction force and net contact force at the physics rate	Penetration depth field on the gel grid, plus a shear field
Fidelity ceiling	Asset and light-transport quality; stereo and ToF failure modes are not rendered	Friction, mass, inertia, solver iterations, contact and rest offset	Gel treated as rigid geometry, so hysteresis, adhesion and creep are missing
Mandatory post-processing	Disparity quantization, $z^2$ noise, invalid mask, blur, frame latency	Low-pass to sensor bandwidth, bias drift, cross-axis coupling, spike clipping	Illumination and marker calibration on real presses, Coulomb-clipped shear
Real data still needed	Little for RGB, moderate for depth on shiny and transparent scenes	Small for quasi-static tasks, large for impacts and insertion	Large: a real fine-tuning set of presses, rolls and slips is usually unavoidable

August 19, 2026

DL0153 Sim-to-Real Transfer

What is sim-to-real transfer in robot learning, and how do domain randomization, system identification, and visual domain adaptation each close a different part of the reality gap?

Answer

Sim-to-real transfer is training a control policy in a physics simulator, where data is fast, safe, and infinitely resettable, and then deploying it on hardware that obeys different physics and produces different pixels. The reality gap is the mismatch between the simulated MDP and the real one, and it decomposes into at least three channels: dynamics parameters that are simply mistuned (mass, friction, joint damping, motor gains), unmodeled effects that the simulator has no term for (cable drag, gear backlash, control latency, deformable contact), and the observation gap between rendered and captured images. The three classic techniques are not competitors; each attacks a different channel. Domain randomization trains one policy over a distribution of simulators so the real robot behaves like just another sample from that distribution, system identification measures the real robot and moves the simulator’s parameters onto it, and visual domain adaptation leaves dynamics alone and instead makes sim and real images look identical to the encoder. Production pipelines for legged locomotion and dexterous manipulation compose them: identify what you can measure, randomize what you cannot, adapt the residual online with a history-conditioned policy.

(1) The Gap Is Not One Number: dynamics, unmodeled effects, and observations fail independently, so a policy can transfer perfectly in torque space and still collapse because the real camera has motion blur.
(2) Domain Randomization Widens The Simulator: sampling $\xi \sim p(\xi)$ each episode buys robustness without any real data, provided the true parameters lie inside the support.
(3) System Identification Shrinks The Gap Itself: fitting $\xi$ to logged real trajectories gives a sharper simulator and a higher-performance policy, but only for effects the simulator can represent.
(4) Visual Domain Adaptation Works In Pixel Space: feature alignment, randomized-to-canonical translation, or image-to-image GANs remove appearance shift that no amount of physics tuning touches.
(5) Support Coverage Is The Binding Constraint: if $\xi_{\mathrm{real}}$ falls outside the randomization support, the policy has literally never trained on the real robot’s physics and no amount of averaging helps.
(6) Online Adaptation Recovers The Lost Performance: conditioning on a short history of states and actions lets the policy infer the latent parameters at run time, which is why teacher-student distillation and rapid motor adaptation beat blind robustness.

Two-panel conceptual diagram. Left panel shows a dynamics parameter space with axes friction coefficient and payload mass, a blue dot for the simulator nominal parameters, a red star for the real robot parameters, a large shaded rectangle marking the randomization support that contains the star, and a dashed arrow from the nominal point to the star labeled system identification. Right panel shows a visual feature space with a tight cluster of sim render points and a separate cluster of real camera image points, a large tilted ellipse marking the spread produced by visual randomization that envelops both clusters, and a curved arrow mapping the real cluster onto the sim cluster labeled domain adaptation.

Figure 1: The same gap, two spaces and two directions of correction. In parameter space randomization inflates the simulator’s support until it contains the real robot, while system identification translates the nominal point onto it. In feature space randomization spreads the sim distribution until it envelops the real look, while adaptation pulls real observations back onto the canonical sim appearance.

The mechanisms differ in what they need and what they cost. Domain randomization needs zero real data: you pick ranges for mass, friction, terrain roughness, motor strength, sensor noise, latency, and texture, then resample every episode so the policy cannot memorize one dynamics model. Its price is conservatism, because a memoryless policy maximizing expected return over a wide $p(\xi)$ converges to behavior that is safe for the average simulator and optimal for none, which is why Automatic Domain Randomization grows the ranges only as fast as the policy can absorb them. System identification runs in the opposite direction: excite the hardware, log $(s_t, a_t, s_{t+1})$ , and minimize prediction residuals to recover $\hat{\xi}$ . It buys back performance and shrinks the randomization ranges you still need, but it can only fit parameters the simulator exposes, so backlash or a compliant tendon that has no corresponding term stays invisible no matter how good the fit is. Visual domain adaptation is orthogonal to both: RCAN-style translation maps heavily randomized or real images to one canonical rendering before the policy sees them, GraspGAN-style translation pushes sim images toward realism, and feature-level alignment penalizes the discrepancy between sim and real encoder activations. The standard modern recipe is a pipeline: identify → randomize the residual uncertainty → train a privileged teacher with access to $\xi$ → distill into a student that reads only onboard history → deploy.

Mathematical Formulation:
$\Delta(\pi) = J_{\mathrm{real}}(\pi) - J_{\mathrm{sim}}(\pi)$
$\pi_{\mathrm{DR}} = \arg\max_{\pi} E_{\xi \sim p(\xi)} [ J_{\xi}(\pi) ]$
$\xi_{\mathrm{real}} \in \mathrm{supp}(p(\xi))$
$\hat{\xi} = \arg\min_{\xi} \sum_{t=1}^{T} \| e_t(\xi) \|^2$
$e_t(\xi) = s_{t+1} - f_{\xi}(s_t, a_t)$
$\min_{g} \; d( g(o_{\mathrm{sim}}), g(o_{\mathrm{real}}) )$

Where:

$J_{\mathrm{real}}$ and $J_{\mathrm{sim}}$ are the expected returns of policy $\pi$ on hardware and in simulation, and $\Delta(\pi)$ is the reality gap measured in task performance rather than in physics units.
$\xi$ is the vector of simulator parameters (masses, friction coefficients, motor gains, latencies, texture and lighting seeds) and $J_{\xi}$ is the return in the simulator instantiated with $\xi$ .
$p(\xi)$ is the randomization distribution; the support condition is the precondition for transfer, since a policy trained on $p(\xi)$ has no guarantee at all for parameters outside $\mathrm{supp}(p(\xi))$ .
$\hat{\xi}$ is the identified parameter estimate, $f_{\xi}$ the simulator’s one-step transition model, and $e_t$ the one-step prediction residual against a logged real transition from state $s_t$ under action $a_t$ .
$t \in \{1, \ldots, T\}$ indexes the real excitation trajectory; residual that cannot be driven to zero by any $\xi$ is unmodeled physics, and it is exactly what must be randomized or adapted away instead.
$g$ is the visual encoder, $o_{\mathrm{sim}}$ and $o_{\mathrm{real}}$ are observations from the two domains, and $d$ is a distribution distance such as an adversarial discriminator loss or a maximum mean discrepancy.

Line chart of task success rate versus randomization half-width as a fraction of the nominal parameter value. The sim-domain success curve starts near 0.97 at zero width and decays gently to about 0.73 at full width. The real-world success curve starts near zero, rises steeply once the support begins to contain the true parameters around a width of 0.3, peaks near 0.89 around 0.45, then declines to about 0.39 at full width. A dotted vertical line marks where the real parameter enters the support and a shaded band marks the practical operating range.

Figure 2: Randomization width has an inverted-U effect on real-world success while sim success decays monotonically, so sim reward is an actively misleading model-selection signal. Too narrow and $\xi_{\mathrm{real}}$ sits outside the support; too wide and the policy averages over incompatible dynamics. System identification moves the useful band left by removing uncertainty you no longer need to cover.

Property	Domain randomization	System identification	Visual domain adaptation
Gap channel it closes	Unknown dynamics and appearance, covered by breadth	Mistuned but representable dynamics parameters	Observation shift between rendered and captured images
Real data required	None, only sensible ranges	Excitation trajectories on the target robot	Unlabeled real images, no reward or actions needed
Main cost	Sample complexity plus a conservative policy	Hardware time, and it must be redone per unit and as parts wear	A second generative or adversarial model to train and maintain
Dominant failure mode	Support misses the true parameters, or breadth destroys performance	Overfits to one robot and cannot fit unmodeled effects	Translation hallucinates or drops task-critical detail; dynamics untouched
Best fit	Contact-rich locomotion and manipulation across a fleet	High-precision tasks on one well-instrumented platform	Image-based grasping and navigation with cheap renderers

August 19, 2026

DL0152 VLA Frequency Gap Bridge

How do you bridge the execution frequency gap between a low-frequency VLA policy running at 3 to 10 Hz, as in a pi-0 or GR00T N1 class model, and a high-frequency joint controller demanding 500 to 1000 Hz torque or impedance updates?

Answer

You never let the VLA command joints directly. The frequency gap is closed by a three-tier cascade in which each tier runs at its own clock: the VLA emits an action chunk covering hundreds of milliseconds of future motion instead of a single next action, a mid-rate head or buffer replays that chunk at 20 to 100 Hz, and a real-time layer upsamples the waypoints to the servo rate and closes an impedance or torque loop at 1 kHz against live encoder feedback. Chunking alone is not enough, because a blocking call stalls the controller for the whole inference window, so the chunk is computed asynchronously while the previous one is still executing, and the first few actions of the new chunk are frozen or blended to match what the robot has already committed to. Between waypoints you interpolate with a cubic or minimum-jerk segment rather than holding the last value, since a zero-order hold turns every chunk step into a velocity impulse the drives cannot follow. The real-time layer also owns safety: a watchdog that decays to gravity compensation when the buffer underruns, plus joint limits and torque saturation that the neural policy never sees.

(1) Chunks, Not Single Actions: the policy predicts $H$ future actions per forward pass, converting a 3 to 10 Hz decision rate into a continuous 20 to 100 Hz stream of targets.
(2) Horizon Must Cover Latency: the chunk has to be long enough to keep the buffer fed through one full inference plus transport delay, otherwise execution stutters at every boundary.
(3) Asynchronous Inference: issue the next forward pass while the current chunk still has actions left, so compute overlaps execution instead of interrupting it.
(4) Frozen Prefix And Blending: the first $d$ actions of a fresh chunk are already stale on arrival, so they are discarded or soft-constrained to the committed trajectory to avoid a jump at the splice.
(5) Interpolate, Never Zero-Order Hold: a spline between waypoints spreads each step over the $r$ controller ticks in between and keeps commanded velocity and acceleration bounded.
(6) The 1 kHz Layer Owns Safety: impedance gains, torque limits, and a buffer-underrun watchdog run on a real-time thread that never waits on a GPU.

Vertical three-tier diagram: a VLA backbone at 3 to 10 Hz with 100 to 300 ms inference passes features down to an action expert that emits a chunk of 50 joint targets at 50 Hz, which feeds a real-time layer performing cubic upsampling and a 1 kHz impedance law before reaching the robot joints, with a feedback path returning observations resampled to 5 Hz

Figure 1: Three clocks on one command path. Each tier only has to meet the deadline of the tier below it, so the 200 ms VLA period never appears as a 200 ms hole in the torque loop. Only the bottom tier is hard real time, and it is the only tier that reads encoders at 1 kHz.

The scheduling detail is what separates a demo from a deployed system. In the naive loop you observe, block on the network and the GPU for 100 to 300 ms, then execute the chunk, which means the controller spends a large fraction of every cycle replaying a stale target or holding still, and the robot visibly pauses at each boundary. Running inference asynchronously removes the hole but introduces a second problem: the chunk that arrives at time $t$ was conditioned on the observation from $t - t_{lat}$ , so its early actions describe a state the robot has already left. Real-time chunking handles this by treating the overlap as an inpainting constraint, keeping the first $d$ actions pinned to the trajectory already in flight and letting the sampler adjust only the free tail. The cheaper approximation used by ACT is temporal ensembling: keep every overlapping prediction for the current timestep and average them with weights $w_k = \exp(-mk)$ , which smooths the splice but adds no latency compensation and biases the command toward older observations.

Timing diagram with two lanes: the upper blocking lane alternates 130 ms inference bars with 200 ms execution blocks separated by red hatched 130 ms hold gaps, while the lower asynchronous lane overlaps inference bars with execution so that chunk blocks butt against each other with no gap

Figure 2: With a 130 ms forward pass and a 200 ms chunk, blocking inference leaves the controller starved for 130 ms out of every 330, roughly 39% dead time. Overlapping the next forward pass with the current execution removes the gaps entirely, and the price is that every chunk acts on an observation that is one cycle old, which is exactly what the frozen prefix compensates for.

Mathematical Formulation:
$T_{chunk} = H / f_a$
$d = \lceil f_a \, t_{lat} \rceil$
$H \geq d + \lceil f_a / f_{vla} \rceil$
$r = f_c / f_a$
$\tau = K_p (q_d - q) + K_d (\dot q_d - \dot q) + g(q)$

Where:

$T_{chunk}$ is the wall-clock horizon a single chunk covers, $H$ is the number of actions in the chunk, and $f_a$ is the rate at which those actions are consumed.
$t_{lat}$ is the end-to-end delay from shutter to first usable action, covering encoding, network transport, and the forward pass; $d$ is the resulting number of stale leading actions.
$f_{vla}$ is the policy replan rate, so $\lceil f_a / f_{vla} \rceil$ is how many actions are consumed per replan and the third relation is the no-underrun condition.
$f_c$ is the servo rate and $r$ the upsampling ratio, the number of interpolated setpoints emitted between two consecutive policy waypoints.
$\tau$ is the joint torque, $q_d$ and $\dot q_d$ the interpolated position and velocity setpoints, $q$ and $\dot q$ the measured state, and $g(q)$ the gravity term.
$K_p$ and $K_d$ set the mechanical impedance; low gains make the arm compliant and forgiving of a slightly wrong setpoint, high gains make it track hard and punish every command discontinuity.

Budget For A 5 Hz VLA On A 1 kHz Arm:
$d = \lceil 50 \times 0.13 \rceil = 7$
$\lceil f_a / f_{vla} \rceil = 50 / 5 = 10$
$H \geq 7 + 10 = 17$
$r = 1000 / 50 = 20$

With a 130 ms latency, a 50 Hz action rate, and a replan every 200 ms, the chunk needs at least 17 actions, and shipping $H = 50$ (a 1.0 s horizon) buys margin for a GPU hiccup or a dropped packet. The controller then produces 20 interpolated setpoints per waypoint, so the neural policy is responsible for shape and the real-time layer for smoothness. Note that a longer horizon is not free: everything past the next replan is open-loop motion, so the chunk length trades buffer robustness against reaction time to disturbances, and only the first $f_a / f_{vla}$ actions of a 50-action chunk are normally executed at all.

Two-panel chart: left panel plots a reference joint trajectory against a 5 Hz zero-order-hold staircase, a 5 Hz linear ramp, and a 50 Hz waypoint sequence cubically upsampled to 1 kHz; right panel is a log-scale bar chart of peak commanded joint velocity for the three command paths with a dashed joint velocity limit line

Figure 3: The same intended motion, three command paths. A zero-order hold asks for a finite position jump inside one 1 ms tick, so the implied velocity sits two orders of magnitude above the joint limit and the drive answers with a torque spike. Both interpolated paths stay under the limit, but only the 50 Hz waypoints actually reproduce the reference shape; upsampling a 5 Hz command smooths the command at the cost of cutting the corners of the trajectory.

Property	Blocking chunk + hold	Overlapping chunks + temporal ensembling	Async chunking + frozen prefix
Controller starvation	One dead window per cycle, about 39% duty loss at 130 ms latency	None if the ensemble buffer stays full	None, inference always overlaps execution
Boundary smoothness	Jump whenever the new chunk disagrees with the held target	Smooth, the exponential average filters the disagreement	Smooth by construction, the prefix is pinned to committed actions
Latency compensation	None, the whole chunk is stale by $t_{lat}$	None, and averaging biases toward older observations	Explicit, the first $d$ actions are skipped or constrained
Extra cost	Cheapest, one forward pass per executed chunk	Keeps several chunks in memory, needs a weighting hyperparameter	Needs a client-server split, a chunk buffer, and guided sampling
Reasonable use	Quasi-static pick and place, teleop replay, offline evaluation	On-board policies with low, stable latency	Remote or large models, dynamic tasks, anything with jittery latency

August 19, 2026

DL0151 Diffusion Policy and pi0 Flow Matching

How does Diffusion Policy generate continuous robot action chunks through denoising, and how does Physical Intelligence’s π0 instantiate the same idea as a flow-matching action expert on top of a pretrained vision-language backbone?

Answer

Neither model emits one action per forward pass. Both treat a whole chunk of $H$ future actions as a single high-dimensional sample from a conditional generative model, and they produce it by starting from Gaussian noise and running an iterative sampler conditioned on the current observation. Diffusion Policy does this with a DDPM: a 1D temporal U-Net (or a transformer variant) predicts the noise inside a noisy action sequence, and $K$ denoising steps turn $A^K \sim \mathcal{N}(0, I)$ into an executable chunk, of which only the first $T_a$ actions are executed before replanning. π0 keeps that output object and changes two things. The sampler becomes conditional flow matching along a straight noise-to-action path integrated with about 10 Euler steps, and the denoiser becomes a 300M-parameter action expert placed inside a PaliGemma 3B VLM as a second set of weights in one transformer. Images and language flow through the VLM weights, the robot state and the 50 noisy action tokens flow through the expert weights, and a single shared self-attention operation joins them, which is why the backbone is initialized from a VLM and fine-tuned rather than kept literally frozen.

(1) Chunks, Not Single Actions: the policy models $p(A_t \mid O_t)$ over an $H \times d$ matrix, which suppresses per-step jitter and makes long idle or contact phases survivable.
(2) Denoising Is The Policy: sampling replaces regression, so the network never has to collapse several valid demonstrated behaviors into their average.
(3) Multimodality Is Preserved: an MSE regressor asked to pass left or right of an obstacle outputs the mean of the two, which hits the obstacle; a denoiser draws one mode per rollout.
(4) Receding Horizon Closes The Loop: predict $H$ , execute $T_a \leq H$ , re-observe, resample. This is the only feedback mechanism the chunk has.
(5) Flow Matching Straightens The Path: a linear interpolation between noise and data gives an almost constant velocity field, so 10 integration steps suffice for a 50-step chunk at 50 Hz.
(6) Two Experts, One Attention: π0 routes tokens to modality-specific weights but keeps one attention operation, and the prefix KV cache is computed once per observation while only the small expert runs on every integration step.

Top row shows three line plots of two action dimensions over the fifty steps of a chunk, starting as pure Gaussian noise, then partially denoised, then a smooth executable trajectory, with denoise arrows between them. Bottom panel shows three overlapping horizontal bars representing chunks predicted at successive replanning times, each with a shaded leading segment marking the executed portion.

Figure 1: The sampler operates on the entire chunk at once, so temporal smoothness is a property of the generated sample rather than something enforced by a filter. At the bottom, only the leading $T_a$ actions of each chunk are executed, so the replanning period sets the reaction latency to anything the model did not anticipate.

The reason to pay for an iterative sampler is the shape of the demonstration data. Teleoperated demonstrations are multimodal and idle-heavy: the same scene is solved in several ways, and a maximum-likelihood Gaussian head trained with MSE returns the conditional mean, which is frequently not a valid action. Discretizing each dimension independently avoids averaging but breaks cross-dimension coordination, and a joint discretization is exponential in $d$ . Diffusion Policy’s published recipe uses observation horizon 2, prediction horizon 16, execution horizon 8, 100 DDPM training steps with 10 DDIM inference steps, FiLM conditioning of the observation embedding into a 1D temporal convolutional U-Net, and end-effector position control rather than velocity control, reporting an average 46.9% relative improvement over prior behavior-cloning baselines across 15 tasks.

Diffusion Policy (DDPM formulation):
$A_t = (a_t, a_{t+1}, \ldots, a_{t+H-1})$
$\hat{\epsilon} = \epsilon_{\theta}(O_t, A_t^k, k)$
$A_t^{k-1} = \alpha_k (A_t^k - \gamma_k \hat{\epsilon}) + \sigma_k z$
$\mathcal{L}_{\mathrm{DP}} = \mathbb{E}\|\epsilon - \hat{\epsilon}\|^2$

The same object, an $H \times d$ chunk, is what π0 produces, but the generative process is a continuous-time flow rather than a discrete Markov chain. Training samples a noise vector and a time $\tau \in [0,1]$ from a beta distribution that deliberately over-weights the noisy end of the path, forms the linear interpolant, and regresses the network onto the constant velocity that carries noise to data. Because the path is straight by construction, inference integrates with a fixed step $\delta = 0.1$ from $\tau = 0$ to $\tau = 1$ , which is 10 network evaluations for a chunk of 50 actions at 50 Hz. Cross-embodiment training is handled crudely and effectively: every state and action vector is zero-padded to the largest action dimension in the mixture (18 in the released model), and robots with fewer joints simply ignore the padded slots.

Architecture diagram with four input lanes for camera images, language instruction, robot state, and the noisy action chunk with its tau embedding, each passing through its own encoder into a single transformer stack that contains two weight sets: PaliGemma VLM weights for the prefix tokens and a 300 million parameter action expert for state and action tokens, joined by shared self-attention, producing a velocity field that is integrated by ten Euler steps into the final action chunk.

Figure 2: π0 is a mixture of two experts inside one transformer: the token type decides which weight matrices are applied, while attention is computed jointly over the whole sequence. Only the small expert is re-run per integration step, so the reported cost of a chunk is one 3B prefill plus ten passes over 300M parameters, roughly 73 ms in the released report.

Flow-matching action expert:
$A_t^{\tau} = \tau A_t + (1-\tau)\epsilon$
$u(A_t^{\tau} \mid A_t) = A_t - \epsilon$
$\mathcal{L}_{\mathrm{FM}} = \mathbb{E}\|v_{\theta}(A_t^{\tau}, o_t) - u\|^2$
$A_t^{\tau+\delta} = A_t^{\tau} + \delta\, v_{\theta}(A_t^{\tau}, o_t)$
$H \Delta t = 50 \times 20\ \text{ms} = 1000\ \text{ms}$
$C = C_{\mathrm{VLM}} + 10\, C_{\mathrm{expert}}$

Where:

$A_t \in \mathbb{R}^{H \times d}$ is the action chunk starting at time $t$ , $a_i$ one action, $H$ the prediction horizon (16 in Diffusion Policy, 50 in π0), and $d$ the padded action dimension.
$O_t$ and $o_t$ are the conditioning observations: for Diffusion Policy a short stack of image features and proprioception, for π0 the image tokens, language tokens, and the state token.
$k \in \{K, \ldots, 1\}$ indexes discrete denoising steps and $\epsilon_{\theta}$ is the noise-prediction network; $\alpha_k, \gamma_k, \sigma_k$ come from the noise schedule and $z \sim \mathcal{N}(0, I)$ is the injected sampling noise.
$\tau \in [0,1]$ is the continuous flow time, $\epsilon \sim \mathcal{N}(0, I)$ the noise endpoint, and $A_t^{\tau}$ the linear interpolant between them.
$u$ is the target velocity field of the straight path, $v_{\theta}$ the action expert’s prediction of it, and $\delta = 0.1$ the Euler step, giving 10 evaluations per chunk.
$\Delta t = 20\ \text{ms}$ is the control period at 50 Hz, so one chunk covers one second; $C_{\mathrm{VLM}}$ and $C_{\mathrm{expert}}$ are the per-pass costs of the 3B prefix and the 300M expert.

Three by three block attention matrix with rows as query blocks and columns as key blocks. The prefix row attends only to the prefix column and is marked bidirectional and cached. The state row attends to prefix and state. The action row attends to all three blocks and is bidirectional within the chunk. The remaining upper right cells are hatched as masked.

Figure 3: The blockwise causal mask is what makes the cost structure possible. Because the image and text prefix never attends to the noisy action tokens, its keys and values do not depend on $\tau$ and stay valid across all 10 integration steps, while the action block attends bidirectionally within the chunk so every predicted timestep sees every other.

Property	Diffusion Policy (2023)	π0 (2024)
Generative process	Discrete-time DDPM, epsilon-prediction, cosine or squared-cosine schedule	Continuous-time conditional flow matching on the straight interpolant
Sampler steps	100 training steps, 10 DDIM steps at inference	10 forward Euler steps with fixed step 0.1
Denoiser	1D temporal U-Net with FiLM, or a small transformer variant	300M action expert sharing attention with PaliGemma 3B weights
Conditioning	ResNet image features plus proprioception, no language	Up to 3 camera views, natural-language instruction, state token
Chunk and rate	Predict 16, execute 8, typically about 10 Hz control	Predict 50 actions, one second of control at 50 Hz
Embodiment scope	One robot and task per trained policy	Cross-embodiment mixture, all vectors zero-padded to 18 dimensions
Dominant failure mode	No semantic generalization; the visual encoder is trained from a few hundred demos	Open-loop within a chunk, and prefill latency dominates the control budget

August 19, 2026

DL0148 Action Chunking Transformer (ACT)

Explain the Action Chunking Transformer (ACT). How does predicting sequences of future action vectors ( $k$ -step horizons) solve temporal inconsistency and compounding error drift?

Answer

ACT is the imitation-learning policy introduced with the ALOHA bimanual setup: a transformer encoder-decoder that takes the current multi-view images plus the 14-dimensional joint state and emits, in a single forward pass, a whole chunk of $k=100$ future joint-position targets rather than one action. It is trained as a conditional VAE (CVAE) with an $L_1$ reconstruction loss on the chunk plus a KL term on a small style latent $z$ , which lets one deterministic-at-test-time policy absorb the multimodality of human teleoperation instead of averaging it away. Chunking attacks drift arithmetically: at 50 Hz a 20-second task is 1,000 control steps but only $1000/100 = 10$ chunk decisions, so the number of places where the policy can leave the training distribution shrinks by a factor of $k$ and the classic $O(\epsilon T^2)$ behaviour-cloning bound falls to roughly $O(\epsilon T^2 / k)$ . Chunking also removes per-step temporal inconsistency, because the actions inside a chunk are generated jointly from one latent and one observation, so the policy cannot flip between two valid modes on consecutive 20 ms steps. To avoid a visible discontinuity when a new chunk starts, ACT still queries the network every step and blends the overlapping predictions with temporal ensembling.

(1) The Chunk Is The Output Unit: the policy models $\pi(a_{t:t+k-1} \mid o_t)$ instead of $\pi(a_t \mid o_t)$ , so $k$ correlated actions are predicted together as one object.
(2) Effective Horizon Divided By $k$ : the agent makes $H = \lceil T/k \rceil$ sequential decisions, and compounding error grows with the number of decisions, not with the number of motor commands.
(3) CVAE For Multimodality: a BERT-style encoder compresses the demonstrated chunk into a 32-dimensional latent $z$ during training and is discarded at test time, where $z = 0$ gives one clean decisive mode.
(4) Non-Markovian Demos Stop Being Fatal: a single-step policy standing at a human pause sees a bimodal target (hold still or move) and can freeze forever; a chunk that contains the pause and the following motion resolves the ambiguity.
(5) Temporal Ensembling For Smoothness: re-query every step and average the overlapping chunk predictions with weights $w_i = \exp(-m i)$ , which keeps reaction latency at one control step while removing chunk-boundary jumps.
(6) $L_1$ Over $L_2$ : the $L_1$ loss on absolute joint targets produces sharper, less smeared trajectories, which matters for millimetre-scale contact tasks.

Architecturally there is nothing exotic. Each of the four RGB streams (two wrist cameras, two static cameras at 480×640) goes through a ResNet-18 whose final feature map is flattened into roughly 300 tokens, the joint vector becomes one more token, and the style latent becomes one more; a 4-layer transformer encoder with width 512 mixes them, and a 7-layer decoder attends to that memory from $k$ fixed learned position embeddings, one per future step, so the $k \times 14$ output is produced non-autoregressively in about 0.01 s. The whole policy is roughly 80M parameters trained from scratch per task on about 50 demonstrations, which is the interesting part: the gain does not come from scale or pretraining, it comes from changing what a single prediction means.

Architecture diagram of ACT: four RGB cameras feed per-camera ResNet-18 backbones producing about 300 tokens each, a joint-position token joins them, a 4-layer transformer encoder mixes the tokens, a 7-layer decoder with k learned queries emits a k by 14 action chunk, and above the main lane a training-only CVAE encoder maps the demonstrated action sequence and joint state to a 32-dimensional style latent z that is injected into the encoder and set to zero at test time

Figure 1: ACT is a CVAE whose decoder is a chunk predictor. The dashed lane exists only during training: it sees the ground-truth action sequence and squeezes the demonstrator’s stylistic choice into $z \in \mathbb{R}^{32}$ . At test time $z$ is fixed to the prior mean, so the same network becomes a deterministic policy that outputs $k \times 14$ absolute joint targets in one pass.

Mathematical Formulation:
$\pi_\theta(\hat a_{t:t+k-1} \mid o_t, q_t, z)$
$\mathcal{L}_1 = \sum_{j=0}^{k-1} \| \hat a_{t+j} - a_{t+j} \|_1$
$\mathcal{L} = \mathcal{L}_1 + \beta \, D_{\mathrm{KL}}(q_\phi \, \| \, \mathcal{N}(0, I))$

Where:

$\hat a_{t:t+k-1}$ is the predicted action chunk, here $k=100$ absolute target joint configurations in $\mathbb{R}^{14}$ for a bimanual 6-DoF-plus-gripper arm pair.
$o_t$ are the four camera images at time $t$ and $q_t$ the measured joint positions, together forming the only observation the chunk is conditioned on.
$z \in \mathbb{R}^{32}$ is the style latent, sampled from the encoder posterior $q_\phi(z \mid a_{t:t+k-1}, q_t)$ in training and set to $z = 0$ at inference.
$j \in \{0, \ldots, k-1\}$ indexes positions inside the chunk, each produced by its own learned decoder query.
$\beta$ weights the KL term; a large $\beta$ collapses $z$ to noise while a small one lets the encoder leak the answer and hurts test-time behaviour.
The $L_1$ norm is deliberate: it penalises large joint errors less quadratically than $L_2$ and therefore averages competing modes less aggressively.

The drift argument is worth writing down. Standard behaviour cloning with per-step error $\epsilon$ suffers covariate shift: a mistake moves the robot to a state the demonstrations never covered, the next prediction is worse, and the regret bound is quadratic in the horizon. Chunking does not make the policy immune, it reduces how many times the loop is closed. With $H = T/k$ decision points, each contributing error $\epsilon$ that persists over the remaining decisions, and each decision covering $k$ timesteps of cost, the accumulated term scales as $\epsilon H^2 k = \epsilon T^2 / k$ . The same factor cuts inference calls, which is why a chunked policy can afford a much heavier network per decision.

Horizon Arithmetic At 50 Hz:
$T = 20 \times 50 = 1000$
$H = \lceil 1000 / 100 \rceil = 10$
$\mathcal{E}_{\mathrm{BC}} = O(\epsilon T^2)$
$\mathcal{E}_{\mathrm{chunk}} = O(\epsilon T^2 / k)$

Semi-log chart of a compounding-error bound scale against episode length in control steps at 50 Hz, with three parabolic curves for chunk size k equal to 1, 10, and 100, showing the k equals 100 curve two orders of magnitude below the single-step curve, annotated with the point where a 20 second task of 1000 steps requires only 10 chunk decisions

Figure 2: The bound is still quadratic in task length, so chunking does not abolish drift, it buys two orders of magnitude at $k=100$ . The practical reading is that a long-horizon task becomes as hard as a short one only if the chunk itself remains executable open-loop, which is exactly the assumption that breaks when the scene moves during the 2 seconds a chunk spans.

Naive open-loop execution of the chunk creates a new problem: every 2 seconds a fresh observation produces a fresh chunk that need not start where the previous one ended, and the robot jerks. ACT therefore runs the policy at the full control rate and aggregates. At timestep $t$ there are up to $k$ different chunks that predict an action for $t$ , one from each of the previous $k$ queries, and the executed command is their exponentially weighted mean with $m = 0.01$ , where index $i = 0$ is the oldest prediction. That value of $m$ keeps the weights nearly uniform (1.00, 0.99, 0.98, …), so new observations are folded in immediately while the average stays smooth; a larger $m$ leans on the oldest chunk and reacts more slowly.

Temporal Ensembling:
$w_i = \exp(-m i)$
$\bar a_t = \sum_i w_i \hat a_t^{(i)} / \sum_i w_i$

Timeline diagram with four horizontal rows of six cells each, one row per chunk issued at timesteps 0, 1, 2 and 3, staggered so that all four rows cover timestep 4; the four cells covering timestep 4 are highlighted and labelled with exponential weights 1.00, 0.99, 0.98 and 0.97, and an arrow leads down to a box stating that the executed action at timestep 4 is the weighted mean of the four overlapping predictions

Figure 3: Chunks overlap because the policy is queried every step, so each control command is a vote of up to $k$ predictions made from $k$ different observations. This is what keeps ACT closed-loop at 50 Hz while its prediction horizon stays 2 seconds long, and it costs one extra forward pass per step rather than any extra training.

Property	Single-step BC (k = 1)	Chunk, open-loop execution	ACT: chunk + temporal ensembling
Policy output	One 14-d joint target	100 x 14 targets, all executed	100 x 14 targets, blended with earlier chunks
Queries per 20 s task	1,000	10	1,000 (each cheap, 0.01 s)
Decision points that can drift	1,000, bound scales as eps T^2	10, bound scales as eps T^2 / k	10 committed motions, continuously re-averaged
Reaction to a disturbance	20 ms	Up to 2 s of stale commands	20 ms, damped by the weighted mean
Human pauses in demos	Bimodal target, policy can freeze	Pause is inside the chunk, resolved	Same, plus no boundary discontinuity
Dominant failure	Jitter and mode switching between steps	Jerk at chunk boundaries, blindness mid-chunk	Averaging across modes can blur a decisive motion

August 18, 2026

DL0147 Multi-Task VLA Negative Transfer

How do multi-task VLA policies, such as the RT-X models trained on the Open X-Embodiment dataset, avoid negative task transfer when trained across heterogeneous robot arms, grippers, and kinematic chains?

Answer

Negative transfer in a cross-embodiment VLA is not a mysterious optimization pathology; it is the direct consequence of the same observation-instruction pair mapping to different correct action vectors on different robots, so the per-embodiment gradients on shared weights partially cancel. The fix is to decide, layer by layer, what genuinely transfers and what does not: vision-language grounding transfers and stays fully shared, while action semantics do not and are pushed into a canonical action space plus embodiment-specific output parameters. In practice that means four moves applied together: canonicalize actions into a common frame with per-dataset percentile normalization, condition the policy on an explicit embodiment token so the target is identifiable, decode through a per-embodiment head or a padded action expert so incompatible outputs never share a final linear layer, and control the data mixture so one 100k-episode domain does not dominate the average gradient. Only after those are in place is gradient surgery (PCGrad, CAGrad) worth its cost, because most measured conflict in early cross-embodiment runs comes from unnormalized, mis-framed action labels rather than from a real task disagreement.

(1) Shared Trunk, Split Output: the ViT plus language backbone sees every dataset, and all embodiment-specific incompatibility is confined to the last block, which is where the conflicting gradients would otherwise meet.
(2) Canonical Action Space: relative end-effector deltas in a fixed camera or base frame, with a normalized gripper channel, make “move 2 cm right” mean the same thing on a Franka and a WidowX.
(3) Per-Dimension Percentile Normalization: mapping each action dimension by its 1st and 99th dataset percentiles into $[-1,1]$ removes the scale mismatch between a 0.08 m gripper stroke and a 0.03 m one.
(4) Explicit Embodiment Conditioning: an embodiment id, proprioceptive state, and control-rate token make the label identifiable; without them the policy is asked to regress a multi-modal target from an ambiguous input.
(5) Mixture Weighting And Capacity: per-domain sampling caps stop large datasets from monopolizing updates, and added capacity (more parameters, or modality/embodiment-aware routing) converts interference into specialization.
(6) Measure Conflict Before Fixing It: log pairwise gradient cosine similarity between embodiment groups and per-domain validation loss, then apply projection methods only to the pairs that are genuinely negative.

The heterogeneity is concrete rather than abstract. A 7-DoF arm commanded in end-effector deltas at 3 Hz, a 6-DoF arm commanded in joint velocities at 5 Hz, and a 14-DoF bimanual rig commanded in absolute joint positions at 50 Hz produce label vectors of different dimension, unit, frame, and scale. Stack them into one tensor and regress with one output layer, and that layer must fit a multi-modal conditional distribution whose modes are mutually contradictory, so the mean-seeking L1 or MSE solution is a blurred command that satisfies nobody. The RT-X study made the asymmetry visible: the small 35M-parameter policy gained substantially on data-poor domains while losing to the original single-domain policies on some data-rich ones, whereas the far larger vision-language-backboned variant did not, which is the classic capacity-limited interference signature. Action chunking makes the stakes higher still, because a head predicting a horizon of $H$ steps outputs $H \times d_a$ numbers whose meaning is embodiment-dependent at every position.

Architecture diagram with three input boxes for RGB observations, the language instruction, and an embodiment id plus proprioceptive spec, all feeding one tall shared vision-language trunk, which produces a shared latent that fans out to three separate action heads for a 7-DoF arm at 3 Hz, a 14-DoF bimanual rig at 50 Hz, and an 8-DoF mobile manipulator, each emitting an action vector of different dimension

Figure 1: Where heterogeneity is absorbed. Everything to the left of the latent is shared across every robot and is exactly the part that benefits from pooled data; everything to the right is embodiment-specific, so a 14-DoF joint-position label and a 7-DoF end-effector delta label never contend for the same output weights. The embodiment id enters as a token in the trunk and also selects the head.

Mathematical Formulation:
$z = f_{\theta}(o_{1:n}, \ell, c_e)$
$\hat{a} = h_{\phi_e}(z)$
$\mathcal{L}(\theta, \phi) = \sum_{e=1}^{E} w_e \mathcal{L}_e$
$g_e = \nabla_{\theta} \mathcal{L}_e$
$\cos(g_i, g_j) = \frac{g_i^{\top} g_j}{\lVert g_i \rVert \lVert g_j \rVert}$
$g_i' = g_i - \frac{g_i^{\top} g_j}{\lVert g_j \rVert^2} g_j$

Where:

$z$ is the shared latent produced by the trunk $f_{\theta}$ from the camera views $o_{1:n}$ , the instruction $\ell$ , and the embodiment context $c_e$ (robot id, proprioceptive state, control rate).
$h_{\phi_e}$ is the head for embodiment group $e$ , with its own output dimension $d_a^{(e)}$ and chunk horizon, so $\hat{a} \in \mathbb{R}^{H \times d_a^{(e)}}$ .
$w_e$ are the mixture weights over $E$ embodiment groups, set by capped sampling rather than raw episode counts, and $\mathcal{L}_e$ is the behaviour-cloning loss on group $e$ .
$g_e$ is that group’s gradient with respect to the shared parameters only; head parameters $\phi_e$ receive gradient from one group and cannot conflict by construction.
The cosine is the diagnostic: a negative value means the two groups disagree about the shared update, and its magnitude tells you how much of each step is being cancelled.
$g_i'$ is the PCGrad projection of $g_i$ onto the normal plane of $g_j$ , applied only when the cosine is negative and symmetrically for $g_j'$ .

Per-Dimension Action Normalization:
$\tilde{a}^{(d)} = 2 \frac{a^{(d)} - q_{1}^{(d)}}{q_{99}^{(d)} - q_{1}^{(d)}} - 1$

Percentiles rather than min-max are used because teleoperation logs contain jitter spikes that would otherwise compress the useful range into a few percent of the interval. Note also what normalization cannot repair: it aligns scales, not semantics. If one dataset logs joint velocities and another logs end-effector deltas, both normalized to $[-1,1]$ , the shared head still sees two incompatible meanings for the same slot, which is precisely the residual conflict that per-embodiment decoding removes.

Two-panel vector diagram in the plane. Left panel shows a bimanual gradient pointing up and a single-arm gradient pointing down-left with an angle of about 117 degrees between them, and their sum drawn as a short vector, illustrating cancellation. Right panel shows the same two gradients as faint dashed arrows plus their PCGrad projections drawn solid, whose sum is a longer vector that still descends both losses

Figure 2: Negative transfer, geometrically. When two embodiment groups have gradient cosine near -0.45, their raw sum is shorter than either gradient, so the shared trunk barely moves while both per-domain losses stall. After projecting each gradient onto the other’s normal plane, the combined step is longer and its inner product with both original gradients stays positive, which is the formal statement of “descends both tasks”.

Choosing where the embodiment-specific parameters live is the main design decision, and three families are in use. A single padded head defines one action vector of maximum width and zero-pads unused dimensions, with the loss masked over the padding, which keeps the model monolithic and transfers well when the padded dimensions are physically comparable. Per-embodiment heads require no cross-robot alignment at all and are the most robust to genuinely different control interfaces, at the cost of one head per group and no head for an unseen robot. Latent action spaces learn a discrete or continuous code from video by inverse dynamics, train the policy in that code, and attach a small decoder per robot, which unlocks human video and action-free data but adds an entire quantization stage that can lose fine-grained precision.

Property	Single padded head	Per-embodiment heads	Latent action space
How heterogeneity is handled	One max-width action vector, unused slots zero-padded and masked in the loss	One output module per action space, no alignment across robots needed	Policy predicts a robot-agnostic code, small per-robot decoder maps it to motors
Negative-transfer risk	Moderate: slots must be semantically comparable or the head averages modes	Lowest at the output layer, residual conflict only in the shared trunk	Low if the code is truly embodiment-invariant, high if it leaks robot identity
Unseen robot at test time	Works if its dimensions fit the padded layout	Needs a new head plus a short fine-tune on target data	Needs only a cheap decoder, which is the main selling point
Extra cost	Wasted output width and careful mask bookkeeping	Linear growth in heads, and per-group data must be sufficient	A separate latent-action pretraining stage and possible precision loss
Representative systems	Flow-matching action experts with padded dimensions, unified-action diffusion policies	Readout-head generalist policies and cross-embodied transformers spanning manipulation and navigation	Latent-action pretraining from human and web video

August 18, 2026

DL0146 VLA Continuous Action Representation

How does a transformer decoder represent multi-modal continuous robot actions (e.g., 7-DoF arm pose, gripper state, base velocity) alongside language tokens, as in Google’s RT-2 and the OpenVLA models?

Answer

The decoder never sees a float. Heterogeneous degrees of freedom are first concatenated into one fixed-width vector (end-effector deltas in metres and radians, a gripper command, base velocities, and a discrete mode flag that says which sub-system is being driven this step), then each dimension is independently normalized by its own 1st and 99th percentile statistics so that metres, radians, and a binary gripper all live on $[-1, 1]$ . In the RT-2 and OpenVLA line each normalized dimension is uniformly discretized into 256 bins and the bin index is mapped onto reserved token ids, typically the 256 least-frequently-used entries of the existing SentencePiece or Llama vocabulary, so an action literally becomes a short string of tokens appended to the same causal sequence that holds the image patches and the instruction. Nothing about the architecture changes: the same causal self-attention, the same softmax, and the same cross-entropy loss that predict the next sub-word now predict the next joint delta, which is exactly what lets a VLA inherit internet-scale pretraining. The cost is a quantization floor plus one sequential decoder step per degree of freedom, which is why newer systems keep the shared prefix but replace the discrete head with a continuous action expert that emits a whole chunk of future actions through diffusion or flow matching in a single pass.

(1) One Flat Vector, Fixed Slots: arm, gripper, and base share a single $D$ -dimensional action vector with fixed slot semantics, and unused slots are zero-padded so one model can serve several embodiments.
(2) A Mode Dimension, Not Separate Heads: RT-1 carries an explicit mode variable that switches between controlling the arm, controlling the base, and terminating the episode, so mutually exclusive sub-spaces are disambiguated inside the same vector.
(3) Per-Dimension Quantile Normalization: statistics are computed per dimension and per dataset, using percentiles rather than min and max, because a handful of teleoperation jerks would otherwise consume most of the dynamic range.
(4) 256 Bins Onto Reserved Vocabulary Ids: the action alphabet is carved out of the language vocabulary, so no new embedding table, output head, or loss function is introduced.
(5) Autoregressive Factorization Across DoFs: the chain rule over dimensions lets the yaw bin depend on the already-emitted $x$ and $y$ bins, which is what keeps the joint action coherent rather than per-axis independent.
(6) Action Chunking And Continuous Heads: predicting $H$ future steps at once fights compounding error and raises the effective control rate, and a flow-matching expert removes the bin grid entirely at the price of extra weights and solver steps.

Architecture diagram showing camera frames, instruction text, and proprioception each tokenized into a shared prefix sequence feeding a decoder-only transformer, which branches into a discrete head that emits 256-bin indices as reserved vocabulary tokens and a continuous action expert that emits an H by D chunk by flow matching, both producing the same 11-dimensional action vector of six arm deltas, one gripper command, three base velocities, and one mode flag

Figure 1: The prefix is modality-agnostic; only the head differs. The discrete head reuses the language softmax and spends one sequential decoder step per degree of freedom, while the continuous action expert attends to the same prefix but emits an entire $H \times D$ chunk of real numbers. Both write into the identical fixed-slot action vector, including the mode flag that selects arm, base, or episode termination.

Normalization is where most of the real accuracy is won or lost, and it is the step engineers most often get wrong. Raw teleoperation logs are heavy-tailed: a few reset motions or dropped-controller frames produce end-effector deltas an order of magnitude larger than anything the policy needs at test time. If the bin edges are set by the observed minimum and maximum, those rare samples stretch the grid so far that the entire working range of the robot collapses into a few dozen of the 256 available levels, and the policy inherits a positional resolution floor of several millimetres that no amount of extra data can fix. Clipping to the 1st and 99th percentile before binning, as OpenVLA does, spends the full alphabet on the range the robot actually operates in and pushes the residual error below the mechanical repeatability of most arms. The same argument applies to the gripper, which is near-binary in the data, so it is usually thresholded into open or closed rather than treated as a smooth continuous axis.

Two histograms of the same end-effector delta-z distribution. The left panel bins over the raw min-max range, where rare large outliers stretch the grid and only a small fraction of the 256 bins are ever occupied. The right panel clips to the 1st and 99th percentile before binning, and nearly all 256 bins are occupied with a much smaller bin width in millimetres.

Figure 2: The same action dimension, two bin grids. Min-max binning lets rare teleoperation outliers dictate the grid, so most of the 256 levels are never used and the effective step size is coarse. Quantile binning throws away 2% of the samples and buys back an order of magnitude in resolution per bin, which is why percentile statistics are standard in production VLA data pipelines.

Mathematical Formulation:
$a_t = (a^{\mathrm{arm}}_t, a^{\mathrm{grip}}_t, a^{\mathrm{base}}_t, m_t)$
$\tilde{a}_i = \mathrm{clip}(a_i, q^{i}_{1}, q^{i}_{99})$
$u_i = 2 (\tilde{a}_i - q^{i}_{1}) / (q^{i}_{99} - q^{i}_{1}) - 1$
$b_i = \lfloor 255 (u_i + 1) / 2 \rfloor$
$\mathrm{id}_i = V - 1 - b_i$
$p(a_t \mid c) = \prod_{i=1}^{D} p(b_i \mid c, b_{1:i-1})$

Where:

$a_t$ is the action at control step $t$ , built from an arm delta pose (three translations and three rotations), a gripper command, a base velocity triple, and a discrete mode flag $m_t$ .
$a_i$ is the raw value of dimension $i$ and $\tilde{a}_i$ its clipped version; $q^{i}_{1}$ and $q^{i}_{99}$ are the per-dimension percentiles estimated on the training corpus.
$u_i \in [-1, 1]$ is the normalized value and $b_i \in \{0, \ldots, 255\}$ the bin index, so the grid holds 256 levels per dimension.
$V$ is the language vocabulary size, and $\mathrm{id}_i$ overwrites the 256 least-used token ids at the tail of that vocabulary.
$c$ is the prefix (image patch tokens, instruction sub-words, optional proprioception) and $i \in \{1, \ldots, D\}$ indexes the action dimensions in a fixed canonical order.
The product is the autoregressive factorization: each bin is conditioned on all previously emitted bins of the same action, so the joint distribution is not a product of independent marginals.

Resolution And Token Budget:
$\Delta_i = (q^{i}_{99} - q^{i}_{1}) / 256$
$\Delta = 0.1\ \mathrm{m} / 256 \approx 0.39\ \mathrm{mm}$
$N = H \times D = 8 \times 11 = 88$

Those two lines set the entire engineering trade-off. A translation axis clipped to $\pm 0.05$ m resolves to about 0.39 mm per bin, which is fine for pick and place and marginal for connector insertion, but an eight-step chunk of an 11-dimensional action costs 88 sequential decoder passes, which caps a 7B VLA at roughly 3 to 10 Hz on a single accelerator. The alternative keeps the shared prefix and attaches a small action expert whose tokens are continuous rather than discrete: starting from noise, it integrates a learned velocity field for a handful of solver steps and emits the whole chunk at once, which is how flow-matching policies reach 50 Hz dexterous control. Crucially, both the categorical head and the flow head are distributional, and that matters more than resolution: when two different behaviours are equally valid, a plain L2 or Gaussian regression head averages them into an action that belongs to neither mode.

$A_t = (a_t, a_{t+1}, \ldots, a_{t+H-1})$
$A^{\tau + \delta} = A^{\tau} + \delta\, v_{\theta}(A^{\tau}, \tau, c)$

Left panel shows a top-down workspace with an obstacle and two valid detour trajectories, one above and one below, plus a straight-line mean trajectory that drives into the obstacle. Right panel shows a bimodal action density with a 256-way softmax step approximation covering both peaks and a red dashed vertical line marking the L2 optimum located in the empty valley between the modes.

Figure 3: Why the head must be distributional. Two demonstrators route around the same obstacle in opposite directions, so the conditional action distribution is bimodal; a regression head minimizing L2 lands on the mean, which is a collision. A 256-way categorical over bins, or a diffusion or flow head over the chunk, keeps both peaks and samples one of them.

Property	Discrete bins as vocabulary tokens	Diffusion / flow action expert	Direct continuous regression
Output form	256-way categorical per DoF over reserved token ids	Real-valued chunk of shape H by D, denoised from noise	One real vector, the predicted mean action
Training objective	Cross-entropy, identical to language pretraining	Flow-matching or denoising regression on noised chunks	L2 or L1 on the action vector
Cost per chunk	H times D sequential decoder steps (88 at H=8, D=11)	One prefix pass plus about 10 solver steps of a small expert	One forward pass
Multimodal actions	Yes, per-DoF peaks are tied together by the chain rule	Yes, jointly over the whole chunk	No, collapses to the mean
Resolution	Bin width, about 0.39 mm on a 0.1 m range	Continuous, limited only by data noise	Continuous
Dominant failure mode	Quantization floor plus slow autoregressive decoding	Extra weights and solver steps, harder to co-train with text	Mode averaging produces an invalid middle action

August 18, 2026