DL0147 Multi-Task VLA Negative Transfer

Negative transfer in a cross-embodiment VLA is not a mysterious optimization pathology; it is the direct consequence of the same observation-instruction pair mapping to different correct action vectors on different robots, so the per-embodiment gradients on shared weights partially cancel. The fix is to decide, layer by layer, what genuinely transfers and what does not: vision-language grounding transfers and stays fully shared, while action semantics do not and are pushed into a canonical action space plus embodiment-specific output parameters. In practice that means four moves applied together: canonicalize actions into a common frame with per-dataset percentile normalization, condition the policy on an explicit embodiment token so the target is identifiable, decode through a per-embodiment head or a padded action expert so incompatible outputs never share a final linear layer, and control the data mixture so one 100k-episode domain does not dominate the average gradient. Only after those are in place is gradient surgery (PCGrad, CAGrad) worth its cost, because most measured conflict in early cross-embodiment runs comes from unnormalized, mis-framed action labels rather than from a real task disagreement.

(1) Shared Trunk, Split Output: the ViT plus language backbone sees every dataset, and all embodiment-specific incompatibility is confined to the last block, which is where the conflicting gradients would otherwise meet.
(2) Canonical Action Space: relative end-effector deltas in a fixed camera or base frame, with a normalized gripper channel, make “move 2 cm right” mean the same thing on a Franka and a WidowX.
(3) Per-Dimension Percentile Normalization: mapping each action dimension by its 1st and 99th dataset percentiles into removes the scale mismatch between a 0.08 m gripper stroke and a 0.03 m one.
(4) Explicit Embodiment Conditioning: an embodiment id, proprioceptive state, and control-rate token make the label identifiable; without them the policy is asked to regress a multi-modal target from an ambiguous input.
(5) Mixture Weighting And Capacity: per-domain sampling caps stop large datasets from monopolizing updates, and added capacity (more parameters, or modality/embodiment-aware routing) converts interference into specialization.
(6) Measure Conflict Before Fixing It: log pairwise gradient cosine similarity between embodiment groups and per-domain validation loss, then apply projection methods only to the pairs that are genuinely negative.

The heterogeneity is concrete rather than abstract. A 7-DoF arm commanded in end-effector deltas at 3 Hz, a 6-DoF arm commanded in joint velocities at 5 Hz, and a 14-DoF bimanual rig commanded in absolute joint positions at 50 Hz produce label vectors of different dimension, unit, frame, and scale. Stack them into one tensor and regress with one output layer, and that layer must fit a multi-modal conditional distribution whose modes are mutually contradictory, so the mean-seeking L1 or MSE solution is a blurred command that satisfies nobody. The RT-X study made the asymmetry visible: the small 35M-parameter policy gained substantially on data-poor domains while losing to the original single-domain policies on some data-rich ones, whereas the far larger vision-language-backboned variant did not, which is the classic capacity-limited interference signature. Action chunking makes the stakes higher still, because a head predicting a horizon of steps outputs numbers whose meaning is embodiment-dependent at every position.

Figure 1: Where heterogeneity is absorbed. Everything to the left of the latent is shared across every robot and is exactly the part that benefits from pooled data; everything to the right is embodiment-specific, so a 14-DoF joint-position label and a 7-DoF end-effector delta label never contend for the same output weights. The embodiment id enters as a token in the trunk and also selects the head.

Mathematical Formulation:

Where:

• is the shared latent produced by the trunk from the camera views , the instruction , and the embodiment context (robot id, proprioceptive state, control rate).
• is the head for embodiment group , with its own output dimension and chunk horizon, so .
• are the mixture weights over embodiment groups, set by capped sampling rather than raw episode counts, and is the behaviour-cloning loss on group .
• is that group’s gradient with respect to the shared parameters only; head parameters receive gradient from one group and cannot conflict by construction.
• The cosine is the diagnostic: a negative value means the two groups disagree about the shared update, and its magnitude tells you how much of each step is being cancelled.
• is the PCGrad projection of onto the normal plane of , applied only when the cosine is negative and symmetrically for .

Per-Dimension Action Normalization:

Percentiles rather than min-max are used because teleoperation logs contain jitter spikes that would otherwise compress the useful range into a few percent of the interval. Note also what normalization cannot repair: it aligns scales, not semantics. If one dataset logs joint velocities and another logs end-effector deltas, both normalized to , the shared head still sees two incompatible meanings for the same slot, which is precisely the residual conflict that per-embodiment decoding removes.

Figure 2: Negative transfer, geometrically. When two embodiment groups have gradient cosine near -0.45, their raw sum is shorter than either gradient, so the shared trunk barely moves while both per-domain losses stall. After projecting each gradient onto the other’s normal plane, the combined step is longer and its inner product with both original gradients stays positive, which is the formal statement of “descends both tasks”.

Choosing where the embodiment-specific parameters live is the main design decision, and three families are in use. A single padded head defines one action vector of maximum width and zero-pads unused dimensions, with the loss masked over the padding, which keeps the model monolithic and transfers well when the padded dimensions are physically comparable. Per-embodiment heads require no cross-robot alignment at all and are the most robust to genuinely different control interfaces, at the cost of one head per group and no head for an unseen robot. Latent action spaces learn a discrete or continuous code from video by inverse dynamics, train the policy in that code, and attach a small decoder per robot, which unlocks human video and action-free data but adds an entire quantization stage that can lose fine-grained precision.