DL0152 VLA Frequency Gap Bridge

You never let the VLA command joints directly. The frequency gap is closed by a three-tier cascade in which each tier runs at its own clock: the VLA emits an action chunk covering hundreds of milliseconds of future motion instead of a single next action, a mid-rate head or buffer replays that chunk at 20 to 100 Hz, and a real-time layer upsamples the waypoints to the servo rate and closes an impedance or torque loop at 1 kHz against live encoder feedback. Chunking alone is not enough, because a blocking call stalls the controller for the whole inference window, so the chunk is computed asynchronously while the previous one is still executing, and the first few actions of the new chunk are frozen or blended to match what the robot has already committed to. Between waypoints you interpolate with a cubic or minimum-jerk segment rather than holding the last value, since a zero-order hold turns every chunk step into a velocity impulse the drives cannot follow. The real-time layer also owns safety: a watchdog that decays to gravity compensation when the buffer underruns, plus joint limits and torque saturation that the neural policy never sees.

(1) Chunks, Not Single Actions: the policy predicts future actions per forward pass, converting a 3 to 10 Hz decision rate into a continuous 20 to 100 Hz stream of targets.
(2) Horizon Must Cover Latency: the chunk has to be long enough to keep the buffer fed through one full inference plus transport delay, otherwise execution stutters at every boundary.
(3) Asynchronous Inference: issue the next forward pass while the current chunk still has actions left, so compute overlaps execution instead of interrupting it.
(4) Frozen Prefix And Blending: the first actions of a fresh chunk are already stale on arrival, so they are discarded or soft-constrained to the committed trajectory to avoid a jump at the splice.
(5) Interpolate, Never Zero-Order Hold: a spline between waypoints spreads each step over the controller ticks in between and keeps commanded velocity and acceleration bounded.
(6) The 1 kHz Layer Owns Safety: impedance gains, torque limits, and a buffer-underrun watchdog run on a real-time thread that never waits on a GPU.

Figure 1: Three clocks on one command path. Each tier only has to meet the deadline of the tier below it, so the 200 ms VLA period never appears as a 200 ms hole in the torque loop. Only the bottom tier is hard real time, and it is the only tier that reads encoders at 1 kHz.

The scheduling detail is what separates a demo from a deployed system. In the naive loop you observe, block on the network and the GPU for 100 to 300 ms, then execute the chunk, which means the controller spends a large fraction of every cycle replaying a stale target or holding still, and the robot visibly pauses at each boundary. Running inference asynchronously removes the hole but introduces a second problem: the chunk that arrives at time was conditioned on the observation from , so its early actions describe a state the robot has already left. Real-time chunking handles this by treating the overlap as an inpainting constraint, keeping the first actions pinned to the trajectory already in flight and letting the sampler adjust only the free tail. The cheaper approximation used by ACT is temporal ensembling: keep every overlapping prediction for the current timestep and average them with weights , which smooths the splice but adds no latency compensation and biases the command toward older observations.

Figure 2: With a 130 ms forward pass and a 200 ms chunk, blocking inference leaves the controller starved for 130 ms out of every 330, roughly 39% dead time. Overlapping the next forward pass with the current execution removes the gaps entirely, and the price is that every chunk acts on an observation that is one cycle old, which is exactly what the frozen prefix compensates for.

Mathematical Formulation:

Where:

• is the wall-clock horizon a single chunk covers, is the number of actions in the chunk, and is the rate at which those actions are consumed.
• is the end-to-end delay from shutter to first usable action, covering encoding, network transport, and the forward pass; is the resulting number of stale leading actions.
• is the policy replan rate, so is how many actions are consumed per replan and the third relation is the no-underrun condition.
• is the servo rate and the upsampling ratio, the number of interpolated setpoints emitted between two consecutive policy waypoints.
• is the joint torque, and the interpolated position and velocity setpoints, and the measured state, and the gravity term.
• and set the mechanical impedance; low gains make the arm compliant and forgiving of a slightly wrong setpoint, high gains make it track hard and punish every command discontinuity.

Budget For A 5 Hz VLA On A 1 kHz Arm:

With a 130 ms latency, a 50 Hz action rate, and a replan every 200 ms, the chunk needs at least 17 actions, and shipping (a 1.0 s horizon) buys margin for a GPU hiccup or a dropped packet. The controller then produces 20 interpolated setpoints per waypoint, so the neural policy is responsible for shape and the real-time layer for smoothness. Note that a longer horizon is not free: everything past the next replan is open-loop motion, so the chunk length trades buffer robustness against reaction time to disturbances, and only the first actions of a 50-action chunk are normally executed at all.

Figure 3: The same intended motion, three command paths. A zero-order hold asks for a finite position jump inside one 1 ms tick, so the implied velocity sits two orders of magnitude above the joint limit and the drive answers with a torque spike. Both interpolated paths stay under the limit, but only the 50 Hz waypoints actually reproduce the reference shape; upsampling a 5 Hz command smooths the command at the cost of cutting the corners of the trajectory.

Property	Blocking chunk + hold	Overlapping chunks + temporal ensembling	Async chunking + frozen prefix
Controller starvation	One dead window per cycle, about 39% duty loss at 130 ms latency	None if the ensemble buffer stays full	None, inference always overlaps execution
Boundary smoothness	Jump whenever the new chunk disagrees with the held target	Smooth, the exponential average filters the disagreement	Smooth by construction, the prefix is pinned to committed actions
Latency compensation	None, the whole chunk is stale by	None, and averaging biases toward older observations	Explicit, the first actions are skipped or constrained
Extra cost	Cheapest, one forward pass per executed chunk	Keeps several chunks in memory, needs a weighting hyperparameter	Needs a client-server split, a chunk buffer, and guided sampling
Reasonable use	Quasi-static pick and place, teleop replay, offline evaluation	On-board policies with low, stable latency	Remote or large models, dynamic tasks, anything with jittery latency