Flow Control

Flow Control

Steering Vision–Language–Action Models with Simple Real-Time Inputs

Jonathan C. Kao1,2,3, Jason Chan2, Andy Wang2
1Electrical & Computer Engineering   2Computer Science   3Neurobiology  ·  UCLA
Conference on Robot Learning (CoRL) 2026
arXiv OpenReview Summary Video More from our lab

Left: Autonomous zero-shot π0.5-DROID makes errors when stacking the cups (shown at 4× speed). Right: With flow control, users give arrow key inputs in real-time that help steer zero-shot π0.5-DROID to successfully stack the cups (real time, 1×). Flow control can be used out-of-the-box (no model fine-tuning) with VLAs that use a flow matching action expert.

Overview

Flow control steers a vision-language-action (VLA) model in real time using generic inputs, such as a keyboard. It works out-of-the-box on a frozen policy, without any retraining or fine-tuning. The VLA moves crude user input (like keypresses) toward the action distribution it learned during training, so the generated actions reflect user intent while staying close to the VLA's own actions.

No retrainingOut-of-the-boxSteers a frozen flow matching VLA.
High conformityClose to the VLAActions stay close to the VLA's own actions.
High fidelityFollows intentCrude inputs reliably convey the user's goal.

We freeze a VLA (here, π0.5) with a flow matching action expert and write the user's input into the flow's initial condition (IC). Because the flow is integrated with a deterministic sampler, information about the IC carries through to the generated action.

Flow control overview diagram
Overview. A frozen π0.5 VLA. Flow control (red) injects user inputs into the flow matching action expert to steer the output action chunk.
Stochastic versus deterministic samplers
Why a deterministic sampler. A stochastic sampler, such as the diffusion SDE, injects noise at each time step and loses information about the initial condition. A deterministic sampler, such as the flow matching ODE, is invertible, so a user nudge to the initial condition carries through to the final action.

Results

RESULT 1

Modifying the flow's initial condition steers the policy.

On an ambiguous two-block task (“put the block in the hole”), π0.5 autonomously picks the right block 85% of the time. Setting joint 1's initial condition redirects it to the left block on nearly 100% of trials, and the pick, transport, and place steps still succeed nearly 100% of the time. We later demonstrate flow control on all joints using more complex tasks.

Two-block task: setting joint-1's initial condition steers block selection without hurting pick-and-place
fig 3 · steering via the initial condition
RESULT 2

Flow control actions stay close to the VLA's action distribution.

When the action distribution is multimodal (which block to grab), the input selects the mode the user wants. When the task is unambiguous, the flow matching action expert transforms even discordant inputs into actions that still complete the task. What if the user wanted to do something discordant with a unimodal action distribution? See our Appendix D.

Flow fields showing steering acts in multimodal states but is absorbed in unimodal ones
fig 4 · steering in multimodal and unimodal settings
RESULT 3

Users steer the VLA to much higher success, faster.

In a study with 16 participants, keyboard steering lifts Marker-in-Bowl success from 53.3% → 99.4% (correcting the VLA's language-following errors) and Cup-Stacking from 48.0% → 87.7%, while cutting cup-stacking time 30% (50.3 s → 35.2 s).

User study: Marker-in-Bowl and Cup-Stacking tasks, a correction, and completion-time distributions
fig 8 · user study (n = 16)
RESULT 4

Flow control data makes the autonomous policy better.

Because flow control actions stay close to the VLA's action distribution, the trajectories, with their corrections, make good training data. Fine-tuning on 60 successful flow control trials (π0.5-FC) raises autonomous cup-stacking success from 48% → 100% (25 trials) and lowers the median trial time to 22 s.

Overview video

Citation

@inproceedings{kao2026flowcontrol,
  title     = {Flow Control: Steering Vision-Language-Action Models
               with Simple Real-Time Inputs},
  author    = {Kao, Jonathan C. and Chan, Jason and Wang, Andy},
  booktitle = {Proceedings of the 10th Conference on Robot Learning (CoRL)},
  year      = {2026},
  url       = {https://arxiv.org/abs/2606.10180}
}