Chapter 3: How Does a VLA Produce Actions? — Model Families, Training Objectives, and Policy Heads

Overview

A vision-language-action model (VLA) is a policy family that consumes images and an instruction and proposes robot actions. That compact definition is useful, but “a large vision-language backbone with an action output attached” is not enough to choose or deploy one. How camera history and robot state enter the network, which data and objectives train each stage, how many future actions are predicted, how many are executed, and when the policy closes the loop on a new observation can all change the result.

S11 contributes a verified physical cell, sensor and clock identities, coordinate frames, calibration, controllers, collision and force limits, watchdogs, protective stops, the E-stop, and accountable human authority. S12 contributes versioned task and skill APIs, the proposal → projected → sent → accepted/executed trace, feasibility projection, classical and BC/ACT/generative baselines, independent evaluation, and a complete rollback tuple. This chapter asks a new question without weakening either inheritance: which VLA configuration creates which kind of action candidate, and how can that candidate be compared fairly and promoted only into a bounded assembly experiment?

After reading this chapter, you should be able to... - decompose a VLA into encoders, multimodal fusion, a policy backbone, an action head, and an execution adapter, with explicit authorities and prohibitions; - distinguish pre-training, co-training, and post-training by data, losses, trainable parameters, and receipts; - compare autoregressive, chunked, diffusion, and flow heads on common action, horizon, closed-loop, and latency axes; - explain what continues directly from S12 BC, ACT, and generative policies and what the VLA conditioning stack adds; - separate free-space proposals from contact authority in the running tabletop assembly task; - record action age, rejection, intervention, force events, recovery, and rollback artifacts rather than reporting success alone.

The experimental question is concrete: with one single-arm, parallel-gripper cell, a fixed assembly-parts distribution, identical observation and action semantics, and the same downstream controllers, does language/vision pre-training plus a VLA action head improve paraphrased-instruction or shifted-layout performance over the simpler S12 policies enough to pay for latency, rejection, contact failures, and operational cost?

1. Read a VLA as a chain of responsibilities

The first useful decomposition has five layers: encoders, multimodal fusion, policy backbone, action head, and execution adapter. A scene or wrist encoder converts images into features; a language encoder converts the instruction into context. Fusion lets image patches, language tokens, proprioceptive state, and observation history interact. The policy backbone produces the context needed for action generation. The head maps that context to an explicit robot action schema. The adapter checks frame, units, timestamps, shape, and validity before passing a candidate to projection and control.

A defensible VLA comparison separates the backbone, training data, fusion design, objectives, action head, action horizon, and closed-loop execution ^{[2] [10] [14]}. Published comparisons often change several of these factors together, so a gain cannot automatically be attributed to the head. RT-2’s web/robot co-training and action tokens, Octo’s flexible observations and diffusion decoder, and controlled studies of discrete versus continuous output structures answer different questions. They do not form one matched leaderboard.

“End-to-end learned” describes a training boundary, not the end of the safety authority chain. A model may emit an end-effector pose, joint delta, gripper state, or skill call, but the result remains a time-bounded proposal. Collision checking, joint/velocity/force constraints, stale-command rejection, protective stop, and responsible-human approval remain independent. A controller value stored next to a checkpoint is not thereby an approved cell configuration.

1.1 Freeze input and output contracts before selecting architecture

The input contract identifies each camera, resolution and exposure, image event and arrival timestamps, robot-state timestamp, frame tree, calibration hash, exact instruction, task phase, and outcome of the previously executed action. A model trained with three observations is not being tested unchanged if deployment provides one. Quietly mirroring a wrist image or substituting a different normalization is also a model-system change. A required missing modality is reported as missing; a zero vector must not hide it.

The output contract is stricter. Suppose a_t=[\Delta x,\Delta y,\Delta z,\Delta r_x,\Delta r_y,\Delta r_z,g]. The receipt must state the reference frame, translation units, rotation representation and composition order, gripper sign, control period, and saturation. If the head predicts K actions and executes H_e, prediction and execution horizons are separate fields. If the output is a skill call, the contract specifies the skill version, argument schema, preconditions, completion predicate, timeout, fallback, and permitted caller.

These fields prevent an architecture comparison from becoming an adapter comparison. A token head evaluated after a forgiving projection and a continuous head evaluated without it do not face the same task. Similarly, a model allowed to retry until success does not share the denominator of a one-attempt baseline. Proposal, projected proposal, sent command, and accepted/executed command need distinct identifiers in every method.

2. Audit information paths and time alignment before fusion labels

Early fusion can place image, language, and state tokens in one sequence so that they interact throughout the backbone. Late fusion can preserve a pre-trained vision-language model while assigning robot state and high-rate action processing to a specialized module. Cross-attention and separate policy heads offer further separations between semantic representations and robot-specific computation. None is superior by name. The assembly-relevant question is whether information needed to ground a part’s color and function, and information needed to reduce pre-insertion pose error, actually reach the head at the necessary time resolution.

RT-2 casts robot actions as text-like tokens and co-fine-tunes on web and robot data, creating one route from semantic pre-training to closed-loop robot control ^[2]. OpenVLA describes projecting multi-resolution visual features into a language-model input space ^[11]. Octo emphasizes support for language or goal images, observation history, multiple camera/proprioceptive configurations, and diffusion-decoded chunks ^[10]. Those distinctions are audit fields about preserved information and training stage, not a ranking of fusion mechanisms.

In tabletop assembly, semantic and geometric responsibilities should be tested separately. For “insert the blue spacer into the right socket,” identifying blue, spacer, and the relational meaning of right can be an upper-model responsibility. The socket center, feasible approach, collision-free transport, and permissible contact force still belong to calibrated geometry, planning, and contact control. CLIPort and PerAct provide useful design points for semantic/spatial pathways and discretized 3D actions, respectively, but planar picks or voxelized poses are not universal evidence for contact insertion ^{[6] [7]}.

The same separation exposes shortcut learning. Swap colors while preserving layout; paraphrase the instruction without moving objects; move the target while keeping the sentence fixed; hide a required part; and issue a prohibited or internally contradictory request. A policy that succeeds only under the original co-occurrence pattern has not established instruction grounding. A policy that names the correct target but proposes an infeasible approach has a different failure, which should be routed to the geometry and action interface rather than merged into “VLA failure.”

3. Define training stages by data roles and changed parameters

Pre-training builds reusable representations or a policy initialization from broad vision-language or multi-robot data. Co-training mixes web/language objectives and robot-action objectives in one training process, typically to retain both semantic and control competence. Post-training draws on high-quality target-robot or target-task demonstrations, preferences, corrections, or specialized data to shape deployment behavior. Papers use these terms differently, so the durable record is the dataset set, sampling weights, loss functions, frozen/trainable parameters, and start/end checkpoint hashes.

This separation matters for contamination and causal attribution. Web pre-training may help connect a novel noun to an object, but it supplies neither the target cell’s coordinate frame nor gripper semantics. Multi-robot pre-training exposes varied observations and actions, but it cannot remove a faulty target embodiment transform. A post-trained model’s improvement can come from its backbone, more target data, a new head, longer history, or selection of an easier evaluation checkpoint. The study must distinguish those explanations.

A minimum training receipt records:

1. every dataset name, version, license, mixture weight, and duplicate/overlap audit;
2. each source embodiment, observation/action schema, control mode, and time resolution;
3. stage-specific losses, weights, frozen/unfrozen modules, and checkpoint hashes;
4. the split used to compute action normalization and the mapping for non-target robots;
5. direct and semantic overlap between evaluation tasks, objects, instructions, and training data;
6. whether failures, interventions, corrections, and recoveries occur in the mixture;
7. the deployment adapter, device, numeric precision, dependencies, and full rollback tuple.

Octo reports imitation learning from a large robot-trajectory mixture and notes limits of training on optimal demonstrations for broader behavior and recovery ^[10]. RoboCat’s self-generated-data loop still depends on human demonstrations and filtering; repeatedly adding model-filtered successes can obscure the failure distribution and evaluation independence ^[9]. Dataset size is therefore not a proxy for failure detection, retreat, or human handoff.

Offline demonstration studies offer another warning. Robomimic shows that algorithm, observation, dataset quality, and training choices interact under fixed benchmarks ^[18]. A VLA does not escape those interactions because its backbone was pre-trained. The local report should keep a target-only policy and an identical-data VLA adaptation as separate baselines, then disclose any additional pre-training mixture rather than treating it as free information.

4. Compare action heads through one execution contract

An autoregressive head predicts the next action symbol or dimension conditioned on earlier outputs. An action-chunk head predicts several future steps together, giving temporal context and trajectory coherence. A diffusion head iteratively transforms noise into a conditional action sequence and can represent several valid modes. A flow-matching head learns a continuous vector field that transports a base distribution toward action sequences. These labels describe output distributions and computation; they do not specify which physical control rate or safety authority a model owns.

The continuity with S12 is direct. A continuous regression head behind a language/vision backbone is behavior cloning. A chunked head with overlapping temporal aggregation shares ACT’s central design choices ^[4]. A diffusion head connects the S12 generative policy to broader conditioning ^[5]. Energy-based imitation also represents multimodal actions by scoring candidates, but candidate optimization cost and out-of-support calibration need separate tests ^[3]. The distinctive VLA addition is broad language, web, and multi-embodiment conditioning—not an exemption from imitation-learning assumptions or the inherited execution contract.

Prediction and execution horizons need not match. A longer K may stabilize free-space transport, while contact entry should shorten H_e and reobserve on force or tactile events. Temporal ensembling can smooth overlapping chunks but can also average across an abrupt contact-state transition. Diffusion and flow heads may create multiple candidates, yet a selector that sees evaluation outcomes leaks future information. If feasibility projection heavily modifies most candidates, the generated distribution’s apparent benefit may disappear downstream; projection delta and rejection must be reported.

4.1 Put flow and tokenization results under their published denominator

\pi_0 uses a flow-based action expert with a separate set of robotics-specific weights, and its source reports brittleness from high-quality-only data and imperfect recovery in some settings ^[13]. The source task, robot, data, post-training, and reset denominator cannot be transferred to the S13 tabletop cell. This is counterevidence to “a flow head guarantees recovery,” not a ranking below another head. Terry’s \pi_0 explainer is companion reading; the original paper remains the claim evidence.

FAST reports an action tokenizer trained on one million trajectories and up to a fivefold training-time reduction in its \pi_0 setting ^[15]. It does not establish a universal controller-speed or end-to-end deployment-latency gain. Tokenization, vision encoding, network transfer, queueing, detokenization, projection, and controller acceptance must be timed independently. Training throughput and physical action age are different quantities.

The implementation decision is consequently empirical. Tokenization can compress smooth, redundant trajectories and make a language-style decoder practical, but quantization error may matter near tight tolerances. Chunk regression can be cheap and stable but collapse valid modes. Diffusion or flow can preserve modes but introduces sampling configuration, stochasticity, and compute. A skill-call head can reduce the model’s low-level burden, but only if the versioned skill API has sufficient parameters and preconditions to express the task.

5. Treat public results as filled cells, not a universal leaderboard

Lists of names, parameter counts, and success rates create a false appearance of comparison. Robots, camera sets, dataset mixtures, target adaptation budgets, action coordinates, control rates, reset protocols, and exclusion rules differ. Even within-paper ablations can change several factors. Public results should establish that a configuration worked under a stated evidence stage; local promotion requires a matched rerun.

OpenVLA reports a 7B-parameter model trained on 970,000 real-world robot demonstrations and gains in its 29-task comparison ^[11]. This is not a matched cross-paper leaderboard because data, robots, adaptation, initial states, reset rules, and denominators differ. Its open backbone and fine-tuning path are useful for reproducibility, but the packet does not establish complete real-hardware evidence for the default S13 tabletop stack. Terry’s structural-comparison reading is a reader cross-link, not quantitative evidence.

The common-axis table fixes what the local evaluation must fill rather than copying headline numbers.

RT-1’s large real-robot dataset and closed-loop deployment are an important scaling design point, but its single robot family and proprietary data constrain transfer ^[1]. Gato demonstrates a tokenized generalist policy across many domains, while robot evidence is only part of the mixture and breadth does not imply specialist mastery ^[8]. A compact model such as TinyVLA may report attractive throughput, but it does not fill the operational timing contract in the next section ^[12].

Evidence stage must remain visible. Offline action error can reject obviously incompatible heads, simulation can exercise collision and delays, and shadow mode can measure action age against real observations. None is real execution. A real-robot success count without intervention, force events, resets, and excluded episodes is also incomplete. The same method can occupy different cells for grounding, free-space action, insertion, and recovery.

6. Separate slow proposals from fast authority in contact-rich assembly

The running task is recognize parts/fixture → grasp → collision-free transport → pre-contact stop → bounded placement/insertion → detect failure → retreat/retry/request human/stop. A VLA can ground paraphrased instructions into target states, propose a part, select a grasp or approach skill, and generate a coarse action sequence. After the pre-contact stop, pose error, rising force, jamming, and slip can evolve faster than a vision-heavy inference cycle. Impedance/force control and stop authority remain in the inherited S12 lower stack.

ForceVLA reports a 23.2% average improvement over its stated \pi_0 baselines and up to 80% success on plug insertion ^[16]. These values are specific to the reported platform, tasks, data, and evaluation and are not an expected S13 success rate. Supplying force as a model input does not transfer force limits, protective-stop authority, or final execution authority to the model. The learned policy may use force features for proposals; an independent monitor rejects commands from raw sensors and approved limits. Terry’s ForceVLA explainer is companion reading.

A minimum implementation uses two authority regimes. In free space, the VLA may propose K end-effector deltas or versioned grasp/transport/preinsert skill calls. The adapter validates frames and units, and IK/collision projection accepts or rejects every candidate. Once an independent evaluator approves the pre-contact pose, a state machine invokes insert, whose short references, force bound, timeout, retreat path, and human-handoff conditions are already specified. The VLA may read results and propose a retry, but it cannot clear a stop or change a limit.

Bimanual and dexterous-hand setups are advanced branches, not extra rows in the default denominator. ACT’s real bimanual experiments are useful evidence for chunking, but they do not supply a matched single-arm result ^[4]. Finger contacts, bilateral synchronization, and higher-dimensional actions require new adapters, controllers, monitors, and independent gates.

This boundary also admits hybrid deployments. If the VLA improves paraphrase grounding but not motion, use it only to populate a bounded skill request and let the classical or S12 policy execute. If it improves free-space motion but fails insertion, stop its horizon at preinsert. A larger model need not own more of the stack to provide value.

7. Measure latency from sensor event to physical application

End-to-end latency begins at the camera event. Record sensor time t_s, preprocessing completion t_p, model completion t_m, feasibility completion t_f, queue insertion t_q, controller acceptance t_a, and application t_x. Action age is t_x-t_s; neural inference is only t_m-t_p. Report p50, p95, maximum, deadline-miss rate, and stale-output rejection rather than a mean alone. Chunked policies can overlap inference and execution, but continuing an old tail after a new observation can break closed-loop behavior despite high throughput.

Compact-model throughput does not establish observation age, queue semantics, deadlines, watchdog behavior, or remote-service outage behavior ^{[12] [17]}. Runtime devices, batching, precision, network path, and execution horizons differ, so their numbers do not transfer directly. A local trace must exercise stale-output rejection, atomic replacement by a newer chunk, and timeout fallback to a classical policy or stop.

Mandatory fault injections include delayed and reordered camera samples, GPU contention, network loss, reversed chunk arrival, missing wrist images, state timestamp skew, projection timeout, and a long execution chunk crossing a contact event. Each test links proposal, projected, sent, and accepted/executed identifiers. Distinguish “the model was late while a still-valid reference remained active” from “both the new output and retained reference were stale, so the system stopped.”

The policy server may be healthy while the client queue is wrong, or the model may be correct while de-normalization uses an old robot’s statistics. Do not collapse recognition, grounding, generation, adapter, projection, controller, monitor, and service failures into one label. Each class has a different owner, fallback, and rollback unit.

8. Diagnose in causal order and preserve counterevidence

If the first symptom is a wrong part, begin with instruction swaps, object swaps, and occlusion to test grounding. If the target is correct but pose is biased, inspect camera calibration, frame transform, action de-normalization, and rotation composition. If free-space motion succeeds but contact oscillates, inspect execution horizon, chunk averaging, force zero, contact-state transition, and the lower controller. If failures grow after sustained runtime, inspect action age, queues, thermal throttling, and memory contention. If only novel phrasing fails, separate data overlap from paraphrase and goal-state evaluation.

Counterevidence is part of promotion. Optimal-demonstration mixtures may underrepresent failure recovery ^[10], and the \pi_0 source explicitly discusses brittleness and imperfect recovery ^[13]. Diffusion can represent multimodal action sequences but pays iterative inference cost ^[5]. ACT-style temporal aggregation can blur abrupt contact transitions ^[4]. Structural studies show interactions among action space, history, and head, so one ablation does not justify a universal rule ^[14].

Do not promote a complex VLA if it fails to beat a classical skill baseline or matched S12 BC/ACT under the intended slice. If it helps only with paraphrased instructions, restrict it to upper-level skill selection. If it helps free-space motion but hurts contact, limit authority to pre-contact. If throughput is high but outage recovery is absent, remain in shadow. “More general” is not a substitute for specifying the tested object, scene, instruction, embodiment, and horizon axes.

9. Build a minimal reproducible experiment with gates

The first local study is not a tournament among giant models. Compare the simplest plausible BC, the S12 ACT or diffusion baseline, and one selected VLA, with a shared encoder budget where practical. Every method uses the same episode IDs, initial-state bins, camera/state history, action conversion, execution horizon, controller, monitor, evaluator, and retry budget. Mark every unmatched field so attribution is correspondingly limited.

Use six evidence stages:

1. Contract inspection: load checkpoint and preprocessing without sending actions; validate input/output schema, frames, units, and gripper meaning.
2. Recorded replay: measure grounding, action error, chunk coherence, latency, and rejection on successful, failed, and intervened episodes.
3. Controller stub: exercise projection, queueing, stale rejection, and stop behavior with no device output path.
4. Simulation: compare collision, pre-contact stop, failure detection, and fallback under fixed resets and perturbations.
5. Shadow execution: read real observations while the approved baseline executes; record VLA proposals without sending them.
6. Human-reviewed bounded-trial card: create a separate artifact only after independent owners approve earlier receipts and the exact rollback tuple.

9.1 Bounded Codex implementation prompt

Goal
- Compare BC, an ACT/diffusion baseline, and one selected VLA on identical recorded episodes.
- Report backbone, data, fusion, head, horizon, closed-loop differences and unassignable confounders.

Context
- Read the S11 cell identity and S12 versioned task/skill/action/controller contracts.
- Inputs are recorded observations, instruction variants, robot state, and calibration hash only.
- Outputs are raw proposal, projected proposal, rejection reason, timing trace, and metrics.

Constraints
- Do not call a real robot driver, send_action, controller switch, fault clear,
  or any force/speed-limit API.
- Pin exact checkpoint, preprocessing, tokenizer, action normalization, commit, and device.
- Separate observation/prediction/execution horizons and event/arrival/inference/accept timestamps.
- Do not construct a ranking from unmatched paper numbers.
- Preserve missing/not measured/not exercised/incompatible rather than guessing.

Done when
- Every method replays the same episode UUID, reset bin, instruction, and seed.
- Tests inject stale images, reordered chunks, GPU contention, network outage,
  infeasible pose, and contact-phase changes, then verify fallback.
- Report task success, grounding error, feasibility rejection, projection delta,
  action-age p50/p95/max, deadline miss, intervention, force event,
  recovery, human handoff, compute, and cost.
- Preserve raw/projected/sent/accepted-executed IDs and the full rollback tuple.

Safety
- Recorded data, controller stubs, kinematic checks, simulation, and shadow only.
- Learned confidence, VLA text, and generated actions are not safety authorization.
- Produce a hardware trial card only for separate human review; do not execute it.

This prompt turns the comparison contract into tests rather than merely running repository examples. First verify in a stub that stale and rejected candidates cannot leak to a lower layer. If save/reload does not reproduce the deterministic evaluation sample and metadata, the model is not ready for shadow mode.

Promotion gates are cumulative. A good offline action metric does not erase a stale-command failure. Simulation success does not erase an untested calibration mapping. Shadow stability does not authorize clearing a protective stop. A bounded physical trial, if later approved, retains the same limits and introduces no model-owned override.

10. Record metrics, artifacts, and owner signatures

Success is reported with denominators separated by task, initial-state bin, and novelty axis. Same object/instruction, paraphrased instruction, shifted layout or occlusion, new object, and limited embodiment change are different slices. Assembly success is decomposed into grasp, transport, pre-contact stop, insertion, and recovery. An episode completed by a human after failure is not silently counted as autonomous success.

The required bundle is model_receipt, data_manifest, io_contract, action_adapter, matched_comparison, latency_trace, failure_taxonomy, gate_results, and rollback_tuple. The rollback tuple includes model, tokenizer, preprocessing, action statistics, adapter, task, skill API, controller, calibration, monitor, and environment versions. “Roll back to the previous version” is not executable if any of those identities are missing.

Owner signatures remain separate. The model owner approves checkpoint identity and candidate quality; the data owner approves lineage, duplicates, and splits; the robot owner approves frame/action mapping and controller compatibility; the evaluation owner approves denominators and failure labels; the safety owner approves collision, force, watchdog, stop, and handoff tests. The model’s own critique is not an independent evaluator, and one person’s blanket signature does not merge these authorities.

11. Limits and the interface handed to Chapter 4

The primary sources establish several VLA and action-head designs and report simulation or physical results under their own protocols. No single source validates the complete S13 tabletop-assembly stack. Robot, task, data, adaptation, and reset denominators differ. Web-data overlap and semantic novelty are difficult to eliminate, and success-focused pre-training may omit the failure and recovery distribution. Compute gains from compact models, token compression, or flow generation are not equivalent to operational reliability with observation age, queues, watchdogs, and remote outages.

The conclusion is therefore not “which VLA wins.” First, decompose the system into responsibility layers. Second, place VLA heads on the same action, horizon, and closed-loop axes as S12 policies. Third, use public results only as condition-labeled evidence cells. Fourth, separate free-space proposal from contact execution authority. Fifth, decide promotion using local fault injection, independent evaluation, and a complete rollback receipt.

Chapter 4 receives an interface, not a model name: a freezable backbone, explicit input contract, replaceable action head, versioned action adapter, matched baselines, failure taxonomy, and complete receipt. It will decide whether a new robot/task requires full tuning, partial tuning, adapters, or a new head. If this chapter confounds backbone and head effects, Chapter 4 cannot interpret adaptation. If the responsibilities and versions are separated, a changed camera, coordinate frame, control mode, action rate, or gripper meaning can be audited one item at a time and rolled back exactly.

Supplementary evidence map

The following verified primary-source groups come from the independent S13 ledger. They document search coverage without changing existing claims or denominators.

• 3D and code-mediated policies: FP3: A 3D Foundation Policy for Robotic Manipulation, RoboScript: Code Generation for Free-Form Manipulation Tasks across Real and Simulation, LEGENT: Open Platform for Embodied Agents, NaVILA: Legged Robot Vision-Language-Action Model for Navigation — source-specific tasks, data, and evidence stages remain separate.
• navigation and bimanual systems: Mobility VLA: Multimodal Instruction Navigation with Long-Context VLMs and Topological Graphs, ReVLA: Reverting Visual Domain Limitation of Robotic Foundation Models, Bi-VLA: Vision-Language-Action Model-Based System for Bimanual Robotic Dexterous Manipulations — source-specific tasks, data, and evidence stages remain separate.
• evaluation and robustness: Benchmarking Vision, Language, & Action Models on Robotic Learning Tasks, A Dual Process VLA: Efficient Robotic Manipulation Leveraging VLM, Run-time Observation Interventions Make Vision-Language-Action Models More Visually Robust — source-specific tasks, data, and evidence stages remain separate.
• Terry Korean cross-reading links (reader navigation, not claim evidence): UMI, π0, RoboVLMs, Octo, ForceVLA, Diffusion Policy.

References

1. Brohan, A., et al. (2023a). RT-1: Robotics Transformer for Real-World Control at Scale. Robotics: Science and Systems. arXiv:2212.06817.
2. Brohan, A., et al. (2023b). RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control. CoRL. arXiv:2307.15818.
3. Florence, P., Lynch, C., Zeng, A., & Ramirez, O. A. (2022). Implicit Behavioral Cloning. CoRL. arXiv:2109.00137.
4. Zhao, T. Z., Kumar, V., Levine, S., & Finn, C. (2023). Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware. Robotics: Science and Systems. arXiv:2304.13705.
5. Chi, C., et al. (2023). Diffusion Policy: Visuomotor Policy Learning via Action Diffusion. Robotics: Science and Systems. arXiv:2303.04137. Terry’s explainer.
6. Shridhar, M., Manuelli, L., & Fox, D. (2022a). CLIPort: What and Where Pathways for Robotic Manipulation. CoRL. arXiv:2109.12098.
7. Shridhar, M., Manuelli, L., & Fox, D. (2022b). Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation. CoRL. arXiv:2209.05451.
8. Reed, S., et al. (2022). A Generalist Agent. Transactions on Machine Learning Research. arXiv:2205.06175.
9. Bousmalis, K., et al. (2023). RoboCat: A Self-Improving Robotic Agent. arXiv:2306.11706.
10. Octo Model Team, et al. (2024). Octo: An Open-Source Generalist Robot Policy. arXiv:2405.12213. Terry’s explainer.
11. Kim, M. J., et al. (2024). OpenVLA: An Open-Source Vision-Language-Action Model. arXiv:2406.09246.
12. Wu, J., et al. (2024). TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation. arXiv:2409.12514.
13. Black, K., et al. (2024). $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control. arXiv:2410.24164. Terry’s explainer.
14. Li, X., et al. (2024). What Matters in Building Vision-Language-Action Models for Generalist Robots. arXiv:2412.14058. Terry’s explainer.
15. Kim, M. J., et al. (2025). FAST: Efficient Action Tokenization for Vision-Language-Action Models. arXiv:2501.09747.
16. Yu, W., et al. (2025). ForceVLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulation. arXiv:2505.22159. Terry’s explainer.
17. Agia, C. (2026). Deployment-Time Reliability of Learned Robot Policies. arXiv:2603.11400.
18. Mandlekar, A., et al. (2021). What Matters in Learning from Offline Human Demonstrations for Robot Manipulation. CoRL. arXiv:2108.03298.
19. Yang, R., et al. (2025). FP3: A 3D Foundation Policy for Robotic Manipulation. arXiv primary preprint (2025-03-11). arXiv:2503.08950.
20. Chen, J., et al. (2024). RoboScript: Code Generation for Free-Form Manipulation Tasks across Real and Simulation. arXiv primary preprint (2024-02-22). arXiv:2402.14623.
21. Cheng, Z., et al. (2024). LEGENT: Open Platform for Embodied Agents. arXiv primary preprint (2024-04-28). arXiv:2404.18243.
22. Cheng, A., et al. (2024). NaVILA: Legged Robot Vision-Language-Action Model for Navigation. arXiv primary preprint (2024-12-05). arXiv:2412.04453.
23. Chiang, H. L., et al. (2024). Mobility VLA: Multimodal Instruction Navigation with Long-Context VLMs and Topological Graphs. arXiv primary preprint (2024-07-10). arXiv:2407.07775.
24. Dey, S., et al. (2024). ReVLA: Reverting Visual Domain Limitation of Robotic Foundation Models. arXiv primary preprint (2024-09-23). arXiv:2409.15250.
25. Gbagbe, K. F., et al. (2024). Bi-VLA: Vision-Language-Action Model-Based System for Bimanual Robotic Dexterous Manipulations. arXiv primary preprint (2024-05-09). arXiv:2405.06039.
26. Guruprasad, P., et al. (2024). Benchmarking Vision, Language, & Action Models on Robotic Learning Tasks. arXiv primary preprint (2024-11-04). arXiv:2411.05821.
27. Han, B., et al. (2024). A Dual Process VLA: Efficient Robotic Manipulation Leveraging VLM. arXiv primary preprint (2024-10-21). arXiv:2410.15549.
28. Hancock, A. J., et al. (2024). Run-time Observation Interventions Make Vision-Language-Action Models More Visually Robust. arXiv primary preprint (2024-10-02). arXiv:2410.01971.