Chapter 6: Can Prediction Support Planning? — World Models, Video/Action Prediction, and Policy Coupling

Overview

A world model lets a robot estimate what may follow an action before exposing hardware to it. “Predicting the future,” however, conflates several different jobs. Generating camera frames, estimating transitions in a compressed state, ranking action candidates, anticipating reward or failure, and training a policy on imagined transitions have different inputs, outputs, errors, and validation procedures. Evidence for one job does not automatically establish performance—or safety—in another.

S11 contributes the physical cell’s cameras, tactile and force sensors, clocks, frames, calibration, controllers, collision and force limits, watchdog, protective stop, E-stop, and accountable human. S12 contributes versioned task and skill APIs, feasibility projection, motion and contact controllers, proposal → projected → sent → accepted/executed lineage, independent evaluation, evidence stages, and the complete rollback tuple. Chapter 5 converted language and scenes into goal states and skill preconditions. This chapter asks a new question: under that contract, where does prediction produce a measurable closed-loop benefit, and what approvals prevent a predicted future from being mistaken for a physical command?

After reading this chapter, you will be able to... - distinguish latent dynamics, video prediction, action-conditioned prediction, reward, value, and uncertainty models by responsibility; - choose among planning, candidate ranking, data generation, failure prediction, and policy learning on matched axes; - diagnose contact, occlusion, long-horizon error, hallucination, and controller–dynamics mismatch separately; - record prediction quality, uncertainty quality, closed-loop task benefit, and safety-gate passage as different measurements; and - build a prediction→candidate→projection→human approval→bounded execution pipeline for tabletop assembly.

The experiment question is: with the tabletop task, initial-state distribution, observation and action semantics, skill APIs, controller, evaluator, and safety limits fixed, does adding a world model reduce bad candidates and physical exposure while improving closed-loop success and recovery over the classical baseline or S12 policy? At which evidence stage—replay, simulation, shadow mode, or a bounded real trial—does that benefit hold?

1. Mental model: prediction proposes; the execution contract approves

Calling a world model “a simulator in the robot” hides too much. The model predicts a selected representation, not every sensor and physical variable. That representation may discard friction, tolerance, cable elasticity, controller delay, or an occluded contact state. An action sequence evaluated by the model is not yet a skill call or controller reference. In this chapter, predictive authority stops at generating and comparing candidates.

versioned goal state + fresh observation + allowlisted skill candidates
                              ↓
                  conditional model rollout
          video | latent state | contact | reward/value | failure
                              ↓
             per-candidate prediction, cost, uncertainty
                              ↓ independent objective, freshness, evidence checks
                    ranked candidate bundle
                              ↓
skill API schema → IK/planning → constraint projection → collision/force/deadline
                              ↓
        shadow mode → human-reviewed bounded trial → accepted/executed

Independent throughout: evaluator · watchdog · protective stop · E-stop · human authority

The predictor may say, “this candidate appears closer to the goal than those alternatives.” Only the execution spine may decide, “this candidate can be sent on this robot, with this calibration, controller, and timestamp.” A high predicted success probability cannot override collision clearance, force limits, observation age, command deadline, or human approval. High uncertainty likewise cannot authorize a reset of the E-stop or a change to a safety boundary.

World models can serve distinct roles in latent-state transition, video prediction, candidate ranking, synthetic transition generation, reward or value estimation, and policy learning ^{[1] [10] [14]}. Dyna combines learning from real experience with planning updates from model-generated experience; World Models trains a controller in a compressed recurrent environment; Dreamer trains an actor and critic on imagined latent rollouts. These sources establish possibilities for particular roles, not that success in one validates another or makes contact assembly safe. Their main evidence is also simulation in tabular or game-like environments, not the sensor drift and discontinuous contact of this cell.

2. What should be predicted? A world-model selection table

Selection begins with the sufficient variables for the decision, not a model name or parameter count. Collision clearance during free-space transport needs time-aligned robot and object geometry. Ranking a pre-contact insertion pose needs part pose plus contact mode and force direction. A sharp video can omit all of these and still be useless to the planner.

Video prediction is easier for a human to inspect, yet averaging multiple possible futures blurs objects ^[5]. Latent prediction reduces computation and supports policy learning, but it is harder to tell what was discarded ^[10]. State prediction connects directly to geometry gates but cannot restore occlusion or deformation omitted by the estimator. Contact prediction offers signals closer to insertion but is sensitive to sensor placement and material shift. These are not ranks in a leaderboard; they are instruments with different blind spots.

2.1 Keep five use paths separate

The planning path rolls out action sequences, chooses low-cost candidates, executes a prefix, and observes again. The candidate-ranking path may compare a small set from a VLA or task planner without generating new actions. The data-generation path adds model transitions to a replay buffer, but every record must retain synthetic=true, generator version, and the real/synthetic mixture. The failure-prediction path flags contact anomaly, loss under occlusion, or deadline failure early. The policy-learning path updates value or policy functions from imagined experience without reporting imagined return as real success.

Dyna’s enduring contribution is a feedback loop in which real experience updates both the model and policy/value process ^[1]. Dreamer provides a different path through latent imagination and actor–critic learning ^[14]. Neural dynamics followed by model-free fine-tuning shows that planning and policy-learning paths can be coupled, but the later policy does not inherit planner constraints automatically ^[11]. Collapsing planning, synthetic data, and policy learning into one “world-model score” removes the ability to attribute either benefit or error.

3. Closed-loop planning: correct replanning matters more than sharp futures

Model predictive control (MPC) compares finite-horizon action sequences from the current observation, applies only a prefix, and solves again after a new observation. Replanning limits long-rollout error; it does not eliminate it. If camera frames arrive late or commands queue, a freshly solved plan can still begin from stale state. The trace therefore needs event time, arrival time, inference start and end, and command validity time.

The cross-entropy method (CEM) samples action sequences, fits the sampling distribution to an elite set, and iteratively narrows candidates. Sampling-based path-integral control instead weights noisy control rollouts by exponentiated cost before updating the first control ^[7]. Both can work without differentiating through the model, but compute grows with candidate count, horizon, and ensemble size. Prediction metrics must be paired with planning_latency_p95, deadline misses, and stale-candidate disposal.

Visual Foresight demonstrates a route from action-conditioned video prediction and pixel goals to short-horizon planning on real robots ^{[6] [9]}. A pixel goal, however, does not completely express whether a part is physically seated in the correct socket, avoided a forbidden object, or stayed within force limits. Occlusion removes the goal pixel, while averaging multiple contact outcomes may present the optimizer with a falsely smooth cost surface.

Sharper futures or lower average prediction error do not necessarily improve closed-loop task success ^{[6] [9]}. The two studies use different robots, prediction objectives, and manipulation settings and therefore cannot be ranked directly. Their shared lesson is narrower: planning needs the correct relative cost among action candidates and state information that supports replanning, not photorealistic reconstruction of every pixel. Local assembly evaluation must pair visual loss with goal-predicate error, candidate-rank correlation, success after selection, and contact or force events.

The same principle applies to an upper model that provides path guidance. HAMSTER separates a high-level 2D path from a low-level policy, while noting that the 2D interface cannot communicate force or rotation fully ^[19]. A plausible path must therefore be projected into the contact skill’s input contract; an unprojectable path is discarded. Receding-horizon action chunks, as in Diffusion Policy, provide a related timing design, but the world model’s observation and action meanings must match the policy and controller ^[17].

4. Uncertainty is a request-for-scrutiny signal, not a safety certificate

Probabilistic dynamics predict a distribution rather than a single next state. PILCO propagates Gaussian-process dynamics uncertainty into long-term cost for data-efficient policy search, but it scales poorly to high-dimensional vision and large datasets, while Gaussian moment matching can miss multimodal contact outcomes ^[3]. PETS combines ensembles of probabilistic neural dynamics with trajectory sampling, but ensemble disagreement is not automatically a calibrated probability of failure ^[8].

At least three sources of uncertainty should be recorded separately. Observation noise or irreducible variation in contact may remain as data grows. Model uncertainty from sparse coverage may shrink with relevant data. Distribution shift appears with a new fixture, reflection, occlusion, sensor replacement, or friction regime. Single-model variance, ensemble disagreement, latent distance, and input density are different proxies; they should not be collapsed into one scale.

Probabilistic dynamics and ensembles can support conservative candidate selection or requests to re-observe, but their uncertainty is conditional on the evaluated distribution and error definition; it is not an independent safety certificate ^{[3] [8] [12]}. Evidence that calibration degrades under dataset shift comes from classification rather than sequential contact control, but it directly warns against transferring an in-distribution calibration curve into a safety probability for a new cell. Collision and force gates, watchdogs, and stop authority stay outside model scoring.

Evaluation should not report only the accuracy of low-uncertainty samples. As the threshold changes, measure unsafe false accepts, unnecessary rejects, human handoffs, shift-specific calibration error, and warning lead time before a physical event. Preserve confident-but-wrong cases as a dedicated suite and verify whether an update removes them or merely changes the failure mode.

5. Tabletop walkthrough: the prediction→execution approval pipeline

The through-task remains component and fixture recognition, grasping, collision-free transport, a pre-contact stop, bounded placement or insertion, failure detection, and retreat, retry, human request, or stop. This chapter adds new objects, layouts, occlusion, and multi-skill sequences. The controlled comparison changes the world-model role while holding the goal, initial distribution, cameras and calibration, skill and action semantics, controller, collision and force limits, evaluator denominator, and recovery policy fixed.

5.1 Case: inserting an occluded slotted bracket

1. Freeze the contract. Record task@6.2, scene and calibration generations, grasp@2.1, transport@3.0, precontact@1.4, insert@2.3, and controller/evaluator hashes. Any absent element is missing, closing the hardware gate.
2. Synchronize observations. Store event, arrival, and use times for scene and wrist cameras, joints, gripper, and force–torque signals. Reject observations older than the declared bound before model input.
3. Create candidates. A classical planner, S12 policy, or VLA proposes different pre-contact poses, insertion speeds, or re-observation actions. Convert all candidates to the same action schema and horizon.
4. Roll out inside the model. Video, latent, and contact models output goal error, object permanence under occlusion, predicted contact state, failure type, and uncertainty. Keep source observations distinct from generated futures.
5. Rank candidates only. Log objective version, cost terms, model-wise scores, uncertainty penalties, and invalidity reasons. The top candidate is not sent automatically.
6. Pass independent execution gates. Check the skill schema, preconditions, IK, collision, constraint projection, force and speed, deadlines, queues, and human approval. Preserve the difference between the original and projected candidate.
7. Promote by evidence stage. Use held-out replay, offline ranking, simulation, shadow mode, and a human-reviewed low-speed bounded trial in order. Each has its own denominator and receipt.
8. Attempt to falsify the model. Compare evaluator judgment, contact or force events, interventions, retreats, retries, and handoffs against prediction. A wrong prediction enters the failure suite; a successful video does not erase it.

For tabletop assembly, predicted outcomes rank candidates; the unchanged skill-schema, feasibility-projection, controller, collision/force, stop, and human gates determine execution ^{[20] [23] [16]}. World4RL learns its world model from fixed data, refines a policy in imagined rollouts, and then evaluates on Meta-World and six physical Franka tasks with 20 rollouts per real task. STORM uses generated futures for search-guided candidate selection on four WidowX tasks in the SimplerEnv simulator, averaging over 24 procedural variations per task. Neither result validates this complete local real-hardware stack. A predictive safety filter supplies a mathematical route for modifying proposals under stated dynamics and backup feasibility, yet model error or optimization latency prevents it from becoming a high-rate safety guarantee. The pipeline is an S13 synthesis of these roles, not evidence that any source transferred command authority to a model.

5.2 Approval states and failure outputs

The prediction service cannot write any state after projected. Keep prediction_id, proposal_id, projection_id, command_id, and execution_id distinct and linked. After a protective stop or E-stop, the system stays closed until the prescribed human reconnection process finishes, even if the model predicts that recovery is possible.

6. What breaks under contact, occlusion, and long horizons?

6.1 Contact discontinuity and controller–dynamics mismatch

In free space, a small state error may produce a small pose error. During insertion, a tiny orientation difference can separate free motion, edge contact, and jamming. A camera model can map the same image to different force states. If the learned action is an end-effector delta but the controller receives filtered joint references or impedance targets, the learned transition differs from the real loop. Leaving out saturation, rate limits, queues, communication delay, gripper internal control, or the contact controller’s integrator means the model predicts the future of a proposed action, not an executed one.

This mismatch becomes visible only when proposed_action, projected_action, sent_command, accepted_command, and executed_state are retained together. Include control mode and controller version in model context, and roll out the action after projection. If the actual command differs from the modeled action, mark the interval controller_mismatch rather than silently mixing it into training.

6.2 Occlusion and object permanence

When the wrist camera blocks the part or the socket disappears during insertion, a video generator may continue drawing a plausible object. That is hallucination, not new evidence about hidden state. Even when scene cameras, touch, joints, and gripper state are fused, distinguish observed, inferred, and predicted, along with the last verification time. If instance identity changes across occlusion or uncertainty exceeds the task bound, transition to re-observation, retreat, or a human request.

RoboNet broadens action-conditioned prediction data across institutions and robot arms, while still requiring platform-specific camera and action handling and concentrating on short pushing interactions ^[13]. Experience-embedded prediction offers a fast adaptation path after observing a few trajectories of a new object, but first incurs interaction and does not establish insertion or multi-skill recovery ^[15]. Data diversity does not manufacture ground truth for hidden contact.

6.3 Long horizons and self-reinforcing error

One-step bias becomes input to the next prediction and compounds with horizon. A policy may exploit model weakness by choosing a high-return future that is physically impossible. Short horizons, frequent reset to real observations, ensemble disagreement penalties, conservative terminal values, skill boundaries, and backup policies can reduce exposure. They are experimental controls, not safety proofs.

Long tasks can avoid one monolithic video prediction by evaluating state again at skill boundaries such as grasp complete, pre-contact pose, and insertion verified. Probabilistic object–action–effect models illustrate effect prediction over a limited vocabulary, but learned correlation does not enforce collision or force feasibility ^[2]. Human-activity anticipation likewise supplies ideas for reactive scenes while leaving a gap between a closed activity vocabulary and assembly preconditions ^[4]. Long horizons demand verifiable intermediate states and stopping points before a larger generator.

7. Read evidence on matched axes

Success rates from different robots, tasks, datasets, and denominators do not belong in one leaderboard. This table states the interpretation each evidence bundle permits.

DayDreamer reports learning specified quadruped skills in about one hour and adapting to changes in roughly ten minutes ^[18]. This is evidence that world-model-based online learning can work on physical robots, not evidence for language-led contact assembly. The task and embodiment differ, and exploration still creates physical exposure. These times therefore do not become our assembly adaptation budget or safety result; the local trial needs matched denominators for initialization, interventions, force events, and stops.

The 2026 ContactWorld preprint reports planning success of 20.7% with wrist vision, 22.0% with front vision, 32.1% with point clouds, and 36.1% with point clouds plus tactile force fields ^[25]. These numbers are comparable only inside that paper’s offline planning setup and Table S3.T1; they provide task-specific evidence that contact sensing can alter representation choice. The recent preprint has not been independently replicated and does not validate a bounded real trial, latency, interventions, force events, or safety authority in the S13 cell. It is not a general leaderboard or release receipt.

RoDyn proposes a 2.5D interactive representation while leaving scale across heterogeneous embodiments and action spaces unresolved ^[22]. World4RL connects a diffusion world model to policy refinement ^[20], and STORM connects generated futures to candidate search ^[23], but their different preprint settings and numbers cannot be combined into a rank. If an official manifest does not disclose train/evaluation overlap, contamination remains unknown.

8. Diagnose from symptoms toward causes

Diagnosis does not begin with retraining. First confirm task, observation, action, and controller versions; confirm that the executed action was logged; confirm the evaluation denominator did not change. Then locate the first boundary at which error appears: one-step prediction, multi-step rollout, candidate rank, or closed-loop outcome. If the cause is not measured, add instrumentation rather than filling it with a larger model.

Recent work that attributes failure between grasping and downstream planning is a useful reminder not to collapse coupled errors into one score ^[24]. Interaction-driven object learning similarly targets gaps in static data while retaining physical exploration cost and sim-to-real mismatch ^[21]. Both are candidates for local failure-taxonomy design, not complete safety and operating contracts.

9. Minimum reproducible workflow and validation gates

Gate W0 — Contract readiness

• Freeze task, initial distribution, goal predicates, and exclusion rules.
• Record observation, action, skill, controller, calibration, evaluator, and safety-device versions.
• Exercise stale observation/command, reconnect, protective-stop, and human-approval semantics.
• Preserve absent, unmeasured, unexercised, and incompatible values as missing, not measured, not exercised, and incompatible.

Gate W1 — Prediction itself

• Measure one-step and multi-step state, video, and contact error on held-out splits.
• Separate occlusion, new object, layout, friction, lighting, and calibration shift.
• Store p95, worst slices, failure-type errors, and calibration—not only means.
• Do not promote to planning if the model cannot predict the task’s goal predicates.

Gate W2 — Candidate ranking

• On a fixed candidate bundle, compare predicted order with actual outcomes and measure top-choice regret.
• Compare against classical cost and S12 policy scores on exactly the same candidates and denominator.
• Close as no_better_than_baseline if benefit is absent or latency exceeds budget.

Gate W3 — Closed-loop non-command test

• After replay and simulation, run shadow inference on live sensor streams.
• Compute candidates without sending them to a robot driver.
• Record observation age, planning p50/p95/max, stale disposal, and expected independent-gate rejection.

Gate W4 — Human-reviewed bounded trial

• Use a few known initial states under low speed and force with explicit stop criteria.
• Require an accountable approval card per trial while watchdog, protective stop, and E-stop remain independent.
• Record denominators for success, intervention, force event, retreat, recovery, and handoff—not one success video.
• On a stable failure, restore the exact model, data, normalization, task, skill, controller, calibration, and evaluator tuple.

A predictive safety filter is an important design for mapping a learning proposal into a backup-feasible constrained action under stated assumptions ^[16]. Passing that filter remains projection evidence at W4; it does not replace collision/force monitoring or human approval.

10. Manufacturing Cell Checkpoint

Required artifacts are world_model_card, observation_action_schema, prediction_dataset_manifest, prediction_eval, calibration_report, candidate_rank_trace, planner_timing, proposal_execution_lineage, failure_attribution, shadow_report, limited_trial_card, and world_model_receipt. Every record distinguishes source observation from prediction, real from synthetic, and proposal from projection, send, and execution.

10.1 Three metric layers

The prediction layer includes one-step and horizon-wise state, video, and contact error, object permanence, goal-predicate accuracy, negative log likelihood, and calibration. The decision layer includes rank correlation, top-choice regret, selection changes relative to baseline, planning latency, and deadline misses. The execution layer includes closed-loop success, intervention, infeasible proposals, collision or force events, stop, retreat, retry, recovery, human handoff, observation age, and operating cost.

Do not collapse the layers into one score. If prediction improves but execution does not, demote the world model to visualization or data diagnosis. If ranking improves but misses the deadline, retain it for offline review only. If success rises together with interventions or force events, do not promote it.

Terry’s discussion of global planning and contact is useful follow-up reading for separating pre-contact global candidates from execution under contact. It is a reader crosslink, not claim evidence here.

11. Bounded Codex implementation prompt

Goal
- Evaluate one world-model role at a time on recorded tabletop-assembly data.
- Predict futures, failures, and uncertainty for a fixed candidate set and produce a ranking report.
- Produce prediction/ranked/projection-request artifacts, never a real command.

Context
- Read S11 cell/sensor/clock/frame/calibration/controller/safety identities.
- Read S12 task/skill/action/projection/evaluator/rollback contracts.
- Freeze Ch5 grounded goal, object/part/relation, precondition, and prohibition records.
- Use only held-out replay, simulator, recorded shadow streams, and controller stubs.

Constraints
- Do not call robot drivers, send_action, controller switch, force-limit change, or stop reset.
- Never merge observed/inferred/predicted or real/synthetic into one field.
- Keep proposed/projected/sent/accepted-executed action IDs distinct.
- Do not impute missing/not measured/not exercised/incompatible values.
- Do not call model uncertainty a safety probability or certification.
- Do not rank success from unmatched robot/task/data/horizon/denominator settings.
- Do not bypass collision/force/watchdog/protective-stop/E-stop/human authority.

Completion
- Produce a world-model selection table and typed prediction schema.
- Separate one-step/multi-step, occlusion, contact, long-horizon, and controller-mismatch tests.
- Report rank correlation, top-choice regret, calibration, and latency on fixed candidates.
- Preserve prediction_id→proposal_id→projection_id lineage and every rejection reason.
- Record replay→simulation→shadow as separate denominators, with hardware command count zero.
- Make no-better-than-baseline, stale, OOD, and infeasible states reproducible.
- Point world_model_receipt to the exact input/model/objective/controller/evaluator tuple and rollback.

Safety
- Outputs are human-reviewable candidates, not execution authority.
- A bounded hardware trial is outside this task and requires a separate approval card.
- Protective stop, E-stop, and final approval remain with independent devices and the named human.

12. Counterevidence, open questions, and the interface to Chapter 7

Current evidence does not support the conclusion that a world model validates a complete tabletop-assembly stack. Classical tabular structures do not scale directly to vision and contact. Gaussian processes become costly with data and dimension, and Gaussian approximations can miss multiple contact modes. Video models blur or hallucinate plausible futures; latent models can discard task-critical variables. Ensembles may share common bias, and long imagination can produce optimistic value. Data-efficient online learning still exposes hardware.

Three questions remain open. First, which combination of vision, touch, and state is sufficient for a particular assembly tolerance while meeting the deadline? Second, how should uncertainty proxies be calibrated under new objects, materials, and occlusion, and how can the evaluator avoid a common-cause failure with the model? Third, does benefit stop at candidate selection or extend to data generation and policy learning, and what matched real denominator supports each role? The 2025–2026 preprints propose candidate answers, but need independent replication and local operational evidence.

Chapter 7 receives neither a free-form future video nor an automatically executable plan. It receives a versioned world-model card, allowed input/action/horizon, per-candidate prediction and uncertainty, objective, rejection reason, evidence stage, predicted completion/failure at skill boundaries, residual against actual outcome, and an explicit no-execution-authority flag. A robot agent may use this record to propose re-observe, another allowlisted skill, a question to a human, or stop.

The next chapter connects these candidates to skill calls, memory, and tools over a long task. Prediction does not give an agent authority to run arbitrary code, invent a skill, or bypass a gate. Chapter 7 asks who may propose what, and who approves, inside versioned skill APIs and a failure/recovery state machine.

What to Learn Next

Chapter 7 separates the authorities of planning, memory, tools, and skill APIs in a robot agent. This chapter’s ranked candidate, prediction residual, uncertainty, and reject reason become agent inputs; the agent’s output becomes another bounded skill-call candidate. The reader carries forward a world model as a falsifiable supplier of possible futures, never an executor.

Supplementary evidence map

The following verified primary-source groups come from the independent S13 ledger. They document search coverage without changing existing claims or denominators.

• contact world models: Revisiting Embodied Chain-of-Thought for Generalizable Robot Manipulation, ARP: Enhancing Quantized Skill Abstractions via Visual Alignment and Iterative Refinement for Robotic Manipulation, OmniVTA: Visuo-Tactile World Modeling for Contact-Rich Robotic Manipulation, TORL-VLA: Tactile Guided Online Reinforcement Learning for Contact-Rich Manipulation — source-specific tasks, data, and evidence stages remain separate.
• prediction and reinforcement learning: Act2Goal: From World Model To General Goal-conditioned Policy, RoboPack: Learning Tactile-Informed Dynamics Models for Dense Packing, DoughNet: A Visual Predictive Model for Topological Manipulation of Deformable Objects — source-specific tasks, data, and evidence stages remain separate.
• generative closed-loop control: Reinforcement Learning Controllers for Soft Robots using Learned Environments, Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulation, Closed-Loop Visuomotor Control with Generative Expectation for Robotic Manipulation — source-specific tasks, data, and evidence stages remain separate.

References

1. Sutton, R. S. (1991). Dyna, an Integrated Architecture for Learning, Planning, and Reacting. ACM SIGART Bulletin.
2. Montesano, L., Lopes, M., Bernardino, A., & Santos-Victor, J. (2008). Learning Object-Action Complexes: A Probabilistic Generative Approach. IEEE Transactions on Robotics.
3. Deisenroth, M. P., & Rasmussen, C. E. (2011). PILCO: A Model-Based and Data-Efficient Approach to Policy Search. Artificial Intelligence.
4. Koppula, H. S., & Saxena, A. (2013). Anticipating Human Activities Using Object Affordances for Reactive Robotic Response. Robotics: Science and Systems. arXiv:1305.5461.
5. Finn, C., Goodfellow, I., & Levine, S. (2016). Unsupervised Learning for Physical Interaction through Video Prediction. NeurIPS. arXiv:1605.07157.
6. Finn, C., & Levine, S. (2017). Deep Visual Foresight for Planning Robot Motion. IEEE ICRA. arXiv:1610.00527; DOI:10.1109/ICRA.2017.7989324.
7. Williams, G., Aldrich, A., & Theodorou, E. (2017). Information-Theoretic Model Predictive Control: Theory and Applications to Autonomous Driving. IEEE Transactions on Robotics.
8. Chua, K., Calandra, R., McAllister, R., & Levine, S. (2018). Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models. NeurIPS. arXiv:1805.12114.
9. Ebert, F., Finn, C., Lee, A. X., & Levine, S. (2018). Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control. arXiv:1710.05268.
10. Ha, D., & Schmidhuber, J. (2018). World Models. arXiv:1803.10122.
11. Nagabandi, A., Kahn, G., Fearing, R., & Levine, S. (2018). Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning. IEEE ICRA. arXiv:1708.02596; DOI:10.1109/ICRA.2018.8463189.
12. Ovadia, Y., et al. (2019). Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift. NeurIPS. arXiv:1906.02530.
13. Dasari, S., et al. (2020). RoboNet: Large-Scale Multi-Robot Learning. CoRL. arXiv:1910.11215.
14. Hafner, D., Lillicrap, T., Ba, J., & Norouzi, M. (2020). Dream to Control: Learning Behaviors by Latent Imagination. ICLR. arXiv:1912.01603.
15. Yen-Chen, L., Bauza, M., & Isola, P. (2020). Experience-Embedded Visual Foresight. CoRL / PMLR 100.
16. Wabersich, K. P., & Zeilinger, M. N. (2021). A Predictive Safety Filter for Learning-Based Control of Constrained Nonlinear Dynamical Systems. Automatica. arXiv:1812.05506; DOI:10.1016/j.automatica.2020.109597.
17. Chi, C., et al. (2023). Diffusion Policy: Visuomotor Policy Learning via Action Diffusion. Robotics: Science and Systems. arXiv:2303.04137.
18. Wu, P., Escontrela, A., Hafner, D., Abbeel, P., & Goldberg, K. (2023). DayDreamer: World Models for Physical Robot Learning. CoRL / PMLR 205.
19. Li, Y., et al. (2025). HAMSTER: Hierarchical Action Models For Open-World Robot Manipulation. arXiv:2502.05485.
20. Jiang, Z., et al. (2025). World4RL: Diffusion World Models for Policy Refinement with Reinforcement Learning for Robotic Manipulation. arXiv:2509.19080v2.
21. Peng, Y., et al. (2025). RoboSeek: You Need to Interact with Your Objects. arXiv:2509.17783.
22. Zhang, C., Wu, Z., Lu, G., Tang, Y., & Wang, Z. (2025). RoDyn: Taming Interactive Robot-Dynamic 2.5D World Model for Robotic Manipulation. arXiv:2510.09036.
23. Lin, W., Zhang, J., Cai, K., & Wang, K. (2025). STORM: Search-Guided Generative World Models for Robotic Manipulation. arXiv:2512.18477.
24. Xu, J., et al. (2026). Grasp-Then-Plan with Failure Attribution: A Closed Two-Stage Framework for Precise and Generalizable Robotic Manipulation. arXiv:2606.03385.
25. Zhang, Z., et al. (2026). ContactWorld: What Matters in Vision-Tactile World Models for Contact-Rich Manipulation. arXiv:2606.13877v1.
26. Sun, N., et al. (2026). Revisiting Embodied Chain-of-Thought for Generalizable Robot Manipulation. arXiv primary preprint (2026-06-02). arXiv:2606.03784.
27. Wang, Y., et al. (2026). ARP: Enhancing Quantized Skill Abstractions via Visual Alignment and Iterative Refinement for Robotic Manipulation. arXiv primary preprint (2026-06-21). arXiv:2606.22480.
28. Zheng, Y., et al. (2026). OmniVTA: Visuo-Tactile World Modeling for Contact-Rich Robotic Manipulation. arXiv primary preprint (2026-03-19). arXiv:2603.19201.
29. Zheng, H., et al. (2026). TORL-VLA: Tactile Guided Online Reinforcement Learning for Contact-Rich Manipulation. arXiv primary preprint (2026-06-08). arXiv:2606.09337.
30. Zhou, P., et al. (2025). Act2Goal: From World Model To General Goal-conditioned Policy. arXiv primary preprint (2025-12-29). arXiv:2512.23541.
31. Ai, B., et al. (2024). RoboPack: Learning Tactile-Informed Dynamics Models for Dense Packing. arXiv primary preprint (2024-07-01). arXiv:2407.01418.
32. Bauer, D., et al. (2024). DoughNet: A Visual Predictive Model for Topological Manipulation of Deformable Objects. arXiv primary preprint (2024-04-18). arXiv:2404.12524.
33. Berdica, U., et al. (2024). Reinforcement Learning Controllers for Soft Robots using Learned Environments. 2024 IEEE 7th International Conference on Soft Robotics (RoboSoft), San Diego, CA, USA, 2024, pp. 933-939. arXiv:2410.18519.
34. Bharadhwaj, H., et al. (2024). Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulation. arXiv primary preprint (2024-09-24). arXiv:2409.16283.
35. Bu, Q., et al. (2024). Closed-Loop Visuomotor Control with Generative Expectation for Robotic Manipulation. arXiv primary preprint (2024-09-13). arXiv:2409.09016.