IFM's new world-model release combines structured plans with video generation. Its potential for physical AI depends on whether the imagined outcome preserves the timing, objects and physical relationships that a real decision requires
Imagine a robot considering how to move an object across a crowded table. Before acting, it could try to predict whether its movement would disturb a nearby item. A visual prediction would let a person inspect the proposed outcome and might help the robot compare candidate actions. This is an illustrative use of a world model, a learned system intended to predict how a situation develops. Its usefulness depends on whether it preserves the relationships that matter to the decision. An attractive depiction of a successful movement can be a poor guide if it hides the collision or changes the object along the way.
On 10 November 2026, the Institute of Foundation Models at Mohamed bin Zayed University of Artificial Intelligence announced PAN 2, an open world model intended to generate future scenes from a starting observation and actions described in words. The release makes a prediction problem accessible through video generation. Its significance for physical AI rests on a further question: whether generated outcomes are reliable enough to help an agent choose what to do. The announcement presents possible applications and demonstrations. Evidence of dependable behaviour in a particular robot or environment would require evaluation in that setting.
The idea has a clear lineage. The earlier PAN research paper described a model for interactable video simulation conditioned on history and natural-language actions. PAN 2 extends that research direction through a new release with public code and model documentation. It is useful to keep the simulator’s output separate from an agent’s eventual decision. The simulator supplies a possible continuation. Another part of the system has to decide which continuation is desirable, whether it is plausible and how to translate a chosen plan into an action the equipment can execute.
From a plan in words to motion in a scene
IFM’s project page describes an architecture that couples conceptual and perceptual reasoning. A multimodal language model develops a structured description of events, while a 29-billion-parameter latent diffusion model produces their visual realisation. The intended division gives the language component responsibility for what happens and when, leaving the generative component to fill in appearance and motion. That separation can make a proposed sequence easier to inspect. It also introduces two places where a prediction can go wrong, either in the plan itself or in how the plan becomes a video.
The released model card describes time-grounded actions associated with actors and a separately specified camera movement. It lists video generation of up to 20 seconds and identifies the released checkpoint as the diffusion backbone, with a text encoder and video autoencoder obtained separately. This matters when interpreting the phrase “world model”. The downloadable component has a defined role in a larger workflow. A user needs to understand which component creates the structured instructions and which components condition and render the resulting sequence.
At a general level, a latent model works with a compressed representation of the visual material. Diffusion generation progressively transforms an initial noisy representation into one consistent with the conditioning information. A decoder turns that representation into visible frames. Compression makes a large prediction problem more manageable, but it also leaves the system with choices about details absent from the input. In the table example, an image might show the objects without revealing their weight, friction or what is hidden behind them. A generated continuation can resolve that uncertainty in a plausible way even when another outcome would occur in the real scene.
Timing and viewpoint are part of the test
A natural-language instruction often leaves more open than it seems to. “Move the object after the person steps back” specifies an order, while a plan with timestamps specifies when that order should unfold. If a model changes the timing but leaves the outcome unchanged, it may be following the broad story while missing a causal condition. IFM’s demonstrations invite viewers to compare such variations. Their value as evaluation examples would depend on whether the altered conditions produce the expected differences consistently, across more than the selected clips displayed on a project page.
Camera movement creates another way to expose inconsistency. A camera can move around a stationary object, changing its visible appearance without changing its location. The model has to preserve those different facts. An illustrative test might begin with an object partly hidden behind another, move the viewpoint and then return. The evaluator could ask whether the same object reappears in the same state. That is a different criterion from asking whether each frame looks credible. A system used for planning needs continuity in the features that its decision depends on, including features that temporarily leave the view.
There is also uncertainty in interpreting an apparent failure. If a prompt describes a physical situation ambiguously, two outcomes might both be defensible. If the starting image lacks relevant information, the model cannot recover it simply by generating more detail. A useful test would identify what is observed, what the task assumes and which relationship is being measured. That approach would make failures easier to classify. Some would call for a better model, others for better observations, and others for a more precise account of the action being proposed.
Physical fidelity needs a measurement
The distinction between realism and useful prediction has been studied directly. The authors of Physics-IQ introduced physical experiments as tests for video models and found that visually convincing output could coexist with limited physical understanding in the systems they examined. That study predates PAN 2 and supplies no score for the new model. It explains why a release in this field needs tests tied to physical relationships. A viewer’s impression of smooth motion offers limited evidence about conservation, contact or the response to a changed initial condition.
Benchmark design is itself an active research problem. A June 2026 audit introducing Physics-IQ Verified examined prompt and reference quality and revised samples and scoring. Its authors reported changes in model rankings after those revisions. The lesson for interpreting a world-model result is methodological. The meaning of a score depends on what the prompt communicates, what counts as a valid reference and how different mistakes are weighted. A comparison is easier to assess when its exact evaluation version and settings are disclosed.
A separate October preprint on measurement-based physical evaluation proposes controlled tasks with explicit observable criteria across several domains of physics. Such an approach can help an evaluator ask whether an object maintains a relevant relationship through time, rather than relying entirely on a model judge’s description of the clip. It also has limits. An event hidden from view may be impossible to measure reliably from the output alone. An evaluation should report those cases and avoid turning an unobservable event into a confident judgement of physical success.
What the open release makes possible
The PAN 2 repository provides training and inference entry points, documents components and identifies Apache 2.0 licensing alongside third-party notices. That gives researchers material to inspect and adapt. Availability still has practical conditions, including the compute required to run the model and the terms of separately supplied components. A team comparing results would need to record the checkpoint, runtime and generation settings. Changing them can alter both the cost and the output, so a demonstration produced under one configuration says little about another without a check.
A useful independent experiment could hold the starting scene constant and vary just one proposed action. It could repeat generation with different random seeds, then compare the outcomes against recordings of the same controlled situation. This is a proposed test, rather than a reported independent validation of PAN 2. It would reveal whether the model responds consistently to the variable the researcher intends to change. It could also show whether occasional plausible clips conceal a wider pattern of unstable results. Reporting both successes and failures would help another team decide which tasks merit closer investigation.
For robotics, the next step would connect a prediction to a measurable action under controlled conditions. A research team might initially use the model to suggest hypotheses or to expose alternative plans for human review. Allowing its predictions to determine a movement would add the need to validate control, perception and recovery from error. A model can be valuable during exploration before there is evidence for that later role. The public release makes it possible to examine the boundary experimentally, with the generated video treated as a prediction whose fidelity has to be established for the task at hand.
Learning what a simulated future can support
Longer planning introduces an additional difficulty. If the system uses its own generated scene as the starting point for the next prediction, a small error can become part of the assumed world. A misplaced object or a missed contact could change every later choice. In an illustrative planning trial, a researcher could measure how performance changes with the number of simulated steps and compare that with a system refreshed by real observations. The difference would help identify when generated history stops being a dependable basis for another decision.
PAN 2 brings a concrete, inspectable implementation to this research direction. The immediate contribution is an opportunity to test how structured action descriptions and visual prediction work together. The consequential evidence will come from tasks where the model must preserve a physical relationship, respond correctly to an intervention and expose uncertainty when the observation is insufficient. Those tests can establish where imagining an outcome helps an agent reason, and where the agent needs another observation or a different kind of simulator before it can choose an action reliably.
Image credit: Universal Robots UR3 at CeDRI, Bragança by Caio.st.oliveira, CC BY-SA 4.0. Illustrative photograph unrelated to PAN 2.
