PAN 2
NexusAi Summary: PAN 2 is a research-grade world model that converts structured text and optional images into short simulated videos, letting agents and researchers preview action outcomes through integrated conceptual reasoning and a latent diffusion perceptual backbone.
NexusAi Overview
Typical workflow runs a conceptual reasoner to rewrite free-text prompts into structured, time-grounded captions, then executes the perceptual pipeline on those captions to produce video. The repository includes scripts to convert checkpoints, generate prompts, and run per-GPU generate.py workers that produce mp4 outputs alongside config metadata for reproducibility.
Capabilities and technical scope
PAN 2 is primarily aimed at AI researchers, robotics teams, and technical product builders who need simulative predictions rather than final production video. It is useful for prototyping agent planning loops, benchmarking physical reasoning, and creating synthetic footage for evaluation, curriculum design, or controlled user studies where interpretability and a structured conditioning format are required.
- Simulates outcomes of text-described actions as short videos up to twenty seconds.
- Accepts an optional first frame for image-to-video continuation and context.
- Produces structured, time-grounded captions used to condition the diffusion backbone.
- Integrates prompt rewriting to fit a fixed text-encoder token budget and format.
- Ships a single diffusion checkpoint with instructions for local diffusers-based inference.
Why engineers choose PAN 2
Who should evaluate it
Follow the repository README to download the diffusion checkpoint and fetch frozen components (Qwen3.5-9B text encoder, Wan 2.2 VAE) via HF cache. Convert transformer files with the provided script, set CKPT_PATH to the converted directory, and run inference/inference.sh or generate.py per GPU. Default sampler settings and environment variables are documented; run prompt rewriting before generation and deploy the pan2-guardrail to screen requests and outputs.
NexusAi: A practical research tool for simulative reasoning and action-conditioned visual prediction.
Getting started
PAN 2 is a research-oriented simulator that prioritises interpretable, conditional predictions over polished production video. Its strength lies in coupling a structured conceptual reasoner with a large diffusion perceptual backbone, making it suitable for agent planning research, controlled datasets, and studies of physical reasoning when used with the recommended guardrails.
Open the tool and review its core product experience.
Create your account or access your existing workspace.
Use your own task to judge speed, quality, and fit.
Check similar AI tools before making a final decision.
PAN 2
The Institute of Foundation Models (IFM) at MBZUAI focuses on foundation models for multimodal understanding and simulation. IFM develops research code, checkpoints, and documentation aimed at reproducible experiments in world modelling and agent reasoning.
PAN 2 pairs a language-driven conceptual reasoner with a 29B latent diffusion perceptual backbone to simulate action-conditioned video outcomes up to 20 seconds. Given an observation (optional first frame plus text) and a text-described action plan, the conceptual module emits a time-grounded structured world-state caption; the diffusion backbone then synthesizes video latents decoded by a dedicated VAE. The released artifacts include the diffusion checkpoint in diffusers layout and instructions for local generation with generate.py, plus guidance on prompt rewriting, guardrails, and precision/step defaults. PAN 2 targets simulative reasoning, robot planning research, and controlled video synthesis where interpretable, checkable predictions matter more than photorealistic fidelity.
Visit PAN 2
Comments (0)
No Comments Found