FluidGen: Physics-Aware Fluid Video Generation via Motion-Guided Post-Training

Yang Zheng*, Zizhang Li*, Boyang Deng*, Jan Ackermann, Lei Shu, Yinghao Xu,
Jiajun Wu, Gordon Wetzstein, Leonidas Guibas
Stanford University
*Equal contribution
Paper arXiv Video Code (soon) Data (soon)
Generated videoVelocity inferred from RGB

FluidGen generates realistic fluid dynamics together with their 3D velocity fields.

Abstract

Video generation models produce visually compelling motion but systematically fail on fluid phenomena, including smoke, fire, and explosion debris, where outputs lack coherent vortical structure and collapse to bland dynamics that violate the physics of real fluids. Classical solvers achieve high physical fidelity but require detailed 3D scenes and carefully tuned parameters, making them impractical for in-the-wild imagery and video generation. We address this gap by injecting a data-driven fluid prior into a pre-trained video model, trained on a mixture of high-fidelity Houdini simulation data with paired velocity-field ground truth and curated in-the-wild fluid video data for appearance diversity. The core of our approach is a dual-stream architecture that predicts per-frame velocity fields alongside RGB, supervised by simulation ground truth. This auxiliary signal regularizes generation toward physically consistent motion and exposes an interpretable physical quantity that we further leverage as a reward for velocity-aware direct preference optimization. Our model produces substantially more realistic fluid dynamics sequences than strong video baselines and optimization-based methods.

Method

FluidGen overview
Overview of FluidGen. We augment a pretrained video diffusion model with a velocity prediction stream and train it in two stages: joint supervised fine-tuning on simulated and in-the-wild fluid videos, followed by velocity-aware DPO using physics-informed rewards.
1

Velocity as a video

Per-pixel 3D velocity is encoded as color via spherical coordinates, so it reuses the pretrained VAE.

2

Dual-head joint diffusion

One shared Wan2.1 denoiser predicts RGB and velocity jointly, with zero-initialized velocity channels.

3

Two-stage post-training

Joint fine-tuning on simulated and real videos, then DPO with a physics reward computed on predicted velocity.

Shape Transitions, with Generated Velocity

Guided by keyframes, FluidGen sculpts smoke into animals while keeping the motion physically plausible. Switch between the generated RGB, the generated velocity encoded as color, and the velocity visualized as arrows.

Smoke → Horse

Horse → Cat

Horse → Winged dragon

Effect of Velocity-Aware DPO

Same input image and seed. The base model Wan2.1-I2V-14B produces unnatural fluid motion with appearance drift, joint fine-tuning (w/o DPO) makes the dynamics coherent, and DPO adds richer fine-scale detail and more vivid fire and smoke motion.

Wan2.1-I2V-14B FluidGen w/o DPO FluidGen (Ours)

Applications

Velocity estimation from real videos

Given a real captured video as the RGB stream, RePaint-style conditional sampling lets FluidGen infer the matching 3D velocity field. The recovered field shows coherent advection consistent with the observed smoke.

Real inputInferred velocityArrow visualization

Driving fluid simulation

The generated velocity fields are physically meaningful enough to be imported directly into Houdini and used to simulate a new fluid source.

FluidGen RGBFluidGen velocityHoudini simulation

Gallery

Image-to-video results generated by FluidGen on in-the-wild images of fire, smoke, explosions, and ink.

BibTeX

@article{zheng2026fluidgen,
  title   = {FluidGen: Physics-Aware Fluid Video Generation via Motion-Guided Post-Training},
  author  = {Zheng, Yang and Li, Zizhang and Deng, Boyang and Ackermann, Jan and Shu, Lei
             and Xu, Yinghao and Wu, Jiajun and Wetzstein, Gordon and Guibas, Leonidas},
  journal = {arXiv preprint},
  year    = {2026}
}