NeurIPS 2026

Policy-DRIFTDynamic Reward-Informed Flow Trajectory Steering

Atharva Mahajan*†, Abhijeet Vishwasrao*†, Yuning Wang, Ricardo Vinuesa* University of Michigan, Ann Arbor. †Equal contribution. *Corresponding authors.

A generative model of turbulent flow, steered by reward at inference, gives a DRL policy targets to track for skin-friction drag reduction.

Streamwise velocity fluctuations in a turbulent channel, from Figure 2 of the paper
Table 1Reτ = 180
48.95% drag reduction, ~37× less actuation energy than TD3-WSE

Reward design and the DRL ceiling

Skin-friction drag accounts for roughly half of the total aerodynamic drag on commercial aircraft. DRL is the leading approach for real-time flow control, but a scalar reward is a proxy for what optimal control looks like, and the proxy limits the policy.

Policy-DRIFT moves reward information from policy gradients to generative model inference. When the control objective changes, the CFM model needs no retraining and only the reward predictor is updated.

Figure 1, top: applications, the paths taken by opposition control, WSE and Policy-DRIFT through flow states, and bars comparing their actuation energy and drag reduction

Policy-DRIFT in 3 steps

1

Conditional flow matching

One conditional flow matching model learns uncontrolled, opposition-controlled and DRL-controlled DNS from 7,350 snapshot pairs per regime, with no regime labels.

2

Terminal Reward Guidance

Terminal Reward Guidance moves ODE trajectories toward terminal states that maximise DR − Eact, using the gradient of a learned reward predictor.

3

Tracking policy

A TD3 policy minimises RMSE to the guided target over each 8-step horizon. Once deployed it senses one wall-parallel plane and queries no generative model.

Figure 1, bottom: CFM trained on opposition control and WSE data, Terminal Reward Guidance with the reward predictor, and the agent tracking the target by policy exploration

Terminal Reward Guidance

TRG corrects each ODE state before the velocity step, and the velocity model then maps the nudged state back toward the physical flow manifold. Applying the same correction after the step causes reward hacking: drag reduction clamps at 1.0 within the first few guidance steps, on physically unrealisable states.

On held-out uncontrolled snapshots, guided rollouts raise drag reduction by ΔDR ≈ 0.026 to 0.030 at comparably low actuation energy. The perturbation is two orders of magnitude smaller than the velocity fluctuations.

ũs+ = ũs + γ δs ∇Rψ(ũs, s)

ũs+δs = ũs+ + δs · vθ(ũs+, s, u0)

Figure 2, first snapshot: streamwise velocity fluctuations for a held-out uncontrolled snapshot, showing the input, the CFM state, the TRG state and their difference

Closed-loop results in Table 1

Policy-DRIFT reaches 48.95% drag reduction, 16.2% higher than TD3-WSE with ~37× less actuation energy, and 38.9% higher than opposition control at comparable energy. The policy optimises neither quantity directly.

MethodDR (%)Eact (×10−1)R̄
Training benchmarks
Opp-Control35.23 ± 10.090.0685 ± 0.02410.345
TD3-WSE42.13 ± 12.713.692 ± 0.04310.052
Experiments
Policy-DRIFT48.95 ± 11.700.0990 ± 0.07110.480
TD3-WEN42.86 ± 11.770.0609 ± 0.04060.423
TD3-WSE42.13 ± 12.713.692 ± 0.04310.052
SAC-WSE32.31 ± 2.330.1193 ± 0.02280.311
Figure 3: drag reduction and actuation energy over time for Policy-DRIFT, TD3-WSE, TD3-WEN, SAC-WSE and opposition control

Joint PDFs at the sensing plane

At y+ ≈ 15, the ejection and sweep tails of the uncontrolled flow show how each controller acts on the near-wall cycle. TD3-WSE suppresses these tails with a bimodal v′+ reaching ±1, a sign of over-actuation.

TD3-WEN recovers a compact v′+ and weakens the suppression. Policy-DRIFT has the most compact distribution, with the strongest attenuation of both quadrants.

Figure 4: joint PDFs of streamwise and wall-normal velocity fluctuations for the uncontrolled flow and each controller

Code, checkpoints and citation

The repository holds the trained CFM model, reward predictor and DRL policies, with DNS restart files and sample snapshots.

# Install (Python 3.11)
conda create -n policy-drift -c conda-forge \
  python=3.11 dedalus=2.2207.3 mpi4py=3.1.4
conda activate policy-drift
pip install -r requirements.txt
pip install -e .
# Two epochs of CFM and reward-predictor training
cd flow_matching_trg
python train_cfm.py --dummy
python train_trg.py --dummy
# Two 20-action episodes of Policy-DRIFT
cd ../drl_pipeline
python train.py --reward trg --dummy
@article{mahajan2026policydrift,
  title   = {Policy-{DRIFT}: Dynamic Reward-Informed Flow Trajectory Steering},
  author  = {Mahajan, Atharva and Vishwasrao, Abhijeet and Wang, Yuning and Vinuesa, Ricardo},
  journal = {arXiv preprint arXiv:2605.14022},
  year    = {2026}
}
Get the code