Fisher Decorator: Refining Flow Policy via A Local Transport Map

Xiaoyuan Cheng*, Haoyu Wang*, Wenxuan Yuan, Ziyan Wang, Zonghao Chen, Li Zeng, Zhuo Sun

Conference on Neural Information Processing Systems (NeurIPS), 2026 Poster

* Equal contribution.

Abstract

Recent advances in flow-based offline reinforcement learning (RL) have achieved strong performance by parameterizing policies via flow matching. However, they still face critical trade-offs among expressiveness, optimality, and efficiency. In particular, existing flow policies interpret the L₂ regularization as an upper bound of the 2-Wasserstein distance (W₂), which can be problematic in offline settings. This issue stems from a fundamental geometric mismatch: the behavioral policy manifold is inherently anisotropic, whereas the L₂ (or upper bound of W₂) regularization is isotropic and density-insensitive, leading to systematically misaligned optimization directions. To address this, we revisit offline RL from a geometric perspective and show that policy refinement can be formulated as a local transport map—an initial flow policy augmented by a residual displacement. By analyzing the induced density transformation, we derive a local quadratic approximation of the KL-constrained objective governed by the Fisher information matrix, enabling a tractable anisotropic optimization formulation. By leveraging the score function embedded in the flow velocity, we obtain a corresponding quadratic constraint for efficient optimization. Our results reveal that the optimality gap in prior methods arises from their isotropic approximation. In contrast, our framework achieves a controllable approximation error within a provable neighborhood of the optimal solution. Extensive experiments demonstrate state-of-the-art performance across diverse offline RL benchmarks.

Resources

Demonstration

Fisher Decorator: Refining Flow Policy via A Local Transport Map — framework and results — Poster

Geometric interpretation of policy refinement (Figure 1 of the paper, CC BY 4.0).

Citation

@inproceedings{cheng2026fisher,
  title = {Fisher Decorator: Refining Flow Policy via A Local Transport Map},
  author = {Xiaoyuan Cheng and Haoyu Wang and Wenxuan Yuan and Ziyan Wang and Zonghao Chen and Li Zeng and Zhuo Sun},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2026},
  url = {https://arxiv.org/abs/2604.17919}
}