Skip to content

README.md

Latest commit

 

History

History
executable file
·
146 lines (108 loc) · 4.96 KB

README.md

File metadata and controls

executable file
·
146 lines (108 loc) · 4.96 KB

FreqPolicy: Frequency Autoregressive Visuomotor Policy with Continuous Tokens

NeurIPS 2025
Yiming Zhong  Yumeng Liu  Chuyang Xiao  Zemin Yang  Youzhuo Wang  Yufei Zhu  Ye Shi  Yujing Sun  Xinge Zhu  Yuexin Ma 

1ShanghaiTech University  2The University of Hong Kong
3Nanyang Technological University  4The Chinese University of Hong Kong

📖 Project Page | 📄 Paper Link |

(a) Heat maps of frequency band energy across action dimensions for different tasks.The top row shows Adroit tasks with high-dimensional actions (26 dimensions), while the bottom row presents Robomimic tasks with low-dimensional actions (10 dimensions). (b) Success rate of actions reconstructed with varying frequency ratios. We reconstruct action sequences using different proportions of frequency components and evaluate their success rates on the original tasks.

Directional Weight Score

📣 News

  • [9/20/2025] 🎉🎉🎉FreqPolicy has been accepted by NeurIPS 2025!!!🎉🎉🎉

😲 Results

Please refer to our homepage for more thrilling results!

🛠️ Setup

Our method provides separate implementations for 2D and 3D inputs, built upon existing frameworks:

📁 2D Version → Freqpolicy_2d/

Based on Diffusion Policy (DP) framework.

Installation Steps:

  1. Follow the environment setup instructions in the original DP repository

  2. Install additional dependencies:

pip install tensorboard==2.10.1
pip install huggingface-hub==0.25.2
pip install torch-dct==0.1.6

📁 3D Version → Freqpolicy_3d/

Based on Diffusion Policy 3D (DP3) framework.

Installation Steps:

  1. Follow the environment setup instructions in the original DP3 repository

  2. Install additional dependencies:

pip install tensorboard==2.14.0
pip install huggingface-hub==0.25.2
pip install torch-dct==0.1.6

🚀 Training & Evaluation

Both 2D and 3D versions support all methods from their respective original repositories.

2D Version

Navigate to the Freqpolicy_2d/ directory:

Training

cd Freqpolicy_2d

# Single run
sh train.sh

# Multi-run
sh train_multirun.sh

Evaluation

python eval.py \
  --checkpoint data/outputs/21.35.20_train_freqpolicy_lowdim_pusht_lowdim/checkpoints/xxxx.ckpt \
  -o data/pusht_eval_output

3D Version

Navigate to the Freqpolicy_3d/ directory:

Training

Step 1: Generate Training Data

cd Freqpolicy_3d
python scripts/gen_xxx.py

Step 2: Train Policy

bash scripts/train_policy.sh Freqpolicy adroit_pen 0428 0 0

Evaluation

bash scripts/eval_policy.sh Freqpolicy adroit_pen 0428 0 0

🎫 License

This project is licensed under the MIT License.

💓 Acknowledgement

We would like to acknowledge that some codes are borrowed from DP3, DP, MAR, FAR. We appreciate the authors for their great contributions to the community and for open-sourcing their code.

🖊️ Citation

@article{zhong2025freqpolicy,
  title={FreqPolicy: Frequency Autoregressive Visuomotor Policy with Continuous Tokens},
  author={Zhong, Yiming and Liu, Yumeng and Xiao, Chuyang and Yang, Zemin and Wang, Youzhuo and Zhu, Yufei and Shi, Ye and Sun, Yujing and Zhu, Xinge and Ma, Yuexin},
  journal={arXiv preprint arXiv:2506.01583},
  year={2025}
}