🦾 VLASH-Piper: VLA Fine-Tuning & Deployment for PIPER Arm (Best Project Runner Up)

Fine-tune π0.5 on your AgileX PIPER robotic arm and deploy with real-time async inference — powered by VLASH and LeRobot.

---

📖 About

This workspace combines two git submodules into a single Pixi-managed workflow for data collection, fine-tuning, and deployment of VLA policies on a PIPER robotic arm running on Jetson AGX Orin.

Submodule	Role	Original Repo
vlash/	Async VLA fine-tuning & inference framework	mit-han-lab/vlash
lerobot_piper/	LeRobot adapted for PIPER arm (CAN bus)	WeGo-Robotics/lerobot_piper

🤗 Hugging Face

Type	Link
Dataset	Frieddeli/vlash
Model	Frieddeli/vlash

---

📁 Project Structure

vlash-piper/
├── data/                    # LeRobot datasets (place datasets here)
├── docs/                    # documentation & course report
│   ├── index.md
│   ├── guide.md
│   ├── integration.md
│   ├── operation_guide.md
│   ├── problems.md
│   └── report/
│       ├── report.qmd       # Quarto source
│       ├── report.pdf       # rendered PDF
│       ├── references.bib
│       ├── system_diagram.png
│       └── inference_timeline.png
├── media/                   # demo video & hardware photos
│   ├── demo.mp4
│   ├── view-1.jpg
│   ├── view_3.jpg
│   └── view_4.jpg
├── models/                  # model checkpoints (output_dir)
├── scripts/                 # helper shell scripts
│   ├── activate.sh
│   ├── init_orin_can.sh
│   ├── record.sh
│   └── teleop.sh
├── lerobot_piper/           # submodule — LeRobot fork for PIPER arm
├── vlash/                   # submodule — VLASH async inference framework
├── pixi.toml                # Pixi task definitions & dependencies
└── zensical.toml            # Zensical docs site config

---

🚀 Quick Start

1. Install Pixi (one-time):

curl -fsSL https://pixi.sh/install.sh | bash

2. Clone and install:

git clone --recursive https://github.com/Alvin0523/vlash-piper.git
cd vlash-piper
pixi install

Already cloned without --recursive? Run git submodule update --init --recursive first.

👉 For hardware setup, data collection, training, inference, and Pixi task reference — see the Operation Guide.

---

🎬 Demo

demo.mp4

π0.5 deployed on AgileX PIPER via async VLASH inference on Jetson AGX Orin.

📷 Setup

---

🏗️ System Architecture

Left: RealSense cameras feed RGB frames into the VLASH Runtime on Jetson AGX Orin. The runtime schedules async inference calls to the PI0.5 Model (flow-matching VLA, PaliGemma backbone, bfloat16) with a lookahead delay d, streams the resulting action chunk to lerobot_piper, which drives the Piper follower arm over CAN bus at 30 Hz.

---

📚 Citation

If this workspace is useful to you, please cite the underlying frameworks:

VLASH (async VLA inference framework):

@article{tang2025vlash,
  title   = {VLASH: Real-Time VLAs via Future-State-Aware Asynchronous Inference},
  author  = {Tang, Jiaming and Sun, Yufei and Zhao, Yilong and Yang, Shang and
             Lin, Yujun and Zhang, Zhuoyang and Hou, James and Lu, Yao and Liu, Zhijian and others},
  journal = {arXiv preprint arXiv:2512.01031},
  year    = {2025},
  url     = {https://arxiv.org/abs/2512.01031}
}

LeRobot (robot learning framework):

@misc{cadene2024lerobot,
  author       = {Cadene, Remi and Alibert, Simon and Soare, Alexander and Gallouedec, Quentin and
                  Zouitine, Adil and Palma, Steven and Kooijmans, Pepijn and Aractingi, Michel and
                  Shukor, Mustafa and Aubakirova, Dana and Russi, Martino and Capuano, Francesco and
                  Pascal, Caroline and Choghari, Jade and Moss, Jess and Wolf, Thomas},
  title        = {LeRobot: State-of-the-art Machine Learning for Real-World Robotics in Pytorch},
  howpublished = {\url{https://github.com/huggingface/lerobot}},
  year         = {2024}
}

---

👥 Authors

HKUST COMP4901-D Embedded AI — Course Project

Name	GitHub
Wong Wei Ming	@Alvin0523
Shao Ying Zhan	@frieddeli
Chen Yusen	@HappyEthan