Release S0 tuned states and training data on Hugging Face

[NielsRogge]

Hi @JackYoung27 🤗

Niels here from the open-source team at Hugging Face. I discovered your work through Hugging Face's daily papers as yours got featured: https://huggingface.co/papers/2604.01168.
The paper page lets people discuss about your paper and lets them find artifacts about it (your models, datasets or demo for instance), you can also claim the paper as yours which will show up on your public profile at HF, and add Github and project page URLs.

It's great to see the s0-tuning library already available on GitHub! It's a very clever approach for hybrid architectures. I noticed that the specific tuned states (the ~48MB files mentioned in the paper) and the curated dataset of 48 verified HumanEval solutions aren't currently linked for download.

It would be awesome to make these checkpoints and the dataset available on the 🤗 hub to improve their discoverability and reproducibility.

Uploading tuned states

Since S0 states are lightweight (~48 MB), they would be perfect to host on https://huggingface.co/models. You could host them as individual repositories or as "adapters" linked to the base models (Qwen3.5/FalconH1). This would allow the community to use your tuned results out-of-the-box using your library via hf_hub_download.

Uploading dataset

Making the verified solutions available on 🤗 would also be a great resource for the community:

from datasets import load_dataset
dataset = load_dataset("your-hf-username/humaneval-s0-train")

See here for a guide: https://huggingface.co/docs/datasets/loading.

Besides that, there's the dataset viewer which allows people to quickly explore the data in the browser.

Let me know if you're interested or need any help regarding this!

Cheers,

Niels
ML Engineer @ HF 🤗

[JackYoung27]

Hi Niels, thanks for reaching out! Great to hear it got featured on Daily Papers. I'll upload both the tuned states and the training dataset to HF Hub this week. Will follow up here when they're live.

[JackYoung27]

Hi Niels, both are on the Hub now:

• States: https://huggingface.co/JackYoung27/s0-tuning-qwen3.5-4b-humaneval
• Dataset: https://huggingface.co/datasets/JackYoung27/humaneval-s0-train

The states load in two lines:

from huggingface_hub import snapshot_download
from s0 import S0Trainer

path = snapshot_download("JackYoung27/s0-tuning-qwen3.5-4b-humaneval")
trainer = S0Trainer.from_pretrained("Qwen/Qwen3.5-4B")
trainer.load(path)
trainer.activate()

The dataset contains 45 execution-verified HumanEval solutions (problems 0-79, 8 samples each, first passing kept). That's the full training set from the paper. Anyone can reproduce the +23.6 pp result by running S0Trainer.train() on it.

One thing I'd like to explore: integrating S0 as a method in the PEFT library. Recurrent layers make up 75% of hybrid models like Qwen3.5 and FalconH1, and no existing PEFT method targets them. The core implementation is ~500 lines. Happy to open an RFC or PR if that's of interest to the team.

Thanks for the suggestion and for featuring the paper on Daily Papers.

[NielsRogge]

Thanks so much for uploading both the tuned states and the training dataset to the Hugging Face Hub. The artifacts are now properly linked to the paper page at https://huggingface.co/papers/2604.01168 and mentioned in the GitHub README. This will greatly improve discoverability and reproducibility for the community.

The integration with PEFT sounds like an interesting direction. I would recommend opening a discussion on the PEFT repository (https://github.com/huggingface/peft/discussions) to get feedback from the maintainers before submitting an RFC or PR.

If you have not already done so, feel free to claim the paper page by clicking on your name on https://huggingface.co/papers/2604.01168. This will display the paper on your Hugging Face profile.

Thanks again for making these artifacts available on the Hub.