Release MLGYM-Bench dataset on Hugging Face - decouple benchmark from MLGym environments

[NielsRogge]

Hi @deepakn97 🤗

I'm Niels from the open-source team at Hugging Face. I discovered your work on MLGym through Hugging Face's daily papers: https://huggingface.co/papers/2502.14499. The paper page lets people discuss your work and find related artifacts. You can also claim the paper, which will show up on your public Hugging Face profile.

It'd be great to make the MLGYM-Bench benchmark available on the 🤗 hub, to improve its discoverability and visibility. We can add tags so that people can easily find it when searching datasets.

While MLGYM-Bench might not be a dataset in the traditional sense, it represents a curated set of tasks and evaluation metrics, which are essential for evaluating research agents. Hosting a structured description of MLGYM-Bench on Hugging Face, alongside the associated code on Github, would make it easier for others to understand, use, and contribute to the benchmark. This could include details on the tasks, evaluation metrics, and the datasets or models used within each task.

Would you be interested in hosting MLGYM-Bench on https://huggingface.co/datasets? Hosting on Hugging Face provides several benefits:

• Increased visibility and discoverability: Make it easier for other researchers to find and use your benchmark.
• Structured format: Provide a clear and organized way to present the benchmark details, including tasks, metrics, and associated resources.
• Version control: Track changes and updates to the benchmark over time.
• Community engagement: Foster discussion and collaboration around the benchmark.

Here's a guide on how to upload a dataset to Hugging Face: https://huggingface.co/docs/datasets/loading.

After uploading, we can link the dataset to the paper page (read here), further increasing its visibility.

Let me know if you're interested or need any help with this!

Cheers,

Niels
ML Engineer @ HF 🤗

[deepakn97]

Hi @NielsRogge

Apologies for the late reply.
Yes, we do plan to host the MLGym-Bench dataset on huggingface. Here is how I was planning to store the data:

Dataset Structure:
The MLGym-Bench dataset includes some raw data files (.zip, csv, etc.), associated code (a bunch of files ), and .yaml config files.

Desired Functionality
Once we have the data up on HuggingFace, I want to have a custom loading script that downloads all the associated files and places them in the correct directory structure in the codebase.

Please let me know if this is possible. If you have any example datasets with a similar structure, that would also be good.

Cheers,
Deepak

[NielsRogge]

Great! Yes this is possible. You can either make the data directly compatible with the Datasets library (for things like zip and csv), or you can use the web interface to just upload the raw files.

[deepakn97]

Thank you! This sounds great, I will work on it.