Skip to content
BranchesTags

Repository files navigation

Exotic-Higgs-Study

HF dataset

This repository contains the code and data for the Exotic Higgs Study, which investigates potential new physics phenomena in the Higgs boson sector.

Installation

To set up the environment for this study, follow these steps:

git clone https://github.com/EveNet-HEP/Exotic-Higgs-Study.git
cd Exotic-Higgs-Study
git clone --recursive https://github.com/EveNet-HEP/EveNet-Full.git

To run the code, we require the following dependencies:

  • Python 3.12 or higher
  • EveNet-Full
conda create --prefix [path] python=3.12
conda activate [path]
pip3 install -r requirements.txt

Set up the environment variables for EveNet-Full, to enable wandb logging, please change the WANDB_API_KEY in src.sh to your own API key from Weights & Biases. Then, source the environment variables:

# This should be run whenever you start a new terminal session to ensure the environment variables are set correctly.
source src.sh

Pull the Docker image for the EveNet training:

shifterimg -v pull docker:avencast1994/evenet:1.5

Install SPANet [Optional]

To install SPANet, we have provided a SPANet repository in our git repo. This is folked from the original SPANet repository with a minor change to adopt event weights, which is needed in our tasks.

git clone https://github.com/UW-EPE-ML/SPANet.git

Download Pre-trained Models

To download the pre-trained models from Hugging Face Hub, you can use the following command (local-dir could be replaced with any path you want)

hf download Avencast/EveNet --local-dir pretrain-weights
# nominal ckpt: checkpoints.20M.a4.last.ckpt
# SSL ckpt: SSL.20M.last.ckpt

Dataset prepare [Hugging Face]

The dataset used in this study is available on Hugging Face. You can access it using the following link: Exotic Higgs Dataset on Hugging Face (Note: The dataset is large, so ensure you have sufficient storage space before downloading. --local-dir option allows you to specify the directory where the dataset will be stored.)

🔹 Download full dataset

hf download Avencast/EveNet-ExoticHiggs-H2a4b \
  --repo-type dataset \
  --local-dir database

🔹 Download a subset of the dataset (e.g., certain mass points)

hf download Avencast/EveNet-ExoticHiggs-H2a4b \
  --include "evenet-train/evenet-ma30/**" \
  --include "evenet-test/evenet-ma30/**" \
  --repo-type dataset \
  --local-dir database

Script Preparation

The training and evaluation scripts are controlled by config/workflow.yaml. You can modify the parameters in this file to customize the training and evaluation process.

working_dir: [PATH_TO_EveNet_WORKING_DIRECTORY]
spanet_dir: [PATH_TO_SPANet_WORKING_DIRECTORY]
train_yaml: train.yaml [relative path under configs dir] # don't need to change if you use the provided train.yaml
predict_yaml: predict.yaml [relative path under configs dir] # don't need to change if you use the provided predict.yaml
process_json: process.json [relative path under configs dir] # don't need to change if you use the provided process.json
stat_yml: Statistics_Test.yml [relative path under configs dir] # don't need to change if you use the provided Statistics_Test.yml
account: # your project account
email: # your email account
time: 04:00:00 # job time
job_flavor: regular # job flavor
spanet:
  project: ExoticHiggsDecay # wandb project name

pretrain_choice:
  scratch:
    path: null 
    option: options.yaml # use this option file if training from scratch
  pretrain:
    path: [your pretrained model path, e.g., pretrain-weights/checkpoints.20M.a4.last.ckpt]
    option: options_pretrain.yaml # use this option file if training from a pretrained model

assign_seg_choice:
  - [true, true] # first bool: assign, second bool: segment
  - [true, false]
  - [false, true]
  - [false, false]

dataset_size_choice:
  - 0.01 # fraction of dataset size to use for training
  - 0.03
  - 0.1
  - 0.3
  - 1.0

mass_choice:
  - 30 # signal mass of a in GeV
  - 40
  - 60

Then you can generate the workflow scripts using the following command:

python3 Make_script.py configs/workflow.yaml --store_dir database --ray_dir [tmp dir] --Lumi 300 --farm Farm

Very Important Note: The provided samples utilize 4 GPUs for running, if this is not allowed in your computer,

  • please tune following parameters in configs/train.yaml and configs/predict.yaml to fit your GPU number, otherwise the training/prediction may fail due to insufficient GPU resource.
platform:
  number_of_workers: [nGPU]
  prefetch_batches: 2
  resources_per_worker:
    CPU: 30
    GPU: 1
  • And add --spanet_gpu 1 when calling Make_script.py to make sure the generated SPANet training/prediction scripts also use 1 GPU.

Reformatting the data

To reformat the data for baseline spanet training, you can use the following command:

sh Farm/prepare-dataset.sh

This script typically performs the following steps:

python3 convert_evenet_to_spanet.py [event-info.yaml] --in_dir database --store_dir [target dir]

Train Evenet

All the needed training commands are saved in Farm/train-evenet.sh, please note that sequentially running all the commands in this script is very time-consuming, better to prepare batch or parallel scripts to run them.

Details

This file basically contains the following command template for training EveNet:

cd /global/u1/t/tihsu/Exotic-Higgs-Study/EveNet-Full; \
  shifter --image=docker:avencast1994/evenet:1.5 python3 scripts/train.py [train yaml] --ray_dir [tmp dir]

Predict EveNet (on Test Dataset)

Similarly, all the needed prediction commands are saved in Farm/predict-evenet.sh, please note that sequentially running all the commands in this script may also be time-consuming, better to prepare batch or parallel scripts to run them.

Details

This file basically contains the following command template for predicting with EveNet:

cd /global/u1/t/tihsu/Exotic-Higgs-Study/EveNet-Full; \
 shifter --image=docker:avencast1994/evenet:1.5 python3 scripts/predict.py [predict yaml]

Train SPANet [Optional]

All the needed training commands for SPANet are saved in Farm/train_spanet.sh.

Details

This file basically contains the following command template for training SPANet:

cd SPANet; \
 python3 -m spanet.train --event_file [event yaml] \
  -tf [test file] \
  --options_file options_files/exotic_higgs_decay/full_training-cls.json \
  --log_dir [log dir] \
   --run_name [name] \
   --epochs 50 --gpus 4 --limit_dataset 10.0 -b 2048 --project [project name]

Predict SPANet [Optional]

All the needed prediction commands for SPANet are saved in Farm/predict_spanet.sh.

Details

This file basically contains the following command template for predicting with SPANet:

cd SPANet; \
 python3 -m spanet.predict --event_file [event yaml] \
  -tf [test file] \

Evaluate the results

To evaluate the results, you can use the following command:

sh Farm/summary.sh

Details

Basically, this script performs the following steps:

# Produce histograms for the signal and background predictions
python3 Produce_ntuple.py /global/u1/t/tihsu/Exotic-Higgs-Study/configs/workflow.yaml --store_dir /pscratch/sd/t/tihsu/test_exotic/ --farm Farm
# For SPANet evaluation, add the network argument to specify the model used for evaluation
python3 Produce_ntuple.py /global/u1/t/tihsu/Exotic-Higgs-Study/configs/workflow.yaml --store_dir /pscratch/sd/t/tihsu/test_exotic/ --farm Farm --network spanet

And run the evaluation script to calculate the limits:

python3 Statistics_test.py \
 --Lumi 300 \
 --signal all \
 --process_json configs/process.json \
 --sourceFile [outdir]/ntuple/[tag]/ntuple.root \
 --observable MVAscoreMASS \
 --config_yml configs/Statistics_Test.yml \
 --outdir [outdir]/fit/[tag] --log_scale &

Summarize the results

To summarize the results, you can use the following command:

python3 Summary_Limit.py --store_dir [database dir]]

Output Format

# The output of the evaluation will be stored in the following format:

# metrics: BackgroundRejection@certainSignalEfficiency, AUC, etc.
[database dir]/fit-summary/None_summaries_[metrics].json

# Distributions plots/histograms for signal and background predictions
[database dir]/fit/[method]-assignment-[on|off]-segmentation-[on|off]-dataset_size[fraction]/haa_ma[mass]/*


# More details for further analysis, such as ROC curves, AUC values, background rejection rates, maxSIC, etc.
[database dir]/fit/[method]-assignment-[on|off]-segmentation-[on|off]-dataset_size[fraction]/summary/roc_results.npz

# The `roc_results.npz` file will contain the following structure:
#output[signal][region] = {
#            "AUC": roc_auc,
#            "FPR-unc": fpr_unc,
#            "FPR": fpr,
#            "TPR": tpr,
#            "BackgroundRejection": bkg_rejections,
#            "BackgroundRejection-unc": bkg_rejections_unc
#            "SIC": sic,
#            "SIC-unc": sic_unc,
#        }

Computing Estimates

The experiments in this work were performed on a computing cluster with NVIDIA A100 40GB GPUs.

Hardware Configuration Estimated Runtime Notes
4 x NVIDIA A100 40GB GPU 20~50 mins / training Cluster setup used in this work. Assume full dataset

It is important to note that this table reflects the cost of a single training/prediction run only. The full study reported in the paper involves multiple trainings for different settings and dataset size, and therefore requires substantially more total compute.

The actual runtime may vary depending on:

  • data loading and I/O performance,
  • software environment (CUDA, PyTorch, etc.),
  • mixed precision settings,
  • batch size and gradient accumulation. Due to the smaller GPU memory on consumer hardware compared to A100 40GB, reproducing the training may require reducing the per-device batch size, which can further increase the runtime.

About

No description, website, or topics provided.

Resources

Readme
Activity
Custom properties

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages