GAME was designed for the functional genomics community to create seamless communication across pre-trained models and genomics datasets. It is a product of the feedback from many model and dataset experts and our hope is that it allows for long-lasting benchmarking of models.

The Orca Predictor returns log scaled interaction matrices for chromatin conformation predictions in H1-ESC cells. The predictions are a binned matrix where each value correspond to a 4kbp-by-4kbp interaction region, for example a 1Mbp input sequence prediction will return a 250x250 matrix.

The Enformer module encapsulates the human head of the large-scale multi-task Enformer model and can make predictions for sequences of any length. Its training data includes chromatin accessibility, DNA binding and expression.

DREAM-RNN model (Rafi et al. 2024, Nature Biotechnology) trained on MPRA data from Agarwal et. al 2023 and can be used for any sequence-to-expression predictions (single task).

CpG predictor serves as a baseline for models which are correlated with many genomic readouts. It ignores the type, cell type and species keys in the Evaluator requests since it is simply counting CpG instances.

The ChromBPNet Predictor encapsulates 9 individual chromatin accessibility models (5 folds for each model, some trained with ATAC-seq data and others with DNase) across five cell types.

Trained on human RNA-seq data from ENCODE (with 866 datasets across diverse biosamples, including cell lines and adult tissues) and GTEx data (with 2-3 replicates for each tissue, processed by the recount3 project). The training dataset also includes epigenomic datasets from Enformer, such as CAGE, DNase-seq, ATAC-seq, and ChIP-seq tracks.

We designed three Evaluators requesting H1 cell interaction matrices for Chr 8/9 (Orca test set) and Chr 10 (validation set). They tile chromosomes in 1Mb steps, extracting hg38 sequences at runtime. Evaluation computes Pearson correlations between predicted and measured Hi-C matrices. These modules also demonstrate msgpack-numpy response support.

Martyn et al. dataset measured the effects of 783 variants in Jurkat T cells and THP-1 monocytes cells. This Evaluator is used to evaluate models' abilities to predict variant effects. We extract 2000bp around the indel and request log scale expression predictions for the reference and mutated sequence.

This synthetic MPRA (Gosai et al. 2024) Evaluator requests expression predictions for ~51k synthetic probes in K562 (erythroid precursors), HepG2 (hepatocytes) and SK-N-SH (neuroblastoma). The Pearson r is calculated between the measured and predicted log2FC values.