AE for SHAPE

Fetch Datasets

Kronecker Graphs

./fetch_datasets/gen_kronecker_data.sh 16

generates a kronecker graph with scale 16.

Law Graphs

First download and install boost library:

./third_party/download_boost.sh
cd third_party/boost_1_84_0/
./bootstrap.sh
./b2

Download and preprocess LAW dataset:

./fetch_datasets/law_misc/fix_webgraph.sh
./fetch_datasets/get_law_data.sh imdb-2021

OGB Graphs

mkdir -p datasets/dgl_dataset
conda env create -f environment.yml
python3 fetch_datasets/get_ogb_data.py --dataset ogbn-arxiv

Preprocess Datasets

Convert datasets to uniform formats

First install cnpy library:

cd third_party/cnpy
mkdir build
cd build
cmake .. -DCMAKE_INSTALL_PREFIX=../install
make
make install

Then back to main folder:

mkdir build
make

Kronecker Graphs

mkdir -p datasets/uni_dataset
./build/kronecker_to_uni kronecker_16

Law Graphs

mkdir -p datasets/uni_dataset
./build/law_to_uni imdb-2021

OGB Graphs

mkdir -p datasets/uni_dataset
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:./third_party/cnpy/install/lib
./build/dgl_to_uni ogbn-arxiv

Graph Partition

Build METIS partitioner

cd third_party/METIS
make config
make

mkdir -p datasets/metis_dataset
python3 scripts/reduce_part.py --dataset ogbn-arxiv --percent 0.01 --method metis --part 4
python3 scripts/reduce_part.py --dataset kronecker_16 --percent 0 --method random --part 2

Evaluation

Communication Volume Statistics

./build/comm_stat_tri ogbn-arxiv.0.01 ./datasets/metis_dataset/ogbn-arxiv.0.01.graph.part.4
./build/comm_stat_opt kronecker_16.0 ./datasets/uni_dataset/kronecker_16.0.graph.random.2