DeepCGP: A Deep Learning Method to Compress Genome-wide Polymorphisms for Predicting Phenotype of Rice
A Deep Learning Compression-based Genomic Prediction (DeepCGP) model that can compress genome-wide polymorphism data and predict phenotypes of a target trait from compressed information was proposed. The DeepCGP model contained two parts: (i) an autoencoder model based on deep neural networks to compress genome-wide polymorphism data, and (ii) regression models based on random forests (RF), genomic best linear unbiased prediction (GBLUP), and Bayesian variable selection (BayesB) to predict phenotypes from compressed information. Two datasets with genome-wide marker genotypes and target trait phenotypes in rice were applied. The DeepCGP model obtained up to 99% prediction accuracy after 98% compression. BayesB required extensive computational time among the three methods, and showed the highest accuracy; however, BayesB could only be used with compressed data. Overall, DeepCGP outperformed state-of-the-art methods in terms of both compression and prediction.
Availability of Datasets: You can access the dataset from here.
For the implementation, we use PyCharm IDE.
C7AIR and HDRA folders contain the scripts for the experiments on the two different datasets, respectively.
1. Data Files
The C7AIR_Data folder contains six subfolders:
a. C7AIR_Raw: This folder contains the original genotype and phenotype data of C7AIR dataset.
b. C7AIR_OneHotEncode: This folder contains processed one hot encoded data of genotype.
c. C7AIR_ModelMetaData: This folder contains the generated data after training autoencoder networks.
d. C7AIR_CompressedData: This folder contains the combined compressed data achieved from different compression levels.
e. C7AIR_Prediction_Result: This folder contains the predicted result after severel compression.
f. C7AIR_Prediction_Accuracy: This folder contains the prediction accuracy plot.
The C7AIR_Src_Python folder contains python source code of DeepCGP model. The C7AIR_Src_R folder contains R source code of DeepCGP model.
2. Preprocessing Data
a. As the first step, we load all the required library and preprocess the C7AIR_Genotype.csv data using the python script C7AIR_DataProcessing.py. For preprocessing, we split the data into several files.
b. We save all the encoded data into C7AIR_OneHotEncode folder.
3. DeepCGP Model Training
a. The C7AIR_Net_1.py trains autoencoder network and records the compressed data in C7AIR_Net_1_EncData folder.
b. Then we take the Net_1 compressed data as input and train C7AIR_Net_2.py network and records the compressed data in C7AIR_Net_2_EncData folder.
c. We also record compression time and loss.
d. We combine the C7AIR_Net_1.py compressed data as C7AIR_Net_1_CompressedData.csv and C7AIR_Net_2.py compressed data as C7AIR_Net_2_CompressedData.csv using C7AIR_CombineCompressData.py.
e. Then we map the combined compress data with phenotype using C7AIR_CompressData_MappingwithPheno.py. \
C7AIR_ModelMetaData folder contains all the model and compressed files generated by DeepCGP.
4. Prediction Models
a. C7AIR_Src_R folder contains R scripts to predict the phenotypes.
b. At first, we generate 0, 1 score from the original genotype data using C7AIR_convATGC201.Rmd. We also generate Phenotype CVID using C7AIR_Phenotype_CVID.Rmd.
c We use ranger package in R for prediction using Random Forest.
d. C7AIR_Ranger_Raw.Rmd and C7AIR_Ranger_Compressed.Rmd predict phenotype from the original genome-wide polymorphism data and compressed genome-wide polymorphism data, respectively.
5. Prediction Accuracy Plot
We calculate the relative accuracy for different compression levels and generate a plot using C7AIR_Ranger_Prediction_Plot.py.
1. Data Files
The HDRA_Data folder contains six subfolders:
a. HDRA_Raw: This folder contains the original genotype and phenotype data of HDRA dataset.
b. HDRA_OneHotEncode: This folder contains processed one hot encoded data of genotype.
c. HDRA_ModelMetaData: This folder contains the generated data after training autoencoder networks.
d. HDRA_CompressedData: This folder contains the combined compressed data achieved from different compression levels.
e. HDRA_Prediction_Result: This folder contains the predicted result after severel compression.
f. HDRA_Prediction_Accuracy: This folder contains the prediction accuracy plot.\
The HDRA_Src_Python folder contains python script of DeepCGP model. The HDRA_Src_R folder contains R script of DeepCGP model.
2. Preprocessing Data
a. As the first step, we load all the required library and preprocess the HDRA_Genotype.csv data using the python sript HDRA_DataProcessing.py. For preprocessing, we split the data into several files.
b. We save all the encoded data into HDRA_OneHotEncode folder.
3. DeepCGP Model Training
a. The HDRA_Net_1.py trains autoencoder network and records the compressed data in HDRA_Net_1_EncData folder.
b. Then we take the Net_1 compressed data as input and train HDRA_Net_2.py network and records the compressed data in HDRA_Net_2_EncData folder.
c. We also record compression time and loss.
d. We combine the HDRA_Net_1.py compressed data as HDRA_Net_1_CompressedData.csv, HDRA_Net_2.py compressed data as HDRA_Net_2_CompressedData.csv and HDRA_Net_3.py compressed data as HDRA_Net_3_CompressedData.csv using HDRA_CombineCompressData.py.
e. Then we map the combined compress data with phenotype using HDRA_CompressData_MappingwithPheno.py.
HDRA_ModelMetaData folder contains all the model and compressed files generated by DeepCGP.
4. Prediction Models
a. HDRA_Src_R folder contains R scripts to predict the phenotypes.
b. At first, we generate 0, 1 score from the original genotype data using HDRA_convATGC201.Rmd. We also generate Phenotype CVID using HDRA_Phenotype_CVID.Rmd.
c We use ranger package in R for prediction using Random Forest.
d. HDRA_Ranger_Raw.Rmd and HDRA_Ranger_Compressed.Rmd predict phenotype from the original genome-wide polymorphism data and compressed genome-wide polymorphism data, respectively.
5. Prediction Accuracy Plot
We calculate the relative accuracy for different compression levels and generate a plot using HDRA_Ranger_Prediction_Plot.py.