Murat Ertas - Model Functionality Testing with CHECKLIST method

Included files and directory structure:

• exam_MURAT_ERTAS folder -dataset folder -bert_input folder (contains the input files for BERT) -challenge_dataset.json (the dataset for the test)

  -scripts folder -predictions folder (contains the predictions for the test) -BERT_prediction.py (the script for BERT prediction) -evaluation.py (the script for evaluation) -preprocess.py (the script for preprocessing) -requirements.txt

Instructions:

• To run the code, you need to have the required libraries installed. You can install the required libraries by running the following command in the terminal:

  • pip install -r requirements.txt

• The fine-tuned BERT models can be downloaded from the following links (both are the same, you can choose one of them):

  • https://drive.google.com/drive/folders/1Ml1bm-rHBeSFapnSEszUqA9EClUAVYRq?usp=sharing
  • https://www.dropbox.com/scl/fo/xv6pkmvqfs4eaptr0aw9i/h?rlkey=jk8ggqbkrngjclxduq2fod3dy&dl=0
  • After downloading the models, you need to link their paths in BERT_prediction.py script.

• To run the preprocessing script, you need to run the following command in the terminal:

  • python preprocess.py
  • This script will create the input files for BERT models in the bert_input folder.
  • This script reads json files from the 'dataset' folder and creates the input files for BERT.

• To run the BERT prediction script, you need to run the following command in the terminal:

  • python BERT_prediction.py
  • This script will create the predictions for the test in the predictions folder.
  • Downloaded model paths should be linked in the script.
  • This script reads the preprocessed files from the 'bert_input' folder and make predictions, then saves predictions in the 'predictions' folder.

• To evaluate the predictions, you need to run the following command in the terminal:

  • python evaluation.py
  • This script will evaluate the predictions and print the results.
  • This script reads the predictions from the 'predictions' folder and evaluates them.