PCREQ Evaluation

This repository contains the evaluation results of PCREQ, assessing its capability to infer compatible requirements for TPL upgrades in Python projects.

📂 Directory Overview

• Motivating-Study/
  Experimental results of our motivating study.

• RQ1/
  Evaluates PCREQ's performance on compatible requirements inference of TPL upgrades in Python projects.

  • Uses the REQBench dataset.

• RQ2/
  Compares PCREQ against PyEGo and ReadPyE on compatible requirements inference of TPL upgrades in Python projects.

  • Uses the REQBench-PyEGo and REQBench-ReadPyE datasets.

• RQ3/
  Compares PCREQ against DeepSeek (V3, R1) and ChatGPT (GPT-4o) on compatible requirements inference of TPL upgrades in Python projects.

  • Uses the REQBench dataset.

• RQ4/
  Measures PCREQ's runtime efficiency.

  • Computes the inference time of test cases.

---

📌 Notes

• Each directory contains the corresponding experiment results and analysis.
• For detailed evaluation metrics and results, please take a look at our paper.

License

PCREQ-evaluation is licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0). See LICENSE for details.