Translational Maturity Classification of Medical AI Articles

This repository contains the R code used to classify scientific articles on artificial intelligence in the medical field according to their level of translational maturity.

The workflow includes article retrieval, data parsing, model training, model evaluation, prediction on the full corpus and evidence mapping.

Requirements

This project was developed with:

• R 4.4.3
• RStudio
• renv

Package versions are stored in renv.lock.

Installation

Clone the repository:

git clone https://github.com/alix-data/trans-maturity.git
cd trans-maturity

Check that you are using R 4.4.3:

R.version.string

Install renv if needed:

install.packages("renv")

Restore the package environment:

renv::restore()

Check that the environment is synchronized:

renv::status()

Environment variables

Create a .Renviron file at the root of the project.

Add your Entrez API key inside .Renviron:

ENTREZ_KEY=your_entrez_key_here

Restart R after creating or modifying .Renviron.

You can check that the key is loaded with:

Sys.getenv("ENTREZ_KEY")

Running the project

All scripts are in the R/ folder.

Run the scripts from the root of the project, in this order:

source(here::here("R", "00_setup.R"))
source(here::here("R", "01_search.R"))
source(here::here("R", "02_parse.R"))
source(here::here("R", "03_baseline.R"))
source(here::here("R", "04_evaluate.R"))
source(here::here("R", "05_apply_at_scale.R"))
source(here::here("R", "06_evidence_map.R"))

Workflow

1. Setup

source(here::here("R", "00_setup.R"))

Loads the required packages and sets the project seed.

2. Search articles

source(here::here("R", "01_search.R"))

Runs the PubMed Central search and saves the search results.

3. Parse articles

source(here::here("R", "02_parse.R"))

Downloads and parses the PMC records into structured article-level data.

4. Train baseline model

source(here::here("R", "03_baseline.R"))

Trains the baseline text classification model using the manually annotated dataset.

The model is based on TF-IDF features (unigrams and bigrams) and multinomial logistic regression with LASSO regularization.

5. Evaluate model

source(here::here("R", "04_evaluate.R"))

Evaluates the baseline model on the test set.

6. Apply model to full corpus

source(here::here("R", "05_apply_at_scale.R"))

Applies the trained model to the full processed corpus.

7. Build evidence map

source(here::here("R", "06_evidence_map.R"))

Summarises the predicted translational maturity labels at corpus level and by publication year.

Data

The main annotated dataset used for model training is:

data/annotated/gold_dataset_v01.xlsx

The manual review file used to assess model predictions is:

data/manual_review/manual_check_predicted_labels_40_per_class.xlsx

Some raw and processed data files are generated by the workflow and may not be stored in the repository.

Reproducibility

The project is designed to be reproducible through:

• R 4.4.3;
• renv.lock;
• a .Renviron file for local environment variables;
• a fixed random seed;
• project-relative paths with here::here();
• separated scripts for each step of the workflow.