Context-Aware Pedestrian Trajectory Prediction

This repository contains the code for my Masters Thesis in "Context-Aware Pedestrian Trajectory Prediction in Urban Scenarios Using Data-Driven Methods", which was done with Scania. This repository provides code for building pedestrian-centered trajectory prediction datasets and training/evaluating context-aware prediction models. The main dataset format used by the project is ped_context_map: each sample contains the target pedestrian history, future trajectory, nearby agents, ego motion, and vector map context.

The code supports nuScenes as the main dataset and includes the Snapshot preprocessing code for Argoverse 2 as a separate third-party component.

Installation

Create a Python environment and install the project requirements:

pip install -r requirements.txt

The model code uses PyTorch. If PyTorch is not already available in your environment, install the version that matches your CUDA setup from the official PyTorch instructions.

The nuScenes preprocessing depends on the bundled nuscenes-devkit/ folder. The requirements file installs the devkit dependencies used by this project.

Repository Structure

config.py                 Main paths and default settings
src/preprocessing/        Dataset generation scripts
src/baseline/             Dataset loaders and shared baseline utilities
src/models/               Model definitions, training scripts, and checkpoint evaluation
src/strat_eval/           Stratified evaluation tools
src/utils/                Shared training, validation, and visualization helpers
src/viz/                  Visualization scripts
src/legacy/               Older scripts not required for the main workflow
nuscenes-devkit/          nuScenes devkit used for dataset access and maps
snapshot/                 Snapshot preprocessing code for Argoverse 2

The most important path setting is DATASET_ROOT in config.py. By default it points to dataset/ inside this repository. You can override it with:

export NUSCENES_DATASET_ROOT=/path/to/dataset

On Windows PowerShell:

$env:NUSCENES_DATASET_ROOT="C:\path\to\dataset"

nuScenes Preprocessing

Download the official nuScenes metadata from https://www.nuscenes.org/nuscenes#download. For this project you need:

• Full Dataset (v1.0) Trainval metadata
• the nuScenes map expansion pack

You do not need the full camera videos for the preprocessing used here. The expected folder layout is:

dataset/
  v1.0-trainval/
  maps/
    expansion/

First build the per-scene all-object tensors:

python src/preprocessing/build_all_tracks_dataset.py \
  --dataset-folder v1.0-trainval

This writes scene files to dataset/preprocessed/all_tracks/ and creates the train/val/test scene split.

Then build the pedestrian context map dataset:

python src/preprocessing/build_ped_context_map_dataset.py \
  --dataset-folder v1.0-trainval \
  --output-root dataset/preprocessed/ped_context_map_v2

This creates the final pedestrian-centered dataset shards, split indices, and manifest.json under dataset/preprocessed/ped_context_map_v2/.

For a quick smoke test, use --dataset-folder v1.0-mini --max-scenes 1 in the second command.

Argoverse 2 Snapshot Preprocessing

The snapshot/ folder contains Snapshot preprocessing code. Its own README.md and LICENSE are kept in that folder because this part is not entirely my own work.

To generate the Argoverse 2 data for this project:

1. Follow the instructions in snapshot/README.md to create the standard Snapshot dataset files.
2. Adapt DEFAULT_SOURCE_ROOT and DEFAULT_OUTPUT_ROOT in snapshot/src/preprocessing/build_av2_ped_context_map.py to your machine.
3. Run the additional preprocessing step:

python snapshot/src/preprocessing/build_av2_ped_context_map.py

This creates the pedestrian context map data used by the rest of this project.

Model Training and Prediction

The main prediction pipeline trains a model on the generated ped_context_map dataset and predicts future pedestrian trajectories on the test split. For example, to train the PointNet model:

python src/models/train_ped_context_pointnet.py \
  --data-root dataset/preprocessed/ped_context_map_v2

The script saves the best checkpoint, test metrics, training curves, and example prediction visualizations under dataset/models/ped_context_pointnet/.

Other models use the same dataset format and workflow. Their training scripts are in src/models/train_ped_context_*.py, including PointNet, attention, transformer, CVAE, multimodal, and Snapshot-style variants.

Testing and Evaluation

To evaluate an existing trajectory prediction run, use:

python src/models/eval_ped_checkpoint.py \
  --run-dir dataset/models/ped_context_pointnet/run_001

This reloads best_model.pt, evaluates the test split, and regenerates standard visualizations. The evaluator can usually detect the model type automatically. It also supports context ablations such as --motion-only, --ablate-map, --ablate-neighbors, and --ablate-ego.

Typical outputs are:

best_model.pt
metrics.json
training_curves.png
visualizations/

Visualization

The training and evaluation scripts already create the most important plots. For manual inspection, these scripts are also useful:

• src/utils/export_ped_context_sample.py exports one dataset sample for debugging.
• src/viz/visualize_model_input_anchors.py visualizes model input anchors.
• src/viz/visualize_anchor_model_prediction.py visualizes one model prediction.
• src/viz/compare_model_predictions_side_by_side.py compares multiple model predictions on the same scene.

Crossing Intention

Crossing intention is handled as a separate binary classification task on top of the same pedestrian-context dataset:

python src/models/train_crossing_intention.py \
  --data-root dataset/preprocessed/ped_context_map_v2

The script trains a crossing/not-crossing classifier and writes metrics, checkpoints, and intent visualizations. Existing crossing-intention checkpoints can be evaluated with:

python src/models/eval_crossing_intention_checkpoint.py \
  --run-dir dataset/models/crossing_intention/run_001

Stratified Evaluation

The src/strat_eval/ folder contains tools for understanding where a model works well or badly. It groups test results by trajectory shape, motion state, crossing behavior, map proximity, and neighbor density.

python src/strat_eval/run_stratified_eval.py \
  --run-dir dataset/models/ped_context_pointnet/run_001

For multiple experiment folders, use run_batch_stratified_eval.py and summarize the batch outputs with summarize_batch_stratified_eval.py.

Third-Party Code and Citation

This repository includes two external codebases that were useful for the dataset work:

• nuscenes-devkit/ is the official nuScenes devkit (https://github.com/nutonomy/nuscenes-devkit). The nuScenes preprocessing code depends on it for reading the dataset tables and map information.
• snapshot/ contains the Snapshot repository for Argoverse 2 pedestrian trajectory preprocessing and modeling (https://github.com/TUMFTM/Snapshot). Its original README.md and LICENSE are kept inside the folder.

If you use the nuScenes or Argoverse 2/Snapshot parts of this repository, please also follow the citation and license instructions from the corresponding projects.

Notes

• src/legacy/ contains older scripts that were useful during development but are not required for the main prediction and evaluation pipeline.
• Most generated files are written under dataset/preprocessed/ and dataset/models/.
• The repository still contains some code for Viscando_traffic_data. This is a proprietary dataset provided by Viscando, so the data itself is not publicly available.