DINOv3 Representation Lab

An evaluation and analysis lab for pretrained DINOv3 visual representations. The project benchmarks frozen backbones on ImageNet-1k, visualizes dense patch features, and compares DINOv3 with a supervised ViT under an identical protocol.

This is not a DINOv3 pre-training reproduction. It studies publicly released pretrained backbones on a single consumer GPU (RTX 3090), with frozen weights: only lightweight evaluation heads are trained.

The central question is:

At which Transformer depth do DINOv3 representations transition from local texture cues to semantic object structure?

Track	Objective	Main outputs
Global evaluation	Measure frozen image embeddings on ImageNet-1k	k-NN, logistic regression, linear probe, Top-1/Top-5, per-class metrics, confusion matrix
Dense visualization	Make patch-level representations interpretable	PCA RGB maps, cosine-similarity maps
Comparative analysis	Identify what makes a representation useful	DINOv3 vs supervised DeiT-S/16; depth, pooling, resolution, robustness studies

Results

Frozen-feature benchmark on ImageNet-1k

Full validation set (50 000 images, 1 281 167 train references), DINOv3 ViT-S/16 frozen final-layer features at 224 px, seed 42, checkpoint revision 1c5ca6c:

Pooling	Method	Top-1	Top-5	Macro per-class
CLS	k-NN (k=20)	0.769	0.923	0.769
CLS	logistic regression	0.617	0.858	0.617
CLS	linear probe	0.780	0.940	0.780
mean_patch	k-NN (k=20)	0.627	0.832	0.627
mean_patch	logistic regression	0.390	0.671	0.390
mean_patch	linear probe	0.700	0.903	0.700

• The CLS linear probe (78.0 % Top-1) is the headline number. The DINOv3 paper evaluates linear probes at 512 px for patch-16 models (1024 patch tokens), so a small gap against the published figure is expected at 224 px; the 512 px configurations are ready (configs/phase2-imagenet-512.toml, configs/phase3-imagenet-512.toml).
• CLS dominates mean-patch pooling at 224 px on this benchmark.
• The multinomial logistic regression (SGD, 20 epochs) is untuned at ImageNet-1k scale; treat its numbers as a lower bound. The AdamW linear probe is the meaningful figure.
• Predictions, per-class accuracies and confusion matrices are under outputs/phase3-imagenet/; the feature caches under outputs/phase2-imagenet/ are reusable for any future 224 px evaluation without re-running the backbone.

Dense patch features: PCA and cosine similarity

Patch tokens form a 14×14 grid at 224 px. The centre panel maps the first three PCA components of the patch embeddings to RGB (PCA fitted on a seeded 20-image ImageNet validation sample); the right panel shows the cosine similarity between one selected query patch and every patch. Foreground objects separate from the background without any label or supervision.

The query patch is chosen by grid coordinates in the configuration (query_patch_row/query_patch_column); here it sits on the snail shell, and the similarity map lights up the whole animal. Any image can be analysed, including your own (see the command table below).

DINOv3 vs supervised DeiT-S/16

Same frozen-feature protocol, same probes, same images. Smoke comparison on Imagenette (200 train / 200 validation images, 10 classes — all ImageNet classes, which favours the supervised model). DeiT-S/16 (22M parameters, supervised on ImageNet-1k) is the same-capacity supervised counterpart of DINOv3 ViT-S/16 (21M).

Model	Pooling	k-NN Top-1	Logistic Top-1	Linear probe Top-1
DINOv3 ViT-S/16	CLS	0.955	0.935	0.935
DINOv3 ViT-S/16	mean_patch	0.885	0.920	0.885
DeiT-S/16 supervised	CLS	0.995	0.985	0.980
DeiT-S/16 supervised	mean_patch	0.915	0.925	0.925

configs/phase6-imagenet.toml scales this comparison to the full ImageNet-1k splits. Supervised ViT baselines are declared with kind = "vit_supervised" and accept a local checkpoint clone (the revision field records the clone's git revision).

Quickstart

Requirements: uv, Python ≥ 3.11, access to the gated facebook/dinov3-vits16-pretrain-lvd1689m checkpoint on Hugging Face, and ImageNet-1k in the ImageFolder layout data/imagenet/{train,val}/<class>/image.jpg (obtained separately under its terms of use). Imagenette (data/imagenette2-160) works for every smoke path.

uv sync

What	Command
Config smoke (no data, no GPU)	uv run dinov3-lab-smoke
Unit tests	uv run python -m unittest discover -s tests -v
Build the 224 px feature cache (resumable)	uv run dinov3-lab-cache-features --config configs/phase2-imagenet.toml --imagenet-root data/imagenet
Run the ImageNet-1k frozen benchmark from the cache	uv run dinov3-lab-benchmark-frozen-features --config configs/phase3-imagenet.toml --imagenet-root data/imagenet
Render the benchmark figure	uv run python scripts/render-phase3-figure.py
Patch PCA + similarity figure (dataset image)	uv run dinov3-lab-visualize-patches --config configs/phase4-imagenet.toml --imagenet-root data/imagenet
Same figure for your own image	add --image path/to/your/image.jpg (a dataset is still required to fit the PCA)
Depth / pooling / resolution / robustness / retrieval analyses	uv run dinov3-lab-analyze-representations --imagenet-root data/imagenette2-160
DINOv3 vs supervised DeiT-S/16 (smoke)	uv run dinov3-lab-compare-baselines --config configs/phase6-deit-smoke.toml --imagenet-root data/imagenette2-160
DINOv3 vs DeiT-S/16 on full ImageNet-1k	uv run dinov3-lab-compare-baselines --config configs/phase6-imagenet.toml --imagenet-root data/imagenet

Every run writes a machine-readable report (metrics, predictions, configuration, model revision, seed) under its outputs/<experiment>/ directory.

Scope and constraints

• Backbones are frozen; only k-NN, logistic regression and a linear probe are evaluated.
• Primary model: facebook/dinov3-vits16-pretrain-lvd1689m (ViT-S/16) at 224 px; 512 px configurations reproduce the paper's 1024-patch-token evaluation protocol.
• Full ImageNet patch-token tensors are never cached; dense features are computed on demand. Global embeddings are cached, keyed by model revision, transform, layer, pooling, resolution, split and dtype.
• DINOv3 pre-training and full backbone fine-tuning are out of scope.

Repository layout

src/dinov3_representation_lab/  feature extraction, evaluation, visualization, utilities
configs/              versioned experiment configurations
scripts/              small command-line entry points (figures, reports)
tests/                unit and smoke tests
data/                 local, git-ignored datasets and caches
outputs/              git-ignored embeddings, figures, predictions, metrics
docs/                 experiment reports and figures selected for publication

Roadmap

Ordered milestones, acceptance criteria and the planned experiment matrix live in ROADMAP.md.