Optimize cuvs-lucene CAGRA_HNSW index build; add example - #2481

Open

jamxia155 wants to merge 22 commits into

NVIDIA:mainfrom

jamxia155:cuvs-lucene-cagra-hnsw-build-optimization-with-example

Commits on Aug 18, 2026

cuvs-lucene__139: This code allows us to construct the HNSW graph on GPU without loading the full set of data on the Java Heap, but instead allows us to stream the set of data to the Java Heap.

EC2 Default User
authored and
jamxia155
committed
c579e87
Consolidate duplicated HNSW graph and field-writing methods

nvzm123
authored and
jamxia155
committed
8598e4f
Add example: chunked sequential ingestion of large .fbin files
jamxia155
committed
f66bdce
Parallelize level-0 HNSW graph serialization in writeGraph
jamxia155
committed
d0b5eb9
Add native flat buffering for single-segment CAGRA_HNSW builds
jamxia155
committed
cd23490
Parallelize CAGRA-to-HNSW conversion in GPUBuiltHnswGraph
jamxia155
committed
3ad0d47
Add prefetching + reused-array to the fbin ingest example
jamxia155
committed
948b41d
Honor cagraGraphBuildAlgo override in HEURISTIC strategy
jamxia155
committed
33c85a2
Expand the fbin ingest example into a full optimized CAGRA-HNSW build
jamxia155
committed
1b9b9d8
Expand OptimizedCagraHnswBuildExample and add early index-sort check
jamxia155
committed
56bc260
Fix merge errors
jamxia155
committed
8ffba09
Add a Lucene-version tripwire and round-trip test for NativeFlatVectorsWriter
jamxia155
committed
cfd3440
Add equivalence test for writerThreads-parallelized graph construction/write
jamxia155
committed
4b4a0d7
Add functional coverage for native flat buffering (numInputVectors)
jamxia155
committed
64ee08d
Fix merge-time vector-count bug causing intermittent EOF during search
jamxia155
committed
22e10fe
Restore unintended M-derivation change in createMultiLayerHnswGraph
jamxia155
committed
725fc9d
Restore unnecessary removals
jamxia155
committed
9faae91
Fix inconsistent handling of explicit CuVSIvfPqParams under HEURISTIC+IVF_PQ
jamxia155
committed
77cbc65
Guard against native flat buffering with quantized fields
jamxia155
committed
3ab2330
Merge remote-tracking branch 'origin/main' into cuvs-lucene-cagra-hnsw-build-optimization-with-example
jamxia155
committed
68245a1
Apply Spotless formatting
jamxia155
committed
563428c
Merge remote-tracking branch 'origin/main' into cuvs-lucene-cagra-hnsw-build-optimization-with-example
jamxia155
committed
b99dae2

Navigation Menu

Uh oh!

Optimize cuvs-lucene CAGRA_HNSW index build; add example - #2481

Optimize cuvs-lucene CAGRA_HNSW index build; add example#2481
jamxia155 wants to merge 22 commits into
NVIDIA:mainfrom
jamxia155:cuvs-lucene-cagra-hnsw-build-optimization-with-example

Commits on Aug 18, 2026

cuvs-lucene__139: This code allows us to construct the HNSW graph on GPU without loading the full set of data on the Java Heap, but instead allows us to stream the set of data to the Java Heap.

Consolidate duplicated HNSW graph and field-writing methods

Add example: chunked sequential ingestion of large .fbin files

Parallelize level-0 HNSW graph serialization in writeGraph

Add native flat buffering for single-segment CAGRA_HNSW builds

Parallelize CAGRA-to-HNSW conversion in GPUBuiltHnswGraph

Add prefetching + reused-array to the fbin ingest example

Honor cagraGraphBuildAlgo override in HEURISTIC strategy

Expand the fbin ingest example into a full optimized CAGRA-HNSW build

Expand OptimizedCagraHnswBuildExample and add early index-sort check

Fix merge errors

Add a Lucene-version tripwire and round-trip test for NativeFlatVectorsWriter

Add equivalence test for writerThreads-parallelized graph construction/write

Add functional coverage for native flat buffering (numInputVectors)

Fix merge-time vector-count bug causing intermittent EOF during search

Restore unintended M-derivation change in createMultiLayerHnswGraph

Restore unnecessary removals

Fix inconsistent handling of explicit CuVSIvfPqParams under HEURISTIC+IVF_PQ

Guard against native flat buffering with quantized fields

Merge remote-tracking branch 'origin/main' into cuvs-lucene-cagra-hnsw-build-optimization-with-example

Apply Spotless formatting

Merge remote-tracking branch 'origin/main' into cuvs-lucene-cagra-hnsw-build-optimization-with-example