We have made amazing progress in 2025: https://github.com/lance-format/lance-spark/issues/47. Here are the roadmap items for 2026 for anyone interested in contributing: # Read Performance - [ ] Better split planning leveraging index and combine fragments: https://github.com/lance-format/lance-spark/issues/34 - [ ] Storage partitioned JOIN without partitioning: https://github.com/lance-format/lance-spark/issues/230 - [ ] Optimizing aggregate pushdown: https://github.com/lance-format/lance-spark/issues/231 - [ ] Integrate with NDV stats: https://github.com/lance-format/lance-spark/issues/60 - [ ] Better caching: https://github.com/lance-format/lance-spark/issues/238 - [ ] Integrate with column statistics: https://github.com/lance-format/lance-spark/issues/121 # Read Features - [ ] Time travel, version travel, branching and tagging travel: https://github.com/lance-format/lance-spark/issues/232 - [ ] Vector search SQL extension: https://github.com/lance-format/lance-spark/issues/65 - [ ] Full text search SQL extension: https://github.com/lance-format/lance-spark/issues/234 - [x] Support float16 vector type: https://github.com/lance-format/lance-spark/issues/245 # Write Features - [x] CREATE OR REPLACE TABLE: https://github.com/lance-format/lance-spark/issues/210 - [ ] Structured streaming: https://github.com/lance-format/lance-spark/issues/246 - [x] Improve distributed index build for btree: https://github.com/lance-format/lance-spark/issues/250 - [ ] Distributed index build for zone map: https://github.com/lance-format/lance-spark/issues/168 - [ ] Distributed index build for bloom filter: https://github.com/lance-format/lance-spark/issues/235 - [ ] Distributed index build for label list: https://github.com/lance-format/lance-spark/issues/236 - [ ] Distributed index build for bitmap: https://github.com/lance-format/lance-spark/issues/237 - [ ] Distributed index build for vector index: https://github.com/lance-format/lance-spark/issues/165 # DDL - [ ] Schema evolution DDLs: https://github.com/lance-format/lance-spark/issues/62 - [x] Branching and tagging management DDLs: https://github.com/lance-format/lance-spark/issues/239 - [x] Drop index DDL: https://github.com/lance-format/lance-spark/issues/244 - [x] ALTER TABLE SET TBLPROPERTIES: https://github.com/lance-format/lance-spark/issues/247 - [x] ALTER TABLE SET UNENFORCED PRIMARY KEY: https://github.com/lance-format/lance-spark/issues/248 - [ ] RENAME TABLE: https://github.com/lance-format/lance-spark/issues/249 # Fundamentals - [x] Spark 4.1 support: https://github.com/lance-format/lance-spark/issues/233 - [ ] Substrait support: https://github.com/lance-format/lance-spark/issues/252 - [ ] Run test against AWS: https://github.com/lance-format/lance-spark/issues/240 - [ ] Run test against Azure: https://github.com/lance-format/lance-spark/issues/241 - [ ] Setup TPCH benchmark: https://github.com/lance-format/lance-spark/issues/242 - [x] Setup TPCDS benchmark: https://github.com/lance-format/lance-spark/issues/243
We have made amazing progress in 2025: #47. Here are the roadmap items for 2026 for anyone interested in contributing:
Read Performance
Read Features
Write Features
DDL
Fundamentals