It could be worth exploring an implementation of deferred tiff tag reading in this repo.
Right now, all tags are read up front, meaning that for large COGs we fetch a good amount of metadata before being able to do anything with the file. I'm thinking we should potentially change that so that tiff tags are instead deferred and loaded on demand for any non-inline tags.
Then for large tags like tile byte counts and tile offset, you don't even need to fetch those on demand, you can read just the element you need when reading a specific tile.
I'm fresh off implementing a client side viewer for cogs, where I implemented a speedup for viewing remote COGs where you don't need to fetch the entire COG header. developmentseed/deck.gl-raster#529
The core implementation is in @cogeotiff/core
I'm thinking of having an enum to describe tag states, including "inline" and "deferred" tags. I.e. tiff tags can be inline, in the case of small tags (4 bytes in tiff or 8 bytes in bigtiff I think?) or for larger tags they're stored separately.
It could be worth exploring an implementation of deferred tiff tag reading in this repo.
Right now, all tags are read up front, meaning that for large COGs we fetch a good amount of metadata before being able to do anything with the file. I'm thinking we should potentially change that so that tiff tags are instead deferred and loaded on demand for any non-inline tags.
Then for large tags like tile byte counts and tile offset, you don't even need to fetch those on demand, you can read just the element you need when reading a specific tile.
I'm fresh off implementing a client side viewer for cogs, where I implemented a speedup for viewing remote COGs where you don't need to fetch the entire COG header. developmentseed/deck.gl-raster#529
The core implementation is in @cogeotiff/core
I'm thinking of having an enum to describe tag states, including "inline" and "deferred" tags. I.e. tiff tags can be inline, in the case of small tags (4 bytes in tiff or 8 bytes in bigtiff I think?) or for larger tags they're stored separately.