I think I want a wholesale reconsideration of how we load metadata from COGs. We want to balance the twin goals of:
- low latency for large files
- reducing the total number of header requests, and reducing the latency to load each additional image tile. That is, in a perfect world where it didn't cost anything to load the header metadata, we would load everything up front because it would mean that each additional image tile could be loaded in one single request.
But right now for really large COGs like in the vermont cog comparison example, we're fetching 60MB of metadata before we can render any single image tile. In that case it would be much better to avoid pre-fetching the entire tile offsets and byte counts. Especially because the initial view might not be at the highest resolution anyways, so the full-resolution image's offsets and byte counts might not be used for the initial render anyways.
I think I want a wholesale reconsideration of how we load metadata from COGs. We want to balance the twin goals of:
But right now for really large COGs like in the vermont cog comparison example, we're fetching 60MB of metadata before we can render any single image tile. In that case it would be much better to avoid pre-fetching the entire tile offsets and byte counts. Especially because the initial view might not be at the highest resolution anyways, so the full-resolution image's offsets and byte counts might not be used for the initial render anyways.