Raster API `rasterio.crs.CRS.from_epsg ValueError: invalid literal for int() with base 10: '4326.0'`

[anayeaye]

What

The sit, dev, and staging instances of the backend are failing to render map and thumbnail previews with an invalid CRS data type error

Exception Group Traceback (most recent call last):\n  |   File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/_utils.py\", line 79, in collapse_excgroups\n  |   File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/middleware/base.py\", line 183, in __call__\n  |   File \"/tmp/pip-target-1t5l1o5b/lib/python/anyio/_backends/_asyncio.py\", line 772, in __aexit__\n  | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)\n  +-+---------------- 1 ----------------\n    | Traceback (most recent call last):\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/middleware/errors.py\", line 164, in __call__\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/middleware/base.py\", line 182, in __call__\n    |   File \"/var/lang/lib/python3.12/contextlib.py\", line 158, in __exit__\n    |     self.gen.throw(value)\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/_utils.py\", line 85, in collapse_excgroups\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/middleware/base.py\", line 184, in __call__\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/src/app.py\", line 268, in add_correlation_id\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/aws_lambda_powertools/tracing/tracer.py\", line 612, in decorate\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/middleware/base.py\", line 159, in call_next\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/middleware/base.py\", line 144, in coro\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette_cramjam/middleware.py\", line 112, in __call__\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette_cramjam/middleware.py\", line 142, in __call__\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/titiler/core/middleware.py\", line 63, in __call__\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/middleware/cors.py\", line 85, in __call__\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/middleware/exceptions.py\", line 63, in __call__\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/_exception_handler.py\", line 53, in wrapped_app\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/_exception_handler.py\", line 42, in wrapped_app\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/routing.py\", line 716, in __call__\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/routing.py\", line 736, in app\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/routing.py\", line 290, in handle\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/routing.py\", line 78, in app\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/_exception_handler.py\", line 53, in wrapped_app\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/_exception_handler.py\", line 42, in wrapped_app\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/routing.py\", line 75, in app\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/src/monitoring.py\", line 59, in route_handler\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/fastapi/routing.py\", line 302, in app\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/fastapi/routing.py\", line 215, in run_endpoint_function\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/concurrency.py\", line 38, in run_in_threadpool\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/anyio/to_thread.py\", line 56, in run_sync\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/anyio/_backends/_asyncio.py\", line 2476, in run_sync_in_worker_thread\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/anyio/_backends/_asyncio.py\", line 967, in run\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/titiler/core/factory.py\", line 933, in tilejson\n    |   File \"<attrs generated methods titiler.pgstac.reader.PgSTACReader>\", line 60, in __init__\n    |     self.__attrs_post_init__()\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/titiler/pgstac/reader.py\", line 52, in __attrs_post_init__\n    |   File \"/tmp/pip-target-1t5l1o5b/lib/python/rio_tiler/io/stac.py\", line 283, in __attrs_post_init__\n    |   File \"rasterio/crs.pyx\", line 852, in rasterio.crs.CRS.from_string\n    |   File \"rasterio/crs.pyx\", line 582, in rasterio.crs.CRS.from_epsg\n    | ValueError: invalid literal for int() with base 10: '4326.0'\n    +------------------------------------\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/middleware/errors.py\", line 164, in __call__\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/middleware/base.py\", line 182, in __call__\n  File \"/var/lang/lib/python3.12/contextlib.py\", line 158, in __exit__\n    self.gen.throw(value)\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/_utils.py\", line 85, in collapse_excgroups\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/middleware/base.py\", line 184, in __call__\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/src/app.py\", line 268, in add_correlation_id\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/aws_lambda_powertools/tracing/tracer.py\", line 612, in decorate\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/middleware/base.py\", line 159, in call_next\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/middleware/base.py\", line 144, in coro\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette_cramjam/middleware.py\", line 112, in __call__\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette_cramjam/middleware.py\", line 142, in __call__\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/titiler/core/middleware.py\", line 63, in __call__\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/middleware/cors.py\", line 85, in __call__\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/middleware/exceptions.py\", line 63, in __call__\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/_exception_handler.py\", line 53, in wrapped_app\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/_exception_handler.py\", line 42, in wrapped_app\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/routing.py\", line 716, in __call__\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/routing.py\", line 736, in app\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/routing.py\", line 290, in handle\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/routing.py\", line 78, in app\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/_exception_handler.py\", line 53, in wrapped_app\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/_exception_handler.py\", line 42, in wrapped_app\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/routing.py\", line 75, in app\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/src/monitoring.py\", line 59, in route_handler\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/fastapi/routing.py\", line 302, in app\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/fastapi/routing.py\", line 215, in run_endpoint_function\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/starlette/concurrency.py\", line 38, in run_in_threadpool\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/anyio/to_thread.py\", line 56, in run_sync\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/anyio/_backends/_asyncio.py\", line 2476, in run_sync_in_worker_thread\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/anyio/_backends/_asyncio.py\", line 967, in run\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/titiler/core/factory.py\", line 933, in tilejson\n  File \"<attrs generated methods titiler.pgstac.reader.PgSTACReader>\", line 60, in __init__\n    self.__attrs_post_init__()\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/titiler/pgstac/reader.py\", line 52, in __attrs_post_init__\n  File \"/tmp/pip-target-1t5l1o5b/lib/python/rio_tiler/io/stac.py\", line 283, in __attrs_post_init__\n  File \"rasterio/crs.pyx\", line 852, in rasterio.crs.CRS.from_string\n  File \"rasterio/crs.pyx\", line 582, in rasterio.crs.CRS.from_epsg\nValueError: invalid literal for int() with base 10: '4326.0'

This problem is not observed in the test and prod stacks.

To cause this error use a map preview request
https://dev.openveda.cloud/api/raster/collections/barc-thomasfire/items/thomas_fire_barc_201712.cog/WebMercatorQuad/tilejson.json?assets=cog_default&nodata=nan&rescale=1%2C4&colormap_name=rdylgn_r

To see the desired result, try the same request in the test stack
https://test.openveda.cloud/api/raster/collections/barc-thomasfire/items/thomas_fire_barc_201712/WebMercatorQuad/map?assets=cog_default&nodata=nan&rescale=1%2C4&colormap_name=rdylgn_r

Note
The map preview is dynamically generated in stac item links and the thumbnail rendered_preview is dynamically generated as an item asset when sufficient metadata are provided in the collection renders config. The items in this collection in all environments have sufficient renders config

barc-thomasfire
https://test.openveda.cloud/api/stac/collections/barc-thomasfire/items

AC

• raster api /map in item links working as expected with sufficient renders metadata
• raster api /preview link in item assets working as expected with sufficient renders metadata

[botanical]

Here is more of the stack trace:

    |   File "/tmp/pip-target-4cl63lh_/lib/python/titiler/core/factory.py", line 933, in tilejson
    |   File "<attrs generated methods titiler.pgstac.reader.PgSTACReader>", line 60, in __init__
    |     self.__attrs_post_init__()
    |   File "/tmp/pip-target-4cl63lh_/lib/python/titiler/pgstac/reader.py", line 52, in __attrs_post_init__
    |   File "/tmp/pip-target-4cl63lh_/lib/python/rio_tiler/io/stac.py", line 283, in __attrs_post_init__
    |   File "rasterio/crs.pyx", line 852, in rasterio.crs.CRS.from_string
    |   File "rasterio/crs.pyx", line 582, in rasterio.crs.CRS.from_epsg
    | ValueError: invalid literal for int() with base 10: '4326.0'
    +------------------------------------

which helped me narrow the issue to this line that errors. It seems to be updated in 1.6.0. Currently, SIT and smce staging are on 1.5.0

[anayeaye]

It is also possible that the metadata data types are different in the lower environments caused by differing versions of airflow and ingest-api packages

#166 is from the last time we had this bug--I may have re-surfaced old metadata problems in the lower environments by digging into some old collections that were never updated (but were published with the correct metadata datatypes in the test and prod environments)