Skip to content

Retrospective change of a result through filter operation #55

@fkroeber

Description

@fkroeber

Description

Due to the usage of filterer.values = utils.null_as_zero(filterer), recipe results are changed in a way that NaN values are converted to zero values if they are used in a subsequent filter operation.

Reproducible example

import json
import geopandas as gpd
import semantique as sq
from semantique.processor.core import QueryProcessor

# define recipe
recipe = sq.QueryRecipe()

recipe["res_a"] = sq.reflectance("s2_band02").\
    filter_time("before", sq.time_instant("2019-12-31"))

recipe["res_b"] = sq.reflectance("s2_band02").\
    filter_time("before", sq.time_instant("2019-12-31"))

recipe["res_c"] = sq.reflectance("s2_band02").\
    filter(sq.result("res_a"))

# create context
with open("files/mapping.json", "r") as file:
    mapping = sq.mapping.Semantique(json.load(file))

with open("files/layout_gtiff.json", "r") as file:
    dc = sq.datacube.GeotiffArchive(json.load(file), src = "files/layers_gtiff.zip")

space = sq.SpatialExtent(gpd.read_file("files/footprint.geojson"))
time = sq.TemporalExtent("2019-01-01", "2020-12-31")

context = {
    "datacube": dc, 
    "mapping": mapping,
    "space": space,
    "time": time,
    "crs": 3035, 
    "tz": "UTC", 
    "spatial_resolution": [-10, 10],
    "track_types": False
}

# execute recipe
fp = QueryProcessor.parse(recipe, **context)
response = fp.optimize().execute()

# evaluate equivalance of both results -> leads to False
# res_a contains zero values at places where res_b contains NaN values 
response["res_a"].equals(response["res_b"])

Expected behavior / Proposed solution

The filter operation should not alter previous results. A deep copy of the filterer needs to be created to ensure this. All occurrences of utils.null_as_zero() within semantique (find . -type f -exec grep -i 'null_as_zero' {} +) should be checked for if they incorrectly mutate an existing object instead of creating a new copy.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bug 🐛Something isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions