fix(core): stop node upsert from nulling unset FlowsintType fields - #228
Merged
Merged
Conversation
dextmorgn
force-pushed
the
fix/node-upsert-clobber-exclude-unset
branch
from
September 20, 2026 17:06
5538794 to
aa97124
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
GraphSerializer.graph_node_to_neo4j_dictdumpednodePropertiesin full, defaults included. The repository upsert (_build_node_query) doesSET n += $props, so every enricher that writes a partial FlowsintType (e.g.Website(url=x)) sent its unset fields asNone/falsy defaults and nulled out richer values a prior enricher had already written on the same node.WebsiteandDomainwriters hit this routinely:to_webtrackers,to_technologies,to_links, andto_indexed_pagesall construct thinWebsiteinstances against a nodeto_website/to_crawlermay have already enriched.nodePropertiesnow dumps withexclude_unset, so a partial write only sends the fields the caller actually set. The rest of theGraphNodedump (id, x/y, nodeColor, etc.) is untouched, since those fields rely on class defaults being present in every write.Scope
flowsint-core/src/flowsint_core/core/graph/serializer.py:GraphSerializer.graph_node_to_neo4j_dictflowsint-core/tests/core/graph/test_graph_serializer.py: repro + regression coverageBlast Radius
Touches every enricher's node write path (
create_node/flowsint_type_to_neo4j_dict), since they all funnel through this one function. Low risk:exclude_unsetonly drops fields a constructor call never set, so nothing that was previously sent on purpose changes. Existing serializer/repository tests (416 in flowsint-core) still pass unmodified.Verification
python3 -m pytest flowsint-core/tests/— 416 passed. New tests confirmed failing against the pre-fix code (thinWebsiteleakedactive/status_code/etc as explicit keys), then passing after the fix.