Skip to content

fix(core): stop node upsert from nulling unset FlowsintType fields - #228

Merged
dextmorgn merged 2 commits into
mainfrom
fix/node-upsert-clobber-exclude-unset
Sep 20, 2026
Merged

dextmorgn merged 2 commits into
mainfrom
fix/node-upsert-clobber-exclude-unset

Conversation

@dextmorgn

Copy link
Copy Markdown
Collaborator

Why

GraphSerializer.graph_node_to_neo4j_dict dumped nodeProperties in full, defaults included. The repository upsert (_build_node_query) does SET n += $props, so every enricher that writes a partial FlowsintType (e.g. Website(url=x)) sent its unset fields as None/falsy defaults and nulled out richer values a prior enricher had already written on the same node. Website and Domain writers hit this routinely: to_webtrackers, to_technologies, to_links, and to_indexed_pages all construct thin Website instances against a node to_website/to_crawler may have already enriched.

nodeProperties now dumps with exclude_unset, so a partial write only sends the fields the caller actually set. The rest of the GraphNode dump (id, x/y, nodeColor, etc.) is untouched, since those fields rely on class defaults being present in every write.

Scope

  • flowsint-core/src/flowsint_core/core/graph/serializer.py: GraphSerializer.graph_node_to_neo4j_dict
  • flowsint-core/tests/core/graph/test_graph_serializer.py: repro + regression coverage

Blast Radius

Touches every enricher's node write path (create_node/flowsint_type_to_neo4j_dict), since they all funnel through this one function. Low risk: exclude_unset only drops fields a constructor call never set, so nothing that was previously sent on purpose changes. Existing serializer/repository tests (416 in flowsint-core) still pass unmodified.

Verification

python3 -m pytest flowsint-core/tests/ — 416 passed. New tests confirmed failing against the pre-fix code (thin Website leaked active/status_code/etc as explicit keys), then passing after the fix.

@dextmorgn
dextmorgn force-pushed the fix/node-upsert-clobber-exclude-unset branch from 5538794 to aa97124 Compare September 20, 2026 17:06
@dextmorgn
dextmorgn merged commit eae1d7a into main Sep 20, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant