fix: JSON Data Encoding

[andrecalil]

Summary

Fixes canvas event JSON handling so numeric payload values are not silently converted to float64 and later rendered in scientific notation or with precision loss.

Changes:

• Replaced CanvasEvent.Data JSON storage with a number-preserving models.JSONValue.
• Decode webhook payloads with json.Decoder.UseNumber().
• Preserve number tokens when events are emitted, re-emitted, and produced from node executions.
• Added protobuf serialization handling for json.Number, stringifying only unsafe/exponent-prone values at the Struct boundary.
• Kept non-number JSON types unchanged: objects, arrays, strings, booleans, and nulls.
• Added tests for storage scanning, webhook parsing, expression interpolation, and canvas event serialization.

[superplanehq-integration]

👋 Commands for maintainers:

• /sp start - Start an ephemeral machine (takes ~30s)
• /sp stop - Stop a running machine (auto-executed on pr close)

[mchalapuk]

This review is more about me trying to understand what's going on in the code than anything else.

The PR seems to me like something that would work.

I'm wondering if there's a simpler solution that doesn't involve a special structure for handling JSON in the code and automatically converting the number values to strings.

Wouldn't it be simpler to explicitly convert those strings to numbers when we're creating the JSON structures instead of this special type and auto-convert logic?

[andrecalil]

@cursoragent update your review

[cursor]

Done.

• Synced to the latest force-updated PR branch.
• Re-checked the review findings.
• Fixed the remaining valid issue: oversized JSON integer tokens now stay exact strings in expression data instead of being rounded through float64.
• Moved JSON integer-token detection into pkg/models and reused it from protobuf conversion.

Pushed commit:

• 7b2392e07 fix: preserve oversized JSON integers in expressions

Verified:

• go test ./pkg/models -run 'JSONValue|UnmarshalJSONValue|IsJSONIntegerToken' -count=1
• go test ./pkg/grpc/actions/canvases -run 'Test__toStructpbCompatible' -count=1
• go test ./pkg/workers/contexts -run 'Test_NodeConfigurationBuilder_JSONNumber' -count=1
• git diff --check

Could not run full Docker-backed checks:

• make lint && make check.build.app fails because docker is not installed in this VM.

  

[lucaspin]

This change is suspicious. Why is this needed?

[andrecalil]

To support expressions (hence the sourceNodeID) and to normalize different number types.
What do you find suspicious? Feel free to suggest a different approach.

[lucaspin]

What do you find suspicious?

I could be misunderstanding this, but to me this looks suspicious to me because all the other changes seem to be about decoding, but for this one, you are actually changing what the ProcessQueueContext.Input value is. @forestileao can you take a look?

[shiroyasha]

What is the state here? @lucaspin @andrecalil @forestileao

[andrecalil]

I was waiting for Pedro to chime in while I looked at other issues/tasks.

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit d73581f. Configure here.}

[cursor]

Duplicate recursive JSON tree-walking logic across packages

Low Severity

toStructpbCompatible in structpb_compatible.go and normalizeExpressionValue/normalizeExpressionMap/normalizeExpressionSlice in node_configuration_builder.go both implement the same recursive tree-walking pattern over map[string]any, []any, and json.Number. The only difference is the leaf conversion strategy for json.Number. Extracting a shared generic walker parameterized by the number-conversion function would reduce the maintenance surface and the risk of one path diverging from the other.

Additional Locations (1)

• pkg/workers/contexts/node_configuration_builder.go#L429-L496

 

^{Reviewed by Cursor Bugbot for commit d73581f. Configure here.}

[cursor]

Multi-value detection uses wrong decode target type

Low Severity

UnmarshalJSONValue detects multiple top-level JSON values by attempting a second Decode into &struct{}{}. This works for extra objects or null, but if the trailing value is a number, string, boolean, or array, the decoder returns a type-mismatch error instead of nil. The function still rejects the input, but the returned error message is misleading (a Go unmarshal error instead of "multiple top-level values"). Using new(any) as the target would correctly detect all trailing JSON value types.

 

^{Reviewed by Cursor Bugbot for commit d73581f. Configure here.}