Skip to content

Add schema artifacts for future nested sourced_from feature - #1228

Merged
myronmarston merged 3 commits into
block:mainfrom
ellisandrews-toast:nested-sourced-from-schema-artifacts
Jun 2, 2026
Merged

Add schema artifacts for future nested sourced_from feature#1228
myronmarston merged 3 commits into
block:mainfrom
ellisandrews-toast:nested-sourced-from-schema-artifacts

Conversation

@ellisandrews-toast

@ellisandrews-toast ellisandrews-toast commented Jun 1, 2026

Copy link
Copy Markdown
Collaborator

Context

ElasticGraph's existing sourced_from feature allows top-level fields on an indexed type to be populated from events of a different type. We're extending this to support nested sourced_from — where fields on embedded objects within an indexed document can also be populated from external events. This requires new runtime metadata to tell the indexing painless script how to navigate into the nested document structure and apply updates to the right element.

Summary

This PR introduces the data model and plumbing for the upcoming nested sourced_from feature. All new params flow through the system as empty values.

What this PR adds

  • ListPathSegment / ObjectPathSegment — Data classes representing path segments for navigating to nested elements (list elements matched by key, or object fields)
  • SourcedFromNestedParams — bundles per-event params (field_params + path_identifier_params) for nested update targets
  • sourced_from_nested_paths_by_relationship field on IndexDefinition — static per-index config describing navigation paths
  • sourced_from_nested_params field on UpdateTarget — per-event param resolution
  • SourcedFromNestedPathSegment module — discriminates segment types via .key?("match_field") (no type field in YAML; painless-facing to_painless_param still includes type for script discrimination)

Why land this separately

Later PRs will add the painless script logic (which references these params) and the schema definition code (which populates them with real values). By landing the data model and plumbing first, those PRs can be reviewed in isolation without worrying about serialization format or param wiring.

myronmarston

This comment was marked as resolved.

@ellisandrews-toast ellisandrews-toast changed the title Add schema artifacts and indexer wiring for future nested sourced_from feature Add schema artifacts for future nested sourced_from feature Jun 2, 2026

@myronmarston myronmarston left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM apart from one minor suggestion.

@myronmarston
myronmarston enabled auto-merge (squash) June 2, 2026 14:17
@myronmarston
myronmarston merged commit b8e7fb1 into block:main Jun 2, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants