Skip to content

GH-22342: [Format][Documentation] Clarify Union type ids - #50839

Merged
pitrou merged 4 commits into
apache:mainfrom
erichanwang:docs/union-typeids-22342
Oct 6, 2026
Merged

pitrou merged 4 commits into
apache:mainfrom
erichanwang:docs/union-typeids-22342

Conversation

@erichanwang

Copy link
Copy Markdown
Contributor

Rationale for this change

The Union layout documentation does not explain that the type IDs stored in the types buffer can differ from the child array indices, or how the optional typeIds metadata maps children to physical type IDs.

What changes are included in this PR?

  • Document the relationship between Union child array indices, physical type IDs, and the optional typeIds metadata.
  • Clarify that the types buffer stores the type ID for each slot.

Fixes #22342.

Are these changes tested?

git diff --check passes and the RST change was reviewed against Schema.fbs and the existing Union layout documentation. A local Sphinx/RST checker is not installed in this environment.

Are there any user-facing changes?

No.

@erichanwang
erichanwang requested a review from pitrou as a code owner August 10, 2026 00:14
@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #22342 has been automatically assigned in GitHub to PR creator.

@erichanwang

Copy link
Copy Markdown
Contributor Author

Formatting follow-up:

  • Wrapped the new Union type-id paragraph to match the surrounding Arrow RST line-wrapping style.
  • git diff --check passes; no semantic content changed.

@uros-b

uros-b commented Aug 10, 2026

Copy link
Copy Markdown
Member

Nice focused improvement, LGTM!

@pitrou pitrou changed the title GH-22342: [C++] [Documentation] Clarify Union type ids GH-22342: [Format][Documentation] Clarify Union type ids Sep 17, 2026
@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #22342 has been automatically assigned in GitHub to PR creator.

@pitrou

pitrou commented Sep 17, 2026

Copy link
Copy Markdown
Member

@paleolimbot @tustvold @alamb What do you think about these clarifications in the format spec?

@tustvold

Copy link
Copy Markdown
Contributor

I think the change makes sense, an example might clear up any remaining ambiguity

Each child type in a union has a type id (an 8-bit signed integer)
that identifies it in the types buffer. By default, these type ids
correspond directly to the child array indices (0, 1, ...), but an optional
``typeIds`` metadata array can provide an explicit mapping from child array

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It isn't clear from this how they actually do this, I think an example would make it clear.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also possibly calling it just typeIds metadata (the term "array" here I believe is talking about a different kind of array than it is later in the paragraph)

@github-actions github-actions Bot added awaiting changes Awaiting changes and removed awaiting review Awaiting review labels Sep 17, 2026

@paleolimbot paleolimbot left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definite improvement! I found this confusing when I implemented it. I don't believe that creating these types of unions is common (and agreed that an example would help).

For what it's worth, GeoArrow does use these and does use meaningful type IDs (although they aren't widely used compared to the other types in the spec). https://geoarrow.org/format.html#geometrycollection

Each child type in a union has a type id (an 8-bit signed integer)
that identifies it in the types buffer. By default, these type ids
correspond directly to the child array indices (0, 1, ...), but an optional
``typeIds`` metadata array can provide an explicit mapping from child array

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also possibly calling it just typeIds metadata (the term "array" here I believe is talking about a different kind of array than it is later in the paragraph)

@github-actions github-actions Bot added awaiting merge Awaiting merge and removed awaiting changes Awaiting changes labels Sep 17, 2026

@alamb alamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @pitrou -- this seems like an improvement to me

I think @tustvold and @paleolimbot 's suggestions are good ones too, but I also think this could be merged as is as it is an improvement over the current version in my mind

Comment thread docs/source/format/Columnar.rst
Comment thread docs/source/format/Columnar.rst Outdated
Comment thread docs/source/format/Columnar.rst Outdated
@pitrou

pitrou commented Oct 6, 2026

Copy link
Copy Markdown
Member

I've applied some minor modifications to address some of the comments, in the interest of moving this forward.

I think adding an example can be done in a followup task if someone is motivated to do so.

@pitrou

pitrou commented Oct 6, 2026

Copy link
Copy Markdown
Member

I'll wait for CI and then merge.

@pitrou
pitrou merged commit fcf8f30 into apache:main Oct 6, 2026
31 checks passed
@pitrou pitrou removed the awaiting merge Awaiting merge label Oct 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Format][Documentation] add discussion of Union.typeIds to Layout.rst

6 participants