Skip to content

[SPARK-57789] Update and bundle LICENSE-binary and licenses-binary to describe all bundled libraries#721

Closed
dongjoon-hyun wants to merge 1 commit into
apache:mainfrom
dongjoon-hyun:SPARK-57789
Closed

[SPARK-57789] Update and bundle LICENSE-binary and licenses-binary to describe all bundled libraries#721
dongjoon-hyun wants to merge 1 commit into
apache:mainfrom
dongjoon-hyun:SPARK-57789

Conversation

@dongjoon-hyun

@dongjoon-hyun dongjoon-hyun commented Jun 30, 2026

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

This PR updates LICENSE-binary to describe all third-party libraries bundled in the operator
shadow jar, adds the licenses-binary/ directory with the license texts of the non-Apache
components, and bundles them into the jar like Apache Hadoop's hadoop-client-runtime.

  • LICENSE-binary: rewritten in Apache Spark's LICENSE-binary format, listing the 170 bundled
    artifacts grouped by license. The list was verified against the actual shadow jar contents.
  • licenses-binary/: adds the 11 license texts for the non-Apache-licensed components.
  • spark-operator/build.gradle: the shadow jar now ships LICENSE-binary/NOTICE-binary as
    META-INF/LICENSE/META-INF/NOTICE plus META-INF/licenses-binary/, instead of the
    source-release LICENSE/NOTICE.
  • .github/.licenserc.yaml: skips the header check for licenses-binary/**.

Why are the changes needed?

LICENSE-binary was a copy of LICENSE and did not enumerate any bundled dependency. Per the
ASF licensing policy, a convenience binary must account for all bundled licenses and include the non-Apache license texts.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Pass the CIs.

Manually verified that META-INF/LICENSE in the shadow jar is identical to LICENSE-binary, and
that every listed artifact actually has classes in the shadow jar (relocation-aware comparison
against the original jars from Maven Central).

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Fable 5

@dongjoon-hyun dongjoon-hyun marked this pull request as draft June 30, 2026 22:28
viirya
viirya previously approved these changes Jun 30, 2026
@dongjoon-hyun dongjoon-hyun marked this pull request as ready for review June 30, 2026 23:38
@dongjoon-hyun dongjoon-hyun changed the title [SPARK-57789] Update LICENSE-binary and add licenses-binary to describe all bundled libraries [SPARK-57789] Update and bundle LICENSE-binary and licenses-binary to describe all bundled libraries Jun 30, 2026
@dongjoon-hyun dongjoon-hyun marked this pull request as draft June 30, 2026 23:55
@dongjoon-hyun

Copy link
Copy Markdown
Member Author

Since we use minimize feature to build uber jar, it becomes a little complicated. I'm rechecking the dependency.

@viirya viirya dismissed their stale review July 1, 2026 00:09

wait for it is ready

@dongjoon-hyun dongjoon-hyun force-pushed the SPARK-57789 branch 3 times, most recently from f72224e to 43a1346 Compare July 1, 2026 05:08
@dongjoon-hyun dongjoon-hyun force-pushed the SPARK-57789 branch 3 times, most recently from 2a7a11a to 8980974 Compare July 1, 2026 17:57
@dongjoon-hyun dongjoon-hyun marked this pull request as ready for review July 2, 2026 06:06
@dongjoon-hyun

Copy link
Copy Markdown
Member Author

Now, this PR is ready. Could you review this, too, @viirya and @peter-toth ?

@dongjoon-hyun

Copy link
Copy Markdown
Member Author

Thank you so much, @viirya and @peter-toth !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants