[improve][build] Improve the gradle build to use similar best practices as apache/pulsar build

[lhotari]

Motivation

The upstream apache/pulsar repository refactored its Gradle build after the initial Gradle support (commit 5b9fe44) to adopt Gradle best practices: convention plugins, composite builds, lazy configuration, and better tooling. The pulsar-connectors build was still using a monolithic root build.gradle.kts with a subprojects {} block for all shared configuration. This PR aligns the connectors build with the upstream patterns and picks up several improvements.

Modifications

Convention plugins (build-logic/ composite build):

• Created pulsar-connectors.java-conventions — shared Java compilation, dependencies, test configuration, Lombok
• Created pulsar-connectors.nar-conventions — NAR packaging with lazy artifact resolution, disables the default JAR task, and publishes NAR as the project artifact for runtimeElements and apiElements configurations
• Created pulsar-connectors.shadow-conventions — Shadow JAR as primary artifact (matching upstream pulsar.shadow-conventions)
• Created pulsar-connectors.code-quality-conventions — Spotless license header enforcement
• Root build.gradle.kts reduced from ~286 lines to ~104 lines (RAT + allprojects + IDEA copyright + version tooling)
• All 37 Java subprojects now apply convention plugins explicitly in their own plugins {} block
• Non-Java modules (distribution/io, docker/pulsar-all) use base plugin instead of disabled java-library

Lazy configuration (configure-on-demand support):

• tasks.withType<X>().configureEach instead of eager tasks.withType<X>
• Lazy NAR bundled-dependencies resolution using configurations.named().map {} instead of eager .get()
• providers.gradleProperty() instead of System.getProperty() for test system properties

Dependency management improvements:

• Renamed pulsar-dependencies to pulsar-connectors-dependencies
• Updated platform module to detect and import BOMs as api(platform(...)) instead of constraints (matching upstream commit d155a47)
• Added 15 BOM entries: slf4j-bom, log4j-bom, protobuf-bom, grpc-bom, jetty-bom, simpleclient-bom, opentelemetry-bom (+ alpha/instrumentation variants), okhttp3-bom, okio-bom, mockito-bom, testcontainers-bom, aws-java-sdk-bom
• Added pulsarConnectorsDependencies extension to java-conventions for controlling how the shared platform is applied — subprojects can exclude specific dependencies from enforcement (e.g., exclude(group = "io.netty", module = "netty-bom")) or switch from enforcedPlatform to platform (via enforced = false) when a module needs to override versions
• Removed unused fastutil dependency (removed in upstream)

Version updates (aligned with upstream):

• Shadow plugin: 9.3.2 → 9.4.1
• commons-lang3: 3.19.0 → 3.20.0
• lightproto: 0.6.2 → 0.6.6
• Checkstyle: 10.14.2 → 13.3.0

New tooling:

• Spotless license header check (./gradlew spotlessCheck / ./gradlew spotlessApply)
• IntelliJ IDEA copyright profile (auto-applies ASL2 header via idea-ext plugin)
• version-catalog-update plugin (./gradlew versionCatalogUpdate / ./gradlew versionCatalogFormat)
• versions plugin (./gradlew dependencyUpdates for HTML report of available updates)
• Java 17 version check in settings.gradle.kts

Connector doc generator (docs/ module):

• Fixed ConnectorDocGenerator to scan NAR files from the classpath using proper jar:file: URLs
• Added generateConnectorDocs Gradle task (JavaExec) with input/output tracking for up-to-date checks
• Added exportClasspath task and pulsar-io-gen.sh shell script as an alternative way to run the generator
• Excluded transitive BOM dependencies from alluxio to avoid resolution conflicts

Alluxio connector fixes:

• Aligned alluxio dependency versions with branch-4.2 upstream
• Excluded jetty-bom from enforcedPlatform added by java-conventions to avoid version conflicts
• Fixed a bug in AlluxioSink implementation
• Improved AlluxioSinkTest

Test improvements:

• testLogging { events("FAILED"); exceptionFormat = FULL } — show test failure details
• failFast property support on test tasks
• Support for -PtestShowOutput to display test stdout/stderr
• Compiler flags: -Xlint:deprecation, -Xlint:unchecked

CI & documentation:

• CI workflow updated to run spotlessCheck alongside RAT
• README updated with prerequisites (Java 17+), Gradle wrapper note, and improved build/test commands

Verifying this change

• Make sure that the change passes the CI checks.

This change is already covered by existing tests. Additionally verified:

• ./gradlew assemble — all modules compile and package successfully
• ./gradlew --configuration-cache assemble — configuration cache works ("Configuration cache entry reused")
• ./gradlew spotlessCheck — all license headers pass
• ./gradlew rat — license audit passes

Does this pull request potentially affect one of the following parts:

• Dependencies (add or upgrade a dependency)
• The public API
• The schema
• The default values of configurations
• The threading model
• The binary protocol
• The REST endpoints
• The admin CLI options
• The metrics
• Anything that affects deployment

Documentation

• doc-complete