Skip to content

Add an enterprise-candidate MySQL profile - #2

Merged
jiangxt2 merged 1 commit into
masterfrom
feat/enterprise-mysql-profile
Aug 11, 2026
Merged

Add an enterprise-candidate MySQL profile#2
jiangxt2 merged 1 commit into
masterfrom
feat/enterprise-mysql-profile

Conversation

@jiangxt2

Copy link
Copy Markdown
Owner

What changes

This PR adds an enterprise-candidate MySQL profile to ray-doris:

  • Environment-referenced credentials. password_env stores only an environment-variable name in the serialized datasource and ReadTask payloads. The driver resolves the value before each DESCRIBE and query-plan request, and a worker resolves it again before every split connection attempt, so Ray retries never reuse a stale credential. It is mutually exclusive with a non-empty literal password. An unavailable variable raises a fixed, redacted DorisConfigurationError before the affected driver request or worker connection attempt, and the resolved value is never written back into the serialized configuration.
  • Private-CA query planning and independent timeouts. http_ca_file loads a private CA into a hostname-verifying TLS context for the _query_plan endpoint and is valid only with http_scheme="https"; there is no trust-all or hostname-bypass option. query_plan_timeout bounds query-plan request I/O independently and defaults to connect_timeout for compatibility.
  • Fixed redacted TLS errors, fail closed. Driver and worker MySQL TLS setup and validation failures, HTTPS CA/hostname failures, missing credentials, and authentication and permission rejections now produce fixed, redacted public errors and never trigger query-plan single_task or transport auto fallback. Sensitive failure tests assert four-layer redaction (message, formatted traceback chain, captured logs, and absent __cause__).
  • Minimum-privilege distributed evidence. The slow distributed suite moves Ray worker-retry and Doris backend-failure scenarios to the MySQL path, using a dedicated SELECT_PRIV reader with strict TLS; Flight remains an experimental regression path.
  • Release evidence binding. Releases now require a successful full slow-suite manifest from the exact candidate SHA and workflow run (discovering both direct and reusable-workflow runs), with the initial cluster topology recorded in the manifest.
  • FE availability boundary. The connector documents one logical FE hostname: it validates TLS and reopens connections per request, while FE election, quorum, and cross-endpoint failover remain deployment responsibilities.

Why

Literal passwords remained in serialized datasource and task state. The query-plan endpoint had no private-CA option and shared connect_timeout with other setup operations. MySQL TLS failures were not consistently classified. Distributed worker-retry and backend-failure evidence lived on the experimental Flight path with the root account. Releases were not bound to slow-suite evidence, so a commit without a successful full slow run could be published.

User-facing changes

Three optional parameters are added with backward-compatible defaults: password_env, http_ca_file, and query_plan_timeout. The enterprise-candidate MySQL profile combines password_env with strict MySQL TLS, HTTPS query planning with hostname verification, on_query_plan_error="error", explicit timeouts, and a minimum-privilege reader. The connector accepts one logical FE hostname and does not discover FE members or fail over between endpoints. An unavailable environment credential fails before the affected driver request or worker connection attempt; all TLS, authentication, and permission failures produce fixed, redacted public errors and never degrade to an unencrypted or fallback path.

How tested

  • Unit suite: 204 passed with 93% coverage on the primary baseline.
  • Static quality: Ruff, mypy, pre-commit, actionlint, and git diff --check all passed.
  • Documentation: contract checks, strict Sphinx build, spelling, and link checks all passed.
  • Packaging: wheel/sdist build, Twine check, and isolated installs all passed.
  • Required Doris 4.0.6 integration suite: 22 passed in 53.876s on the final tree, covering environment credentials on the driver and Ray workers, fail-before-network behavior, re-resolution at split start, wrong-password fail-closed, and minimum-privilege reads.
  • Full slow distributed suite: pending on the candidate commit SHA; a successful full run on the exact release SHA is a release gate.
  • Release verification: tools/check_slow_result.py is unit-tested for exact-SHA, run-ID, profile, and topology binding; the full release dry-run follows the candidate slow run.

This PR does not implement FE discovery, leader election, quorum, health checks, or cross-endpoint failover; it does not change tablet grouping, add SQL or type capabilities, or modify branch protection, tag rulesets, or release environments.

Resolve environment-referenced credentials independently before each
driver planning request and each worker connection attempt, without
serializing resolved values.

Add private-CA query planning, an independent query-plan timeout, and
fixed redacted MySQL TLS error classification. Authentication,
permission, TLS, and missing-credential failures fail closed without
transport fallback.

Move distributed worker-retry and backend-failure evidence to the
minimum-privilege MySQL path with strict TLS, and bind releases to a
successful full slow-suite manifest from the exact candidate SHA and
workflow run, recording the initial cluster topology in the manifest.

Document the logical FE endpoint contract and keep FE election, quorum,
and backend failover as deployment concerns.

Signed-off-by: jiangxt2 <jiangxt2@vip.qq.com>
@jiangxt2
jiangxt2 marked this pull request as ready for review August 11, 2026 15:20
@jiangxt2
jiangxt2 merged commit 086160e into master Aug 11, 2026
13 of 14 checks passed
@jiangxt2
jiangxt2 deleted the feat/enterprise-mysql-profile branch August 11, 2026 15:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant