Skip to content

Add ScyllaDB Operator compatibility scraper - #4164

Open
goodguypeci-design wants to merge 3 commits into
pluralsh:masterfrom
goodguypeci-design:feat/scylladb-operator-compatibility
Open

Add ScyllaDB Operator compatibility scraper#4164
goodguypeci-design wants to merge 3 commits into
pluralsh:masterfrom
goodguypeci-design:feat/scylladb-operator-compatibility

Conversation

@goodguypeci-design

@goodguypeci-design goodguypeci-design commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

I added ScyllaDB Operator to the compatibility catalog.

The scraper uses the operator's versioned release docs for the Kubernetes support matrix, then matches stable operator releases to chart versions from the official ScyllaDB chart index. I only included releases where both the versioned docs and chart entry are present. Older releases without those sources are skipped rather than guessed.

Sources:

I checked this with py_compile, the ScyllaDB scraper tests, a direct live scraper run through Helm, a generated YAML sanity check for versions/chart versions/images, and git diff --check.

Related to #4132.

@greptile-apps

greptile-apps Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds ScyllaDB Operator to the compatibility catalog and introduces a scraper that combines stable GitHub releases, versioned support-matrix documentation, and Helm chart metadata.

  • Registers the operator in the compatibility manifest.
  • Seeds compatibility metadata for operator releases 1.22.0 and 1.21.1.
  • Selects the latest stable release from each minor line and skips releases without matching documentation or charts.
  • Adds parser and row-generation tests, with one gap in validating per-minor support data.

Confidence Score: 4/5

The PR appears safe to merge, with a non-blocking test-coverage gap around per-minor Kubernetes support mapping.

Scraper discovery, output paths, chart mapping, and static schema integration follow repository conventions; the only accepted concern is that the new test would not detect accidental reuse of one minor’s documentation for another.

Files Needing Attention: utils/compatibility/tests/test_scylladb_operator.py

Important Files Changed

Filename Overview
utils/compatibility/scrapers/scylladb-operator.py Adds the release, documentation, and Helm chart scraper; its integration follows existing compatibility utility contracts.
utils/compatibility/tests/test_scylladb_operator.py Covers support-range parsing and documented-release filtering but does not verify the distinct support matrix generated for the second minor.
static/compatibilities/scylladb-operator.yaml Adds schema-compatible static compatibility data for two ScyllaDB Operator releases.
static/compatibilities/manifest.yaml Registers the new scraper and static compatibility entry using established naming conventions.

Reviews (1): Last reviewed commit: "Add ScyllaDB Operator compatibility scra..." | Re-trigger Greptile

Comment on lines +78 to +80
self.assertEqual([row["version"] for row in rows], ["1.22.0", "1.21.1"])
self.assertEqual(rows[0]["kube"], ["1.33", "1.34", "1.35", "1.36"])
self.assertEqual(rows[0]["chart_version"], "1.22.0")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Minor support goes unchecked

The fixture gives the 1.21 documentation a different Kubernetes support range, but the test only checks the 1.22 row. It would therefore still pass if the scraper accidentally reused the 1.22 documentation for every minor, leaving the per-minor compatibility mapping unprotected.

Suggested change
self.assertEqual([row["version"] for row in rows], ["1.22.0", "1.21.1"])
self.assertEqual(rows[0]["kube"], ["1.33", "1.34", "1.35", "1.36"])
self.assertEqual(rows[0]["chart_version"], "1.22.0")
self.assertEqual([row["version"] for row in rows], ["1.22.0", "1.21.1"])
self.assertEqual(rows[0]["kube"], ["1.33", "1.34", "1.35", "1.36"])
self.assertEqual(rows[1]["kube"], ["1.32", "1.33", "1.34", "1.35"])
self.assertEqual(rows[0]["chart_version"], "1.22.0")

@goodguypeci-design
goodguypeci-design force-pushed the feat/scylladb-operator-compatibility branch from e919f9d to 3e408c5 Compare September 8, 2026 13:55
@soffi-ai

soffi-ai Bot commented Sep 8, 2026

Copy link
Copy Markdown
Soffi AI Summary

This PR adds ScyllaDB Operator to the Plural Console compatibility catalog — giving the platform visibility into which Kubernetes versions each ScyllaDB Operator release supports.

The implementation consists of a Python scraper (utils/compatibility/scrapers/scylladb-operator.py) that:

  1. Fetches stable ScyllaDB Operator releases from the GitHub API, grouping them by minor version to track the latest patch per minor.
  2. Pulls the versioned release docs from operator.docs.scylladb.com for each minor and parses the "Support matrix" Markdown table to extract the supported Kubernetes version range (handling both comma-separated lists and hyphen-delimited ranges).
  3. Cross-references operator versions against chart entries from the official ScyllaDB Helm index (scylla-operator-charts.storage.googleapis.com/stable) — skipping any release that lacks either docs or a published chart.
  4. Writes the resulting compatibility rows (version, Kubernetes versions, chart version, image reference) to static/compatibilities/scylladb-operator.yaml.

Tests cover the Kubernetes version range parser (pure ranges, mixed list+range) and the row-building logic (ensuring only releases with published docs and chart entries are included). A secondary fix patches the test loader to inject the local utils module so the scraper can import it correctly in isolation.

The generated static/compatibilities/scylladb-operator.yaml (20 rows) and updated manifest.yaml are committed alongside the scraper.

Changes

ScyllaDB Operator compatibility scraper

  • Adds the initial ScyllaDB Operator scraper. It fetches stable releases from GitHub, resolves the latest patch per minor version, cross-references chart entries from the Helm index, parses the Kubernetes support matrix from the versioned release docs, and writes the compatibility rows to static/compatibilities/scylladb-operator.yaml. (4392e04)
  • Extends the Kubernetes version parser to handle per-minor support mappings — supporting both hyphen-delimited ranges (e.g. 1.33 - 1.36) and mixed comma-separated lists with embedded ranges (e.g. 1.31, 1.33 - 1.35), ensuring full expansion to individual minor versions. (ae72635)
  • Fixes the test loader so that the local utils module is correctly injected into sys.modules before the scraper is imported, allowing the scraper's from utils import ... statements to resolve during isolated unit test runs. (3e408c5)

Updated: 2026-09-08 13:57 UTC

Deploy in Soffi

@goodguypeci-design

Copy link
Copy Markdown
Contributor Author

Rebased this on the current master and re-ran the focused compatibility checks locally.

The required-label check is failing because the PR has no label, but GitHub will not let me add the enhancement label from my fork account. Could you add the label when you get a chance?

@goodguypeci-design

Copy link
Copy Markdown
Contributor Author

I checked the failing CI logs as well. The compatibility checks themselves are still green; the red Test job stops during docker/login-action with Password required, then the Slack notification step fails because SLACK_WEBHOOK_URL is empty.

So this looks like the usual missing repo secrets on a forked PR, plus the label check I mentioned above, rather than a scraper/test failure from this patch.

@goodguypeci-design

Copy link
Copy Markdown
Contributor Author

Pushed 3e408c5 to tighten the fixture around the review note: the test now checks both ScyllaDB operator rows, so reusing one minor support range for every release would fail. python -m unittest tests.test_scylladb_operator -v passes locally (3 tests).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant