feat: Add GitHub Action to auto-sync capabilities matrix#56
Open
paigealleman-sketch wants to merge 1 commit intomainfrom
Open
feat: Add GitHub Action to auto-sync capabilities matrix#56paigealleman-sketch wants to merge 1 commit intomainfrom
paigealleman-sketch wants to merge 1 commit intomainfrom
Conversation
Add automated workflow that detects mismatches between connector documentation and the capabilities matrix, then creates a PR with fixes. Components: - .github/workflows/sync-capabilities.yml: Workflow triggered on pushes to baton/*.mdx files (except capabilities.mdx) - .github/scripts/sync-capabilities.js: Script that parses connector docs and compares against the matrix Key features: - Only processes changed connector files (not full scan) - Supports manual trigger with specific connector or "all" option - High-confidence detection to minimize false positives Detection logic: - Parses capabilities tables in connector docs for provisioning info - Detects account provisioning/deprovisioning from explicit text patterns - Compares only changed files against the matrix Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Contributor
📝 WalkthroughWalkthroughIntroduces an automated workflow that synchronizes a capabilities matrix with connector documentation files. A Node.js script parses both the matrix and connector docs, detects mismatches, updates the matrix, and outputs results. A GitHub Actions workflow orchestrates this process on pushes to main or manual trigger, creating a pull request if changes are detected. Changes
Sequence Diagram(s)sequenceDiagram
actor GitHub
participant Workflow as GitHub Actions<br/>Workflow
participant NodeScript as Sync Script<br/>(Node.js)
participant FileSystem as File System
participant PRTool as PR Creation<br/>(peter-evans)
GitHub->>Workflow: Push event to main OR manual dispatch
Workflow->>Workflow: Determine changed connector files<br/>(based on trigger type)
Workflow->>Workflow: Set up Node.js environment
Workflow->>NodeScript: Execute sync-capabilities.js<br/>with file list
NodeScript->>FileSystem: Read capabilities.mdx matrix
NodeScript->>FileSystem: Read connector .mdx files
NodeScript->>NodeScript: Parse matrix entries &<br/>connector documentation
NodeScript->>NodeScript: Detect capability mismatches<br/>(accounts, entitlements, deprovisions)
NodeScript->>FileSystem: Update capabilities.mdx<br/>with corrected icons
NodeScript->>FileSystem: Write sync-output.json<br/>with results & summary
NodeScript->>Workflow: Return exit code & outputs
Workflow->>Workflow: Check if changes detected<br/>from sync-output.json
alt Changes detected
Workflow->>PRTool: Trigger PR creation
PRTool->>GitHub: Create pull request<br/>with summary & labels
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Fix all issues with AI agents
In @.github/scripts/sync-capabilities.js:
- Around line 84-130: The current logic sets capabilities.deprovisionsAccounts
to false by default which causes silent docs to be treated as "does not support"
in downstream findMismatches; change deprovisionsAccounts to a tri-state (e.g.,
null or 'unknown') initially and only set it to true when
hasDeprovisioningMention is true and not hasNegativeDeprovisioning, or
explicitly set to false when hasNegativeDeprovisioning is true; update
confidence (e.g., set to 'high' when explicit negative or positive) and modify
the consumer logic (findMismatches) to skip comparisons when
capabilities.deprovisionsAccounts is unknown/null so only explicit
positive/negative mentions are compared.
- Around line 36-39: The capabilities-table regex used to set tableMatch (the
content.match call) fails to detect a table at EOF because its lookahead
requires a following blank line/heading/HTML tag; update that regex to also
allow end-of-string by adding an alternative for end-of-input (e.g., include |$
in the lookahead). Locate the content.match(...) expression that produces
tableMatch and adjust the final (?=\n\n|\n##|\n<) lookahead to
(?=\n\n|\n##|\n<|$) so tables at file end are recognized.
- Around line 148-151: The extraction of fileName from link currently captures a
path like "v1/aws" (linkMatch → fileName) which doesn't match the matrix keys
that use the basename ("aws"); update the normalization in the block that
computes fileName (the linkMatch/fileName logic) to take only the last path
segment (e.g., split on '/' and use the final element or use path.basename) and
also strip any file extension like ".mdx" so the resulting key exactly matches
the connector map lookup; adjust usages of fileName accordingly (same symbol:
fileName) so versioned paths no longer miss matches.
In @.github/workflows/sync-capabilities.yml:
- Around line 27-28: The workflow currently uses fetch-depth: 2 and computes
changed files with git diff HEAD~1 HEAD which misses multi-commit pushes; update
the actions/checkout step(s) (replace fetch-depth: 2) to fetch full history
(fetch-depth: 0) and change the diff command to use the push range, e.g. git
diff --name-only ${{ github.event.before }} ${{ github.sha }} (also update the
identical occurrences noted around the other checkout block at lines 40-42) so
the job computes changed files across the full push range.
Comment on lines
+36
to
+39
| // Find the capabilities table | ||
| const tableMatch = content.match(/##\s*Capabilities[\s\S]*?\|[^|]*Resource[^|]*\|[^|]*Sync[^|]*\|[^|]*Provision[^|]*\|([\s\S]*?)(?=\n\n|\n##|\n<)/i); | ||
|
|
||
| if (!tableMatch) return result; |
Contributor
There was a problem hiding this comment.
Allow capabilities table parsing at EOF.
The regex requires a blank line/heading/HTML tag after the table, so a table at end-of-file won’t be detected and hasTable stays false.
🛠️ Suggested fix
- const tableMatch = content.match(/##\s*Capabilities[\s\S]*?\|[^|]*Resource[^|]*\|[^|]*Sync[^|]*\|[^|]*Provision[^|]*\|([\s\S]*?)(?=\n\n|\n##|\n<)/i);
+ const tableMatch = content.match(/##\s*Capabilities[\s\S]*?\|[^|]*Resource[^|]*\|[^|]*Sync[^|]*\|[^|]*Provision[^|]*\|([\s\S]*?)(?=\n\n|\n##|\n<|$)/i);📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| // Find the capabilities table | |
| const tableMatch = content.match(/##\s*Capabilities[\s\S]*?\|[^|]*Resource[^|]*\|[^|]*Sync[^|]*\|[^|]*Provision[^|]*\|([\s\S]*?)(?=\n\n|\n##|\n<)/i); | |
| if (!tableMatch) return result; | |
| // Find the capabilities table | |
| const tableMatch = content.match(/##\s*Capabilities[\s\S]*?\|[^|]*Resource[^|]*\|[^|]*Sync[^|]*\|[^|]*Provision[^|]*\|([\s\S]*?)(?=\n\n|\n##|\n<|$)/i); | |
| if (!tableMatch) return result; |
🤖 Prompt for AI Agents
In @.github/scripts/sync-capabilities.js around lines 36 - 39, The
capabilities-table regex used to set tableMatch (the content.match call) fails
to detect a table at EOF because its lookahead requires a following blank
line/heading/HTML tag; update that regex to also allow end-of-string by adding
an alternative for end-of-input (e.g., include |$ in the lookahead). Locate the
content.match(...) expression that produces tableMatch and adjust the final
(?=\n\n|\n##|\n<) lookahead to (?=\n\n|\n##|\n<|$) so tables at file end are
recognized.
Comment on lines
+84
to
+130
| const capabilities = { | ||
| fileName, | ||
| cloudHosted: false, | ||
| selfHosted: false, | ||
| provisionsEntitlements: false, | ||
| provisionsAccounts: false, | ||
| deprovisionsAccounts: false, | ||
| confidence: 'low', | ||
| }; | ||
|
|
||
| // Check for hosting options | ||
| capabilities.cloudHosted = /<Tab\s+title=["']Cloud-hosted["']/i.test(content); | ||
| capabilities.selfHosted = /<Tab\s+title=["']Self-hosted["']/i.test(content); | ||
|
|
||
| // Use table info as primary source | ||
| if (tableInfo.hasTable) { | ||
| capabilities.provisionsAccounts = tableInfo.accountsCanProvision; | ||
| capabilities.provisionsEntitlements = tableInfo.entitlementsCanProvision; | ||
| capabilities.confidence = 'high'; | ||
| } | ||
|
|
||
| // Check for account provisioning text (high confidence patterns) | ||
| const hasExplicitAccountProvisioning = | ||
| /supports?\s+(?:\[)?automatic\s+account\s+provisioning/i.test(content) || | ||
| /connector\s+supports?\s+(?:\[)?(?:automatic\s+)?account\s+provisioning/i.test(content); | ||
|
|
||
| if (hasExplicitAccountProvisioning) { | ||
| capabilities.provisionsAccounts = true; | ||
| capabilities.confidence = 'high'; | ||
| } | ||
|
|
||
| // Check for deprovisioning - must have explicit positive mention | ||
| const hasDeprovisioningMention = | ||
| /provisioning\s+and\s+deprovisioning/i.test(content) || | ||
| /account\s+provisioning\s+and\s+deprovisioning/i.test(content); | ||
|
|
||
| const hasNegativeDeprovisioning = | ||
| /does\s+not\s+support\s+(?:account\s+)?deprovisioning/i.test(content) || | ||
| /cannot\s+deprovision/i.test(content) || | ||
| /must\s+deprovision\s+(?:accounts?\s+)?directly/i.test(content); | ||
|
|
||
| if (hasDeprovisioningMention && !hasNegativeDeprovisioning) { | ||
| capabilities.deprovisionsAccounts = true; | ||
| } else if (hasNegativeDeprovisioning) { | ||
| capabilities.deprovisionsAccounts = false; | ||
| capabilities.confidence = 'high'; | ||
| } |
Contributor
There was a problem hiding this comment.
Avoid flagging deprovisioning mismatches when docs are silent.
deprovisionsAccounts defaults to false, and findMismatches always compares it, so connectors without any deprovisioning mention are treated as “does not support,” which can trigger noisy PRs. Track an “unknown” state and only compare when there’s an explicit positive/negative mention.
🛠️ Suggested fix (tri-state + gated comparison)
const capabilities = {
fileName,
cloudHosted: false,
selfHosted: false,
provisionsEntitlements: false,
provisionsAccounts: false,
- deprovisionsAccounts: false,
+ deprovisionsAccounts: null,
confidence: 'low',
};
@@
- if (hasDeprovisioningMention && !hasNegativeDeprovisioning) {
- capabilities.deprovisionsAccounts = true;
- } else if (hasNegativeDeprovisioning) {
- capabilities.deprovisionsAccounts = false;
- capabilities.confidence = 'high';
- }
+ if (hasDeprovisioningMention && !hasNegativeDeprovisioning) {
+ capabilities.deprovisionsAccounts = true;
+ capabilities.confidence = 'high';
+ } else if (hasNegativeDeprovisioning) {
+ capabilities.deprovisionsAccounts = false;
+ capabilities.confidence = 'high';
+ }
@@
- if (docCaps.deprovisionsAccounts !== matrixCaps.provisioning.deprovisions) {
+ if (docCaps.deprovisionsAccounts !== null &&
+ docCaps.deprovisionsAccounts !== matrixCaps.provisioning.deprovisions) {Also applies to: 198-209
🤖 Prompt for AI Agents
In @.github/scripts/sync-capabilities.js around lines 84 - 130, The current
logic sets capabilities.deprovisionsAccounts to false by default which causes
silent docs to be treated as "does not support" in downstream findMismatches;
change deprovisionsAccounts to a tri-state (e.g., null or 'unknown') initially
and only set it to true when hasDeprovisioningMention is true and not
hasNegativeDeprovisioning, or explicitly set to false when
hasNegativeDeprovisioning is true; update confidence (e.g., set to 'high' when
explicit negative or positive) and modify the consumer logic (findMismatches) to
skip comparisons when capabilities.deprovisionsAccounts is unknown/null so only
explicit positive/negative mentions are compared.
Comment on lines
+148
to
+151
| // Extract the file name from the link | ||
| const linkMatch = link.match(/\/baton\/([^)]+)/); | ||
| const fileName = linkMatch ? linkMatch[1] : null; | ||
|
|
Contributor
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Inspect connector links in capabilities.mdx
fd -a 'capabilities.mdx' -x rg -n '\]\([^)]*baton/[^)]*\)' -A 1 -B 1Repository: ConductorOne/docs
Length of output: 31027
🏁 Script executed:
#!/bin/bash
# Examine the sync-capabilities.js file around the flagged lines and downstream
cat -n .github/scripts/sync-capabilities.js | head -200Repository: ConductorOne/docs
Length of output: 7912
🏁 Script executed:
#!/bin/bash
# Check if the map key usage is actually mismatching
# Search for where fileName is used after being set
rg -n 'fileName' .github/scripts/sync-capabilities.js -B 3 -A 3Repository: ConductorOne/docs
Length of output: 2108
Normalize connector filename keys to match file structure.
The link extraction captures the full path component (e.g., v1/aws from /baton/v1/aws), but the matrix lookup uses only the basename of the actual file (e.g., aws from baton/v1/aws.mdx). This causes versioned connectors to be silently skipped from comparisons because the map lookup fails.
🛠️ Suggested fix
const linkMatch = link.match(/\/baton\/([^)]+)/);
- const fileName = linkMatch ? linkMatch[1] : null;
+ const linkedPath = linkMatch ? linkMatch[1].split('#')[0] : null;
+ const fileName = linkedPath ? path.basename(linkedPath, '.mdx') : null;🤖 Prompt for AI Agents
In @.github/scripts/sync-capabilities.js around lines 148 - 151, The extraction
of fileName from link currently captures a path like "v1/aws" (linkMatch →
fileName) which doesn't match the matrix keys that use the basename ("aws");
update the normalization in the block that computes fileName (the
linkMatch/fileName logic) to take only the last path segment (e.g., split on '/'
and use the final element or use path.basename) and also strip any file
extension like ".mdx" so the resulting key exactly matches the connector map
lookup; adjust usages of fileName accordingly (same symbol: fileName) so
versioned paths no longer miss matches.
Comment on lines
+27
to
+28
| with: | ||
| fetch-depth: 2 |
Contributor
There was a problem hiding this comment.
Compute changed files across the full push range.
git diff HEAD~1 HEAD only captures the last commit in a push. Multi-commit pushes will miss earlier changes, and fetch-depth: 2 may not include github.event.before. Use the push range and fetch enough history.
🛠️ Suggested fix
- name: Checkout repository
uses: actions/checkout@v4
with:
- fetch-depth: 2
+ fetch-depth: 0
@@
- CHANGED=$(git diff --name-only HEAD~1 HEAD -- 'baton/*.mdx' | grep -v capabilities.mdx | grep -v '^baton/_' | grep -v '^baton/baton-' | tr '\n' ' ')
+ CHANGED=$(git diff --name-only "${{ github.event.before }}" "${{ github.sha }}" -- 'baton/*.mdx' | grep -v capabilities.mdx | grep -v '^baton/_' | grep -v '^baton/baton-' | tr '\n' ' ')Also applies to: 40-42
🤖 Prompt for AI Agents
In @.github/workflows/sync-capabilities.yml around lines 27 - 28, The workflow
currently uses fetch-depth: 2 and computes changed files with git diff HEAD~1
HEAD which misses multi-commit pushes; update the actions/checkout step(s)
(replace fetch-depth: 2) to fetch full history (fetch-depth: 0) and change the
diff command to use the push range, e.g. git diff --name-only ${{
github.event.before }} ${{ github.sha }} (also update the identical occurrences
noted around the other checkout block at lines 40-42) so the job computes
changed files across the full push range.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds automated workflow to detect and fix mismatches between connector documentation and the capabilities matrix (
baton/capabilities.mdx).How it works
Components
Workflow (
.github/workflows/sync-capabilities.yml):baton/*.mdx(exceptcapabilities.mdx)git diffScript (
.github/scripts/sync-capabilities.js):Detection Logic
Usage
Test plan
🤖 Generated with Claude Code
Summary by CodeRabbit