-
Notifications
You must be signed in to change notification settings - Fork 85
feat: add chaitin cloudwalker service adapter #457
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
bingtaosong484-ops
wants to merge
4
commits into
chaitin:main
Choose a base branch
from
bingtaosong484-ops:feature/Chaitin_CloudWalker
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
743cf34
feat: add chaitin cloudwalker service adapter
Lunaille e385bae
fix: align cloudwalker service with OctoBus conventions
Lunaille e6928cb
fix: address PR review comments
Lunaille a3a741e
fix: detect 302 session redirect as UNAUTHENTICATED
Lunaille File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| #!/usr/bin/env node | ||
|
|
||
| import { fileURLToPath } from "node:url"; | ||
| import { runServiceMain } from "@chaitin-ai/octobus-sdk"; | ||
|
|
||
| import { service } from "../chaitin__cloudwalker/src/service.js"; | ||
|
|
||
| runServiceMain(service, { | ||
| entryFile: fileURLToPath(new URL("../chaitin__cloudwalker/bin/cloudwalker.js", import.meta.url)), | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,158 @@ | ||
| # CloudWalker OctoBus Service | ||
|
|
||
| 长亭科技牧云(CloudWalker)集群与漏洞只读查询适配器。 | ||
|
|
||
| ```bash | ||
| octobus service import cloudwalker ./services/chaitin__cloudwalker | ||
| ``` | ||
|
|
||
| ## Package Files | ||
|
|
||
| - `service.json` — OctoBus 服务清单,声明运行模式与 proto 入口。 | ||
| - `proto/cloudwalker.proto` — gRPC API 定义,6 个 unary 方法。 | ||
| - `config.schema.json` — 非密配置:`baseUrl`、`referer`。 | ||
| - `secret.schema.json` — 密钥配置:`token`(必填)、`cookie`(可选)。 | ||
| - `src/cloudwalker.js` — 上游 REST API 请求映射、响应归一化、错误分类。 | ||
| - `src/service.js` — OctoBus SDK `defineService` 封装。 | ||
| - `bin/cloudwalker.js` — 服务本地可执行入口。 | ||
| - `test/cloudwalker.test.js` — node:test 覆盖:请求映射、响应归一化、错误分类、SDK handler。 | ||
| - `test/cloudwalker-client.test.js` — 扩展客户端测试:HTML 响应检测、fallback 逻辑、零值保留。 | ||
| - `test/mock_upstream.js` — 本地 mock 上游,覆盖成功 / 认证失败 / 5xx / 超时。 | ||
|
|
||
| ## 支持版本 | ||
|
|
||
| - **目标产品**: 长亭科技牧云 CloudWalker | ||
| - **适配版本**: VM-S10-26.06.002 | ||
|
|
||
| ## 认证方式 | ||
|
|
||
| Token + Browser Session Cookie 组合认证: | ||
|
|
||
| | 字段 | 必填 | 说明 | | ||
| |------|------|------| | ||
| | `token` | 是 | API Token,通过牧云控制台「个人中心 → API Token」生成 | | ||
| | `cookie` | 视环境 | 浏览器 Session Cookie;Demo 环境需要,正式环境可能仅需 token | | ||
|
|
||
| 认证头同时发送三种格式以兼容不同牧云版本:`Authorization: Bearer <token>`、`token: <token>`、`x-auth-token: <token>`。 | ||
|
|
||
| ## Configuration | ||
|
|
||
| ```json | ||
| { | ||
| "baseUrl": "https://cnapp.demo.chaitin.cn", | ||
| "referer": "https://cnapp.demo.chaitin.cn/profile/apitoken" | ||
| } | ||
| ``` | ||
|
|
||
| | 字段 | 类型 | 必填 | 说明 | | ||
| |------|------|------|------| | ||
| | `baseUrl` | string | 是 | 牧云 API 基础 URL | | ||
| | `referer` | string | 否 | 浏览器 Referer 头,部分环境需要 | | ||
|
|
||
| Secret: | ||
|
|
||
| ```json | ||
| { | ||
| "token": "TMCpan#2VB44wwF...", | ||
| "cookie": "_c_WBKFRo=...; veinmind=..." | ||
| } | ||
| ``` | ||
|
|
||
| | 字段 | 类型 | 必填 | 说明 | | ||
| |------|------|------|------| | ||
| | `token` | string | 是 | API Token | | ||
| | `cookie` | string | 否 | 浏览器 Session Cookie(含 httpOnly 的 veinmind) | | ||
|
|
||
| Handler 优先使用 `ctx.config` / `ctx.secret`,其次回退到环境变量 `CLOUDWALKER_BASE_URL`、`CLOUDWALKER_TOKEN`、`CLOUDWALKER_COOKIE`、`CLOUDWALKER_REFERER`。 | ||
|
|
||
| ## RPC Methods | ||
|
|
||
| | 方法 | 上游 API | 说明 | | ||
| |------|----------|------| | ||
| | `Chaitin_CloudWalker.Chaitin_CloudWalker/ListClusters` | `GET /cluster/cluster_list` | 查询集群列表,支持 name / status 过滤与分页 | | ||
| | `Chaitin_CloudWalker.Chaitin_CloudWalker/GetClusterInfo` | `GET /cluster/cluster_info` | 按集群 ID 获取详情 | | ||
| | `Chaitin_CloudWalker.Chaitin_CloudWalker/ListClusterVulnEvents` | `GET /cluster_vuln/vuln_event_list` | 查询集群漏洞事件列表,支持 CVE / CNVD / risk / state 等过滤 | | ||
| | `Chaitin_CloudWalker.Chaitin_CloudWalker/GetClusterVulnEvent` | `GET /cluster_vuln/vuln_event_info` | 按事件 ID 获取集群漏洞详情 | | ||
| | `Chaitin_CloudWalker.Chaitin_CloudWalker/ListMicroserviceVulnEvents` | `GET /cluster_microservice/vuln_event_list` | 查询微服务漏洞事件列表,支持 serviceName / clusterName 等过滤 | | ||
| | `Chaitin_CloudWalker.Chaitin_CloudWalker/GetMicroserviceVulnEvent` | `GET /cluster_microservice/vuln_event_info` | 按事件 ID 获取微服务漏洞详情 | | ||
|
|
||
| ## Behavior Notes | ||
|
|
||
| - **Proto3 零值参数跳过**:gRPC 反序列化后 int32 字段(如 `status`、`order`)默认值为 0,`appendScalarQuery` 主动跳过值为 0 的数字参数,避免发送 `status=0` 导致上游 "Validation Failed"。 | ||
| - **Fallback 过滤**:`clusterName`、`cnvd`、`cnnvd` 等参数在 Demo 上游不稳定,客户端先尝试直接查询,失败后自动降级为全量扫描 + 客户端过滤。 | ||
| - **HTML 响应检测**:Session 过期时上游返回 200 + HTML 页面而非 JSON,客户端识别 `<!doctype html>` 并触发 fallback 或抛出 `UNAVAILABLE` 错误。 | ||
| - **snake_case → camelCase**:上游 API 返回 snake_case 字段,客户端自动转换为 camelCase 以匹配 proto JSON 格式。 | ||
| - **非 JSON 内容守卫**:响应 Content-Type 非 JSON 时直接拒绝,防止将 HTML 误解析为业务数据。 | ||
|
|
||
| ### 错误映射 | ||
|
|
||
| | 上游 HTTP 状态 | gRPC 状态码 | | ||
| |----------------|-------------| | ||
| | 400 | `INVALID_ARGUMENT` | | ||
| | 401 | `UNAUTHENTICATED` | | ||
| | 403 | `PERMISSION_DENIED` | | ||
| | 404 | `NOT_FOUND` | | ||
| | 409 | `ALREADY_EXISTS` | | ||
| | 412 | `FAILED_PRECONDITION` | | ||
| | 429 | `RESOURCE_EXHAUSTED` | | ||
| | 504 | `DEADLINE_EXCEEDED` | | ||
| | 其他 5xx / 网络错误 | `UNAVAILABLE` | | ||
|
|
||
| ## Risk Boundary | ||
|
|
||
| - **风险等级**: `read-only` | ||
| - **写操作**: 本版本无写操作,所有 6 个方法均为只读查询。 | ||
| - 本适配器不修改牧云平台任何数据。 | ||
|
|
||
| ## Suggested Capset | ||
|
|
||
| - `prod` — 生产环境只读查询 | ||
|
|
||
| ```bash | ||
| octobus capset create prod | ||
| octobus capset add-instance prod --service cloudwalker --instance cloudwalker-demo | ||
| ``` | ||
|
|
||
| ## Known Limitations | ||
|
|
||
| - 第一版仅覆盖 6 个只读查询能力,不支持漏洞处置、集群纳管等写操作。 | ||
| - Demo 环境的 `clusterName`、`cnvd`、`cnnvd` 过滤参数不稳定,需通过 fallback 机制兜底。 | ||
| - Cookie 中的 `veinmind` 为 httpOnly,需通过浏览器 CDP 获取,无法从 `document.cookie` 读取。 | ||
| - `ListClustersRequest.status` 为 int32 类型,proto3 默认值为 0,不支持通过 0 值筛选集群状态;需传正整数(如 1=运行中、2=异常)。 | ||
| - Fallback 模式下(`clusterName`/`cnvd`/`cnnvd` 触发客户端过滤),`collectFilteredItems` 在命中 `pageSize` 数量后提前中断当前页扫描,返回的 `nextPageToken` 为上游最后一页的全局 token,可能跳过当前页剩余未扫描项。建议一次性拉取较大 `pageSize` 以减少分页遗漏风险。 | ||
|
|
||
| ## Local Checks | ||
|
|
||
| ```bash | ||
| cd services | ||
| npm run validate -- --service-dir chaitin__cloudwalker | ||
| npm test -- --service-dir chaitin__cloudwalker | ||
| npm run pack:check | ||
| ``` | ||
|
|
||
| ## OctoBus Runtime Validation | ||
|
|
||
| ```bash | ||
| # 导入服务 | ||
| octobus service import cloudwalker ./services/chaitin__cloudwalker | ||
|
|
||
| # 创建实例 | ||
| octobus instance create cloudwalker-demo \ | ||
| --service cloudwalker \ | ||
| --config-json '{"baseUrl":"https://cnapp.demo.chaitin.cn","referer":"https://cnapp.demo.chaitin.cn/profile/apitoken"}' \ | ||
| --secret-json '{"token":"<TOKEN>","cookie":"<COOKIE>"}' | ||
|
|
||
| # 创建 capset 并绑定实例 | ||
| octobus capset create prod | ||
| octobus capset add-instance prod --service cloudwalker --instance cloudwalker-demo | ||
|
|
||
| # 验证方法注册 | ||
| octobus catalog prod --all --json | ||
|
|
||
| # 调用测试 | ||
| curl -X POST http://127.0.0.1:9000/capsets/prod/connect/cloudwalker-demo/CloudWalker.CloudWalker/ListClusters \ | ||
| -H 'Content-Type: application/json' \ | ||
| -d '{"pageSize": 5}' | ||
| ``` | ||
|
|
||
| **注意**:方法路径格式为 `Chaitin_CloudWalker.Chaitin_CloudWalker/<Method>`,与 proto `package Chaitin_CloudWalker` 保持一致。 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| #!/usr/bin/env node | ||
|
|
||
| import { runServiceMain } from '@chaitin-ai/octobus-sdk'; | ||
|
|
||
| import { service } from '../src/service.js'; | ||
|
|
||
| runServiceMain(service); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| { | ||
| "$schema": "http://json-schema.org/draft-07/schema#", | ||
| "type": "object", | ||
| "additionalProperties": false, | ||
| "properties": { | ||
| "baseUrl": { | ||
| "type": "string", | ||
| "description": "CloudWalker API base URL." | ||
| }, | ||
| "referer": { | ||
| "type": "string", | ||
| "description": "Browser referer header used by CloudWalker session-based authentication flows." | ||
| } | ||
| }, | ||
| "required": ["baseUrl"] | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| { | ||
| "name": "cloudwalker", | ||
| "version": "0.0.0", | ||
| "private": true, | ||
| "type": "module", | ||
| "bin": { | ||
| "cloudwalker": "bin/cloudwalker.js" | ||
| }, | ||
| "scripts": { | ||
| "test": "node --test test/cloudwalker.test.js test/cloudwalker-client.test.js" | ||
| }, | ||
| "dependencies": { | ||
| "@chaitin-ai/octobus-sdk": "^0.6.0" | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,135 @@ | ||
| syntax = "proto3"; | ||
|
|
||
| package Chaitin_CloudWalker; | ||
|
|
||
| option go_package = "miner/grpc-service/Chaitin_CloudWalker"; | ||
|
|
||
| service Chaitin_CloudWalker { | ||
| rpc ListClusters(ListClustersRequest) returns (ListClustersResponse) {} | ||
| rpc GetClusterInfo(GetClusterInfoRequest) returns (Cluster) {} | ||
| rpc ListClusterVulnEvents(ListClusterVulnEventsRequest) returns (ListVulnEventsResponse) {} | ||
| rpc GetClusterVulnEvent(GetClusterVulnEventRequest) returns (VulnEvent) {} | ||
| rpc ListMicroserviceVulnEvents(ListMicroserviceVulnEventsRequest) returns (ListVulnEventsResponse) {} | ||
| rpc GetMicroserviceVulnEvent(GetMicroserviceVulnEventRequest) returns (VulnEvent) {} | ||
| } | ||
|
|
||
| message ListClustersRequest { | ||
| int32 page_size = 1; | ||
| string page_token = 2; | ||
| string name = 3; | ||
| int32 status = 4; | ||
| } | ||
|
|
||
| message GetClusterInfoRequest { | ||
| string cluster_id = 1; | ||
| } | ||
|
|
||
| message ListClusterVulnEventsRequest { | ||
| string cluster_id = 1; | ||
| int32 page_size = 2; | ||
| string page_token = 3; | ||
| string cve = 4; | ||
| string name = 5; | ||
| string cnvd = 6; | ||
| string cnnvd = 7; | ||
| string node_name = 8; | ||
| string cluster_name = 9; | ||
| string order_by = 10; | ||
| repeated int32 risk = 11; | ||
| repeated int32 state = 12; | ||
| repeated string characteristic = 13; | ||
| int32 order = 14; | ||
| } | ||
|
|
||
| message GetClusterVulnEventRequest { | ||
| string event_id = 1; | ||
| } | ||
|
|
||
| message ListMicroserviceVulnEventsRequest { | ||
| int32 page_size = 1; | ||
| string page_token = 2; | ||
| string service_name = 3; | ||
| string service_type = 4; | ||
| string cluster_name = 5; | ||
| string name = 6; | ||
| string cve = 7; | ||
| string cnvd = 8; | ||
| string cnnvd = 9; | ||
| string order_by = 10; | ||
| repeated string characteristic = 11; | ||
| repeated int32 risk = 12; | ||
| repeated int32 state = 13; | ||
| int32 order = 14; | ||
| } | ||
|
|
||
| message GetMicroserviceVulnEventRequest { | ||
| string event_id = 1; | ||
| } | ||
|
|
||
| message ListClustersResponse { | ||
| repeated Cluster clusters = 1; | ||
| string next_page_token = 2; | ||
| } | ||
|
|
||
| message ListVulnEventsResponse { | ||
| repeated VulnEvent vuln_events = 1; | ||
| string next_page_token = 2; | ||
| } | ||
|
|
||
| message ModuleStatus { | ||
| string version = 1; | ||
| int32 module_type = 2; | ||
| int32 status = 3; | ||
| } | ||
|
|
||
| message Cluster { | ||
| string cluster_id = 1; | ||
| string cluster_name = 2; | ||
| string status = 3; | ||
| string risk_level = 4; | ||
| string created_at = 5; | ||
| string updated_at = 6; | ||
|
|
||
| // Extended fields from CloudWalker API | ||
| string api_version = 7; | ||
| repeated string master_ips = 8; | ||
| repeated ModuleStatus module_status = 9; | ||
| int32 cluster_type = 10; | ||
| int32 reachable = 11; | ||
| int32 integration_status = 12; | ||
| } | ||
|
|
||
| message VulnEvent { | ||
| string event_id = 1; | ||
| string cluster_id = 2; | ||
| string microservice_id = 3; | ||
| string microservice_name = 4; | ||
| string level = 5; | ||
| string status = 6; | ||
| string title = 7; | ||
| string cve = 8; | ||
| string package_name = 9; | ||
| string package_version = 10; | ||
| string fixed_version = 11; | ||
| string image_name = 12; | ||
| string discovered_at = 13; | ||
| string updated_at = 14; | ||
|
|
||
| // Extended fields for better data coverage | ||
| string node_name = 15; | ||
| string cluster_name = 16; | ||
| int32 risk = 17; | ||
| int32 original_risk = 18; | ||
| int32 custom_risk = 19; | ||
| repeated string characteristic = 20; | ||
| string service_uid = 21; | ||
| string service_type = 22; | ||
| string description = 23; | ||
| string solution = 24; | ||
| int32 manage_status = 25; | ||
| bool node_exist = 26; | ||
| string first_discovery_time = 27; | ||
| string last_discovery_time = 28; | ||
| string cnvd = 29; | ||
| string cnnvd = 30; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| { | ||
| "$schema": "http://json-schema.org/draft-07/schema#", | ||
| "type": "object", | ||
| "additionalProperties": false, | ||
| "properties": { | ||
| "token": { | ||
| "type": "string", | ||
| "description": "CloudWalker API token header value." | ||
| }, | ||
| "cookie": { | ||
| "type": "string", | ||
| "description": "Browser session cookie. Required for demo environments that enforce session-based auth. May be omitted for environments that accept token-only authentication." | ||
| } | ||
| }, | ||
| "required": ["token"] | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.