-
Notifications
You must be signed in to change notification settings - Fork 84
feat(services): add ThreatBook HFish honeypot service package #276
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
boy331
wants to merge
7
commits into
chaitin:main
Choose a base branch
from
boy331:feat/threatbook-hfish
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
ce81106
feat(services): add ThreatBook HFish honeypot service package
5aec002
fix: resolve CI validate errors
32b828c
fix(hfish): remove TLS env var check entirely
b0fecfa
fix(hfish): add missing UNAUTHENTICATED to grpcCodeFor mapping
544803f
fix: set executable permissions on bin entry files
d69982a
fix(hfish): handle subpath endpoints and align data type with proto
9d66e0a
test(hfish): add regression test for subpath endpoint URLs
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| #!/usr/bin/env node | ||
|
|
||
| import { fileURLToPath } from "node:url"; | ||
| import { runServiceMain } from "@chaitin-ai/octobus-sdk"; | ||
|
|
||
| import { service } from "../threatbook__hfish/src/service.js"; | ||
|
|
||
| runServiceMain(service, { | ||
| entryFile: fileURLToPath(new URL("../threatbook__hfish/bin/threatbook-hfish.js", import.meta.url)), | ||
| }); |
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,83 @@ | ||
| # ThreatBook HFish Honeypot OctoBus Service | ||
|
|
||
| OctoBus service package for [HFish](https://hfish.net/) honeypot by ThreatBook (Chaitin). Provides APIs to query attack source IPs, attack details, captured credentials, and system status. | ||
|
|
||
| ## Package Files | ||
|
|
||
| - `service.json`: OctoBus service manifest. | ||
| - `proto/hfish.proto`: gRPC API definition. | ||
| - `config.schema.json`: non-secret endpoint, headers, timeout, and TLS settings. | ||
| - `secret.schema.json`: HFish API key. | ||
| - `src/hfish.js`: HFish REST API proxy implementation. | ||
| - `src/service.js`: OctoBus SDK `defineService` wrapper. | ||
| - `bin/threatbook-hfish.js`: service-local executable entrypoint. | ||
| - `test/hfish.test.js`: node:test coverage for request validation, REST mapping, error mapping, and SDK handler invocation. | ||
| - `test/mock_upstream.js`: optional local HFish HTTP mock. | ||
|
|
||
| ## Configuration | ||
|
|
||
| Use `endpoint` for the HFish server base URL. | ||
|
|
||
| ```json | ||
| { | ||
| "endpoint": "https://hfish.example.com:4433", | ||
| "headers": { | ||
| "X-Extra": "demo" | ||
| }, | ||
| "timeoutMs": 1500, | ||
| "skipTlsVerify": true | ||
| } | ||
| ``` | ||
|
|
||
| Use `secret.apiKey` for the HFish API key: | ||
|
|
||
| ```json | ||
| { | ||
| "apiKey": "replace-with-hfish-api-key" | ||
| } | ||
| ``` | ||
|
|
||
| Requests may still pass `api_key` or `apiKey`; configured secret values take precedence over request values (because gRPC proto3 string fields default to `""` which would shadow the real secret). | ||
|
|
||
| ## RPC Methods | ||
|
|
||
| | Method | HTTP Mapping | Description | | ||
| |--------|-------------|-------------| | ||
| | `ThreatBook_HFISH.ThreatBook_HFISH/ListAttackIPs` | `POST /api/v1/attack/ip?api_key=...&page=...&limit=...` | List attack source IPs with pagination | | ||
| | `ThreatBook_HFISH.ThreatBook_HFISH/ListAttackDetails` | `POST /api/v1/attack/detail?api_key=...&page=...&limit=...` | List attack details with pagination | | ||
| | `ThreatBook_HFISH.ThreatBook_HFISH/ListAttackAccounts` | `POST /api/v1/attack/account?api_key=...&page=...&limit=...` | List captured credentials from attacks | | ||
| | `ThreatBook_HFISH.ThreatBook_HFISH/GetSystemInfo` | `GET /api/v1/hfish/sys_info?api_key=...` | Get honeypot system status | | ||
|
|
||
| ## Parameter Notes | ||
|
|
||
| - All POST endpoints send an empty `{}` JSON body; authentication and pagination are via query parameters. | ||
| - `page` defaults to 1, `limit` defaults to 20. | ||
| - HFish API response codes: `0` = success, `1003` = authentication failure (maps to `PERMISSION_DENIED`). | ||
|
|
||
| ## Behavior Notes | ||
|
|
||
| - HTTP 401 maps to `UNAUTHENTICATED`, 403 maps to `PERMISSION_DENIED`. | ||
| - Other HTTP 4xx responses map to `FAILED_PRECONDITION`. | ||
| - HTTP 5xx, network, and TLS failures map to `UNAVAILABLE`. | ||
| - Non-JSON success bodies map to `UNKNOWN`. | ||
| - HFish response code `1003` (illegal apikey) maps to `PERMISSION_DENIED`. | ||
| - Other non-zero HFish response codes map to `FAILED_PRECONDITION`. | ||
|
|
||
| ## Local Checks | ||
|
|
||
| ```bash | ||
| cd services | ||
| npm run validate -- --service-dir threatbook__hfish | ||
| npm test -- --service-dir threatbook__hfish --coverage | ||
| npm run pack:check | ||
| ``` | ||
|
|
||
| ## Supported Versions | ||
|
|
||
| - HFish v3.x (tested on v3.3.6) | ||
|
|
||
| ## Risk & CapSet Notes | ||
|
|
||
| - **Read-only operations**: all four methods are read-only queries. | ||
| - **Suggested CapSet**: `threat-intel` with read-only constraints. | ||
| - **No side effects**: these APIs do not modify any HFish configuration or data. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| #!/usr/bin/env node | ||
|
|
||
| import { runServiceMain } from "@chaitin-ai/octobus-sdk"; | ||
|
|
||
| import { service } from "../src/service.js"; | ||
|
|
||
| runServiceMain(service); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| { | ||
| "$schema": "https://json-schema.org/draft/2020-12/schema", | ||
| "type": "object", | ||
| "additionalProperties": true, | ||
| "properties": { | ||
| "endpoint": { | ||
| "type": "string", | ||
| "description": "HFish server base URL, for example https://hfish.example.com:4433." | ||
| }, | ||
| "headers": { | ||
| "type": "object", | ||
| "additionalProperties": { | ||
| "type": "string" | ||
| }, | ||
| "description": "Optional extra HTTP headers sent to HFish." | ||
| }, | ||
| "timeoutMs": { | ||
| "type": "integer", | ||
| "minimum": 1, | ||
| "default": 1500, | ||
| "description": "HTTP timeout in milliseconds." | ||
| }, | ||
| "skipTlsVerify": { | ||
| "type": "boolean", | ||
| "default": false, | ||
| "description": "Skip TLS certificate verification for private HFish deployments." | ||
| } | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| { | ||
| "name": "threatbook-hfish", | ||
| "version": "0.0.0", | ||
| "private": true, | ||
| "type": "module", | ||
| "bin": { | ||
| "threatbook-hfish": "bin/threatbook-hfish.js" | ||
| }, | ||
| "dependencies": { | ||
| "@chaitin-ai/octobus-sdk": "^0.5.0" | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,153 @@ | ||
| syntax = "proto3"; | ||
|
|
||
| package ThreatBook_HFISH; | ||
|
|
||
| import "google/protobuf/wrappers.proto"; | ||
| import "google/protobuf/struct.proto"; | ||
|
|
||
| option go_package = "miner/grpc-service/ThreatBook_HFISH"; | ||
|
|
||
| service ThreatBook_HFISH { | ||
| // 获取攻击来源 IP 列表,支持分页 | ||
| rpc ListAttackIPs(ListAttackIPsRequest) returns (ListAttackIPsResponse) {} | ||
|
|
||
| // 获取攻击详情列表,支持分页 | ||
| rpc ListAttackDetails(ListAttackDetailsRequest) returns (ListAttackDetailsResponse) {} | ||
|
|
||
| // 获取攻击 IP 使用的账号信息列表 | ||
| rpc ListAttackAccounts(ListAttackAccountsRequest) returns (ListAttackAccountsResponse) {} | ||
|
|
||
| // 获取蜜罐系统运行状态信息 | ||
| rpc GetSystemInfo(GetSystemInfoRequest) returns (GetSystemInfoResponse) {} | ||
| } | ||
|
|
||
| // ==================== ListAttackIPs ==================== | ||
|
|
||
| message ListAttackIPsRequest { | ||
| string api_key = 1; // API 访问凭证,必填 | ||
| google.protobuf.Int64Value page = 2; // 页码,可选,默认 1 | ||
| google.protobuf.Int64Value limit = 3; // 每页条数,可选,默认 20 | ||
| } | ||
|
|
||
| message ListAttackIPsResponse { | ||
| int64 response_code = 1; // 响应码,0 表示成功 | ||
| string verbose_msg = 2; // 响应消息 | ||
| repeated AttackIPRecord data = 3; // 攻击 IP 列表 | ||
| } | ||
|
|
||
| message AttackIPRecord { | ||
| string ip = 1; // 攻击者 IP | ||
| int64 attack_count = 2; // 攻击次数 | ||
| string first_attack_time = 3; // 首次攻击时间 | ||
| string last_attack_time = 4; // 最近攻击时间 | ||
| repeated string attack_types = 5; // 攻击类型列表 | ||
| string country = 6; // 国家 | ||
| string province = 7; // 省份 | ||
| string city = 8; // 城市 | ||
| string group = 9; // 所属分组 | ||
| string comment = 10; // 备注 | ||
| int64 attack_chain_count = 11; // 攻击链数量 | ||
| int64 port_count = 12; // 端口数量 | ||
| int64 related_info_count = 13; // 关联信息数量 | ||
| } | ||
|
|
||
| // ==================== ListAttackDetails ==================== | ||
|
|
||
| message ListAttackDetailsRequest { | ||
| string api_key = 1; // API 访问凭证,必填 | ||
| google.protobuf.Int64Value page = 2; // 页码,可选,默认 1 | ||
| google.protobuf.Int64Value limit = 3; // 每页条数,可选,默认 20 | ||
| string ip = 4; // 按 IP 过滤,可选 | ||
| string type = 5; // 按蜜罐类型过滤,可选 | ||
| } | ||
|
|
||
| message ListAttackDetailsResponse { | ||
| int64 response_code = 1; // 响应码,0 表示成功 | ||
| string verbose_msg = 2; // 响应消息 | ||
| AttackDetailData data = 3; // 攻击详情数据 | ||
| } | ||
|
|
||
| message AttackDetailData { | ||
| int64 total_num = 1; // 总记录数 | ||
| int64 page_no = 2; // 当前页码 | ||
| int64 page_size = 3; // 每页大小 | ||
| int64 total_page = 4; // 总页数 | ||
| repeated AttackDetailRecord detail_list = 5; // 攻击详情列表 | ||
| } | ||
|
|
||
| message AttackDetailRecord { | ||
| int64 id = 1; // 记录 ID | ||
| string src_ip = 2; // 源 IP | ||
| string src_port = 3; // 源端口 | ||
| string dest_ip = 4; // 目的 IP | ||
| string dest_port = 5; // 目的端口 | ||
| string protocol = 6; // 协议 | ||
| string type = 7; // 蜜罐类型 | ||
| string app_name = 8; // 蜜罐名称 | ||
| string client_name = 9; // 节点名称 | ||
| string raw_data = 10; // 原始攻击载荷 | ||
| string country = 11; // 国家 | ||
| string province = 12; // 省份 | ||
| string city = 13; // 城市 | ||
| string create_time = 14; // 攻击时间 | ||
| string attack_chain = 15; // 攻击链信息 | ||
| string crawl_info = 16; // 扫描信息 | ||
| string user_info = 17; // 账号信息 | ||
| } | ||
|
|
||
| // ==================== ListAttackAccounts ==================== | ||
|
|
||
| message ListAttackAccountsRequest { | ||
| string api_key = 1; // API 访问凭证,必填 | ||
| google.protobuf.Int64Value page = 2; // 页码,可选,默认 1 | ||
| google.protobuf.Int64Value limit = 3; // 每页条数,可选,默认 20 | ||
| } | ||
|
|
||
| message ListAttackAccountsResponse { | ||
| int64 response_code = 1; // 响应码,0 表示成功 | ||
| string verbose_msg = 2; // 响应消息 | ||
| repeated AttackAccountRecord data = 3; // 账号信息列表 | ||
| } | ||
|
|
||
| message AttackAccountRecord { | ||
| int64 id = 1; // 记录 ID | ||
| string ip = 2; // 攻击者 IP | ||
| string account = 3; // 账号 | ||
| string password = 4; // 密码 | ||
| string type = 5; // 蜜罐类型 | ||
| string create_time = 6; // 捕获时间 | ||
| } | ||
|
|
||
| // ==================== GetSystemInfo ==================== | ||
|
|
||
| message GetSystemInfoRequest { | ||
| string api_key = 1; // API 访问凭证,必填 | ||
| } | ||
|
|
||
| message GetSystemInfoResponse { | ||
| int64 response_code = 1; // 响应码,0 表示成功 | ||
| string verbose_msg = 2; // 响应消息 | ||
| SystemInfoData data = 3; // 系统状态数据 | ||
| } | ||
|
|
||
| message SystemInfoData { | ||
| int64 total_honeypots = 1; // 蜜罐总数 | ||
| int64 total_cardinal_honeypots = 2; // 主要蜜罐数 | ||
| int64 total_online_honeypots = 3; // 在线蜜罐数 | ||
| int64 total_offline_honeypots = 4; // 离线蜜罐数 | ||
| map<string, int64> honeypot_self_cnt = 5; // 各类型蜜罐数量 | ||
| repeated ClientInfo clients = 6; // 节点列表 | ||
| } | ||
|
|
||
| message ClientInfo { | ||
| string name = 1; // 节点名称 | ||
| string ip = 2; // 节点 IP | ||
| int64 create_time = 3; // 创建时间 | ||
| repeated HoneypotInfo honeypots = 4; // 蜜罐列表 | ||
| } | ||
|
|
||
| message HoneypotInfo { | ||
| string type = 1; // 蜜罐类型 | ||
| string name = 2; // 蜜罐名称 | ||
| int64 state = 3; // 状态:2=在线,3=离线 | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| { | ||
| "$schema": "https://json-schema.org/draft/2020-12/schema", | ||
| "type": "object", | ||
| "additionalProperties": true, | ||
| "properties": { | ||
| "apiKey": { | ||
| "type": "string", | ||
| "description": "HFish API key for authentication." | ||
| } | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| { | ||
| "schema": "chaitin.octobus.service.v1", | ||
| "name": "threatbook-hfish", | ||
| "displayName": "ThreatBook HFish Honeypot", | ||
| "description": "OctoBus package for ThreatBook HFish honeypot attack IPs, attack details, attack accounts, and system info APIs.", | ||
| "runtime": { | ||
| "mode": "long-running" | ||
| }, | ||
| "proto": { | ||
| "roots": [ | ||
| "proto" | ||
| ], | ||
| "files": [ | ||
| "proto/hfish.proto" | ||
| ] | ||
| }, | ||
| "configSchema": "config.schema.json", | ||
| "secretSchema": "secret.schema.json", | ||
| "sdk": { | ||
| "cli": { | ||
| "commands": { | ||
| "ThreatBook_HFISH.ThreatBook_HFISH/ListAttackIPs": { | ||
| "name": "list-attack-ips", | ||
| "description": "List HFish attack source IPs with pagination." | ||
| }, | ||
| "ThreatBook_HFISH.ThreatBook_HFISH/ListAttackDetails": { | ||
| "name": "list-attack-details", | ||
| "description": "List HFish attack details with pagination." | ||
| }, | ||
| "ThreatBook_HFISH.ThreatBook_HFISH/ListAttackAccounts": { | ||
| "name": "list-attack-accounts", | ||
| "description": "List HFish attack accounts with pagination." | ||
| }, | ||
| "ThreatBook_HFISH.ThreatBook_HFISH/GetSystemInfo": { | ||
| "name": "get-system-info", | ||
| "description": "Get HFish honeypot system status." | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.