feat: community detection rule library — 9 curated rules embedded in binary#13
Merged
Conversation
…binary Ships three rule files in internal/config/rules.d/ (embedded via go:embed): recon.toml — port_scan_detector, host_sweep_detector, connection_flood c2_malware.toml — tls_fingerprint_flood, rapid_beaconing, slow_beaconing lateral_movement.toml — internal_host_spread, ssh_connection_burst, dst_port_spread All rules use sliding windows where appropriate to catch bursts straddling tumbling boundaries. Rules ship ready-to-use; operators tune thresholds or disable rules by adding a same-named entry to custom_rules.toml (user rules shadow community rules of the same name via MergeRules). Community rules are static (embedded in binary, not reloaded on SIGHUP). Custom rules via custom_rules.toml continue to hot-reload on SIGHUP. Tests: LoadCommunityRules_Valid, NoDuplicateNames, MergeRules_* (5 cases). All 11 packages green. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Crank-Git
added a commit
that referenced
this pull request
Apr 14, 2026
…scription fix Addresses findings from post-merge eng review of PR #13: - Add new_ja4_burst and dga_cert_diversity rules to c2_malware.toml. These two rules use JA4-specific fields (ja4, ja4x) that a NetFlow sensor cannot see — the actual differentiator vs. volumetric-only detection. Both include a note that they only fire on completed TLS handshakes. - Fix ssh_connection_burst description: was "opened more than 20 SSH connections" but the rule uses count_distinct(dst_ip), which measures distinct destination hosts, not total connection count. Corrected to "connected to more than 20 distinct hosts via SSH". - Add TestMergeRules_BothNil: guards against panic if both community and custom rule sets are nil (edge case, not reachable in normal operation). - Add TestCommunityRules_FireThroughEvaluator (integration test): end-to-end verification that LoadCommunityRules → NewCustomRules → Evaluate fires an alert. Specifically triggers port_scan_detector with 21 distinct dst_ports. Guards against silent regression if the wiring in initPipeline breaks. - Add logShadowedRules helper in main.go: logs a warning when a user custom rule shadows a community rule of the same name. Called at startup and on SIGHUP. Previously silent; helps operators debug unexpected detection gaps. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
3 tasks
Crank-Git
added a commit
that referenced
this pull request
May 9, 2026
feat: community detection rule library — 9 curated rules embedded in binary
Crank-Git
added a commit
that referenced
this pull request
May 9, 2026
…scription fix Addresses findings from post-merge eng review of PR #13: - Add new_ja4_burst and dga_cert_diversity rules to c2_malware.toml. These two rules use JA4-specific fields (ja4, ja4x) that a NetFlow sensor cannot see — the actual differentiator vs. volumetric-only detection. Both include a note that they only fire on completed TLS handshakes. - Fix ssh_connection_burst description: was "opened more than 20 SSH connections" but the rule uses count_distinct(dst_ip), which measures distinct destination hosts, not total connection count. Corrected to "connected to more than 20 distinct hosts via SSH". - Add TestMergeRules_BothNil: guards against panic if both community and custom rule sets are nil (edge case, not reachable in normal operation). - Add TestCommunityRules_FireThroughEvaluator (integration test): end-to-end verification that LoadCommunityRules → NewCustomRules → Evaluate fires an alert. Specifically triggers port_scan_detector with 21 distinct dst_ports. Guards against silent regression if the wiring in initPipeline breaks. - Add logShadowedRules helper in main.go: logs a warning when a user custom rule shadows a community rule of the same name. Called at startup and on SIGHUP. Previously silent; helps operators debug unexpected detection gaps. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Crank-Git
added a commit
that referenced
this pull request
May 9, 2026
…scription fix Addresses findings from post-merge eng review of PR #13: - Add new_ja4_burst and dga_cert_diversity rules to c2_malware.toml. These two rules use JA4-specific fields (ja4, ja4x) that a NetFlow sensor cannot see — the actual differentiator vs. volumetric-only detection. Both include a note that they only fire on completed TLS handshakes. - Fix ssh_connection_burst description: was "opened more than 20 SSH connections" but the rule uses count_distinct(dst_ip), which measures distinct destination hosts, not total connection count. Corrected to "connected to more than 20 distinct hosts via SSH". - Add TestMergeRules_BothNil: guards against panic if both community and custom rule sets are nil (edge case, not reachable in normal operation). - Add TestCommunityRules_FireThroughEvaluator (integration test): end-to-end verification that LoadCommunityRules → NewCustomRules → Evaluate fires an alert. Specifically triggers port_scan_detector with 21 distinct dst_ports. Guards against silent regression if the wiring in initPipeline breaks. - Add logShadowedRules helper in main.go: logs a warning when a user custom rule shadows a community rule of the same name. Called at startup and on SIGHUP. Previously silent; helps operators debug unexpected detection gaps. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
go:embed, organized into 3 files:recon.toml—port_scan_detector,host_sweep_detector,connection_floodc2_malware.toml—tls_fingerprint_flood,rapid_beaconing,slow_beaconinglateral_movement.toml—internal_host_spread,ssh_connection_burst,dst_port_spreadconfig.LoadCommunityRules()— loads and validates all embedded TOML files at startupconfig.MergeRules(community, custom)— user rules shadow community rules of the same name, enabling per-rule disable/override without editing the binaryinitPipeline+ SIGHUP handler updated to load community rules and merge before passing to evaluator; community rules stay static across SIGHUP (only custom rules reload)Override model
To disable a community rule, add an entry with the same name to
custom_rules.toml:User rules shadow community rules of the same name — no binary changes needed.
Test plan
TestLoadCommunityRules_Valid— all embedded files parse + validate, at least one rule returnedTestLoadCommunityRules_NoDuplicateNames— no name collisions across rule filesTestMergeRules_UserShadowsCommunity— user rule takes priority when names collideTestMergeRules_EmptyCustom— community-only scenarioTestMergeRules_EmptyCommunity— custom-only scenarioTestMergeRules_OrderPreserved— community rules appear before custom rules🤖 Generated with Claude Code