fix(*): truncate json_content/json_map keys over 31 chars to match engine-stored keys - #704
Merged
Merged
Conversation
…gine-stored keys The legacy engine's JSON_MAX_KEY_SIZE is 32 bytes, so Parse_JSON() clips any dotted key path over 31 chars before storing it. Both JSON_Content() and the json_map lookup (Get_Key_Value, via engine.c) do an exact strcmp against that stored key, so a rule naming the real (untruncated) field never matches -- confirmed live against a real event and against the engine source (parsers/json.c, json-content.c, processors/engine.c). Since the key names come from the log formats themselves (AWS CloudTrail, Entra, Confluent, Okta, Oracle OCI) and can't be shortened, this truncates each affected key to the first 31 chars -- the exact string the engine actually stores -- so the 28 affected rules match again. Each change is commented with the original full key for restoration if the engine's JSON_MAX_KEY_SIZE is ever raised. Two additional json_map rules (cisco-amp.rules, screenconnect.rules) target array elements and are not fixed here -- arrays are never expanded by the parser regardless of key length, so truncation doesn't help them. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
JSON_MAX_KEY_SIZEis 32 bytes, soParse_JSON()(parsers/json.c) clips any dotted JSON key path over 31 chars before storing it. BothJSON_Content()(json-content.c) and thejson_maplookup (Get_Key_Value, called fromprocessors/engine.c) do an exactstrcmpagainst that stored key, so a rule naming the real, untruncated field path never matches -- silently, with no error or log line.JSON_MAX_KEY_SIZEis ever raised.json_maprules (cisco-amp.rules,screenconnect.rules) target array elements and are not fixed here -- arrays are never expanded by the parser regardless of key length, so truncation doesn't help; they need a different rewrite.Rules affected (28)
azureEventHub_entra.rules-- 1 rule (2 keys)azure-eventhub-ad.rules-- 1 ruleaws-cloudtrail.rules-- 1 ruleokta.rules-- 1 ruleconfluent.rules-- 9 rulesddr.rules-- 7 rulesgcp-cloud-audit.rules-- 5 rulesoracle_oci.rules-- 1 ruleTest plan
.properties.deviceDetail.isManagednever fires against a live event containing that field; the same rule matching the truncated (31-char).properties.deviceDetail.isManafires correctly.🤖 Generated with Claude Code