Draft
Conversation
Agent-Logs-Url: https://github.com/benhoyt/goawk/sessions/29bc6e5b-0551-41a0-83cf-9d679344b4d0 Co-authored-by: benhoyt <999033+benhoyt@users.noreply.github.com>
…ing, better error message Agent-Logs-Url: https://github.com/benhoyt/goawk/sessions/29bc6e5b-0551-41a0-83cf-9d679344b4d0 Co-authored-by: benhoyt <999033+benhoyt@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Add JSON Lines support for GoAWK
Add JSON Lines (JSONL) input mode
Apr 12, 2026
Owner
|
@copilot thanks. For nested objects and stats, I'd much rather these were flattened to @"foo.nested" or @"foo.0". Can you please try that? |
Agent-Logs-Url: https://github.com/benhoyt/goawk/sessions/c8ca21c9-3654-4775-b9ab-54cdd0583883 Co-authored-by: benhoyt <999033+benhoyt@users.noreply.github.com>
Author
Done in 9612689. Nested JSON values are now flattened recursively using dot notation:
For example: gives Accessing |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
jsonlas a new input mode, enabling GoAWK to process JSON Lines format where each line is a JSON value. Complements the existing CSV/TSV modes.New mode:
INPUTMODE="jsonl"/-i jsonlJSON arrays → positional fields
$1,$2, ...JSON objects → named fields via
@"key", also accessible as$1,$2, ... in document key order;FIELDSarray updated per recordNested objects and arrays are flattened using dot notation for objects and numeric indexes for arrays:
Accessing a non-scalar key (e.g.
@"four"wherefouris an array) returns"". Positional fields$1,$2, ... and theFIELDSarray reflect all flattened scalar leaves in document order.Type conversions
true"1"false"0"null""Implementation
interp/jsonl.go(new):jsonlSplitter(mirrorscsvSplitter— pre-parses each line before action bodies run, soFIELDSand per-record state are ready),parseJSONLineToFields(),flattenObject()/flattenArray()/flattenJSONValue()for recursive flattening, and JSON→AWK conversion helpersinterp/interp.go:JSONLMode IOMode = 3;getFieldByName()callsensureFields()first in JSONL mode (per-record object keys, unlike CSV's fixed header); updatedparseInputMode()/inputModeString()interp/io.go:newScanner()usesjsonlSplitterfor JSONLMode;ensureFields()re-parses from$0on reassignment;setFieldNames(nil)correctly clears FIELDS for array records$0holds the raw JSON line;$0reassignment re-parses as JSON