Skip to content

2.4.x field type json format bug#308

Open
CharlesTHN wants to merge 2 commits intobrokercap:v2.4.xfrom
CharlesTHN:2.4.x-field-type-json-format-bug
Open

2.4.x field type json format bug#308
CharlesTHN wants to merge 2 commits intobrokercap:v2.4.xfrom
CharlesTHN:2.4.x-field-type-json-format-bug

Conversation

@CharlesTHN
Copy link

修复问题:
运行过程中频发事件解析的空指针错误

parseEvent err:parseEvent err recover err:runtime error: invalid memory address or nil pointer dereference ;lastMapEvent:*mysql.TableMapEvent ;binlogFileName:mysql-bin.004480 ;binlogPosition:497749251
2025/10/20 11:14:03 goroutine 2909 [running]:
runtime/debug.Stack()
/Users/xiezuofu/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.8.darwin-arm64/src/runtime/debug/stack.go:26 +0x5e
github.com/brokercap/Bifrost/Bristol/mysql.(*mysqlConn).DumpBinlog0.func1.1()
/Users/xiezuofu/go/src/github.com/brokercap/Bifrost/Bristol/mysql/conn_dump.go:132 +0x149
panic({0x10f7a00?, 0x1c7adc0?})
/Users/xiezuofu/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.8.darwin-arm64/src/runtime/panic.go:791 +0x132
encoding/binary.Read({0x141c160, 0xc00064f230}, {0x14283b0, 0x1cc5e20}, {0x0, 0x0})
/Users/xiezuofu/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.8.darwin-arm64/src/encoding/binary/binary.go:264 +0x23f
github.com/brokercap/Bifrost/Bristol/mysql.read_offset_or_inline(0x141c160?, 0x30?)
/Users/xiezuofu/go/src/github.com/brokercap/Bifrost/Bristol/mysql/field_type_json_format.go:273 +0x149
github.com/brokercap/Bifrost/Bristol/mysql.read_binary_json_object(0xc00064f230, 0x1454a, 0x1)
/Users/xiezuofu/go/src/github.com/brokercap/Bifrost/Bristol/mysql/field_type_json_format.go:222 +0x395
github.com/brokercap/Bifrost/Bristol/mysql.get_field_json_data0(0x10?, 0xa0?, 0xc00084d560?)
/Users/xiezuofu/go/src/github.com/brokercap/Bifrost/Bristol/mysql/field_type_json_format.go:49 +0x46
github.com/brokercap/Bifrost/Bristol/mysql.get_field_json_data({0xc002cc60b1, 0x1454b, 0x289c1}, 0x1454b)
/Users/xiezuofu/go/src/github.com/brokercap/Bifrost/Bristol/mysql/field_type_json_format.go:39 +0xa5
github.com/brokercap/Bifrost/Bristol/mysql.(*eventParser).parseEventRow(0x10e85e0?, 0xc00064f1d0, 0xc0015d2210, {0xc004ea8780, 0xd, 0xc0015d4c60?})
/Users/xiezuofu/go/src/github.com/brokercap/Bifrost/Bristol/mysql/event_row.go:588 +0x3205
github.com/brokercap/Bifrost/Bristol/mysql.(*eventParser).parseRowsEvent(0xc00089e120, 0xc00064f1d0)
/Users/xiezuofu/go/src/github.com/brokercap/Bifrost/Bristol/mysql/event_row.go:62 +0x352
github.com/brokercap/Bifrost/Bristol/mysql.(*eventParser).parseEvent(0xc00089e120, {0xc002cc6001, 0x28a71, 0x28a71})
/Users/xiezuofu/go/src/github.com/brokercap/Bifrost/Bristol/mysql/parser.go:256 +0xa39
github.com/brokercap/Bifrost/Bristol/mysql.(*mysqlConn).DumpBinlog0.func1(0xc0004c1e20, 0xc0015d22c0?, 0xc0004c1e18, {0xc002cc6000?, 0x1?, 0x28?})
/Users/xiezuofu/go/src/github.com/brokercap/Bifrost/Bristol/mysql/conn_dump.go:135 +0x85
github.com/brokercap/Bifrost/Bristol/mysql.(*mysqlConn).DumpBinlog0(0xc00223ae70, 0xc00089e120, 0xc000138390)
/Users/xiezuofu/go/src/github.com/brokercap/Bifrost/Bristol/mysql/conn_dump.go:136 +0x205
github.com/brokercap/Bifrost/Bristol/mysql.(*mysqlConn).DumpBinlogMySQLGtid(0xc00223ae70, 0xc00089e120, 0xc000138390)
/Users/xiezuofu/go/src/github.com/brokercap/Bifrost/Bristol/mysql/conn_dump.go:66 +0x165
github.com/brokercap/Bifrost/Bristol/mysql.(*mysqlConn).DumpBinlogGtid(0xc00223ae70, 0xc00089e120, 0xc000138390)
/Users/xiezuofu/go/src/github.com/brokercap/Bifrost/Bristol/mysql/conn_dump.go:52 +0x105
github.com/brokercap/Bifrost/Bristol/mysql.(*BinlogDump).startConnAndDumpBinlog.func2()
/Users/xiezuofu/go/src/github.com/brokercap/Bifrost/Bristol/mysql/binlog.go:267 +0x58
created by github.com/brokercap/Bifrost/Bristol/mysql.(*BinlogDump).startConnAndDumpBinlog in goroutine 73
/Users/xiezuofu/go/src/github.com/brokercap/Bifrost/Bristol/mysql/binlog.go:261 +0x43f

修复项:
1.修复了数组类型判断错误(之前数组分支错误地检查了 JSONB_TYPE_LARGE_OBJECT,已改为检查 JSONB_TYPE_LARGE_ARRAY)。
2.修复了在 large object 分支中 key offsets 的索引错误(把原来错误的 key_offset_lengths[0] = ... 改为 key_offset_lengths[i] = ...)。
3.修改了打包语句,Go 1.13+ 的 -trimpath 参数不需要指定路径值,它会自动移除所有本地路径信息。

单元测试已过,运行 一周没有再复现此异常

tianhaonan added 2 commits October 20, 2025 14:28
1.修复了数组类型判断错误(之前数组分支错误地检查了 JSONB_TYPE_LARGE_OBJECT,已改为检查 JSONB_TYPE_LARGE_ARRAY)。
2.修复了在 large object 分支中 key offsets 的索引错误(把原来错误的 key_offset_lengths[0] = ... 改为 key_offset_lengths[i] = ...)。
Go 1.13+ 的 -trimpath 参数不需要指定路径值,它会自动移除所有本地路径信息。
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant