Skip to content

CockroachDB connector accepts envelope values it cannot parse (wrapped, bare) #2000

@viragtripathi

Description

@viragtripathi

cockroachdb.changefeed.envelope validated against enriched, wrapped, and bare, but the connector's event parser only handles the enriched envelope. Setting either other value passed config validation and created the changefeed, then failed or silently misbehaved at runtime:

  • bare emits row data at the top level with metadata under __crdb__, with no after/before wrapper, so the parser cannot extract change data at all.
  • wrapped has after/before but no op and no ts_ns. Operation type degrades to before/after guessing, and because the deduplication key is tableId:op:ts_ns, the missing ts_ns collapses every same-operation change on a table to a single key, so events after the first are silently deduplicated away.

Only enriched carries the op and ts_ns fields the connector relies on.

Fix

Because enriched is the only valid value, the option could only ever be set wrong. Rather than keep the option and validate it back to a single value, remove it:

  • Remove the cockroachdb.changefeed.envelope property. The connector always creates changefeeds with envelope='enriched' internally.
  • Guard the existing-changefeed reuse path: if a running changefeed matches the connector's topic prefix and tables but was not created with envelope='enriched', fail to start with a clear message instead of consuming a feed the connector cannot parse.
  • Update the property documentation and examples.

Existing configurations that still set cockroachdb.changefeed.envelope=enriched continue to work: Kafka Connect ignores the now-unknown property with a warning.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    Status
    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions