Skip to content
This repository was archived by the owner on Dec 7, 2022. It is now read-only.

Add a cli option for producer encoding#86

Open
denisacostaq wants to merge 5 commits intomasterfrom
forward_data_using_a_model_based_encoding
Open

Add a cli option for producer encoding#86
denisacostaq wants to merge 5 commits intomasterfrom
forward_data_using_a_model_based_encoding

Conversation

@denisacostaq
Copy link
Copy Markdown

@denisacostaq denisacostaq commented Jul 29, 2021

Description

What

Be able to configure the encoder for the producer side
Options: model or base64
--producer_encoding Two possible values are accepted: model - if you want to send data encoded in a json mapping the model; or base64 if you want a base64 encoding. (env $PRODUCER_ENCODING) (default "model")

Why

To be able to use kafka-connect to forward data directly from kafka to elasticsearch using:
'value.converter' = 'org.apache.kafka.connect.json.JsonConverter' is required to have the messages encoded in json in the topic, base64 messages does not work.

Anything, in particular, you'd like to highlight to reviewers

This depends on Financial-Times/message-queue-go-producer#10

Would appreciate a second pair of eyes on the test
I am not quite sure how this bit works
Is there a better library for doing x

Scope and particulars of this PR (Please tick all that apply)

  • Tech hygiene (dependency updating & other tech debt)
  • Bug fix
  • Feature
  • Documentation
  • Breaking change
  • Minor change (e.g. fixing a typo, adding config)

This Pull Request follows the rules described in our Pull Requests Guide

Be able to configure the encoder for the producer side

Options: `model` or `base64`
@denisacostaq denisacostaq force-pushed the forward_data_using_a_model_based_encoding branch from 1bb43fa to 81b33b5 Compare July 29, 2021 07:59
@denisacostaq denisacostaq self-assigned this Jul 29, 2021
@denisacostaq denisacostaq force-pushed the forward_data_using_a_model_based_encoding branch 2 times, most recently from 142a546 to d54b1f2 Compare July 29, 2021 08:49
@coveralls
Copy link
Copy Markdown

coveralls commented Jul 29, 2021

Coverage Status

Coverage decreased (-3.5%) to 31.921% when pulling ec02b38 on forward_data_using_a_model_based_encoding into 0ef127a on master.

@denisacostaq denisacostaq force-pushed the forward_data_using_a_model_based_encoding branch from d54b1f2 to 172395c Compare August 5, 2021 09:04
@denisacostaq denisacostaq marked this pull request as ready for review August 5, 2021 09:14
@denisacostaq denisacostaq requested a review from a team as a code owner August 5, 2021 09:14
@denisacostaq denisacostaq force-pushed the forward_data_using_a_model_based_encoding branch from 172395c to ec02b38 Compare August 5, 2021 09:15
@denisacostaq denisacostaq force-pushed the forward_data_using_a_model_based_encoding branch from f4ebe09 to 701dab8 Compare August 5, 2021 14:10
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants