Skip to content

Comerica branch for streaming #46

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
akshaya-kumar-t wants to merge 11 commits into
base: master
Choose a base branch
from
Open

Comerica branch for streaming #46

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
5836beb
Use tuple-type struct for Parquet
janmejay Oct 23, 2025
82d5f65
tmp placeholder commit
janmejay Oct 29, 2025
cd523f4
Chore[OB-7853] - Make decoders Eq to help with assertions in config s…
janmejay Oct 30, 2025
59f05e7
Chore[OB-7853] - Support stream termination and tick in SCol runner-e…
janmejay Nov 5, 2025
ab89f9c
Feat[OB-7853] - Enhance SCol fetcher to support streaming
janmejay Nov 20, 2025
92a8d5d
Feat[OB-7853] - Add support for streams-per SCol fetcher
janmejay Nov 25, 2025
73de49e
Chore[OB-7853] - Merge `master` into SCol streaming
janmejay Nov 25, 2025
4a97469
Fix[OB-7853] - Fix streams-per-fetch enforcement for http-fetch
janmejay Nov 27, 2025
7c26b7e
Fix[OB-7853] - Bump scol (exec stream stdin poll fix)
janmejay Nov 27, 2025
4e066bc
Add support for json_paths as a streaming json decoder.
jsbali Nov 25, 2025
5d4a04b
Add PartialEq, Eq to JsonPathDeserializerConfig
jsbali Nov 25, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 38 additions & 13 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions lib/codecs/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ obvrl = { path = "../observo/obvrl", optional = true }
futures.workspace = true
indexmap.workspace = true
bytesize.workspace = true
actson = { git = "https://github.com/jsbali/actson-rs.git" }

[dev-dependencies]
futures.workspace = true
Expand Down
2 changes: 1 addition & 1 deletion lib/codecs/src/decoding/format/avro.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ impl From<&AvroDeserializerOptions> for AvroSerializerOptions {
}
/// Apache Avro serializer options.
#[configurable_component]
#[derive(Clone, Debug)]
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct AvroDeserializerOptions {
/// The Avro schema definition.
/// Please note that the following [`apache_avro::types::Value`] variants are currently *not* supported:
Expand Down
2 changes: 1 addition & 1 deletion lib/codecs/src/decoding/format/gelf.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ use crate::{gelf_fields::*, VALID_FIELD_REGEX};

/// Config used to build a `GelfDeserializer`.
#[configurable_component]
#[derive(Debug, Clone, Default)]
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct GelfDeserializerConfig {
/// GELF-specific decoding options.
#[serde(default, skip_serializing_if = "vector_core::serde::is_default")]
Expand Down
2 changes: 1 addition & 1 deletion lib/codecs/src/decoding/format/influxdb.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ use super::Deserializer;
/// Config used to build a `InfluxdbDeserializer`.
/// - [InfluxDB Line Protocol](https://docs.influxdata.com/influxdb/v1/write_protocols/line_protocol_tutorial/):
#[configurable_component]
#[derive(Debug, Clone, Default)]
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct InfluxdbDeserializerConfig {
/// Influxdb-specific decoding options.
#[serde(default, skip_serializing_if = "vector_core::serde::is_default")]
Expand Down
2 changes: 1 addition & 1 deletion lib/codecs/src/decoding/format/json.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ use super::{default_lossy, Deserializer};

/// Config used to build a `JsonDeserializer`.
#[configurable_component]
#[derive(Debug, Clone, Default)]
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct JsonDeserializerConfig {
/// JSON-specific decoding options.
#[serde(default, skip_serializing_if = "vector_core::serde::is_default")]
Expand Down
Loading