-
Notifications
You must be signed in to change notification settings - Fork 544
service_section: added parameter for 'storage.max_chunk_size' #2331
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
Parameter 'storage.max_chunk_size' introduced in the 'input_chunk' module to provide user option to configure maximum size for buffer chunks of input modules. Signed-off-by: Castor Sky <csky57@gmail.com>
📝 WalkthroughWalkthroughDocumentation update to Fluent Bit's storage configuration section: reformats the storage configuration table with improved readability and adds a new Changes
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~3 minutes Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In `@administration/configuring-fluent-bit/yaml/service-section.md`:
- Line 38: Remove the invalid table row for `storage.max_chunk_size` (it is not
a Fluent Bit option); either delete that entry from the service-section table or
replace it with the correct setting `storage.max_chunks_up` and document its
meaning (limits the max number of in-memory chunks, default `128`) so the table
reflects valid Fluent Bit configuration options.
| | `storage.sync` | Configures the synchronization mode used to store data in the file system. Using `full` increases the reliability of the filesystem buffer and ensures that data is guaranteed to be synced to the filesystem even if Fluent Bit crashes. On Linux, `full` corresponds with the `MAP_SYNC` option for [memory mapped files](https://man7.org/linux/man-pages/man2/mmap.2.html). Accepted values: `normal`, `full`. | `normal` | | ||
| | `storage.checksum` | Enables data integrity check when writing and reading data from the filesystem. The storage layer uses the CRC32 algorithm. Accepted values: `off` or `on`. | `off` | | ||
| | `storage.max_chunks_up` | Sets the number of chunks that can be `up` in memory for input plugins that use filesystem storage. | `128` | | ||
| | `storage.max_chunk_size` | Sets the maximum size of buffer chunk for input plugins that use filesystem storage. | `2M` | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧩 Analysis chain
🌐 Web query:
Fluent Bit storage.max_chunk_size default value configuration format
💡 Result:
There is no Fluent Bit option named storage.max_chunk_size. Fluent Bit chunks are ~2 MB in size by design, and the relevant service setting to control in-memory chunk counts is storage.max_chunks_up (default 128), configured in the [SERVICE] / service section of the Fluent Bit config (classic fluent-bit.conf or the YAML format introduced in v3.2). [1][2][3]
Sources:
- Buffering & Storage (Fluent Bit docs) — chunk size ≈ 2 MB; storage.max_chunks_up default 128. [1]
- Buffering (Fluent Bit docs) — per-input/service configuration (storage.* settings). [2]
- Configuring Fluent Bit — supported configuration formats: classic and YAML (v3.2+). [3]
References:
[1] https://docs.fluentbit.io/manual/4.0/administration/buffering-and-storage
[2] https://docs.fluentbit.io/manual/administration/buffering-and-storage
[3] https://docs.fluentbit.io/manual/3.2/administration/configuring-fluent-bit
Remove storage.max_chunk_size — this option does not exist in Fluent Bit.
storage.max_chunk_size is not a valid Fluent Bit configuration option. Chunk size is fixed at approximately 2 MB and is not configurable. The documented option appears to conflate chunk size with the actual storage.max_chunks_up setting (default: 128), which limits the maximum number of in-memory chunks, not their size. Replace this table entry with the correct option or remove it entirely if not intended for this service section.
🤖 Prompt for AI Agents
In `@administration/configuring-fluent-bit/yaml/service-section.md` at line 38,
Remove the invalid table row for `storage.max_chunk_size` (it is not a Fluent
Bit option); either delete that entry from the service-section table or replace
it with the correct setting `storage.max_chunks_up` and document its meaning
(limits the max number of in-memory chunks, default `128`) so the table reflects
valid Fluent Bit configuration options.
Parameter 'storage.max_chunk_size' introduced in the
input_chunkmodule to provide the user with an option to configure maximum size for buffer chunks of input modules.PR in the
fluent-bitrepo: fluent/fluent-bit#11373Summary by CodeRabbit
storage.max_chunk_sizeconfiguration option (default: 2M) for managing chunk sizes.✏️ Tip: You can customize this high-level summary in your review settings.