Skip to content

Conversation

@castorsky
Copy link

@castorsky castorsky commented Jan 20, 2026

Parameter 'storage.max_chunk_size' introduced in the input_chunk module to provide the user with an option to configure maximum size for buffer chunks of input modules.

PR in the fluent-bit repo: fluent/fluent-bit#11373

Summary by CodeRabbit

  • Documentation
    • Updated Fluent Bit storage configuration documentation with improved table formatting and clarity.
    • Added new storage.max_chunk_size configuration option (default: 2M) for managing chunk sizes.
    • Reorganized storage configuration section with updated descriptions for better readability.

✏️ Tip: You can customize this high-level summary in your review settings.

Parameter 'storage.max_chunk_size' introduced in the 'input_chunk' module to provide user option to configure maximum size for buffer chunks of input modules.

Signed-off-by: Castor Sky <csky57@gmail.com>
@castorsky castorsky requested review from a team as code owners January 20, 2026 01:05
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Jan 20, 2026

📝 Walkthrough

Walkthrough

Documentation update to Fluent Bit's storage configuration section: reformats the storage configuration table with improved readability and adds a new storage.max_chunk_size configuration option with a default value of 2M, while retaining all existing storage parameters.

Changes

Cohort / File(s) Summary
Documentation: Storage Configuration
administration/configuring-fluent-bit/yaml/service-section.md
Restructures storage configuration table with enhanced formatting; adds new storage.max_chunk_size parameter (default: 2M); updates descriptions and spacing for all storage configuration keys while preserving semantic content

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Poem

🐰 A table once tight, now spreads its ears wide,
With storage settings organized with pride,
A new max_chunk_size joins the fold,
Documentation fresh, yet structured like gold! ✨

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title accurately and concisely describes the primary change: adding documentation for the 'storage.max_chunk_size' parameter to the service section configuration table.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In `@administration/configuring-fluent-bit/yaml/service-section.md`:
- Line 38: Remove the invalid table row for `storage.max_chunk_size` (it is not
a Fluent Bit option); either delete that entry from the service-section table or
replace it with the correct setting `storage.max_chunks_up` and document its
meaning (limits the max number of in-memory chunks, default `128`) so the table
reflects valid Fluent Bit configuration options.

| `storage.sync` | Configures the synchronization mode used to store data in the file system. Using `full` increases the reliability of the filesystem buffer and ensures that data is guaranteed to be synced to the filesystem even if Fluent Bit crashes. On Linux, `full` corresponds with the `MAP_SYNC` option for [memory mapped files](https://man7.org/linux/man-pages/man2/mmap.2.html). Accepted values: `normal`, `full`. | `normal` |
| `storage.checksum` | Enables data integrity check when writing and reading data from the filesystem. The storage layer uses the CRC32 algorithm. Accepted values: `off` or `on`. | `off` |
| `storage.max_chunks_up` | Sets the number of chunks that can be `up` in memory for input plugins that use filesystem storage. | `128` |
| `storage.max_chunk_size` | Sets the maximum size of buffer chunk for input plugins that use filesystem storage. | `2M` |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🌐 Web query:

Fluent Bit storage.max_chunk_size default value configuration format

💡 Result:

There is no Fluent Bit option named storage.max_chunk_size. Fluent Bit chunks are ~2 MB in size by design, and the relevant service setting to control in-memory chunk counts is storage.max_chunks_up (default 128), configured in the [SERVICE] / service section of the Fluent Bit config (classic fluent-bit.conf or the YAML format introduced in v3.2). [1][2][3]

Sources:

  • Buffering & Storage (Fluent Bit docs) — chunk size ≈ 2 MB; storage.max_chunks_up default 128. [1]
  • Buffering (Fluent Bit docs) — per-input/service configuration (storage.* settings). [2]
  • Configuring Fluent Bit — supported configuration formats: classic and YAML (v3.2+). [3]

References:
[1] https://docs.fluentbit.io/manual/4.0/administration/buffering-and-storage
[2] https://docs.fluentbit.io/manual/administration/buffering-and-storage
[3] https://docs.fluentbit.io/manual/3.2/administration/configuring-fluent-bit


Remove storage.max_chunk_size — this option does not exist in Fluent Bit.

storage.max_chunk_size is not a valid Fluent Bit configuration option. Chunk size is fixed at approximately 2 MB and is not configurable. The documented option appears to conflate chunk size with the actual storage.max_chunks_up setting (default: 128), which limits the maximum number of in-memory chunks, not their size. Replace this table entry with the correct option or remove it entirely if not intended for this service section.

🤖 Prompt for AI Agents
In `@administration/configuring-fluent-bit/yaml/service-section.md` at line 38,
Remove the invalid table row for `storage.max_chunk_size` (it is not a Fluent
Bit option); either delete that entry from the service-section table or replace
it with the correct setting `storage.max_chunks_up` and document its meaning
(limits the max number of in-memory chunks, default `128`) so the table reflects
valid Fluent Bit configuration options.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant