From ed60278dfe825f41df13ba29e783f29b22c923ea Mon Sep 17 00:00:00 2001 From: Andy Stark Date: Fri, 10 Jul 2026 15:26:38 +0100 Subject: [PATCH 1/8] DOC-6832 Add go-redis automatic pipelining page Document the experimental AutoPipeline() and AsyncAutoPipeline() APIs (blocking and async faces), AutoPipelineConfig options, cluster slot sharding, and caveats, based on go-redis PR #3867. Cross-link from the Pipelines/transactions page. Written against an unmerged draft PR, so two things are deliberately provisional: the version note reads "v9.XX.0" as a placeholder because no release ships the API yet, and the examples are static go blocks rather than tested clients-example doctests because the TCE example set cannot exist until a release lands. Both resolve at pickup, not now. The config field list and defaults are also draft-derived and may move. Learned: page docs an unreleased draft API; version and examples are placeholders to resolve at pickup Constraint: the "v9.XX.0" version note is a placeholder; do not invent a real version until the release that ships #3867 is known Directive: convert the static go code blocks to {{< clients-example >}} doctests once go-redis ships the API and an example set exists Recheck: when go-redis PR #3867 merges and a release ships the AutoPipeline API Ticket: DOC-6832 Co-Authored-By: Claude Opus 4.8 (1M context) --- content/develop/clients/go/autopipeline.md | 169 +++++++++++++++++++++ content/develop/clients/go/transpipe.md | 4 + 2 files changed, 173 insertions(+) create mode 100644 content/develop/clients/go/autopipeline.md diff --git a/content/develop/clients/go/autopipeline.md b/content/develop/clients/go/autopipeline.md new file mode 100644 index 0000000000..c4011639cf --- /dev/null +++ b/content/develop/clients/go/autopipeline.md @@ -0,0 +1,169 @@ +--- +bannerText: Automatic pipelining is an experimental feature that is not yet released and may be subject to change. +categories: +- docs +- develop +- stack +- oss +- rs +- rc +- oss +- kubernetes +- clients +description: Batch concurrent go-redis commands into pipelines automatically for high-throughput workloads. +linkTitle: Automatic pipelining +title: Automatic pipelining +weight: 42 +--- + +{{< note >}} +Automatic pipelining is an **experimental** feature and its API may still +change. It requires `github.com/redis/go-redis/v9` v9.XX.0 or later. +{{< /note >}} +  + +[Pipelining]({{< relref "/develop/using-commands/pipelining" >}}) sends a batch +of commands to the server in a single communication, which avoids the network +and processing overhead of sending each command separately. Normally you build +a pipeline by hand (see [Pipelines and transactions]({{< relref "/develop/clients/go/transpipe" >}})), +but this means you must know in advance which commands you want to batch. + +*Automatic pipelining* removes that requirement. When many goroutines issue +commands concurrently, `go-redis` coalesces them into deep pipelines for you, +without any pipeline code in your application. Reach for it in high-throughput, +high-concurrency, or scale scenarios. At low concurrency, a plain client is +simpler and just as fast, and a hand-written pipeline is still fastest when you +can batch by hand. + +## The two faces + +Automatic pipelining has two forms that share the same underlying engine: + +- **Blocking** (`AutoPipeline()`) is a drop-in replacement for a normal + client. Each command call blocks until it executes and returns its own + value and error, exactly like a plain client, so existing code keeps + working unchanged. Under concurrency, the engine batches commands from all + goroutines into back-to-back pipelines behind the scenes. Per-goroutine + ordering is preserved. +- **Async** (`AsyncAutoPipeline()`) is deferred and offers the highest + throughput. Command calls return immediately; reading a result with + `Val()`, `Result()`, or `Err()` blocks until the batch executes. Submit a + window of commands and then drain the results to keep each pipeline as deep + as possible. + +Both methods are available on `Client` and `ClusterClient`. + +## Blocking usage + +Call `AutoPipeline()` to get an `AutoPipeliner`, then call command methods on it +just as you would on a normal client. Each call blocks until it executes, but +concurrent callers' commands are batched together automatically: + +```go +rdb := redis.NewClient(&redis.Options{Addr: "localhost:6379"}) +defer rdb.Close() +ctx := context.Background() + +// Blocking face: a drop-in for a normal client, batched under the hood. +ap, err := rdb.AutoPipeline(nil) +if err != nil { // only returned for an invalid AutoPipelineConfig + log.Fatal(err) +} +defer ap.Close() + +var wg sync.WaitGroup +for i := 0; i < 1000; i++ { + wg.Add(1) + go func(i int) { + defer wg.Done() + key := fmt.Sprintf("key:%d", i) + if err := ap.Set(ctx, key, i, 0).Err(); err != nil { // blocks until executed + log.Printf("set %s: %v", key, err) + } + }(i) +} +wg.Wait() +``` + +## Windowed usage + +For maximum throughput, use the async face. Command calls return immediately, so +you can submit a window of commands and read their results afterwards: + +```go +ctx := context.Background() + +ap, err := rdb.AsyncAutoPipeline(nil) // ordered by default +if err != nil { + log.Fatal(err) +} +defer ap.Close() + +cmds := make([]*redis.StatusCmd, 0, 200) +for i := 0; i < 200; i++ { + // Returns immediately without executing. + cmds = append(cmds, ap.Set(ctx, fmt.Sprintf("key:%d", i), i, 0)) +} +for _, cmd := range cmds { + if err := cmd.Err(); err != nil { // blocks until the batch executes + log.Printf("set: %v", err) + } +} +``` + +## Configuration + +Both `AutoPipeline()` and `AsyncAutoPipeline()` take an optional +`*AutoPipelineConfig` and return `(*AutoPipeliner, error)`. Pass `nil` to use +the defaults. The error is non-nil only when the configuration is invalid (for +example, setting `MaxConcurrentBatches` greater than 1 without also setting +`Unordered`); an invalid config is never a panic. + +```go +ap, err := rdb.AsyncAutoPipeline(&redis.AutoPipelineConfig{ + MaxConcurrentBatches: 80, + Unordered: true, +}) +``` + +The main configuration options are: + +| Field | Description | +| :---- | :---------- | +| `MaxBatchSize` | Maximum number of commands the engine coalesces into a single pipeline before flushing. | +| `MaxFlushDelay` | Maximum time the engine waits to accumulate more commands before flushing a batch. Larger values build deeper pipelines at the cost of latency. | +| `MaxConcurrentBatches` | Number of batches that may be in flight at once. Values greater than 1 require `Unordered` because concurrent batches do not preserve a single ordered stream. | +| `NumShards` | Number of independent queue-and-flusher shards. The default funnels every caller into one queue so batches stay deep. | +| `PipelinePoolSize` | Number of pooled pipeline connections shared across batches. Because batches share these connections, automatic pipelining needs far fewer connections than a plain client at the same concurrency. | +| `Unordered` | Allows commands to execute without preserving a single ordered stream, which enables higher concurrency. | +| `MaxRetries` | Number of times a whole batch is retried if its connection drops. | + +The `AutoPipeliner` instance is cached and shared per client: the first call's +configuration wins, and `Close()` stops the instance for all callers. + +## Cluster usage + +`AutoPipeline()` and `AsyncAutoPipeline()` also work on `ClusterClient`. +Commands are routed to the correct shard by key, so the client installs +slot-based shard routing to keep each shard's batch on a single master node +(rather than splitting every batch across all nodes at flush time). A single +batch may span many slots. Ordering is per key: same-key commands stay in +order, while sub-pipelines on different nodes run concurrently. + +## Caveats and limitations + +- A command's context is not honored once it is queued, because batches + execute on the autopipeliner's own context. Use a plain client if you need + per-command deadlines. +- Blocking commands such as [`BLPOP`]({{< relref "/commands/blpop" >}}) and + [`WAIT`]({{< relref "/commands/wait" >}}) are never batched and run directly + on your context. +- The generic `Do` method bypasses batching and behaves like `Client.Do`. + Prefer the typed methods (`ap.Set()`, `ap.Get()`, and so on). +- On a dropped connection, a batch is retried as a whole (up to + `MaxRetries`), so non-idempotent commands may execute twice. + +## More information + +See the [`go-redis`](https://github.com/redis/go-redis) repository for the +`example/autopipeline` usage tour and further API details. diff --git a/content/develop/clients/go/transpipe.md b/content/develop/clients/go/transpipe.md index 1155004a83..4cefc3c299 100644 --- a/content/develop/clients/go/transpipe.md +++ b/content/develop/clients/go/transpipe.md @@ -28,6 +28,10 @@ There are two types of batch that you can use: See the [Transactions]({{< relref "develop/using-commands/transactions" >}}) page for more information. +If you want the client to batch concurrent commands into pipelines for you +without writing any pipeline code, see +[Automatic pipelining]({{< relref "/develop/clients/go/autopipeline" >}}). + ## Execute a pipeline To execute commands in a pipeline, you first create a pipeline object From 6051201db72fdcc6b55dae8d9130bb67794a5d31 Mon Sep 17 00:00:00 2001 From: Andy Stark Date: Tue, 4 Aug 2026 10:31:50 +0100 Subject: [PATCH 2/8] DOC-6832 Reconcile autopipelining page with released go-redis v9.22.0 go-redis PR #3942 (the master-bound umbrella that superseded #3867) merged on 2026-08-03 and shipped in v9.22.0 the same day, so the park trigger is met. Reconcile the page against the released source rather than the draft it was written from: - Version note, v9.XX.0 placeholder becomes v9.22.0. - AutoPipelineConfig becomes AutoPipelineOptions; AutoPipeline(nil) becomes a no-arg AutoPipeline() plus AutoPipelineWithOptions(), same for the async face. Configuration also shown via Options.AutoPipelineOptions. - Config fields, dropping MaxRetries (gone from the type; batch retries are governed by the client's own MaxRetries) and PipelinePoolSize (a client option, and only live when a pipeline buffer size is set), adding MaxBatchBytes and AdaptiveDelay, and recording the real defaults (MaxBatchSize 200/300, MaxConcurrentBatches 1, NumShards 0, MaxFlushDelay 0). - Widened scope, since Ring exposes all four methods and failover clients get them via *Client. - Cluster, noting the several-slot-routed-shards default and that multi-node commands are rejected from batches. - Caveats, since Do/DoRaw/DoRawWriteTo run outside the pipeline, and retries are capped by the client MaxRetries and disabled with -1. - bannerText, still experimental at release, so the banner stays but "not yet released" goes. Predicted-versus-actual, closing the park loop. The prose survived almost whole; every identifier in it moved. The page's architecture (two faces, blocking versus windowed, cluster slot sharding, the four caveats) needed no structural change, and the cluster ordering and context/blocking/Do caveats all read correctly against the released code. What churned was exclusively naming and field-level detail, and the park snapshot's two flagged placeholders were the cheapest items to fix. The one thing the snapshot got wrong in substance rather than in naming was PipelinePoolSize: it was recorded as "moved to the client Options", implying it still belonged on the page, but autopipeline.go never references the pipeline pool at all and the pool is only created when a pipeline buffer size is set, so the honest fix was to demote it to a tuning aside rather than relocate it as an autopipeline knob. Also worth recording, the trigger fired within 44 minutes of the merge (merged 16:59Z, v9.22.0 published 17:43Z), so "merged" and "released" were effectively the same event here; the tag still had to be checked separately, because the preceding tag was a beta the version note must not cite. The static go blocks stay static: go-redis has no autopipeline doctest upstream, so no TCE example set can exist yet. Learned: park snapshot got the page's shape and caveats right and every identifier wrong; incremental re-verification during the park kept the final reconcile cheap Constraint: batch retries are governed by the client's MaxRetries, not an autopipeline option; AutoPipelineOptions has no retry field Rejected: relocating PipelinePoolSize as an autopipeline config knob | autopipeline.go never touches the pipeline pool, which is only created when a pipeline buffer size is set Directive: keep the code blocks static go until go-redis ships an autopipeline doctest; a TCE example set cannot be authored from the docs side alone Gaps: the examples are not run-verified against a live server, and the throughput figures in the release notes were not reproduced Recheck: if automatic pipelining leaves experimental status, drop the bannerText and the note Ticket: DOC-6832 Co-Authored-By: Claude Opus 5 (1M context) --- content/develop/clients/go/autopipeline.md | 86 +++++++++++++++------- 1 file changed, 58 insertions(+), 28 deletions(-) diff --git a/content/develop/clients/go/autopipeline.md b/content/develop/clients/go/autopipeline.md index c4011639cf..8b8f0b6651 100644 --- a/content/develop/clients/go/autopipeline.md +++ b/content/develop/clients/go/autopipeline.md @@ -1,5 +1,5 @@ --- -bannerText: Automatic pipelining is an experimental feature that is not yet released and may be subject to change. +bannerText: Automatic pipelining is an experimental feature and may be subject to change. categories: - docs - develop @@ -18,7 +18,7 @@ weight: 42 {{< note >}} Automatic pipelining is an **experimental** feature and its API may still -change. It requires `github.com/redis/go-redis/v9` v9.XX.0 or later. +change. It requires `github.com/redis/go-redis/v9` v9.22.0 or later. {{< /note >}}   @@ -51,7 +51,8 @@ Automatic pipelining has two forms that share the same underlying engine: window of commands and then drain the results to keep each pipeline as deep as possible. -Both methods are available on `Client` and `ClusterClient`. +Both methods are available on `Client`, `ClusterClient`, and `Ring`. A failover +client created with `NewFailoverClient()` is a `*Client`, so it has them too. ## Blocking usage @@ -65,8 +66,8 @@ defer rdb.Close() ctx := context.Background() // Blocking face: a drop-in for a normal client, batched under the hood. -ap, err := rdb.AutoPipeline(nil) -if err != nil { // only returned for an invalid AutoPipelineConfig +ap, err := rdb.AutoPipeline() +if err != nil { // only returned for invalid AutoPipelineOptions log.Fatal(err) } defer ap.Close() @@ -93,7 +94,7 @@ you can submit a window of commands and read their results afterwards: ```go ctx := context.Background() -ap, err := rdb.AsyncAutoPipeline(nil) // ordered by default +ap, err := rdb.AsyncAutoPipeline() // ordered by default if err != nil { log.Fatal(err) } @@ -113,42 +114,67 @@ for _, cmd := range cmds { ## Configuration -Both `AutoPipeline()` and `AsyncAutoPipeline()` take an optional -`*AutoPipelineConfig` and return `(*AutoPipeliner, error)`. Pass `nil` to use -the defaults. The error is non-nil only when the configuration is invalid (for -example, setting `MaxConcurrentBatches` greater than 1 without also setting -`Unordered`); an invalid config is never a panic. +`AutoPipeline()` and `AsyncAutoPipeline()` take no arguments. They use the +`AutoPipelineOptions` set on the client's options, if any, and otherwise the +built-in default for their face. To pass options for a single autopipeliner, +use `AutoPipelineWithOptions()` or `AsyncAutoPipelineWithOptions()` instead: ```go -ap, err := rdb.AsyncAutoPipeline(&redis.AutoPipelineConfig{ +// On the client, shared by both faces. +rdb := redis.NewClient(&redis.Options{ + Addr: "localhost:6379", + AutoPipelineOptions: &redis.AutoPipelineOptions{MaxFlushDelay: 100 * time.Microsecond}, +}) + +// Or for a single autopipeliner. +ap, err := rdb.AsyncAutoPipelineWithOptions(&redis.AutoPipelineOptions{ MaxConcurrentBatches: 80, Unordered: true, }) ``` -The main configuration options are: +All four methods return `(*AutoPipeliner, error)`. The error is non-nil only +when the options are invalid (for example, setting `MaxConcurrentBatches` +greater than 1 without also setting `Unordered`); invalid options are never a +panic, and no instance is cached. + +The configuration options are: | Field | Description | | :---- | :---------- | -| `MaxBatchSize` | Maximum number of commands the engine coalesces into a single pipeline before flushing. | -| `MaxFlushDelay` | Maximum time the engine waits to accumulate more commands before flushing a batch. Larger values build deeper pipelines at the cost of latency. | -| `MaxConcurrentBatches` | Number of batches that may be in flight at once. Values greater than 1 require `Unordered` because concurrent batches do not preserve a single ordered stream. | -| `NumShards` | Number of independent queue-and-flusher shards. The default funnels every caller into one queue so batches stay deep. | -| `PipelinePoolSize` | Number of pooled pipeline connections shared across batches. Because batches share these connections, automatic pipelining needs far fewer connections than a plain client at the same concurrency. | +| `MaxBatchSize` | Target number of commands the engine coalesces into a single pipeline before flushing. This is a soft threshold rather than a hard cap, so a busy queue can flush a larger batch. Defaults to 200, or to 300 for the blocking face's built-in default. | +| `MaxBatchBytes` | Approximate limit on the argument bytes in a batch, so that large values flush as several bounded writes instead of one very large one. Also a soft threshold. Defaults to 0, meaning no byte limit. | +| `MaxFlushDelay` | Maximum time the engine waits to accumulate more commands before flushing a batch. Larger values build deeper pipelines at the cost of latency. Defaults to 0, which adds no accumulation wait. | +| `AdaptiveDelay` | Scales `MaxFlushDelay` down as the queue fills, so a busy queue flushes sooner. Requires `MaxFlushDelay` greater than 0. Defaults to `false`. | +| `MaxConcurrentBatches` | Number of batches that may execute at once. Defaults to 1, which gives a single ordered stream. Values greater than 1 require `Unordered` because concurrent batches do not preserve a single ordered stream. | | `Unordered` | Allows commands to execute without preserving a single ordered stream, which enables higher concurrency. | -| `MaxRetries` | Number of times a whole batch is retried if its connection drops. | +| `NumShards` | Number of independent queue-and-flusher shards. Defaults to 0, meaning a single shard, which funnels every caller into one queue so batches stay deep. Cluster clients default to several slot-routed shards instead. On the async face, more than one shard requires `Unordered`. | -The `AutoPipeliner` instance is cached and shared per client: the first call's -configuration wins, and `Close()` stops the instance for all callers. +Connection and buffer tuning is not part of `AutoPipelineOptions`. Batches use +the client's pipeline connections, which you size with the +`PipelineReadBufferSize`, `PipelineWriteBufferSize`, and `PipelinePoolSize` +fields of the client's options. + +Each face has its own cached instance, shared by every caller of that method on +the client: the first call's options win and later calls return the same +instance. `Close()` stops that instance for all callers, and a later call to the +same method then builds a fresh one. Closing the client stops it permanently, +after which the methods return `ErrClosed`. ## Cluster usage `AutoPipeline()` and `AsyncAutoPipeline()` also work on `ClusterClient`. Commands are routed to the correct shard by key, so the client installs slot-based shard routing to keep each shard's batch on a single master node -(rather than splitting every batch across all nodes at flush time). A single -batch may span many slots. Ordering is per key: same-key commands stay in -order, while sub-pipelines on different nodes run concurrently. +(rather than splitting every batch across all nodes at flush time). This is why +cluster clients default to several shards instead of one. A single batch may +span many slots. Ordering is per key: same-key commands stay in order, while +sub-pipelines on different nodes run concurrently. + +Commands that must reach every node or shard, such as +[`FLUSHALL`]({{< relref "/commands/flushall" >}}), cannot ride a pipeline, so +the cluster client rejects them with an error rather than let them spoil a +batch shared with other callers. Run them on the plain client instead. ## Caveats and limitations @@ -158,10 +184,14 @@ order, while sub-pipelines on different nodes run concurrently. - Blocking commands such as [`BLPOP`]({{< relref "/commands/blpop" >}}) and [`WAIT`]({{< relref "/commands/wait" >}}) are never batched and run directly on your context. -- The generic `Do` method bypasses batching and behaves like `Client.Do`. - Prefer the typed methods (`ap.Set()`, `ap.Get()`, and so on). -- On a dropped connection, a batch is retried as a whole (up to - `MaxRetries`), so non-idempotent commands may execute twice. +- The generic `Do`, `DoRaw`, and `DoRawWriteTo` methods run outside the + pipeline, on a normal connection, because an arbitrary command name can + carry connection state or block the connection. Prefer the typed methods + (`ap.Set()`, `ap.Get()`, and so on), which are always batched. +- On a dropped connection, a batch is retried as a whole, up to the client's + `MaxRetries`, so non-idempotent commands may execute twice. Set + `MaxRetries: -1`, or use a plain client, for commands that must never be + retransmitted. ## More information From 6ca3d05bb9b86fb3c0d7e7dfe24e60987c766517 Mon Sep 17 00:00:00 2001 From: Andy Stark Date: Tue, 4 Aug 2026 10:38:47 +0100 Subject: [PATCH 3/8] DOC-6832 Record the park-snapshot calibration lesson in /park Promote the cross-cutting lesson from unparking this PR into the skill that can act on it, rather than leaving it in a commit nobody re-reads. Three additions, each from something that actually bit on DOC-6832. Step 1 now says a trigger must name the source that will really ship the feature (the draft #3867 was one of a series that landed as the consolidated #3942, so a trigger watching the draft would have watched the wrong PR) and must resolve "released" as an ancestry check rather than by dates (merge and the v9.22.0 release were 44 minutes apart, and the preceding tag was a beta the version note must not cite). Step 3 now asks for the observed shape to be split into semantics and identifiers, because they decayed at completely different rates here, plus the two traps that cost the most on the reconcile: a mis-attributed field is indistinguishable from a renamed one unless you record where you saw it, and the release notes under-reported which client types expose the API where the source was authoritative. Limits records the measured outcome so a future author reads a moved identifier as normal rather than as evidence the page was written too early. Learned: the identifier/semantics split is the load-bearing distinction in a park snapshot; recording where an identifier was seen is what separates a rename from a mis-attribution Directive: keep the manifest's required-section format in _shared/park-manifest.md; if the identifier-list structure needs to become a hard requirement, add it there and point at it from here Ticket: DOC-6832 Co-Authored-By: Claude Opus 5 (1M context) --- .claude/skills/park/SKILL.md | 40 ++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/.claude/skills/park/SKILL.md b/.claude/skills/park/SKILL.md index 2b23b42ece..7e1030ac97 100644 --- a/.claude/skills/park/SKILL.md +++ b/.claude/skills/park/SKILL.md @@ -46,6 +46,19 @@ Park only when **the docs are as complete as the source allows** and the sole bl external event. If the work is merely unfinished, keep working. If the source may never ship, don't park — say so. There must be a concrete, testable **trigger condition**. +Two things make a trigger actually testable, and both have bitten: + +- **Point it at the source that will ship the feature, not the one you read.** A draft PR is + often one step in a series that lands on the default branch as a different, consolidated PR. + Watch the branch the release is cut from; if the PR you wrote against targets an integration + branch rather than `master`/`main`, say so in the trigger and name the umbrella once it + exists. +- **Require "in a released version", and make it mean an ancestry check.** Merge and release can + be minutes apart, so date order proves nothing, and the newest tag is often a beta the page's + version note must not cite. The trigger is met when the merge commit is an **ancestor of a + non-prerelease tag** — record the command that shows it (`gh api + repos///compare/...`, expecting `behind_by: 0`). + ## Step 2 — Harvest the loose ends already on the branch The branch's own commits hold the re-check items, from `/reflect`: @@ -84,6 +97,24 @@ Record URL + snapshot in the sources table, and the re-fetch command for unpark. against an unmerged diff — and say *why* it's low, e.g. "signatures differed between two reads of the diff"). +**Split that summary in two: the semantics, and the identifiers.** They decay at completely +different rates. What the feature *does* — the shape of the page, the ordering guarantees, the +caveats — usually survives to release intact. The **names** usually don't: types get renamed, +optional-arg methods split into no-arg plus `WithOptions` variants, fields appear, and fields +leave the type they were on. So enumerate every type, method, field and default the page +commits to as an explicit list unpark can tick off one by one, rather than describing them in +prose. A prose sentence covering four field names is one checklist item that can be half-right; +four listed names are four verdicts. + +Two traps worth calling out while you write that list: + +- **Distinguish "renamed" from "was never there".** Record *where* you saw each identifier + (which file, which struct), because the useful unpark finding is sometimes "this field isn't + part of this API at all" — a mis-attributed field looks exactly like a renamed one in the + snapshot, and only the location tells them apart. +- **Note where the changelog and the source disagree.** Release notes summarize and under-report; + the source is authoritative for which client types expose the API. + ## Step 4 — Compose the manifest & PR body Build the PR body: a short human summary, a prominent **do-not-merge warning** linking the @@ -119,6 +150,15 @@ outward-facing — if the branch isn't pushed or the user hasn't asked, confirm - **A stale snapshot is worse than none** — if you record a head SHA or API shape you didn't actually verify, unpark diffs against fiction. Snapshot only what you checked; leave the rest out and flag it in the checklist. +- **Expect the identifiers to be wrong, and don't let that shake your confidence in the page.** + Measured on DOC-6832 (go-redis automatic pipelining, parked ~3.5 weeks across two re-parks): + at release the page's structure, ordering semantics and every caveat still read correctly, + while nearly every type, method and field name had moved. That is the *normal* outcome, not a + sign the page was written too early — which is why the LOW confidence tag belongs on the + identifier list specifically, not smeared over the whole page. +- **Re-verifying during the park is what makes the final reconcile cheap.** Each re-park that + refreshes the snapshot converts churn into an already-answered checklist item; skip them and + every delta arrives at once, at the moment you least want a surprise. - Park **cannot judge whether the source will ship.** It records a trigger; it doesn't predict the future. A parked PR that never triggers is dead weight — `/unpark`'s scan mode is how you find and close those. From 5cd5fdd23705cd7cacfa2c879c284e09223bf2b8 Mon Sep 17 00:00:00 2001 From: Andy Stark Date: Tue, 4 Aug 2026 11:19:52 +0100 Subject: [PATCH 4/8] DOC-6832 Address review on the automatic pipelining page Three review points from Andy. Drop the upstream "face" terminology. go-redis' own source and release notes call the blocking and deferred APIs "the two faces", and writing the page from that source pulled the word in throughout, but it is that project's internal jargon rather than Go vocabulary a reader would recognize. The page now talks about two methods, with the section retitled "Blocking and asynchronous pipelining" and "Windowed usage" becoming "Asynchronous usage" to match. Where the text described a per-API default or cached instance, it now names the method it means, which is more precise than "face" was: the cached instances are per-API, not per-method, so that sentence says so explicitly. Remove the note shortcode that restated the bannerText. The banner already carries the experimental warning, so the note only added the version requirement, which now sits in the intro prose instead. Say "Sentinel" when mentioning NewFailoverClient. The bare word "failover" reads as the cross-region failover feature currently in preview for other clients; this is the long-standing Sentinel client (sentinel.go, "uses Redis Sentinel for automatic failover"), and go-redis has no geo-failover API at all. Naming Sentinel removes the ambiguity rather than dropping the sentence, since the scope statement is accurate and useful. Learned: NewFailoverClient is Sentinel failover and long released, unrelated to the previewed cross-region failover on the Lettuce and redis-py failover pages Directive: do not reintroduce "face" for the two autopipelining APIs; it is go-redis' internal jargon, not reader-facing Go terminology Constraint: the bannerText carries the experimental warning on this page; do not add a note shortcode that repeats it Ticket: DOC-6832 Co-Authored-By: Claude Opus 5 (1M context) --- content/develop/clients/go/autopipeline.md | 48 +++++++++++----------- 1 file changed, 23 insertions(+), 25 deletions(-) diff --git a/content/develop/clients/go/autopipeline.md b/content/develop/clients/go/autopipeline.md index 8b8f0b6651..caff0d94dc 100644 --- a/content/develop/clients/go/autopipeline.md +++ b/content/develop/clients/go/autopipeline.md @@ -16,12 +16,6 @@ title: Automatic pipelining weight: 42 --- -{{< note >}} -Automatic pipelining is an **experimental** feature and its API may still -change. It requires `github.com/redis/go-redis/v9` v9.22.0 or later. -{{< /note >}} -  - [Pipelining]({{< relref "/develop/using-commands/pipelining" >}}) sends a batch of commands to the server in a single communication, which avoids the network and processing overhead of sending each command separately. Normally you build @@ -33,11 +27,12 @@ commands concurrently, `go-redis` coalesces them into deep pipelines for you, without any pipeline code in your application. Reach for it in high-throughput, high-concurrency, or scale scenarios. At low concurrency, a plain client is simpler and just as fast, and a hand-written pipeline is still fastest when you -can batch by hand. +can batch by hand. Automatic pipelining requires +`github.com/redis/go-redis/v9` v9.22.0 or later. -## The two faces +## Blocking and asynchronous pipelining -Automatic pipelining has two forms that share the same underlying engine: +Automatic pipelining has two methods that share the same underlying engine: - **Blocking** (`AutoPipeline()`) is a drop-in replacement for a normal client. Each command call blocks until it executes and returns its own @@ -45,14 +40,15 @@ Automatic pipelining has two forms that share the same underlying engine: working unchanged. Under concurrency, the engine batches commands from all goroutines into back-to-back pipelines behind the scenes. Per-goroutine ordering is preserved. -- **Async** (`AsyncAutoPipeline()`) is deferred and offers the highest +- **Asynchronous** (`AsyncAutoPipeline()`) is deferred and offers the highest throughput. Command calls return immediately; reading a result with `Val()`, `Result()`, or `Err()` blocks until the batch executes. Submit a window of commands and then drain the results to keep each pipeline as deep as possible. -Both methods are available on `Client`, `ClusterClient`, and `Ring`. A failover -client created with `NewFailoverClient()` is a `*Client`, so it has them too. +Both methods are available on `Client`, `ClusterClient`, and `Ring`. A Sentinel +failover client created with `NewFailoverClient()` is a `*Client`, so it has +them too. ## Blocking usage @@ -65,7 +61,7 @@ rdb := redis.NewClient(&redis.Options{Addr: "localhost:6379"}) defer rdb.Close() ctx := context.Background() -// Blocking face: a drop-in for a normal client, batched under the hood. +// Blocking: a drop-in for a normal client, batched under the hood. ap, err := rdb.AutoPipeline() if err != nil { // only returned for invalid AutoPipelineOptions log.Fatal(err) @@ -86,10 +82,11 @@ for i := 0; i < 1000; i++ { wg.Wait() ``` -## Windowed usage +## Asynchronous usage -For maximum throughput, use the async face. Command calls return immediately, so -you can submit a window of commands and read their results afterwards: +For maximum throughput, use the asynchronous method. Command calls return +immediately, so you can submit a window of commands and read their results +afterwards: ```go ctx := context.Background() @@ -116,11 +113,12 @@ for _, cmd := range cmds { `AutoPipeline()` and `AsyncAutoPipeline()` take no arguments. They use the `AutoPipelineOptions` set on the client's options, if any, and otherwise the -built-in default for their face. To pass options for a single autopipeliner, -use `AutoPipelineWithOptions()` or `AsyncAutoPipelineWithOptions()` instead: +built-in default for the method you called. To pass options for a single +autopipeliner, use `AutoPipelineWithOptions()` or +`AsyncAutoPipelineWithOptions()` instead: ```go -// On the client, shared by both faces. +// On the client, used by both methods. rdb := redis.NewClient(&redis.Options{ Addr: "localhost:6379", AutoPipelineOptions: &redis.AutoPipelineOptions{MaxFlushDelay: 100 * time.Microsecond}, @@ -142,23 +140,23 @@ The configuration options are: | Field | Description | | :---- | :---------- | -| `MaxBatchSize` | Target number of commands the engine coalesces into a single pipeline before flushing. This is a soft threshold rather than a hard cap, so a busy queue can flush a larger batch. Defaults to 200, or to 300 for the blocking face's built-in default. | +| `MaxBatchSize` | Target number of commands the engine coalesces into a single pipeline before flushing. This is a soft threshold rather than a hard cap, so a busy queue can flush a larger batch. Defaults to 200, or 300 when `AutoPipeline()` falls back to its built-in default. | | `MaxBatchBytes` | Approximate limit on the argument bytes in a batch, so that large values flush as several bounded writes instead of one very large one. Also a soft threshold. Defaults to 0, meaning no byte limit. | | `MaxFlushDelay` | Maximum time the engine waits to accumulate more commands before flushing a batch. Larger values build deeper pipelines at the cost of latency. Defaults to 0, which adds no accumulation wait. | | `AdaptiveDelay` | Scales `MaxFlushDelay` down as the queue fills, so a busy queue flushes sooner. Requires `MaxFlushDelay` greater than 0. Defaults to `false`. | | `MaxConcurrentBatches` | Number of batches that may execute at once. Defaults to 1, which gives a single ordered stream. Values greater than 1 require `Unordered` because concurrent batches do not preserve a single ordered stream. | | `Unordered` | Allows commands to execute without preserving a single ordered stream, which enables higher concurrency. | -| `NumShards` | Number of independent queue-and-flusher shards. Defaults to 0, meaning a single shard, which funnels every caller into one queue so batches stay deep. Cluster clients default to several slot-routed shards instead. On the async face, more than one shard requires `Unordered`. | +| `NumShards` | Number of independent queue-and-flusher shards. Defaults to 0, meaning a single shard, which funnels every caller into one queue so batches stay deep. Cluster clients default to several slot-routed shards instead. With `AsyncAutoPipeline()`, more than one shard requires `Unordered`. | Connection and buffer tuning is not part of `AutoPipelineOptions`. Batches use the client's pipeline connections, which you size with the `PipelineReadBufferSize`, `PipelineWriteBufferSize`, and `PipelinePoolSize` fields of the client's options. -Each face has its own cached instance, shared by every caller of that method on -the client: the first call's options win and later calls return the same -instance. `Close()` stops that instance for all callers, and a later call to the -same method then builds a fresh one. Closing the client stops it permanently, +The blocking and asynchronous autopipeliners are cached separately, and each is +shared by all of its callers: the first call's options win and later calls +return the same instance. `Close()` stops that instance for every caller, and a +later call then builds a fresh one. Closing the client stops it permanently, after which the methods return `ErrClosed`. ## Cluster usage From e5b7424acdef14d586181bde0160e81b56add07d Mon Sep 17 00:00:00 2001 From: Andy Stark Date: Tue, 4 Aug 2026 11:41:42 +0100 Subject: [PATCH 5/8] DOC-6832 De-emphasize Sentinel and drop more upstream jargon Includes Andy's own edits: the Sentinel failover sentence is gone (still supported, but a legacy feature we aren't emphasizing), the intro is reworded, the version requirement is its own paragraph, and "drain the results" became "read the results afterwards". Continuing the same jargon sweep, "a window of commands" becomes "a sequence of commands" in both places it appeared. "Sequence" rather than "batch", the other candidate, because this page already uses "batch" two dozen times for the thing the engine flushes, and the asynchronous section's point is precisely that the caller submits commands while the engine decides how to batch them; reusing the word would blur that. Two more carried-over words found by sweeping the page against the upstream source rather than waiting to be told again. "is deferred and" is dropped from the asynchronous bullet: it was upstream's predicate for that API, and with the bullet already labelled Asynchronous and the next sentence explaining that calls return immediately, it was redundant as well as jargon. "queue-and-flusher shards" becomes "command queues, or shards, that the engine flushes separately", since "flusher" names an internal goroutine that nobody tuning NumShards needs to know about, while keeping the word "shards" tied to the field name. Left alone deliberately: "coalesces" and "soft threshold rather than a hard cap" also come from the upstream source, but both are ordinary technical English that a reader parses without knowing go-redis internals, which is the line this sweep is drawing. Learned: the jargon a source-derived page inherits is not only coined metaphors like "face" but ordinary-looking nouns ("window", "flusher") that silently name upstream internals Directive: sweep a source-derived page against the upstream vocabulary before review, not after; two rounds of this were reader-caught Ticket: DOC-6832 Co-Authored-By: Claude Opus 5 (1M context) --- content/develop/clients/go/autopipeline.md | 26 +++++++++++----------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/content/develop/clients/go/autopipeline.md b/content/develop/clients/go/autopipeline.md index caff0d94dc..9a9aad38dc 100644 --- a/content/develop/clients/go/autopipeline.md +++ b/content/develop/clients/go/autopipeline.md @@ -24,10 +24,12 @@ but this means you must know in advance which commands you want to batch. *Automatic pipelining* removes that requirement. When many goroutines issue commands concurrently, `go-redis` coalesces them into deep pipelines for you, -without any pipeline code in your application. Reach for it in high-throughput, -high-concurrency, or scale scenarios. At low concurrency, a plain client is -simpler and just as fast, and a hand-written pipeline is still fastest when you -can batch by hand. Automatic pipelining requires +without any pipeline code in your application. This is useful in high-throughput or +high-concurrency scenarios. At low concurrency, a plain client is +simpler and just as fast, and a hand-written pipeline is generally faster than +an auto-generated one. + +Automatic pipelining requires `github.com/redis/go-redis/v9` v9.22.0 or later. ## Blocking and asynchronous pipelining @@ -40,15 +42,13 @@ Automatic pipelining has two methods that share the same underlying engine: working unchanged. Under concurrency, the engine batches commands from all goroutines into back-to-back pipelines behind the scenes. Per-goroutine ordering is preserved. -- **Asynchronous** (`AsyncAutoPipeline()`) is deferred and offers the highest - throughput. Command calls return immediately; reading a result with +- **Asynchronous** (`AsyncAutoPipeline()`) offers the highest throughput. + Command calls return immediately; reading a result with `Val()`, `Result()`, or `Err()` blocks until the batch executes. Submit a - window of commands and then drain the results to keep each pipeline as deep - as possible. + sequence of commands and then read the results afterwards to keep each + pipeline as deep as possible. -Both methods are available on `Client`, `ClusterClient`, and `Ring`. A Sentinel -failover client created with `NewFailoverClient()` is a `*Client`, so it has -them too. +Both methods are available on `Client`, `ClusterClient`, and `Ring`. ## Blocking usage @@ -85,7 +85,7 @@ wg.Wait() ## Asynchronous usage For maximum throughput, use the asynchronous method. Command calls return -immediately, so you can submit a window of commands and read their results +immediately, so you can submit a sequence of commands and read their results afterwards: ```go @@ -146,7 +146,7 @@ The configuration options are: | `AdaptiveDelay` | Scales `MaxFlushDelay` down as the queue fills, so a busy queue flushes sooner. Requires `MaxFlushDelay` greater than 0. Defaults to `false`. | | `MaxConcurrentBatches` | Number of batches that may execute at once. Defaults to 1, which gives a single ordered stream. Values greater than 1 require `Unordered` because concurrent batches do not preserve a single ordered stream. | | `Unordered` | Allows commands to execute without preserving a single ordered stream, which enables higher concurrency. | -| `NumShards` | Number of independent queue-and-flusher shards. Defaults to 0, meaning a single shard, which funnels every caller into one queue so batches stay deep. Cluster clients default to several slot-routed shards instead. With `AsyncAutoPipeline()`, more than one shard requires `Unordered`. | +| `NumShards` | Number of independent command queues, or shards, that the engine flushes separately. Defaults to 0, meaning a single shard, which funnels every caller into one queue so batches stay deep. Cluster clients default to several slot-routed shards instead. With `AsyncAutoPipeline()`, more than one shard requires `Unordered`. | Connection and buffer tuning is not part of `AutoPipelineOptions`. Batches use the client's pipeline connections, which you size with the From 510e238a2150fcc855d72cdf797812bb2fd1ab4d Mon Sep 17 00:00:00 2001 From: Andy Stark Date: Tue, 4 Aug 2026 12:57:14 +0100 Subject: [PATCH 6/8] DOC-6832 Clarify the MaxBatchSize default instead of listing two Andy asked why one field had two default values. It does not, and the table row said so misleadingly: "Defaults to 200, or 300 when AutoPipeline() falls back to its built-in default" implied a per-method default when the truth is a precedence chain. newAutoPipeliner coerces any MaxBatchSize <= 0 to 200 regardless of which method built the autopipeliner, so 200 is the engine's only default. The 300 exists solely inside DefaultBlockingAutoPipelineOptions, which is consulted only when no options are supplied anywhere: explicit options beat Options.AutoPipelineOptions, which beats the per-method preset. The consequence the old wording hid is that supplying AutoPipelineOptions at all, even an empty struct, drops the blocking target from 300 back to 200 for an unset MaxBatchSize -- a silent change from a seemingly unrelated edit. The row now states the single default, 200, and a following paragraph explains the preset and the precedence. Prose is the right home because MaxBatchSize is the only field where the two presets differ at all: both set MaxConcurrentBatches to 1 and both leave MaxFlushDelay at 0, so this is one field's quirk rather than a general per-method-defaults story that a whole column of the table would need to carry. Not documented, because upstream does not say it: there is no stated rationale for 300 over 200. DefaultBlockingAutoPipelineOptions explains at length why the blocking preset keeps MaxConcurrentBatches at 1, but never ties the batch target to that reasoning, so the page describes the value without inventing a why. Learned: MaxBatchSize 200 is the engine default and 300 is a no-options-only preset, so supplying any AutoPipelineOptions silently drops the blocking target to 200 Constraint: do not restate 300 as a per-method default for MaxBatchSize; it applies only when no options are set anywhere Gaps: upstream gives no rationale for the 300 preset value; the page deliberately does not offer one Ticket: DOC-6832 Co-Authored-By: Claude Opus 5 (1M context) --- content/develop/clients/go/autopipeline.md | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/content/develop/clients/go/autopipeline.md b/content/develop/clients/go/autopipeline.md index 9a9aad38dc..c2afa639cd 100644 --- a/content/develop/clients/go/autopipeline.md +++ b/content/develop/clients/go/autopipeline.md @@ -84,7 +84,7 @@ wg.Wait() ## Asynchronous usage -For maximum throughput, use the asynchronous method. Command calls return +For maximum throughput, use asynchronous execution. Command calls return immediately, so you can submit a sequence of commands and read their results afterwards: @@ -112,8 +112,8 @@ for _, cmd := range cmds { ## Configuration `AutoPipeline()` and `AsyncAutoPipeline()` take no arguments. They use the -`AutoPipelineOptions` set on the client's options, if any, and otherwise the -built-in default for the method you called. To pass options for a single +`AutoPipelineOptions` set on the client's options, if any, and otherwise use +reasonable default values. To pass options for a single autopipeliner, use `AutoPipelineWithOptions()` or `AsyncAutoPipelineWithOptions()` instead: @@ -133,21 +133,27 @@ ap, err := rdb.AsyncAutoPipelineWithOptions(&redis.AutoPipelineOptions{ All four methods return `(*AutoPipeliner, error)`. The error is non-nil only when the options are invalid (for example, setting `MaxConcurrentBatches` -greater than 1 without also setting `Unordered`); invalid options are never a -panic, and no instance is cached. +greater than one without also setting `Unordered`). Invalid options never cause a +panic. The configuration options are: | Field | Description | | :---- | :---------- | -| `MaxBatchSize` | Target number of commands the engine coalesces into a single pipeline before flushing. This is a soft threshold rather than a hard cap, so a busy queue can flush a larger batch. Defaults to 200, or 300 when `AutoPipeline()` falls back to its built-in default. | -| `MaxBatchBytes` | Approximate limit on the argument bytes in a batch, so that large values flush as several bounded writes instead of one very large one. Also a soft threshold. Defaults to 0, meaning no byte limit. | +| `MaxBatchSize` | Target number of commands the engine coalesces into a single pipeline before flushing. This is a soft threshold rather than a hard cap, so a busy queue can flush a larger batch. Defaults to 200. | +| `MaxBatchBytes` | Soft limit on the argument bytes in a batch, so that large values flush as several bounded writes instead of one very large one. Defaults to 0, meaning no byte limit. | | `MaxFlushDelay` | Maximum time the engine waits to accumulate more commands before flushing a batch. Larger values build deeper pipelines at the cost of latency. Defaults to 0, which adds no accumulation wait. | | `AdaptiveDelay` | Scales `MaxFlushDelay` down as the queue fills, so a busy queue flushes sooner. Requires `MaxFlushDelay` greater than 0. Defaults to `false`. | | `MaxConcurrentBatches` | Number of batches that may execute at once. Defaults to 1, which gives a single ordered stream. Values greater than 1 require `Unordered` because concurrent batches do not preserve a single ordered stream. | | `Unordered` | Allows commands to execute without preserving a single ordered stream, which enables higher concurrency. | | `NumShards` | Number of independent command queues, or shards, that the engine flushes separately. Defaults to 0, meaning a single shard, which funnels every caller into one queue so batches stay deep. Cluster clients default to several slot-routed shards instead. With `AsyncAutoPipeline()`, more than one shard requires `Unordered`. | +`MaxBatchSize` is the one default that differs between the two methods. If you +set no options at all, `AutoPipeline()` uses a built-in preset that targets 300 +commands instead of 200. As soon as you supply `AutoPipelineOptions`, either on +the client or to `AutoPipelineWithOptions()`, that preset no longer applies, and +a `MaxBatchSize` you leave unset means 200. + Connection and buffer tuning is not part of `AutoPipelineOptions`. Batches use the client's pipeline connections, which you size with the `PipelineReadBufferSize`, `PipelineWriteBufferSize`, and `PipelinePoolSize` From c867787b41585161435ad60be1e229e80af73310 Mon Sep 17 00:00:00 2001 From: Andy Stark Date: Tue, 4 Aug 2026 13:13:35 +0100 Subject: [PATCH 7/8] DOC-6832 Describe the autopipeliners as singletons Andy suggested the singleton pattern as a framing readers of this page already know, with a link to the Wikipedia article. It fits exactly: each client keeps one blocking and one asynchronous autopipeliner, created lazily on first use and shared thereafter, which is what the old "cached separately, and each is shared by all of its callers" wording was groping towards. Naming the pattern also let a real trap be stated plainly instead of implied. The old text said "the first call's options win", which is true but leaves the reader to work out the consequence. Verified in the source: AutoPipeline() and AutoPipelineWithOptions() both resolve through &c.autopipeliner, so they are two entry points to one singleton rather than one shared and one bespoke instance. Calling AutoPipelineWithOptions() when the singleton already exists therefore returns the existing instance and silently discards the options you passed. The page now says so, and points out that closing is consequently how you apply different options, since the next call builds a fresh one. The Wikipedia link follows established practice here: 407 content pages already link to Wikipedia, and the italic-on-first-use form matches existing entries such as [*idempotent*]. Learned: AutoPipeline() and AutoPipelineWithOptions() share one cached instance per client, so the WithOptions variant silently ignores its options once the singleton exists Constraint: describe the blocking and asynchronous autopipeliners as two singletons per client; the WithOptions variants are entry points to them, not separate instances Ticket: DOC-6832 Co-Authored-By: Claude Opus 5 (1M context) --- content/develop/clients/go/autopipeline.md | 27 ++++++++++++++-------- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/content/develop/clients/go/autopipeline.md b/content/develop/clients/go/autopipeline.md index c2afa639cd..eef6694b7b 100644 --- a/content/develop/clients/go/autopipeline.md +++ b/content/develop/clients/go/autopipeline.md @@ -141,12 +141,12 @@ The configuration options are: | Field | Description | | :---- | :---------- | | `MaxBatchSize` | Target number of commands the engine coalesces into a single pipeline before flushing. This is a soft threshold rather than a hard cap, so a busy queue can flush a larger batch. Defaults to 200. | -| `MaxBatchBytes` | Soft limit on the argument bytes in a batch, so that large values flush as several bounded writes instead of one very large one. Defaults to 0, meaning no byte limit. | +| `MaxBatchBytes` | Soft limit on the total size of arguments (in bytes) for a batch, so that large values flush as several bounded writes instead of one very large one. Defaults to 0, meaning no byte limit. | | `MaxFlushDelay` | Maximum time the engine waits to accumulate more commands before flushing a batch. Larger values build deeper pipelines at the cost of latency. Defaults to 0, which adds no accumulation wait. | -| `AdaptiveDelay` | Scales `MaxFlushDelay` down as the queue fills, so a busy queue flushes sooner. Requires `MaxFlushDelay` greater than 0. Defaults to `false`. | -| `MaxConcurrentBatches` | Number of batches that may execute at once. Defaults to 1, which gives a single ordered stream. Values greater than 1 require `Unordered` because concurrent batches do not preserve a single ordered stream. | +| `AdaptiveDelay` | Scales `MaxFlushDelay` down as the queue fills, so a busy queue flushes sooner. Requires `MaxFlushDelay` to be greater than 0. Defaults to `false`. | +| `MaxConcurrentBatches` | Number of batches that may execute at once. Defaults to 1, which gives a single ordered stream. Values greater than 1 require `Unordered` set to `true` because concurrent batches do not preserve a single ordered stream. | | `Unordered` | Allows commands to execute without preserving a single ordered stream, which enables higher concurrency. | -| `NumShards` | Number of independent command queues, or shards, that the engine flushes separately. Defaults to 0, meaning a single shard, which funnels every caller into one queue so batches stay deep. Cluster clients default to several slot-routed shards instead. With `AsyncAutoPipeline()`, more than one shard requires `Unordered`. | +| `NumShards` | Number of independent command queues, or shards, that the engine flushes separately. Defaults to 0, meaning a single shard, which funnels every caller into one queue so batches stay deep. Cluster clients default to several slot-routed shards instead. With `AsyncAutoPipeline()`, `Unordered` to be set to `true` if `NumShards` is greater than 1. | `MaxBatchSize` is the one default that differs between the two methods. If you set no options at all, `AutoPipeline()` uses a built-in preset that targets 300 @@ -159,11 +159,18 @@ the client's pipeline connections, which you size with the `PipelineReadBufferSize`, `PipelineWriteBufferSize`, and `PipelinePoolSize` fields of the client's options. -The blocking and asynchronous autopipeliners are cached separately, and each is -shared by all of its callers: the first call's options win and later calls -return the same instance. `Close()` stops that instance for every caller, and a -later call then builds a fresh one. Closing the client stops it permanently, -after which the methods return `ErrClosed`. +Each client holds at most two autopipeliners: one for the blocking method and +one for the asynchronous method. Each of them is a +[*singleton*](https://en.wikipedia.org/wiki/Singleton_pattern) that the client +creates on first use and then shares with every later caller. + +Options therefore only take effect on the call that creates the singleton. If a +blocking autopipeliner already exists, a later `AutoPipelineWithOptions()` call +returns that same instance and ignores the options you passed, because +`AutoPipeline()` and `AutoPipelineWithOptions()` share one singleton between +them. `Close()` stops the singleton for every caller and the next call creates a +fresh one, so closing is also how you apply different options. Closing the +client is permanent: both methods then return `ErrClosed`. ## Cluster usage @@ -176,7 +183,7 @@ span many slots. Ordering is per key: same-key commands stay in order, while sub-pipelines on different nodes run concurrently. Commands that must reach every node or shard, such as -[`FLUSHALL`]({{< relref "/commands/flushall" >}}), cannot ride a pipeline, so +[`FLUSHALL`]({{< relref "/commands/flushall" >}}), cannot be added to a pipeline, so the cluster client rejects them with an error rather than let them spoil a batch shared with other callers. Run them on the plain client instead. From b2e883a28a01ddcf5e51f8b1a81dfe59b824d000 Mon Sep 17 00:00:00 2001 From: Andy Stark Date: Tue, 4 Aug 2026 13:17:03 +0100 Subject: [PATCH 8/8] DOC-6832 Fix missing verb in the NumShards table row Ticket: DOC-6832 Co-Authored-By: Claude Opus 5 (1M context) --- content/develop/clients/go/autopipeline.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/develop/clients/go/autopipeline.md b/content/develop/clients/go/autopipeline.md index eef6694b7b..274a4b22e0 100644 --- a/content/develop/clients/go/autopipeline.md +++ b/content/develop/clients/go/autopipeline.md @@ -146,7 +146,7 @@ The configuration options are: | `AdaptiveDelay` | Scales `MaxFlushDelay` down as the queue fills, so a busy queue flushes sooner. Requires `MaxFlushDelay` to be greater than 0. Defaults to `false`. | | `MaxConcurrentBatches` | Number of batches that may execute at once. Defaults to 1, which gives a single ordered stream. Values greater than 1 require `Unordered` set to `true` because concurrent batches do not preserve a single ordered stream. | | `Unordered` | Allows commands to execute without preserving a single ordered stream, which enables higher concurrency. | -| `NumShards` | Number of independent command queues, or shards, that the engine flushes separately. Defaults to 0, meaning a single shard, which funnels every caller into one queue so batches stay deep. Cluster clients default to several slot-routed shards instead. With `AsyncAutoPipeline()`, `Unordered` to be set to `true` if `NumShards` is greater than 1. | +| `NumShards` | Number of independent command queues, or shards, that the engine flushes separately. Defaults to 0, meaning a single shard, which funnels every caller into one queue so batches stay deep. Cluster clients default to several slot-routed shards instead. With `AsyncAutoPipeline()`, values greater than 1 require `Unordered` to be set to `true`. | `MaxBatchSize` is the one default that differs between the two methods. If you set no options at all, `AutoPipeline()` uses a built-in preset that targets 300