Skip to content

transaction: Support file based transaction#1998

Draft
pingyu wants to merge 16 commits into
tikv:masterfrom
pingyu:txn-file-cse
Draft

transaction: Support file based transaction#1998
pingyu wants to merge 16 commits into
tikv:masterfrom
pingyu:txn-file-cse

Conversation

@pingyu

@pingyu pingyu commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Changes

Summary by CodeRabbit

  • New Features
    • Added new configuration options for file-based transactions (chunk sizing, minimum mutation size, RU discounting, and request-source whitelisting).
    • Added an option to disable file-based transaction execution and use the standard 2PC path.
  • Bug Fixes
    • Improved region split handling by resolving split-key locks before continuing.
  • Observability
    • Added Prometheus metrics for txn-file requests, write bytes, mutation size, and duration (including internal vs. general and success vs. error).
  • Tests
    • Expanded unit and integration tests for txn-file sizing, batching across splits/regroups, retry behavior, and request decision logic.

Signed-off-by: Ping Yu <yuping@pingcap.com>
@ti-chi-bot ti-chi-bot Bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. dco-signoff: yes Indicates the PR's author has signed the dco. labels Jun 9, 2026
@ti-chi-bot

ti-chi-bot Bot commented Jun 9, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign zyguan for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@pingyu
pingyu marked this pull request as draft June 9, 2026 14:42
@coderabbitai

coderabbitai Bot commented Jun 9, 2026

Copy link
Copy Markdown

Review Change Stack

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 41b05079-6d56-41ea-a830-9400f152007f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Introduces transaction-file execution for large transactions through chunked HTTP uploads, region-aware prewrite/commit/rollback, lock resolution, optional region pre-splitting, configuration, metrics, 2PC integration, and unit/integration tests.

Changes

Txn File 2PC Feature

Layer / File(s) Summary
Configuration and metrics setup
config/client.go, kv/variables.go, metrics/*.go, internal/locate/region_cache.go, internal/resourcecontrol/resource_control.go
Adds txn-file configuration and variable fields, Prometheus metrics and shortcuts, a region-cache codec accessor, and a Resource Control request constructor.
2PC and range helpers
txnkv/transaction/2pc.go, txnkv/transaction/test_probe.go, util/misc.go, txnkv/transaction/2pc_test.go, util/misc_test.go
Adds txn-file-aware primary and heartbeat handling, range mutation lookup, and byte-range helper functions with tests.
Lock-aware region splitting
tikv/split_region.go
Resolves locks reported in split-key errors and applies TTL-based backoff before continuing region processing.
Transaction-level txn-file control
txnkv/transaction/txn.go
Adds DisableTxnFile() and conditional selection between txn-file execution and standard 2PC.
Txn-file 2PC execution
txnkv/transaction/txn_file.go
Implements chunk construction and HTTP upload, region batching, prewrite/commit/rollback orchestration, retries, pre-splitting, Resource Control accounting, metrics, and request-source gating.
Txn-file validation
txnkv/transaction/txn_file_test.go, integration_tests/txn_file_test.go, tikv/kv_test.go
Adds unit and integration coverage for commit timestamps, RPC outcomes, chunking, request-source decisions, prewrite sizing, region-split retries, and intest-gated error injection.

Estimated code review effort: 5 (Critical) | ~120 minutes

Sequence Diagram(s)

sequenceDiagram
    participant Client as KVTxn
    participant Committer as twoPhaseCommitter
    participant ChunkWriter as chunkWriterClient
    participant RegionCache as RegionCache
    participant TiKV as TiKV Region

    Client->>Committer: Commit()
    Committer->>Committer: buildTxnFiles(mutations)
    Committer->>ChunkWriter: POST chunk data with CRC32
    ChunkWriter-->>Committer: chunk IDs and ranges
    Committer->>RegionCache: group chunk ranges into region batches
    Committer->>TiKV: Prewrite chunk IDs and TxnSize
    TiKV-->>Committer: Prewrite response
    Committer->>TiKV: Commit or BatchRollback
    TiKV-->>Committer: Commit or rollback response
    Committer-->>Client: commit result
Loading
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 26.32% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title matches the main change: adding support for file-based transactions across the transaction stack.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ti-chi-bot ti-chi-bot Bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Jun 9, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 9

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
metrics/metrics.go (1)

1120-1215: ⚠️ Potential issue | 🟠 Major

Register txn-file metrics in RegisterMetrics.

metrics/metrics.go defines TiKVTxnFileRequestCounter, TiKVTxnFileWriteBytes, TiKVTxnFileMutationSizeHistogram, and TiKVTxnFileDuration, but RegisterMetrics has no prometheus.MustRegister(...) calls for any of them, so they won’t be exposed to Prometheus.

🔧 Proposed fix
 func RegisterMetrics() {
@@
 	prometheus.MustRegister(TiKVTxnLagCommitTSAttemptHistogram)
 	prometheus.MustRegister(TiKVStaleBucketFromPDCounter)
+	prometheus.MustRegister(TiKVTxnFileRequestCounter)
+	prometheus.MustRegister(TiKVTxnFileWriteBytes)
+	prometheus.MustRegister(TiKVTxnFileMutationSizeHistogram)
+	prometheus.MustRegister(TiKVTxnFileDuration)
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@metrics/metrics.go` around lines 1120 - 1215, RegisterMetrics is missing
registrations for the txn-file metrics; add prometheus.MustRegister calls for
TiKVTxnFileRequestCounter, TiKVTxnFileWriteBytes,
TiKVTxnFileMutationSizeHistogram, and TiKVTxnFileDuration inside the
RegisterMetrics function so those metrics are exposed to Prometheus (use the
same pattern as the other metrics already registered).
🧹 Nitpick comments (3)
integration_tests/txn_file_test.go (1)

259-268: ⚡ Quick win

Consider replacing time.Sleep with deterministic synchronization.

The 3-second sleep adds latency to CI and may be flaky if the commit goroutine doesn't reach the paused failpoint within that window. A more robust approach would use a channel or another failpoint to signal when the commit has reached the desired state before triggering the split.

For example, you could add a "reached prewrite" failpoint that signals a channel, then wait on that channel before calling cluster.Split.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@integration_tests/txn_file_test.go` around lines 259 - 268, Replace the
non-deterministic time.Sleep with a synchronization channel signaled by a new
failpoint: instrument the commit path with a "reached prewrite" failpoint that
calls close(prewriteReachedCh) (or sends on it) when the commit goroutine
reaches the paused state, then in the test wait on that channel before calling
cluster.Split(...) instead of sleeping; keep the existing goroutine that runs
txn.Commit(ctx), ensure you still disable the "tikvclient/invalidCacheAndRetry"
failpoint after split, and preserve the final <-done wait to ensure Commit
finishes. Use identifiers txn.Commit, cluster.Split, failpoint.Disable and the
done channel to locate and update the test and add the new failpoint signal in
the commit execution path.
txnkv/transaction/txn_file_test.go (1)

51-56: ⚡ Quick win

Use a local seeded RNG for deterministic test data.

Using package-level math/rand here can couple outcomes to global RNG state from other tests. Prefer a local RNG with a fixed seed.

♻️ Suggested change
 func TestChunkSliceSortAndDedup(t *testing.T) {
 	assert := assert.New(t)
+	rng := rand.New(rand.NewSource(1))
 
 	genRndChunkIDs := func() []uint64 {
-		n := rand.Intn(10)
+		n := rng.Intn(10)
 		ids := make([]uint64, 0, n)
 		for i := 0; i < n; i++ {
-			ids = append(ids, uint64(rand.Intn(n+n/2+1)))
+			ids = append(ids, uint64(rng.Intn(n+n/2+1)))
 		}
 		return ids
 	}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@txnkv/transaction/txn_file_test.go` around lines 51 - 56, The test's
genRndChunkIDs uses package-level math/rand causing non-determinism; replace it
with a local RNG created via rand.New(rand.NewSource(<fixedSeed>)) (e.g., 42)
and use that RNG (r.Intn) for n and id generation inside genRndChunkIDs so the
test data is deterministic and not affected by global RNG state; ensure the seed
is fixed and deterministic for the test run and that all rand.* calls in
genRndChunkIDs are switched to the local RNG variable.
txnkv/transaction/2pc.go (1)

343-343: ⚡ Quick win

Document or unexport MutationsHasDataInRange.

This new exported symbol lacks a Go doc comment. If it’s intended to be public, add a doc comment; otherwise make it unexported to keep package surface tight.

As per coding guidelines, "{tikv,rawkv,txnkv,kv,tikvrpc,oracle,config,metrics,trace,error}/**/*.go: Exported identifiers must have clear Go doc comments when they are part of the public client API".

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@txnkv/transaction/2pc.go` at line 343, MutationsHasDataInRange is an exported
function without a Go doc comment; either add a clear Go doc comment describing
its purpose, parameters (mutations, start, end), return values (firstDataKey,
found bool) and behavior, or make it unexported by renaming it to
mutationsHasDataInRange if it is not part of the public API; update all call
sites accordingly (look for MutationsHasDataInRange references) so the package
surface adheres to the exported-identifiers documentation rule.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@txnkv/transaction/2pc.go`:
- Around line 356-371: MutationsHasDataInRange currently can return (nil, true)
when the loop never assigns firstDataKey (e.g., range contains only non-write
ops); change the logic in the MutationsHasDataInRange function around the
isInRange(pos) block so you only return true when a valid key was found: inside
the loop set firstDataKey = mutations.GetKey(pos) when pos==0 or
isOpForWrite(mutations.GetOp(pos)), break, and after the loop check if
firstDataKey != nil before returning (firstDataKey, true); if firstDataKey is
nil, return (nil, false) instead. Ensure you reference the existing helpers/vars
(isInRange, mutations.GetOp, mutations.GetKey, isOpForWrite, pos, firstDataKey)
so the change is localized to that block.

In `@txnkv/transaction/txn_file.go`:
- Around line 960-967: Log call is using the wrong error variable: change
logutil.Logger(bo.GetCtx()).Error(..., zap.Error(err)) to pass the
channel-result error (r.err) instead of the outer err; update the occurrences
inside the resultCh receive loop (where r := <-resultCh is handled) so the error
logged and the wrapped error return use r.err together with buildChunkErrMsg
(same fix needed for both occurrences around the resultCh handling).
- Around line 1274-1298: The defer resp.Body.Close() inside the retry loop leaks
resources; instead, close resp.Body immediately on each iteration after you're
done with it. Update the loop around w.cli.Do(req) so that you call
resp.Body.Close() (not defer) after reading the body or before any
continue/return paths (both the non-200 branch where you read bodyStr and the
success branch that reads data via io.ReadAll), ensuring every resp is closed on
that iteration; use the existing variables resp, data, err and the same
retry/backoff logic (retry.BoTiKVRPC, retry.BoTiKVServerBusy, bo.Backoff) but
remove the deferred close.
- Around line 736-762: The dispatch loop can skip spawning goroutines when
rateLimiter.GetToken(exitCh) signals exit, causing the receiver to block waiting
for len(batches) responses; fix by tracking only the number of goroutines
actually launched and closing the result channel when they finish. Replace the
manual receive-count approach with a sync.WaitGroup: before launching a
goroutine call wg.Add(1), inside the goroutine defer wg.Done(), and after the
dispatch loop start a goroutine that does wg.Wait() then close(ch); change the
consumer to range over ch until closed (instead of for i := 0; i < len(batches);
i++), keeping existing uses of rateLimiter.GetToken, rateLimiter.PutToken,
executeTxnFileSliceSingleBatch, regionErrChunks.appendSlice and respecting
returnEarly behavior when reading results.
- Line 332: The condition uses locate.IsFakeRegionError which doesn't exist;
replace that call with retry.IsFakeRegionError(regionErr) so the check reads: if
regionErr.GetEpochNotMatch() == nil || retry.IsFakeRegionError(regionErr) {;
update the reference where this appears (e.g., around the
regionErr/GetEpochNotMatch check in txn_file.go) to use the imported retry
package and its IsFakeRegionError function.
- Around line 889-912: The async goroutine is using the forked backoffer `bo`
that is cancelled by the parent function's deferred `cancel()` (from
`bo.Fork()`), causing `executeTxnFileSliceWithRetry` to fail; to fix, create an
independent backoffer/context for the async path (e.g., construct `asyncBO` from
a non-cancelled context such as `context.Background()` or the store's long-lived
context) and pass that `asyncBO` into the async call inside the `store.Go`
closure instead of the parent `bo`; ensure the async goroutine cleans up/cancels
`asyncBO` when it finishes if needed and keep all references to
`executeTxnFileSliceWithRetry`, `executeTxnFileAction`, `bo.Fork()`, and
`store.Go` so the change is localized.
- Around line 259-263: The call site in MutationsHasDataInRange (in txn_file.go)
uses missing helpers util.GetMaxStartKey and util.GetMinEndKey; add these two
functions to the util package (e.g., func GetMaxStartKey(a, b []byte) []byte and
func GetMinEndKey(a, b []byte) []byte) that return the lexicographically greater
start key and the lexicographically smaller end key respectively (preserving
nil/empty semantics used elsewhere), or replace the calls with existing util
equivalents if they already exist; update imports and any unit tests to
reference the new util functions so MutationsHasDataInRange compiles.
- Around line 1117-1122: The call to resourcecontrol.NewRequestInfo in
txnkv/transaction/txn_file.go fails because only
resourcecontrol.MakeRequestInfo(req *tikvrpc.Request) exists and RequestInfo
fields are unexported; fix by either constructing a *tikvrpc.Request with the
appropriate payload/metadata and replacing NewRequestInfo(...) with
resourcecontrol.MakeRequestInfo(req) in the txn_file code, or (if you prefer API
addition) add an exported constructor function in
internal/resourcecontrol/resource_control.go (e.g., NewRequestInfoFromParams)
that accepts requestSize, accessType, replicaID/leaderStoreID and bypass and
returns a *resourcecontrol.RequestInfo with all internal fields (requestSize,
accessType, bypass, etc.) correctly initialized, then call that new constructor
from txn_file.go.

In `@txnkv/transaction/txn.go`:
- Around line 793-795: The exported method DisableTxnFile on type KVTxn lacks a
Go doc comment; add a one-line Go-style doc comment immediately above the
DisableTxnFile declaration explaining its purpose and behavior (e.g.,
"DisableTxnFile disables writing/using the transaction file for this KVTxn.") so
it complies with the public API doc rule; ensure the comment starts with
"DisableTxnFile" and briefly states that it sets disableTxnFile to true for the
KVTxn instance.

---

Outside diff comments:
In `@metrics/metrics.go`:
- Around line 1120-1215: RegisterMetrics is missing registrations for the
txn-file metrics; add prometheus.MustRegister calls for
TiKVTxnFileRequestCounter, TiKVTxnFileWriteBytes,
TiKVTxnFileMutationSizeHistogram, and TiKVTxnFileDuration inside the
RegisterMetrics function so those metrics are exposed to Prometheus (use the
same pattern as the other metrics already registered).

---

Nitpick comments:
In `@integration_tests/txn_file_test.go`:
- Around line 259-268: Replace the non-deterministic time.Sleep with a
synchronization channel signaled by a new failpoint: instrument the commit path
with a "reached prewrite" failpoint that calls close(prewriteReachedCh) (or
sends on it) when the commit goroutine reaches the paused state, then in the
test wait on that channel before calling cluster.Split(...) instead of sleeping;
keep the existing goroutine that runs txn.Commit(ctx), ensure you still disable
the "tikvclient/invalidCacheAndRetry" failpoint after split, and preserve the
final <-done wait to ensure Commit finishes. Use identifiers txn.Commit,
cluster.Split, failpoint.Disable and the done channel to locate and update the
test and add the new failpoint signal in the commit execution path.

In `@txnkv/transaction/2pc.go`:
- Line 343: MutationsHasDataInRange is an exported function without a Go doc
comment; either add a clear Go doc comment describing its purpose, parameters
(mutations, start, end), return values (firstDataKey, found bool) and behavior,
or make it unexported by renaming it to mutationsHasDataInRange if it is not
part of the public API; update all call sites accordingly (look for
MutationsHasDataInRange references) so the package surface adheres to the
exported-identifiers documentation rule.

In `@txnkv/transaction/txn_file_test.go`:
- Around line 51-56: The test's genRndChunkIDs uses package-level math/rand
causing non-determinism; replace it with a local RNG created via
rand.New(rand.NewSource(<fixedSeed>)) (e.g., 42) and use that RNG (r.Intn) for n
and id generation inside genRndChunkIDs so the test data is deterministic and
not affected by global RNG state; ensure the seed is fixed and deterministic for
the test run and that all rand.* calls in genRndChunkIDs are switched to the
local RNG variable.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 4fc0fea7-71aa-49c6-b6f1-29a2e40dc67c

📥 Commits

Reviewing files that changed from the base of the PR and between 8df1c02 and 74c5b70.

📒 Files selected for processing (12)
  • config/client.go
  • integration_tests/txn_file_test.go
  • kv/variables.go
  • metrics/metrics.go
  • metrics/shortcuts.go
  • tikv/split_region.go
  • txnkv/transaction/2pc.go
  • txnkv/transaction/2pc_test.go
  • txnkv/transaction/txn.go
  • txnkv/transaction/txn_file.go
  • txnkv/transaction/txn_file_test.go
  • txnkv/txnlock/lock_resolver.go

Comment thread txnkv/transaction/2pc.go
Comment thread txnkv/transaction/txn_file.go
Comment thread txnkv/transaction/txn_file.go Outdated
Comment thread txnkv/transaction/txn_file.go
Comment thread txnkv/transaction/txn_file.go
Comment thread txnkv/transaction/txn_file.go
Comment thread txnkv/transaction/txn_file.go
Comment thread txnkv/transaction/txn_file.go
Comment thread txnkv/transaction/txn.go
pingyu added 2 commits June 9, 2026 23:33
Signed-off-by: Ping Yu <yuping@pingcap.com>
Signed-off-by: Ping Yu <yuping@pingcap.com>
pingyu and others added 2 commits July 6, 2026 21:17
Signed-off-by: Ping Yu <yuping@pingcap.com>
@pingyu pingyu changed the title [WIP] transaction: Support file based transaction transaction: Support file based transaction Jul 6, 2026
@pingyu
pingyu marked this pull request as ready for review July 6, 2026 14:51
@ti-chi-bot ti-chi-bot Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 6, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@util/misc.go`:
- Around line 179-198: Add Go doc comments for the exported util functions
GetMaxStartKey and GetMinEndKey. Describe their comparison semantics and make
the empty-key behavior explicit, including that an empty start key acts as an
unbounded low value and an empty end key acts as an unbounded high value. Keep
the comments attached to the function declarations so the public API in util is
properly documented.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: afc08df7-a227-43a6-bbe6-2a9fc280e62f

📥 Commits

Reviewing files that changed from the base of the PR and between 74c5b70 and f558577.

📒 Files selected for processing (12)
  • integration_tests/txn_file_test.go
  • internal/locate/region_cache.go
  • internal/resourcecontrol/resource_control.go
  • kv/variables.go
  • metrics/metrics.go
  • tikv/kv_test.go
  • txnkv/transaction/2pc.go
  • txnkv/transaction/test_probe.go
  • txnkv/transaction/txn.go
  • txnkv/transaction/txn_file.go
  • util/misc.go
  • util/misc_test.go
🚧 Files skipped from review as they are similar to previous changes (6)
  • kv/variables.go
  • metrics/metrics.go
  • txnkv/transaction/txn.go
  • txnkv/transaction/2pc.go
  • integration_tests/txn_file_test.go
  • txnkv/transaction/txn_file.go

Comment thread util/misc.go
Comment thread txnkv/transaction/txn_file.go Outdated
}

commitBo := retry.NewBackofferWithVars(ctx, int(CommitMaxBackoff), c.txn.vars)
c.commitTS, err = c.store.GetTimestampWithRetry(commitBo, c.txn.GetScope())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the txn-file path use the same commitTS preparation/check path as normal 2PC here?

Normal 2PC gets the commitTS through txn.GetTimestampForCommit(...), then checks schema validity, MaxTxnTimeUse, and commitTSUpperBoundCheck before committing. This path calls store.GetTimestampWithRetry(...) directly, and the CommitTsExpired retry path does the same, so it seems to bypass SetCommitWaitUntilTSO, schema lease validation, and commitTS upper-bound constraints.

}

prewriteBo := retry.NewBackofferWithVars(ctx, int(PrewriteMaxBackoff.Load()), c.txn.vars)
err = c.executeTxnFileAction(prewriteBo, c.txnFileCtx.slice, txnFilePrewriteAction{})

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should txn-file either handle binlog the same way as normal 2PC, or fall back when c.shouldWriteBinlog() is true?

The normal 2PC path calls binlog.Prewrite(...) before KV prewrite and later calls binlog.Commit(...) or binlog.Skip(...). I don't see equivalent handling in executeTxnFile, and useTxnFile does not appear to exclude transactions with a binlog executor. This may cause a large txn-file transaction to commit to TiKV without producing the corresponding binlog.

c.setUndeterminedErr(errors.WithStack(sender.GetRPCError()))
}
// Unexpected error occurs, return it.
if err != nil {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this return ErrResultUndetermined for primary txn-file commit RPC errors, matching normal 2PC?

ResourceGroupName: c.resourceGroupName,
},
})
markTxnFileRetryRequest(req, bo)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the txn-file path expected to preserve dynamic resource-group tags from resourceGroupTagger?

The normal prewrite/commit/rollback paths call c.resourceGroupTagger(req) when c.resourceGroupTag == nil && c.resourceGroupTagger != nil. I don't see the same handling for txn-file prewrite, commit, or rollback requests, so a transaction may lose its dynamic resource-group tag once it crosses the txn-file threshold.

Comment thread metrics/metrics.go
ConstLabels: constLabels,
}, []string{LblResult})

TiKVTxnFileRequestCounter = prometheus.NewCounterVec(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like the new txn-file metrics may be missing from RegisterMetrics()?

Comment thread config/client.go
// TxnChunkWriterConcurrency is the concurrency to request the txn chunk writer for file-based txn.
TxnChunkWriterConcurrency uint `toml:"txn-chunk-writer-concurrency" json:"txn-chunk-writer-concurrency"`
// TxnChunkMaxSize is the maximum size of a txn chunk of file-based txn.
TxnChunkMaxSize uint64 `toml:"txn-chunk-max-size" json:"txn-chunk-max-size"`

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should validate the new txn-file config values before accepting them, especially TxnChunkMaxSize > 0? If this is set to 0, the txn-file path can divide by zero when calculating chunk counts or parallelism, so rejecting or normalizing it in Valid() would make the failure mode clearer.

pingyu added 4 commits July 14, 2026 00:24
Signed-off-by: Ping Yu <yuping@pingcap.com>
Signed-off-by: Ping Yu <yuping@pingcap.com>
Signed-off-by: Ping Yu <yuping@pingcap.com>
Signed-off-by: Ping Yu <yuping@pingcap.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
txnkv/transaction/txn_file_test.go (1)

558-804: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Add a duplicate-ID case to TestChunkSliceSortAndDedup
The test only covers duplicate IDs with identical range metadata, so it doesn’t pin down the first-occurrence-wins behavior when duplicates have different smallest/biggest values.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@txnkv/transaction/txn_file_test.go` around lines 558 - 804, Extend
TestChunkSliceSortAndDedup with a deterministic duplicate-ID case where repeated
IDs have different txnChunkRange metadata, and assert that sortAndDedup retains
the range from the first occurrence while producing sorted, deduplicated
chunkIDs. Keep the existing randomized coverage unchanged.
🧹 Nitpick comments (2)
txnkv/transaction/txn_file_test.go (2)

548-548: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Simplify redundant embedded-field selector.

Static analysis (staticcheck QF1008) flags this: ttlManager is embedded in twoPhaseCommitter, so committer.ttlManager.state can be written as committer.state.

♻️ Proposed simplification
-	committer.ttlManager.state = stateRunning
+	committer.state = stateRunning
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@txnkv/transaction/txn_file_test.go` at line 548, In the test setup around the
embedded twoPhaseCommitter field, update the assignment to use the promoted
state field directly via committer.state instead of the redundant
committer.ttlManager.state selector.

Source: Linters/SAST tools


491-506: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Direct manipulation of unexported package singletons (once, cli, errCli, scheme) for test setup is fragile.

Resetting these package-level vars works because tests in this file run sequentially, but it silently couples the test to the internal implementation names/shape of the chunk-writer client singleton in txn_file.go. Any rename/refactor of that singleton (or a future t.Parallel() addition elsewhere in the package) breaks or races this test without an obvious signal. Consider exposing a small internal test hook (e.g., a resetChunkWriterClientForTest() in txn_file.go) instead of poking at the vars directly.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@txnkv/transaction/txn_file_test.go` around lines 491 - 506, The test directly
resets the chunk-writer singleton internals, coupling it to implementation
details. Add an internal resetChunkWriterClientForTest hook near the singleton
implementation in txn_file.go that clears once, cli, errCli, and scheme, then
replace both direct reset blocks in the test with calls to that hook while
preserving the existing configuration restoration.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@txnkv/transaction/txn_file_test.go`:
- Around line 484-488: Update the HTTP handler in the chunkWriter test server to
replace require.Equal and require.NoError with non-terminating assertions such
as t.Errorf or assert calls. Keep the method validation and response-writing
behavior unchanged, ensuring failures from the request goroutine are reported
without invoking FailNow.

---

Outside diff comments:
In `@txnkv/transaction/txn_file_test.go`:
- Around line 558-804: Extend TestChunkSliceSortAndDedup with a deterministic
duplicate-ID case where repeated IDs have different txnChunkRange metadata, and
assert that sortAndDedup retains the range from the first occurrence while
producing sorted, deduplicated chunkIDs. Keep the existing randomized coverage
unchanged.

---

Nitpick comments:
In `@txnkv/transaction/txn_file_test.go`:
- Line 548: In the test setup around the embedded twoPhaseCommitter field,
update the assignment to use the promoted state field directly via
committer.state instead of the redundant committer.ttlManager.state selector.
- Around line 491-506: The test directly resets the chunk-writer singleton
internals, coupling it to implementation details. Add an internal
resetChunkWriterClientForTest hook near the singleton implementation in
txn_file.go that clears once, cli, errCli, and scheme, then replace both direct
reset blocks in the test with calls to that hook while preserving the existing
configuration restoration.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 519fb185-007a-4c49-9860-68a2d948d972

📥 Commits

Reviewing files that changed from the base of the PR and between f558577 and 26bd55b.

📒 Files selected for processing (2)
  • txnkv/transaction/txn_file.go
  • txnkv/transaction/txn_file_test.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • txnkv/transaction/txn_file.go

Comment thread txnkv/transaction/txn_file_test.go
Signed-off-by: Ping Yu <yuping@pingcap.com>
@pingyu
pingyu marked this pull request as draft July 15, 2026 01:29
@ti-chi-bot ti-chi-bot Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 15, 2026
pingyu added 3 commits July 15, 2026 15:16
Signed-off-by: Ping Yu <yuping@pingcap.com>
Signed-off-by: Ping Yu <yuping@pingcap.com>
Signed-off-by: Ping Yu <yuping@pingcap.com>
pingyu and others added 3 commits July 15, 2026 18:35
Signed-off-by: Ping Yu <yuping@pingcap.com>
Signed-off-by: Ping Yu <yuping@pingcap.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dco-signoff: yes Indicates the PR's author has signed the dco. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants