TOOLS-4073 Make mongorestore work when restoring a dump with replicated record IDs to a cluster that doesn't support this by autarch · Pull Request #875 · mongodb/mongo-tools

autarch · 2026-02-03T03:50:50Z

This was written by Copilot using Claude Opus and reviewed by me. I also made some additional changes based on issues I saw when testing these changes.

There are a few things this does:

If the cluster we're restoring doesn't have replicated record IDs enabled, it removes the recordIdsReplicated flag from collection options when they're being created.
- In the future, this collection option will go away entirely. Instead, if the cluster is configured for replicated record IDs, collections will always have this enabled. When that happens, we will remove this part of the code. See TOOLS-4076 for more details.
It always removes the record ID field from oplog entries (rid) when restoring them. Per discussion with Server folks working on this, the cluster will handle assigning record IDs and does not need this information.
It always filters out ci and cd ops when restoring from the oplog. These are new ops specific to replicated record IDs and index creation and deletion. They are internal operations that do not need to be replicated when restoring. The cluster will always manage indexes appropriately.

autarch · 2026-02-03T03:51:04Z

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

autarch · 2026-02-05T03:17:02Z

See this eng proposal for more context.

…ed record IDs to a cluster that doesn't support this

FGasper

The record ID type issue must be fixed.

We should also add tests to catch this & any other new/unknown BSON types for record IDs.

FGasper · 2026-02-06T15:07:42Z

common/db/command.go

+	if result.Err() != nil {
+		// If the command fails, the feature flag doesn't exist or isn't enabled. Or maybe something
+		// is totally broken, in which case we will find that out when we attempt other DB
+		// operations.


Can we instead check for an error code?

The failure may be intermittent, so ISTM we really shouldn’t just throw it away. At the very least it should be logged, but even then we’d want a good comment about why we can’t reliably fail.

FGasper · 2026-02-09T00:27:00Z

mongorestore/metadata.go

+				log.DebugLow,
+				"removing recordIdsReplicated option (target server does not support this feature)",
+			)
+			return append(options[:i], options[i+1:]...)


IMO slices.Delete() would be clearer.

Or, lo.Filter.

FGasper · 2026-02-09T00:29:05Z

common/db/oplog.go

 	TxnNumber   *int64              `bson:"txnNumber,omitempty"`
 	PrevOpTime  bson.Raw            `bson:"prevOpTime,omitempty"`
 	MultiOpType *int                `bson:"multiOpType,omitempty"`
+	RecordId    *int64              `bson:"rid,omitempty"`


Record IDs are not all int64s, though. In clustered collections they’re binary strings. And in 5.0 timeseries they’re strings.

Suggest: *bson.RawValue

visemet

Adding my own thoughts here.

visemet · 2026-02-12T06:42:01Z

mongorestore/oplog.go

+
+		filtered := make([]db.Oplog, len(ops))
+		for i, v := range ops {
+			filtered[i], err = restore.filterRecordIds(v)


Is there a test case which shows this logic is necessary? My understanding is HandleNonTxnOp() already unrolls top-level applyOps oplog entries. The oplog will not contain a nested applyOps oplog entry in any server version an "rid" field may be present.

I would have expected omitting the RecordId field from the Oplog struct to work as desired automatically. What am I missing?

mongo-tools/mongorestore/oplog.go

Lines 307 to 324 in 8279a5a

case "applyOps":

rawOps, ok := op.Object[0].Value.(bson.A)

if !ok {

return fmt.Errorf("unknown format for applyOps: %#v", op.Object)

}

for _, rawOp := range rawOps {

bytesOp, err := bson.Marshal(rawOp)

if err != nil {

return fmt.Errorf("could not marshal applyOps operation: %v: %v", rawOp, err)

}

var nestedOp db.Oplog

err = bson.Unmarshal(bytesOp, &nestedOp)

if err != nil {

return fmt.Errorf("could not unmarshal applyOps command: %v: %v", rawOp, err)

}

err = restore.HandleOp(oplogCtx, nestedOp)

[note] Prior to SERVER-45033 (7.0), the server would record the applyOps command the client had specified which means that the applyOps oplog entry could have another applyOps oplog entry within it. There was a finite limit imposed by the server of 10 nested applyOps.

visemet · 2026-02-12T06:53:34Z

mongorestore/metadata.go

+// filterRecordIdsReplicatedOption removes the `recordIdsReplicated` option from the collection
+// options if the target server doesn't have the `featureFlagRecordIdsReplicated` feature enabled.
+func (restore *MongoRestore) filterRecordIdsReplicatedOption(options bson.D) bson.D {
+	if restore.recordIdsReplicatedEnabled {


If featureFlagRecordIdsReplicated is enabled by default on the target system, then the create command is going to create a collection which has all replica set members using the same mapping of record ID → BSON document. The target system will do so even if recordIdsReplicated: true is omitted from the create command request.

There is zero interest in preserving the absence of recordIdsReplicated in the collection metadata as recordIdsReplicated: false on a target system with featureFlagRecordIdsReplicated enabled by default. Why not always remove the recordIdsReplicated collection option and let the target system decide based on its own default? This means there would be no need to run the getParameter command earlier.

@henrikedin @erreina would you please confirm that following SERVER-85589, the recordIdsReplicated option will no longer appear in op=c create collection oplog entries? If not, then there would be another place in mongorestore to always remove the recordIdsReplicated option from.

mongo-tools/mongorestore/oplog.go

Lines 363 to 364 in 8279a5a

case "create":

collName, ok := op.Object[0].Value.(string)

It will not be on the options in the create collection oplog entries, but it will now be on the o2 field.

Always removing the recordIdsReplicated field from the collection metadata and let the target decide seems reasonable to me.

This was referenced Feb 3, 2026

Centralize the connection logic for tests in one place and fix it for Atlas clusters #872

Merged

TOOLS-4074 Add a script to create clusters with the recordIdsReplicated feature enabled #873

Open

autarch mentioned this pull request Feb 3, 2026

TOOLS-4074 Add a tool to perform manual dump/restore testing across two clusters #874

Open

autarch force-pushed the 02-03-tools-4073_make_mongorestore_work_when_restoring_a_dump_with_replicated_record_ids_to_a_cluster_that_doesn_t_support_this branch from 07fe9a7 to c769b37 Compare February 4, 2026 04:05

autarch force-pushed the 02-03-add_a_tool_to_perform_manual_dump/restore_testing_across_two_clusters branch from b879434 to 03b7e3b Compare February 4, 2026 04:05

autarch mentioned this pull request Feb 4, 2026

TOOLS-4073 Simplify MongoRestore.ApplyOps method #876

Merged

autarch force-pushed the 02-03-tools-4073_make_mongorestore_work_when_restoring_a_dump_with_replicated_record_ids_to_a_cluster_that_doesn_t_support_this branch from c769b37 to 9e7e2c5 Compare February 4, 2026 04:13

autarch force-pushed the 02-03-add_a_tool_to_perform_manual_dump/restore_testing_across_two_clusters branch from 03b7e3b to 85fb02a Compare February 4, 2026 04:13

autarch force-pushed the 02-03-tools-4073_make_mongorestore_work_when_restoring_a_dump_with_replicated_record_ids_to_a_cluster_that_doesn_t_support_this branch from 9e7e2c5 to 12a6c61 Compare February 4, 2026 06:51

autarch force-pushed the 02-03-add_a_tool_to_perform_manual_dump/restore_testing_across_two_clusters branch from 85fb02a to 3dc8fcb Compare February 4, 2026 06:51

autarch force-pushed the 02-03-tools-4073_make_mongorestore_work_when_restoring_a_dump_with_replicated_record_ids_to_a_cluster_that_doesn_t_support_this branch from 12a6c61 to c224ed2 Compare February 4, 2026 06:55

autarch force-pushed the 02-03-add_a_tool_to_perform_manual_dump/restore_testing_across_two_clusters branch from 3dc8fcb to bd21afe Compare February 4, 2026 06:55

autarch force-pushed the 02-03-tools-4073_make_mongorestore_work_when_restoring_a_dump_with_replicated_record_ids_to_a_cluster_that_doesn_t_support_this branch from c224ed2 to f4746cf Compare February 4, 2026 14:13

autarch force-pushed the 02-03-add_a_tool_to_perform_manual_dump/restore_testing_across_two_clusters branch from bd21afe to 1c5b465 Compare February 4, 2026 14:13

autarch force-pushed the 02-03-tools-4073_make_mongorestore_work_when_restoring_a_dump_with_replicated_record_ids_to_a_cluster_that_doesn_t_support_this branch from f4746cf to 865983d Compare February 4, 2026 15:08

autarch force-pushed the 02-03-add_a_tool_to_perform_manual_dump/restore_testing_across_two_clusters branch from 1c5b465 to 0712cf7 Compare February 4, 2026 15:08

autarch force-pushed the 02-03-tools-4073_make_mongorestore_work_when_restoring_a_dump_with_replicated_record_ids_to_a_cluster_that_doesn_t_support_this branch from 865983d to a73c37a Compare February 5, 2026 03:11

autarch force-pushed the 02-03-add_a_tool_to_perform_manual_dump/restore_testing_across_two_clusters branch from 0712cf7 to afedc4d Compare February 5, 2026 03:11

autarch requested a review from FGasper February 5, 2026 03:16

autarch marked this pull request as ready for review February 5, 2026 03:16

autarch requested a review from a team as a code owner February 5, 2026 03:16

autarch force-pushed the 02-03-tools-4073_make_mongorestore_work_when_restoring_a_dump_with_replicated_record_ids_to_a_cluster_that_doesn_t_support_this branch from a73c37a to b351dd5 Compare February 5, 2026 03:20

autarch force-pushed the 02-03-add_a_tool_to_perform_manual_dump/restore_testing_across_two_clusters branch 2 times, most recently from 0f7684d to 5b02458 Compare February 5, 2026 15:52

autarch force-pushed the 02-03-tools-4073_make_mongorestore_work_when_restoring_a_dump_with_replicated_record_ids_to_a_cluster_that_doesn_t_support_this branch 2 times, most recently from 954005c to ff4ac8f Compare February 5, 2026 15:58

autarch force-pushed the 02-03-add_a_tool_to_perform_manual_dump/restore_testing_across_two_clusters branch from 5b02458 to 30faca2 Compare February 5, 2026 15:58

autarch mentioned this pull request Feb 5, 2026

Fix a post-merge bug #878

Merged

autarch force-pushed the 02-03-add_a_tool_to_perform_manual_dump/restore_testing_across_two_clusters branch from 30faca2 to 84ee53f Compare February 5, 2026 16:29

TOOLS-4073 Make mongorestore work when restoring a dump with replicat…

8279a5a

…ed record IDs to a cluster that doesn't support this

autarch force-pushed the 02-03-tools-4073_make_mongorestore_work_when_restoring_a_dump_with_replicated_record_ids_to_a_cluster_that_doesn_t_support_this branch from ff4ac8f to 8279a5a Compare February 5, 2026 16:30

FGasper requested changes Feb 9, 2026

View reviewed changes

visemet suggested changes Feb 12, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TOOLS-4073 Make mongorestore work when restoring a dump with replicated record IDs to a cluster that doesn't support this#875

autarch commented Feb 3, 2026 •

edited

Loading

Uh oh!

autarch commented Feb 3, 2026 •

edited

Loading

Uh oh!

autarch commented Feb 5, 2026

Uh oh!

FGasper left a comment

Uh oh!

FGasper Feb 6, 2026

Uh oh!

FGasper Feb 9, 2026

Uh oh!

FGasper Feb 12, 2026

Uh oh!

FGasper Feb 9, 2026

Uh oh!

FGasper Feb 12, 2026

Uh oh!

visemet left a comment

Uh oh!

visemet Feb 12, 2026

Uh oh!

visemet Feb 12, 2026

Uh oh!

visemet Feb 12, 2026

Uh oh!

erreina Feb 12, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

	case "applyOps":
	rawOps, ok := op.Object[0].Value.(bson.A)
	if !ok {
	return fmt.Errorf("unknown format for applyOps: %#v", op.Object)
	}

	for _, rawOp := range rawOps {
	bytesOp, err := bson.Marshal(rawOp)
	if err != nil {
	return fmt.Errorf("could not marshal applyOps operation: %v: %v", rawOp, err)
	}
	var nestedOp db.Oplog
	err = bson.Unmarshal(bytesOp, &nestedOp)
	if err != nil {
	return fmt.Errorf("could not unmarshal applyOps command: %v: %v", rawOp, err)
	}

	err = restore.HandleOp(oplogCtx, nestedOp)

Conversation

autarch commented Feb 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

autarch commented Feb 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

autarch commented Feb 5, 2026

Uh oh!

FGasper left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

visemet left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

erreina Feb 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

autarch commented Feb 3, 2026 •

edited

Loading

autarch commented Feb 3, 2026 •

edited

Loading

erreina Feb 12, 2026 •

edited

Loading