feat(enterprise): Implement cold storage archiving for read receipts#38623
feat(enterprise): Implement cold storage archiving for read receipts#38623Copilot wants to merge 20 commits intochore/new-data-migrationfrom
Conversation
|
Looks like this PR is not ready to merge, because of the following issues:
Please fix the issues and try again If you have any trouble, please check the PR guidelines |
|
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## chore/new-data-migration #38623 +/- ##
============================================================
+ Coverage 70.91% 70.98% +0.06%
============================================================
Files 3207 3201 -6
Lines 114120 113813 -307
Branches 20721 20638 -83
============================================================
- Hits 80932 80788 -144
+ Misses 31145 30976 -169
- Partials 2043 2049 +6
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
|
@copilot fix the CI issues |
The PR Title Checker is failing because the title has a typo: "code storage" should be "cold storage". The PR title needs to be updated through the GitHub UI to: |
|
@copilot fix code lint and type check which failed on CI |
Fixed TypeScript type errors in the
All methods now properly match the |
|
@copilot read the CI logs of "TypeScript" and "Code Lint" and fix the reported errors |
I'm unable to access the CI logs as the workflows are still in progress for commit 42ec30f. The TypeScript and Code Lint checks haven't completed yet. I've proactively verified my recent changes:
Once the CI completes, I can access the specific error messages and fix them. Alternatively, if you can share the specific errors from the CI logs, I can address them immediately. |
Co-authored-by: rodrigok <234261+rodrigok@users.noreply.github.com>
Co-authored-by: rodrigok <234261+rodrigok@users.noreply.github.com>
Co-authored-by: rodrigok <234261+rodrigok@users.noreply.github.com>
Co-authored-by: rodrigok <234261+rodrigok@users.noreply.github.com>
Co-authored-by: rodrigok <234261+rodrigok@users.noreply.github.com>
Co-authored-by: rodrigok <234261+rodrigok@users.noreply.github.com>
Co-authored-by: rodrigok <234261+rodrigok@users.noreply.github.com>
Co-authored-by: rodrigok <234261+rodrigok@users.noreply.github.com>
Co-authored-by: rodrigok <234261+rodrigok@users.noreply.github.com>
… delay Co-authored-by: rodrigok <234261+rodrigok@users.noreply.github.com>
Co-authored-by: rodrigok <234261+rodrigok@users.noreply.github.com>
Co-authored-by: rodrigok <234261+rodrigok@users.noreply.github.com>
…ondary collection Co-authored-by: rodrigok <234261+rodrigok@users.noreply.github.com>
Co-authored-by: rodrigok <234261+rodrigok@users.noreply.github.com>
… based on old extra data filters
92eece2 to
d18b09b
Compare
d18b09b to
0607593
Compare
|
@coderabbitai full review |
✅ Actions performedFull review triggered. |
WalkthroughThis PR implements a cold storage archiving system for read receipts in Enterprise edition. It introduces a Changes
Sequence DiagramsequenceDiagram
participant Cron as Cron Job
participant Settings as Settings
participant ReadReceipts as ReadReceipts
participant Archive as ReadReceiptsArchive
participant Messages as Messages
Cron->>Settings: Get retention days & batch size
activate Cron
loop Until no old receipts
Cron->>ReadReceipts: findOlderThan(cutoff date)
activate ReadReceipts
ReadReceipts-->>Cron: Old receipt documents
deactivate ReadReceipts
alt Receipts found
Cron->>Archive: insertMany(receipts)
activate Archive
Archive-->>Cron: Insert result
deactivate Archive
Cron->>Messages: updateMany(receiptsArchived: true)
Cron->>ReadReceipts: deleteMany(old receipts)
Cron->>Cron: Delay between batches
else No receipts
Cron->>Cron: Break loop
end
end
deactivate Cron
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Suggested labels
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. 📝 Coding Plan
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment Tip CodeRabbit can use Trivy to scan for security misconfigurations and secrets in Infrastructure as Code files.Add a .trivyignore file to your project to customize which findings Trivy reports. |
There was a problem hiding this comment.
Actionable comments posted: 6
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
apps/meteor/app/livechat/server/lib/rooms.ts (1)
287-305:⚠️ Potential issue | 🟠 Major
Promise.allSettledindex mapping broke after adding archive deletion.After Line 290 inserted a new operation, Lines 300 and 303 still use old indexes.
result[3]andresult[4]now point to different tasks, so notifications can be triggered from the wrong deletion outcomes.✅ Suggested fix (use named settled results)
const result = await Promise.allSettled([ Messages.removeByRoomId(rid), ReadReceipts.removeByRoomId(rid), ReadReceiptsArchive.removeByRoomId(rid), Subscriptions.removeByRoomId(rid, { async onTrash(doc) { void notifyOnSubscriptionChanged(doc, 'removed'); }, }), LivechatInquiry.removeByRoomId(rid), LivechatRooms.removeById(rid), ]); -if (result[3]?.status === 'fulfilled' && result[3].value?.deletedCount && inquiry) { +const [, , , subscriptionsResult, inquiryResult, roomResult] = result; + +if (inquiryResult?.status === 'fulfilled' && inquiryResult.value?.deletedCount && inquiry) { void notifyOnLivechatInquiryChanged(inquiry, 'removed'); } -if (result[4]?.status === 'fulfilled' && result[4].value?.deletedCount) { +if (roomResult?.status === 'fulfilled' && roomResult.value?.deletedCount) { void notifyOnRoomChanged(room, 'removed'); }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/meteor/app/livechat/server/lib/rooms.ts` around lines 287 - 305, The Promise.allSettled call added ReadReceiptsArchive.removeByRoomId which shifted positional indexes so result[3] and result[4] no longer refer to Subscriptions and LivechatInquiry outcomes; update the code to capture named settled results instead of relying on numeric indexes (e.g., const [messagesResult, readReceiptsResult, readReceiptsArchiveResult, subscriptionsResult, inquiryResult, livechatRoomsResult] = await Promise.allSettled([...]) ) and then use subscriptionsResult to drive notifyOnSubscriptionChanged (and its deletedCount check), inquiryResult to drive notifyOnLivechatInquiryChanged, and livechatRoomsResult to drive notifyOnRoomChanged; reference the existing functions/subroutines Messages.removeByRoomId, ReadReceipts.removeByRoomId, ReadReceiptsArchive.removeByRoomId, Subscriptions.removeByRoomId, LivechatInquiry.removeByRoomId, LivechatRooms.removeById, notifyOnSubscriptionChanged, notifyOnLivechatInquiryChanged, and notifyOnRoomChanged to locate the affected logic and update the conditional checks accordingly.
🧹 Nitpick comments (7)
apps/meteor/server/startup/dataMigrations/00001_disable-archive-for-large-read-receipts.ts (1)
8-8: Consider extracting the 1,000,000 threshold into a named constant.This avoids drift between the human-readable description and the runtime condition.
♻️ Suggested refactor
import { Settings, ReadReceipts } from '@rocket.chat/models'; import { addDataMigration } from '../../lib/dataMigrations'; +const MAX_READ_RECEIPTS_FOR_DEFAULT_ARCHIVE_ENABLE = 1_000_000; + addDataMigration({ order: 1, id: 'disable-archive-for-large-read-receipts', - description: 'Do not enable read receipts archive by default if there are more than 1 million records in the read receipts collection', + description: `Do not enable read receipts archive by default if there are more than ${MAX_READ_RECEIPTS_FOR_DEFAULT_ARCHIVE_ENABLE.toLocaleString()} records in the read receipts collection`, strategy: 'once', direction: 'upgrade', requiresManualReversion: false, async run() { const count = await ReadReceipts.col.estimatedDocumentCount(); - if (count <= 1_000_000) { + if (count <= MAX_READ_RECEIPTS_FOR_DEFAULT_ARCHIVE_ENABLE) { return; }Also applies to: 15-15
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/meteor/server/startup/dataMigrations/00001_disable-archive-for-large-read-receipts.ts` at line 8, Extract the magic number 1000000 into a named constant (e.g., READ_RECEIPTS_ARCHIVE_THRESHOLD) and use that constant both in the migration description string and in the runtime check that currently compares the read receipts count to 1000000; update both occurrences (the description line and the runtime condition) to reference the constant so the human-readable text and logic cannot drift apart (see the description: 'Do not enable read receipts archive by default if there are more than 1 million records...' and the runtime comparison in this migration file).apps/meteor/ee/server/startup/readReceiptsArchive.ts (1)
4-8: Remove inline implementation comments in this module.The comments at Lines 4 and 7 restate obvious behavior and can be dropped.
🧹 Suggested cleanup
-// Initialize the cron job void readReceiptsArchiveCron(); -// Watch for settings changes and update the cron schedule settings.watch<string>('Message_Read_Receipt_Archive_Cron', async (value) => {As per coding guidelines: "Avoid code comments in the implementation".
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/meteor/ee/server/startup/readReceiptsArchive.ts` around lines 4 - 8, Remove the redundant inline implementation comments that restate obvious behavior around the cron initialization and settings watcher; specifically delete the comment lines immediately above the call to readReceiptsArchiveCron() and above the settings.watch<string>('Message_Read_Receipt_Archive_Cron', ...) block so the file only contains the call to readReceiptsArchiveCron() and the settings.watch handler without those explanatory comments.apps/meteor/ee/server/cron/readReceiptsArchive.spec.ts (2)
63-65: Freeze time for the retention assertions.These tests derive the expected age from the current wall clock. Since fake timers are already enabled, set a fixed system time and assert the exact cutoff passed to
ReadReceipts.findOlderThan; that keeps the test deterministic and easier to reason about.Also applies to: 82-84
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/meteor/ee/server/cron/readReceiptsArchive.spec.ts` around lines 63 - 65, Tests rely on the real clock when asserting retention days; freeze system time with jest.setSystemTime(...) before triggering the code so the cutoff passed to ReadReceipts.findOlderThan is deterministic, then assert the exact cutoff Date (or its getTime()) equals new Date(Date.now() - 30 * 24 * 60 * 60 * 1000). Add matching fixes for the second assertion block (lines around the other ReadReceipts.findOlderThan call) and restore timers with jest.useRealTimers() or jest.restoreAllMocks() after the assertions. Ensure you reference the mocked call ReadReceipts.findOlderThan (mock.calls[0][0]) to compare against the fixed system time.
155-165: Assert the 1-second delay explicitly.
jest.runAllTimersAsync()will still make this pass if the inter-batch timeout is removed or changed. Advance the clock in smaller steps and assert that batch 2 does not start before the full 1000ms.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/meteor/ee/server/cron/readReceiptsArchive.spec.ts` around lines 155 - 165, The test for archiveOldReadReceipts should explicitly assert the 1s inter-batch delay by advancing timers in controlled steps: after starting archiveOldReadReceipts(), advance timers less than 1000ms (e.g., 500ms) and assert that ReadReceiptsArchive.insertMany / Messages.updateMany / ReadReceipts.deleteMany have not been called a second time, then advance the remaining time to reach 1000ms and assert the second batch calls occur; update the test around archivePromise and the jest timer calls to use jest.advanceTimersByTime with the incremental checks to ensure batch 2 doesn't start before 1000ms.apps/meteor/ee/server/lib/message-read-receipt/ReadReceipt.ts (1)
141-156: Please drop the step-by-step comments ingetReceipts.The method is short enough that these comments are only narrating obvious steps, and the repo guideline asks us to keep TS/JS implementation files comment-free.
As per coding guidelines, "Avoid code comments in the implementation."
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/meteor/ee/server/lib/message-read-receipt/ReadReceipt.ts` around lines 141 - 156, Remove the step-by-step inline comments inside the getReceipts implementation: delete the explanatory comment lines that precede the ReadReceipts.findByMessageId call, the conditional ReadReceiptsArchive query, the receipts combination, the userIds extraction, and the users lookup; leave the existing code and variable names (getReceipts, ReadReceipts, ReadReceiptsArchive, receipts, userIds) intact so behavior is unchanged but the method contains no narrative comments.apps/meteor/ee/server/models/raw/ReadReceiptsArchive.ts (1)
16-21: The interface mismatch should be solved in types, not comments.These comments are compensating for an overly broad
IReadReceiptsModelsurface and restating behavior the code already shows. If the archive backend only needs part of that contract, prefer splitting the interface and dropping the implementation comments here.As per coding guidelines, "Avoid code comments in the implementation."
Also applies to: 43-44
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/meteor/ee/server/models/raw/ReadReceiptsArchive.ts` around lines 16 - 21, The ReadReceiptsArchive class is implementing the broad IReadReceiptsModel but only needs a subset; instead of leaving explanatory comments, define a narrower interface (e.g., IReadReceiptsArchive or IReadReceiptsMinimal) that declares only the methods the archive actually requires (e.g., the findByMessageId/read method used in ReadReceiptsArchive), update the ReadReceiptsArchive class to implement that new interface, remove the no-op/delete method implementations and their comments, and update any call sites or DI/typing references that expected IReadReceiptsModel to use the new narrower interface where appropriate so the type system enforces the reduced surface instead of relying on comments.apps/meteor/ee/server/cron/readReceiptsArchive.ts (1)
10-10: Please remove the inline implementation comments from this cron.This new file is carrying step-by-step narration and inline ESLint suppressions for control flow that can be expressed directly in code. A small extraction/restructure here would keep the implementation cleaner and align it with the repo style.
As per coding guidelines, "Avoid code comments in the implementation."
Also applies to: 27-34, 43-54, 68-88, 101-101
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/meteor/ee/server/cron/readReceiptsArchive.ts` at line 10, Remove the inline narration comments and ESLint suppressions from the cron implementation and instead express the behavior via small, named helpers and constants: keep the BATCH_DELAY_MS constant but extract narrated blocks into functions like validateInputs(), fetchBatch(), processBatch(), and delay(ms) (or use an existing sleep helper) and call them from the main cron handler (readReceiptsArchiveCron / exported cron function); replace suppressed control-flow comments with explicit early returns, try/catch, and an awaited delay(BATCH_DELAY_MS) in the batch loop so the code reads clearly without inline implementation comments.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@apps/meteor/app/lib/server/functions/cleanRoomHistory.ts`:
- Around line 146-149: The cleanup currently only calls
ReadReceipts.removeByMessageIds and ReadReceiptsArchive.removeByMessageIds when
`limit && selectedMessageIds`, which leaves orphaned receipts in the
unlimited/delete-all path of `cleanRoomHistory()`; modify `cleanRoomHistory` so
that when the unlimited path is taken (limit is 0/undefined), you either
materialize the IDs of messages being deleted into `selectedMessageIds` before
removing messages, or perform a filter-based cleanup (e.g., call
ReadReceipts.removeByFilter and ReadReceiptsArchive.removeByFilter with the same
room/message predicates) so archived and hot read receipts are removed in both
the limited and unlimited flows.
In `@apps/meteor/ee/server/cron/readReceiptsArchive.ts`:
- Around line 52-59: The catch currently treats any MongoBulkWriteError as a
duplicate-key-only situation; update the handler around the insertMany call to
inspect mongoError.writeErrors and only treat it as benign if every entry in
writeErrors has code === 11000, otherwise rethrow or log and abort the batch;
use the existing mongoError.result?.insertedCount for the archived count, and
reference insertMany, mongoError, writeErrors, batchNumber and logger when
making this change so we only proceed with marking messages archived and
deleting hot receipts when all write errors are duplicate-key errors.
In `@apps/meteor/ee/server/lib/message-read-receipt/ReadReceipt.ts`:
- Around line 140-151: getReceipts currently concatenates hotReceipts and
coldReceipts and can return duplicates when the archiver leaves the same receipt
in both collections; deduplicate the combined array by receipt identity (e.g.,
_id or the tuple { messageId, userId }) before further processing so only one
receipt per read event is enriched and returned. Update the getReceipts
implementation in ReadReceipt.ts to merge hotReceipts and coldReceipts into a
single map keyed by the chosen unique key (use ReadReceipts and
ReadReceiptsArchive as sources and respect message.receiptsArchived), then
convert the map back to an array of IReadReceipt or IReadReceiptWithUser for the
existing enrichment flow. Ensure the dedupe key matches the rest of the code
that uses receipt identity (prefer _id if present, otherwise { messageId, userId
}).
In `@apps/meteor/ee/server/startup/readReceiptsArchive.ts`:
- Around line 8-11: The settings.watch callback currently only calls
readReceiptsArchiveCron() when the new value is truthy, which leaves the old
cron running if the setting is cleared; remove the truthy guard so the callback
always calls readReceiptsArchiveCron(value) (or call it unconditionally and let
it infer clearing) and update the readReceiptsArchiveCron function to accept the
new value (e.g., readReceiptsArchiveCron(schedule?: string)) and stop/unregister
the existing cron when the value is falsy, otherwise (re)configure the cron
according to the provided schedule.
In `@apps/meteor/server/settings/message.ts`:
- Around line 68-76: The setting 'Message_Read_Receipt_Archive_Enabled' is
currently defaulting to true causing automatic archive/backfill; change its
default to false to make the archive opt-in. Locate the call to this.add that
creates 'Message_Read_Receipt_Archive_Enabled' in
apps/meteor/server/settings/message.ts (the invocation with
modules:['message-read-receipt'] and enableQuery:{ _id:
'Message_Read_Receipt_Store_Users', value: true }) and update the default value
argument from true to false so workspaces must opt in before the
archive/backfill flow runs.
In `@packages/i18n/src/locales/pt.i18n.json`:
- Around line 1704-1712: The new translation keys
(Message_Read_Receipt_Archive_Enabled,
Message_Read_Receipt_Archive_Enabled_Description,
Message_Read_Receipt_Archive_Enabled_Alert,
Message_Read_Receipt_Archive_Retention_Days,
Message_Read_Receipt_Archive_Retention_Days_Description,
Message_Read_Receipt_Archive_Cron,
Message_Read_Receipt_Archive_Cron_Description,
Message_Read_Receipt_Archive_Batch_Size,
Message_Read_Receipt_Archive_Batch_Size_Description) should not be committed
directly into pt.i18n.json; remove these entries from
packages/i18n/src/locales/pt.i18n.json and instead add the new keys and English
texts to packages/i18n/src/locales/en.i18n.json so the translation
pipeline/fallback will propagate them to other locales.
---
Outside diff comments:
In `@apps/meteor/app/livechat/server/lib/rooms.ts`:
- Around line 287-305: The Promise.allSettled call added
ReadReceiptsArchive.removeByRoomId which shifted positional indexes so result[3]
and result[4] no longer refer to Subscriptions and LivechatInquiry outcomes;
update the code to capture named settled results instead of relying on numeric
indexes (e.g., const [messagesResult, readReceiptsResult,
readReceiptsArchiveResult, subscriptionsResult, inquiryResult,
livechatRoomsResult] = await Promise.allSettled([...]) ) and then use
subscriptionsResult to drive notifyOnSubscriptionChanged (and its deletedCount
check), inquiryResult to drive notifyOnLivechatInquiryChanged, and
livechatRoomsResult to drive notifyOnRoomChanged; reference the existing
functions/subroutines Messages.removeByRoomId, ReadReceipts.removeByRoomId,
ReadReceiptsArchive.removeByRoomId, Subscriptions.removeByRoomId,
LivechatInquiry.removeByRoomId, LivechatRooms.removeById,
notifyOnSubscriptionChanged, notifyOnLivechatInquiryChanged, and
notifyOnRoomChanged to locate the affected logic and update the conditional
checks accordingly.
---
Nitpick comments:
In `@apps/meteor/ee/server/cron/readReceiptsArchive.spec.ts`:
- Around line 63-65: Tests rely on the real clock when asserting retention days;
freeze system time with jest.setSystemTime(...) before triggering the code so
the cutoff passed to ReadReceipts.findOlderThan is deterministic, then assert
the exact cutoff Date (or its getTime()) equals new Date(Date.now() - 30 * 24 *
60 * 60 * 1000). Add matching fixes for the second assertion block (lines around
the other ReadReceipts.findOlderThan call) and restore timers with
jest.useRealTimers() or jest.restoreAllMocks() after the assertions. Ensure you
reference the mocked call ReadReceipts.findOlderThan (mock.calls[0][0]) to
compare against the fixed system time.
- Around line 155-165: The test for archiveOldReadReceipts should explicitly
assert the 1s inter-batch delay by advancing timers in controlled steps: after
starting archiveOldReadReceipts(), advance timers less than 1000ms (e.g., 500ms)
and assert that ReadReceiptsArchive.insertMany / Messages.updateMany /
ReadReceipts.deleteMany have not been called a second time, then advance the
remaining time to reach 1000ms and assert the second batch calls occur; update
the test around archivePromise and the jest timer calls to use
jest.advanceTimersByTime with the incremental checks to ensure batch 2 doesn't
start before 1000ms.
In `@apps/meteor/ee/server/cron/readReceiptsArchive.ts`:
- Line 10: Remove the inline narration comments and ESLint suppressions from the
cron implementation and instead express the behavior via small, named helpers
and constants: keep the BATCH_DELAY_MS constant but extract narrated blocks into
functions like validateInputs(), fetchBatch(), processBatch(), and delay(ms) (or
use an existing sleep helper) and call them from the main cron handler
(readReceiptsArchiveCron / exported cron function); replace suppressed
control-flow comments with explicit early returns, try/catch, and an awaited
delay(BATCH_DELAY_MS) in the batch loop so the code reads clearly without inline
implementation comments.
In `@apps/meteor/ee/server/lib/message-read-receipt/ReadReceipt.ts`:
- Around line 141-156: Remove the step-by-step inline comments inside the
getReceipts implementation: delete the explanatory comment lines that precede
the ReadReceipts.findByMessageId call, the conditional ReadReceiptsArchive
query, the receipts combination, the userIds extraction, and the users lookup;
leave the existing code and variable names (getReceipts, ReadReceipts,
ReadReceiptsArchive, receipts, userIds) intact so behavior is unchanged but the
method contains no narrative comments.
In `@apps/meteor/ee/server/models/raw/ReadReceiptsArchive.ts`:
- Around line 16-21: The ReadReceiptsArchive class is implementing the broad
IReadReceiptsModel but only needs a subset; instead of leaving explanatory
comments, define a narrower interface (e.g., IReadReceiptsArchive or
IReadReceiptsMinimal) that declares only the methods the archive actually
requires (e.g., the findByMessageId/read method used in ReadReceiptsArchive),
update the ReadReceiptsArchive class to implement that new interface, remove the
no-op/delete method implementations and their comments, and update any call
sites or DI/typing references that expected IReadReceiptsModel to use the new
narrower interface where appropriate so the type system enforces the reduced
surface instead of relying on comments.
In `@apps/meteor/ee/server/startup/readReceiptsArchive.ts`:
- Around line 4-8: Remove the redundant inline implementation comments that
restate obvious behavior around the cron initialization and settings watcher;
specifically delete the comment lines immediately above the call to
readReceiptsArchiveCron() and above the
settings.watch<string>('Message_Read_Receipt_Archive_Cron', ...) block so the
file only contains the call to readReceiptsArchiveCron() and the settings.watch
handler without those explanatory comments.
In
`@apps/meteor/server/startup/dataMigrations/00001_disable-archive-for-large-read-receipts.ts`:
- Line 8: Extract the magic number 1000000 into a named constant (e.g.,
READ_RECEIPTS_ARCHIVE_THRESHOLD) and use that constant both in the migration
description string and in the runtime check that currently compares the read
receipts count to 1000000; update both occurrences (the description line and the
runtime condition) to reference the constant so the human-readable text and
logic cannot drift apart (see the description: 'Do not enable read receipts
archive by default if there are more than 1 million records...' and the runtime
comparison in this migration file).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: a3d920df-cf09-4d10-9ba7-3afb5ccaf99e
📒 Files selected for processing (36)
apps/meteor/app/lib/server/functions/cleanRoomHistory.tsapps/meteor/app/lib/server/functions/deleteMessage.tsapps/meteor/app/lib/server/functions/deleteUser.tsapps/meteor/app/lib/server/functions/relinquishRoomOwnerships.tsapps/meteor/app/livechat/server/lib/guests.tsapps/meteor/app/livechat/server/lib/rooms.tsapps/meteor/app/message-pin/server/pinMessage.tsapps/meteor/app/slackbridge/server/SlackAdapter.tsapps/meteor/app/threads/server/functions.tsapps/meteor/definition/IRoomTypeConfig.tsapps/meteor/ee/app/message-read-receipt/server/hooks/afterDeleteRoom.tsapps/meteor/ee/server/cron/readReceiptsArchive.spec.tsapps/meteor/ee/server/cron/readReceiptsArchive.tsapps/meteor/ee/server/lib/message-read-receipt/ReadReceipt.tsapps/meteor/ee/server/methods/getReadReceipts.tsapps/meteor/ee/server/models/ReadReceiptsArchive.tsapps/meteor/ee/server/models/raw/ReadReceipts.tsapps/meteor/ee/server/models/raw/ReadReceiptsArchive.tsapps/meteor/ee/server/models/startup.tsapps/meteor/ee/server/startup/index.tsapps/meteor/ee/server/startup/readReceiptsArchive.tsapps/meteor/jest.config.tsapps/meteor/server/lib/moderation/deleteReportedMessages.tsapps/meteor/server/lib/rooms/roomCoordinator.tsapps/meteor/server/lib/rooms/roomTypes/livechat.tsapps/meteor/server/settings/message.tsapps/meteor/server/startup/dataMigrations/00001_disable-archive-for-large-read-receipts.tsapps/meteor/server/startup/dataMigrations/index.tspackages/core-typings/src/IMessage/IMessage.tspackages/core-typings/src/IReadReceipt.tspackages/i18n/src/locales/en.i18n.jsonpackages/i18n/src/locales/pt-BR.i18n.jsonpackages/i18n/src/locales/pt.i18n.jsonpackages/model-typings/src/models/IReadReceiptsModel.tspackages/models/src/dummy/ReadReceipts.tspackages/models/src/index.ts
💤 Files with no reviewable changes (1)
- apps/meteor/server/lib/rooms/roomTypes/livechat.ts
| if (limit && selectedMessageIds) { | ||
| await ReadReceipts.removeByMessageIds(selectedMessageIds); | ||
| await ReadReceiptsArchive.removeByMessageIds(selectedMessageIds); | ||
| } |
There was a problem hiding this comment.
Handle the unlimited prune path too.
Line 146 gates receipt cleanup on limit. cleanRoomHistory() also supports deleting every matching message when limit is 0/unset, and in that path both read-receipt collections keep orphaned rows for the removed messages. Please delete archived/hot receipts for the unlimited path as well, either by materializing the IDs before the delete or by adding a filter-based cleanup.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@apps/meteor/app/lib/server/functions/cleanRoomHistory.ts` around lines 146 -
149, The cleanup currently only calls ReadReceipts.removeByMessageIds and
ReadReceiptsArchive.removeByMessageIds when `limit && selectedMessageIds`, which
leaves orphaned receipts in the unlimited/delete-all path of
`cleanRoomHistory()`; modify `cleanRoomHistory` so that when the unlimited path
is taken (limit is 0/undefined), you either materialize the IDs of messages
being deleted into `selectedMessageIds` before removing messages, or perform a
filter-based cleanup (e.g., call ReadReceipts.removeByFilter and
ReadReceiptsArchive.removeByFilter with the same room/message predicates) so
archived and hot read receipts are removed in both the limited and unlimited
flows.
| } catch (error: unknown) { | ||
| // If we get duplicate key errors, some receipts were already archived, which is fine | ||
| // We'll continue to mark messages and delete from hot storage | ||
| if (error && typeof error === 'object' && ('code' in error || 'name' in error)) { | ||
| const mongoError = error as { code?: number; name?: string; result?: { insertedCount?: number } }; | ||
| if (mongoError.code === 11000 || mongoError.name === 'MongoBulkWriteError') { | ||
| const insertedCount = mongoError.result?.insertedCount || 0; | ||
| logger.info(`Archived ${insertedCount} read receipts in batch ${batchNumber} (some were already archived)`); |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
cat apps/meteor/ee/server/cron/readReceiptsArchive.ts | head -120Repository: RocketChat/Rocket.Chat
Length of output: 4844
🌐 Web query:
MongoDB MongoBulkWriteError structure error codes what are the error types
💡 Result:
MongoBulkWriteError (Node.js driver) — structure / “error types”
When collection.bulkWrite() has failures, the Node.js driver throws MongoBulkWriteError and includes:
message(string)writeErrors: WriteError[](per-operation/server write errors)writeConcernErrors(write concern–related errors during the bulk execution)result(theBulkWriteResultfor the ops that succeeded before the error)err(underlying error object, often with more detail) [1]
At the API/class level, MongoBulkWriteError extends MongoServerError and commonly exposes code, codeName, writeErrors, and result (plus standard server-error fields like error labels, etc.) [2]
The two main “error types” you handle are:
-
Write errors (
writeErrors): one entry per failed operation in the batch. EachWriteErrorincludes at least:index(which operation in your input array failed)code(MongoDB server error code)errmsg(message)errInfo(optional details)op(the operation/document that caused the error) [3]
This aligns with the server write-command error-document pattern (index,code,errmsg, …). [4]
-
Write concern errors (
writeConcernErrors/WriteConcernError): errors where the write may have been applied but the server could not satisfy the requested write concern (e.g.,w: "majority"can’t be met in time). These expose:code/errmsgerrInfo(optional) [5]
Error codes: what they mean
MongoBulkWriteError does not define its own special code list—the code you see in writeErrors[i].code (and in write concern errors) is a standard MongoDB server error code (e.g., 11000 duplicate key, 121 document validation failure, etc.). You interpret codes using MongoDB’s server error codes, and the failing operation is identified via writeErrors[i].index (and often writeErrors[i].op). [3][4]
Related (newer) bulk-write error class
For client-level bulk writes (client.bulkWrite()), the driver uses MongoClientBulkWriteError with a different shape (notably a Map of per-op errors and a partialResult). [1]
🏁 Script executed:
sed -n '40,90p' apps/meteor/ee/server/cron/readReceiptsArchive.tsRepository: RocketChat/Rocket.Chat
Length of output: 2608
🏁 Script executed:
sed -n '1,120p' apps/meteor/ee/server/cron/readReceiptsArchive.ts | grep -n '//'Repository: RocketChat/Rocket.Chat
Length of output: 1148
Don't treat every MongoBulkWriteError as a duplicate-key batch.
When insertMany(…, { ordered: false }) throws MongoBulkWriteError, it means at least one write failed—but not necessarily due to duplicate keys. The error name alone does not indicate the cause. A non-duplicate failure (validation error, permission error, etc.) will still match mongoError.name === 'MongoBulkWriteError' and allow the code to proceed with marking messages archived and deleting hot receipts, even though some inserts were not persisted. Only continue when every write error in the writeErrors array has code === 11000.
🛡️ Safer duplicate-only handling
- if (error && typeof error === 'object' && ('code' in error || 'name' in error)) {
- const mongoError = error as { code?: number; name?: string; result?: { insertedCount?: number } };
- if (mongoError.code === 11000 || mongoError.name === 'MongoBulkWriteError') {
+ if (error && typeof error === 'object' && ('code' in error || 'writeErrors' in error)) {
+ const mongoError = error as {
+ code?: number;
+ result?: { insertedCount?: number };
+ writeErrors?: Array<{ code?: number }>;
+ };
+ const onlyDuplicateErrors = mongoError.writeErrors?.length
+ ? mongoError.writeErrors.every((writeError) => writeError.code === 11000)
+ : mongoError.code === 11000;
+
+ if (onlyDuplicateErrors) {
const insertedCount = mongoError.result?.insertedCount || 0;
logger.info(`Archived ${insertedCount} read receipts in batch ${batchNumber} (some were already archived)`);📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| } catch (error: unknown) { | |
| // If we get duplicate key errors, some receipts were already archived, which is fine | |
| // We'll continue to mark messages and delete from hot storage | |
| if (error && typeof error === 'object' && ('code' in error || 'name' in error)) { | |
| const mongoError = error as { code?: number; name?: string; result?: { insertedCount?: number } }; | |
| if (mongoError.code === 11000 || mongoError.name === 'MongoBulkWriteError') { | |
| const insertedCount = mongoError.result?.insertedCount || 0; | |
| logger.info(`Archived ${insertedCount} read receipts in batch ${batchNumber} (some were already archived)`); | |
| } catch (error: unknown) { | |
| // If we get duplicate key errors, some receipts were already archived, which is fine | |
| // We'll continue to mark messages and delete from hot storage | |
| if (error && typeof error === 'object' && ('code' in error || 'writeErrors' in error)) { | |
| const mongoError = error as { | |
| code?: number; | |
| result?: { insertedCount?: number }; | |
| writeErrors?: Array<{ code?: number }>; | |
| }; | |
| const onlyDuplicateErrors = mongoError.writeErrors?.length | |
| ? mongoError.writeErrors.every((writeError) => writeError.code === 11000) | |
| : mongoError.code === 11000; | |
| if (onlyDuplicateErrors) { | |
| const insertedCount = mongoError.result?.insertedCount || 0; | |
| logger.info(`Archived ${insertedCount} read receipts in batch ${batchNumber} (some were already archived)`); |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@apps/meteor/ee/server/cron/readReceiptsArchive.ts` around lines 52 - 59, The
catch currently treats any MongoBulkWriteError as a duplicate-key-only
situation; update the handler around the insertMany call to inspect
mongoError.writeErrors and only treat it as benign if every entry in writeErrors
has code === 11000, otherwise rethrow or log and abort the batch; use the
existing mongoError.result?.insertedCount for the archived count, and reference
insertMany, mongoError, writeErrors, batchNumber and logger when making this
change so we only proceed with marking messages archived and deleting hot
receipts when all write errors are duplicate-key errors.
| async getReceipts(message: Pick<IMessage, '_id' | 'receiptsArchived'>): Promise<IReadReceiptWithUser[]> { | ||
| // Query hot storage (always) | ||
| const hotReceipts = await ReadReceipts.findByMessageId(message._id).toArray(); | ||
|
|
||
| // Query cold storage only if message has archived receipts | ||
| let coldReceipts: IReadReceipt[] = []; | ||
| if (message.receiptsArchived) { | ||
| coldReceipts = await ReadReceiptsArchive.findByMessageId(message._id).toArray(); | ||
| } | ||
|
|
||
| // Combine receipts from both storages | ||
| const receipts = [...hotReceipts, ...coldReceipts]; |
There was a problem hiding this comment.
De-duplicate hot and cold receipts before enriching them.
The archiver can legitimately leave the same receipt in both collections for a while: it sets receiptsArchived before hot deletes complete, and it also tolerates duplicate-key replays. Concatenating the arrays here will surface duplicate receipts to clients during that window. Collapse by _id (or { messageId, userId }) before building the user map.
♻️ Suggested change
- const receipts = [...hotReceipts, ...coldReceipts];
+ const receipts = [...new Map([...hotReceipts, ...coldReceipts].map((receipt) => [receipt._id, receipt])).values()];📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| async getReceipts(message: Pick<IMessage, '_id' | 'receiptsArchived'>): Promise<IReadReceiptWithUser[]> { | |
| // Query hot storage (always) | |
| const hotReceipts = await ReadReceipts.findByMessageId(message._id).toArray(); | |
| // Query cold storage only if message has archived receipts | |
| let coldReceipts: IReadReceipt[] = []; | |
| if (message.receiptsArchived) { | |
| coldReceipts = await ReadReceiptsArchive.findByMessageId(message._id).toArray(); | |
| } | |
| // Combine receipts from both storages | |
| const receipts = [...hotReceipts, ...coldReceipts]; | |
| async getReceipts(message: Pick<IMessage, '_id' | 'receiptsArchived'>): Promise<IReadReceiptWithUser[]> { | |
| // Query hot storage (always) | |
| const hotReceipts = await ReadReceipts.findByMessageId(message._id).toArray(); | |
| // Query cold storage only if message has archived receipts | |
| let coldReceipts: IReadReceiptArchived[] = []; | |
| if (message.receiptsArchived) { | |
| coldReceipts = await ReadReceiptsArchive.findByMessageId(message._id).toArray(); | |
| } | |
| // Combine receipts from both storages | |
| const receipts = [...new Map([...hotReceipts, ...coldReceipts].map((receipt) => [receipt._id, receipt])).values()]; |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@apps/meteor/ee/server/lib/message-read-receipt/ReadReceipt.ts` around lines
140 - 151, getReceipts currently concatenates hotReceipts and coldReceipts and
can return duplicates when the archiver leaves the same receipt in both
collections; deduplicate the combined array by receipt identity (e.g., _id or
the tuple { messageId, userId }) before further processing so only one receipt
per read event is enriched and returned. Update the getReceipts implementation
in ReadReceipt.ts to merge hotReceipts and coldReceipts into a single map keyed
by the chosen unique key (use ReadReceipts and ReadReceiptsArchive as sources
and respect message.receiptsArchived), then convert the map back to an array of
IReadReceipt or IReadReceiptWithUser for the existing enrichment flow. Ensure
the dedupe key matches the rest of the code that uses receipt identity (prefer
_id if present, otherwise { messageId, userId }).
| settings.watch<string>('Message_Read_Receipt_Archive_Cron', async (value) => { | ||
| if (value) { | ||
| await readReceiptsArchiveCron(); | ||
| } |
There was a problem hiding this comment.
Reconfigure the cron job on every cron-setting change.
At Line 9, the truthy guard can skip reconfiguration when the setting is cleared/invalidated, leaving the previous schedule active.
💡 Suggested fix
-settings.watch<string>('Message_Read_Receipt_Archive_Cron', async (value) => {
- if (value) {
- await readReceiptsArchiveCron();
- }
+settings.watch<string>('Message_Read_Receipt_Archive_Cron', async () => {
+ await readReceiptsArchiveCron();
});📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| settings.watch<string>('Message_Read_Receipt_Archive_Cron', async (value) => { | |
| if (value) { | |
| await readReceiptsArchiveCron(); | |
| } | |
| settings.watch<string>('Message_Read_Receipt_Archive_Cron', async () => { | |
| await readReceiptsArchiveCron(); | |
| }); |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@apps/meteor/ee/server/startup/readReceiptsArchive.ts` around lines 8 - 11,
The settings.watch callback currently only calls readReceiptsArchiveCron() when
the new value is truthy, which leaves the old cron running if the setting is
cleared; remove the truthy guard so the callback always calls
readReceiptsArchiveCron(value) (or call it unconditionally and let it infer
clearing) and update the readReceiptsArchiveCron function to accept the new
value (e.g., readReceiptsArchiveCron(schedule?: string)) and stop/unregister the
existing cron when the value is falsy, otherwise (re)configure the cron
according to the provided schedule.
| await this.add('Message_Read_Receipt_Archive_Enabled', true, { | ||
| type: 'boolean', | ||
| enterprise: true, | ||
| invalidValue: false, | ||
| modules: ['message-read-receipt'], | ||
| i18nDescription: 'Message_Read_Receipt_Archive_Enabled_Description', | ||
| alert: 'Message_Read_Receipt_Archive_Enabled_Alert', | ||
| enableQuery: { _id: 'Message_Read_Receipt_Store_Users', value: true }, | ||
| }); |
There was a problem hiding this comment.
Make the archive toggle opt-in.
Line 68 defaults Message_Read_Receipt_Archive_Enabled to true. Any workspace that already has detailed read receipts enabled will start the archive/backfill flow immediately after upgrade, which bypasses the warning path and can trigger a long migration unexpectedly.
Suggested change
- await this.add('Message_Read_Receipt_Archive_Enabled', true, {
+ await this.add('Message_Read_Receipt_Archive_Enabled', false, {📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| await this.add('Message_Read_Receipt_Archive_Enabled', true, { | |
| type: 'boolean', | |
| enterprise: true, | |
| invalidValue: false, | |
| modules: ['message-read-receipt'], | |
| i18nDescription: 'Message_Read_Receipt_Archive_Enabled_Description', | |
| alert: 'Message_Read_Receipt_Archive_Enabled_Alert', | |
| enableQuery: { _id: 'Message_Read_Receipt_Store_Users', value: true }, | |
| }); | |
| await this.add('Message_Read_Receipt_Archive_Enabled', false, { | |
| type: 'boolean', | |
| enterprise: true, | |
| invalidValue: false, | |
| modules: ['message-read-receipt'], | |
| i18nDescription: 'Message_Read_Receipt_Archive_Enabled_Description', | |
| alert: 'Message_Read_Receipt_Archive_Enabled_Alert', | |
| enableQuery: { _id: 'Message_Read_Receipt_Store_Users', value: true }, | |
| }); |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@apps/meteor/server/settings/message.ts` around lines 68 - 76, The setting
'Message_Read_Receipt_Archive_Enabled' is currently defaulting to true causing
automatic archive/backfill; change its default to false to make the archive
opt-in. Locate the call to this.add that creates
'Message_Read_Receipt_Archive_Enabled' in apps/meteor/server/settings/message.ts
(the invocation with modules:['message-read-receipt'] and enableQuery:{ _id:
'Message_Read_Receipt_Store_Users', value: true }) and update the default value
argument from true to false so workspaces must opt in before the
archive/backfill flow runs.
| "Message_Read_Receipt_Archive_Enabled": "Ativar Arquivo de Recibos de Leitura", | ||
| "Message_Read_Receipt_Archive_Enabled_Description": "Quando ativado, os recibos de leitura mais antigos que o período de retenção serão movidos do armazenamento ativo para o armazenamento frio.", | ||
| "Message_Read_Receipt_Archive_Enabled_Alert": "Atenção: Ativar esta opção num workspace com grande volume de recibos de leitura pode fazer com que o sistema processe a migração para o armazenamento frio por um período muito prolongado, afetando o desempenho em produção. É aconselhável limpar os dados de recibos de leitura ou migrá-los manualmente antes de ativar esta configuração.", | ||
| "Message_Read_Receipt_Archive_Retention_Days": "Dias de Retenção no Arquivo", | ||
| "Message_Read_Receipt_Archive_Retention_Days_Description": "Número de dias para manter os recibos de leitura no armazenamento ativo antes de arquivar no armazenamento frio", | ||
| "Message_Read_Receipt_Archive_Cron": "Agendamento do Arquivo", | ||
| "Message_Read_Receipt_Archive_Cron_Description": "Expressão cron para o agendamento do arquivamento (por exemplo, '0 2 * * *' para diariamente às 2h da manhã)", | ||
| "Message_Read_Receipt_Archive_Batch_Size": "Tamanho do Lote de Arquivo", | ||
| "Message_Read_Receipt_Archive_Batch_Size_Description": "Número de recibos de leitura a processar por lote durante o arquivamento. Valores mais baixos reduzem a carga do servidor, mas levam mais tempo para concluir.", |
There was a problem hiding this comment.
Keep the new keys in en.i18n.json only.
These pt.i18n.json additions should not be committed directly; this repo backfills non-English locales through the translation pipeline / English fallback, so manual edits here will drift or be overwritten.
Based on learnings, new translation keys should be added to packages/i18n/src/locales/en.i18n.json only; other locale files are populated via the external translation pipeline and/or fall back to English.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/i18n/src/locales/pt.i18n.json` around lines 1704 - 1712, The new
translation keys (Message_Read_Receipt_Archive_Enabled,
Message_Read_Receipt_Archive_Enabled_Description,
Message_Read_Receipt_Archive_Enabled_Alert,
Message_Read_Receipt_Archive_Retention_Days,
Message_Read_Receipt_Archive_Retention_Days_Description,
Message_Read_Receipt_Archive_Cron,
Message_Read_Receipt_Archive_Cron_Description,
Message_Read_Receipt_Archive_Batch_Size,
Message_Read_Receipt_Archive_Batch_Size_Description) should not be committed
directly into pt.i18n.json; remove these entries from
packages/i18n/src/locales/pt.i18n.json and instead add the new keys and English
texts to packages/i18n/src/locales/en.i18n.json so the translation
pipeline/fallback will propagate them to other locales.
https://rocketchat.atlassian.net/browse/CORE-1850
Depends on #39578
Implements automatic archiving of read receipts older than a configurable retention period to a separate collection, reducing hot storage size and query load.
Architecture
Collections:
read_receipts- Hot storage (recent receipts)read_receipts_archive- Cold storage (archived receipts)Message Flag:
receiptsArchived: trueset on messages with archived receiptsArchive Model:
readPreference: SECONDARY_PREFERREDfor all readsIReadReceiptsModelinterface for consistencyArchiving Process
Batch Processing:
Workflow:
Cron Job:
Settings
All settings are enterprise-gated and require
message-read-receiptmodule:Message_Read_Receipt_Archive_Retention_Days(int, default: 30)Message_Read_Receipt_Archive_Cron(string, default: '0 2 * * *')Message_Read_Receipt_Archive_Batch_Size(int, default: 10000)Includes EN, PT, PT-BR translations.
Read Receipt Retrieval
Deletion Consistency
All deletion operations (by user, room, message) updated to remove from both hot and archive collections.
Testing
8 unit tests covering:
💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.
Summary by CodeRabbit
New Features
Chores