Skip to content

fix: filter learner search by requested usernames instead of scanning all users - #61

Merged
Alam-2U merged 1 commit into
release-ulmofrom
LP-956
Jul 21, 2026
Merged

fix: filter learner search by requested usernames instead of scanning all users#61
Alam-2U merged 1 commit into
release-ulmofrom
LP-956

Conversation

@Alam-2U

@Alam-2U Alam-2U commented Jul 20, 2026

Copy link
Copy Markdown

Description

The _get_stats_for_usernames function in forum/api/users.py calls backend.get_users() with no arguments, triggering a full collection scan on the users collection for every activity_stats request.

This means viewing the Learners tab for a single course — or searching for a single learner — causes the database to iterate over millions of user documents across all courses, only to discard them via Python-side filtering.

Impact

  • 500 errors on the Learners tab — the full scan hits a legacy document containing a corrupt byte (0xff at position 24) that PyMongo's UTF-8 decoder cannot handle, raising InvalidBSON
  • Severe performance degradation — every Learners tab page load or search queries millions of documents instead of the 1–25 actually needed

Fix

Pass the usernames filter to backend.get_users() so the filtering happens at the database query level instead of in Python.

Ticket

LP-956

@Alam-2U Alam-2U changed the title fix: filter learner search by requested usernames instead of scanning… fix: filter learner search by requested usernames instead of scanning all users Jul 21, 2026
@Alam-2U
Alam-2U requested a review from Copilot July 21, 2026 06:08

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR optimizes the learner stats lookup path by pushing username filtering down into the database query (MongoDB/MySQL) instead of fetching all users and filtering in Python, preventing expensive full-collection scans and related failures.

Changes:

  • Updated _get_stats_for_usernames to call backend.get_users(username__in=...) and removed Python-side filtering.
  • Added username__in support to both MongoDB and MySQL backend get_users() implementations by translating it to the respective query syntax.
  • Bumped the package version from 0.6.8 to 0.6.9.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
forum/backends/mysql/api.py Maps username__in to Django ORM user__username__in for DB-level filtering.
forum/backends/mongodb/api.py Maps username__in to MongoDB { username: { $in: [...] } } filtering.
forum/api/users.py Uses backend filtering for username-specific stats requests and removes Python filtering.
forum/init.py Version bump to reflect the change set.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread forum/api/users.py
@Alam-2U
Alam-2U merged commit 5475de8 into release-ulmo Jul 21, 2026
13 checks passed
@Alam-2U
Alam-2U deleted the LP-956 branch July 21, 2026 06:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants