fix: rework Auth0 API calls to allow large numbers of users#190
Merged
marius-mather merged 18 commits intomainfrom Mar 4, 2026
Merged
fix: rework Auth0 API calls to allow large numbers of users#190marius-mather merged 18 commits intomainfrom
marius-mather merged 18 commits intomainfrom
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Updates Auth0 integration to support syncing more than 1000 users by switching role-user fetching to checkpoint-based iteration and switching full user sync to Auth0’s user export job flow.
Changes:
- Implement checkpoint-based iteration for fetching all users assigned to a role.
- Add Auth0 user export job creation/polling + CSV download/parsing, and update
sync_auth0_usersto use it. - Update/extend tests and introduce Faker as a dev dependency for data generation.
Reviewed changes
Copilot reviewed 6 out of 7 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
auth0/client.py |
Adds export job APIs + download logic; updates role-user retrieval to support checkpoints. |
scheduled_tasks/tasks.py |
Introduces ExportedUser, CSV parsing, and switches sync_auth0_users to export-based sync. |
tests/test_auth0_client.py |
Updates role-user pagination tests and adds a checkpoint-iteration test. |
tests/scheduled_tasks/test_tasks.py |
Updates sync test to mock exported-user flow instead of paged API calls. |
tests/datagen.py |
Adds ExportedUserFactory and introduces Faker usage. |
pyproject.toml |
Adds faker to dev dependencies. |
uv.lock |
Lockfile update for new dev dependency. |
uwwint
requested changes
Mar 4, 2026
uwwint
approved these changes
Mar 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
AAI-743: need to allow fetching more than 1000 users at a time to properly sync users into the backend from Auth0
Changes
get_all_role_usersto use checkpoint-based iteration, page-based only allows 1000 userssync_auth0_usersto use a user export, since otherwise we can't get more than 1000 usersChecklist
How to Test Manually (if necessary)
Run
uv run pytest