Skip to content

fix: replace O(n) analytics full-scan with campaign-isolated pipeline - #153

Merged
BarryArinze merged 1 commit into
aid-linkk:masterfrom
JamesEjembi:fix/analytics-campaign-isolated-scan
Aug 26, 2026
Merged

fix: replace O(n) analytics full-scan with campaign-isolated pipeline#153
BarryArinze merged 1 commit into
aid-linkk:masterfrom
JamesEjembi:fix/analytics-campaign-isolated-scan

Conversation

@JamesEjembi

@JamesEjembi JamesEjembi commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Summary

Replaces the O(n·pages × m·records) analytics full-scan with a campaign-isolated, cursor-resumable, server-side aggregation pipeline.

Changes

  • Campaign-isolated fetching: Uses /transactions endpoint with memo filtering to only fetch records for the target campaign
  • Cursor-based resumption: Supports startCursor parameter for incremental fetches
  • Configurable limits: Adds maxRecords parameter (default 5000) to prevent unbounded scanning
  • Rate limiting: Implements 110ms delay between Horizon requests for compliance
  • Server-side API route: Creates /api/v1/analytics endpoint for pre-aggregated data
  • Direct memo access: Uses /transactions endpoint which includes memo directly, eliminating per-record fetches

Closes

Closes #140

Key Improvements

  • Before: Fetches ALL payments for the analytics account, then filters by campaign
  • After: Only fetches transactions with matching campaign memo prefix
  • Before: Per-record GET /transactions/:hash for memo lookup
  • After: Memo is available directly on /transactions response
  • Before: No cursor support, full rescan every 60s
  • After: Cursor-based resumption for incremental updates

@JamesEjembi
fix: replace O(n) analytics full-scan with campaign-isolated pipeline
fdfd680 
- Replace /payments endpoint with /transactions endpoint for direct memo access
- Implement cursor-based resumption for incremental fetches
- Add configurable maxRecords limit (default 5000)
- Add rate limiting (110ms between requests) for Horizon compliance
- Create server-side API route /api/v1/analytics for pre-aggregated data
- Maintain CampaignAnalytics interface compatibility
- Early exit when target campaign records are not found in page
- Proper deduplication using transaction_hash only

Closes aid-linkk#140
@BarryArinze
BarryArinze merged commit 746706c into aid-linkk:master Aug 26, 2026
2 of 4 checks passed
@grantfox-oss grantfox-oss Bot mentioned this pull request Aug 26, 2026
Closed
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Replace the O(n·pages × m·records) analytics full-scan with a campaign-isolated, cursor-resumable, server-side aggregation pipeline

2 participants

@JamesEjembi @BarryArinze