feat(telemetry): implement Android-native telemetry collection and TUS upload pipeline

[DELTA-45-G]

Description

This PR introduces a robust, decoupled telemetry system for the Android Registration Client, mirroring the desktop architecture. It establishes a secure, efficient bridge between Flutter and the Android native layer to collect performance metrics and user behavior data based on issue #719 and ensures reliable background transmission to the our created TUSD ingestion server .

Implementation Phases

This implementation follows a structured approach to maintain performance and modularity:

[Phase 1] Pigeon Bridge: Defined a strict TelemetryApi contract for type-safe asynchronous communication between the Flutter UI and Android native layer.
[Phase 2] Native Collection: Implemented AndroidMetricCollector to handle file I/O operations on a dedicated background thread, ensuring thread-safe JSON-line logging with automatic 5MB log rotation.
[Phase 3] Host Integration: Integrated the TelemetryHandler into HostApiModule.java to register the telemetry service within the Flutter Engine.
[Phase 4] Background Upload: Leveraged WorkManager via a new TelemetryUploadWorker to perform periodic, resumable uploads of the metrics.log file using the TUS protocol, ensuring network-efficient background data transmission.
[Phase 5] Flutter Service Layer: Created TelemetryService to standardize event structure and provide a clean API for UI components to trigger telemetry logging.

Key Technical Highlights

Performance: All file I/O and network transmission occur off the main UI thread to prevent frame drops.
Reliability: Implemented resumable uploads using the TUS protocol; local log files are cleared only upon successful transmission confirmation.
Maintenance: Utilized the Pigeon generated-code pattern to minimize boilerplate and enforce type safety across layers.
Observability: Integrated with existing Android WorkManager infrastructure, reusing existing scheduler patterns to minimize resource footprint.

Verification & Testing

Verified local metrics writing via logcat and file system inspection.
Validated end-to-end flow using the existing TUSD ingestion backend.
Verified background job execution consistency via WorkManager.

Associated Issue

#719

Thanks
Team TuluTech

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (2)

• develop
• release-*

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 502d8bbd-572c-48f6-ab36-b4f537a544c0

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[yakshithkd23]

@SachinPremkumar Phase 1 implemented and verified. Testing passed—everything is working fine

[DELTA-45-G]

Hi @SachinPremkumar and @Pragya279 ,

We are happy to report that Phase 2 (Native Telemetry Collection) is successfully implemented and verified.

Key updates:

Schema Alignment: The Android telemetry collection now produces JSON logs identical to the Desktop Registration Client (Outer Envelope + Inner Message structure). This ensures seamless integration with our existing backend ingestion (Vector/Kafka) pipeline without any modifications.

Production Ready: We have implemented system-level metric tracking (Memory, Battery, Uptime) using thread-safe, non-blocking background I/O and automatic log rotation (5MB limit).

Verification:
We have attached the captured telemetry logs from the device, confirming the structure matches the expected backend schema exactly.

We are now proceeding with Phase 3 (HostApiModule integration) to wire these native collectors to the Flutter UI events.

Regards,
Team TuluTech

[yakshithkd23]

Phase 3 Complete — Pigeon Bridge Active

Hi @SachinPremkumar

Phase 3 (Host Integration) is verified and complete.

What we changed:

• We activated collectAndLogSystemMetrics() in configureFlutterEngine().
• The Pigeon bridge is now live: Flutter → TelemetryApiImpl → AndroidMetricCollector → metrics.log.

Verification:

We have confirmed the end-to-end flow by checking the generated metrics.log and validating the logcat output. All system metrics (uptime, memory, battery) are being captured correctly with the desktop-aligned schema, and the bridge test event was successfully logged.

File changed:

• MainActivity.java — uncommented collectAndLogSystemMetrics()

Pipeline status:

Phase 1 ✅ Pigeon contract
Phase 2 ✅ Native collection + schema aligned
Phase 3 ✅ Host integration verified
Phase 4 ⏳ TUS upload worker — next
Phase 5 ⏳ UI event triggers — upcoming

Regards,
Team TuluTech

[yakshithkd23]

---

✅ Phase 4 Complete — Full End-to-End Pipeline Verified

Hi @SachinPremkumar

Phase 4 (Background TUS Upload) is implemented and fully verified end-to-end.

What was implemented:

• TelemetryUploadWorker using WorkManager for periodic background upload
• TUS resumable protocol — survives network drops and app restarts
• TusPreferencesURLStore — persists upload URL across crashes
• metrics.log deleted only after confirmed upload success
• Result.retry() on failure — WorkManager handles automatic backoff
• Runs every 15 minutes with CONNECTED network constraint
• Matches desktop TelemetryUploadService behavior exactly

Verified End-to-End Pipeline:

Android Device
    → metrics.log (AndroidMetricCollector)
    → TUS Upload (TelemetryUploadWorker)
    → TUSD Server (mosip-tusd:8080)
    → Vector Sidecar (file → Kafka)
    → Kafka Topic: registration-client-telemetry

Verification Screenshots:

Docker Stack — TUSD, Vector, Kafka, Zookeeper all healthy

TUSD Container — metrics.log received and stored successfully

Kafka Consumer — all metrics flowing through topic: registration-client-telemetry

Files changed:

• telemetry/TelemetryUploadWorker.java — new
• android/app/build.gradle — tus-android-client dependency added
• MainActivity.java — WorkManager schedule block added

Full pipeline status:

Phase 1 ✅ Pigeon contract defined
Phase 2 ✅ Native collection + schema aligned to desktop
Phase 3 ✅ Host integration + Pigeon bridge verified
Phase 4 ✅ TUS upload → TUSD → Vector → Kafka verified
Phase 5 ⏳ Real UI event triggers — next

Proceeding to Phase 5 (Flutter UI event triggers).

Regards,

Team TuluTech

[DELTA-45-G]

Atomic Handoff Verified & Resolved

Hi @SachinPremkumar and @yakshithkd23,

The data loss issue during concurrent writes has been resolved using the Atomic Handoff strategy.

@yakshithkd23 i changed some code lines
What was changed:
Instead of deleting the active log, AndroidMetricCollector now atomically renames metrics.log to metrics.log.processing immediately before the background upload begins.

Verification:

1. Concurrent Writes: The app successfully generates a fresh metrics.log to capture new events while the .processing file is actively uploading.
2. Safe Cleanup: Only the .processing file is deleted upon a successful TUS upload.
3. Network Drops: If the worker fails (e.g., offline), the .processing file remains safely on the device for the next WorkManager retry cycle without being overwritten.

Proof of Concurrent Execution:

Proof of Safe Cleanup:

Ready for the next review!

[yakshithkd23]

Atomic Handoff implementation verified. The code now safely handles concurrent writes as discussed. Ready to merge.

[DELTA-45-G]

Phase 5 (Part 1) Complete — Auth & Onboarding Triggers Verified

Hi @SachinPremkumar, @Pragya279

We have successfully implemented and verified the core pipeline for Phase 5 (Flutter UI Event Triggers), focusing on the Authentication and Onboarding flows.

What was implemented:

1. TelemetryService created in lib/utils/telemetry_service.dart as a singleton.

2. Added registration lifecycle event tracking for:

• login.success / login.failure — lib/ui/login_page.dart
• registration.started — lib/ui/onboard/onboard_landing_page.dart
• registration.started (Onboarding) — lib/ui/onboard/onboarding_page.dart

All events successfully flow via:
Flutter UI → TelemetryService → Pigeon → AndroidMetricCollector → metrics.log

Verified End-to-End Pipeline:
We successfully verified the login event trigger with a real app interaction. The app captured a live login failure event with the exact error context:
"name": "registration.login.failure", "reason": "REG_CRED_EXPIRED", "machine": "22031116AI_TP1A.2"
This confirms the full Flutter → Native → File pipeline is working correctly.

Blockers for Remaining Phase 5 Goals:
We have not included the biometric capture or packet upload triggers in this commit for the following reasons:

1. lib/ui/scanner/scanner.dart: As noted earlier, this file only contained commented-out code. When we uncommented the scanner implementation to add our telemetry logic, it triggered 9+ build errors, preventing the app from compiling. We have left this out of the commit so we don't break the development build. Could you advise on how we should handle this, or if there is a different active scanner file we should target?

2. lib/ui/export_packet/widgets/upload_button.dart: We have held off on updating this file until the scanner issue is resolved so we can push them in a single follow-up commit.

3. Testing Access Limitations: Currently, we only have access up to the login page and lack the active test credentials required to proceed further into the application. Because of this, we are unable to test the remaining downstream features (biometrics, packet upload, etc.). Could you please provide test credentials or confirm if this should be validated in the review environment?

Regards,
Team TuluTech