Skip to content

"tiny fix"#332

Merged
AnkushMalaker merged 107 commits into
devfrom
fix/optimizations-2
Jul 14, 2026
Merged

"tiny fix"#332
AnkushMalaker merged 107 commits into
devfrom
fix/optimizations-2

Conversation

@AnkushMalaker

Copy link
Copy Markdown
Collaborator

No description provided.

- Updated `useAudioStreamer` to accept an options parameter for token refresh handling.
- Implemented automatic re-login on token expiration, updating the WebSocket connection with a new token.
- Enhanced error handling in WebSocket authentication to provide specific failure messages.
- Modified audio retrieval endpoints to support both Opus and WAV formats, improving efficiency and flexibility.
- Updated Docker configurations to replace Qdrant with Neo4j, adjusting environment variables and service dependencies accordingly.
- Cleaned up unused code and improved logging for better traceability.
Symptom: iPhone app worked for a day after install, then failed to
launch after backgrounding. No iOS .ips crash reports were produced.
Strong signal it's a JS-side startup hang (no native crash) caused by
expo-updates loading a bad/partial downloaded bundle at boot — reinstall
clears the cache and restores it for another day.

- Disable expo-updates via updates.enabled=false in app.json
  (no code calls Updates.*, so OTA was not in active use)
- Add src/utils/logger.ts: file logger at
  FileSystem.documentDirectory/chronicle-logs/chronicle-log.txt,
  1 MB rotation, session header with updates state, ErrorUtils
  global handler, unhandledrejection listener
- Add src/components/ErrorBoundary.tsx: root JS error boundary with
  share/retry UI
- _layout.tsx: initialize logger + wrap in ErrorBoundary
- ConnectionLogContext: mirror every event to file logger
- diagnostics.tsx: Share Log File / Clear File buttons
- eas.json: add ascAppId for non-interactive TestFlight submits
- .easignore (root + app): exclude node_modules, android/app/build,
  ios/Pods, build artifacts (project tarball 2.1 GB -> 154 MB)
- expo@~55.0.15
- @siteed/expo-audio-studio@^2.18.6 (2.18.1 pinned old
  expo-modules-core@~2.4.0, incompatible with SDK 55)
- @types/react@~19.2.0, typescript@~5.9.2
- Regenerated ios/ via 'expo prebuild --clean'
- Removed newArchEnabled from app.json (default in SDK 55)
- Removed android.usesCleartextTraffic (duplicated by
  expo-build-properties plugin)
- eas.json: pin testflight ios.image to
  macos-sequoia-15.6-xcode-26.2 to satisfy Apple's Apr 28 2026
  Xcode 26 upload requirement
EAS Build runs strict 'npm ci' which rejected the lock file produced
by earlier 'npm install --legacy-peer-deps' commands (react-dom@19.2.5
pulled transitively by expo-router's radix-ui stack requires
react@^19.2.5, but root had react@19.2.0).
expo-router in SDK 55 imports @expo/metro-runtime transitively from
node_modules/expo-router/node_modules/. With disableHierarchicalLookup=true
Metro can only resolve from the paths we explicitly listed, causing
'Unable to resolve module @expo/metro-runtime' during EAGER_BUNDLE.

This was the SDK 53-era monorepo recommendation; SDK 55's expo-router
deps require hierarchical lookup.
1) index.tsx: onDeviceConnect/onDeviceDisconnect referenced 'orchestrator'
   and 'autoReconnect' which are declared later in the function (TDZ).
   Accessing 'orchestrator.handleStartAudioListeningAndStreaming' in the
   deps array caused 'Cannot read property ... of undefined' at render.
   Introduce orchestratorRef/autoReconnectRef, assign them after the
   hooks are declared. Pre-existing latent bug that became reproducible
   on SDK 55.

2) logger.ts: Expo SDK 55 removed the default 'expo-file-system' runtime
   in favour of 'expo-file-system/legacy' for the classic API (or the
   new File/Directory classes). Calls to getInfoAsync etc. were throwing
   on every write, so the file logger silently produced nothing.
   Switch to 'expo-file-system/legacy' and add expo-file-system as a
   direct dep so TS can resolve its types (it was only transitively
   installed under expo/node_modules/).
- Updated environment variables and configuration files to replace Neo4j settings with FalkorDB.
- Modified Docker Compose files to use FalkorDB services instead of Neo4j.
- Adjusted application logic and routes to accommodate FalkorDB for entity management and knowledge graph functionalities.
- Introduced new graph client utilities for FalkorDB integration.
- Updated documentation to reflect changes in the architecture and service dependencies.
Captures recent log lines in an in-memory deque handler and exposes
them via a scrollable rumps.Window so users can inspect activity
without leaving the menu bar.
rumps.Window wraps an NSTextField, which doesn't scroll — long log
output was unreachable past the visible area. Replace with a native
NSAlert + NSScrollView containing an NSTextView (monospace, read-only,
scrolled to the latest entry on open).
start.sh forwards args to main.py (supports subcommands like kickstart),
making the verbose-only relay.sh redundant. Also tracks logs.sh helper
for streaming ESPHome firmware logs.
The logging commands for the qdrant-test service have been removed from the full-tests-with-api.yml, pr-tests-with-api.yml, and robot-tests.yml workflows to streamline log management and reduce unnecessary output during CI runs.
A new play-audio now cancels the Opus clip still streaming to the same
device instead of interleaving frames on the socket (which sounded like
two replies alternating). Tracked per client_id so it holds across a
stale + fresh downlink subscriber; the stream runs as a background task
so the downlink loop stays free to supersede it.
…ator

- Add AppSettingsProvider so home + settings share one settings instance
- New Settings screen (gear in header) holds backend pairing, login,
  system admin, and network overview
- Home screen shows live status + a setup CTA only when unpaired/logged out
- Replace full-screen auto-reconnect takeover with inline spinner banner
  in the Connection tab
- Bump app version to 1.0.8
…alignment

Fix over-identification where a 2-person conversation was labeled as many
speakers (short segments spuriously matching a growing 42-speaker gallery,
plus cross-chunk label fragmentation from the 2-min diarization cap).

- audio_backend: replace label-string chunk stitching with embedding-based
  global speaker reconciliation — cluster chunk-local centroids (average
  linkage, same-chunk cannot-link sentinel, max_speakers hard cap). Keeps
  chunking to bound pyannote's O(N^2) CPU clustering while producing
  globally-consistent labels.
- identification: cluster-then-identify — one pooled centroid decision per
  diarized speaker (not per segment), with an open-set best-vs-runner-up
  margin guard and exclusive assignment.
- unified_speaker_db: add identify_with_candidates() returning ranked
  candidates for margin/exclusive logic; identify() behavior unchanged.
- asr-services: expose /align endpoint (existing MMS_FA forced aligner),
  reading WAV via stdlib wave to avoid the torchcodec dependency;
  per-segment alignment with proportional fallback.
- backend: synthesize word timestamps via /align when a provider (e.g.
  VibeVoice) emits segments without words, routing word-less conversations
  onto the full pyannote re-diarization path instead of per-segment id.

Result on the test conversation: 10 labels (incl. 4 spurious identities)
-> 2 clean speakers (ankush, vipul g) + a few stray segments.

Claude-Session: https://claude.ai/code/session_01CkoBkd7ZM3SPUBCHctNKRt
Make the chunk-reconciliation and open-set identification parameters tunable
from config instead of hardcoded:

- reconciliation_threshold (default 0.4): min cosine similarity to merge
  chunk-local centroids into one global speaker
- identify_margin (default 0.1): open-set best-vs-runner-up gap guard
- exclusive (default true): one enrolled speaker per diarized speaker

Flows config (diarization settings) -> speaker_recognition_client form fields
-> /v1/diarize-identify-match Form params -> async_diarize / cluster-identify.
max_speakers was already wired; it caps the global speaker count after
reconciliation. defaults lowered the async_diarize reconciliation default
0.5 -> 0.4 to merge same-speaker chunks more readily.

Claude-Session: https://claude.ai/code/session_01CkoBkd7ZM3SPUBCHctNKRt
Silent/degenerate segments embed to a zero vector, which the embedder normalizes
to NaN (0/0). A NaN centroid produced NaN cosine distances, and sklearn's
AgglomerativeClustering rejects NaN -> ValueError -> HTTP 500 in
diarize-identify-match (seen reprocessing streaming-only conversations).

- _embed_local_speakers: drop non-finite per-segment embeddings; skip a local
  speaker entirely if it has no finite embedding (and guard zero-norm centroids).
- _reconcile_chunk_speakers: nan_to_num the distance matrix as a safety net so a
  stray non-finite value becomes "maximally far" instead of crashing clustering.

Claude-Session: https://claude.ai/code/session_01CkoBkd7ZM3SPUBCHctNKRt
Add 'elato' to the wearable name filters so the OMI-compatible Elato BLE
peripheral is auto-discovered alongside OMI/Friend/Neo, in both the macOS
menu-bar client (main.py, menu_app.py) and the mobile app. Classify Elato
as an OMI-type device (it speaks the OMI GATT contract) and update the
filter toggle labels.
Plumbs the backend's existing Opus speaker downlink (speak-start / binary Opus /
speak-end, plus speak-stop) through the relay to the device's new BLE speaker
characteristic, so Elato gets TTS/playback over BLE the way it already does over WiFi.

- friend-lite-sdk: add ELATO_SPEAKER_CHAR_UUID + speaker opcodes; OmiConnection gains
  speaker_start/end/stop and write_speaker_audio (MTU-fragmented write-without-response).
- local-wearable-client: receive_handler now forwards binary Opus frames and
  speak-start/end/stop to the BLE speaker char (threaded the OmiConnection through
  stream_to_backend); only for OmiConnection devices (OMI/Neo have no speaker).
- backend: guard websocket.send with WebSocketState.CONNECTED in the interim-result
  and device-downlink subscribers, fixing the 'websocket.send after websocket.close'
  ASGI error when a result/downlink races the socket close.
…t count from active transcript version. Add TIMING annotation type and related API endpoints for adjusting segment timings. Implement waveform data fetching and region editing components in the web UI for improved audio segment management.
Sweep of the mid-function-import anti-pattern across the backend,
extras services, edge/root scripts, and test libs (~350 sites hoisted
or deduped across 114 files, net -150 lines).

- Hoist stdlib, light third-party, and internal imports to module top
  per the project import guidelines; dedupe repeated per-function
  imports (e.g. rq.registry x6 in queue_routes, plugin_service x13 in
  system_controller)
- Keep genuinely-lazy imports and document each with a comment:
  circular-dependency breakers (queue_controller<->workers,
  conversation_jobs<->transcription_jobs, chronicle->agent->llm_client,
  speaker-recognition router DI), optional deps (opentelemetry,
  langfuse, minidisc), heavy ML model loads, macOS-only AppKit/rumps,
  torch-less ASR provider images, and init-ordering in app_factory /
  models/job / rq_worker_entry
- Verified: fresh-process import sweep of all changed backend modules
  (no new failures vs baseline), ruff F401/F811/F821 vs HEAD (0 new,
  ~80 pre-existing unused imports eliminated), black + isort, and the
  no-API robot suite (110 pass; 18 failures are pre-existing stale
  tests for removed knowledge-graph/falkordb endpoints, RQ worker
  count, and the processing-status rename)
Plugins whose external dependency is unreachable (e.g. Home Assistant
on a powered-off server) now degrade instead of failing permanently:

- New PluginConnectivityError -> DEGRADED health state (vs FAILED for
  real config/setup errors); system event recorded only on transition
  so retry ticks don't spam the log
- Shared initialize_plugins() used by every plugin-router host
  (FastAPI app, streaming worker, wakeword worker, RQ workers,
  hot-reload) replaces four copies of the init loop
- run_plugin_recovery() background task: per-plugin exponential
  backoff re-init (30s..600s) plus a 5-min health probe that demotes
  initialized plugins whose health_check starts failing
- homeassistant plugin raises PluginConnectivityError on unreachable
  HA and uses a 5s connect timeout
defaults.fallback_llm names a model (default: OpenAI gpt-5-nano) that
llm_client wrappers retry against exactly once when the primary LLM
fails at the transport level (APIConnectionError, InternalServerError,
timeout) — auth/4xx config errors are not retried since the fallback
would not fix them.

- model_registry: get_fallback_llm_operation() resolves an operation
  against defaults.fallback_llm via a new model_override param; skips
  when unset, invalid, or identical to the primary model.
  reasoning_effort "none" downgrades to "minimal" for pre-gpt-5.1
  reasoning models that reject it
- health: /readiness reports fallback_llm status (non-critical)
- wizard: new "Fallback LLM" setup step with enable/disable and
  model/key prompts; defaults.yml gains the fallback-llm model entry
Two annotation-driven workflows:

Deletion annotations
- New DELETION annotation type: mark a transcript segment for removal,
  staged as a pending correction and committed by /apply, which
  re-derives a new transcript version with the segment dropped
  (inserts are positioned against original indices first, then
  deletions applied by object identity so indices stay valid)
- TranscriptEditor: per-segment delete toggle with pending-state UI

Curated speaker enrollment (replaces blunt annotation replay)
- GET /api/finetuning/enrollment-candidates builds a quality-gated,
  per-speaker candidate list from the ACTIVE transcript version of
  labelled conversations: >=3s duration, no cross-talk overlap with
  another enrollable speaker, near-duplicate spans deduped; longest 5
  clean clips per speaker pre-selected
- POST /api/finetuning/enroll-selected enrolls only user-promoted
  clips, re-validating each span against the current active version
  (guards a version change between review and submit), then marks the
  source diarization annotations trained
- New EnrollmentCandidates review UI on the Finetuning page
- health_routes: stop constructing the memory service, transcription
  provider, and model registry at import time. Memory-service creation
  raises when LLM defaults are unconfigured, which killed every route in
  routers.modules on import; all three are now acquired inside the
  handlers (creation failure reports 'unavailable' instead of a 500).
- user_controller: delete_all_user_memories is async — running it in an
  executor returned an un-awaited coroutine, so user deletion never
  deleted memories and the count comparison would TypeError. Await it
  directly.
- enroll_speaker: --list/--delete called list_speakers_with_url/
  delete_speaker_with_url, which never existed (NameError). Thread
  --service-url through list_speakers/delete_speaker instead.

Also drops now-dead unused imports in the touched files and updates the
data_audit_controller lazy-import comment to cite the real (still
present) package cycle rather than the removed side effect.
@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 7cf63648-4d96-470b-8257-0c8208057c3e

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/optimizations-2

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@AnkushMalaker
AnkushMalaker merged commit 07e194e into dev Jul 14, 2026
1 of 2 checks passed
@github-actions

Copy link
Copy Markdown

⚠️ Robot Framework Test Results (No API Keys)

Status: ❌ Some tests failed

ℹ️ Note: This run excludes tests requiring external API keys (Deepgram, OpenAI).
Tests tagged with requires-api-keys will run on dev/main branches.

Metric Count
✅ Passed 108
❌ Failed 22
📊 Total 130

📊 View Reports

GitHub Pages (Live Reports):

Download Artifacts:


View full workflow run

AnkushMalaker added a commit that referenced this pull request Jul 15, 2026
…ker-count expectations

The knowledge-graph feature (FalkorDB routes, wizard toggle) and the
/api/admin/chat/config endpoints were removed in #332, but their tests
were left behind — every one 404s in CI:

- delete knowledge_graph_tests.robot, knowledge_graph_integration_test.robot
  and knowledge_graph_keywords.robot
- health_tests: stop asserting the removed config.falkordb_host key
- system_admin_tests: drop the three chat-config tests and the
  /api/admin/chat/config entry in the non-admin access list
- test_wizard_defaults: drop select_knowledge_graph tests (function
  removed from wizard.py)

Also sync expectations that drifted behind intentional refactors:

- always_persist_audio_tests: processing_status is now the 3-state
  machine's 'active', not 'pending_transcription'
- infra_tests: 10 RQ workers now (the dedicated memory worker was added
  alongside the 6 general + 3 audio persistence)
AnkushMalaker added a commit that referenced this pull request Jul 15, 2026
'Button Press Should Close Active Conversation' failed in every CI run
since #332: the button_control rework maps single_press to stop_playback
(barge-in) and double_press to close_conversation, and the 'actions:'
override in plugins.test.yml was never read — load_plugin_config() only
merges enabled/events/condition from config/plugins.yml, so the plugin
ran with its own config.yml defaults. The conversation never closed and
the WS eventually died with end_reason=websocket_disconnect.

- test sends DOUBLE_PRESS (the close_conversation mapping) and documents
  where the mapping lives
- plugins.test.yml: drop the dead actions block, note that this file is
  orchestration-only
- plugin_service: warn when config/plugins.yml carries non-orchestration
  keys instead of silently dropping them

Verified locally against real Deepgram: test passes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant