feat(scaling): bulk import batching for high volume ingestion and Prometheus metrics endpoint

[thehaven]

Note

Disclosure: This pull request was developed with assistance from Google Antigravity / Gemini CLI (gemini-cli) AI coding assistant. All implementation logic and unit tests (vitest 100% passing) have been empirically verified in production container deployments.

📦 Summary & Problem Solved

• Problem 1 (Bulk Ingestion Scaling): Importing large catalogs (500+ or 3,000+ books at once) caused UI thread lockups and database connection pool exhaustion.
• Problem 2 (Observability): Lack of standardized container metrics for Prometheus/Grafana monitoring.
• Fix:
  1. Implemented bulk-import-batch.processor.ts to process high-volume book imports in chunked background batches of 100 with Bull queue rate-limiting (max: 10, duration: 1000).
  2. Added /api/admin/metrics endpoint providing Prometheus-compatible runtime telemetry (node_memory_bytes, bull_queue_jobs_total, db_requests_count, etc.).

🧪 Unit Test Verification

 ✓ tests/processors/bulk-import-batch.processor.test.ts (1 test)
   ✓ should break large import requests into chunks of 100 and add jobs to queue
 ✓ tests/api/metrics.test.ts (1 test)
   ✓ should return Prometheus-formatted metrics text