Eightsleep Test

Requirements

1. Assumptions

• Incoming events are append-only and can be stored in a Redis stream.
• Each event contains a fixed-size feature vector of exactly 3 floats.
• Timestamps are integer seconds and are used to maintain a rolling time window.
• The data posted is of the expected format and matches rough expectations provided by the assignment
• The events are not so significantly unordered that they can be ignored by the user median calculation (it will be ignored by default but currently no record of it will be kept as we do not persist events)

2. Design

Python Architecture

• api/inference_endpoint.py

  • FastAPI service for event ingestion, user median lookup, and cached median-of-medians retrieval.
  • Uses Pydantic for request validation.
  • Writes events to a Redis stream (events_stream) and exposes user lookup endpoints.

• worker/median_worker.py

  • Separate Redis stream consumer using a consumer group.
  • Batches inference with InferenceModel.predict_batch.
  • Maintains per-user sliding windows in memory via worker/user_window.py.
  • Writes per-user medians and cached median-of-medians values back to Redis.
  • Computes median-of-medians periodically rather than on every API request.

• worker/user_window.py

  • UserScoreWindow maintains a time-windowed deque of scores.
  • UserWindowStore keeps one window per user and computes rolling medians.

Median-of-medians caching

• The worker computes a single cached "median of medians" value from all active user medians and stores it in Redis under median_of_medians.
• This happens on a timed interval (MEDIAN_OF_MEDIANS_UPDATE_SECONDS) to avoid recomputing across every API request.
• The API serves /stats from the cached value, which is fast and avoids scanning all users or recomputing a second-level median in real time.
• Because updates are periodic and driven by worker activity, the cached value can lag behind the latest per-user medians — especially if there are few incoming events or if the worker has not yet refreshed the cache.
• This design choice favors read speed and low latency for /stats at the cost of potentially stale aggregation data.

Core Tradeoffs

• In-memory user windows are fast but not durable. A restart loses recent per-user state.
• Redis stream decouples ingestion from processing and supports multiple worker consumers.
• Python median computation on sorted lists is simple and sufficient for moderate window sizes, but it is not optimized for very large sliding windows.
• The worker currently keeps state in-process, which limits horizontal scaling unless the window state is externalized.

Kappa-style Architecture

• The Redis stream acts as the immutable event log and the single source of truth.
• The API writes events to the stream and remains stateless, enabling immediate ingestion and query routing.
• The worker consumes the same stream, performs inference, updates per-user medians, and writes materialized results back to Redis.
• This mirrors Kappa by using one stream for both live processing and replay-based reconciliation; a future implementation would replay the same stream for corrections, model updates, or batch analytics.

Stretch Goals

• Provided a separate API service and worker service with containerized setup via docker-compose.yml.
• If additional stretch goals were implemented, they would include:
  • more health checks and readiness probes
  • metrics and observability
  • persistence of rolling windows for crash recovery
  • multiple worker consumers with partitioned load
  • bottleneck search instrumentation and query support to find slow or overloaded user flows

3. Next Steps

a. Production Readiness

• Add structured logging and observability metrics.
• Harden Redis connection handling with retries and backoff.
• Add authentication and authorization for API endpoints.
• Persist user windows or reconstruct them from Redis data on restart. We would likely create a consumer that persists this data in a DB for downstream analytics or model building purposes.
• Adopt a Kappa-style architecture: use the Redis stream as the single source of truth, provide an immediate "speed layer" for low-latency user median queries, and support replay/reconciliation from the same stream for more accurate or historical batch recalculation.
• Add a real model instead of the current placeholder inference path.
• Improve median_of_medians freshness by moving from periodic batch recompute to either incremental maintenance, on-demand recompute with caching, or a more scalable quantile algorithm.

b. Scaling Prediction

• The API can scale horizontally behind a load balancer because it is stateless.
• The worker can scale using Redis consumer groups, but current in-memory window state is per-process.
• Redis stream ingestion is the likely bottleneck under heavy load; future work should include partitioning and sharding. We could move this to a better more durable message system such as Kafka.
• The worker’s sliding-window memory usage grows with the number of active users and window size. If we offload this to an in memory store it would likely be more efficient

c. Next Work

• Implement durable user window state in Redis or a distributed cache.
• More thorough testing
• A Kappa-style architecture that treats the Redis stream as the immutable source of truth, with a fast materialized-view path for live queries and a replayable stream path for reconciliation and batch analytics.
• Add a lightweight load test harness for end-to-end ingestion and processing.
• Add bottleneck search support by instrumenting the worker and stream path with per-event latency, backlog, and user activity metadata.
  • This would let the system answer queries like "which users or streams are creating the longest processing delay?" or "which partitions are exceeding throughput capacity?"
  • In practice, it would be used for operational visibility, alerting, and capacity planning, helping identify whether a slow median computation is caused by a hot user, an overloaded worker, or late-arriving events.

Running the Project

docker-compose up --build --detach

Running Tests

Install pytest and run:

python3 -m pip install pytest
python3 -m pytest tests