TTFT aware Latency Scorer

[Mohammad-nassar10]

What type of PR is this?

/kind feature

What this PR does / why we need it:

Adds a new ttft-aware-scorer that routes each request to the model with the lowest predicted TTFT under current load.

It reads the per-model snapshot published by the ttft-percentile-extractor — the service floor P10Low, the operating points P25/P50, and their in-flight anchors inflightAtP25/inflightAtP50 — and predicts the effective TTFT for a request arriving now as a line through the high operating point (inflightAtP50, P50) and a low anchor blended between the in-cloud point (inflightAtP25, P25) and the load-free floor (0, P10Low):

w             = clamp((inflightAtP50 - inflightAtP25) / anchorGapScale, 0, 1)
lowInflight   = w * inflightAtP25
lowTTFT       = w * P25 + (1 - w) * P10Low
effectiveTTFT = lowTTFT + (inflight - lowInflight) * (P50 - lowTTFT) / (inflightAtP50 - lowInflight)

Models are then scored (maxTTFT - effectiveTTFT) / (maxTTFT - minTTFT). Under-observed (uncalibrated) models are seeded optimistically at the best observed TTFT; with optional explorationRate, each under-observed model is independently given that probability of being forced to the top score so it receives a calibration probe — preventing a cold pool from being starved by an already-calibrated one.

Parameters: explorationRate (default 0, disabled) and anchorGapScale (default 2).

Depends on PR #204.

Which issue(s) this PR fixes:

Fixes #54.

Release note (write NONE if no user-facing change):

NONE

[github-actions]

⚠️ Large PR detected

Your PR is large. Please consider breaking it into multiple PRs.

The do-not-merge/hold label has been added and can be removed by the reviewers based on their judgement.

[ronenkat]

The code is missing algorithm style documentation to help understand the intent of the code.

[github-actions]

This PR is marked as stale after 21d of inactivity. After an additional 14d of inactivity (7d to become rotten, then 7d more), it will be closed. To prevent this PR from being closed, add a comment or remove the lifecycle/stale label.