A strategy names the segment, not the algorithm: restore model_realtime_* - #11
Merged
Merged
Conversation
…me_* als_covis_youtravel reported two strategies of its own, als_covis_blend and covis_session, for the same two segments the ALS item-sim paths always served: a hot user scored with their live session, and an unknown visitor scored from theirs. That was a mistake. A strategy says what was known about the visitor and which signal was used. Which artifact answered - and therefore which algorithm ran - is already carried by model_name right next to it, so the new strings added nothing while breaking everyone: dashboards and analytics watching for model_realtime_* went silent, and a teammate waiting to see his profile warm up could not have seen it, because on als_covis those values are unreachable by construction. Both ALS artifacts now report model_realtime_hot_users and model_realtime_warm_users; split A/B readouts on model_name. The two values stay in the enum, unemitted, because their numeric ids in DEBUG_INFO_CODEC.md are append-only - same treatment as model_hot_and_cold_users. CLAUDE.md states the rule so the next algorithm change does not repeat this. Reverting the labels costs the tests their discriminator: strategy alone no longer proves the blend ran. Two tests now compare the covis-configured model against the same model without the sub-config on identical input and require the outputs to differ, and the item-sim fallback is pinned by output equality instead. 101 passed. Verified on the production dev artifact (als_covis_youtravel, 13.08): hot -> model_hot_users, hot+session -> model_realtime_hot_users, unknown+session -> model_realtime_warm_users, unknown -> model_cold_users.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
als_covis_youtravelreported two strategies of its own —als_covis_blendandcovis_session— for the same two segments the ALS item-sim paths always served: a hot user scored with their live session, and an unknown visitor scored from theirs.That was a mistake. A strategy says what was known about the visitor and which signal was used. Which artifact answered, and therefore which algorithm ran, is already carried by
model_nameright next to it. So the new strings added nothing, while breaking every consumer:model_realtime_*went silentals_covisthose values are unreachable by constructionWhat
Both ALS artifacts now report
model_realtime_hot_users(hot + session) andmodel_realtime_warm_users(unknown + session). Split A/B readouts onmodel_name.MODEL_ALS_COVIS_BLENDandMODEL_COVIS_SESSIONstay in the enum, unemitted, because their numeric ids inapi/docs/DEBUG_INFO_CODEC.mdare append-only — the same treatmentMODEL_HOT_AND_COLD_USERSalready gets.CLAUDE.mdnow states the rule, so the next algorithm change doesn't repeat this.Tests
Reverting the labels costs the tests their discriminator:
strategyalone no longer proves the blend ran. So two tests now compare the covis-configured model against the same model without the sub-config on identical input and require the outputs to differ, and the item-sim fallback is pinned by output equality instead of by a distinct label.101 passed. black and the flake8 gate clean.
Verified against production
Loaded the live dev artifact (
als_covis_youtravel, trained 2026-08-13, 214 MB, read straight from the bucket Triton serves):model_hot_usersmodel_realtime_hot_usersmodel_realtime_warm_usersmodel_cold_usersThe paired change in the parent repo restores the same label for the API's flag-gated covis re-route.