Make the artifact a set of models, in the code and in the config

[Varfalamei]

Why

RecommenderALS was the whole feed. It owned a PopularModel for cold users, an optional RecommenderCoVis for live sessions, the blend weights, the four-way segment routing and the entire Strategy vocabulary. Its config declared popular, covis and blend — who serves cold users, whether there is a session layer, and how two lists are fused. None of that is a hyperparameter of ALS.

Two consequences: "add a model" meant editing ALS, and als_covis_youtravel read as "the ALS artifact with extras" rather than as what it is — a set of models behind one name.

What ships

The config keys by ROLE; the value's type picks the model.

ModelSetSettings(
    main=ALSSettings(...),                 # the main ranker
    fallback=PopularSettings(...),         # everyone the main ranker cannot represent
    realtime=CoVisSettings(...),           # optional, scores the live session
    blend=BlendSettings(MAIN_WEIGHT=1.0, REALTIME_WEIGHT=1.0, RRF_K=60),
)

RankerSettings = Union[ALSSettings, EASESettings, CoVisSettings, PopularSettings], and MODEL_FOR_SETTINGS is the single place mapping a settings type to its model class. So putting LightFM in the main role is main=LightFMSettings(...) — no schema change, no conditional anywhere, and adding a model is one registry entry plus the type in the union. ALSSettings is a leaf again: four ALS hyperparameters and the two common fields. Passing popular=/covis=/blend= to it is a validation error, with a test asserting it.

Field names were als/popular/covis in the first two commits of this branch — named after the models that happen to fill the roles today, which is the same category error as ALSSettings.covis one level down.

The code separates routing from scoring.

RecommenderModelSet   which member answers, and under what Strategy
members               how to score, and nothing else

RecommenderModelSet.recommend() is the only routing table in the package. A member knows its own algorithm: it answers can_serve(user) and returns candidates. It does not import a sibling, hold one as a field, read the set's config, or name a segment.

Members return (RecomItems, session_used) — a fact — and the set names the segment from it. My first version had members stamping their own strategy, and test_hot_user_ignores_history caught the lie: a hot user whose session seeds are all unknown to the models gets a plain ALS ranking, and labelling that model_realtime_hot_users silently corrupts every A/B readout split on strategy.

The role is realtime, the signal is session.

The role that consumes live events was called session while the strategies it emits are model_realtime_hot_users / model_realtime_warm_users — a published, append-only vocabulary mirrored by numeric id in api/docs/DEBUG_INFO_CODEC.md. Config and wire disagreed about the same thing. The rule is now explicit in CLAUDE.md §4: the signal is a session (history=, has_session, session_used — the API and Redis vocabulary), and the role, its weight and its strategy are realtime. BlendSettings.SESSION_WEIGHT would also have been the third distinct meaning of "session weight" in this package, next to CoVisSettings.COVIS_SESSION_WEIGHTS (a per-seed event multiplier) and the co-occurrence kernel's session_weight argument.

Strategy drops three members it could never emit: MODEL_HOT_AND_COLD_USERS, MODEL_ALS_COVIS_BLEND, MODEL_COVIS_SESSION (10 members to 7). The first lost its only emitter when RecommenderRandom was deleted; the other two briefly named algorithms instead of segments and were reverted in #11. Their ids stay reserved in DEBUG_INFO_CODEC.md forever — that table, not this enum, is the contract, and the consumer decodes historical records by id. Removing a ROW would break it; removing a MEMBER does not.

Manifest artifacts are declined, not deferred (48fb081). Three of the four problems a manifest.json would solve dissolve on their own, and the fourth is already answered by the trainer's log lines — against a runtime repack plus a loader change under the deploy ordering below. MODEL_SET_ROUTING.md records the routing-as-data design as deferred with its trigger conditions, so neither doc reads as a commitment nobody made.

Legacy artifacts: three hooks, shape only

Two generations exist in the buckets and both must serve. The old artifact is a RecommenderALS __dict__ carrying every member itself plus a flat or nested ALSSettings; the new one is a RecommenderModelSet __dict__ with main/fallback/realtime.

• ALSSettings.__setstate__ — rebuilds sub-configs from flat fields on unpickle;
• ModelSetSettings.from_legacy_als_settings — either legacy config shape into a set config;
• RecommenderModelSet.from_legacy_als_state — a legacy ARTIFACT into a model set, member by member, so there is exactly one routing path in the library rather than a second copy kept alive for old pickles.

This is load-bearing, and its failure mode is silent. At serving time the only composition field read is blend — fallback and realtime configs are read by train()/calc_metrics(), which always get a freshly built config. So a missing shim would not raise on a legacy artifact: the RRF fusion would run at the default 1.0/1.0/60 instead of the trained weights and quietly reorder the feed. test_legacy_artifact_blends_with_its_own_weights_not_defaults pins that, and test_legacy_artifact_loads_and_recommends_identically pins that a legacy artifact answers all four segments with identical items, scores and strategy — a member assigned from the wrong key raises nothing and just serves worse.

serving/model.py therefore resolves the class from the pickle's shape first and only then from the artifact name: the name cannot tell the generations apart, since als_covis_youtravel is a public query parameter and cannot change (§5.9). New: test_serving_loader.py stubs Triton's backend utils so the real _load_model — which decides what production loads and had no test — runs off-cluster.

Delete all three hooks once both artifacts have been retrained twice.

Renaming ModelSetSettings fields and BlendSettings weights was free only right now: no artifact in either bucket carries a session key or a role-shaped config (verified against both real pickles). After the first retrain each rename would have cost another legacy hook.

Verification

• 109 tests pass, black clean. The flake8 gate CI actually blocks on (E9,F63,F7,F82) is clean; the advisory --exit-zero pass reports 9 warnings, the same 9 already present on master — this branch adds none.
• Gate against real artifacts, old code vs new: both dev artifacts pulled from S3, loaded through the real TritonPythonModel._load_model, 14 request cases each — 28/28 identical items, scores and strategies. The runner pins BLAS to one thread the way app/src/settings.py does in production; without that, ALS dot products sum in a thread-dependent order and scores drift by ~1e-7, which reads as a diff while items and labels are unaffected.
• Deployed to dev end to end: runtime repacked, trainer image 2026.08.25-dev.87, DAG tag bumped, retrained. The resulting artifact loads as a RecommenderModelSet and answers every route.

Deploy ordering

A code change under §5.11: upload runtime_env.tgz first, then bump the trainer image tag and retrain. Reversing it raises ImportError on model reload for the whole bucket, including artifacts that were not retrained. Done for dev; not yet for prod.

The API-side counterpart lives in youtravel-recsys MR !9 and is not part of this PR.

[Varfalamei]

The S3 real-artifact gate: PASSED

The one open gate from the PR description is closed. Both REAL artifacts were pulled from the dev bucket (models/*/20260813183106, the flat legacy RecommenderALS shape — same generation prod serves) and run through both generations of code on an identical 14-case request matrix (3 hot users from the artifact's own id map × profile/session, weighted and plain history formats, garbage-history honesty cases, candidate restriction, cold, cold+candidates):

Artifact	Old code (master 6658124, RecommenderALS.load_model)	New code (this branch, via TritonPythonModel._load_model)	Result
als_covis_youtravel	god-class	adapted RecommenderModelSet	14/14 identical — items, scores (9 dp), strategies
als_youtravel	god-class	adapted RecommenderModelSet	14/14 identical

Strategy honesty cases confirmed on real data: hot user + unusable history → model_hot_users (not realtime) in both generations; unknown user + unusable history → empty result keeping model_realtime_warm_users, no silent fallback to popular.

The new run goes through the exact serving entry point (_load_model with shape sniffing → from_legacy_als_state), not a shortcut.

Also in the branch since the description was written: docs/MODEL_SET_ROUTING.md — a worked-out routing-as-data design, explicitly deferred 2026-08-24 in favour of shipping this simple shape.

Deploy ordering reminder (§5.11): runtime repack first, then retrain.