What happened:
Structured log lines that include a datalayer.Model value render the
model as {}, even when the model is actually populated. Example from a
live DEBUG-level run:
{"level":"debug","caller":"modelselector/model_selector_pipeline.go:264", "msg":"Completed running picker plugin", "plugin":"max-score-picker/max-score-picker", "result" {"TargetModel":{}}}
The picker returned a real, non-nil model. The nil check at
pkg/modelselector/modelselector.go:73 would have failed the request
otherwise. Only the log rendering is empty.
Root cause: datalayer.Model is an interface (pkg/framework/interface/datalayer/model.go:19)
and its only concrete implementation, the unexported model struct at model.go:34, has unexported fields (name, attributes) and no MarshalJSON method. encoding/json cannot see unexported fields, so
every Model value marshals to {} silently.
Affected log sites (all render the model as {}):
pkg/modelselector/model_selector_pipeline.go:264
pkg/framework/plugins/modelselector/picker/maxscore/picker.go:71-72
pkg/framework/plugins/modelselector/picker/random/picker.go:74-75
pkg/framework/plugins/modelselector/picker/weightedrandom/picker.go:108-109
What you expected to happen:
Log lines containing a datalayer.Model should render the model's name
(and, ideally, its attributes) rather than {}. Debug logs are the primary
signal for triaging model-selection behavior. A silently empty payload
makes them worse than useless for that use case.
How to reproduce it (as minimally and precisely as possible):
- Run the processor with
DEBUG logging enabled (-v=4).
- Configure any model selector with a picker (
max-score, random, or
weighted-random) and at least one candidate model.
- Send a single inference request that succeeds.
- Inspect the log line
"Completed running picker plugin" result.TargetModel
is {} despite the request succeeding and being routed correctly.
Anything else we need to know?:
Fix scope is observability only, no behavior change.
One suggested fix: add a MarshalJSON method to the unexported model type at
pkg/framework/interface/datalayer/model.go:34.
Anything else we need to know?:
Fix scope is observability-only, no behavior change. Suggested fix: add
a MarshalJSON method to the unexported model type at
pkg/framework/interface/datalayer/model.go:34 that projects the
model's name to an exported JSON key, e.g.:
func (m *model) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
Name string `json:"name"`
}{Name: m.name})
}
Fixes every current and future log site that serializes a
Model (bare, nested in PipelineRunResult, or inside a ScoredModel slice) without touching call sites.
Attributes intentionally excluded. AttributeMap (pkg/framework/interface/datalayer/attributemap.go:36) is itself an
interface backed by a struct with an unexported sync.Map field, so projecting attributes here would re-introduce the same {} problem one level down. Making attributes serialize correctly is a separate concern and should be a follow-up if/when a real use case surfaces.
Environment:
- Kubernetes version (use
kubectl version): Client Version: v1.34.1, Kustomize Version: v5.7.1, Server Version: v1.34.0
- llm-d-scheduler version (use
git describe --tags --dirty --always if you built from source, or specify the tag if you used a tagged version or image): 58cd88d-dirty built from source
What happened:
Structured log lines that include a
datalayer.Modelvalue render themodel as
{}, even when the model is actually populated. Example from alive DEBUG-level run:
{"level":"debug","caller":"modelselector/model_selector_pipeline.go:264", "msg":"Completed running picker plugin", "plugin":"max-score-picker/max-score-picker", "result" {"TargetModel":{}}}The picker returned a real, non-nil model. The nil check at
pkg/modelselector/modelselector.go:73would have failed the requestotherwise. Only the log rendering is empty.
Root cause:
datalayer.Modelis an interface (pkg/framework/interface/datalayer/model.go:19)and its only concrete implementation, the unexported
modelstruct atmodel.go:34, has unexported fields (name,attributes) and noMarshalJSONmethod.encoding/jsoncannot see unexported fields, soevery
Modelvalue marshals to{}silently.Affected log sites (all render the model as
{}):pkg/modelselector/model_selector_pipeline.go:264pkg/framework/plugins/modelselector/picker/maxscore/picker.go:71-72pkg/framework/plugins/modelselector/picker/random/picker.go:74-75pkg/framework/plugins/modelselector/picker/weightedrandom/picker.go:108-109What you expected to happen:
Log lines containing a
datalayer.Modelshould render the model's name(and, ideally, its attributes) rather than
{}. Debug logs are the primarysignal for triaging model-selection behavior. A silently empty payload
makes them worse than useless for that use case.
How to reproduce it (as minimally and precisely as possible):
DEBUGlogging enabled (-v=4).max-score,random, orweighted-random) and at least one candidate model."Completed running picker plugin"result.TargetModelis
{}despite the request succeeding and being routed correctly.Anything else we need to know?:
Fix scope is observability only, no behavior change.
One suggested fix: add a
MarshalJSONmethod to the unexportedmodeltype atpkg/framework/interface/datalayer/model.go:34.Anything else we need to know?:
Fix scope is observability-only, no behavior change. Suggested fix: add
a
MarshalJSONmethod to the unexportedmodeltype atpkg/framework/interface/datalayer/model.go:34that projects themodel's
nameto an exported JSON key, e.g.:Fixes every current and future log site that serializes a
Model (bare, nested in
PipelineRunResult, or inside aScoredModelslice) without touching call sites.Attributes intentionally excluded.
AttributeMap(pkg/framework/interface/datalayer/attributemap.go:36) is itself aninterface backed by a struct with an unexported
sync.Mapfield, so projecting attributes here would re-introduce the same{}problem one level down. Making attributes serialize correctly is a separate concern and should be a follow-up if/when a real use case surfaces.Environment:
kubectl version): Client Version: v1.34.1, Kustomize Version: v5.7.1, Server Version: v1.34.0git describe --tags --dirty --alwaysif you built from source, or specify the tag if you used a tagged version or image): 58cd88d-dirty built from source