Skip to content

[Bug] datalayer.Model serializes as {} in structured logs in picker/selector #280

Description

@davidbreitgand

What happened:

Structured log lines that include a datalayer.Model value render the
model as {}, even when the model is actually populated. Example from a
live DEBUG-level run:

{"level":"debug","caller":"modelselector/model_selector_pipeline.go:264", "msg":"Completed running picker plugin", "plugin":"max-score-picker/max-score-picker", "result" {"TargetModel":{}}}

The picker returned a real, non-nil model. The nil check at
pkg/modelselector/modelselector.go:73 would have failed the request
otherwise. Only the log rendering is empty.

Root cause: datalayer.Model is an interface (pkg/framework/interface/datalayer/model.go:19)
and its only concrete implementation, the unexported model struct at model.go:34, has unexported fields (name, attributes) and no MarshalJSON method. encoding/json cannot see unexported fields, so
every Model value marshals to {} silently.

Affected log sites (all render the model as {}):

  • pkg/modelselector/model_selector_pipeline.go:264
  • pkg/framework/plugins/modelselector/picker/maxscore/picker.go:71-72
  • pkg/framework/plugins/modelselector/picker/random/picker.go:74-75
  • pkg/framework/plugins/modelselector/picker/weightedrandom/picker.go:108-109

What you expected to happen:

Log lines containing a datalayer.Model should render the model's name
(and, ideally, its attributes) rather than {}. Debug logs are the primary
signal for triaging model-selection behavior. A silently empty payload
makes them worse than useless for that use case.

How to reproduce it (as minimally and precisely as possible):

  1. Run the processor with DEBUG logging enabled (-v=4).
  2. Configure any model selector with a picker (max-score, random, or
    weighted-random) and at least one candidate model.
  3. Send a single inference request that succeeds.
  4. Inspect the log line "Completed running picker plugin" result.TargetModel
    is {} despite the request succeeding and being routed correctly.

Anything else we need to know?:

Fix scope is observability only, no behavior change.

One suggested fix: add a MarshalJSON method to the unexported model type at
pkg/framework/interface/datalayer/model.go:34.
Anything else we need to know?:

Fix scope is observability-only, no behavior change. Suggested fix: add
a MarshalJSON method to the unexported model type at
pkg/framework/interface/datalayer/model.go:34 that projects the
model's name to an exported JSON key, e.g.:

func (m *model) MarshalJSON() ([]byte, error) {
    return json.Marshal(struct {
        Name string `json:"name"`
    }{Name: m.name})
}

Fixes every current and future log site that serializes a
Model (bare, nested in PipelineRunResult, or inside a ScoredModel slice) without touching call sites.

Attributes intentionally excluded. AttributeMap (pkg/framework/interface/datalayer/attributemap.go:36) is itself an
interface backed by a struct with an unexported sync.Map field, so projecting attributes here would re-introduce the same {} problem one level down. Making attributes serialize correctly is a separate concern and should be a follow-up if/when a real use case surfaces.

Environment:

  • Kubernetes version (use kubectl version): Client Version: v1.34.1, Kustomize Version: v5.7.1, Server Version: v1.34.0
  • llm-d-scheduler version (use git describe --tags --dirty --always if you built from source, or specify the tag if you used a tagged version or image): 58cd88d-dirty built from source

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

kind/bugCategorizes issue or PR as related to a bug.needs-triageIndicates an issue or PR lacks a triage label and requires one.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions