Skip to content
This repository was archived by the owner on Nov 10, 2025. It is now read-only.
This repository was archived by the owner on Nov 10, 2025. It is now read-only.

vllm in-process handles sampling temperature incorrectly #194

Description

@jlquinn

Describe the bug

When using decoding method "sample", the temperature is not handled correctly.

  1. The default temperature is 0.0, instead of 1.0.
  2. If the user sets a temperature, the default of 0.0 will override the user value.

Platform

Please provide details about the environment you are using, including the following:

  • Interpreter version: 3.12.7
  • Library version: 4151c97

The bug is at vllm_in_proc.py:461

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions