## Describe the bug When using decoding method "sample", the temperature is not handled correctly. 1) The default temperature is 0.0, instead of 1.0. 2) If the user sets a temperature, the default of 0.0 will override the user value. ## Platform Please provide details about the environment you are using, including the following: - Interpreter version: 3.12.7 - Library version: 4151c972f0e3ef2d9c6278534054c9989f071df4 The bug is at vllm_in_proc.py:461
Describe the bug
When using decoding method "sample", the temperature is not handled correctly.
Platform
Please provide details about the environment you are using, including the following:
The bug is at vllm_in_proc.py:461