cannot import name 'LogitsWarper' from 'transformers.generation.logits_process'

[Kenwwww]

where is the requirements？？？？

[musab-mk]

Hi, could you clarify which command did you use and got this error?

[Kenwwww]

hello，i just run the commend as readme.  python3 vllm...   i think it's something wrong with the env. is this repo fit the latest vllm version?   i just pull the model ,pip install vllm, and run.

…

---Original--- From: "Musab ***@***.***> Date: Wed, Feb 19, 2025 22:56 PM To: ***@***.***>; Cc: ***@***.******@***.***>; Subject: Re: [MeetKai/functionary] cannot import name 'LogitsWarper' from'transformers.generation.logits_process' (Issue #312) Hi, could you clarify which command did you use and got this error? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***> musab-mk left a comment (MeetKai/functionary#312) Hi, could you clarify which command did you use and got this error? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[Kenwwww]

and if i want to use original vllm，insted of server_vllm.py，what should i do?

…

---Original--- From: "Musab ***@***.***> Date: Wed, Feb 19, 2025 22:56 PM To: ***@***.***>; Cc: ***@***.******@***.***>; Subject: Re: [MeetKai/functionary] cannot import name 'LogitsWarper' from'transformers.generation.logits_process' (Issue #312) Hi, could you clarify which command did you use and got this error? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***> musab-mk left a comment (MeetKai/functionary#312) Hi, could you clarify which command did you use and got this error? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[alicehau]

I am having this same issue when I run python3 server_sglang.py --model-path "meetkai/functionary-v4r-small-preview" --host 0.0.0.0 --port 8000 --context-length 8192

I get the following even if I have transformers installed:

Traceback (most recent call last):
File "/home/ahau/Goose/functionary/functionary-original/.venv/lib/python3.12/site-packages/lmformatenforcer/integrations/transformers.py", line 5, in
from transformers.generation.logits_process import LogitsWarper, PrefixConstrainedLogitsProcessor
ImportError: cannot import name 'LogitsWarper' from 'transformers.generation.logits_process' (/home/ahau/Goose/functionary/functionary-original/.venv/lib/python3.12/site-packages/transformers/generation/logits_process.py). Did you mean: 'EtaLogitsWarper'?

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/ahau/Goose/functionary/functionary-original/server_sglang.py", line 61, in
from functionary.sglang_inference import v1_chat_completions
File "/home/ahau/Goose/functionary/functionary-original/functionary/sglang_inference.py", line 40, in
from functionary.inference_stream import generate_openai_format_from_stream_async
File "/home/ahau/Goose/functionary/functionary-original/functionary/inference_stream.py", line 15, in
from functionary.inference import prepare_messages_for_inference
File "/home/ahau/Goose/functionary/functionary-original/functionary/inference.py", line 5, in
from lmformatenforcer.integrations.vllm import build_vllm_logits_processor
File "/home/ahau/Goose/functionary/functionary-original/.venv/lib/python3.12/site-packages/lmformatenforcer/integrations/vllm.py", line 8, in
from lmformatenforcer.integrations.transformers import build_token_enforcer_tokenizer_data
File "/home/ahau/Goose/functionary/functionary-original/.venv/lib/python3.12/site-packages/lmformatenforcer/integrations/transformers.py", line 8, in
raise ImportError('transformers is not installed. Please install it with "pip install transformers[torch]"')
ImportError: transformers is not installed. Please install it with "pip install transformers[torch]"

[alicehau]

Ah I think that you need to version lock to transformers[torch]==4.45.2

[Kenwwww]

try to downgrade transformers，i succeed. cuz this repo is too old.

…

---Original--- From: "Alice ***@***.***> Date: Tue, Feb 25, 2025 10:40 AM To: ***@***.***>; Cc: ***@***.******@***.***>; Subject: Re: [MeetKai/functionary] cannot import name 'LogitsWarper' from'transformers.generation.logits_process' (Issue #312) I am having this same issue when I run python3 server_sglang.py --model-path "meetkai/functionary-v4r-small-preview" --host 0.0.0.0 --port 8000 --context-length 8192 I get the following even if I have transformers installed: Traceback (most recent call last): File "/home/ahau/Goose/functionary/functionary-original/.venv/lib/python3.12/site-packages/lmformatenforcer/integrations/transformers.py", line 5, in from transformers.generation.logits_process import LogitsWarper, PrefixConstrainedLogitsProcessor ImportError: cannot import name 'LogitsWarper' from 'transformers.generation.logits_process' (/home/ahau/Goose/functionary/functionary-original/.venv/lib/python3.12/site-packages/transformers/generation/logits_process.py). Did you mean: 'EtaLogitsWarper'? During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/ahau/Goose/functionary/functionary-original/server_sglang.py", line 61, in from functionary.sglang_inference import v1_chat_completions File "/home/ahau/Goose/functionary/functionary-original/functionary/sglang_inference.py", line 40, in from functionary.inference_stream import generate_openai_format_from_stream_async File "/home/ahau/Goose/functionary/functionary-original/functionary/inference_stream.py", line 15, in from functionary.inference import prepare_messages_for_inference File "/home/ahau/Goose/functionary/functionary-original/functionary/inference.py", line 5, in from lmformatenforcer.integrations.vllm import build_vllm_logits_processor File "/home/ahau/Goose/functionary/functionary-original/.venv/lib/python3.12/site-packages/lmformatenforcer/integrations/vllm.py", line 8, in from lmformatenforcer.integrations.transformers import build_token_enforcer_tokenizer_data File "/home/ahau/Goose/functionary/functionary-original/.venv/lib/python3.12/site-packages/lmformatenforcer/integrations/transformers.py", line 8, in raise ImportError('transformers is not installed. Please install it with "pip install transformers[torch]"') ImportError: transformers is not installed. Please install it with "pip install transformers[torch]" — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***> alicehau left a comment (MeetKai/functionary#312) I am having this same issue when I run python3 server_sglang.py --model-path "meetkai/functionary-v4r-small-preview" --host 0.0.0.0 --port 8000 --context-length 8192 I get the following even if I have transformers installed: Traceback (most recent call last): File "/home/ahau/Goose/functionary/functionary-original/.venv/lib/python3.12/site-packages/lmformatenforcer/integrations/transformers.py", line 5, in from transformers.generation.logits_process import LogitsWarper, PrefixConstrainedLogitsProcessor ImportError: cannot import name 'LogitsWarper' from 'transformers.generation.logits_process' (/home/ahau/Goose/functionary/functionary-original/.venv/lib/python3.12/site-packages/transformers/generation/logits_process.py). Did you mean: 'EtaLogitsWarper'? During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/ahau/Goose/functionary/functionary-original/server_sglang.py", line 61, in from functionary.sglang_inference import v1_chat_completions File "/home/ahau/Goose/functionary/functionary-original/functionary/sglang_inference.py", line 40, in from functionary.inference_stream import generate_openai_format_from_stream_async File "/home/ahau/Goose/functionary/functionary-original/functionary/inference_stream.py", line 15, in from functionary.inference import prepare_messages_for_inference File "/home/ahau/Goose/functionary/functionary-original/functionary/inference.py", line 5, in from lmformatenforcer.integrations.vllm import build_vllm_logits_processor File "/home/ahau/Goose/functionary/functionary-original/.venv/lib/python3.12/site-packages/lmformatenforcer/integrations/vllm.py", line 8, in from lmformatenforcer.integrations.transformers import build_token_enforcer_tokenizer_data File "/home/ahau/Goose/functionary/functionary-original/.venv/lib/python3.12/site-packages/lmformatenforcer/integrations/transformers.py", line 8, in raise ImportError('transformers is not installed. Please install it with "pip install transformers[torch]"') ImportError: transformers is not installed. Please install it with "pip install transformers[torch]" — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[jeffrey-fong]

@Kenwwww Hi, please follow the installation instructions here to install the correct dependencies versions.

@alicehau Hi, #313 should fix your problem once merged. It refactors some deprecated features. Nevertheless, please follow the installation instructions using the link above too to ensure that you install the correct dependencies versions.

Thank you!

[alicehau]

Cool! @jeffrey-fong why are you removing grammar sampling out of curiosity?

[musab-mk]

During our tests, it improves the performance only in some cases, so it's not a huge deal. The biggest accuracy changing factor is the model itself.

Also, prompt template updates requiring another refactor of grammar sampling, and it breaks down every time vllm updates the engine. So we would rather not spend too much time on maintaining it.

[alicehau]

Makes sense. As FYI I'm also not able to run the sglang server even after following those installation instructions exactly. Getting more import errors:

ImportError: cannot import name 'PILImageResampling' from 'transformers.image_utils' (/home/ahau/functionary-venv/lib/python3.12/site-packages/transformers/image_utils.py)

Will try the latest once the grammar sampling is removed to see if that fixes anything

[jeffrey-fong]

The PR is merged. Can you try again? If the same problem persists, can you follow this to install the correct dependencies versions?