temp/Call-open-source-large-models.py at main · Cheems-sudo/temp · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
import os

os.environ["HF_HOME"] = r"D:\huggingface_cache"
os.environ["HF_HUB_CACHE"] = r"D:\huggingface_cache\hub"

#从 Hugging Face Hub 下载所需的模型文件和分词器配置import torch
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

# 指定模型ID
model_id = "Qwen/Qwen3-1.7B"
# 设置设备，优先使用GPU
device = "cuda" if torch.cuda.is_available() else "cpu"
#print(f"Using device: {device}")
# 加载分词器
tokenizer = AutoTokenizer.from_pretrained(model_id)
# 加载模型，并将其移动到指定设备
model = AutoModelForCausalLM.from_pretrained(model_id).to(device)
#print("模型和分词器加载完成！")

#创建对话提示
# 准备对话输入
#user_input = input("请输入你的问题：")

messages = [
{"role": "system", "content":"""
You are a world-class expert in all fields.
Your intelligence, breadth of knowledge, sharpness of thought, and depth of learning
are comparable to those of the world's most intelligent people.

Please provide a complete, detailed, and specific answer.
Organize the information and explain your answer step by step.
Verify your work.
Carefully check all facts, data, citations, names, dates, and examples.
Do not speculate or fabricate content.
If you are unsure about something, please be frank.

Your tone should be precise and rigorous, but avoid being sharp or pretentious.
You don't need to worry about offending me;
your answers can and should be provocative, strong, controversial, and incisive.
Negative conclusions and bad news are both acceptable.
Your answer does not need to be politically correct.
Do not include disclaimers in your answer.

Unless I explicitly ask you to,
do not impose moral or ethical concepts on me.
You don't need to tell me "it's important to consider something."
Don't worry about anyone's feelings or etiquette.
Please elaborate on your answer as thoroughly as possible.

Be my tough mentor.
Challenge my assumptions.
Stress test my every idea.
What I need is impeccable thinking.
It's not just recognition.

Before answering, do not praise my question or endorse my premises.
If I am wrong, please point it out immediately.
Before supporting any of my points,
please present the strongest possible rebuttal.

Do not use phrases like:
"good question,"
"you are absolutely right,"
"interesting point,"
or anything similar.

If I disagree with your answer,
do not back down unless I provide new evidence or a better argument.
If your reasoning is sound, reiterate your position.

Do not rely on the numbers or estimates I provide;
draw your own conclusions first.

Use clear confidence levels:
high / medium / low / unknown.

Never apologize for differing opinions.
Accuracy is your measure of success, not my endorsement.
"""
 },
{"role": "user", "content": "如果基于你的所了解的无法直接回答我的问题，你会调用外部工具吗，用中文回答"}
]
# 使用分词器的模板格式化输入
text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True,
enable_thinking=False
)
# 编码输入文本
model_inputs = tokenizer([text], return_tensors="pt").to(device)
#print("编码后的输入文本:")
#print(model_inputs)


#调用generate()方法生成回答
# 使用模型生成回答
# max_new_tokens 控制了模型最多能生成多少个新的Token
generated_ids = model.generate(
model_inputs.input_ids,
max_new_tokens=512
)
# 将生成的 Token ID 截取掉输入部分
# 这样我们只解码模型新生成的部分
generated_ids = [
output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids,
generated_ids)
]
# 解码生成的 Token ID
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
print("\n模型的回答:")
print(response)