Skip to content

chore(repo): remove unused deepseek_v3_tokenizer offline package - #123

Merged
songdiyang merged 1 commit into
devfrom
songdiyang/remove-deepseek-tokenizer
Aug 16, 2026
Merged

chore(repo): remove unused deepseek_v3_tokenizer offline package#123
songdiyang merged 1 commit into
devfrom
songdiyang/remove-deepseek-tokenizer

Conversation

@songdiyang

Copy link
Copy Markdown
Collaborator

更新了哪些特性

  • 移除仓库中无活跃调用者的 DeepSeek tokenizer 离线包(deepseek_v3_tokenizer 目录,约 7.5MB)

变更内容

  • 删除 deepseek_v3_tokenizer/tokenizer.json(7.48MB)、tokenizer_config.json、deepseek_tokenizer.py
  • 背景:该目录仅被 aether-ai 的 DeepSeekTokenizer 运行时加载,其对外 API(AiClient::count_tokens / count_messages_tokens)全工作区无调用者;安装包也未打包此目录;设置页的 Token 统计来自 billing_history,不依赖该文件
  • 无任何 Rust 代码改动,不影响编译与测试

@songdiyang
songdiyang merged commit 5dd6de8 into dev Aug 16, 2026
2 checks passed
@songdiyang
songdiyang deleted the songdiyang/remove-deepseek-tokenizer branch August 16, 2026 02:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant