Skip to content

Releases: tiktoken-go/tokenizer

v0.8.1

Choose a tag to compare

@bluescreen10 bluescreen10 released this 17 Jul 16:07

Bugfix with generated regular expressions

Added GPT-5 Support

Choose a tag to compare

@bluescreen10 bluescreen10 released this 08 Aug 21:09
629192d
Merge pull request #21 from ando-masaki/gpt-5-fix

Added GPT-5 models

v0.6.2 Revert the last commit

Choose a tag to compare

@bluescreen10 bluescreen10 released this 09 May 22:25

Full Changelog: v0.6.1...v0.6.2

v0.6.1

Choose a tag to compare

@bluescreen10 bluescreen10 released this 27 Mar 14:38

🏎️ Small performance improvements

v0.6.0

Choose a tag to compare

@bluescreen10 bluescreen10 released this 07 Mar 14:52

Support for O3-mini model

Full Changelog: v0.5.1...v0.6.0

v0.5.1

Choose a tag to compare

@bluescreen10 bluescreen10 released this 25 Feb 15:57

Full Changelog: v0.5.0...v0.5.1

v0.5.0

Choose a tag to compare

@bluescreen10 bluescreen10 released this 17 Feb 20:46
8ac6fdf

What's Changed

New Contributors

Full Changelog: v0.4.0...v0.5.0

v0.4.0

Choose a tag to compare

@bluescreen10 bluescreen10 released this 12 Feb 23:00
03e0e5f

What's Changed

  • pre-compile regexp2 with regexp2cg by @terwey in #11

New Contributors

Full Changelog: v0.3.0...v0.4.0

Added support for the latest OpenAI model

Choose a tag to compare

@bluescreen10 bluescreen10 released this 28 Jan 18:45
025023d
Merge pull request #12 from DenisPalnitsky/chatgpt-4o-support

Add support of chatgpt-4o-latest model

Support for O1 models

Choose a tag to compare

@bluescreen10 bluescreen10 released this 22 Nov 22:16
2ad5811
Merge pull request #10 from ando-masaki/o1-preview-and-mini-fix

Added o1-preview and o1-mini