Rare find

tiktoken. tiktoken is a fast BPE tokeniser for use with OpenAI's models.

github.com/openai/tiktoken
by @openai

Problem

Audience

Keep exploring

Updates

No founder updates yet. Check back soon.