Rare find

tokenizer. Pure Go implementation of OpenAI's tiktoken tokenizer

github.com/tiktoken-go/tokenizer

by

@tiktoken-go

Problem

Audience

Keep exploring

Updates

No founder updates yet. Check back soon.