tokenizers-cpp. Universal cross-platform tokenizers binding to HF and sentencepiece

github.com/BBuf/tokenizers-cpp
by @BBuf

Problem

Audience

Keep exploring

Updates

No founder updates yet. Check back soon.