Rethinking tokenization: Crafting better tokenizers for large language models
Jinbiao Yang
Who it is from
- Max Planck Institute for Psycholinguisticsaffiliation declared by the authors, according to OpenAlex
Other papers from the same organizations
- iSign: A Benchmark for Indian Sign Language ProcessingMax Planck Institute for Psycholinguistics · 53 citations
- Deep neural networks and humans both benefit from compositional language structureMax Planck Institute for Psycholinguistics · 39 citations