I am pretty sure Jeff Dean (and the rest of the LLM world) is wrong about the lack of super words in the tokenizer.
Reach out at rrenaud@gmail.com .
Based in SF.
submissions | comments