Let's build the GPT Tokenizer

Name: Let's build the GPT Tokenizer
Uploaded: 2024-02-20T17:11:35Z
Channel: Andrej Karpathy
Description: The Tokenizer is a necessary and pervasive component of Large Language Models (LLMs), where it translates between strings and tokens (text chunks). To

1.2M views · 2 years ago · 2:13:35

Andrej Karpathy

Educational creator

Feb 20, 2024 · 1.2M views

The Tokenizer is a necessary and pervasive component of Large Language Models (LLMs), where it translates between strings and tokens (text chunks). To