Train and run a small Llama 2 model from scratch on the TinyStories dataset.
- Iters vs Val Loss Learning Words and Grammar Visualised
- 105 Token Vocab
!cd llama2.c && python tinystories.py train_vocab --vocab_size=256
trainer_interface.cc(558) LOG(INFO) Alphabet size=102
Vocabulary size is smaller than required_chars. 256 vs 361.