simpsons / tokenizer

Commit History

  1. Add tokenizer/vocab.json

    Verified signature
    25a57a9
  2. Add tokenizer/tokenizer_config.json

    Verified signature
    482ea37
  3. Add tokenizer/special_tokens_map.json

    Verified signature
    3ce6b7c
  4. Add tokenizer/merges.txt

    Verified signature
    3f89983