Arabic
arabic
tokenizer
morphology
nlp
dialect
df-arc / tokenizer.model

Commit History

  1. vocab_size Increased from 64000 to 128000

    Verified signature
    34fc84b

  1. Upload custom Unigram tokenizer (v1)

    Verified signature
    ea23987