Hugging Face's logo

flexitok
/
bpe_script_SEAS_16000

Indonesian
Vietnamese
tokenizer
bpe
flexitok
fineweb2
Model card Files
xet
 Community
bpe_script_SEAS_16000 / script_1

Commit History

  1. Upload script_1/flexitok--bpe_script_SEAS_16000_overlap.json with huggingface_hub

    gsaltintas
    Verified signature
    5bb2748

  2. Upload script_1/flexitok--bpe_script_SEAS_16000_vocab.json with huggingface_hub

    gsaltintas
    Verified signature
    e61849a

  3. Upload script_1/flexitok--bpe_script_SEAS_16000_super_mapping.json with huggingface_hub

    gsaltintas
    Verified signature
    e01a238

  1. Upload script_1/flexitok--bpe_script_SEAS_16000_overlap.json with huggingface_hub

    gsaltintas
    Verified signature
    5efb5f1

  2. Upload script_1/participating_tokenizers.json with huggingface_hub

    gsaltintas
    Verified signature
    61a4274

  3. Upload script_1/flexitok--bpe_script_SEAS_16000.yaml with huggingface_hub

    gsaltintas
    Verified signature
    3779e6b

  4. Upload script_1/flexitok--bpe_script_SEAS_16000_info.json with huggingface_hub

    gsaltintas
    Verified signature
    e7c0617

  5. Upload script_1/flexitok--bpe_script_SEAS_16000_vocab.json with huggingface_hub

    gsaltintas
    Verified signature
    083d4d3

  6. Upload script_1/flexitok--bpe_script_SEAS_16000_super_mapping.json with huggingface_hub

    gsaltintas
    Verified signature
    103ad38