Download tokenizer_config.json from bigscience-data/tokenizer_alpha_weight_NFKC: direct link, hf CLI and curl.
- Browser
- Download file 131 Bytes
-
https://huggingface.co/bigscience-data/tokenizer_alpha_weight_NFKC/resolve/main/tokenizer_config.json
- Command line
-
hf download hf://bigscience-data/tokenizer_alpha_weight_NFKC/tokenizer_config.json
-
curl -L -o tokenizer_config.json https://huggingface.co/bigscience-data/tokenizer_alpha_weight_NFKC/resolve/main/tokenizer_config.json
131 Bytes
| {"unk_token": "<unk>", "eos_token": "</s>", "bos_token": "<s>", "pad_token": "<pad>", "tokenizer_class": "PreTrainedTokenizerFast"} |