LlamaRoboticsTokenizer / tokenizer.json

Commit History

  1. Robotics/routing tags packed into reserved slots (vocab 128256) + <think>/<tool_call>/<tool_response> tokens

    Verified signature
    dd9c19b

  1. updated upload of JiRackTokenizer with improvement for Robotics and Routing tags

    Verified signature
    9fabd23

  1. Initial upload of JiRackTokenizer

    Verified signature
    3c89c4c