Instructions to use tiny-random/flint with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use tiny-random/flint with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="tiny-random/flint")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("tiny-random/flint")
model = AutoModelForCausalLM.from_pretrained("tiny-random/flint", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use tiny-random/flint with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tiny-random/flint"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tiny-random/flint",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/tiny-random/flint

SGLang

How to use tiny-random/flint with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "tiny-random/flint" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tiny-random/flint",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "tiny-random/flint" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tiny-random/flint",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use tiny-random/flint with Docker Model Runner:
```
docker model run hf.co/tiny-random/flint
```

This tiny model is intended for debugging. It is randomly initialized using the configuration adapted from RecursiveSI/flint.

Structural comparison with RecursiveSI/flint:

Feature	Source	Tiny
Loading implementation	source SGLang plugin	same source plugin wheel
Stored precision	BF16	BF16
Vocabulary / embedding and LM-head ties	150,144 / untied	same
Layers / dense prefix	49 / layers 0-1	8 / layers 0-1
Attention cadence / RoPE-free layers	49-layer source schedule	first 8 source layers
Hidden size / Q and KV heads / head dimension	2,048 / 32 and 4 / 128	32 / 1 and 1 / 32
Routed experts / active experts	256 / 6	64 / 6
MoE / shared-expert widths	512 / 1,024	32 / 64
Sigmoid router, top-k renormalization, learned sinks, QK norms	present	same checkpoint schema

Direct checkpoint generation is required because Transformers does not implement Flint. Tensor names and shapes come from the pinned source safetensors headers and the source-provided flint_sglang loader wheel. The source plugin explicitly validates TP=1 only, so this model carries no TP=4 claim.

File path	Size
model.safetensors	21.9MB

Example usage:

SGLang (unverified)
The source Flint plugin validates one GPU per replica only; it explicitly marks tensor/expert/pipeline parallelism as unvalidated. The tiny checkpoint therefore does not claim TP=4 support.
Install the source-provided plugin wheel before this command. Its learned attention sinks, sigmoid-router top-k, shared expert, tool grammar and reasoning parser are not implemented by stock SGLang.

pip install ./sglang/flint_sglang-0.1.4-py3-none-any.whl
SGLANG_EXTERNAL_MODEL_PACKAGE=flint_sglang FLASHINFER_USE_CUDA_NORM=1 \
python -m sglang.launch_server \
  --model-path tiny-random/flint --served-model-name flint --tp-size 1 \
  --attention-backend fa3 --moe-runner-backend triton --enable-fp32-lm-head \
  --tool-call-parser flint --reasoning-parser flint --grammar-backend flint

Codes to create this repo:

Click to expand

import json
from pathlib import Path
from shutil import copyfile

import torch
from huggingface_hub import hf_hub_download
from safetensors.torch import save_file

source_model_id = "RecursiveSI/flint"
save_folder = "/tmp/tiny-random/flint"
source_revision = "2a247be0c0aec895ccb0ac7ca92229ec6df37288"
folder = Path(save_folder)
folder.mkdir(parents=True, exist_ok=True)
raw = json.loads(Path(hf_hub_download(source_model_id, "config.json", revision=source_revision)).read_text())
# Transformers has no Flint model class.  The source SGLang plugin is the
# authoritative executable schema, so create its public checkpoint tensors directly.
# The fixed 150,144-row untied embedding and LM head make hidden_size=32 the
# smallest practical BF16 geometry; one 32-wide attention head retains the
# required LLM head-dimension floor.
layers = 8
raw.update(
    hidden_size=32, head_dim=32, num_attention_heads=1, num_key_value_heads=1,
    intermediate_size=128, moe_intermediate_size=32, shared_expert_intermediate_size=64,
    num_experts=64, num_experts_per_tok=6, max_position_embeddings=4096,
    num_hidden_layers=layers,
    layer_types=raw["layer_types"][:layers],
    no_rope_layer_indices=[index for index in raw["no_rope_layer_indices"] if index < layers],
)
(folder / "config.json").write_text(json.dumps(raw, indent=2) + "\n", encoding="utf-8")
generator = torch.Generator(device="cpu").manual_seed(42)

def random_tensor(*shape: int) -> torch.Tensor:
    return torch.randn(shape, generator=generator, dtype=torch.bfloat16).mul_(0.02)

hidden = raw["hidden_size"]
head_dim = raw["head_dim"]
experts = raw["num_experts"]
state: dict[str, torch.Tensor] = {
    "model.embed_tokens.weight": random_tensor(raw["vocab_size"], hidden),
    "lm_head.weight": random_tensor(raw["vocab_size"], hidden),
    "model.norm.weight": torch.ones(hidden, dtype=torch.bfloat16),
}
dense_layers = set(raw["mlp_only_layers"])
for layer in range(raw["num_hidden_layers"]):
    prefix = f"model.layers.{layer}"
    state[f"{prefix}.input_layernorm.weight"] = torch.ones(hidden, dtype=torch.bfloat16)
    state[f"{prefix}.post_attention_layernorm.weight"] = torch.ones(hidden, dtype=torch.bfloat16)
    state[f"{prefix}.self_attn.q_proj.weight"] = random_tensor(hidden, hidden)
    state[f"{prefix}.self_attn.k_proj.weight"] = random_tensor(head_dim, hidden)
    state[f"{prefix}.self_attn.v_proj.weight"] = random_tensor(head_dim, hidden)
    state[f"{prefix}.self_attn.o_proj.weight"] = random_tensor(hidden, hidden)
    state[f"{prefix}.self_attn.q_norm.weight"] = torch.ones(head_dim, dtype=torch.bfloat16)
    state[f"{prefix}.self_attn.k_norm.weight"] = torch.ones(head_dim, dtype=torch.bfloat16)
    state[f"{prefix}.self_attn.softmax_offset"] = random_tensor(raw["num_attention_heads"])
    if layer in dense_layers:
        for projection in ("gate_proj", "up_proj"):
            state[f"{prefix}.mlp.{projection}.weight"] = random_tensor(raw["intermediate_size"], hidden)
        state[f"{prefix}.mlp.down_proj.weight"] = random_tensor(hidden, raw["intermediate_size"])
        continue
    state[f"{prefix}.mlp.gate.weight"] = random_tensor(experts, hidden)
    for projection in ("gate_proj", "up_proj"):
        state[f"{prefix}.mlp.shared_expert.{projection}.weight"] = random_tensor(
            raw["shared_expert_intermediate_size"], hidden
        )
    state[f"{prefix}.mlp.shared_expert.down_proj.weight"] = random_tensor(
        hidden, raw["shared_expert_intermediate_size"]
    )
    for expert in range(experts):
        expert_prefix = f"{prefix}.mlp.experts.{expert}"
        for projection in ("gate_proj", "up_proj"):
            state[f"{expert_prefix}.{projection}.weight"] = random_tensor(raw["moe_intermediate_size"], hidden)
        state[f"{expert_prefix}.down_proj.weight"] = random_tensor(hidden, raw["moe_intermediate_size"])
save_file(state, folder / "model.safetensors", metadata={"format": "pt"})
for filename in (
    "LICENSE", "chat_template.jinja", "generation_config.json", "tokenizer.json", "tokenizer_config.json",
    "sglang/flint_sglang-0.1.4-py3-none-any.whl", "sglang/serve_h200.sh", "sglang/README.md",
):
    destination = folder / filename
    destination.parent.mkdir(parents=True, exist_ok=True)
    copyfile(hf_hub_download(source_model_id, filename, revision=source_revision), destination)

Test environment:

huggingface_hub: 1.27.0
safetensors: 0.8.0
tokenizers: 0.23.0rc0
torch: 2.13.0+cu126
transformers: 5.15.0.dev0

Change log:

2026-10-23: Initial version.

This repo is generated by GPT-6.1-Sol

Downloads last month: 66

Safetensors

Model size

10.9M params

Tensor type

BF16

Model tree for tiny-random/flint

Base model

RecursiveSI/flint

Finetuned

(17)

this model

Collection including tiny-random/flint

tiny ramdom models

Collection

158 items • Updated about 22 hours ago • 11