Datdanboi25 (Dan P)

THE SLM FRONTIER ADVANCES!

bench-labs/cagliostro-v3 just hit an Intelligence Index of 26.13 on the AxiomicLabs/Open_SLM_Leaderboard a 146M-param model trained completely from scratch on a single consumer GPU. That's 2nd place overall, and as far as I can tell, the most capable SLM trained on consumer hardware to date. Beating SmolLM-135m on 1/8th of the data is just silly levels of efficiency.

Big congrats to the @BenchLabs team and specifically @TobiasLogic !

6 replies

·

reacted to FlameF0X's post with 🔥 about 18 hours ago

Post

1506

Personally, I don't think bot accounts on Hugging Face are a good thing, as we don't know how many accounts are run by automated systems versus how many actual users there are. Dead Internet theory is already a thing.

12 replies

·

liked a model about 20 hours ago

SupraLabs/Supra2-IMG

Text-to-Image • Updated 1 day ago • 137 • 96

replied to Banaxi-Tech's post 1 day ago

@BananaMindBot train GPT-X3

liked 2 models 1 day ago

AlgoDriveAI/AIMO_EP_25

Text Generation • Updated 1 day ago • 2

AlgoDriveAI/Akkadian_English_DenseLLM_1B

Text Generation • Updated 10 days ago • 103 • 2

New activity in bench-labs/cagliostro-v3 1 day ago

What was the peak learning rate used?

#8 opened 1 day ago by

Datdanboi25

reacted to their post with ❤️ 1 day ago

Post

3360

THE SLM FRONTIER ADVANCES!

bench-labs/cagliostro-v3 just hit an Intelligence Index of 26.13 on the AxiomicLabs/Open_SLM_Leaderboard a 146M-param model trained completely from scratch on a single consumer GPU. That's 2nd place overall, and as far as I can tell, the most capable SLM trained on consumer hardware to date. Beating SmolLM-135m on 1/8th of the data is just silly levels of efficiency.

Big congrats to the @BenchLabs team and specifically @TobiasLogic !

6 replies

·

liked 2 models 2 days ago

#159 opened 2 days ago by

dawidmajewski

Update cagliostro-v3 to final weights (75B tokens)

2

#158 opened 2 days ago by

TobiasLogic

reacted to KlondikeDev's post with 😔 3 days ago

Post

188

Important Boris-2 news:

Boris-2 is 30B out of 200B tokens in, and it is severely behind its competitors in training.

We have determined the bug to be a configuration error. Boris-2 has been in training for ~1 week, and was projected to finish on November 3rd, 2026.

We are unfortunately going to restart training, with proper configuration.

The new projected finish date is ~15-18th of November.

We apologize for the delay.

17 replies

·

replied to KlondikeDev's post 3 days ago

You will NOT be forgiven for this delay!

reacted to TobiasLogic's post with 🔥 3 days ago

Post

3372

We’ve been cooking something new at Bench Labs.

Introducing Cagliostro-v3, our new 146M parameter language model trained completely from scratch.

The run isn’t even finished yet.

At the current checkpoint:

• 146M parameters
• 72.7B / 75B tokens trained
• 26.27 Open SLM Index
• 43.80 ArithMark-3
• Trained on a single RTX 5090
• ~90K to 103K tokens/sec during training
• ~9 days for the full run
• Apache 2.0

For some context, SmolLM2-135M scores 27.13 on the same Index after being trained on roughly 2 trillion tokens.

Cagliostro-v3 is currently at 26.27 with only ~72.7B.

That’s around 27x fewer training tokens.

The model also currently Hold the number 3rd spot for ArithMark-3, scoring 43.80

This wasn’t achieved by just throwing more tokens at the model. A huge part of v3 has been figuring out architecture, data mixture, and training dynamics at this scale.

The model uses a custom 30-layer decoder architecture with grouped-query attention and cross-head subspace attenuation, SwiGLU, RMSNorm, RoPE, tied embeddings, and a warmup-stable-decay training schedule.

During cooldown we also substantially shifted the data mixture toward higher-quality synthetic textbook and mathematics data, with the mathematics share increasing from 10% to 28%.

And everything is open.

The repository contains the training history with checkpoints pushed roughly every 30 minutes, so you can inspect how the model evolved throughout training rather than only seeing the final weights.

This is still a pre-final checkpoint. We have roughly 2.3B tokens left and the learning-rate cooldown is still running.

So 26.27 isn’t the final number.

Really excited to see where the last part of the run lands.

Cagliostro-v3:
bench-labs/cagliostro-v3
Built by Bench Labs.

Open SLM Leaderboard:
AxiomicLabs/Open_SLM_Leaderboard

8 replies

·

reacted to their post with 🔥 3 days ago

Post

3360

THE SLM FRONTIER ADVANCES!

bench-labs/cagliostro-v3 just hit an Intelligence Index of 26.13 on the AxiomicLabs/Open_SLM_Leaderboard a 146M-param model trained completely from scratch on a single consumer GPU. That's 2nd place overall, and as far as I can tell, the most capable SLM trained on consumer hardware to date. Beating SmolLM-135m on 1/8th of the data is just silly levels of efficiency.

Big congrats to the @BenchLabs team and specifically @TobiasLogic !

6 replies

·

replied to their post 3 days ago

Well deserved, congrats on the incredible model!

Dan P

AI & ML interests

Recent Activity

Organizations

[BUG] Kiyo is broken

GPT-X3

Add <1M filter

SupraLabs/Supra2-IMG

AlgoDriveAI/AIMO_EP_25

AlgoDriveAI/Akkadian_English_DenseLLM_1B

What was the peak learning rate used?

KlondikeDev/Boris-2-0917

KlondikeDev/Boris-2-0907

Add Pollock 1.5 (127.57M, from scratch, Index 12.59)

Update cagliostro-v3 to final weights (75B tokens)

Dan P

AI & ML interests

Recent Activity

Organizations

Datdanboi25's activity

[BUG] Kiyo is broken

GPT-X3

Add <1M filter

What was the peak learning rate used?

Add Pollock 1.5 (127.57M, from scratch, Index 12.59)

Update cagliostro-v3 to final weights (75B tokens)