Post
82
Releasing PhiloLM ( gustavecortal/PhiloLM), a 112M base model trained from scratch on open philosophy texts.
PhiloLM was trained on PhiloText ( gustavecortal/PhiloText): 0.5B tokens of open philosophy encyclopedias (SEP, IEP), public-domain books, reviews, textbooks and educational web pages.
Trained on (only!) 1.5B tokens in 15 hours on a $350 GPU (Intel Arc A770). PhiloLM builds on efficient methods: Muon optimizer, bigram embeddings, sliding-window attention, GQA.
PhiloLM is optimized for efficient inference, reaching ~500 tokens/s on a $150 CPU. Try it in your browser: https://gustavecortal.com/philosophy-lm/
PhiloLM was trained on PhiloText ( gustavecortal/PhiloText): 0.5B tokens of open philosophy encyclopedias (SEP, IEP), public-domain books, reviews, textbooks and educational web pages.
Trained on (only!) 1.5B tokens in 15 hours on a $350 GPU (Intel Arc A770). PhiloLM builds on efficient methods: Muon optimizer, bigram embeddings, sliding-window attention, GQA.
PhiloLM is optimized for efficient inference, reaching ~500 tokens/s on a $150 CPU. Try it in your browser: https://gustavecortal.com/philosophy-lm/