compactlm-5m / README.md

Commit History

  1. Fix over-stated quality claims: model is first-sentence-coherent then loops, not "fluent/on-topic" (#9)⁠

    9323e39
  2. Add version note: v2 replaces earlier 6.16M word-salad build

    Verified signature
    5dbd13e
  3. Add model card: CompactLM-5M from-scratch 4.9M-param GQA LLaMA

    Verified signature
    a6ec395

  1. Fix Results section to report the measured val loss/ppl from the shipped checkpoint's own eval (eval_fresh.json: 3.8719 / 48.03). The previous card cited 3.8775 / 48.30 / "step 20000", but the training run diverged to NaN at step 14300 and the log died at step 16000 — step 20000 was never reached. Also correct the eval filename reference (eval_shipped.json -> eval_fresh.json, the file actually in the repo). (#8)⁠

    23f2b03
  2. Fix: quote verbatim samples from the shipped eval, correct over-optimistic framing

    Verified signature
    186d75e
  3. Add model card (honest: real samples, final val 3.8775 / ppl 48.30).

    Verified signature
    c353fb6
  4. Fix card: replace hand-written sample sentences with real output from the shipped weights, and correct the capability claims to match what the model actually produces (#7)⁠

    ed33551
  5. Add model card (honest: arch, data, measured val PPL 48.03, 0/15 degenerate) (#6)⁠

    cafd02f
  6. Add CompactLM-5M: from-scratch LLaMA-style 6.2M-param English LM on fineweb-edu

    Verified signature
    da4f145