EngRandom_1.0

Eksperymentalny, ultra-mały model językowy wytrenowany od zera na własnym słowniku.

Opis

Model powstał za pomocą skryptu w Google Colab. Jest to zabawkowy GPT-2 (2 warstwy, 64 wymiary, kontekst 64 tokenów) przeznaczony głównie do zabawy i nauki.

Słownik (vocab)

hob, twee, dense, run, nuts, fever, fetch, rival, consider, occasionally, upchange, mauve, extricate, pass, pain, dilute, prune, stake, impregnate, oral, tree, holster, donor, ample, petition, mooch, exciting, sob, scan, deaden, runic, wily, lecture, taboo, breakfast, evoke, saver, teeth, void, jolt, grief, ring, black, livid, golem, nearly, imagine, baby, weary, mural, mushy, rank, mod, album, lion, zebra, own, pied, urinate, lambast, ambush, mothball, corner, rash, two, faster, mix, face, ticket, wean, successful, regard, belly, feel, disrobe, ruled, dread, fine, muddy, cat, salon, once, plow, macro, rasp, smoky, shiny, live, view, plan, prize, coerce, grass, mighty, lift, take, entire, rear, smile, round, hasty, comfort, tiger, plump, lane, enjoy, ruminate, glad, damn, rife, indicate, bleat, remount, clump, clock, usually, force, husky, mummy, growl, pony, jolly, line, fantasize, knowledge, spine, achieve, inseminate, tow, gummy, incommode, kill, valet, rosin, hazy, bared, summer, sewn, girl, search, freedom, reason, bet, testy, shed, tidy, tea, vague, skulk, world, retro, some, dead, snow, reset, structure, dab, carbonise, chart, wild, fighting, loud, gent, cross, kick, rocky, blag, tubby, crawl, have, rosy, dial, blear, strap, make, mock, habit, lunge, secede, ego, spumy, warm, raise, bar, pace, rubber, taken, paper, conquer, three, lefty, right, tub, carefully, nick, darn, stonewall, nice, cadet, mince, bated, shut, bumble, sing, job, picture, redeploy, cache, noted, raw, decorate, loamy, seal, lemur, below, fun, hoagy, peal, quick, ready, gate, string, slate, nanna, row, filch, choice, struggle, purloin, space, pat, gesture, lope, hops, recall, abet, pricy, mimer, arm, fated, promo, empty, antic, arrow, four, awash, natural, times, tape, beat, arms, sour, moony, mantle, fashion, showy, shawl, thrill, room, swing, ridge, parts, astir, has, fluid, diet, port, oaken, too, stopped, video, ink, able, frail, favor, catalyze, wacky, factor, limp, rib, zany, chuckle, lay, color, discard, replied, summon, fifth, jumbo, fabricate, fruit, pop, fracture, remix, low, fence, brand, men, basin, kinky, hump, evil, gawky, team, noise, liken, slew, elude, scandalize, tribe, lank, flex, farce, you, timer, shower, sugar, ion, bliss, instrument, dowel, nag, redo, quell, comfortable, fortify, task, elder, expiate, dip, mangy, plane, lent, till, rejoice, drape, cello, flavour, frank, cleft, crisp, tip, halo, television, agonize, wool, muggy, ham, aqua, palmy, found, produce, leach, doing, wits, forum, grain, legitimise, spoil, walk, indeed, clothes, soak, sixty, using, babble, hail, ordinary, shot, ten, barn, gulch, seat, during, acme, vial, herald, tomorrow, fudge, mice, lowly, goon, ace, rake
  • tokeny specjalne: [UNK], [PAD], <|endoftext|>

Dane treningowe

Wygenerowane losowe zdania ze słownika użytkownika (200 przykładów)

Architektura

  • n_layer = 2
  • n_embd = 64
  • n_head = 2
  • n_positions = 64
  • vocab_size = 387

Pliki GGUF (kwantyzacje)

  • (brak plików GGUF)

Użycie

Transformers (Python)

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("EngRandom_1.0")
tokenizer = AutoTokenizer.from_pretrained("EngRandom_1.0")

inputs = tokenizer("Miau", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=20)
print(tokenizer.decode(outputs[0]))

llama.cpp / Ollama / LM Studio

Po prostu wskaż wybrany plik .gguf.

Ograniczenia

  • Model jest bardzo mały – generuje głównie sekwencje ze swojego wąskiego słownika.
  • Nie nadaje się do normalnych zadań językowych.
  • Przeznaczony wyłącznie do eksperymentów i edukacji.

Licencja

MIT

Downloads last month
21
Safetensors
Model size
129k params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support