|
Download README.md from NLP-RISE/gptbase_het3_1000mb: direct link, hf CLI and curl.
- Browser
- Download file 1.54 kB
-
https://huggingface.co/NLP-RISE/gptbase_het3_1000mb/resolve/main/README.md
- Command line
-
hf download hf://NLP-RISE/gptbase_het3_1000mb/README.md
-
curl -L -H "Authorization: Bearer $HF_TOKEN" -o README.md https://huggingface.co/NLP-RISE/gptbase_het3_1000mb/resolve/main/README.md
1.54 kB
| language: | |
| - el | |
| - tr | |
| - et | |
| - fr | |
| - ar | |
| tags: | |
| - generated-from-wandb | |
| - gptbase | |
| metrics: | |
| - loss | |
| # gptbase_het3_1000mb | |
| <!-- collection-note:start --> | |
| GPT2 (base) trained on a balanced set of languages | |
| Balanced model training set (1000mb with byte premium scaling for each language) | |
| Balanced tokenizer training set (100mb with byte premium scaling for each language) | |
| <!-- collection-note:end --> | |
| <!-- wandb-aggregate:start --> | |
| ## Training results | |
| Based on **1** W&B runs named `gptbase_het3_1000mb` in [sjzahra-ri-se/huggingface](https://wandb.ai/sjzahra-ri-se/huggingface). | |
|  | |
| Loss: **1** run(s); warmup: **1**; GPU telemetry: **1** run(s). | |
| **Training hardware:** Trained on a single H100 GPU. | |
| ### Languages | |
| Greek, Turkish, Estonian, French, Arabic | |
| ### Training metadata | |
| | Parameter | Value | | |
| |---|---| | |
| | Runs included | 1 | | |
| | Training hardware | Trained on a single H100 GPU | | |
| | Total epochs | 9.99974 | | |
| | Total training steps | 385,460 | | |
| | Total training FLOPs | 6.4458e+18 | | |
| | Learning rate | 0.0001 | | |
| | Train batch size | 64 | | |
| | Eval batch size | 64 | | |
| | Gradient accumulation | 1 | | |
| | Weight decay | 0.01 | | |
| | Warmup steps | 38546 | | |
| | Warmup ratio | 0 | | |
| | Optimizer | adamw_torch_fused | | |
| | Scheduler | linear | | |
| | Max sequence length | 20 | | |
| | Precision | False | | |
| | Seed | 43 | | |
| | GPU utilization | mean 98.7%, peak 100.0% | | |
| W&B runs: [buh64hkk](https://wandb.ai/sjzahra-ri-se/huggingface/runs/buh64hkk) | |
| _Generated 2026-11-17T17:25:27+00:00._ | |
| <!-- wandb-aggregate:end --> | |