Front-loading study
Four web datasets pretrained with and without MixtureVitae's instruction and reasoning subset, then extended to 16K and given Tulu3 SFT and DPO.
2B • Updated • 9Note MixtureVitae web part, without front-loading · base
ali-elganzory/open-sci-ref-v0.02-1.7b-mixturevitae-noinstruct-300B-4096-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 17Note MixtureVitae web part, without front-loading · Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-mixturevitae-noinstruct-300B-4096-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 7Note MixtureVitae web part, without front-loading · Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-mixturevitae-noinstruct-300B-4096-longsft_16k
2B • Updated • 11Note MixtureVitae web part, without front-loading · extended to 16K
ali-elganzory/open-sci-ref-v0.02-1.7b-mixturevitae-noinstruct-300B-4096-longsft_16k-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 16Note MixtureVitae web part, without front-loading · 16K, Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-mixturevitae-noinstruct-300B-4096-longsft_16k-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 15Note MixtureVitae web part, without front-loading · 16K, Tulu3 DPO
ali-elganzory/1.7b-MixtureVitae-300BT-v1-decontaminated
Text Generation • 2B • Updated • 23Note MixtureVitae web part, with front-loading · base
ali-elganzory/1.7b-MixtureVitae-300BT-v1-decontaminated-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 9Note MixtureVitae web part, with front-loading · Tulu3 SFT
ali-elganzory/1.7b-MixtureVitae-300BT-v1-decontaminated-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 12Note MixtureVitae web part, with front-loading · Tulu3 DPO
ali-elganzory/1.7b-MixtureVitae-300BT-v1-decontaminated-16k
Text Generation • 2B • Updated • 17Note MixtureVitae web part, with front-loading · extended to 16K
ali-elganzory/1.7b-MixtureVitae-300BT-v1-decontaminated-16k-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 13Note MixtureVitae web part, with front-loading · 16K, Tulu3 SFT
ali-elganzory/1.7b-MixtureVitae-300BT-v1-decontaminated-16k-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 16Note MixtureVitae web part, with front-loading · 16K, Tulu3 DPO
ali-elganzory/1.7b-Comma0.1-300BT-WithChatTemplate
Text Generation • 2B • Updated • 11Note Comma0.1, without front-loading · base
ali-elganzory/1.7b-Comma0.1-300BT-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 9Note Comma0.1, without front-loading · Tulu3 SFT
ali-elganzory/1.7b-Comma0.1-300BT-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 6Note Comma0.1, without front-loading · Tulu3 DPO
ali-elganzory/1.7b-Comma0.1-300BT-longsft_16k
Text Generation • 2B • Updated • 29Note Comma0.1, without front-loading · extended to 16K
ali-elganzory/1.7b-Comma0.1-300BT-longsft_16k-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 25Note Comma0.1, without front-loading · 16K, Tulu3 SFT
ali-elganzory/1.7b-Comma0.1-300BT-longsft_16k-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 23Note Comma0.1, without front-loading · 16K, Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-comma-mv_reasoning-300B-4096
2B • Updated • 10Note Comma0.1, with front-loading · base
ali-elganzory/open-sci-ref-v0.02-1.7b-comma-mv_reasoning-300B-4096-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 62Note Comma0.1, with front-loading · Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-comma-mv_reasoning-300B-4096-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 18Note Comma0.1, with front-loading · Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-comma-mv_reasoning-300B-4096-longsft_16k
2B • Updated • 11Note Comma0.1, with front-loading · extended to 16K
ali-elganzory/open-sci-ref-v0.02-1.7b-comma-mv_reasoning-300B-4096-longsft_16k-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 28Note Comma0.1, with front-loading · 16K, Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-comma-mv_reasoning-300B-4096-longsft_16k-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 13Note Comma0.1, with front-loading · 16K, Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-300B-4096
Text Generation • 2B • Updated • 9Note Nemotron-CC-HQ, without front-loading · base
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-300B-4096-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 15Note Nemotron-CC-HQ, without front-loading · Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-300B-4096-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 19Note Nemotron-CC-HQ, without front-loading · Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-300B-4096-longsft_16k
2B • Updated • 10Note Nemotron-CC-HQ, without front-loading · extended to 16K
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-300B-4096-longsft_16k-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 22Note Nemotron-CC-HQ, without front-loading · 16K, Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-300B-4096-longsft_16k-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 20Note Nemotron-CC-HQ, without front-loading · 16K, Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-mv_reasoning-300B-4096
2B • Updated • 17Note Nemotron-CC-HQ, with front-loading · base
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-mv_reasoning-300B-4096-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 11Note Nemotron-CC-HQ, with front-loading · Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-mv_reasoning-300B-4096-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 11Note Nemotron-CC-HQ, with front-loading · Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-mv_reasoning-300B-4096-longsft_16k
2B • Updated • 12Note Nemotron-CC-HQ, with front-loading · extended to 16K
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-mv_reasoning-300B-4096-longsft_16k-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 20Note Nemotron-CC-HQ, with front-loading · 16K, Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-mv_reasoning-300B-4096-longsft_16k-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 14Note Nemotron-CC-HQ, with front-loading · 16K, Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-300B-4096
Text Generation • 2B • Updated • 9 • 1Note FineWeb-Edu, without front-loading · base
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-300B-4096-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 15Note FineWeb-Edu, without front-loading · Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-300B-4096-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 15Note FineWeb-Edu, without front-loading · Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-300B-4096-4096-longsft_16k
Text Generation • 2B • Updated • 15 • 1Note FineWeb-Edu, without front-loading · extended to 16K
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-300B-4096-longsft_16k-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 33Note FineWeb-Edu, without front-loading · 16K, Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-300B-4096-longsft_16k-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 21Note FineWeb-Edu, without front-loading · 16K, Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-mv_reasoning-300B-4096
2B • Updated • 11Note FineWeb-Edu, with front-loading · base
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-mv_reasoning-300B-4096-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 10Note FineWeb-Edu, with front-loading · Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-mv_reasoning-300B-4096-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 9Note FineWeb-Edu, with front-loading · Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-mv_reasoning-300B-4096-longsft_16k
2B • Updated • 12Note FineWeb-Edu, with front-loading · extended to 16K
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-mv_reasoning-300B-4096-longsft_16k-SFT-Tulu3-decontam
Text Generation • 2B • Updated • 15Note FineWeb-Edu, with front-loading · 16K, Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-mv_reasoning-300B-4096-longsft_16k-DPO-Tulu3-decontam
Text Generation • 2B • Updated • 10Note FineWeb-Edu, with front-loading · 16K, Tulu3 DPO
ontocord/1.7b-Comma0.1-300BT
2B • Updated • 13Note Comma0.1, without front-loading · base (language suite)
permissive-post-train/evaluation-records
Updated • 381Note The evaluation records of every checkpoint here