J C
dark-pen
AI & ML interests
None yet
Recent Activity
updated a Space about 3 hours ago
dark-pen/quant-my-repo reacted to juiceb0xc0de's post with ๐ฅ about 9 hours ago
Dude, Where's My Update? I'll tell you where! ~97.6% of my BF16 parameter coordinates didn't move at all, and the ones that did overshot by ~1.33x.
It's nice to do research that doesn't end in disproving yourself once again and moving on to the next subject once in awhile.
Back to the topic, if you've ever wondered why most of your weights are basically ghosting you nearly every step when you store your weights at bf16, Dude, I Measured It.
https://huggingface.co/blog/juiceb0xc0de/intended-and-realized-updates-in-bf16-fine-tuning#dude-wheres-my-update upvoted a paper about 10 hours ago
Causal Foundation Models with Continuous TreatmentsOrganizations
