@wassname is the most recent version of the notebook broken? running it as is doesn't seem to ablate the refusals. The intervention generations still refuse, but the fully "merged" orthogonalized ones are more helpful.
wing lian PRO
winglian
AI & ML interests
None yet
Recent Activity
published a model about 2 hours ago
axolotl-ai-co/Qwen3.8-27B-Q5-GGUF updated a model about 2 hours ago
axolotl-ai-co/Qwen3.8-27B-Q5-GGUF published a model 1 day ago
axolotl-ai-co/Qwen3.8-27B-Q4-GGUFOrganizations
replied to their post almost 3 years ago
Sorry about that! Public now.
posted an update almost 3 years ago
Post
Full fine-tuning of Microsoft's Phi2 on a single 4090 is now supported in axolotl. Thanks to @abacaj and @vikhyatk for their help with gradient checkpointing and flash attention fixes.
alpaca finetune: openaccess-ai-collective/phi2-alpaca
wandb: https://wandb.ai/oaaic/phi2/runs/00pc4ugb?workspace=user-wing-lian
merged PR: https://github.com/OpenAccess-AI-Collective/axolotl/pull/1058
alpaca finetune: openaccess-ai-collective/phi2-alpaca
wandb: https://wandb.ai/oaaic/phi2/runs/00pc4ugb?workspace=user-wing-lian
merged PR: https://github.com/OpenAccess-AI-Collective/axolotl/pull/1058
- 8 replies