remove root README
Browse files
README.md
DELETED
|
@@ -1,58 +0,0 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
-
tags:
|
| 4 |
-
- code
|
| 5 |
-
- latent-reasoning
|
| 6 |
-
- codi
|
| 7 |
-
- experiment-archive
|
| 8 |
-
---
|
| 9 |
-
|
| 10 |
-
# codi-trace — checkpoint archive
|
| 11 |
-
|
| 12 |
-
CODI latent-reasoning distillation 实验的 **checkpoint 备份**(非成果发布)。学生模型在每个执行帧把
|
| 13 |
-
`$LOCALS` 状态换成一个递归 latent block(`<|latent_start|>` + latent steps + `<|latent_end|>`),
|
| 14 |
-
从 Stage-1 SFT 模型蒸馏而来。基座为 Qwen2.5-Coder-1.5B/3B。
|
| 15 |
-
|
| 16 |
-
> ⚠️ 这些 **不是** 标准 `from_pretrained` 权重。`pytorch_model.bin` 是训练用 `CodiModel` 的
|
| 17 |
-
> `state_dict`(键带 `model.` / `prj.` 前缀),需用随仓附带的 `code/` 加载。
|
| 18 |
-
|
| 19 |
-
## 目录结构
|
| 20 |
-
|
| 21 |
-
```
|
| 22 |
-
README.md
|
| 23 |
-
code/ # 冻结的加载代码快照(只读),与这批权重对应
|
| 24 |
-
checkpoints/<run>/checkpoint-<step>/
|
| 25 |
-
pytorch_model.bin # CodiModel state_dict(含 model.* 与 prj.*)
|
| 26 |
-
thought_projector.pt # 投影器权重(亦含于 pytorch_model.bin)
|
| 27 |
-
config.json # 基座 arch 配置
|
| 28 |
-
tokenizer.json, merges.txt, vocab.json, tokenizer_config.json,
|
| 29 |
-
special_tokens_map.json, added_tokens.json, chat_template.jinja
|
| 30 |
-
trainer_state.json # 训练 loss 历史
|
| 31 |
-
```
|
| 32 |
-
|
| 33 |
-
训练恢复用文件(`optimizer.pt` / `scheduler.pt` / `rng_state_*.pth` / `training_args.bin`)**未包含**,
|
| 34 |
-
故无法精确 resume,仅供加载推理 / 评测。
|
| 35 |
-
|
| 36 |
-
## run 命名图例
|
| 37 |
-
|
| 38 |
-
| 片段 | 含义 |
|
| 39 |
-
|---|---|
|
| 40 |
-
| `sft…` | Stage-1 SFT(显式 trace,teacher) |
|
| 41 |
-
| `codi<size>_a<α>_b<β>_g<γ>_ls<n>` | 多帧 CODI:α/β/γ = teacher/student/KD 损失权重,ls = latent_steps |
|
| 42 |
-
| `…_ss<p>` | scheduled sampling 概率 p |
|
| 43 |
-
| `codi_frozen_<size>_<hidden\|logit>` | 冻结 SFT teacher;KD 对齐最后层 hidden 或 logit |
|
| 44 |
-
| `codi-single-<size>` | 单 block CODI(arXiv 2502.21074 忠实版) |
|
| 45 |
-
|
| 46 |
-
## 加载
|
| 47 |
-
|
| 48 |
-
```python
|
| 49 |
-
import sys; sys.path.insert(0, "code")
|
| 50 |
-
from eval.eval_cruxeval_codi import load_codi
|
| 51 |
-
|
| 52 |
-
# latent_steps 要与该 run 名里的 ls<n> 一致(默认 1)
|
| 53 |
-
tok, ids, codi = load_codi(
|
| 54 |
-
"checkpoints/codi_frozen_3b_hidden/checkpoint-1000", latent_steps=1, dev=0)
|
| 55 |
-
```
|
| 56 |
-
|
| 57 |
-
依赖:`torch`、`transformers`(`wandb` 仅训练时惰性需要,加载无需)。CRUXEval-O 评测见
|
| 58 |
-
`code/eval/eval_cruxeval_codi.py`。
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|