adhikjoshi commited on
Commit
fd53df3
·
verified ·
1 Parent(s): 5a32fe9

Add MiniMax-H3 Turbo V4 + RL (rl_s3) rank-128 LoRA and model card

Browse files
Files changed (2) hide show
  1. README.md +68 -0
  2. minimax_h3_turbo_v4_rl_s3.safetensors +3 -0
README.md ADDED
@@ -0,0 +1,68 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ license_name: minimax-h3-community-license-agreement
4
+ license_link: https://huggingface.co/MiniMaxAI/MiniMax-H3
5
+ base_model:
6
+ - MiniMaxAI/MiniMax-H3
7
+ - larryvrh/MiniMax-H3-Turbo-Lora
8
+ library_name: diffusers
9
+ pipeline_tag: text-to-video
10
+ tags:
11
+ - lora
12
+ - video-generation
13
+ - text-to-audio-video
14
+ - minimax-h3
15
+ - ref2va
16
+ - flow-dpo
17
+ ---
18
+
19
+ # MiniMax-H3 Turbo V4 + RL (rl_s3)
20
+
21
+ One LoRA file for **MiniMax-H3 Ref2VA** (`MiniMaxAI/MiniMax-H3`, `transformer_ref`) that combines:
22
+
23
+ 1. **Turbo LoRA V4** by larryvrh (`larryvrh/MiniMax-H3-Turbo-Lora`, `minimax_h3_turbo_v4_step600_ema.safetensors`, Apache-2.0): the 8-step distillation adapter.
24
+ 2. **rl_s3**: a rank-64 LoRA trained by ModelsLab on top of Turbo V4 with closed-loop, pure-ODE online Flow-DPO (best-of-N rollouts scored by in-RAM video/audio critics, paired against frozen V4 on the same seed, high-frequency KL lock to V4).
25
+
26
+ The two are stored as exact branches side by side, not merged: for every attention/MLP module `ΔW = [B_v4 | B_rl] · [A_v4 ; A_rl]` (rank 128), and V4's rank-16 `adaln_proj.linear` branches are kept as they are. Merging into BF16 base weights would round away most of these small deltas, so load the file as LoRA branches.
27
+
28
+ | File | Size | Layout |
29
+ | --- | --- | --- |
30
+ | `minimax_h3_turbo_v4_rl_s3.safetensors` | 1.40 GB (BF16) | ComfyUI / ai-toolkit keys (`blocks.N.attn.qkv_proj.lora_A.weight`, ...), same layout as Turbo V4. 208 attention/MLP modules at rank 128 (200 DiT-block modules = V4 ⊕ RL; 8 token-refiner modules = V4 zero-padded) + 51 AdaLN modules at rank 16 (V4 only). 518 BF16 tensors. |
31
+
32
+ ## How to run it
33
+
34
+ The recommended sampler is **E4a**: Turbo V4's 9-point shifted grid (`u = 1, .875, …, .125, 0`, shift 12 video / 3 audio) with one extra node at `u = 0.04` (`σ_video = 0.333`) and a 2nd-order Heun corrector on `[σ 0.632 → 0.333]`: 10 sigma points, 10 transformer forwards. Reference implementation for the NVIDIA Sol-H3 engine: `heun_tail.py` in the ModelsLab H3 server repo; the plain 8-forward Euler schedule also works.
35
+
36
+ Load it with any loader that accepts Turbo V4's file (same keys, larger rank). Sol-H3 / PEFT loaders read the rank from the tensors.
37
+
38
+ ## Evaluation (16 public clips, seed 11, ratios × the real clip)
39
+
40
+ | Arm | Forwards | Detail | Flow | Motion | Made-up speech |
41
+ | --- | :---: | :---: | :---: | :---: | :---: |
42
+ | Turbo V4, Euler | 8 | 1.43 | 0.87 | 0.94 | 1/7 |
43
+ | Turbo V4, E4a | 10 | 1.59 | 0.92 | 1.02 | 1/7 |
44
+ | **Turbo V4 + rl_s3, E4a** | 10 | **1.59** | **0.92** | **1.01** | 1/7 |
45
+
46
+ Honest attribution: on this public set almost all of the gain over Turbo V4 comes from the E4a sampler; rl_s3 matches V4 + E4a within noise on these metrics (closer to the real clip than V4 + E4a on Lab b for 11/16 clips and on saturation for 8/16). Measured with a BF16 reference pipeline, not with the MXFP8 production engine.
47
+
48
+ ## Training data and licences
49
+
50
+ The RL stages (rl_s0 → rl_s3) used short clips (only their cached latents and the prompts written for them) from:
51
+
52
+ | Source | Licence |
53
+ | --- | --- |
54
+ | DVIDS (dvidshub.net) | Public domain (US federal government work); DVIDS API terms |
55
+ | Wikimedia Commons | Per-file free licences |
56
+ | Netflix Open Content | CC BY 4.0 |
57
+ | UltraVideo (`APRIL-AIGC/UltraVideo`) | UltraVideo License (CC BY 4.0 with restrictions, non-commercial research) |
58
+ | Replay (`facebookresearch/replay_dataset`) | CC BY-NC 4.0 |
59
+ | Xiph test media: SVT | Only for developing, testing and presenting technology standards |
60
+ | Xiph test media: Netflix | CC BY-NC-ND 4.0 |
61
+
62
+ No media from these datasets is redistributed here. No customer or user data was used for training.
63
+
64
+ ## Licence and use
65
+
66
+ - The weights are a derivative of MiniMax-H3 and are released under the **MiniMax-H3 Community License Agreement**, including its territory and use restrictions. Read it before use.
67
+ - The Turbo V4 component keeps its Apache-2.0 licence and credit: larryvrh, `larryvrh/MiniMax-H3-Turbo-Lora` (revision `43a74557`).
68
+ - Several training sources are licensed for non-commercial research only (see the table). Check that your use is compatible with them.
minimax_h3_turbo_v4_rl_s3.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7ea1f6eee5de47e1a56c172d4a26efef90240dab2964df598eb899fb28b69799
3
+ size 1400083496