Instructions to use ModelsLab/MiniMax-H3-Turbo-RL with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use ModelsLab/MiniMax-H3-Turbo-RL with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import export_to_video # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("MiniMaxAI/MiniMax-H3", dtype=torch.bfloat16, device_map="cuda") pipe.load_lora_weights("ModelsLab/MiniMax-H3-Turbo-RL") prompt = "A man with short gray hair plays a red electric guitar." output = pipe(prompt=prompt).frames[0] export_to_video(output, "output.mp4") - Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
Add MiniMax-H3 Turbo V4 + RL (rl_s3) rank-128 LoRA and model card
Browse files- README.md +68 -0
- minimax_h3_turbo_v4_rl_s3.safetensors +3 -0
README.md
ADDED
|
@@ -0,0 +1,68 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: minimax-h3-community-license-agreement
|
| 4 |
+
license_link: https://huggingface.co/MiniMaxAI/MiniMax-H3
|
| 5 |
+
base_model:
|
| 6 |
+
- MiniMaxAI/MiniMax-H3
|
| 7 |
+
- larryvrh/MiniMax-H3-Turbo-Lora
|
| 8 |
+
library_name: diffusers
|
| 9 |
+
pipeline_tag: text-to-video
|
| 10 |
+
tags:
|
| 11 |
+
- lora
|
| 12 |
+
- video-generation
|
| 13 |
+
- text-to-audio-video
|
| 14 |
+
- minimax-h3
|
| 15 |
+
- ref2va
|
| 16 |
+
- flow-dpo
|
| 17 |
+
---
|
| 18 |
+
|
| 19 |
+
# MiniMax-H3 Turbo V4 + RL (rl_s3)
|
| 20 |
+
|
| 21 |
+
One LoRA file for **MiniMax-H3 Ref2VA** (`MiniMaxAI/MiniMax-H3`, `transformer_ref`) that combines:
|
| 22 |
+
|
| 23 |
+
1. **Turbo LoRA V4** by larryvrh (`larryvrh/MiniMax-H3-Turbo-Lora`, `minimax_h3_turbo_v4_step600_ema.safetensors`, Apache-2.0): the 8-step distillation adapter.
|
| 24 |
+
2. **rl_s3**: a rank-64 LoRA trained by ModelsLab on top of Turbo V4 with closed-loop, pure-ODE online Flow-DPO (best-of-N rollouts scored by in-RAM video/audio critics, paired against frozen V4 on the same seed, high-frequency KL lock to V4).
|
| 25 |
+
|
| 26 |
+
The two are stored as exact branches side by side, not merged: for every attention/MLP module `ΔW = [B_v4 | B_rl] · [A_v4 ; A_rl]` (rank 128), and V4's rank-16 `adaln_proj.linear` branches are kept as they are. Merging into BF16 base weights would round away most of these small deltas, so load the file as LoRA branches.
|
| 27 |
+
|
| 28 |
+
| File | Size | Layout |
|
| 29 |
+
| --- | --- | --- |
|
| 30 |
+
| `minimax_h3_turbo_v4_rl_s3.safetensors` | 1.40 GB (BF16) | ComfyUI / ai-toolkit keys (`blocks.N.attn.qkv_proj.lora_A.weight`, ...), same layout as Turbo V4. 208 attention/MLP modules at rank 128 (200 DiT-block modules = V4 ⊕ RL; 8 token-refiner modules = V4 zero-padded) + 51 AdaLN modules at rank 16 (V4 only). 518 BF16 tensors. |
|
| 31 |
+
|
| 32 |
+
## How to run it
|
| 33 |
+
|
| 34 |
+
The recommended sampler is **E4a**: Turbo V4's 9-point shifted grid (`u = 1, .875, …, .125, 0`, shift 12 video / 3 audio) with one extra node at `u = 0.04` (`σ_video = 0.333`) and a 2nd-order Heun corrector on `[σ 0.632 → 0.333]`: 10 sigma points, 10 transformer forwards. Reference implementation for the NVIDIA Sol-H3 engine: `heun_tail.py` in the ModelsLab H3 server repo; the plain 8-forward Euler schedule also works.
|
| 35 |
+
|
| 36 |
+
Load it with any loader that accepts Turbo V4's file (same keys, larger rank). Sol-H3 / PEFT loaders read the rank from the tensors.
|
| 37 |
+
|
| 38 |
+
## Evaluation (16 public clips, seed 11, ratios × the real clip)
|
| 39 |
+
|
| 40 |
+
| Arm | Forwards | Detail | Flow | Motion | Made-up speech |
|
| 41 |
+
| --- | :---: | :---: | :---: | :---: | :---: |
|
| 42 |
+
| Turbo V4, Euler | 8 | 1.43 | 0.87 | 0.94 | 1/7 |
|
| 43 |
+
| Turbo V4, E4a | 10 | 1.59 | 0.92 | 1.02 | 1/7 |
|
| 44 |
+
| **Turbo V4 + rl_s3, E4a** | 10 | **1.59** | **0.92** | **1.01** | 1/7 |
|
| 45 |
+
|
| 46 |
+
Honest attribution: on this public set almost all of the gain over Turbo V4 comes from the E4a sampler; rl_s3 matches V4 + E4a within noise on these metrics (closer to the real clip than V4 + E4a on Lab b for 11/16 clips and on saturation for 8/16). Measured with a BF16 reference pipeline, not with the MXFP8 production engine.
|
| 47 |
+
|
| 48 |
+
## Training data and licences
|
| 49 |
+
|
| 50 |
+
The RL stages (rl_s0 → rl_s3) used short clips (only their cached latents and the prompts written for them) from:
|
| 51 |
+
|
| 52 |
+
| Source | Licence |
|
| 53 |
+
| --- | --- |
|
| 54 |
+
| DVIDS (dvidshub.net) | Public domain (US federal government work); DVIDS API terms |
|
| 55 |
+
| Wikimedia Commons | Per-file free licences |
|
| 56 |
+
| Netflix Open Content | CC BY 4.0 |
|
| 57 |
+
| UltraVideo (`APRIL-AIGC/UltraVideo`) | UltraVideo License (CC BY 4.0 with restrictions, non-commercial research) |
|
| 58 |
+
| Replay (`facebookresearch/replay_dataset`) | CC BY-NC 4.0 |
|
| 59 |
+
| Xiph test media: SVT | Only for developing, testing and presenting technology standards |
|
| 60 |
+
| Xiph test media: Netflix | CC BY-NC-ND 4.0 |
|
| 61 |
+
|
| 62 |
+
No media from these datasets is redistributed here. No customer or user data was used for training.
|
| 63 |
+
|
| 64 |
+
## Licence and use
|
| 65 |
+
|
| 66 |
+
- The weights are a derivative of MiniMax-H3 and are released under the **MiniMax-H3 Community License Agreement**, including its territory and use restrictions. Read it before use.
|
| 67 |
+
- The Turbo V4 component keeps its Apache-2.0 licence and credit: larryvrh, `larryvrh/MiniMax-H3-Turbo-Lora` (revision `43a74557`).
|
| 68 |
+
- Several training sources are licensed for non-commercial research only (see the table). Check that your use is compatible with them.
|
minimax_h3_turbo_v4_rl_s3.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7ea1f6eee5de47e1a56c172d4a26efef90240dab2964df598eb899fb28b69799
|
| 3 |
+
size 1400083496
|