Add diffusers format weights

#5
by multimodalart HF Staff - opened
MiniMax org

Adds the diffusers format weights at the root of the repository, alongside the original checkpoint folders. The layout is component wise, so diffusers loading fetches exactly the subfolders a task needs and never touches FL2VA/ or Ref2VA/.

transformer/ serves text to video+audio and first/last frame conditioning, transformer_ref/ serves omni reference conditioning, and text_encoder/, tokenizer/, processor/, vae/, audio_vae/, scheduler/ and audio_scheduler/ are shared by both. modular_model_index.json is the loading manifest.

Requires the diffusers integration: https://github.com/huggingface/diffusers/pull/14355 (usage and per hardware guidance: https://github.com/huggingface/diffusers/blob/minimax-h3/docs/source/en/api/pipelines/minimax_h3.md)

import torch
from diffusers import ModularPipeline

pipe = ModularPipeline.from_pretrained("MiniMaxAI/MiniMax-H3")
pipe.load_components(dtype=torch.bfloat16)

The converted transformer reproduces the reference implementation's denoising trajectories bit for bit across all 15 documented use case configurations at 30 steps.

multimodalart changed pull request status to open
ryanlee-dev changed pull request status to merged

Sign up or log in to comment