Isotr0py commited on
Commit
ead029e
·
verified ·
1 Parent(s): c7c5a76

Update vllm-omni deployment guide

Browse files
Files changed (1) hide show
  1. README.md +22 -1
README.md CHANGED
@@ -152,10 +152,31 @@ hf download MiniMaxAI/MiniMax-H3-Image-PE
152
  We recommend the following inference frameworks to serve the model:
153
 
154
  - [SGLang](https://docs.sglang.io/) - see [cookbook](https://docs.sglang.io/cookbook/diffusion/MiniMax/MiniMax-H3-image)
155
- - [vLLM](https://github.com/vllm-project/vllm) - see [vllm recipes](https://recipes.vllm.ai/MiniMaxAI/MiniMax-H3-Image)
156
  - [diffusers](https://github.com/huggingface/diffusers) - see [diffusers docs]()
157
  - [ComfyUI](https://github.com/Comfy-Org/ComfyUI) - see [Comfy tutorial](); use [T2I template]() / [Edit template]()
158
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
159
  #### SGLang Deployment
160
 
161
  Here we use SGLang as a deployment example. See the [MiniMax-H3-Image deployment guide](https://docs.sglang.io/cookbook/diffusion/MiniMax/MiniMax-H3-image) for
 
152
  We recommend the following inference frameworks to serve the model:
153
 
154
  - [SGLang](https://docs.sglang.io/) - see [cookbook](https://docs.sglang.io/cookbook/diffusion/MiniMax/MiniMax-H3-image)
155
+ - [vLLM-Omni](https://github.com/vllm-project/vllm-omni) - see [vllm-omni recipe](https://recipes.vllm.ai/MiniMaxAI/MiniMax-H3-Image)
156
  - [diffusers](https://github.com/huggingface/diffusers) - see [diffusers docs]()
157
  - [ComfyUI](https://github.com/Comfy-Org/ComfyUI) - see [Comfy tutorial](); use [T2I template]() / [Edit template]()
158
 
159
+ #### vLLM-Omni Deployment
160
+
161
+ See the [vLLM-Omni recipe](https://recipes.vllm.ai/MiniMaxAI/MiniMax-H3-Image) for
162
+ per-hardware deployment configurations and measured latency. Both NVIDIA and AMD GPUs are
163
+ supported, including NVIDIA H200, RTX PRO 6000 Blackwell and AMD
164
+ Instinct MI350X etc.
165
+
166
+ Start the server on one GPU:
167
+
168
+ ```bash
169
+ vllm serve MiniMaxAI/MiniMax-H3-Image --omni --host 0.0.0.0 --port 8000
170
+ ```
171
+
172
+ For lower latency, split each request over several GPUs with sequence parallelism
173
+ (Ulysses), for example on 4 GPUs:
174
+
175
+ ```bash
176
+ vllm serve MiniMaxAI/MiniMax-H3-Image --omni --host 0.0.0.0 --port 8000 \
177
+ --num-gpus 4 --usp 4 --text-encoder-tp-size 4
178
+ ```
179
+
180
  #### SGLang Deployment
181
 
182
  Here we use SGLang as a deployment example. See the [MiniMax-H3-Image deployment guide](https://docs.sglang.io/cookbook/diffusion/MiniMax/MiniMax-H3-image) for