Hugging Face's logo

SD-Next
/
MicroDecoder

Text-to-Image
Diffusers
Model card Files
xet
 Community
vladmandic commited on
Commit
f9898d3
·
0 Parent(s):

Duplicate from vladmandic/MicroDecoder

Browse files
Files changed (11) hide show
  1. .gitattributes +36 -0
  2. MicroDecoder.jpg +3 -0
  3. README.md +64 -0
  4. microdecoder-f1.safetensors +3 -0
  5. microdecoder-f2.safetensors +3 -0
  6. microdecoder-minimaxh3.safetensors +3 -0
  7. microdecoder-qwen.safetensors +3 -0
  8. microdecoder-qwen21.safetensors +3 -0
  9. microdecoder-sd.safetensors +3 -0
  10. microdecoder-sdxl.safetensors +3 -0
  11. microdecoder-wan21.safetensors +3 -0
.gitattributes ADDED
@@ -0,0 +1,36 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ MicroDecoder.jpg filter=lfs diff=lfs merge=lfs -text
MicroDecoder.jpg ADDED

Git LFS Details

  • SHA256: fed3617eaa55e4433d3d4fd01da73e3208f8bbff451830134c67734ae2a5f7b8
  • Pointer size: 131 Bytes
  • Size of remote file: 677 kB
README.md ADDED
@@ -0,0 +1,64 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ pipeline_tag: text-to-image
4
+ library_name: diffusers
5
+ ---
6
+
7
+ # MicroDecoder
8
+
9
+ **MicroDecoder** VAE can be trained on *any* diffusion model in **~5-15min** and provides low-quality reconstruction from latents to RGB in **~0.01sec**.
10
+
11
+ Inference includes noise correction based on current timestep, blurring corrections plus upscale interpolation: all with intention of providing as fast-as-possible reconstruction that is viable and consistent with any noise levels.
12
+
13
+ Intended use-case is live-preview during generative model inference.
14
+
15
+ Shapes/Channels/etc are inferred from the base VAE, so no configuration changes are needed between different models.
16
+
17
+ ## Models
18
+
19
+ Models included in this repo are pre-trained for following architectures:
20
+ - **SD, SDXL, Flux.1, Flux.2, Qwen, Qwen-21, Wan-21, MiniMax-H3**
21
+
22
+ ## Code
23
+
24
+ - Model definition [here](https://github.com/vladmandic/sdnext/blob/dev/modules/vae/sd_vae_micro_model.py)
25
+ - Training code [here](https://github.com/vladmandic/sdnext/blob/dev/modules/vae/sd_vae_micro_train.py)
26
+ - Example inference code [here](https://github.com/vladmandic/sdnext/blob/dev/modules/vae/sd_vae_micro.py)
27
+
28
+ ## Example
29
+
30
+ Example using **MicroDecoder** with `Flux.2-Klein-9B` and compared with official final VAE processing at the end:
31
+
32
+ ![MicroDecoder](https://huggingface.co/vladmandic/MicroDecoder/resolve/main/MicroDecoder.jpg)
33
+
34
+ ## Training
35
+
36
+ ```shell
37
+ sd_vae_micro_train.py \
38
+ --dim 256 \
39
+ --epochs 350 \
40
+ --resolution 512 \
41
+ --scale 4 \
42
+ --lr 0.0003
43
+ --folder ~/generative/Input/vae/ \
44
+ --vae AutoencoderKLQwenImage21 \
45
+ --repo Qwen/Qwen-Image-2.1
46
+ --output MicroVAE-qwen21.safetensors
47
+ ```
48
+
49
+ ```log
50
+ MicroDecoder Train
51
+ Args: Namespace(folder='/home/vlado/generative/Input/vae/', max=500, resolution=512, crop='center', vae='AutoencoderKLMiniMaxH3', repo='OzzyGT/MiniMax_H3_sdnq_dynamic_4bit', subfolder='vae', dim=256, epochs=400, scale=4, batch=16, lr=0.0003, ema=0.999, val=0.1, output='microdecoder-minimaxh3.safetensors', device='cuda')
52
+ Base VAE: cls=<class 'diffusers.models.autoencoders.autoencoder_kl_minimax_h3.AutoencoderKLMiniMaxH3'> repo="OzzyGT/MiniMax_H3_sdnq_dynamic_4bit" subfolder="vae"
53
+ Init VAE: model=MicroDecoder ema=EMAModel optimizer=AdamW scheduler=CosineAnnealingLR criterion=EnhancedQualityLoss metrics=DetailedMetricsTracker
54
+ Epoch | tTotal | tPSNR | tSSIM | vTotal | vPSNR | vSSIM | vL1 | vLAB
55
+ -------------------------------------------------------------------------------------------
56
+ 001/400 | 7.0891 | 10.49 | 0.2540 | 7.1642 | 10.75 | 0.2357 | 0.2614 | 0.4146
57
+ ...
58
+ 400/400 | 4.5365 | 21.03 | 0.6402 | 1.9644 | 30.61 | 0.9266 | 0.0204 | 0.0465
59
+ Load ━━━━━━━━━━━━━━━━━━━━ 400/400 100% 0:00:00 0:00:08 Images=400 Shape=torch.Size([400, 3, 512, 512])
60
+ Encode ━━━━━━━━━━━━━━━━━━━━ 400/400 100% 0:00:00 0:00:46 Train=360 Validation=40 EMA=0.999
61
+ Train ━━━━━━━━━━━━━━━━━━━━ 400/400 100% 0:00:00 0:06:33 Train(psnr=21.029 ssim=0.640 lpips=2.784 l1=0.089 lab=0.156 grad=0.073 sat=0.124 fft=0.071 total=4.536) Validate(psnr=30.607 ssim=0.927 lpips=1.242 l1=0.020 lab=0.046 grad=0.046 sat=0.057 fft=0.032 total=1.964)
62
+ Complete: Time=393.89 Epoch/Sec=1.02 PSNR=30.61 dB @ Epoch=400
63
+ Save: filename="microdecoder-minimaxh3.safetensors"
64
+ ```
microdecoder-f1.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d2d8b0c603d7a387cef03afbfde5b19f7e47294679c04825f4f6d2e6d20567f9
3
+ size 9676622
microdecoder-f2.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9914d3c3ef783d5659111c45cf50fa52d45d08418e89c368c0f07d31a561a915
3
+ size 9750358
microdecoder-minimaxh3.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eb9427a1d8f92bdd123feda388f9865738095689df0a395eb49956bcd3b0415d
3
+ size 9713494
microdecoder-qwen.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4726e301518ca1c4931fce396aa66086662bb4ab522a0d936a2f673f27cab921
3
+ size 9676622
microdecoder-qwen21.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8e852db1de30d9191d78a463e739d9acfb2ad861a4c02487d245f25dfecec3f5
3
+ size 9897814
microdecoder-sd.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:34f4b4a95a0a6ff3e3773ca1e070152d71fbfc366ecb712d71d0aaf3e43d3580
3
+ size 9621318
microdecoder-sdxl.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fc29e419d9da15a7f4f6299eb8296424bffeea16a5f525f1711471137197183c
3
+ size 9621318
microdecoder-wan21.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:46d6b7207a797cd65d7950b1c60302387814b06cbaeedef268dc60a4a61ec8d5
3
+ size 9676622