metadata
license: mit
TokenFlow🚀: Unified Image Tokenizer for Multimodal Understanding and Generation
🌿 Introduction
We present TokenFlow, a unified image tokenizer that bridges the long-standing gap between multimodal understanding and generation. TokenFlow introduce an innovative dual-codebook architecture that decouples semantic and pixel-level feature learning while maintaining their alignment through a shared mapping mechanism.
TokenFlow excels in both multimodal understanding and image generation. For multimodal understanding, we surpass the flagship models such as LLaVA-1.5 and EMU3 by a large margin. For text-to-image generation, we also achieve comparable performance to SDXL in 256×256 resolution.
⚙️ Getting Started
See GETTING_STARTED.md for detailed instructions of training and evaluation of TokenFlow tokenizer.