Commit History
-
feat(model): add boundary-aware FA3 cross-attention
Verified signatureThis adds an opt-in path for the specialized MOSS-VL FA3 cross-attention kernel. With `cross_attention_implementation="flash_attention_3"`, the model converts each frame-level mask row to an int32 visible-KV boundary and passes it to FA3. This path no longer materializes a dense text-by-vision mask. Existing eager, SDPA, and FA2 behavior is unchanged. Setting global `attn_implementation="flash_attention_3"` alone also keeps the previous cross-attention dispatch. The separate override lets Ampere runs keep FA2 or SDPA for self-attention while using the MOSS FA3 kernel for cross-attention. Tested: - 11 CPU unit tests covering mask parity, invalid masks, config serialization, dispatch, and default compatibility - A800 BF16 forward/backward parity at the model head shape: 32 query heads, 8 KV heads, head dimension 128 - MOSS-VL 11B two-image prefill and four cached decode steps: 60 FA3 calls, no dense mask expansion, stable 48-token vision KV cache, and matching greedy top-1 against SDP…0dbae6f
-
Upload README.md with huggingface_hub
Verified signatureabd2185
-
Update video_processing_moss_vl.py
Verified signature9700ecf
-
Update README.md
Verified signature5bf9cbe
-
Update processing_moss_vl.py
Verified signaturef8d900c
-
Update processing_moss_vl.py
Verified signaturee59a15d -
Update README.md
Verified signaturec0d80c1
-
Upload benchmark-offline.png
Verified signature2c4d532 -
Delete assets/benchmark-offline.png
Verified signature305e490 -
Upload benchmark-offline.png
Verified signatured396355 -
Delete assets/benchmark-offline.png
Verified signaturee891b68 -
Update README.md
Verified signature6cc79e6 -
Update README.md
Verified signature8554b6f -
Update model card and assets
Verified signature3f11010
-
Upload folder using huggingface_hub
Verified signature2cb1408 -
initial commit
Verified signatureaad8579