audio-cpp/Audio-Flamingo-GGUF

Audio Flamingo 3, Next, and Music Flamingo GGUF

For use with audio.cpp.

These self-contained GGUFs package the model tensors, tokenizer, configuration, and audio.cpp model spec for Audio Flamingo 3, Audio Flamingo Next, and Music Flamingo. AF3 and Next support transcription, captioning, and questions about speech, music, and environmental sounds. Music Flamingo specializes in music understanding; it produces text, not audio.

File	Upstream checkpoint	Size	Maximum audio
audio-flamingo-3-bf16.gguf	Audio Flamingo 3	15.4 GiB	10 minutes
audio-flamingo-3-q8_0.gguf	Audio Flamingo 3	8.7 GiB	10 minutes
audio-flamingo-3-q4_k.gguf	Audio Flamingo 3	5.1 GiB	10 minutes
audio-flamingo-next-bf16.gguf	Audio Flamingo Next	15.4 GiB	30 minutes
audio-flamingo-next-q8_0.gguf	Audio Flamingo Next	8.7 GiB	30 minutes
audio-flamingo-next-q4_k.gguf	Audio Flamingo Next	5.1 GiB	30 minutes
music-flamingo-2601-q8_0.gguf	Music Flamingo 2601	8.7 GiB	20 minutes

Usage

audiocpp_cli --task asr --family audio_flamingo \
  --model /path/to/audio-flamingo-3-bf16.gguf --backend cuda \
  --audio input.wav \
  --request-option "instruct=Transcribe the input speech." \
  --text-out response.txt --log

Select a GGUF with --model. Use a question such as Describe the instruments and tempo. for general audio understanding. Requested timestamps or speaker descriptions are generated text, not structured alignment or diarization results.

For server use, configure family audio_flamingo, task asr, and mode offline, then submit audio to /v1/audio/transcriptions with the instruction in prompt.

Music Flamingo

audiocpp_cli --task asr --family audio_flamingo \
  --model music-flamingo-2601-q8_0.gguf --backend cuda \
  --audio music.wav \
  --request-option "instruct=Describe the instruments, rhythm, and mood of this music." \
  --text-out description.txt --log

The Q8 package was tested on CUDA with an official music example and produced a coherent description. This is a functional check, not a BF16 parity or performance comparison.

SpotSound LoRA

SpotSound adds sound-event temporal grounding to Audio Flamingo 3. Use an AF3 base, not Next or Music Flamingo. Download its adapter and configuration together:

hf download Loie/SpotSound adapter_model.safetensors adapter_config.json \
  --local-dir adapters/SpotSound

audiocpp_cli --task asr --family audio_flamingo \
  --model audio-flamingo-3-bf16.gguf --backend cuda \
  --audio recording.wav \
  --session-option audio_flamingo.lora=adapters/SpotSound \
  --request-option template_name=temporal_grounding \
  --request-option "instruct=ambulance siren" \
  --text-out intervals.txt --log

The template applies SpotSound's task prompt and timestamp-interleaved audio layout. Results are generated timestamp text; boundaries can be inaccurate. Upstream processing keeps complete one-second feature groups and omits the final fractional group. Use audio_flamingo.lora_scale to adjust adapter strength (default 1). Ordinary requests default to template_name=default.

Correctness Reference

Official Python BF16 output is the reference. Correctness comparisons and performance measurements are separate tests.

Model	Runtime / weights	Result compared with official Python
Audio Flamingo 3	Official Python BF16	Reference
Audio Flamingo 3	audio.cpp BF16	Exact transcription and music answer
Audio Flamingo 3	audio.cpp Q8_0	Exact transcription; coherent but different music answer
Audio Flamingo 3	audio.cpp Q4_K	Same transcription words with different wrapper and punctuation; coherent but different music answer
Audio Flamingo Next	Official Python BF16	Reference
Audio Flamingo Next	audio.cpp BF16	Exact short transcription; matching words and line boundaries on the three-minute test after timestamps were removed
Audio Flamingo Next	audio.cpp Q8_0	Same short transcription words with one comma omitted
Audio Flamingo Next	audio.cpp Q4_K	Exact short transcription

Audio Flamingo Next generates timestamps as text rather than structured alignment. Its BF16 three-minute response used the same words and line boundaries as Python after timestamps were removed, but the generated timestamps drifted.

CUDA Performance and Quantization

The following results use an RTX 5090 and the same decoded 10-second, 16-kHz mono speech input. Python uses the official Transformers BF16 model with SDPA. audio.cpp uses the CUDA backend and 8 CPU threads. Wall time and RTF are from the second request in an already-loaded session. Peak VRAM covers the complete process and was sampled every 100 ms.

Model	Runtime / weights	Warm wall time	RTF	Peak VRAM
Audio Flamingo 3	Official Python BF16	292.4 ms	0.0292	16,766 MiB
Audio Flamingo 3	audio.cpp BF16	283.3 ms	0.0283	16,854 MiB
Audio Flamingo 3	audio.cpp Q8_0	184.9 ms	0.0185	9,956 MiB
Audio Flamingo 3	audio.cpp Q4_K	151.7 ms	0.0152	6,280 MiB
Audio Flamingo Next	Official Python BF16	249.0 ms	0.0249	16,784 MiB
Audio Flamingo Next	audio.cpp BF16	246.3 ms	0.0246	16,890 MiB
Audio Flamingo Next	audio.cpp Q8_0	160.9 ms	0.0161	9,992 MiB
Audio Flamingo Next	audio.cpp Q4_K	129.9 ms	0.0130	6,314 MiB

Quantized outputs remained coherent on the transcription and music-understanding requests. Q4_K is the default package and offers the lowest tested VRAM. Q8_0 is the higher-precision quantized alternative. Quantization can change wording, punctuation, or factual details in open-ended answers and should not be treated as exact parity with BF16.

Audio Preprocessing

For comparisons with Python, supply the same decoded 16-kHz mono WAV to both implementations. Python's optional TorchCodec loader and librosa fallback use different downmixing and resampling paths. audio.cpp averages channels and uses SOXR when resampling is needed.

The base checkpoints use the NVIDIA OneWay Noncommercial License. SpotSound licenses its adapter under MIT; the base-model terms still apply. The upstream license is included in this directory. Review its terms before use or redistribution.

See the audio.cpp model documentation for request options, conversion, and additional usage details.