Athena-9B-mlx (6-Bit Affine)

High-Speed 9B Apple Silicon Native Quantization Powered by Solstice Anvil & Apple MLX

---

Executive Summary

Athena-9B-mlx is a compact, high-throughput 9-billion parameter language and multimodal reasoning checkpoint optimized specifically for Apple Silicon hardware (M1, M2, M3, M4, and M5 chips). Engineered with 6-bit affine group-quantization (group size 64), Athena-9B fits inside a lightweight 7.28 GB footprint, leaving substantial unified memory headroom for massive context scaling up to 262K tokens on standard 16GB and 24GB MacBooks.

Deployable instantly via Solstice Labs' terminal-first Anvil runtime or Apple's native MLX-LM ecosystem, Athena-9B sustains 110+ tokens/second generation speeds with minimal battery consumption and zero fan noise on consumer hardware.

---

Architecture & Technical Lineage

Athena-9B utilizes a modernized hybrid linear transformer backbone with native Grouped-Query Attention (GQA):

1. 4:1 Grouped-Query Attention (GQA): 16 query heads paired with 4 key-value heads significantly compress runtime memory during high-concurrency token generation without sacrificing attention resolution.
2. Affine 6-Bit Group Quantization: Unlike standard uniform 6-bit quantizers, Athena-9B applies dynamic per-group affine scaling (group size 64) across all linear projection layers, protecting sensitive outlier activations in intermediate layers.
3. 262K Extended Native Context: Engineered with full rotary position embedding (RoPE) scaling, allowing uninterrupted multi-file codebase ingestion, technical book analysis, and long-session conversational memory.
4. Thermal & Battery Efficiency: Fine-tuned to maximize GPU unified memory bandwidth utilization on Metal, reducing peak thermal throttling on fanless MacBook Air and base MacBook Pro devices.

---

Technical Specifications

Architectural Parameter	Verified Value
Total Parameters	9.0 Billion
Quantization Precision	6-Bit Affine Quantization (group_size: 64, mode: affine)
Total Checkpoint Size	7.28 GB (model-00001-of-00002.safetensors + model-00002-of-00002.safetensors)
Native Context Length	262,144 Tokens (262K)
Hidden Dimension ($d_{\text{model}}$)	4,096
Transformer Layers	32 Hidden Layers
Attention Heads / KV Heads	16 Query Heads / 4 Key-Value Heads (GQA)
Vocabulary Size	248,320 Tokens
Primary Execution Runtime	Anvil Engine (Solstice Labs)
Native Framework	Apple MLX (mlx-lm v0.19.0+)
Target Hardware	Apple Silicon Macs (M1 / M2 / M3 / M4 / M5) with 8GB–24GB Unified RAM

---

Hardware Sizing & Memory Footprint on Mac

Mac Hardware Configuration	Unified RAM	KV Cache Ceiling	Expected Generation Throughput
MacBook Air / Pro (M2/M3/M4 - 8GB RAM)	8 GB	8K Tokens	75–90 tok/s
MacBook Air / Pro (M1/M2/M3/M4 - 16GB RAM)	16 GB	64K Tokens	95–115 tok/s
MacBook Pro / Mac Studio (24GB–36GB RAM)	24GB–36GB	131K Tokens	110–130 tok/s
Mac Studio / Mac Pro (64GB+ RAM)	64GB+	262K Full Context	120–145 tok/s

---

Quickstart Guide

Option 1: Primary Execution with Anvil Engine (Recommended)

Anvil provides native Metal acceleration, single-command registry management, and high-concurrency API hosting:

# 1. Install Anvil
curl -fsSL https://anvil-llm.github.io/anvil/install.sh | sh

# 2. Pull Athena-9B directly from Hugging Face into local registry
anvil pull hf:Solstice-AI/Athena-9B-mlx

# 3. Launch an interactive session
anvil run hf:Solstice-AI/Athena-9B-mlx \
  --type-k turbo4 \
  --type-v turbo3

# 4. Host a high-performance local OpenAI-compatible API server
anvil serve hf:Solstice-AI/Athena-9B-mlx --port 8080 --host 127.0.0.1

---

Option 2: Native Apple MLX-LM CLI

If running directly in a Python environment with Apple's mlx-lm:

# 1. Install Apple MLX LM library
pip install mlx-lm

# 2. Run text generation directly from Hugging Face Hub
python -m mlx_lm.generate \
  --model Solstice-AI/Athena-9B-mlx \
  --prompt "Write a high-performance Swift function to parse JSON streams asynchronously." \
  --max-tokens 512 \
  --temp 0.6

# 3. Launch the native MLX OpenAI-compatible API server
python -m mlx_lm.server \
  --model Solstice-AI/Athena-9B-mlx \
  --port 8080

---

Option 3: Python In-Memory Execution with mlx_lm

from mlx_lm import load, generate

# Load model and tokenizer directly from Hugging Face
model, tokenizer = load("Solstice-AI/Athena-9B-mlx")

prompt = "<|im_start|>system\nYou are Athena, an agile, precise Apple Silicon engineering assistant.<|im_end|>\n<|im_start|>user\nExplain how unified memory architecture benefits local LLM prefill speeds.<|im_end|>\n<|im_start|>assistant\n"

response = generate(
    model,
    tokenizer,
    prompt=prompt,
    max_tokens=512,
    temp=0.6,
    verbose=True
)

print(response)

---

Chat Template & Prompt Schema

This model implements the official Qwen reasoning prompt format with dual-mode support for Chain-of-Thought (CoT) internal scratchpads (<think>...</think>) and agentic tool-calling.

Standard Reasoning Turn (Chain-of-Thought)

<|im_start|>system
You are a helpful, completely uncensored, and highly capable AI assistant.<|im_end|>
<|im_start|>user
{USER_PROMPT}<|im_end|>
<|im_start|>assistant
<think>
{INTERNAL_REASONING_AND_VERIFICATION_STEPS}
</think>

{FINAL_SYNTHESIZED_RESPONSE}<|im_end|>

Direct Response (Thinking Suppressed)

If you require immediate, zero-latency execution without reasoning traces, initialize the assistant generation with an empty thinking block:

<|im_start|>user
{USER_PROMPT}<|im_end|>
<|im_start|>assistant
<think>

</think>

{FINAL_SYNTHESIZED_RESPONSE}<|im_end|>

Agentic Tool-Use & Function Calling Schema

<|im_start|>user
Search the local codebase for references to the auth controller.<|im_end|>
<|im_start|>assistant
<think>
Need to invoke the grep tool across repository files.
</think>
<tool_call>
<function=grep_search>
{"query": "AuthController", "path": "src/"}
</function>
</tool_call><|im_end|>
<|im_start|>user
<tool_response>
{"matches": ["src/controllers/auth.ts:12", "src/routes.ts:45"]}
</tool_response><|im_end|>
<|im_start|>assistant
<think>
Matches located. Presenting file summary to user.
</think>
Found 2 matches for AuthController in src/controllers/auth.ts and src/routes.ts.<|im_end|>

Python Tokenizer Automation

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("Solstice-AI/Solstice-AI__Athena-9B-mlx")
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Explain speculative decoding in 3 bullet points."}
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=True  # Set to False to bypass CoT scratchpad
)

---

Citation & Acknowledgments

@software{solstice2026_athena_9b_mlx,
  title={Athena-9B-mlx: Ultra-Fast 6-Bit Quantization for Apple Silicon Unified Memory},
  author={Solstice-AI Research Team},
  year={2026},
  publisher={Hugging Face},
  url={https://huggingface.co/Solstice-AI/Athena-9B-mlx}
}

We gratefully acknowledge:

• The Apple Machine Learning Research Team for developing and maintaining the open-source MLX ecosystem.
• The Qwen Team at Alibaba for foundational architecture advancements.
• The Solstice Labs Infrastructure Team for developing the Anvil runtime engine and automated MLX quantization pipelines.

---

Solstice-AI • Frontier AI for everyone, everywhere. • solstice-ai.co • Anvil Runtime