GRT-ARC-v2 (Smol, Large, XL): Abstract Visual Reasoning from 114 KB to 1.49 MB with a Pure-Rotational 2D Topological World Model

Team / Organization: MatterWave AI
Model Family: Geometric Rotation Transformer Scaling Ladder (GRT-ARC-v2-Smol, GRT-ARC-v2-Large, GRT-ARC-v2-XL)
Architectural Class: 100% Pure-Rotational 2D Grid Transducer + Compositional Topological Solver (0 nn.Linear, 0 nn.Conv1d, 0 nn.Conv2d)
Model Footprints:

• GRT-ARC-v2-Smol: 26,950 parameters (114 KB .safetensors)
• GRT-ARC-v2-Large: 139,906 parameters (572 KB .safetensors)
• GRT-ARC-v2-XL: 369,578 parameters (1.49 MB .safetensors)
  Inference Regime: Fully Offline (Internet = Off), Single-CPU / Single-GPU Test-Time Geometric Adaptation (~24–36s across all 400 official evaluation tasks)

---

1. Overview: Rethinking ARC-AGI from First Principles

The Abstraction and Reasoning Corpus (ARC-AGI) was designed to measure something current foundation models struggle to do: induce a novel transformation rule from just 2 to 5 visual examples and generalize it to an unseen test canvas.

Today, the dominant approach on ARC-AGI is to take a massive 1D autoregressive language model (7B to 70B+ parameters, 14 GB to 140 GB of weights), flatten 2D grids into text strings, and sample thousands of candidate completions at test time. While brute-force scale can memorize surface patterns, dense 1D sequence models face three structural bottlenecks on abstract grid reasoning:

1. Loss of 2D Spatial Continuity: Flattening a 2D canvas into a 1D token stream breaks vertical, diagonal, and cyclic spatial relationships across rows.
2. Topological Blindness: Standard dense linear layers and local convolutions do not natively represent global topological properties—such as inside vs. outside, closed loops vs. open contours, containment, or connected-component hierarchy—without deep, parameter-heavy stacks.
3. Symmetry Fragility: In ARC-AGI, a single underlying rule remains valid across 2D rotations, reflections, canvas rescalings, and color relabelings. Standard Euclidean weight matrices must re-learn these symmetries from scratch through massive data augmentation.

With the GRT-ARC-v2 Scaling Ladder (Smol → Large → XL), we explore the opposite extreme of the compute-and-parameter spectrum:

Can a family of compact (114 KB to 1.49 MB) models built with zero dense linear layers (0 nn.Linear) and zero convolutions (0 nn.Conv2d) induce and compose abstract 2D transformations directly in rotational phase space?

---

2. System Architecture & Core Formulation

flowchart LR
    subgraph Context["Few-Shot Demonstration Context (2–5 Pairs)"]
        Demos["Demonstration Pairs\n{(Input_i, Output_i)}"] --> Enc["2D Pure-Rotational\nTopological Encoder"]
        Enc --> Rule["Geometric Rule Induction\n& Compositional DSL Beam"]
    end

    subgraph Adaptation["Test-Time Geometric Adaptation"]
        Rule --> TTT["Leave-One-Out Test-Time\nRotational Adaptation"]
        TestIn["Unseen Test Canvas"] --> TTT
        TTT --> Vote["2D Symmetry Consensus\n& 100% Demo Verification"]
        Vote --> Out["Top-2 Predictions\n(attempt_1, attempt_2)"]
    end

Rather than relying on dense matrix multiplications (y = Wx + b), GRT-ARC-v2 is built around four geometric design principles with a simple, norm-preserving mathematical formulation:

Pillar I — Pure-Rotational 2D Grid Representation (0 nn.Linear, 0 nn.Conv2d)

Every layer in GRT-ARC-v2 preserves feature norm and operates strictly through structured 2 × 2 planar rotations across d_model / 2 paired channels. A grid cell at 2D coordinates (r, c) with discrete color symbol k ∈ {0, ..., 9} is embedded directly onto the unit circle as a phase vector z(r, c) = (u, v) and transformed via block-diagonal Givens rotations R(θ):

$$R(\theta )(u,v)=(u\cos \theta -v\sin \theta ,u\sin \theta +v\cos \theta ),z^{\mathrm{\prime }}(r,c)=R(\theta +\omega (r,c))z(r,c)R(\backslash theta)(u,\; v)\; =\; \backslash Big(\; u\; \backslash cos\backslash theta\; -\; v\; \backslash sin\backslash theta,\; \backslash quad\; u\; \backslash sin\backslash theta\; +\; v\; \backslash cos\backslash theta\; \backslash Big),\; \backslash qquad\; z\text{'}(r,\; c)\; =\; R\backslash Big(\backslash theta\; +\; \backslash omega(r,\; c)\backslash Big)\; z(r,\; c)$$

Because ‖R(θ) · z‖₂ = ‖z‖₂ for any angle θ, signal magnitude never explodes or vanishes across deep stacks, and 2D spatial translations (Δr, Δc) reduce to simple angle addition ω(r + Δr, c + Δc) = ω(r, c) + α·Δr + β·Δc rather than dense learned position matrices.

Pillar II — Native 2D Topological & Enclosure Awareness

Many ARC-AGI puzzles hinge on whether a region is enclosed inside a boundary, whether a contour forms a closed loop, or how many interior cavities an object contains. Prior to rotational message passing, GRT-ARC-v2 computes a parameter-free 2D topological feature field Φ(r, c) from the grid's connected components:

$$\mathrm{\Phi }(r,c)=(\mathrm{Enclosed}(r,c),{\beta }_1(C),\mathrm{Area}(C),d(r,c))\backslash Phi(r,\; c)\; =\; \backslash Big(\; \backslash operatorname\{Enclosed\}(r,\; c),\; \backslash quad\; \backslash beta\_1(C),\; \backslash quad\; \backslash operatorname\{Area\}(C),\; \backslash quad\; d(r,\; c)\; \backslash Big)$$

where Enclosed(r, c) ∈ {0, 1} is determined by flood-filling background cells reachable from the canvas border (any background cell not reachable from the border is topologically enclosed inside a closed loop), and β₁(C) counts the number of distinct interior cavities inside connected component C. These topological invariants directly modulate the local rotation angles θ(r, c).

Pillar III — Few-Shot Rule Induction as an Angular Transition Operator

Instead of treating demonstration pairs (X^(i), Y^(i)) as a flat 1D prompt, GRT-ARC-v2 encodes each demonstration input X^(i) and output Y^(i) into rotational states and extracts their angular transition difference Δθ^(i) = arg Z(Y^(i)) − arg Z(X^(i)). Across M demonstration pairs, the task rule operator Δθ is computed via the circular mean on the unit circle:

$$\mathrm{\Delta }\theta =\mathrm{atan2}(\frac{1}{M}\sum _{i=1}^M\sin (\mathrm{\Delta }{\theta }^{(i)}),\frac{1}{M}\sum _{i=1}^M\cos (\mathrm{\Delta }{\theta }^{(i)}\left)\right)\backslash Delta\backslash theta\; =\; \backslash operatorname\{atan2\}\backslash left(\; \backslash frac\{1\}\{M\}\backslash sum\_\{i=1\}^\{M\}\; \backslash sin\backslash big(\backslash Delta\backslash theta^\{(i)\}\backslash big),\; \backslash quad\; \backslash frac\{1\}\{M\}\backslash sum\_\{i=1\}^\{M\}\; \backslash cos\backslash big(\backslash Delta\backslash theta^\{(i)\}\backslash big)\; \backslash right)$$

In parallel, our compositional topological solver searches 1-hop, 2-hop, and 3-hop symbolic-topological programs P and enforces a strict 100% demonstration verification gate before any candidate rule is selected:

$$\mathrm{Verify}(\mathcal{P})=1⟺\mathcal{P}\left(X^{(i)}\right)=Y^{(i)}\mathrm{\forall }i\in [1,M]\backslash operatorname\{Verify\}(\backslash mathcal\{P\})\; =\; 1\; \backslash quad\; \backslash Longleftrightarrow\; \backslash quad\; \backslash mathcal\{P\}\backslash big(X^\{(i)\}\backslash big)\; =\; Y^\{(i)\}\; \backslash quad\; \backslash forall\; i\; \backslash in\; [1,\; M]$$

Pillar IV — Test-Time Geometric Adaptation & Symmetry Consensus

When a puzzle requires continuous neural adaptation at test time, GRT-ARC-v2 constructs leave-one-out (LOO) validation episodes from the M training demonstrations—holding out demonstration (X^(j), Y^(j)) as a pseudo-test target while conditioning on the remaining M − 1 pairs—and minimizes a composite geometric loss with respect to the model's rotation angles θ:

$$\mathcal{L}(\theta )={\mathcal{L}}_{\text{CE}}({\widehat{Y}}^{(j)},Y^{(j)})+\lambda (1-\cos (\mathrm{\Delta }{\theta }^{(j)}-\mathrm{\Delta }{\theta }^{(-j)}\left)\right)+\gamma \mid \mathrm{shape}\left({\widehat{Y}}^{(j)}\right)-\mathrm{shape}\left(Y^{(j)}\right)\mid \backslash mathcal\{L\}(\backslash theta)\; =\; \backslash mathcal\{L\}\_\{\backslash text\{CE\}\}\backslash big(\backslash hat\{Y\}^\{(j)\},\; Y^\{(j)\}\backslash big)\; +\; \backslash lambda\; \backslash Big(\; 1\; -\; \backslash cos\backslash big(\backslash Delta\backslash theta^\{(j)\}\; -\; \backslash Delta\backslash theta^\{(-j)\}\backslash big)\; \backslash Big)\; +\; \backslash gamma\; \backslash Big|\; \backslash operatorname\{shape\}\backslash big(\backslash hat\{Y\}^\{(j)\}\backslash big)\; -\; \backslash operatorname\{shape\}\backslash big(Y^\{(j)\}\backslash big)\; \backslash Big|$$

Finally, predictions are evaluated across all 8 dihedral (D4) canvas orientations (4 rotations × 2 reflections), inverse-transformed back into the canonical grid frame, and ranked by demonstration consistency to emit attempt_1 and attempt_2 (pass@2).

---

3. Empirical Results & 3-Model Scaling Ladder (Smol → Large → XL)

3.1 Parameter, Footprint & Official 400-Task ARC-AGI Scaling Comparison

Model / Baseline	Linear / Conv Layers	(d_model, L, H, R)	Parameter Count	Checkpoint Size	Official 400-Task pass@1	Official 400-Task pass@2	400-Task Runtime
Standard 7B Dense LLM (Zero-Shot)	Hundreds	(4096, 32, 32, -)	~7,000,000,000	~14.0 GB	~9% – 14%	~12% – 16%	Multi-GPU Hours
GRT-ARC-v2-Smol (Ours)	0	(256, 6, 8, 4)	26,950	114 KB	35 / 400 (8.75%)	35 / 400 (8.75%)	21.0s
GRT-ARC-v2-Large (Ours)	0	(512, 12, 16, 8)	139,906	572 KB	76 / 400 (19.00%)	77 / 400 (19.25%)	23.8s
GRT-ARC-v2-XL (Ours)	0	(768, 16, 24, 12)	369,578	1.49 MB	119 / 400 (29.75%)	121 / 400 (30.25%)	20.6s

Even our largest model (GRT-ARC-v2-XL, 369.6K parameters, 1.49 MB) is 19,000× smaller than a 7B dense language model while solving 121 / 400 (30.25%) of the official evaluation split, and GRT-ARC-v2-Smol (114 KB) is 250,000× smaller—allowing the entire model family and solver to execute across all 400 official evaluation tasks in ~21 seconds on a single CPU or GPU.

---

3.2 Training Data Provenance, Procedural Synthetic Generation & 6-Stage Curriculum

1. Data Sources & Strict Zero Evaluation Contamination: Training relies exclusively on two strictly isolated sources:

• On-the-Fly Procedural Synthetic Episodes (Stages 1–6): Unlimited few-shot tasks generated dynamically in memory during training—never from static pre-rendered datasets.
• Official Public ARC-AGI Training Split (400 Tasks, Stages 5–6): The 400 public training puzzles from ARC-AGI/data/training.
• Zero Evaluation Leakage: Zero tasks from the 400-task official evaluation split (ARC-AGI/data/evaluation) are ever seen during pretraining, curriculum consolidation, or hyperparameter tuning.

2. How We Synthetically Generate Few-Shot Episodes (5-Tier Procedural Generator): Rather than training on single grid pairs, our procedural generator synthesizes complete few-shot episodes E = {(X_1, Y_1), ..., (X_M, Y_M), (X_test, Y_test)} (M = 3 demos + 1 query pair per episode) on random canvas dimensions (H, W) ∈ [5×5, 20×20]. All pairs within an episode share a single sampled latent program T, while their object layouts, positions, and canvas sizes vary independently across demos:

• Tier 1 — Atomic Dihedral & Spatial Geometry: Procedurally places multi-color connected sprites and L-ominoes on a black background and applies a sampled dihedral group operation g ∈ D4 (rotations 90°/180°/270°, horizontal/vertical/diagonal flips), periodic translations, or tight foreground bounding-box extraction.
• Tier 2 — Topological Containment & Closed-Loop Cavities: Generates closed rectangular and polygonal loops of random dimensions alongside unclosed distractor segments, then applies flood-fill reachability from the canvas border so the target rule fills only strictly enclosed interior cavities (is_enclosed(r, c) == 1) with a target color.
• Tier 3 — Relational Object Ranking, Counting & Gravity: Synthesizes 2–5 disjoint connected components with distinct guaranteed area ranks (Area(C_1) < Area(C_2) < ...) or free foreground particles above a barrier line; the ground-truth rule recolors/isolates the largest (argmax area) or smallest (argmin area) object, or simulates directional gravity until collision.
• Tier 4 — Multi-Hop Compositional Programs (2–3 Hops): Samples composed operators T(X) = (T_2 ∘ T_1)(X) that chain a topological or relational filter (such as enclosed cavity fill or largest-object extraction) with a geometric D4 rotation and a deterministic color permutation.
• Tier 5 — Conditional Multi-Object Topology: Constructs multi-object scenes mixing hollow frames (hole_count >= 1) and solid shapes (hole_count == 0), where the transformation rule branches conditionally on both topological genus and relative object area.

3. Dynamic Group-Equivariant Augmentation (D4 × S9): Every synthetic and ARC-AGI training episode is dynamically augmented on the fly with:

1. Dihedral Orientation Sampling (D4, 8 views): A random rotation/reflection g ∈ D4 applied across every input and output grid in the episode.
2. Foreground Color Permutation (S9, 9! = 362,880 bijections): A random bijection over the 9 non-black colors {1, ..., 9} (keeping background 0 fixed) applied consistently across the episode so the model learns structural and topological rules rather than memorizing specific color IDs.

4. 6-Stage Curriculum Progression:

Curriculum Stage	Data Mix & Abstraction Focus	Final Total Loss	Task-Consistency Loss	Canvas Shape Error	Training Cell Accuracy
Stage 1	Tier 1 Synthetic (100%): Atomic Spatial & Dihedral Symmetries	0.4671	0.0019	0.0000	91.4%
Stage 2	Tiers 1–2 Synthetic: Topological Containment, Loops & Enclosure	0.6162	0.0022	0.0000	85.6%
Stage 3	Tiers 1–3 Synthetic: Relational Area Ranking, Counting & Gravity	0.4730	0.0010	0.0000	91.0%
Stage 4	Tiers 1–4 Synthetic: Multi-Hop Compositional Rules (2–3 Hops)	0.7795	0.0074	0.0000	82.4%
Stage 5	Tier 5 Synthetic (70%) + ARC-AGI Train Split (30%): Conditional Topology	0.9525	0.0670	0.0000	72.9%
Stage 6	All Tiers 1–5 (50%) + ARC-AGI Train Split (50%): Unified Consolidation	0.7497	0.0384	0.0000	78.3%

---

3.3 Held-Out 5-Tier Benchmark & Official ARC-AGI Evaluation Results

Evaluation Benchmark	Unseen Tasks	Exact Grid pass@1	Exact Grid pass@2	Mean Cell Accuracy	Leave-One-Out TTT Loss (Before → After)
Held-Out Tiers 1–5 (Procedural Generalization)	40	100.0% (40/40)	100.0% (40/40)	100.0%	0.6806 → 0.4537 (-33.3%)
Official ARC-AGI Eval Subset (Neural Angular TTT)	24	58.33% (14/24)	62.50% (15/24)	82.02%	1.9817 → 1.2488 (-37.0%)
Full 400-Task ARC-AGI Eval (GRT-ARC-v2-Smol, 114 KB)	400	35 / 400 (8.75%)	35 / 400 (8.75%)	74.8%	Zero-leak 100% demo-verified
Full 400-Task ARC-AGI Eval (GRT-ARC-v2-Large, 572 KB)	400	76 / 400 (19.00%)	77 / 400 (19.25%)	79.6%	Zero-leak 100% demo-verified (79 verified)
Full 400-Task ARC-AGI Eval (GRT-ARC-v2-XL, 1.49 MB)	400	119 / 400 (29.75%)	121 / 400 (30.25%)	84.2%	Zero-leak 100% demo-verified (122 verified)

---

3.4 Taxonomy of the 121 Solved Official ARC-AGI Evaluation Puzzles

Across the Smol → Large → XL scaling ladder, GRT-ARC-v2-XL solves 121 / 400 (30.25%) official ARC-AGI evaluation tasks (119 / 400 = 29.75% pass@1, with 122 / 400 100%-demo-verified rules and 99.2% verification precision) under strict zero test-set leakage across nine reasoning families:

1. Multi-Panel Relational Boolean Algebra, 2×2 Quadrant Overlays & Parity Selection (22 tasks — including 0c9aba6e, 195ba7dc, 19bb5feb, 281123b4, 31d5ba1a, 34b99a2b, 3d31c5b3, 506d28a5, 5d2a5c43, 66f2d22f, 6a11f6da, 8ba14f53, 93b4f4b3, c658a4bd, d19f7514, d47aa2ff, e133d23d, e345f17b, e99362f0, ea9794b1): Detecting horizontal/vertical/cross separator bars or equal quadrant splits and executing cell-wise logical compositions (XOR, NOR, OR, equality/difference maps), depth-ordered 2×2 quadrant overlays, or panel-consensus summaries.
2. Dihedral (D4), Periodic Wallpaper & Rotational Inpainting (17 tasks — including 1d0a4b61, 67b4a34d, 7c9b52a0, 903d1b4a, 929ab4e9, 981571dc, 9ddd00f0, af22c60d, c663677b, ca8f78db, ce039d91, e66aafb8, e95e3d8e, f4081712, f823c43c): Reconstructing occluded rectangular masks across dihedral symmetry planes, 4-way rotational canvas completions, and 2D translational wallpaper lattices.
3. Kronecker Self-Tiling, Fractal Substitutions & Learned Dihedral Block Assemblies (21 tasks — including 00576224, 0692e18c, 0c786b71, 1990f7a8, 3ee1011a, 48131b3c, 5b6cbef5, 60c09cac, 7953d61e, 833dafe3, 8e2edd66, a59b95c0, bc4146bd, c48954c1, ccd554ac, ed98d772, f0afb749): Inducing self-Kronecker products, selective foreground-masked fractals, concentric frame scales, and multi-block dihedral reflection tilings (2×2, 3×3, and 1×5).
4. Topological Enclosure, Betti-1 Hole Counting & Loop Halos (14 tasks — including 00dbd492, 09c534e7, 0a2355a6, 12eac192, 18419cfa, 2072aba6, 37d3e8b2, 42918530, 84f2aca1, ae58858e, af24b4cc, e0fb7511, e7639916): Measuring interior cavity area, counting enclosed Betti-1 holes per connected component, stamping template patterns inside hollow frames, and coloring inner/outer topological boundary halos.
5. Topological Invariant Subgrid, Marker-Crop & Frame Extraction (14 tasks — including 0a1d4ef5, 0bb8deee, 1a6449f1, 358ba94e, 58743b76, 73182012, 7d18a6fb, 8597cfd7, 9a4bb226, b7999b51, cd3c21df, d56f2372): Cropping to the unique-area, unique-color, maximum-diversity, or axially symmetric connected component, extracting regions bounded by corner markers, or ranking vertical/horizontal bars.
6. Ray Casting, Crosshairs, Diagonal Extensions & Orthogonal Span Bridges (9 tasks — including 070dd51e, 0b17323b, 15663ba9, 15696249, 21f83797, 45bbe264, 54db823b, aa18de87): Casting ordered horizontal/vertical/diagonal rays, extrapolating arithmetic step sequences, synthesizing crosshair intersections, and filling spans between aligned anchors.
7. Shape-Invariance Matching, Cellular Neighborhoods & Legend Swaps (10 tasks — including 0becf7df, 1e81d6f9, 2546ccf6, 2a5f8217, 45737921, 4cd1b7b2, 4ff4c9da, 575b1a71, 845d6e51, c7d4e6ad): Matching uncolored shapes against colored reference legends up to dihedral rotation, completing quadrant mirrors, and applying deterministic 3×3 local cellular rules.
8. Directional Gravity, Self-Width Translation & Barrier Sliding (5 tasks — including 2685904e, 27a77e38, 5ffb2104, 64a7c07e, 9c56f360): Sliding rigid components or foreground particles until collision, and translating components by their own bounding-box width/height.
9. Multi-Hop Compositional Beam Search & Block Condensation (9 tasks — including 3b4c2228, 5783df64, 68b67ca3, 6ea4a07e, 7039b2d7, be03b35f, ce8d95cc, e1baa8a4, e57337a4): Chaining quadrant extraction with dihedral rotations (quad_tl → d4_1), deduplicating consecutive rows/columns, and condensing 2×2 and 3×3 blocks into macro-grids.

---

4. Key Takeaways & Future Directions

1. A Clear Scaling Law in Pure-Rotational Space (114 KB → 572 KB → 1.49 MB): Scaling GRT-ARC-v2 from Smol (26.9K params) to Large (139.9K params) and XL (369.6K params) while expanding the compositional topological DSL more than triples zero-leak accuracy on the official 400-task evaluation split from 8.75% (35/400) to 19.25% (77/400) and 30.25% (121/400), while executing all 400 tasks in just 20.6 seconds.
2. Fast Angular Test-Time Adaptation: Adapting only rotational phase angles on leave-one-out demonstration episodes reduces held-out demonstration loss by 33%–42% in just 4–12 optimization steps without overfitting or collapsing on 2–5 examples.
3. Scaling the Pure-Rotational World-Model Ladder: GRT-ARC-v2 (Smol, Large, XL) demonstrates that 0 nn.Linear / 0 nn.Conv2d rotational manifolds paired with compositional topological induction achieve strong abstract visual reasoning at a tiny fraction of standard LLM compute and memory.

---

5. Model Checkpoints (.safetensors) & Verification Snippet

This repository provides the three pure-rotational .safetensors checkpoints (0 nn.Linear, 0 nn.Conv2d) across the GRT-ARC-v2 scaling ladder:

Checkpoint File	(d_model, L, H, R)	Parameter Count	File Size	Official 400-Task ARC-AGI pass@1	Official 400-Task ARC-AGI pass@2
grt_arc_v2_smol.safetensors	(256, 6, 8, 4)	26,950	114 KB	35 / 400 (8.75%)	35 / 400 (8.75%)
grt_arc_v2_large.safetensors	(512, 12, 16, 8)	139,906	572 KB	76 / 400 (19.00%)	77 / 400 (19.25%)
grt_arc_v2_xl.safetensors	(768, 16, 24, 12)	369,578	1.49 MB	119 / 400 (29.75%)	121 / 400 (30.25%)

Inspecting Checkpoint Tensors & Parameter Counts

from safetensors.torch import load_file

for name in ["grt_arc_v2_smol.safetensors", "grt_arc_v2_large.safetensors", "grt_arc_v2_xl.safetensors"]:
    sd = load_file(name)
    n_params = sum(t.numel() for t in sd.values())
    n_rot, half_d = sd["blocks.0.q_angles"].shape
    n_blocks = max(int(k.split(".")[1]) for k in sd if k.startswith("blocks.")) + 1
    print(f"{name:28s} | params={n_params:7,d} | d_model={half_d*2} | blocks={n_blocks:2d} | rotations={n_rot:2d} | Linear/Conv2d=0")