jburtoft commited on
Commit
5d204e3
·
1 Parent(s): 954a874

v1.9.1: V-JEPA2 re-measured on public SDK 2.33 (docs only, no code change)

Browse files
Files changed (2) hide show
  1. CHANGELOG.md +18 -0
  2. README.md +55 -5
CHANGELOG.md CHANGED
@@ -1,5 +1,23 @@
1
  # Changelog
2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ## v1.9 (2026-10)
4
 
5
  * **New: padding masks on the kernel path for `NeuronXLMRobertaModel`.** v1.8 sent any batch with a padding
 
1
  # Changelog
2
 
3
+ ## v1.9.1 (2026-11-08)
4
+
5
+ Documentation only. No code change: the kernels and layers are byte-identical to v1.9.
6
+
7
+ * **V-JEPA2 re-measured on the public SDK 2.33 DLAMI** (`torch_neuronx_2_14_0` venv: torch 2.14.0,
8
+ torch-neuronx 2.14.3.0, NKI 0.7.0, neuronx-cc 2.28.7631; transformers 5.22.2, kernels 0.17.3) on trn2.3xlarge.
9
+ The kernel source needed no change for this toolchain.
10
+ * Per instance: 44.5 to **159.6 clips/s (3.59x)** at LNC=2, batch 1 per core. On Beta 6 this was 4.27x.
11
+ * Single core, LNC=2, batch 1: 82.6 to **23.8 ms/clip (3.48x)** wall-clock; 23.54 ms device time at 45.1% MFU.
12
+ On Beta 6 this was 3.93x.
13
+ * The speedup narrowed because the stock path got faster on 2.33. The kernels themselves are unchanged or slightly faster.
14
+ * Accuracy: rel_l2 vs fp32 is 3.9e-2 in all four configurations, identical to Beta 6.
15
+ * Faster than the original `aws-neuron/vjepa2-neuron-kernels` v1 layer on the same instance: 23.8 vs 26.7 ms/clip.
16
+ * Documented that the stock V-JEPA2 graph at LNC=1, batch 1 is 2.1e-1 from fp32 on SDK 2.33. The LNC=2 and
17
+ LNC=1 batch-2 graphs are 1.4e-2.
18
+ * Requirements: SDK 2.33 install notes (transformers 5.22.2 requires `kernels>=0.17.2,<0.18`), and use the
19
+ `forward` graft rather than `KernelConfig`. DINOv3 is still verified on Beta 6 only.
20
+
21
  ## v1.9 (2026-10)
22
 
23
  * **New: padding masks on the kernel path for `NeuronXLMRobertaModel`.** v1.8 sent any batch with a padding
README.md CHANGED
@@ -38,7 +38,9 @@ have been validated.
38
 
39
  ## Performance (trn2.3xlarge, bf16, `torch.compile(backend="neuron")`)
40
 
41
- Measured 2026-10 on PyTorch Native Beta 6 (torch 2.13.0, torch-neuronx 2.13.3, NKI 0.7.0b1).
 
 
42
  Real Hugging Face weights. "Stock" is the same model compiled with
43
  `torch.compile(backend="neuron")` without these kernels: DINOv3 cast to bf16, V-JEPA2 as its fp32
44
  checkpoint (stock V-JEPA2 is the encoder as shipped, and its accuracy is in the table below).
@@ -52,7 +54,7 @@ checkpoint (stock V-JEPA2 is the encoder as shipped, and its accuracy is in the
52
  | DINOv3 ViT-L/16 | 448 | 201.1 | **491.9** | 2.45x |
53
  | DINOv3 ViT-H+/16 | 224 | 400.0 | **626.6** | 1.57x |
54
  | DINOv3 ViT-H+/16 | 448 | 106.8 | **200.3** | 1.88x |
55
- | V-JEPA2 ViT-L, 16 frames x 256 px | - | 36.8 clips/s | **157.3 clips/s** | 4.27x |
56
 
57
  Configuration: LNC=1 (8 logical cores) for every row except V-JEPA2 stock, whose best was LNC=2
58
  (4 cores). DINOv3 kernels run at batch 8 (224 px) or batch 2 (448 px). DINOv3 stock runs at batch 4
@@ -67,12 +69,47 @@ Configuration: LNC=1 (8 logical cores) for every row except V-JEPA2 stock, whose
67
  | DINOv3 ViT-L/16 | 448 | 25.1 img/s | **83.0 img/s** | 3.31x |
68
  | DINOv3 ViT-H+/16 | 224 | 21.4 img/s | **88.2 img/s** | 4.12x |
69
  | DINOv3 ViT-H+/16 | 448 | 17.7 img/s | **32.4 img/s** | 1.83x |
70
- | V-JEPA2 ViT-L, 16 frames x 256 px | - | 101.5 ms/clip | **25.8 ms/clip** | 3.93x |
71
 
72
  The gain grows with resolution: attention cost grows with token count, and these kernels keep
73
  attention and its surroundings on chip. At 224 px with large batches the gap narrows. For example,
74
  stock ViT-L/16 at LNC=2 batch 4 is 252 img/s and the kernels give 321 img/s (1.27x).
75
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
76
  ## Accuracy
77
 
78
  Every result above passes a per-image check against the fp32 CPU model (rel_l2 <= 5e-2).
@@ -85,10 +122,16 @@ LayerNorm statistics stay in fp32:
85
  | DINOv3 ViT-L/16 | 1.3-1.6e-2 | 0.8-1.0e-2 |
86
  | DINOv3 ViT-H+/16 | 1.0-1.7e-2 | 0.6-1.0e-2 |
87
  | V-JEPA2 ViT-L | 1.4e-2 (fp32 model, compiler-chosen precision) | 3.9e-2 (bf16 matmuls) |
 
88
 
89
  For V-JEPA2 the stock path compiles the fp32 checkpoint and keeps more of it in higher precision,
90
  so it is closer to fp32 than the kernels (which run every matmul in bf16).
91
 
 
 
 
 
 
92
  ## Quick start
93
 
94
  ```python
@@ -205,9 +248,16 @@ RoPE pairings via `rope=`:
205
 
206
  * trn2 (verified on trn2.3xlarge, LNC=1 and LNC=2).
207
  * PyTorch Native (TorchNeuron) with NKI 0.7 and `nkilib`. Verified on Beta 6 (torch 2.13.0,
208
- torch-neuronx 2.13.3, NKI 0.7.0b1, neuronx-cc 2.0.404056) and (v1.8) on the **SDK 2.33 DLAMI**
209
  `torch_neuronx_2_14_0` venv (torch 2.14.0, torch-neuronx 2.14.3.0, NKI 0.7.0, neuronx-cc 2.28.7631).
210
- * Transformers with DINOv3 / V-JEPA2 (verified on 4.57.6); `kernels` for `get_kernel`.
 
 
 
 
 
 
 
211
 
212
  ## Repository layout
213
 
 
38
 
39
  ## Performance (trn2.3xlarge, bf16, `torch.compile(backend="neuron")`)
40
 
41
+ The two tables below were measured 2026-10 on PyTorch Native Beta 6 (torch 2.13.0, torch-neuronx
42
+ 2.13.3, NKI 0.7.0b1). V-JEPA2 has been re-measured on the public SDK 2.33 release; see
43
+ [V-JEPA2 on SDK 2.33](#v-jepa2-on-sdk-233) below. The DINOv3 rows have not been re-measured on 2.33.
44
  Real Hugging Face weights. "Stock" is the same model compiled with
45
  `torch.compile(backend="neuron")` without these kernels: DINOv3 cast to bf16, V-JEPA2 as its fp32
46
  checkpoint (stock V-JEPA2 is the encoder as shipped, and its accuracy is in the table below).
 
54
  | DINOv3 ViT-L/16 | 448 | 201.1 | **491.9** | 2.45x |
55
  | DINOv3 ViT-H+/16 | 224 | 400.0 | **626.6** | 1.57x |
56
  | DINOv3 ViT-H+/16 | 448 | 106.8 | **200.3** | 1.88x |
57
+ | V-JEPA2 ViT-L, 16 frames x 256 px | - | 36.8 clips/s | **157.3 clips/s** | 4.27x (SDK 2.33: 3.59x, below) |
58
 
59
  Configuration: LNC=1 (8 logical cores) for every row except V-JEPA2 stock, whose best was LNC=2
60
  (4 cores). DINOv3 kernels run at batch 8 (224 px) or batch 2 (448 px). DINOv3 stock runs at batch 4
 
69
  | DINOv3 ViT-L/16 | 448 | 25.1 img/s | **83.0 img/s** | 3.31x |
70
  | DINOv3 ViT-H+/16 | 224 | 21.4 img/s | **88.2 img/s** | 4.12x |
71
  | DINOv3 ViT-H+/16 | 448 | 17.7 img/s | **32.4 img/s** | 1.83x |
72
+ | V-JEPA2 ViT-L, 16 frames x 256 px | - | 101.5 ms/clip | **25.8 ms/clip** | 3.93x (SDK 2.33: 3.48x, below) |
73
 
74
  The gain grows with resolution: attention cost grows with token count, and these kernels keep
75
  attention and its surroundings on chip. At 224 px with large batches the gap narrows. For example,
76
  stock ViT-L/16 at LNC=2 batch 4 is 252 img/s and the kernels give 321 img/s (1.27x).
77
 
78
+ ### V-JEPA2 on SDK 2.33
79
+
80
+ Re-measured 2026-11-08 with v1.9 (`get_kernel(..., version=1)`). The environment was the public
81
+ SDK 2.33 DLAMI `torch_neuronx_2_14_0` venv on trn2.3xlarge: torch 2.14.0, torch-neuronx 2.14.3.0,
82
+ NKI 0.7.0, neuronx-cc 2.28.7631.0, transformers 5.22.2, kernels 0.17.3. The input was 16 frames
83
+ x 256 px, from the fp32 checkpoint. Stock is that model compiled with
84
+ `torch.compile(backend="neuron")`. The kernel source needed no change for this toolchain.
85
+
86
+ | | Stock | **These kernels** | Speedup |
87
+ |---|---|---|---|
88
+ | Per instance, best config | 44.5 clips/s | **159.6 clips/s** | **3.59x** |
89
+ | Single core, LNC=2, batch 1, wall-clock | 82.6 ms/clip | **23.8 ms/clip** | 3.48x |
90
+ | Single core, LNC=2, batch 1, device time (neuron-explorer) | 82.57 ms, 12.7% MFU | **23.54 ms, 45.1% MFU** | 3.51x |
91
+ | Single core, LNC=1, batch 1, wall-clock | 165.2 ms/clip | **51.8 ms/clip** | 3.19x |
92
+
93
+ **Per-instance runs** (one process per logical core, clips/s):
94
+
95
+ | Batch per core | Stock LNC=2 (4 cores) | Stock LNC=1 (8 cores) | Kernels LNC=2 (4 cores) | Kernels LNC=1 (8 cores) |
96
+ |---|---|---|---|---|
97
+ | 1 | 44.5 | 44.7 | **159.6** | 149.8 |
98
+ | 2 | 44.3 | 45.1 | 143.1 | 147.4 |
99
+ | 4 | 42.2 | 44.7 | - | - |
100
+
101
+ - **Best configuration changed.** On SDK 2.33 the kernels do best at LNC=2 with batch 1. On Beta 6
102
+ the best was LNC=1 with 8 cores.
103
+ - **Stock is flat.** It stays at 42-45 clips/s across every batch size and LNC setting.
104
+ - **Compared with Beta 6:**
105
+ - The stock path got faster: 36.8 to 44.5 clips/s per instance, and 101.5 to 82.6 ms single-core.
106
+ - The kernels stayed about the same: 157.3 to 159.6 clips/s, and 25.8 to 23.8 ms.
107
+ - The speedup therefore narrowed, from 4.27x to 3.59x per instance and from 3.93x to 3.48x
108
+ single-core.
109
+ - **This repo is faster than the original.** On the same SDK 2.33 instance, wall-clock and same
110
+ input, the original [`aws-neuron/vjepa2-neuron-kernels`](https://huggingface.co/kernels/aws-neuron/vjepa2-neuron-kernels)
111
+ v1 layer is 26.7 ms/clip. This repo's v1.9 is 23.8 ms/clip.
112
+
113
  ## Accuracy
114
 
115
  Every result above passes a per-image check against the fp32 CPU model (rel_l2 <= 5e-2).
 
122
  | DINOv3 ViT-L/16 | 1.3-1.6e-2 | 0.8-1.0e-2 |
123
  | DINOv3 ViT-H+/16 | 1.0-1.7e-2 | 0.6-1.0e-2 |
124
  | V-JEPA2 ViT-L | 1.4e-2 (fp32 model, compiler-chosen precision) | 3.9e-2 (bf16 matmuls) |
125
+ | V-JEPA2 ViT-L, SDK 2.33 | 1.4e-2 (LNC=2, and LNC=1 batch 2); **2.1e-1 at LNC=1 batch 1** | 3.9e-2 (all four configs, identical to Beta 6) |
126
 
127
  For V-JEPA2 the stock path compiles the fp32 checkpoint and keeps more of it in higher precision,
128
  so it is closer to fp32 than the kernels (which run every matmul in bf16).
129
 
130
+ On SDK 2.33, the stock V-JEPA2 graph compiled at LNC=1 with batch 1 is 2.1e-1 from fp32. That
131
+ result was the same across 4 runs and 2 processes. The LNC=2 graph and the LNC=1 batch-2 graph
132
+ are 1.4e-2. The kernels are 3.9e-2 in every configuration. If you compare against stock at LNC=1,
133
+ use batch 2 or LNC=2 as the reference.
134
+
135
  ## Quick start
136
 
137
  ```python
 
248
 
249
  * trn2 (verified on trn2.3xlarge, LNC=1 and LNC=2).
250
  * PyTorch Native (TorchNeuron) with NKI 0.7 and `nkilib`. Verified on Beta 6 (torch 2.13.0,
251
+ torch-neuronx 2.13.3, NKI 0.7.0b1, neuronx-cc 2.0.404056) and on the **SDK 2.33 DLAMI**
252
  `torch_neuronx_2_14_0` venv (torch 2.14.0, torch-neuronx 2.14.3.0, NKI 0.7.0, neuronx-cc 2.28.7631).
253
+ SDK 2.33 coverage: XLM-RoBERTa since v1.8, and V-JEPA2 since v1.9.1. DINOv3 has been verified only
254
+ on Beta 6.
255
+ * Transformers with DINOv3 / V-JEPA2 (verified on 4.57.6 and 5.22.2); `kernels` for `get_kernel`.
256
+ The SDK 2.33 venv does not include either package. One way to add them without touching its torch:
257
+ `pip install --target ~/xpkgs --no-deps "transformers==5.22.2" "kernels>=0.17.2,<0.18" tokenizers safetensors regex typer-slim accelerate`,
258
+ then `export PYTHONPATH=~/xpkgs`. `transformers` 5.22.2 rejects `kernels` 0.18.x.
259
+ * Use the `forward` graft shown in the quick start. `transformers.KernelConfig` cannot map
260
+ `VJEPA2Encoder`, because it is not registered with `use_kernel_forward_from_hub`.
261
 
262
  ## Repository layout
263