Add the 2026-10-23 parameter shard, ring masks, and README notes.

A stage with two or more layers keeps one flat copy and unpacks at most two layers. A sliding window is a softmax mask. JSON and the position seed change only the pick. DeepGEMM's 128-channel scale stays out of the GEMM. The four docs list what was added and what was left out.

GUIDE.md

@@ -26,6 +26,14 @@ The rest of this file is the previous guide. This list, and every later note mar
 - One weight range stays in flight while the previous body is widened to f32.
 - A dropped peer or a stalled lap is replayed once. Weights still come from the Hub.
 
+## Added 2026-10-23
+
+- A stage with two or more layers packs those weights into one flat buffer and unpacks at most two layers during a forward. Embeddings and `lm_head` stay put. Weights do not cross the ring.
+- Optimizer states, reduced gradients, a peer parameter all-gather, and tiled re-quantization are not implemented.
+- From DeepInfer: sliding-window softmax mask, sampling tied to the token position, a JSON mask on the top 40 completion tokens, and tokenizing the next queued prompt during the current turn.
+- Not from DeepInfer: CUDA kernels, expert parallelism, a confidence-head draft model, compact KV replay, disaggregated prefill.
+- DeepGEMM's FP8 and FP4 kernels are not used. A scale every 128 channels of K is a different number from the one INT8 scale per row and per column.
+
 ```bash
 npm install
 npm start            # http://localhost:8788
@@ -78,6 +86,25 @@ The compute units are the live DaisyChain-Train browser cores (Hub revision `a28
 - One weight range stays in flight while the previous body is widened to f32.
 - A dropped peer or a stalled lap is replayed **once** from the start. Weights still come from the Hub.
 
+## Parameter shard *(added 2026-10-23)*
+
+ZeRO stages 1 and 2 do not apply here. There is no optimizer and no backward. Parameters are not gathered from other devices.
+
+When a stage owns two or more layers, those weights sit in one flat buffer. A forward unpacks the current layer and the next one, runs the layer, then drops the one just used. At most **2** layers are unpacked. A single layer stays resident. The embedding and `lm_head` are not in the shard. The bytes the GEMM reads are the same bytes, so a full-window split stays bit-identical. The flat buffer does not reduce the stage below one copy of its parameters.
+
+## DeepInfer *(added 2026-10-23)*
+
+DeepInfer's CUDA kernels, expert parallelism, and KV cluster are not used.
+
+- A model with `use_sliding_window` and a positive `sliding_window` masks attention outside that window on layers at or after `max_window_layers`. The GEMM still includes every key, so the int8 scales do not change. Compact replay, which drops old keys, is not done.
+- Above temperature 0, each token position has its own draw from the seed.
+- Output set to JSON keeps the completion inside a JSON value, using the top **40** tokens. The hidden state is already finished before the mask.
+- The next queued prompt is tokenized while the current turn runs. One turn still owns the ring.
+
+## DeepGEMM *(added 2026-10-23)*
+
+DeepGEMM's CUDA GEMMs, grouped expert GEMMs, and indexer are not used. The verified GEMM keeps one INT8 scale for a whole activation row and one for a whole weight column. A 128-channel scale would change the products. K above **131071** is still refused.
+
 ## The ring
 
 ```
@@ -178,6 +205,7 @@ Without those, a test where both sides call the same code proves nothing.
 *(added 2026-10-22)* The same file also checks a dropped probe, a refused non-finite activation,
 a chunked prefill across a split, and a lookahead draft across a split. The
 chunked path is compared to itself, not to the full-window GEMM.
+*(added 2026-10-23)* The same file checks that a flat parameter shard matches a resident forward, on the full window and on one chunk, and that a third live layer is refused. It also checks a sliding-window mask across a split, a JSON mask, a position seed, a prepared prompt, and that a 128-channel weight scale differs from the verified column scale.
 
 This was also confirmed against a real model: SmolLM-135M's 30 layers split
 `[10,10,10]`, `[1,14,15]`, `[0,15,15]` and `[5,5,5,5,5,5]` all produced token

README.md

@@ -31,6 +31,14 @@ The rest of this file is the previous README. Everything in this list, and every
 - One weight range stays in flight while the previous body is widened to f32.
 - A planned peer other than the head dropping, or a stalled lap, replays the prompt once. Weights still come from the Hub.
 
+## Added 2026-10-23
+
+- ZeRO and FSDP, forward only, inside one stage. A stage with two or more layers packs those weights into one flat buffer and unpacks at most two layers for the forward. Embeddings and `lm_head` stay as they are. Weights still come from the Hub and do not cross the ring.
+- ZeRO stages 1 and 2 are not implemented. Infer has no optimizer and no backward. Parameters are not all-gathered across peers. `TiledLinear` is not used.
+- From DeepInfer: a sliding-window mask on layers that configure one, sampling tied to the token position, a JSON mask on the top 40 completion tokens, and tokenizing the next queued prompt while the current turn still runs.
+- Not taken from DeepInfer: CUDA kernels, expert parallelism, a draft-model confidence head, compact KV replay, disaggregated prefill, and a KV store.
+- DeepGEMM is not a second GEMM. Its FP8 and FP4 kernels, 128-channel scales, grouped MoE GEMM, and indexer stay out. The verified path keeps one INT8 scale per activation row and one per weight column.
+
 ---
 
 > **In plain terms:** point it at a Hugging Face model, open the page on two or
@@ -45,7 +53,7 @@ machines:
 ```bash
 npm install
 npm start            # http://localhost:8788
-npm test             # 47 checks: pipeline equivalence, wire protocol, loader
+npm test             # 59 checks: pipeline equivalence, wire protocol, loader
 ```
 
 Full walkthrough: **[docs/GETTING_STARTED.md](docs/GETTING_STARTED.md)** ·
@@ -98,6 +106,60 @@ On top of that, the head does a few things the old loop did not:
 - Weight fetches keep one HTTP range in flight while the previous body is widened to f32. The bytes are the same.
 - If a peer disconnects or a lap stalls, the head drops that peer, replans, and reruns the prompt **once** from the start. Weights still come from the Hub, not from another device.
 
+## Parameter shard *(added 2026-10-23)*
+
+ZeRO and FSDP split training state across data-parallel ranks. This project does not train, so the parts that move optimizer moments or gradients are left out.
+
+What a stage does when it owns two or more layers:
+
+- Those layers' float32 weights are copied once into one flat buffer. The per-layer arrays are then dropped. The embedding, the final norm, and `lm_head` stay where they are. `lm_head` stays on the head.
+- A forward gathers the current layer as views into that buffer, prefetches the next layer, runs the same layer, then releases the one just used. At most **2** layers are unpacked.
+- The views alias the flat buffer. The GEMM reads them. The full-window path stays bit-identical to keeping every layer resident. The chunked path uses the same gather.
+- A stage with one layer keeps that layer resident.
+- Peak RAM is still one copy of the stage's parameters. The flat buffer does not make the slice smaller.
+- Weights are not sent to another device. Each device still fetches its own slice from the Hub.
+
+Left out on purpose:
+
+- ZeRO stage 1 (optimizer states) and stage 2 (reduced gradients). There is no Adam state and no backward.
+- A parameter all-gather across the ring. That would move weights on every token.
+- `TiledLinear`. Re-quantizing a matrix per tile would change the int8 scales.
+- NVMe offload. The flat buffer is the CPU-resident copy.
+
+## DeepInfer *(added 2026-10-23)*
+
+[DeepInfer](https://github.com/deepseek-ai/DeepInfer) serves DeepSeek-V4.1-Flash on NVIDIA SM100 and Ascend. Most of that stack does not fit a browser ring that only runs Llama-style and GPT-2-style blocks.
+
+Taken:
+
+- **Sliding-window mask.** If `use_sliding_window` is set and `sliding_window` is positive, layers with index `>= max_window_layers` drop attention outside that window. The GEMM still sees every key column, so the int8 scales stay the scales of the full matrix. A window wider than the cache matches full causal attention. `use_sliding_window: false` leaves the mask off.
+- **Position seed.** Above temperature 0, the draw for token position `p` depends on the seed and `p`. Scoring a longer draft does not move a later draw.
+- **JSON mask.** The Output control can require the completion to stay inside a JSON value. The head checks the top **40** tokens after the hidden state is already computed. No legal token in that set falls back to the unconstrained pick. This does not change the hop.
+- **Queued prompt.** The next prompt is tokenized while the current turn still holds the ring. The forward stays one turn at a time.
+
+Left out:
+
+- FlashMLA, DeepGEMM, DeepEP, TileKernels, and the other CUDA kernels. The verified INT8 units stay.
+- Attention data parallel, context parallel, MoE expert parallel, and expert load balancing.
+- DSpark's confidence head and next-n draft model. The prompt-lookup draft stays capped at **4** and is scored in the same window. Shortening that draft would not shrink the GEMM.
+- Compact SWA replay (the V4.1 keep of **140**, or a **128**-token recompute). Dropping key rows would change int8 scales. Prefix hits still recompute.
+- Disaggregated prefill and decode, the 3FS KV store, FP4 weights, and the multi-protocol recipe tokenizer.
+
+## DeepGEMM *(added 2026-10-23)*
+
+[DeepGEMM](https://github.com/deepseek-ai/DeepGEMM) is a CUDA kernel library for SM90 and SM100. It compiles FP8, FP4, and BF16 GEMMs at runtime, plus grouped MoE GEMMs, a fused Mega MoE, and the V3.2 indexer. None of those kernels run in this browser ring.
+
+The numeric recipe is the part that would matter, and it does not match. DeepGEMM's FP8 GEMM uses one scale per **128** channels of K (`BLOCK_K == 128`). A spike in a later block does not widen the scale of an earlier block. `vgemmBlock` uses one INT8 scale for the whole activation row and one for the whole weight column. Swapping in the 128-channel scale would change every product, including the split against the unsplit run. The test `a 128-channel weight scale is not the verified column scale` pins that difference. The forward path does not call a block scale.
+
+Also left out:
+
+- Grouped GEMMs, masked grouped GEMMs, and the K-grouped backward. This ring has no experts and no backward.
+- Mega MoE, NVLink overlap, and the lightning-indexer ReLU logits.
+- UE8M0 packed scales, TMA layout, programmatic dependent launch, and JIT CUDA.
+- A non-deterministic fast GEMM. The CPU mirror and the GPU kernel stay on the same INT8 units. K above **131071** is refused so the int32 accumulator does not wrap.
+
+Weights are already stored `k x n` once at load. That is the layout a non-transposed A times a transposed B expects. It is not a new kernel.
+
 ## The ring
 
 ```
@@ -198,7 +260,13 @@ Without those, a test where both sides call the same code proves nothing.
 *(added 2026-10-22)* The same file checks that a disagreeing probe is dropped, that a non-finite
 activation is refused, that a chunked prefill matches across a split, and that
 a lookahead draft matches across a split. The chunked path is compared to
-itself, not to the full-window GEMM.
+itself, not to the full-window GEMM. *(added 2026-10-23)* It also checks that a
+flat parameter shard matches a resident forward on both the full-window path
+and a chunk, that the flat bytes are unchanged, and that a third live layer
+is refused. *(added 2026-10-23)* It also checks a sliding-window mask across a
+split, a JSON mask, a position seed, and a prepared prompt id list. A
+128-channel weight scale is checked to be a different number from the
+verified whole-column scale.
 
 Confirmed on a real model too: SmolLM-135M's 30 layers split `[10,10,10]`,
 `[1,14,15]`, `[0,15,15]` and `[5,5,5,5,5,5]` all produced token sequences
@@ -236,8 +304,8 @@ public/safetensors.js     header parsing, exact BF16/F16 widening, range coalesc
 public/hf.js              Hub client: range reads, gated repos, token redaction
 public/arch.js            config.json -> normalized spec; per-family tensor names
 public/tokenizer.js       byte-level BPE from the repo's own tokenizer.json
-public/shard.js           the plan, probe filter, lap estimate, prefix index, drafts   (added 2026-10-22)
-public/infer.js           forward pass, chunked prefill, lookahead readout          (added 2026-10-22)
+public/shard.js           the plan, probe filter, lap estimate, prefix index, drafts   (added 2026-10-22); flat parameter shard (added 2026-10-23)
+public/infer.js           forward pass, chunked prefill, lookahead readout          (added 2026-10-22); gather window of 2 (added 2026-10-23)
 public/wire.js            binary protocol + routing, as pure functions
 public/app.js             WebRTC mesh, assignments, queue, replay, token loop       (added 2026-10-22)
 public/verified_core.js   the verified INT8 units      (copied from live DaisyChain-Train, 2026-10-22)
@@ -262,6 +330,9 @@ docs/                     GETTING_STARTED, ARCHITECTURE
   4 bytes per parameter it holds.
 - **The head is a single point of failure.** *(added 2026-10-22.)* A stage that drops is removed and the
   prompt is replayed once. If that also fails, press Generate again.
+- **No optimizer shard, and no weight gather across peers.** *(added 2026-10-23.)* A stage with two or more layers keeps one flat copy and unpacks at most two layers. The slice is not smaller than one copy of its parameters. Weights still come from the Hub.
+- **No DeepInfer kernel stack.** *(added 2026-10-23.)* No expert parallel, no draft-model head, no compact KV drop, no disaggregated prefill. A sliding window is a softmax mask. JSON is a pick mask.
+- **No DeepGEMM kernel.** *(added 2026-10-23.)* FP8, FP4, 128-channel scales, and grouped MoE GEMMs are not used. One INT8 scale covers each activation row and each weight column.
 - **No authentication of activations.** A malicious stage that runs correct math
   but returns a crafted activation is not caught by any of this. Peers see each
   other's IPs, and the head sees your prompt. Ring with people you trust.

docs/ARCHITECTURE.md

@@ -15,6 +15,15 @@ The rest of this file is the previous architecture note. This list is what chang
 - Prompts longer than 32 tokens on more than one stage use even chunks of about 16, two in flight, with per-stage KV. Not the full-window GEMM.
 - A second prompt is queued. A dropped planned peer or a stalled lap replays the prompt once. A non-finite hidden state is not forwarded.
 
+## Added 2026-10-23
+
+- `makeParamShard` packs a stage's layers into one flat Float32 buffer when the stage owns two or more. `runLayers` and `runLayersChunk` gather the current layer, prefetch the next, and release the one just used. The limit is 2 live layers.
+- Head tensors stay outside that buffer. A one-layer stage is not sharded. No optimizer state is stored. No parameter is sent to a peer.
+- A configured sliding window is a softmax mask. Key columns stay in the GEMM. `swaFrom` is `max_window_layers`.
+- `seqDraw(seed, pos)` samples one position. `pickStructured` applies a JSON mask to the top 40 logits on the head. `preparedIds` is the prompt tokenized during the previous turn.
+- DeepInfer kernels, expert parallel, the DSpark confidence head, compact KV truncation, and prefill/decode disaggregation are not implemented.
+- DeepGEMM is not linked. `quantizeCols` keeps one scale per weight column for the whole K. A 128-channel scale is a different recipe and is not called from `vgemmBlock`. `MAX_INT32_K` is 131071.
+
 ## The pieces
 
 | File | Role |
@@ -110,6 +119,8 @@ credentials. Weights never travel between peers, which means:
   rather than nine per layer;
 - *(added 2026-10-22)* the next range is requested while the previous body is widened to f32.
   The bytes do not change.
+- *(added 2026-10-23)* after the slice is arranged, a stage with two or more layers copies those layer weights into one flat buffer and drops the per-layer arrays. Embeddings, the final norm, and `lm_head` stay as separate arrays. A forward gathers at most two layers as views into the flat buffer (`forward_prefetch`, `limit_all_gathers` of 2, reshard after the layer). The views alias the buffer, so the GEMM sees the same bytes. ZeRO stage 1, ZeRO stage 2, a cross-peer all-gather, and tiled re-quantization are not done.
+- *(added 2026-10-23)* `layerForward` reads `ly.swa`. A positive window limits the softmax, not the score GEMM. The next queued prompt is tokenized during the wait. JSON masking happens in `sampleAt` after logits exist.
 
 ## One token
 

docs/GETTING_STARTED.md

@@ -11,6 +11,12 @@ The rest of this file is the previous getting-started note. This list is what ch
 - New log lines: lap estimate for the whole plan, queued prompts, overlap prefill, draft span. A disagreeing probe is left out of the plan.
 - New failures: "a peer left the ring" (one replay), "REFUSED activation" for a non-finite hop, "refusing the run" when the lap estimate exceeds 120 seconds, "stays out of the plan: kernel probe".
 
+## Added 2026-10-23
+
+- A device that owns two or more layers shows `flat shard, 2 layers live` on its slice line. The weights are one buffer on that device. They are not fetched from a peer. A device with one layer still says `resident`.
+- Output can be `free` or `JSON`. JSON masks the completion. The seed above temperature 0 is tied to the token position. A prompt typed while a turn is running is tokenized before that turn ends.
+- DeepGEMM is not used. The multiply stays the verified INT8 GEMM, one scale per row and one per column.
+
 ## Run the server
 
 ```bash
@@ -119,7 +125,8 @@ few layers and no vocabulary.
 - **overlap prefill** — *(added 2026-10-22)* chunks of about 16, two in flight, per-stage KV. Not the
   full-window GEMM. Verify repeats this path.
 - **draft span** — *(added 2026-10-22)* a repeated suffix scored in one lap on the full-window path.
-- **my slice** — which layers this device holds and how much memory they take.
+- **my slice** — which layers this device holds and how much memory they take. *(added 2026-10-23)* Two or more layers are kept in one flat shard, with at most two unpacked during a forward. That does not shrink the download.
+- **Output** — *(added 2026-10-23)* `free` or `JSON`. JSON applies after the hop, to the top 40 tokens of the completion.
 
 ## Verify
 

public/app.js

@@ -18,7 +18,7 @@ const ui = {};
 for (const id of ["status", "backend", "me", "peers", "log", "requests", "roomInfo", "roomCode",
                   "copyLink", "model", "tokenizer", "plan", "myShard", "prompt", "genBtn",
                   "stopBtn", "out", "tps", "tokcount", "hops", "temp", "vtemp", "ntok", "vntok",
-                  "seed", "verify", "verifyRow", "ringState", "repo", "revision", "loadRepo",
+                  "seed", "schema", "verify", "verifyRow", "ringState", "repo", "revision", "loadRepo",
                   "modelState", "ctxLen", "vctxLen", "tokenState", "clearToken",
                   "tokModal", "tokInput", "tokOk", "tokCancel", "tokWhy",
                   "tokPasteRow", "tokSignIn", "tokSignInBtn",
@@ -636,7 +636,8 @@ async function loadMyStage(assign, st) {
   stageObj = Infer.makeStage(s, st, w, kernelCtx(), assign.ctx || ctxLen());
   stageObj.pass = assign.pass === "chunk" ? "chunk" : "window";
   spec = s;
-  ui.myShard.textContent = `${st.head ? "head + " : ""}layers ${st.lo}–${st.hi - 1} · ${mb.toFixed(0)} MB resident`;
+  ui.myShard.textContent = `${st.head ? "head + " : ""}layers ${st.lo}–${st.hi - 1} · ${mb.toFixed(0)} MB` +
+    (stageObj.paramShard ? " flat shard, 2 layers live" : " resident");
   return stageObj;
 }
 
@@ -764,11 +765,25 @@ async function onAct(peerId, buf) {
 }
 
 const promptQueue = [];
+let scheduledPrompt = null;
+
+// Tokenize the next queued prompt while this turn still holds the ring.
+// The forward itself stays serial. Weights are not touched here.
+function scheduleQueued() {
+  if (!tok || !promptQueue.length) return;
+  const text = promptQueue[0];
+  if (!text) return;
+  if (scheduledPrompt && scheduledPrompt.text === text) return;
+  let ids = [...Tokenizer.encode(tok, text)];
+  if (!ids.length) ids = [0];
+  scheduledPrompt = { text, ids };
+}
 
 async function generate(isReplay) {
   if (running) {
     const waiting = (ui.prompt.value || "").slice(0, 80);
     promptQueue.push(ui.prompt.value);
+    scheduleQueued();
     log(`ring busy — queued (${promptQueue.length} waiting): "${waiting}". The turn already generating keeps the ring.`);
     return;
   }
@@ -782,8 +797,11 @@ async function generate(isReplay) {
     plan = buildPlan(); modelHash = repoInfo.fingerprint; spec = repoInfo.spec;
     renderPlan();
     const promptText = ui.prompt.value || "The";
-    let idsPreview = [...Tokenizer.encode(tok, promptText)];
+    const prepared = Shard.preparedIds(promptText, scheduledPrompt);
+    scheduledPrompt = null;
+    let idsPreview = prepared ? prepared : [...Tokenizer.encode(tok, promptText)];
     if (!idsPreview.length) idsPreview = [0];
+    if (prepared) log("prompt was scheduled while the previous turn held the ring.");
     const fits = Shard.headCanHold(spec, repoInfo.tensors, navigator.deviceMemory || 0);
     const local = Shard.preferLocal(idsPreview.length, plan.length - 1, fits);
     const chatKey = String((prefixIndex.blocks(idsPreview)[0]) || "empty");
@@ -815,6 +833,7 @@ async function generate(isReplay) {
           pass: willChunk ? "chunk" : "window",
         }));
       log("waiting for every stage to fetch its layers…");
+      scheduleQueued();
       const ok = await waitFor(() => readyStages.size >= plan.length - 1, 600000);
       if (!ok) { log("not every stage reported ready — generating anyway will stall, so stopping here."); throw new Error("stages not ready"); }
     } else {
@@ -826,7 +845,12 @@ async function generate(isReplay) {
 
     const seq = ++runSeq;
     const nTok = +ui.ntok.value, temp = +ui.temp.value / 100;
-    const opts = { temperature: temp, topK: 40, rng: Infer.mulberry32(+ui.seed.value | 0) };
+    const schema = ui.schema && ui.schema.value === "json" ? "json" : "";
+    const opts = {
+      temperature: temp, topK: 40, seqSeed: +ui.seed.value | 0, schema,
+      decodeSeq: (arr) => Tokenizer.decode(tok, arr), promptLen: idsPreview.length,
+    };
+    if (schema === "json") log("JSON mask on the completion. Top 40 tokens. The hidden state is unchanged.");
     let ids = idsPreview.slice();
     prefixIndex.note(chatKey, ids);
     // The bytes kept here are the prefix-block hashes, not layer KV. Reusing
@@ -859,6 +883,7 @@ async function generate(isReplay) {
         await sendTo(plan[1].id, packAct(seq, index, plan[1].index, hidden));
         hops += plan.length;
         const key = `${seq}:${index}`;
+        scheduleQueued();
         const ok = await waitFor(() => actIn.has(key) || peerDropped, 60000);
         if (peerDropped) throw new Error("peer dropped");
         if (!ok) throw new Error("stalled");
@@ -882,7 +907,8 @@ async function generate(isReplay) {
       let produced = 0;
       while (produced < nTok && !abortRun && last) {
         const row = (last.length / spec.hidden) - 1;
-        const id = Infer.pickToken((await Infer.logitsAt(stageObj, last, [row]))[0], opts);
+        const id = Infer.sampleAt((await Infer.logitsAt(stageObj, last, [row]))[0], opts, ids.length,
+                                  opts.decodeSeq, ids.slice(promptLen));
         ids.push(id); produced++;
         ui.out.textContent = Tokenizer.decode(tok, ids);
         ui.tokcount.textContent = String(produced);
@@ -909,13 +935,16 @@ async function generate(isReplay) {
         await sendTo(plan[1].id, packAct(seq, produced, plan[1].index, x));
         hops += plan.length;
         const key = `${seq}:${produced}`;
+        scheduleQueued();
         const ok = await waitFor(() => actIn.has(key) || peerDropped, 60000);
         if (peerDropped) throw new Error("peer dropped");
         if (!ok) throw new Error("stalled");
         x = actIn.get(key); actIn.delete(key);
       }
       const rows = Infer.draftRows(T, frame.shown, draft.length);
-      const picked = (await Infer.logitsAt(stageObj, x, rows)).map(l => Infer.pickToken(l, opts));
+      const logitRows = await Infer.logitsAt(stageObj, x, rows);
+      const picked = logitRows.map((l, i) => Infer.sampleAt(
+        l, opts, ids.length + i, opts.decodeSeq, ids.slice(promptLen).concat(draft.slice(0, i))));
       const got = Shard.acceptDraft(picked, draft);
       if (draft.length) log(`draft span ${draft.length}, accepted ${Math.max(0, got.length - 1)} before the bonus token. One lap.`);
       for (const id of got) {
@@ -940,11 +969,14 @@ async function generate(isReplay) {
       const all = { id: myId, index: 0, lo: 0, hi: spec.layers, head: true };
       const solo = await loadMyStage({ repo: repoInfo.repo, revision: repoInfo.revision, spec, ctx: T }, all);
       const promptIds = Tokenizer.encode(tok, ui.prompt.value || "The");
+      const verifyOpts = {
+        temperature: temp, topK: 40, seqSeed: +ui.seed.value | 0, schema,
+        decodeSeq: (arr) => Tokenizer.decode(tok, arr),
+        promptLen: promptIds.length ? promptIds.length : 1, maxDraft: 4,
+      };
       const ref = useChunks
-        ? await Infer.generateChunked([solo], promptIds.length ? promptIds : [0], nTok, 16,
-                                      { temperature: temp, topK: 40, rng: Infer.mulberry32(+ui.seed.value | 0) })
-        : await Infer.generateLookahead([solo], promptIds.length ? promptIds : [0], nTok, T,
-                                      { temperature: temp, topK: 40, rng: Infer.mulberry32(+ui.seed.value | 0), maxDraft: 4 });
+        ? await Infer.generateChunked([solo], promptIds.length ? promptIds : [0], nTok, 16, verifyOpts)
+        : await Infer.generateLookahead([solo], promptIds.length ? promptIds : [0], nTok, T, verifyOpts);
       const same = ref.length === ids.length && ref.every((v, i) => v === ids[i]);
       log(same ? "VERIFIED: the distributed run and the single-device run produced identical token ids."
                : `MISMATCH: the ring and this device disagree. Distributed "${Tokenizer.decode(tok, ids).slice(0, 60)}" ` +

public/arch.js

@@ -63,7 +63,16 @@
       // GPT-2's Conv1D stores (in, out) — already the k x n our GEMM wants.
       // torch.nn.Linear stores (out, in) and must be transposed at load.
       weightLayout: family === "gpt2" ? "in_out" : "out_in",
+      // Sliding window is mask-only. 0 means full causal attention.
+      // Layers with index < swaFrom stay full even when a window is set
+      // (Qwen2's max_window_layers). Compact KV truncation is not done here.
+      slidingWindow: 0,
+      swaFrom: 0,
     };
+    if (family === "llama" && cfg.use_sliding_window && +cfg.sliding_window > 0) {
+      spec.slidingWindow = cfg.sliding_window | 0;
+      spec.swaFrom = cfg.max_window_layers != null ? cfg.max_window_layers | 0 : 0;
+    }
     if (spec.gated && !cfg.intermediate_size)
       throw new Error("a gated MLP needs intermediate_size in config.json");
     if (spec.kvHeads && spec.heads % spec.kvHeads !== 0)

public/index.html

@@ -177,6 +177,12 @@
     <p class="note" style="margin:.3rem 0 .6rem">0 = greedy, which is deterministic and therefore checkable. Above 0, sampling follows the seed so runs stay reproducible.</p>
     <label class="slbl">Seed</label>
     <input type="number" id="seed" value="1234" style="width:140px">
+    <label class="slbl">Output</label>
+    <select id="schema" style="width:140px">
+      <option value="">free</option>
+      <option value="json">JSON</option>
+    </select>
+    <p class="note" style="margin:.3rem 0 .6rem">JSON constrains the completion only. The mask checks the top 40 tokens. The hidden state stays the same. Above temperature 0, the seed is tied to the token position.</p>
     <div class="row" id="verifyRow" style="display:none;margin-top:10px">
       <label class="note" style="display:flex;gap:8px;align-items:center;cursor:pointer">
         <input type="checkbox" id="verify"> Verify afterwards against a single-device run

public/infer.js

@@ -180,13 +180,17 @@
       const khT = TC.transpose(kh, N, hd);                       // hd x N
       const scores = await V.vgemmBlock(qh, khT, { m: T, k: hd, n: N, batch: 1 }, ctx.L, ctx.bgemm, ctx.audit);
       const a = new Float32Array(T * N);                          // causal softmax
+      const swa = ly.swa | 0;
       for (let i = 0; i < T; i++) {
         const see = n0 + i;                                       // this query's own position in the cache
+        // Mask mode only. Columns outside the window stay in the GEMM, so
+        // per-column int8 scales match a full causal matrix. Softmax drops them.
+        const oldest = swa > 0 ? Math.max(0, see - (swa - 1)) : 0;
         let mx = -Infinity;
-        for (let j = 0; j <= see; j++) mx = Math.max(mx, scores[i * N + j] * scale);
+        for (let j = oldest; j <= see; j++) mx = Math.max(mx, scores[i * N + j] * scale);
         let z = 0;
-        for (let j = 0; j <= see; j++) { const e = Math.exp(scores[i * N + j] * scale - mx); a[i * N + j] = e; z += e; }
-        for (let j = 0; j <= see; j++) a[i * N + j] /= z;
+        for (let j = oldest; j <= see; j++) { const e = Math.exp(scores[i * N + j] * scale - mx); a[i * N + j] = e; z += e; }
+        for (let j = oldest; j <= see; j++) a[i * N + j] /= z;
       }
       const oh = await V.vgemmBlock(a, vh, { m: T, k: N, n: hd, batch: 1 }, ctx.L, ctx.bgemm, ctx.audit);
       for (let t = 0; t < T; t++) ctxOut.set(oh.subarray(t * hd, t * hd + hd), t * qDim + h * hd);
@@ -217,8 +221,13 @@
   }
 
   // ---- stage ------------------------------------------------------------------
-  function makeStage(spec, st, w, ctx, T) {
+  function makeStage(spec, st, w, ctx, T, opts) {
     const S = { spec, st, w, ctx, T: T || Math.min(spec.maxPos, 128) };
+    // Two or more layers: one flat parameter shard, gathered per layer.
+    // opts.resident keeps the old layout for the bit-identity check.
+    const resident = opts && opts.resident;
+    S.paramShard = (!resident && w.layers && w.layers.length >= 2)
+      ? SH.makeParamShard(w.layers) : null;
     if (spec.rope) S.rope = ropeTables(spec.headDim, spec.ropeTheta, S.T + 1);
     return S;
   }
@@ -236,12 +245,39 @@
     return x;
   }
 
+  // Walk this stage's layers. With a parameter shard, gather layer i, prefetch
+  // i+1, run the same layerForward, then release i. At most two layers hold
+  // views. Without a shard the arrays stay on the layer objects.
+  async function forwardLayers(S, x, span) {
+    const layers = S.w.layers;
+    const shard = S.paramShard;
+    if (!shard) {
+      if (span) {
+        for (let li = 0; li < layers.length; li++) {
+          span.li = li;
+          x = await layerForward(S, x, layers[li], span);
+        }
+      } else {
+        for (const ly of layers) x = await layerForward(S, x, ly);
+      }
+      return x;
+    }
+    for (let li = 0; li < layers.length; li++) {
+      SH.gatherLayer(shard, layers, li);
+      if (li + 1 < layers.length) SH.gatherLayer(shard, layers, li + 1);
+      if (span) span.li = li;
+      x = await layerForward(S, x, layers[li], span || undefined);
+      SH.releaseLayer(shard, layers, li);
+    }
+    return x;
+  }
+
   async function runLayers(S, x) {
     // A non-finite activation used to quantize to a silent zero and keep
     // travelling. Train refuses it (NonFiniteError). A hop must too: the
     // next stage would otherwise agree on a wrong sum.
     V.assertFinite(x, "activation");
-    for (const ly of S.w.layers) x = await layerForward(S, x, ly);
+    x = await forwardLayers(S, x, null);
     V.assertFinite(x, "activation");
     return x;
   }
@@ -254,10 +290,7 @@
     const C = S.spec.hidden;
     const T = (x.length / C) | 0;
     if (T < 1 || T * C !== x.length) throw new Error("chunk length is not a whole number of rows");
-    for (let li = 0; li < S.w.layers.length; li++) {
-      span.li = li;
-      x = await layerForward(S, x, S.w.layers[li], span);
-    }
+    x = await forwardLayers(S, x, span);
     V.assertFinite(x, "activation");
     return x;
   }
@@ -319,7 +352,8 @@
     }
     for (let n = 0; n < nNew; n++) {
       const row = (last.length / stages[0].spec.hidden) - 1;
-      const id = pickToken((await logitsAt(stages[0], last, [row]))[0], opts);
+      const id = sampleAt((await logitsAt(stages[0], last, [row]))[0], opts, ids.length,
+                          opts && opts.decodeSeq, ids.slice(ids0.length));
       ids.push(id);
       const pos = ids.length - 1;
       let x = embedRows(stages[0], [id], pos);
@@ -347,6 +381,12 @@
 
   // ---- sampling ---------------------------------------------------------------
   function mulberry32(a) { return function () { a |= 0; a = a + 0x6D2B79F5 | 0; let t = Math.imul(a ^ a >>> 15, 1 | a); t = t + Math.imul(t ^ t >>> 7, 61 | t) ^ t; return ((t ^ t >>> 14) >>> 0) / 4294967296; }; }
+  // One draw for absolute position `pos`. A longer or shorter draft does not
+  // move the draw for a later position.
+  function seqDraw(seed, pos) {
+    let a = (seed | 0) ^ Math.imul((pos | 0) + 1, 0x9E3779B1);
+    return mulberry32(a)();
+  }
   function pickToken(logits, opts) {
     const o = opts || {}, temp = o.temperature ?? 0, vocab = logits.length;
     if (!temp) {
@@ -355,16 +395,55 @@
       return best;
     }
     const k = Math.min(o.topK || 40, vocab);
-    const idx = Array.from({ length: vocab }, (_, i) => i).sort((a, b) => logits[b] - logits[a]).slice(0, k);
+    const idx = Array.from({ length: vocab }, (_, i) => i).sort((a, b) => logits[b] - logits[a] || a - b).slice(0, k);
     let mx = -Infinity;
     for (const i of idx) mx = Math.max(mx, logits[i] / temp);
     let z = 0;
     const p = idx.map(i => { const e = Math.exp(logits[i] / temp - mx); z += e; return e; });
-    let r = (o.rng || Math.random)() * z;
+    const draw = (o.seqSeed != null && o.pos != null) ? () => seqDraw(o.seqSeed, o.pos) : (o.rng || Math.random);
+    let r = draw() * z;
     for (let i = 0; i < idx.length; i++) { r -= p[i]; if (r <= 0) return idx[i]; }
     return idx[idx.length - 1];
   }
 
+  // Head-side JSON mask. The hidden state is already computed. Only the pick
+  // changes, and only among the top-K logits. No legal candidate falls back
+  // to the unconstrained pick.
+  function pickStructured(logits, opts) {
+    const o = opts || {};
+    if (o.schema !== "json" || typeof o.decodePiece !== "function") return pickToken(logits, o);
+    const produced = o.produced || "";
+    const vocab = logits.length;
+    const k = Math.min(o.topK || 40, vocab);
+    const idx = Array.from({ length: vocab }, (_, i) => i).sort((a, b) => logits[b] - logits[a] || a - b).slice(0, k);
+    const legal = [];
+    for (const id of idx) {
+      let piece;
+      try { piece = o.decodePiece(id); } catch (e) { continue; }
+      if (piece == null) continue;
+      if (SH.jsonAllows(produced, piece)) legal.push(id);
+    }
+    if (!legal.length) return pickToken(logits, o);
+    if (!(o.temperature ?? 0)) return legal[0];
+    const fake = new Float32Array(vocab);
+    fake.fill(-Infinity);
+    for (const id of legal) fake[id] = logits[id];
+    return pickToken(fake, o);
+  }
+
+  function sampleAt(logits, opts, pos, decodeSeq, baseIds) {
+    const local = Object.assign({}, opts || {}, { pos: pos | 0 });
+    if (local.schema === "json" && typeof decodeSeq === "function" && baseIds) {
+      local.produced = decodeSeq(baseIds);
+      local.decodePiece = (id) => {
+        const prev = local.produced;
+        const next = decodeSeq(baseIds.concat([id]));
+        return next.startsWith(prev) ? next.slice(prev.length) : next;
+      };
+    }
+    return pickStructured(logits, local);
+  }
+
   // ---- single-device reference ------------------------------------------------
   async function generateLocal(stages, ids, nTokens, opts) {
     const head = stages[0], T = head.T;
@@ -386,6 +465,8 @@
     const ids = [...ids0];
     const head = stages[0];
     const maxDraft = opts && opts.maxDraft != null ? opts.maxDraft : 4;
+    const promptLen = opts && opts.promptLen != null ? opts.promptLen | 0 : ids0.length;
+    const decodeSeq = opts && opts.decodeSeq;
     let produced = 0;
     while (produced < nNew) {
       const frame = SH.lookaheadFrame(ids, T, maxDraft);
@@ -396,7 +477,11 @@
       for (const S of stages) x = await runLayers(S, x);
       const rows = draftRows(T, frame.shown, draft.length);
       const picked = [];
-      for (const logits of await logitsAt(head, x, rows)) picked.push(pickToken(logits, opts));
+      const logitRows = await logitsAt(head, x, rows);
+      for (let i = 0; i < logitRows.length; i++) {
+        const base = ids.slice(promptLen).concat(draft.slice(0, i));
+        picked.push(sampleAt(logitRows[i], opts, ids.length + i, decodeSeq, base));
+      }
       const got = SH.acceptDraft(picked, draft);
       if (!got.length) break;
       for (const id of got) {
@@ -410,7 +495,7 @@
 
   const api = { makeStage, embed, embedRows, runLayers, runLayersChunk, readout, logitsAt, draftRows,
                 generateLookahead,
-                pickToken, generateLocal, generateChunked,
+                pickToken, pickStructured, sampleAt, seqDraw, generateLocal, generateChunked,
                 rmsNorm, layerNorm, norm, ropeTables, applyRope, silu, gelu, mulberry32 };
   if (typeof module !== "undefined" && module.exports) {
     TC = require("./traincore.js"); V = require("./verified_core.js"); SH = require("./shard.js");

public/shard.js

@@ -114,6 +114,8 @@
         ly.Wfc = toKN(pick(t.Wfc), spec.inter, C, L); ly.bfc = pick(t.bfc, true);
       }
       ly.Wdown = toKN(pick(t.Wdown), C, spec.inter, L); ly.bdown = pick(t.bdown, true);
+      const sw = spec.slidingWindow | 0;
+      ly.swa = sw > 0 && l >= (spec.swaFrom | 0) ? sw : 0;
       w.layers.push(ly);
     }
     return w;
@@ -125,6 +127,68 @@
     return out;
   }
 
+  // ---- parameter shard (ZeRO-3 / FSDP FULL_SHARD, forward only) --------------
+  // A stage that owns two or more layers keeps those weights in one flat
+  // Float32 buffer. A forward gathers the current layer as views into that
+  // buffer, prefetches the next, runs it, then drops the views. At most two
+  // layers are unpacked (FSDP limit_all_gathers + forward_prefetch +
+  // reshard-after-forward). The flat buffer is the CPU-resident copy.
+  // Embeddings, the final norm, and lm_head are not in this shard.
+  // The views alias the buffer. The GEMM only reads them, so the bytes the
+  // kernel sees are the bytes that were resident. No optimizer state, no
+  // gradient, and no copy of a parameter is sent to a peer.
+  const PARAM_KEYS = ["nrm1", "nrm1b", "nrm2", "nrm2b",
+                      "Wqkv", "bqkv", "Wq", "bq", "Wk", "bk", "Wv", "bv",
+                      "Wo", "bo", "Wgate", "Wup", "Wfc", "bfc", "Wdown", "bdown"];
+  const GATHER_LIMIT = 2;
+
+  function makeParamShard(layers) {
+    if (!layers || layers.length < 2) return null;
+    const slices = [];
+    let n = 0;
+    for (let i = 0; i < layers.length; i++) {
+      const parts = [];
+      const ly = layers[i];
+      for (const k of PARAM_KEYS) {
+        const a = ly[k];
+        if (a && a.length) {
+          parts.push({ k, off: n, len: a.length });
+          n += a.length;
+        }
+      }
+      slices.push(parts);
+    }
+    const flat = new Float32Array(n);
+    for (let i = 0; i < layers.length; i++)
+      for (const p of slices[i]) flat.set(layers[i][p.k], p.off);
+    for (let i = 0; i < layers.length; i++) {
+      for (const p of slices[i]) layers[i][p.k] = null;
+      layers[i]._gathered = false;
+    }
+    return { flat, slices, live: 0, maxLive: 0, limit: GATHER_LIMIT };
+  }
+
+  function gatherLayer(shard, layers, i) {
+    const ly = layers[i];
+    if (ly._gathered) return ly;
+    if (shard.live >= shard.limit)
+      throw new Error("param gather would exceed " + shard.limit + " live layers");
+    for (const p of shard.slices[i])
+      ly[p.k] = shard.flat.subarray(p.off, p.off + p.len);
+    ly._gathered = true;
+    shard.live++;
+    if (shard.live > shard.maxLive) shard.maxLive = shard.live;
+    return ly;
+  }
+
+  function releaseLayer(shard, layers, i) {
+    const ly = layers[i];
+    if (!ly || !ly._gathered) return;
+    for (const p of shard.slices[i]) ly[p.k] = null;
+    ly._gathered = false;
+    shard.live--;
+  }
+
   // ---- hashes -----------------------------------------------------------------
   // FNV-1a over raw bytes, identical to the function DaisyChain-Web hashes
   // replicas with, so hashes stay comparable across the projects.
@@ -333,6 +397,136 @@
     return { draft, shown: shown.length, win: shown.slice() };
   }
 
+  // The text of a prompt that was tokenized while an earlier turn still held
+  // the ring. Used only when the text still matches. Returns a copy.
+  function preparedIds(text, prepared) {
+    if (!prepared || prepared.text !== text || !prepared.ids || !prepared.ids.length) return null;
+    return prepared.ids.slice();
+  }
+
+  // JSON value scanner for a completion constraint. "ok" is a finished value,
+  // "need" is a legal prefix, "bad" is a broken one. Incomplete is allowed.
+  function jsonStatus(text) {
+    const s = String(text || "");
+    let i = 0;
+    const n = s.length;
+    function ws() {
+      while (i < n && (s[i] === " " || s[i] === "\n" || s[i] === "\r" || s[i] === "\t")) i++;
+    }
+    function string() {
+      i++;
+      while (i < n) {
+        const c = s[i++];
+        if (c === "\"") return "ok";
+        if (c === "\\") {
+          if (i >= n) return "need";
+          const e = s[i++];
+          if ("\"\\/bfnrt".indexOf(e) >= 0) continue;
+          if (e === "u") {
+            if (i + 4 > n) return "need";
+            const hex = s.slice(i, i + 4);
+            if (!/^[0-9a-fA-F]{4}$/.test(hex)) return "bad";
+            i += 4;
+            continue;
+          }
+          return "bad";
+        }
+        if (c < " ") return "bad";
+      }
+      return "need";
+    }
+    function number() {
+      if (s[i] === "-") i++;
+      if (i >= n) return "need";
+      if (s[i] === "0") i++;
+      else if (s[i] >= "1" && s[i] <= "9") { while (i < n && s[i] >= "0" && s[i] <= "9") i++; }
+      else return "bad";
+      if (i >= n) return "need";
+      if (s[i] === ".") {
+        i++;
+        if (i >= n) return "need";
+        if (s[i] < "0" || s[i] > "9") return "bad";
+        while (i < n && s[i] >= "0" && s[i] <= "9") i++;
+      }
+      if (i < n && (s[i] === "e" || s[i] === "E")) {
+        i++;
+        if (i >= n) return "need";
+        if (s[i] === "+" || s[i] === "-") i++;
+        if (i >= n) return "need";
+        if (s[i] < "0" || s[i] > "9") return "bad";
+        while (i < n && s[i] >= "0" && s[i] <= "9") i++;
+      }
+      return "ok";
+    }
+    function literal() {
+      const rest = s.slice(i);
+      const words = ["true", "false", "null"];
+      for (let w = 0; w < words.length; w++) {
+        const word = words[w];
+        if (rest.indexOf(word) === 0) { i += word.length; return "ok"; }
+        if (word.indexOf(rest) === 0) return "need";
+      }
+      return "bad";
+    }
+    function value() {
+      ws();
+      if (i >= n) return "need";
+      const c = s[i];
+      if (c === "{") return object();
+      if (c === "[") return array();
+      if (c === "\"") return string();
+      if (c === "-" || (c >= "0" && c <= "9")) return number();
+      return literal();
+    }
+    function object() {
+      i++;
+      ws();
+      if (i >= n) return "need";
+      if (s[i] === "}") { i++; return "ok"; }
+      for (;;) {
+        ws();
+        if (i >= n) return "need";
+        if (s[i] !== "\"") return "bad";
+        const key = string();
+        if (key !== "ok") return key;
+        ws();
+        if (i >= n) return "need";
+        if (s[i] !== ":") return "bad";
+        i++;
+        const v = value();
+        if (v !== "ok") return v;
+        ws();
+        if (i >= n) return "need";
+        if (s[i] === "}") { i++; return "ok"; }
+        if (s[i] !== ",") return "bad";
+        i++;
+      }
+    }
+    function array() {
+      i++;
+      ws();
+      if (i >= n) return "need";
+      if (s[i] === "]") { i++; return "ok"; }
+      for (;;) {
+        const v = value();
+        if (v !== "ok") return v;
+        ws();
+        if (i >= n) return "need";
+        if (s[i] === "]") { i++; return "ok"; }
+        if (s[i] !== ",") return "bad";
+        i++;
+      }
+    }
+    const st = value();
+    if (st !== "ok") return st;
+    ws();
+    if (i < n) return "bad";
+    return "ok";
+  }
+  function jsonAllows(soFar, more) {
+    return jsonStatus(String(soFar || "") + String(more == null ? "" : more)) !== "bad";
+  }
+
   function acceptDraft(picked, draft) {
     const out = [];
     let i = 0;
@@ -345,7 +539,8 @@
                 fnv1a, hashF32, modelFingerprint, ringMembers, estimateLapMs,
                 headCanHold, preferLocal, makePrefixIndex, makeKvSpill,
                 prefillChunks, promptLookupDraft, acceptDraft, lookaheadFrame,
-                estimatePlanMs };
+                estimatePlanMs, makeParamShard, gatherLayer, releaseLayer,
+                PARAM_KEYS, GATHER_LIMIT, preparedIds, jsonStatus, jsonAllows };
   if (typeof module !== "undefined" && module.exports) module.exports = api;
   else root.Shard = api;
 })(typeof self !== "undefined" ? self : this);

test_pipeline.js

@@ -279,6 +279,155 @@ const GPT2 = Arch.fromConfig({
     assert.strictEqual(whole.length, prompt.length + 4);
   });
 
+  await t("a parameter shard matches a resident forward and keeps at most 2 layers live", async () => {
+    for (const spec of [LLAMA, GPT2]) {
+      const got = makeWeights(spec);
+      const st = { id: "p0", index: 0, lo: 0, hi: spec.layers, head: true };
+      const wRes = Shard.stageWeights(spec, st, got, Arch);
+      const wSh = Shard.stageWeights(spec, st, got, Arch);
+      const resident = Infer.makeStage(spec, st, wRes, ctx(), T, { resident: true });
+      const sharded = Infer.makeStage(spec, st, wSh, ctx(), T);
+      assert.ok(sharded.paramShard, "a multi-layer stage should flat-shard");
+      assert.strictEqual(sharded.paramShard.limit, 2);
+      assert.strictEqual(resident.paramShard, null);
+      assert.ok(wSh.emb && wSh.emb.length, "embeddings stay outside the shard");
+      assert.ok(wSh.lmHead && wSh.lmHead.length, "lm_head stays resident");
+      const firstKey = sharded.paramShard.slices[0][0].k;
+      assert.strictEqual(wSh.layers[0][firstKey], null, "weights are packed, not left on the layer");
+      const win = new Int32Array(T);
+      for (let i = 0; i < PROMPT.length; i++) win[T - PROMPT.length + i] = PROMPT[i];
+      const a = await Infer.runLayers(resident, Infer.embed(resident, win));
+      const flatBefore = sharded.paramShard.flat.slice();
+      const b = await Infer.runLayers(sharded, Infer.embed(sharded, win));
+      assert.ok(bitsEqual(a, b), "sharded runLayers diverged from resident");
+      assert.strictEqual(sharded.paramShard.live, 0);
+      assert.strictEqual(sharded.paramShard.maxLive, 2);
+      assert.strictEqual(wSh.layers[0][firstKey], null);
+      assert.strictEqual(wSh.layers[spec.layers - 1][firstKey], null);
+      assert.ok(bitsEqual(flatBefore, sharded.paramShard.flat), "the forward wrote the flat shard");
+      for (let i = 0; i < wRes.layers.length; i++) {
+        for (const p of sharded.paramShard.slices[i]) {
+          const src = wRes.layers[i][p.k];
+          assert.strictEqual(src.length, p.len);
+          for (let j = 0; j < p.len; j++)
+            if (sharded.paramShard.flat[p.off + j] !== src[j])
+              throw new Error("flat shard byte mismatch at layer " + i);
+        }
+      }
+      const ids = [1, 2, 3, 4];
+      const spanR = { pos0: 0, cache: [] };
+      const spanS = { pos0: 0, cache: [] };
+      const cr = await Infer.runLayersChunk(resident, Infer.embedRows(resident, ids, 0), spanR);
+      const cs = await Infer.runLayersChunk(sharded, Infer.embedRows(sharded, ids, 0), spanS);
+      assert.ok(bitsEqual(cr, cs), "sharded chunk diverged from resident");
+      assert.ok(bitsEqual(spanR.cache[0].k, spanS.cache[0].k));
+      assert.ok(bitsEqual(spanR.cache[0].v, spanS.cache[0].v));
+      assert.strictEqual(sharded.paramShard.live, 0);
+      assert.ok(bitsEqual(flatBefore, sharded.paramShard.flat));
+      Shard.gatherLayer(sharded.paramShard, wSh.layers, 0);
+      Shard.gatherLayer(sharded.paramShard, wSh.layers, 1);
+      assert.throws(() => Shard.gatherLayer(sharded.paramShard, wSh.layers, 2), /2 live/);
+      Shard.releaseLayer(sharded.paramShard, wSh.layers, 0);
+      Shard.releaseLayer(sharded.paramShard, wSh.layers, 1);
+      assert.strictEqual(sharded.paramShard.live, 0);
+    }
+    const st1 = { id: "p0", index: 0, lo: 0, hi: 1, head: true };
+    const one = Infer.makeStage(LLAMA, st1, Shard.stageWeights(LLAMA, st1, makeWeights(LLAMA), Arch), ctx(), T);
+    assert.strictEqual(one.paramShard, null, "a single layer stays resident");
+    assert.ok(one.w.layers[0].Wq && one.w.layers[0].Wq.length);
+    assert.strictEqual(Shard.makeParamShard([one.w.layers[0]]), null);
+  });
+
+  await t("sliding-window mask matches across a split and stays full-width", async () => {
+    const got = makeWeights(LLAMA);
+    const win = new Int32Array(T);
+    for (let i = 0; i < PROMPT.length; i++) win[T - PROMPT.length + i] = PROMPT[i];
+    const run = async (spec, splits) => {
+      const stages = stagesFor(spec, got, splits, T);
+      let x = Infer.embed(stages[0], win);
+      for (const S of stages) x = await Infer.runLayers(S, x);
+      return x;
+    };
+    const plain = await run(LLAMA, [LLAMA.layers]);
+    const wide = Object.assign({}, LLAMA, { slidingWindow: 1000, swaFrom: 0 });
+    assert.ok(bitsEqual(plain, await run(wide, [LLAMA.layers])), "a window wider than the cache changed the GEMM");
+    const late = Object.assign({}, LLAMA, { slidingWindow: 2, swaFrom: LLAMA.layers });
+    assert.ok(bitsEqual(plain, await run(late, [LLAMA.layers])), "layers before swaFrom must stay full causal");
+    const sw = Object.assign({}, LLAMA, { slidingWindow: 2, swaFrom: 0 });
+    const masked = await run(sw, [sw.layers]);
+    assert.ok(!bitsEqual(plain, masked), "a window of 2 did not change attention");
+    const split = await run(sw, [2, 2, 2]);
+    assert.ok(bitsEqual(masked, split), "sliding-window split diverged");
+    const cfg = Arch.fromConfig({
+      model_type: "qwen2", hidden_size: 64, num_hidden_layers: 4, num_attention_heads: 4,
+      num_key_value_heads: 2, intermediate_size: 128, vocab_size: 32,
+      use_sliding_window: true, sliding_window: 8, max_window_layers: 2,
+    });
+    assert.strictEqual(cfg.slidingWindow, 8);
+    assert.strictEqual(cfg.swaFrom, 2);
+    const off = Arch.fromConfig({
+      model_type: "qwen2", hidden_size: 64, num_hidden_layers: 4, num_attention_heads: 4,
+      num_key_value_heads: 2, intermediate_size: 128, vocab_size: 32,
+      use_sliding_window: false, sliding_window: 8,
+    });
+    assert.strictEqual(off.slidingWindow, 0);
+  });
+
+  await t("JSON mask, position seed, and a prepared prompt", async () => {
+    assert.strictEqual(Shard.jsonStatus("{\"a\":1}"), "ok");
+    assert.strictEqual(Shard.jsonAllows("", "{"), true);
+    assert.strictEqual(Shard.jsonAllows("{", "a"), false);
+    assert.strictEqual(Shard.jsonAllows("{", "\"a\":"), true);
+    assert.strictEqual(Shard.jsonAllows("{\"a\":1}", "x"), false);
+    assert.strictEqual(Shard.jsonAllows("tru", "e"), true);
+    assert.strictEqual(Shard.jsonAllows("tru", "k"), false);
+    assert.strictEqual(Shard.jsonAllows("[1,", "2]"), true);
+    assert.strictEqual(Shard.jsonAllows("{\"a\":1,", "}"), false);
+    assert.strictEqual(Shard.jsonAllows("", " "), true);
+    const logits = Float32Array.from([5, 0.1, 1]);
+    const decodePiece = (id) => ["x", "{", "}"][id];
+    assert.strictEqual(Infer.pickStructured(logits, {
+      schema: "json", temperature: 0, topK: 3, produced: "", decodePiece,
+    }), 1);
+    assert.strictEqual(Infer.pickStructured(logits, { temperature: 0 }), 0);
+    const sample = Float32Array.from([0, 0.2, 0.5, 1.2, 0.1]);
+    const a = Infer.pickToken(sample, { temperature: 0.8, topK: 5, seqSeed: 99, pos: 4, rng: () => 0.1 });
+    const b = Infer.pickToken(sample, { temperature: 0.8, topK: 5, seqSeed: 99, pos: 4, rng: () => 0.9 });
+    assert.strictEqual(a, b, "a position seed must ignore the mutable rng");
+    assert.strictEqual(Infer.seqDraw(99, 4), Infer.seqDraw(99, 4));
+    assert.deepStrictEqual(Shard.preparedIds("hi", { text: "hi", ids: [1, 2] }), [1, 2]);
+    assert.strictEqual(Shard.preparedIds("no", { text: "hi", ids: [1] }), null);
+    const gotIds = Shard.preparedIds("hi", { text: "hi", ids: [1, 2] });
+    gotIds.push(9);
+    assert.deepStrictEqual(Shard.preparedIds("hi", { text: "hi", ids: [1, 2] }), [1, 2]);
+  });
+
+  await t("a 128-channel weight scale is not the verified column scale", async () => {
+    // DeepGEMM's FP8 GEMM scales every 128 channels of K (BLOCK_K == 128).
+    // vgemmBlock uses one scale for the whole column. A spike in a later
+    // block must not tighten the earlier block under our recipe, and it
+    // must widen the single column scale. The forward path does not call
+    // the block recipe.
+    const V = require("./public/verified_core.js");
+    const k = 256, n = 3, block = 128;
+    const W = new Float32Array(k * n);
+    for (let i = 0; i < W.length; i++) W[i] = ((i * 17) % 50) / 25 - 1;
+    W[200 * n] = 40;
+    const col = V.quantizeCols(W, k, n);
+    assert.strictEqual(col.s.length, n);
+    const blockS = new Float32Array((k / block) * n);
+    for (let c = 0; c < n; c++) {
+      for (let b = 0; b < k / block; b++) {
+        let mx = 0;
+        for (let r = b * block; r < (b + 1) * block; r++) mx = Math.max(mx, Math.abs(W[r * n + c]));
+        blockS[b * n + c] = Math.max(mx / 127, 1e-8);
+      }
+    }
+    assert.ok(blockS[0] < col.s[0], "the early block must not inherit a later spike");
+    assert.strictEqual(col.s[0], Math.fround(40 / 127));
+    assert.strictEqual(V.MAX_INT32_K, 131071);
+  });
+
   await t("a non-finite activation is refused before any layer runs", async () => {
     const S = { w: { layers: [] }, spec: LLAMA, T: 2 };
     const bad = new Float32Array([0, 1, NaN, 2]);