Date the 2026-10-22 README additions and finish the scheduling pass.

Each doc now opens with an Added 2026-10-22 list, and the new sentences in the body are marked the same way. The chunk path is chosen by the assignment, the lap budget uses the whole plan, a prefix record is restored on load, and only a planned peer aborts a run.

GUIDE.md

@@ -11,6 +11,21 @@ where available, the identical math on CPU where not.
 **This runs locally.** It is a project you clone and start on your own
 machines; there is no hosted deployment, and it is not a Space.
 
+## Added 2026-10-22
+
+The rest of this file is the previous guide. This list, and every later note marked *(added 2026-10-22)*, is new.
+
+- Compute cores from DaisyChain-Train `a28437c`. Non-finite activations are refused.
+- A disagreeing kernel probe is left out of the plan.
+- Lap budget is the plan, each stage at its own rate. Above 120 seconds per token the run stops before download.
+- A second prompt waits.
+- If the browser reports enough RAM for the whole model, the turn stays on the head.
+- A 64-token prefix hash is kept per conversation and read back on load. The full-window pass still recomputes it.
+- Long multi-device prompts use even chunks of about 16 tokens, two in flight, with per-stage KV. Verify reruns that path.
+- A repeated suffix is drafted and scored in one lap on the full-window path.
+- One weight range stays in flight while the previous body is widened to f32.
+- A dropped peer or a stalled lap is replayed once. Weights still come from the Hub.
+
 ```bash
 npm install
 npm start            # http://localhost:8788
@@ -48,6 +63,21 @@ Running SmolLM-135M across three devices, each one downloads and holds
 135–243 MB of a 513 MB model. The activation moving between them is tens of KB.
 That asymmetry is why this works.
 
+## Scheduling *(added 2026-10-22)*
+
+The compute units are the live DaisyChain-Train browser cores (Hub revision `a28437c`, 20 October 2026). A non-finite activation throws `NonFiniteError` and is not forwarded.
+
+- A peer whose kernel probe disagrees is left out of the plan.
+- Layers are still split by measured GEMMs/sec.
+- Above **120 seconds** per estimated token, the run stops before any weight download.
+- A second prompt waits. The turn already generating keeps the ring.
+- If the browser reports enough RAM for the whole model, the turn stays on the head. Unknown RAM uses the ring.
+- A **64-token** prefix hash is recorded per conversation, keyed by the first block, so two chats do not erase each other. The full-window pass still recomputes it.
+- A prompt longer than **32 tokens** on more than one device is prefetched in even chunks of about **16** tokens, two in flight, with per-stage KV. That path is not the full-window GEMM. Verify reruns the chunked path.
+- On the full-window path, a repeated suffix is drafted and scored in one lap. Matching drafts are kept, plus the first disagreement.
+- One weight range stays in flight while the previous body is widened to f32.
+- A dropped peer or a stalled lap is replayed **once** from the start. Weights still come from the Hub.
+
 ## The ring
 
 ```
@@ -124,7 +154,7 @@ Four things close that:
 
 1. **The kernel probe**, which matters more here than in the trainer — the same
    seeded GEMM on every device, so it stays comparable even when the real work
-   is not. A stage whose probe disagrees is flagged before it is given layers.
+   is not. A stage whose probe disagrees is left out of the plan. *(added 2026-10-22)*
 2. **Activation integrity hashes** on every hop, plus the model fingerprint, so
    a stage still holding a slice of a *different* model refuses the hop instead
    of silently mixing two models.
@@ -145,6 +175,9 @@ architecture families, across several uneven splits including a head that keeps
 no layers. Two cases are mutation checks — a stage that silently drops a layer,
 and stages applied out of order — both of which *must* fail the comparison.
 Without those, a test where both sides call the same code proves nothing.
+*(added 2026-10-22)* The same file also checks a dropped probe, a refused non-finite activation,
+a chunked prefill across a split, and a lookahead draft across a split. The
+chunked path is compared to itself, not to the full-window GEMM.
 
 This was also confirmed against a real model: SmolLM-135M's 30 layers split
 `[10,10,10]`, `[1,14,15]`, `[0,15,15]` and `[5,5,5,5,5,5]` all produced token
@@ -179,13 +212,14 @@ behaviour; it was checked.
 - **Latency, not bandwidth, is the cost.** Every token pays one round trip per
   stage. More stages buy capacity, not speed — expect tokens/sec to *fall* as
   you add devices.
-- **No KV cache.** Every token re-runs the whole window, which is also what
-  keeps the split bit-comparable against an unsplit run. Context length is the
-  dominant per-token cost.
+- **The full-window path has no KV cache.** *(added 2026-10-22, replaces "No KV cache".)* Every token re-runs the window, which
+  is what keeps that path bit-comparable against an unsplit run. Context length
+  is the dominant per-token cost. A long multi-device prompt uses chunked
+  prefill instead, and that is a different number.
 - **f32 in memory.** Weights are widened from F16/BF16 on load, so a stage
   costs 4 bytes per parameter it holds.
-- **The head is a single point of failure**, and a stage that drops stalls the
-  ring; press Generate again to re-plan around whoever is still connected.
+- **The head is a single point of failure.** *(added 2026-10-22.)* A stage that drops is removed and the
+  prompt is replayed once. If that also fails, press Generate again.
 - **No authentication of activations.** A malicious stage that runs correct
   math but returns a crafted activation is not caught by any of this — the
   verification proves the *computation* is right on every honest device. Peers

README.md

@@ -14,6 +14,23 @@ tags:
 **Part of DaisyChain on 🤗 Hugging Face → https://huggingface.co/DaisyChainAI**
 **Live Space:** https://huggingface.co/spaces/Quazim0t0/DaisyChain-Infer
 
+## Added 2026-10-22
+
+The rest of this file is the previous README. Everything in this list, and every later note marked *(added 2026-10-22)*, is new.
+
+- Compute cores copied from live DaisyChain-Train, Hub revision `a28437c` (20 October 2026). Lookup tables unchanged.
+- A non-finite activation throws `NonFiniteError` and is not forwarded. The same cores refuse an int32 accumulator that would wrap, and a non-finite gradient before DaisyAdam.
+- A peer whose kernel probe disagrees is left out of the plan.
+- Layers stay split by measured GEMMs/sec, so a slower device gets fewer layers.
+- Lap budget uses `estimatePlanMs`: each stage's share of the model, at that stage's own rate, after the plan is final. Above 120 seconds per token the run stops before any download.
+- A second prompt is queued. The turn already generating keeps the ring.
+- If `navigator.deviceMemory` says the whole model fits in a quarter of that RAM, the turn stays on the head. Unknown memory uses the ring.
+- A 64-token prefix hash is kept per conversation, keyed by the first block, written to IndexedDB, and read back when the page loads. Two chats do not wipe each other. The full-window pass still recomputes a matched prefix. Oldest record drops at 32 MB.
+- Prompt longer than 32 tokens, more than one device: even chunks of about 16 tokens, two in flight, per-stage KV. Not bit-identical to the full-window GEMM. Verify reruns this path. The assignment field `pass` is `chunk` or `window`. Stages do not guess from the activation length.
+- Full-window path: a repeated suffix is drafted and scored in one lap (`lookaheadFrame`). Matches are kept, plus the first disagreement.
+- One weight range stays in flight while the previous body is widened to f32.
+- A planned peer other than the head dropping, or a stalled lap, replays the prompt once. Weights still come from the Hub.
+
 ---
 
 > **In plain terms:** point it at a Hugging Face model, open the page on two or
@@ -64,6 +81,23 @@ Running SmolLM-135M across three devices, each downloads and holds 135–243 MB
 of a 513 MB model, and the activation moving between them is tens of KB. That
 asymmetry is why this works.
 
+## Scheduling *(added 2026-10-22)*
+
+The compute units are the live DaisyChain-Train browser cores (Hub revision `a28437c`, 20 October 2026), not the older snapshot this repo shipped with. A non-finite activation throws `NonFiniteError` before any layer runs, and the next stage does not forward it. The same cores refuse an int32 accumulator that would wrap, and they refuse a non-finite gradient before DaisyAdam. The lookup tables did not change.
+
+On top of that, the head does a few things the old loop did not:
+
+- A peer whose kernel probe disagrees is left out of the plan, not only warned.
+- Layers are still split by measured GEMMs/sec, so a slower device gets fewer layers.
+- Before any weight download, `estimatePlanMs` sums each stage's share at that stage's own rate. Above **120 seconds** per token the run stops. *(added 2026-10-22)*
+- A second prompt while a turn is generating is queued. The turn already on the ring finishes first.
+- If `navigator.deviceMemory` says the whole model fits in a quarter of that RAM, the turn runs on the head and the ring is not used. Unknown memory uses the ring. A split that the head can hold only adds a hop.
+- A **64-token** block hash is kept per conversation, keyed by the first block, in memory and in IndexedDB when the browser has it. Two chats do not wipe each other's record. The oldest record is dropped at **32 MB**. The full-window pass still recomputes. A prefix hit is logged and then computed again, because reusing K and V would change the per-column int8 scales.
+- With more than one device and a prompt longer than **32 tokens**, prefill goes out in even chunks of about **16** tokens, two chunks in flight. A short tail is not left as its own hop. Each stage keeps K and V for its own layers. Decode stays one new token per lap. This path is not bit-identical to one full-window GEMM. Verify reruns the same chunked path.
+- On the full-window path, a repeated suffix proposes the tokens that followed the earlier copy (prompt lookup, no second model). One lap scores that span. Matching drafts are kept, plus the first token that disagreed. If nothing repeats, the lap still produces one token.
+- Weight fetches keep one HTTP range in flight while the previous body is widened to f32. The bytes are the same.
+- If a peer disconnects or a lap stalls, the head drops that peer, replans, and reruns the prompt **once** from the start. Weights still come from the Hub, not from another device.
+
 ## The ring
 
 ```
@@ -145,7 +179,7 @@ Four things close that:
 
 1. **The kernel probe**, which matters more here than in the trainer — the same
    seeded GEMM on every device, so it stays comparable even when the real work is
-   not. A stage whose probe disagrees is flagged before it is given layers.
+   not. A stage whose probe disagrees is left out of the plan. *(added 2026-10-22)*
 2. **Activation integrity hashes** on every hop, plus the model fingerprint, so a
    stage still holding a slice of a *different* model refuses the hop instead of
    silently mixing two models.
@@ -161,6 +195,10 @@ architecture families, across several uneven splits including a head that keeps
 no layers. Two cases are mutation checks — a stage that silently drops a layer,
 and stages applied out of order — both of which *must* fail the comparison.
 Without those, a test where both sides call the same code proves nothing.
+*(added 2026-10-22)* The same file checks that a disagreeing probe is dropped, that a non-finite
+activation is refused, that a chunked prefill matches across a split, and that
+a lookahead draft matches across a split. The chunked path is compared to
+itself, not to the full-window GEMM.
 
 Confirmed on a real model too: SmolLM-135M's 30 layers split `[10,10,10]`,
 `[1,14,15]`, `[0,15,15]` and `[5,5,5,5,5,5]` all produced token sequences
@@ -198,13 +236,13 @@ public/safetensors.js     header parsing, exact BF16/F16 widening, range coalesc
 public/hf.js              Hub client: range reads, gated repos, token redaction
 public/arch.js            config.json -> normalized spec; per-family tensor names
 public/tokenizer.js       byte-level BPE from the repo's own tokenizer.json
-public/shard.js           the plan, per-stage tensor sets, weight-layout handling
-public/infer.js           forward pass: embed / runLayers / readout
+public/shard.js           the plan, probe filter, lap estimate, prefix index, drafts   (added 2026-10-22)
+public/infer.js           forward pass, chunked prefill, lookahead readout          (added 2026-10-22)
 public/wire.js            binary protocol + routing, as pure functions
-public/app.js             WebRTC mesh, assignments, credentials, token loop
-public/verified_core.js   the verified INT8 units      (unchanged from DaisyChain-Web)
-public/webgpu.js          WGSL kernels + exact gates   (unchanged)
-public/*.bin              the units as lookup tables   (unchanged)
+public/app.js             WebRTC mesh, assignments, queue, replay, token loop       (added 2026-10-22)
+public/verified_core.js   the verified INT8 units      (copied from live DaisyChain-Train, 2026-10-22)
+public/webgpu.js          WGSL kernels + exact gates   (copied from live DaisyChain-Train, 2026-10-22)
+public/*.bin              the units as lookup tables   (unchanged; same as Train)
 test_pipeline.js          split vs unsplit, bit-exact, both families
 test_wire.js              protocol round-trips, refusals, ring routing
 test_loader.js            safetensors, dtype widening, BPE
@@ -216,12 +254,14 @@ docs/                     GETTING_STARTED, ARCHITECTURE
 - **Latency, not bandwidth, is the cost.** Every token pays one round trip per
   stage. More stages buy capacity, not speed — expect tokens/sec to *fall* as you
   add devices.
-- **No KV cache.** Every token re-runs the whole window, which is also what keeps
-  the split bit-comparable against an unsplit run.
+- **The full-window path has no KV cache.** *(added 2026-10-22, replaces "No KV cache".)* Every token re-runs the window, which
+  is what keeps that path bit-comparable against an unsplit run. A long prompt on
+  a multi-device ring uses the chunked path instead, and that path is a different
+  number.
 - **f32 in memory.** Weights are widened from F16/BF16 on load, so a stage costs
   4 bytes per parameter it holds.
-- **The head is a single point of failure**, and a stage that drops stalls the
-  ring; press Generate again to re-plan around whoever is still connected.
+- **The head is a single point of failure.** *(added 2026-10-22.)* A stage that drops is removed and the
+  prompt is replayed once. If that also fails, press Generate again.
 - **No authentication of activations.** A malicious stage that runs correct math
   but returns a crafted activation is not caught by any of this. Peers see each
   other's IPs, and the head sees your prompt. Ring with people you trust.

docs/ARCHITECTURE.md

@@ -3,6 +3,18 @@
 How a page load becomes one stage of a model, and how a token gets produced by
 several machines that each hold only part of it.
 
+## Added 2026-10-22
+
+The rest of this file is the previous architecture note. This list is what changed.
+
+- A disagreeing kernel probe is removed before apportionment (`ringMembers`).
+- `estimatePlanMs` is the lap budget: each stage's share at that stage's rate. Above 120000 ms the run is refused.
+- Assignments carry `pass`: `chunk` or `window`. Stages do not infer the path from the row count.
+- The next weight range is fetched while the previous body is widened to f32.
+- Full-window laps can score a prompt-lookup draft of at most 4 tokens in one window.
+- Prompts longer than 32 tokens on more than one stage use even chunks of about 16, two in flight, with per-stage KV. Not the full-window GEMM.
+- A second prompt is queued. A dropped planned peer or a stalled lap replays the prompt once. A non-finite hidden state is not forwarded.
+
 ## The pieces
 
 | File | Role |
@@ -68,6 +80,17 @@ Apportionment is largest-remainder, so it is a pure function of the capacity
 report: every device derives the same plan from the same inputs, and the plan
 never has to be trusted, only compared. A device that ends up with zero blocks
 is dropped from the ring unless it is the head — an empty hop is pure latency.
+*(added 2026-10-22)* A peer whose kernel probe hash disagrees with the head is removed before that
+apportionment (`ringMembers`). The head is always kept.
+
+*(added 2026-10-22)* Before the download, `estimatePlanMs` sums each stage's share of
+`estimateLapMs` at that stage's own GEMM rate. A slow stage with a real share
+of the layers raises the figure. It is a budget check. Above 120000 ms the
+head refuses the run. It is not a latency model.
+
+*(added 2026-10-22)* The assignment may set `pass` to `chunk` or `window`. Stages must not guess
+the path from the activation length: a chunk can be exactly `T` rows, and
+treating it as a full window would drop the KV the other stages kept.
 
 `unpackAssign` refuses any plan whose stages do not cover every layer exactly
 once, and any plan whose stage 0 is not the head. A gap there would not throw
@@ -84,7 +107,9 @@ credentials. Weights never travel between peers, which means:
 - no device needs to hold the whole model, not even the head;
 - a token is never sent over the wire — each device is prompted separately;
 - adjacent byte ranges are coalesced, so a stage costs roughly one request
-  rather than nine per layer.
+  rather than nine per layer;
+- *(added 2026-10-22)* the next range is requested while the previous body is widened to f32.
+  The bytes do not change.
 
 ## One token
 
@@ -101,6 +126,28 @@ The head is both stage 0 and the terminus, so the return leg has its own
 address rather than a stage index. Reading it as "stage 0, your turn" was a
 real bug: the head re-ran its own layers and the lap never closed.
 
+*(added 2026-10-22)* Two other ways a lap can start:
+
+- **Full window, with a draft.** If a suffix of the token ids already occurred
+  earlier, the head appends the tokens that followed that copy (at most 4) and
+  reads logits at each of those rows plus one. Matches are kept. The first
+  disagreement is kept too. An empty draft is one new token, the same window
+  as before. `runLayers` still sees the whole window, so this path stays on
+  the bit-exact GEMM.
+- **Chunked prefill.** Prompt longer than 32 tokens, and more than one stage.
+  Even chunks of about 16 tokens, two in flight. A short tail is folded into
+  those chunks so it is not its own hop. `runLayersChunk` appends K and V on the stage
+  that owns those layers and attends only the new rows. Decode after that is
+  one new token per lap on the same cache. Per-column int8 scales move as the
+  cache grows, so this is not bit-identical to the full-window GEMM. A split
+  of the chunked path matches itself. `tokenIdx` is the chunk index, and a
+  stage applies chunks in order even if a later chunk arrives first.
+
+A second prompt while a run is in progress is queued. If the activation wait
+fails or a peer channel closes, the head drops that peer, rebuilds the plan,
+and reruns the prompt once. A non-finite hidden state throws inside
+`runLayers` / `runLayersChunk` and is not forwarded.
+
 ## The wire
 
 All messages are binary. Sentinels continue DaisyChain-Web's numbering (it uses

docs/GETTING_STARTED.md

@@ -3,6 +3,14 @@
 This is a local project. There is no hosted version — you clone it and run it
 on your own machines.
 
+## Added 2026-10-22
+
+The rest of this file is the previous getting-started note. This list is what changed.
+
+- Context length still defaults to 64. A prompt longer than 32 tokens on a multi-device ring uses even chunked prefill. If this device can hold the whole model, the ring is not used.
+- New log lines: lap estimate for the whole plan, queued prompts, overlap prefill, draft span. A disagreeing probe is left out of the plan.
+- New failures: "a peer left the ring" (one replay), "REFUSED activation" for a non-finite hop, "refusing the run" when the lap estimate exceeds 120 seconds, "stays out of the plan: kernel probe".
+
 ## Run the server
 
 ```bash
@@ -87,7 +95,7 @@ The first person in is the **host** and approves each device individually.
 | Setting | Default | What it does |
 |---|---|---|
 | Tokens | 60 | how many to generate |
-| Context length | 64 | window size; the main cost per token (there is no KV cache) |
+| Context length | 64 | *(added 2026-10-22)* full-window size; the main cost per token on that path. A prompt longer than 32 tokens on a multi-device ring uses even chunked prefill instead. If this device can hold the whole model, the ring is not used |
 | Temperature ÷100 | 0 | 0 = greedy, deterministic and therefore checkable |
 | Seed | 1234 | drives sampling above temperature 0, so runs stay reproducible |
 | Verify | off | afterwards, re-run with every layer here and compare |
@@ -101,9 +109,16 @@ few layers and no vocabulary.
   stages: each one adds a round trip per token. More stages buy capacity, not
   speed.
 - **ring hops** — stages × tokens.
-- **kernel probe** (in the log) — the same number on every honest device,
-  whatever its backend. A peer reporting a different one is computing different
-  arithmetic, and the log says so.
+- **kernel probe** (in the log) — *(added 2026-10-22)* the same number on every device that is
+  allowed into the plan. A peer reporting a different one is left out.
+- **lap estimate** — *(added 2026-10-22)* milliseconds per token for the plan that will actually
+  run, each stage at its own rate. Above 120 seconds the run is refused
+  before the download.
+- **queued** — *(added 2026-10-22)* a prompt typed while a turn is generating. It starts when the
+  ring is free.
+- **overlap prefill** — *(added 2026-10-22)* chunks of about 16, two in flight, per-stage KV. Not the
+  full-window GEMM. Verify repeats this path.
+- **draft span** — *(added 2026-10-22)* a repeated suffix scored in one lap on the full-window path.
 - **my slice** — which layers this device holds and how much memory they take.
 
 ## Verify
@@ -127,8 +142,11 @@ npm test
 
 ## When it goes wrong
 
-- **"the ring stalled at token N"** — a stage went quiet. Press Generate again
-  to re-plan around whoever is still connected.
+- *(added 2026-10-22)* **"a peer left the ring"** — the head replans and replays the prompt once.
+  If the replay also fails, press Generate again.
+- *(added 2026-10-22)* **"REFUSED activation"** — a non-finite hop. It is not forwarded.
+- *(added 2026-10-22)* **"refusing the run: estimated … ms per token"** — the lap budget of 120
+  seconds was exceeded. Add a faster device or lower the context length.
 - **"unsupported architecture"** — the model is not Llama-style or
   GPT-2-style.
 - **"tokenizer type … is not supported"** — the repo uses SentencePiece or
@@ -139,8 +157,8 @@ npm test
   a slice; refused rather than accepted, since that defeats the point.
 - **"REFUSED activation … belongs to model X"** — a stage still holds a slice
   of a previous model. Reload that device.
-- **"⚠ … disagrees with this device's kernel probe"** — that device's
-  arithmetic differs from yours. Do not give it layers.
+- *(added 2026-10-22)* **"stays out of the plan: kernel probe"** — that device's arithmetic
+  differs. It is not given layers.
 - **out of memory** — the stage's slice does not fit. Add devices, or pick a
   smaller model; the plan shows the per-device cost before you start.
 

public/app.js

@@ -426,7 +426,7 @@ function cleanupPeer(id) {
   const dc = chans.get(id);
   if (dc && !dc.isRelay) chans.delete(id);       // keep a working relay channel
   for (const k of fragIn.keys()) if (k.startsWith(id + ":")) fragIn.delete(k);
-  if (running) peerDropped = true;
+  if (running && plan && plan.some(s => s.id === id && s.id !== myId)) peerDropped = true;
   caps.delete(id);
   wake();
 }
@@ -559,8 +559,30 @@ async function loadRepo() {
 }
 
 const prefixIndex = Shard.makePrefixIndex(64);
+function restorePrefixes() {
+  if (typeof indexedDB === "undefined") return;
+  const open = indexedDB.open("daisychain-infer", 1);
+  open.onupgradeneeded = () => {
+    if (!open.result.objectStoreNames.contains("prefix")) open.result.createObjectStore("prefix");
+  };
+  open.onsuccess = () => {
+    const db = open.result;
+    if (!db.objectStoreNames.contains("prefix")) return;
+    const cur = db.transaction("prefix").objectStore("prefix").openCursor();
+    cur.onsuccess = () => {
+      const row = cur.result;
+      if (!row) return;
+      const bytes = row.value;
+      if (bytes && bytes.byteLength >= 4) {
+        const u32 = new Uint32Array(bytes.buffer, bytes.byteOffset, (bytes.byteLength / 4) | 0);
+        prefixIndex.restore(String(row.key), Array.from(u32));
+        kvSpill.put(String(row.key), bytes);
+      }
+      row.continue();
+    };
+  };
+}
 const kvSpill = Shard.makeKvSpill(32 * 1048576);
-const CHAT = "chat";
 const LAP_BUDGET_MS = 120000;
 
 function memberList() {
@@ -612,6 +634,7 @@ async function loadMyStage(assign, st) {
                              `${assign.repo} weights`);
   const w = Shard.stageWeights(s, st, got, Arch);
   stageObj = Infer.makeStage(s, st, w, kernelCtx(), assign.ctx || ctxLen());
+  stageObj.pass = assign.pass === "chunk" ? "chunk" : "window";
   spec = s;
   ui.myShard.textContent = `${st.head ? "head + " : ""}layers ${st.lo}–${st.hi - 1} · ${mb.toFixed(0)} MB resident`;
   return stageObj;
@@ -692,9 +715,7 @@ async function onAct(peerId, buf) {
   const kind = Wire.classifyAct(nextIndex, me ? me.index : -99);
   if (kind === "return") { actIn.set(`${seq}:${tokenIdx}`, hidden); wake(); return; }
   if (kind === "other" || !me || !stageObj) return;
-  const rows = hidden.length / spec.hidden;
-  const chunked = Number.isInteger(rows) && rows !== stageObj.T;
-  if (chunked) {
+  if (stageObj.pass === "chunk") {
     if (!stageObj.inbox || stageObj.boxSeq !== seq) {
       stageObj.inbox = new Map();
       stageObj.boxSeq = seq;
@@ -763,25 +784,27 @@ async function generate(isReplay) {
     const promptText = ui.prompt.value || "The";
     let idsPreview = [...Tokenizer.encode(tok, promptText)];
     if (!idsPreview.length) idsPreview = [0];
-    const lapMs = Shard.estimateLapMs(spec, T, myCapacity);
-    log(`lap estimate ${Math.round(lapMs)} ms/token on this device's capacity (budget ${LAP_BUDGET_MS} ms).`);
-    if (lapMs > LAP_BUDGET_MS) {
-      throw new Error(`refusing the run: estimated ${Math.round(lapMs)} ms per token exceeds ${LAP_BUDGET_MS} ms. ` +
-                      `Add a faster device or lower the context length.`);
-    }
     const fits = Shard.headCanHold(spec, repoInfo.tensors, navigator.deviceMemory || 0);
     const local = Shard.preferLocal(idsPreview.length, plan.length - 1, fits);
-    const matched = prefixIndex.matchedBlocks(CHAT, idsPreview);
+    const chatKey = String((prefixIndex.blocks(idsPreview)[0]) || "empty");
+    const matched = prefixIndex.matchedBlocks(chatKey, idsPreview);
     if (matched) {
       log(`prefix index: ${matched} block(s) of 64 tokens match this chat. ` +
           `The window is still recomputed so the token ids stay bit-identical to a full pass.`);
     }
     if (local) {
-      log(`short prompt (${idsPreview.length} tokens) fits on this device ` +
-          `(${(navigator.deviceMemory)} GB reported). Running here. The ring is not used for this turn.`);
+      log(`this device can hold the model (${navigator.deviceMemory} GB reported). ` +
+          `Running here. The ring is not used for this turn.`);
       plan = [{ id: myId, index: 0, lo: 0, hi: spec.layers, head: true, capacity: myCapacity, backend: compute.backend }];
       renderPlan();
     }
+    const lapMs = Shard.estimatePlanMs(spec, T, plan);
+    log(`lap estimate ${Math.round(lapMs)} ms/token across ${plan.length} stage(s) (budget ${LAP_BUDGET_MS} ms).`);
+    if (lapMs > LAP_BUDGET_MS) {
+      throw new Error(`refusing the run: estimated ${Math.round(lapMs)} ms per token exceeds ${LAP_BUDGET_MS} ms. ` +
+                      `Add a faster device or lower the context length.`);
+    }
+    const willChunk = plan.length > 1 && idsPreview.length > 32;
     if (plan.length > 1) {
       log(`plan: ${spec.layers} layers over ${plan.length} device(s) — ` +
           plan.map(s => `${s.id === myId ? "me" : nmeOf(s.id)}:${s.hi - s.lo}`).join(", "));
@@ -789,6 +812,7 @@ async function generate(isReplay) {
         await sendTo(plan[i].id, Wire.packAssign({
           repo: repoInfo.repo, revision: repoInfo.revision, spec, plan,
           mine: i, fingerprint: repoInfo.fingerprint, ctx: T,
+          pass: willChunk ? "chunk" : "window",
         }));
       log("waiting for every stage to fetch its layers…");
       const ok = await waitFor(() => readyStages.size >= plan.length - 1, 600000);
@@ -804,17 +828,17 @@ async function generate(isReplay) {
     const nTok = +ui.ntok.value, temp = +ui.temp.value / 100;
     const opts = { temperature: temp, topK: 40, rng: Infer.mulberry32(+ui.seed.value | 0) };
     let ids = idsPreview.slice();
-    prefixIndex.note(CHAT, ids);
+    prefixIndex.note(chatKey, ids);
     // The bytes kept here are the prefix-block hashes, not layer KV. Reusing
     // K and V would change per-column int8 scales, so the pass still recomputes.
     const hashBytes = new Uint8Array(new Uint32Array(prefixIndex.blocks(ids)).buffer);
-    if (!kvSpill.put(CHAT, hashBytes)) log("prefix record is larger than the spill cap; this chat is not retained.");
+    if (!kvSpill.put(chatKey, hashBytes)) log("prefix record is larger than the spill cap; this chat is not retained.");
     if (typeof indexedDB !== "undefined") {
       const open = indexedDB.open("daisychain-infer", 1);
       open.onupgradeneeded = () => open.result.createObjectStore("prefix");
       open.onsuccess = () => {
         const db = open.result;
-        db.transaction("prefix", "readwrite").objectStore("prefix").put(hashBytes, CHAT);
+        db.transaction("prefix", "readwrite").objectStore("prefix").put(hashBytes, chatKey);
       };
     }
     const promptLen = ids.length;
@@ -822,12 +846,14 @@ async function generate(isReplay) {
     const tStart = performance.now();
     let hops = 0;
 
-    const useChunks = plan.length > 1 && ids.length > 32;
+    stageObj.pass = willChunk ? "chunk" : "window";
+    actIn.clear();
+    const useChunks = willChunk;
     let chunkedIds = null;
     if (useChunks) {
-      log(`overlap prefill: ${ids.length} prompt tokens, chunks of 16, 2 in flight. ` +
-          `Each stage keeps KV for its own layers. This is not the full-window GEMM.`);
       const chunks = Shard.prefillChunks(ids.length, 16, 2);
+      log(`overlap prefill: ${ids.length} prompt tokens as [${chunks.chunks.map(c => c.hi - c.lo).join("+")}], ` +
+          `${chunks.window} in flight. Each stage keeps KV for its own layers. This is not the full-window GEMM.`);
       const span = { cache: [], pos0: 0, li: 0 };
       const hop = async (index, hidden) => {
         await sendTo(plan[1].id, packAct(seq, index, plan[1].index, hidden));
@@ -872,13 +898,10 @@ async function generate(isReplay) {
 
     if (!useChunks) {
     for (let produced = 0; produced < nTok && !abortRun; ) {
-      let draft = Shard.promptLookupDraft(ids, 4);
-      const seqIds = ids.concat(draft);
-      const shown = seqIds.slice(-T);
-      const shownDraft = Math.min(draft.length, Math.max(0, shown.length - 1));
-      draft = draft.slice(draft.length - shownDraft);
+      const frame = Shard.lookaheadFrame(ids, T, 4);
+      const draft = frame.draft;
       const win = new Int32Array(T);
-      for (let i = 0; i < shown.length; i++) win[T - shown.length + i] = shown[i];
+      for (let i = 0; i < frame.win.length; i++) win[T - frame.win.length + i] = frame.win[i];
       let x = Infer.embed(stageObj, win);
       x = await Infer.runLayers(stageObj, x);
       if (auditFailure) { log(`KERNEL AUDIT FAILED: ${auditFailure} — stopping`); break; }
@@ -891,7 +914,7 @@ async function generate(isReplay) {
         if (!ok) throw new Error("stalled");
         x = actIn.get(key); actIn.delete(key);
       }
-      const rows = Infer.draftRows(T, shown.length, draft.length);
+      const rows = Infer.draftRows(T, frame.shown, draft.length);
       const picked = (await Infer.logitsAt(stageObj, x, rows)).map(l => Infer.pickToken(l, opts));
       const got = Shard.acceptDraft(picked, draft);
       if (draft.length) log(`draft span ${draft.length}, accepted ${Math.max(0, got.length - 1)} before the bonus token. One lap.`);
@@ -1031,6 +1054,7 @@ function kernelCtx() {
 
   await measureCapacity();
   log(`measured capacity: ${myCapacity.toFixed(0)} verified GEMMs/sec — this decides how many layers this device takes`);
+  restorePrefixes();
 
   ui.me.textContent = deviceName;
   if (!hfToken) setToken(null);                  // keep a token adopted from the fragment

public/infer.js

@@ -308,11 +308,11 @@
     const spans = stages.map(() => ({ cache: [], pos0: 0, li: 0 }));
     const size = Math.max(1, chunkSize | 0);
     let last = null;
-    for (let lo = 0; lo < ids.length; lo += size) {
-      const piece = ids.slice(lo, Math.min(ids.length, lo + size));
-      let x = embedRows(stages[0], piece, lo);
+    for (const ch of SH.prefillChunks(ids.length, size, 1).chunks) {
+      const piece = ids.slice(ch.lo, ch.hi);
+      let x = embedRows(stages[0], piece, ch.lo);
       for (let s = 0; s < stages.length; s++) {
-        spans[s].pos0 = lo;
+        spans[s].pos0 = ch.lo;
         x = await runLayersChunk(stages[s], x, spans[s]);
       }
       last = x;
@@ -388,16 +388,13 @@
     const maxDraft = opts && opts.maxDraft != null ? opts.maxDraft : 4;
     let produced = 0;
     while (produced < nNew) {
-      let draft = SH.promptLookupDraft(ids, maxDraft);
-      const seqIds = ids.concat(draft);
-      const shown = seqIds.slice(-T);
-      const shownDraft = Math.min(draft.length, Math.max(0, shown.length - 1));
-      draft = draft.slice(draft.length - shownDraft);
+      const frame = SH.lookaheadFrame(ids, T, maxDraft);
+      const draft = frame.draft;
       const win = new Int32Array(T);
-      for (let i = 0; i < shown.length; i++) win[T - shown.length + i] = shown[i];
+      for (let i = 0; i < frame.win.length; i++) win[T - frame.win.length + i] = frame.win[i];
       let x = embed(head, win);
       for (const S of stages) x = await runLayers(S, x);
-      const rows = draftRows(T, shown.length, draft.length);
+      const rows = draftRows(T, frame.shown, draft.length);
       const picked = [];
       for (const logits of await logitsAt(head, x, rows)) picked.push(pickToken(logits, opts));
       const got = SH.acceptDraft(picked, draft);

public/shard.js

@@ -163,6 +163,21 @@
   // count below is the MAC volume of one window through every layer, divided
   // by that unit. It is there to refuse a split that cannot produce a token
   // in `budgetMs`, before any weight download.
+  // Serial time of one lap on this plan. Each stage pays for the layers it
+  // holds, at its own measured rate. A slow stage with a real share of the
+  // layers dominates. The head with no layers still pays a small share for
+  // the embedding and the readout.
+  function estimatePlanMs(spec, T, stages) {
+    const layers = Math.max(spec.layers, 1);
+    let ms = 0;
+    for (const s of stages) {
+      const owned = Math.max(0, (s.hi | 0) - (s.lo | 0));
+      const share = owned > 0 ? owned / layers : (s.head ? 0.05 : 0);
+      ms += estimateLapMs(spec, T, s.capacity || 0) * share;
+    }
+    return ms;
+  }
+
   function estimateLapMs(spec, T, gemmPerSec) {
     const unit = 32 * 64 * 64;
     const C = spec.hidden;
@@ -186,10 +201,10 @@
     return bytes < deviceGB * 0.25 * 1073741824;
   }
 
-  // A short prompt stays on the head when the head can hold every layer.
-  // The ring still exists for prompts that do not fit that rule.
+  // If the head can hold every layer, the ring only adds a hop. Run here.
+  // Unknown memory leaves `fits` false, so the ring is used rather than guessed.
   function preferLocal(promptTokens, peerCount, fits) {
-    return !!(fits && peerCount > 0 && promptTokens > 0 && promptTokens <= 64);
+    return !!(fits && peerCount > 0 && promptTokens > 0);
   }
 
   // Rolling FNV blocks over token ids. This is the approximate prefix index:
@@ -212,6 +227,7 @@
       blockSize: B,
       blocks,
       note(session, ids) { sessions.set(session, blocks(ids)); },
+      restore(session, hashes) { sessions.set(session, hashes.slice()); },
       matchedBlocks(session, ids) {
         const prev = sessions.get(session);
         if (!prev) return 0;
@@ -260,12 +276,23 @@
   // Prompt chunks for a pipelined prefill. `window` is how many chunks may be
   // in flight across the ring at once. Decode stays one token, one lap:
   // the next token is not known until the lap returns.
+  // Chunks are as even as the token count allows. A fixed size leaves a short
+  // tail, and the tail still pays a full hop while the other stages wait.
   function prefillChunks(nTokens, chunkSize, inFlight) {
     const size = Math.max(1, chunkSize | 0);
     const window = Math.max(1, inFlight | 0);
+    const n = Math.max(1, Math.ceil(Math.max(nTokens, 1) / size));
+    const base = Math.floor(nTokens / n);
+    let rem = nTokens - base * n;
     const chunks = [];
-    for (let lo = 0; lo < nTokens; lo += size)
-      chunks.push({ index: chunks.length, lo, hi: Math.min(nTokens, lo + size) });
+    let lo = 0;
+    for (let i = 0; i < n; i++) {
+      const len = base + (rem > 0 ? 1 : 0);
+      if (rem > 0) rem--;
+      if (len < 1) break;
+      chunks.push({ index: chunks.length, lo, hi: lo + len });
+      lo += len;
+    }
     return { chunks, window, chunkSize: size };
   }
 
@@ -295,6 +322,17 @@
 
   // picked[i] proposes draft[i]. picked has one extra entry: the token at the
   // first position that rejected, which is always kept.
+  // One description of the window both the ring and the local check run.
+  // `win` is the token ids placed at the right edge of a context of length T.
+  function lookaheadFrame(ids, T, maxDraft) {
+    let draft = promptLookupDraft(ids, maxDraft);
+    const seqIds = ids.concat(draft);
+    const shown = seqIds.slice(-T);
+    const shownDraft = Math.min(draft.length, Math.max(0, shown.length - 1));
+    draft = draft.slice(draft.length - shownDraft);
+    return { draft, shown: shown.length, win: shown.slice() };
+  }
+
   function acceptDraft(picked, draft) {
     const out = [];
     let i = 0;
@@ -306,7 +344,8 @@
   const api = { planStages, stageBytes, stageWeights, toKN, transposeEmb,
                 fnv1a, hashF32, modelFingerprint, ringMembers, estimateLapMs,
                 headCanHold, preferLocal, makePrefixIndex, makeKvSpill,
-                prefillChunks, promptLookupDraft, acceptDraft };
+                prefillChunks, promptLookupDraft, acceptDraft, lookaheadFrame,
+                estimatePlanMs };
   if (typeof module !== "undefined" && module.exports) module.exports = api;
   else root.Shard = api;
 })(typeof self !== "undefined" ? self : this);

test_pipeline.js

@@ -207,6 +207,12 @@ const GPT2 = Arch.fromConfig({
     const fast = Shard.estimateLapMs(spec, 8, 1000);
     assert.ok(slow > fast, "a longer window on a slower device must cost more");
     assert.ok(fast > 0);
+    const solo = Shard.estimatePlanMs(spec, 32, [{ lo: 0, hi: spec.layers, head: true, capacity: 1000 }]);
+    const ring = Shard.estimatePlanMs(spec, 32, [
+      { lo: 0, hi: 3, head: true, capacity: 1000 },
+      { lo: 3, hi: spec.layers, head: false, capacity: 10 },
+    ]);
+    assert.ok(ring > solo, "a slow stage must raise the lap estimate");
   });
 
   await t("a short prompt stays local only when the head can hold the model", async () => {
@@ -215,7 +221,7 @@ const GPT2 = Arch.fromConfig({
     assert.strictEqual(Shard.headCanHold(spec, tensors, 0), false);
     assert.strictEqual(Shard.headCanHold(spec, tensors, 8), true);
     assert.strictEqual(Shard.preferLocal(16, 2, true), true);
-    assert.strictEqual(Shard.preferLocal(128, 2, true), false);
+    assert.strictEqual(Shard.preferLocal(128, 2, true), true);
     assert.strictEqual(Shard.preferLocal(16, 2, false), false);
     assert.strictEqual(Shard.preferLocal(16, 0, true), false);
   });
@@ -223,8 +229,13 @@ const GPT2 = Arch.fromConfig({
   await t("prefix blocks match a continuation and the spill evicts the oldest block", async () => {
     const idx = Shard.makePrefixIndex(4);
     const a = [1, 2, 3, 4, 5, 6, 7, 8];
+    const kept = [1, 2, 3, 4, 5];
+    idx.restore("kept", idx.blocks(kept).slice(0, 1));
+    assert.strictEqual(idx.matchedBlocks("kept", kept.concat([6])), 1);
     idx.note("chat", a);
+    idx.note("other", [9, 9, 9, 9, 8]);
     assert.strictEqual(idx.matchedBlocks("chat", a.concat([9])), 2);
+    assert.strictEqual(idx.matchedBlocks("other", [9, 9, 9, 9, 1]), 1);
     assert.strictEqual(idx.matchedBlocks("chat", [9, 9, 9, 9]), 0);
     const spill = Shard.makeKvSpill(12);
     assert.strictEqual(spill.put("a", new Uint8Array(8)), true);
@@ -237,7 +248,12 @@ const GPT2 = Arch.fromConfig({
   await t("prefill chunks, prompt-lookup drafts, and accept keep a bonus token", async () => {
     const c = Shard.prefillChunks(40, 16, 2);
     assert.strictEqual(c.chunks.length, 3);
-    assert.deepStrictEqual([c.chunks[2].lo, c.chunks[2].hi], [32, 40]);
+    assert.deepStrictEqual(c.chunks.map(x => x.hi - x.lo), [14, 13, 13]);
+    assert.strictEqual(c.chunks[2].hi, 40);
+    assert.strictEqual(c.chunks[0].lo, 0);
+    const frame = Shard.lookaheadFrame([1, 2, 3, 9, 1, 2, 3], 16, 4);
+    assert.deepStrictEqual(frame.draft, [9]);
+    assert.strictEqual(frame.win.length, 8);
     assert.strictEqual(c.window, 2);
     const ids = [1, 2, 3, 9, 1, 2, 3];
     assert.deepStrictEqual(Shard.promptLookupDraft(ids, 4), [9]);