Spaces:
Running
Running
Commit History
-
Settings: model section to top; default WebLLM + Qwen3 0.6B; copyable debug
- Inject the "Local AI Model" section at the TOP of Gradio's settings page. - Default engine = WebLLM (best on mobile WebGPU; falls back to wllama with no WebGPU) and default model = Qwen3 0.6B. - Persona + diary panels gain a "📋 Copy debug" button that copies a paste-ready report (engine · backend · model · input · outcome · raw output · stripped text, plus exception/stack), so failures like the empty-JSON persona can be sent back verbatim. Clipboard-blocked contexts fall back to selecting the text. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>898540a -
Cache status + delete for WebLLM and Transformers.js (like wllama)
The model bar already shows "✓ downloaded" + a delete button for any engine that exposes cachedSet()/deleteCached(); wire those up for the other two: - WebLLM: MLC's own hasModelInCache() / deleteModelAllInfoInCache() over prebuiltAppConfig (matches the q4f16/q4f32 variant we'd load). - Transformers.js: inspect the 'transformers-cache' Cache API store; only count a model as cached when its *.onnx weights are present (Transformers.js sometimes caches the config/tokenizer JSONs but not the big model file — known upstream issue), so the badge is honest. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>b0f48f8 -
Fix Qwen3 on WebLLM: budget for thinking + capture reasoning_content
Symptom: WebLLM Qwen3 "rambled" but the output was just unfinished. MLC doesn't honour the `/no_think` text switch (wllama does), so Qwen3 keeps reasoning, and max_tokens:220 caps the TOTAL (reasoning + answer) — the JSON gets truncated before it closes, so parsing fails. - thinkMaxTokens(): give thinking models (qwen3) ≥768 tokens so they can reason AND still finish the answer. When /no_think IS honoured (wllama/transformers) the model stops early at EOS, so this costs nothing there. - WebLLM stream now captures MLC's separate `reasoning_content`, re-wrapped as <think>…</think>, so stripThink + the raw "thinking" view treat every engine the same (and the budget math is visible). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>eba5aae -
Fix Qwen3 parsing (/no_think) + WebLLM double-load guard & progress clarity
Qwen3 is a thinking model: with a 220-token budget it spent everything on a <think> block and never emitted the JSON/answer, so persona parsing failed after the thinking. Append Qwen3's `/no_think` soft switch for these structured tasks (persona + diary) so it goes straight to the answer. Helper noThink(modelId) keyed on the model id; no-op for non-Qwen3. WebLLM "downloads twice / no cache": - ensure() guarded on _loadedId (only set AFTER load), so a re-entrant ensure during a slow download could start a SECOND load. Guard on _loadingId (set up front) instead — same fix applied to the Transformers.js engine. - MLC runs two phases through one progress callback (Fetching from network, then Loading from cache into GPU) — both 0→100%, which reads as "downloading twice". Pass MLC's progress text through so the status line shows which phase it is; the second pass is a cache-load, not a re-download. (Caching = MLC Cache API + our storage.persist(); evictable under quota pressure.…ab87288 -
Fix model re-downloads (persistent storage) + WebLLM shader-f16 crash
Two issues with the multi-engine setup: - Models re-downloaded across engine switches because cached data was best-effort and evicted under quota pressure (three engines park 0.5-2GB each on one origin). Add storage.js → navigator.storage.persist() before the first download, flipping the cache to durable. Surface cache usage (usage/quota) in the model bar so the footprint is visible. (Note: the three engines use different formats/repos/caches — GGUF vs ONNX vs MLC — so a download genuinely can't be shared across them; only repeat use of the SAME engine is now cached reliably.) - WebLLM threw "Invalid ShaderModule … index_kernel" on GPUs/drivers without the WebGPU shader-f16 feature, because every MLC model was a q4f16_1 build. Detect shader-f16 via adapter.features and fall back to q4f32_1 when absent. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>fa27f81 -
Multi-engine benchmark: wllama vs Transformers.js vs WebLLM (engine selector)
Add a runtime facade (runtime.js) over three in-browser LLM engines so the same model can be A/B'd and tok/s compared: - engineWllama.js — llama.cpp WASM + WebGPU (GGUF from HF; 🦙 + local-first) - engineTransformers.js — HF Transformers.js (ONNX, WebGPU/WASM) - engineWebllm.js — MLC WebLLM (WebGPU only; greyed out without GPU) - genStats.js — shared tok/s tracker (decode rate, TTFT) for fair benchmarking modelBar.js gains an Engine selector (unavailable engines disabled); persona + diary panels now import from runtime.js. Removes the old wllamaLlm.js (folded into engineWllama.js). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>f8d0843