Turn PyBot into an in-browser web demo of the best bot (3185a7a, 26.55%)

Pyodide runs SwarmChat-v8 commit 3185a7a9bc50 (the leaderboard's best) in a Web Worker; no server. Keeps the chat, trace and file treemap; drops the story text, progress chart, FastAPI server and old bot zip. Switches the Space to sdk: static.

.gitattributes

@@ -33,5 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
-site/fonts/LetterGothicCode-B.ttf filter=lfs diff=lfs merge=lfs -text
-site/fonts/LetterGothicCode-R.ttf filter=lfs diff=lfs merge=lfs -text
+fonts/LetterGothicCode-B.ttf filter=lfs diff=lfs merge=lfs -text
+fonts/LetterGothicCode-R.ttf filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -2,3 +2,4 @@
 __pycache__/
 *.py[cod]
 scratch/
+node_modules/

Dockerfile

@@ -1,12 +0,0 @@
-FROM python:3.12-slim
-RUN useradd -m -u 1000 user
-USER user
-WORKDIR /home/user/app
-ENV PATH="/home/user/.local/bin:$PATH" PYTHONUNBUFFERED=1
-COPY --chown=user requirements.txt ./
-RUN pip install --no-cache-dir --user -r requirements.txt
-COPY --chown=user app.py bot_runner.py ./
-COPY --chown=user bot ./bot
-COPY --chown=user site ./site
-EXPOSE 7860
-CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "7860"]

README.md

@@ -3,60 +3,41 @@ title: PyBot
 emoji: 🐍
 colorFrom: gray
 colorTo: gray
-sdk: docker
-app_port: 7860
+sdk: static
+app_file: index.html
 pinned: false
+datasets:
+  - HuggingFaceSwarms/SwarmChat-v8
 ---
 
-PyBot: a conversational bot without LLMs, written by a swarm of LLM agents. One page: the story and a live chat that shows
-which of PyBot's modules and files produced each reply. Pinned to commit `64eb2c844738` of `HuggingFaceSwarms/SwarmChat-v8`.
+PyBot: a conversational bot without LLMs, written by a swarm of LLM agents. This Space is a chat demo that runs
+PyBot entirely in your browser: [Pyodide](https://pyodide.org) (CPython compiled to WebAssembly) runs the bot's Python
+in a Web Worker, so there is no server. Each reply shows which of PyBot's modules and files produced it.
 
-`app.py` serves `site/` and `POST /api/chat` / `POST /api/trace`; `bot/bot-<sha12>.zip` holds the bot's source (`bot/**.py` only);
-`bot_runner.py` runs it in a fresh worker per request. Built by `build_pybot.py` in the swarmchat-release workspace.
+The bot is commit `3185a7a9bc50` of `HuggingFaceSwarms/SwarmChat-v8`, the best run on the
+[SwarmChat leaderboard](https://huggingface.co/spaces/HuggingFaceSwarms/swarmchat-leaderboard) (26.55% length-penalised
+win rate on SmolTalkBench). Replaying 40 of that run's recorded conversations in the browser bundle gives byte-identical
+replies.
 
-## Run locally
+## Files
 
-Use Python 3.12 (required by the trace endpoint) on macOS or Linux:
+- `index.html`: the page (chat, the reply's trace, and a treemap of PyBot's files; Plotly from jsDelivr).
+- `worker.js`: loads Pyodide 314.0.7 from jsDelivr and the bot bundle, then answers turns. Like transformers.js does
+  for model files, it keeps the bundle in Cache Storage, so a second visit doesn't download it again.
+- `pybot_web.py`: the glue, ported from the previous server version of this Space (`bot_runner.py`, `app.py`): fixed
+  random seeds as in the eval server, the trace via `sys.monitoring`, and the file that holds a reply's text.
+- `pybot-<sha12>.zip`: `bot/**.pyc` compiled for Pyodide's CPython 3.14 (loading skips compiling 33 MB of source),
+  plus `norm.json` (each file's text, letters and digits only) for finding the file that holds a reply.
+- `tree.json`: PyBot's files and line counts for the treemap.
 
-```sh
-uv venv --python 3.12 .venv
-source .venv/bin/activate
-uv pip install -r requirements.txt
-uvicorn app:app --host 127.0.0.1 --port 7860
-```
-
-Open http://127.0.0.1:7860. The health endpoint is `/health`.
-Keep local logs and temporary files in `scratch/`, which is ignored by Git.
-
-### Progress chart
-
-Generate the chart snapshot from the leaderboard's v3 cache before starting the app:
+## Rebuild for another commit
 
 ```sh
-python build_progress.py /path/to/swarmchat-leaderboard-v3.json --source-repo scratch/v8-history.git
+hf download HuggingFaceSwarms/SwarmChat-v8 --repo-type dataset --revision <sha> --include "bot/**" --local-dir src
+(cd src && zip -qr ../bot-src.zip bot -i '*.py')
+npm install pyodide@314.0.7   # the same version as worker.js: bytecode only loads in the Python that compiled it
+node build/build.mjs bot-src.zip build/build_bundle.py pybot-<sha12>.zip
 ```
 
-The generated data stays in `scratch/progress.json` and is never committed. In the
-hosted Space, the private `HuggingFaceSwarms/pybot-chart` bucket is mounted read-only
-at `/home/user/app/scratch`, supplying `progress.json` across rebuilds and restarts.
-To publish a regenerated snapshot, activate the local environment and run:
-
-```sh
-hf buckets cp scratch/progress.json hf://buckets/HuggingFaceSwarms/pybot-chart/progress.json
-```
-
-The bucket mount is configured in the Space settings, separately from Git. Preserve
-it when changing volumes, and recreate it when duplicating the Space. No Hub token
-is needed by the app. The chart shows an unavailable message if the snapshot is absent.
-It uses the leaderboard's length-penalised percentage metric and latest successful
-1,000-prompt run per revision, with all three SmolLM baselines. The contiguous outage
-plateau from November 7, 2026, 12:12:47 through 13:15:11 UTC (commit times) is omitted;
-other score fluctuations and the original revision positions are preserved.
-
-The right axis overlays physical lines in `bot/**/*.py` at each retained evaluated
-commit, including blank lines and comments but excluding tests. Counts come from
-the source repository's first-parent Git diffs, including deletions and merges.
-Fetch the source history into `scratch/v8-history.git` before building the snapshot;
-unavailable commits get a null code-size value, rather than an estimate. The first
-evaluated seed (`541b8def5107`) predates the current main history and is no longer
-available from the Hub; its win-rate point remains, but its code-size point is absent.
+Then regenerate `tree.json` (`[path, line count]` for every `bot/**.py`, plus `repo` and `sha`) and update `BUNDLE`
+in `worker.js`. To test locally, serve this folder (e.g. `python -m http.server`) and open it.

app.py

@@ -1,120 +0,0 @@
-"""PyBot release Space: the story page (site/) and the PyBot demo's chat endpoint.
-
-The bot is one pinned commit's bot/**.py, shipped in bot/bot-<sha12>.zip, run by bot_runner (from SwarmChat-Demo):
-one loader process, and a fresh forked worker per request that replays the conversation's user lines, so visitors
-never share bot memory. Run only trusted bot code here and add no secrets: the worker is not a security sandbox.
-"""
-import re
-import sys
-import tempfile
-import threading
-import zipfile
-from pathlib import Path
-from typing import Literal
-
-from fastapi import FastAPI, HTTPException
-from fastapi.staticfiles import StaticFiles
-from fastapi.responses import FileResponse
-from pydantic import BaseModel, Field
-
-from bot_runner import Bot, user_lines
-
-HERE = Path(__file__).parent
-TREE = tempfile.TemporaryDirectory(prefix="swarmchat-source-")
-zipfile.ZipFile(next(HERE.glob("bot/bot-*.zip"))).extractall(TREE.name)
-app = FastAPI(title="PyBot")
-_bot, _bot_lock = None, threading.Lock()
-_slots = threading.BoundedSemaphore(4)  # conversations answered at once; cpu-basic has 2 vCPUs
-_norm = lambda s: re.sub(r"[^a-z0-9]", "", s.lower())
-# Every bot file, letters and digits only, to find which one holds a reply's text verbatim.
-SOURCES = {p.relative_to(TREE.name).as_posix(): _norm(p.read_text(errors="replace")) for p in Path(TREE.name, "bot").rglob("*.py")}
-
-
-def bot():
-    global _bot
-    with _bot_lock:
-        if _bot is None or not _bot.alive():
-            _bot = Bot(Path(TREE.name), seed=0, timeout=30, workers=2, log=sys.stderr)
-        return _bot
-
-
-threading.Thread(target=bot, daemon=True).start()  # load at boot, so the first visitor does not wait
-
-
-class Message(BaseModel):
-    role: Literal["user", "assistant"]
-    content: str = Field(min_length=1, max_length=2000)
-
-
-class ChatRequest(BaseModel):
-    messages: list[Message] = Field(min_length=1, max_length=60)
-
-
-@app.post("/api/chat")
-def chat(req: ChatRequest):
-    lines = user_lines([m.model_dump() for m in req.messages])
-    if not lines or req.messages[-1].role != "user":
-        raise HTTPException(400, "A user message is required.")
-    if sum(map(len, lines)) > 20000:
-        raise HTTPException(400, "This conversation is too long. Reload the page to start again.")
-    if not _slots.acquire(timeout=15):
-        raise HTTPException(503, "The bot is busy. Please try again in a moment.")
-    try:
-        result = bot().ask(lines)
-    finally:
-        _slots.release()
-    if result.get("status") != "ok" or len(result.get("replies", [])) != len(lines):
-        raise HTTPException(502, "The bot failed on that line. Try another one.")
-    return {"reply": result["replies"][-1]}
-
-
-def answer_source(reply: str, used: list[str]) -> str | None:
-    """The one file that holds the reply verbatim (its first 40+ letters/digits), preferring files the winner called;
-    None when the reply is composed, too short, or found in several files."""
-    text = _norm(reply)
-    for n in (60, 40):
-        key = text[:n]
-        if len(key) < 20:  # too short to point at one place
-            continue
-        hits = [f for f, body in SOURCES.items() if key in body]
-        hits = hits if len(hits) == 1 else [f for f in hits if f in used]
-        if len(hits) == 1:
-            return hits[0]
-    return None
-
-
-@app.post("/api/trace")
-def trace(req: ChatRequest):
-    """Like /api/chat, plus how the last reply was made: the module that won, the ones that also proposed a reply,
-    the bot files the winner called, and the file holding the reply's text."""
-    lines = user_lines([m.model_dump() for m in req.messages])
-    if not lines or req.messages[-1].role != "user":
-        raise HTTPException(400, "A user message is required.")
-    if sum(map(len, lines)) > 20000:
-        raise HTTPException(400, "This conversation is too long. Reload the page to start again.")
-    if not _slots.acquire(timeout=15):
-        raise HTTPException(503, "The bot is busy. Please try again in a moment.")
-    try:
-        result = bot().ask(lines, trace=True)
-    finally:
-        _slots.release()
-    if result.get("status") != "ok" or len(result.get("replies", [])) != len(lines) or not result.get("trace"):
-        raise HTTPException(502, "The bot failed on that line. Try another one.")
-    reply, info = result["replies"][-1], result["trace"]
-    return {"reply": reply, **info, "source": answer_source(reply, info["files"])}
-
-
-@app.get("/health")
-def health():
-    return {"ok": True}
-
-
-@app.get("/api/progress")
-def progress():
-    snapshot = HERE / "scratch" / "progress.json"
-    if not snapshot.is_file():
-        raise HTTPException(503, "The progress chart data is unavailable.")
-    return FileResponse(snapshot, media_type="application/json")
-
-
-app.mount("/", StaticFiles(directory=HERE / "site", html=True), name="site")

bot_runner.py

@@ -1,195 +0,0 @@
-# From Space HuggingFaceSwarms/SwarmChat-Demo (bot_runner.py, itself adapted from flock's run4 eval server),
-# plus a trace mode: which modules proposed a reply, which won, and the bot files the winner called.
-"""Serve a checkout of a swarm's chatbot as SwarmChat, an OpenAI-compatible endpoint for evals.
-
-    python experiments/run4/eval/swarmchat_server.py <chatbot checkout> --port 8000
-
-Loading the bot takes seconds (hundreds of modules), answering takes milliseconds. So one
-process loads the bot and forks a worker per conversation, and every conversation starts from
-the same just-loaded state. The same lines get the same replies whatever the order or the
-concurrency of the requests, in this run or the next: the hash seed, the global `random` state
-(restored in each worker) and the seeds of the bot's own random.Random() instances are fixed.
-Replies that read the clock or seed from an object id (a few modules use id(memory)) can still
-change: ids move with address randomisation, which HF Jobs do not let us turn off.
-
-POST /v1/chat/completions feeds the request's user messages to the bot in order (system and
-assistant messages are ignored: the bot only reads user lines) and returns its last reply.
-There is no sandbox: the bot runs as the server's user, with an empty environment, so run it
-somewhere disposable such as an HF Job.
-"""
-
-import json
-import os
-import signal
-import select
-import socket
-import subprocess
-import sys
-import tempfile
-from pathlib import Path
-
-LOADER = r'''
-import io, json, os, random, resource, runpy, signal, socket, sys, time
-root, seed, timeout, path, workers = sys.argv[1], int(sys.argv[2]), float(sys.argv[3]), sys.argv[4], int(sys.argv[5])
-sys.path.insert(0, root)
-status = os.fdopen(os.dup(1), "w")
-os.dup2(2, 1)  # stray prints from the bot go to the log
-listener = socket.socket(socket.AF_UNIX)
-listener.bind(path)
-listener.listen(256)
-# A worker writes a byte here when it takes a conversation; the loader forks one per byte.
-spawn_r, spawn_w = os.pipe()
-os.write(spawn_w, bytes(workers))
-
-
-def converse():
-    """A worker: take one connection, hold that conversation, send the result and exit."""
-    conn = listener.accept()[0]
-    os.write(spawn_w, b"\0")
-    random.setstate(loaded)  # random reseeds itself from the OS in every forked child
-    started = time.monotonic()
-    request = json.loads(conn.makefile("rb").readline())
-    lines, trace = (request, False) if isinstance(request, list) else (request["lines"], bool(request.get("trace")))
-    limit = int(timeout) + 1
-    signal.alarm(limit)
-    resource.setrlimit(resource.RLIMIT_CPU, (limit, limit + 1))
-    if trace:
-        result, out = "ok", {}
-        try:
-            out = traced(lines)
-        except BaseException as exc:
-            result = f"error {type(exc).__name__}"
-        conn.sendall((json.dumps({"replies": out.get("replies", []), "trace": out.get("trace"), "status": result,
-                                  "seconds": round(time.monotonic() - started, 3)}) + "\n").encode())
-        os._exit(0)
-    sys.stdin = io.StringIO("".join(line + "\n" for line in lines))
-    sys.stdout = buffer = io.StringIO()
-    result = "ok"
-    try:
-        runpy.run_module("bot", run_name="__main__")
-    except BaseException as exc:  # the fixed __main__ catches reply errors; this is the rest
-        result = f"error {type(exc).__name__}"
-    conn.sendall((json.dumps({"replies": buffer.getvalue().splitlines(), "status": result,
-                              "seconds": round(time.monotonic() - started, 3)}) + "\n").encode())
-    os._exit(0)
-
-
-def traced(lines):
-    """Replies like the fixed __main__ (one per line, whitespace-normalised, "Hello!" on errors), plus, for the last
-    line, the bot's own decision (bot.runtime.turn: candidates and winner) and the bot files each proposing handler called.
-    sys.monitoring records a function once and then disables that event, so tracing costs about nothing."""
-    import bot.runtime as rt
-    def say(text, fn):
-        try:
-            return " ".join(str(fn(text)).split()) or "Hello!"
-        except Exception:  # noqa: BLE001 - the fixed __main__'s rule
-            return "Hello!"
-    memory = {}
-    replies = [say(line, lambda t: rt.reply(t, memory)) for line in lines[:-1]]
-    mon, tool, bot_dir, seen, files = sys.monitoring, 5, os.path.join(root, "bot") + os.sep, set(), {}
-    mon.use_tool_id(tool, "pybot-trace")
-    def start(code, offset):
-        if code.co_filename.startswith(bot_dir):
-            seen.add("bot/" + code.co_filename[len(bot_dir):])
-        return mon.DISABLE
-    mon.register_callback(tool, mon.events.PY_START, start)
-    def watch(name, respond):
-        def run(text, mem):
-            seen.clear(); mon.restart_events(); mon.set_events(tool, mon.events.PY_START)
-            try:
-                return respond(text, mem)
-            finally:
-                mon.set_events(tool, 0); files[name] = sorted(seen)
-        return run
-    rt._HANDLERS = [(name, priority, watch(name, respond)) for name, priority, respond in rt._HANDLERS]
-    decision = {}
-    def last(text):
-        decision.update(rt.turn(text, memory))
-        return decision["reply"]
-    replies.append(say(lines[-1], last))
-    module = decision.get("module")
-    return {"replies": replies, "trace": {"module": module,
-            "candidates": [c["module"] for c in decision.get("candidates", [])],
-            "files": files.get(module, []) if module else []}}
-
-
-# Modules make their own random.Random(), which seeds itself from the OS: derive those seeds
-# from the run's seed and a counter instead, so every run starts in the same state.
-os_seeded, created = random.Random.seed, iter(range(1 << 62))
-def derived_seed(self, a=None, version=2):
-    os_seeded(self, f"{seed}:{next(created)}" if a is None else a, version)
-random.Random.seed = derived_seed
-random.seed = random._inst.seed
-random.seed(seed)
-started = time.monotonic()
-import bot.runtime
-loaded = random.getstate()
-print(json.dumps({"load_s": round(time.monotonic() - started, 2),
-                  "handlers": len(bot.runtime._HANDLERS)}), file=status, flush=True)
-# Keep `workers` forks waiting for a connection. The workers read the requests, never the
-# loader, and the kernel reaps them.
-signal.signal(signal.SIGCHLD, signal.SIG_IGN)
-while True:
-    os.read(spawn_r, 1)
-    if os.fork() == 0:
-        converse()
-'''
-
-
-class Bot:
-    """The loader process; ask() holds one conversation in a fresh worker."""
-
-    def __init__(self, tree: Path, *, seed: int, timeout: float, workers: int, log):
-        self.timeout = timeout
-        self.tmp = tempfile.TemporaryDirectory(prefix="swarmchat-")
-        self.path = os.path.join(self.tmp.name, "bot.sock")
-        # No -I: PYTHONHASHSEED must apply. The environment holds nothing else.
-        self.proc = subprocess.Popen(
-            [sys.executable, "-s", "-S", "-P", "-c", LOADER, str(tree), str(seed), str(timeout),
-             self.path, str(workers)],
-            cwd=self.tmp.name, env={"PYTHONHASHSEED": str(seed)}, stdout=subprocess.PIPE,
-            stderr=log, text=True, start_new_session=True)
-        if not select.select([self.proc.stdout], [], [], 90)[0]:
-            self.close()
-            raise RuntimeError("The bot did not load within 90 seconds.")
-        line = self.proc.stdout.readline()
-        if not line:
-            self.close()
-            raise RuntimeError("This commit could not load its bot; try another commit.")
-        self.ready = json.loads(line)
-
-    def ask(self, lines: list[str], trace: bool = False) -> dict:
-        with socket.socket(socket.AF_UNIX) as conn:
-            conn.settimeout(self.timeout + 5)
-            conn.connect(self.path)
-            conn.sendall((json.dumps({"lines": lines, "trace": True} if trace else lines) + "\n").encode())
-            try:
-                raw = conn.makefile("rb").readline()
-            except TimeoutError:
-                return {"replies": [], "status": "timeout"}
-        # The worker died without replying, e.g. on its CPU or wall-clock limit.
-        return json.loads(raw) if raw else {"replies": [], "status": "worker exited"}
-
-    def alive(self) -> bool:
-        return self.proc.poll() is None
-
-    def close(self):
-        if self.alive():
-            os.killpg(self.proc.pid, signal.SIGKILL)
-        self.proc.wait()
-        self.proc.stdout.close()
-        self.tmp.cleanup()
-
-
-def user_lines(messages) -> list[str]:
-    """User message texts as single bot input lines (the bot reads one line per turn)."""
-    lines = []
-    for m in messages:
-        if m.get("role") != "user":
-            continue
-        content = m.get("content") or ""
-        if isinstance(content, list):
-            content = " ".join(p.get("text", "") for p in content if p.get("type") == "text")
-        lines.append(" ".join(str(content).split()))
-    return lines
-

build/build.mjs

@@ -0,0 +1,11 @@
+// Build the browser bundle inside the same Pyodide the page uses: node build.mjs <bot source zip> <build_bundle.py> <out zip>
+import { loadPyodide } from "pyodide";
+import { readFileSync, writeFileSync } from "node:fs";
+
+const [src, script, out] = process.argv.slice(2);
+const py = await loadPyodide();
+py.unpackArchive(new Uint8Array(readFileSync(src)), "zip", { extractDir: "/in" });
+py.FS.mkdirTree("/out");
+py.runPython(readFileSync(script, "utf8"));
+writeFileSync(out, py.FS.readFile("/out/pybot.zip"));
+console.log("python", py.runPython("import sys; sys.version"), "->", out);

build/build_bundle.py

@@ -0,0 +1,30 @@
+"""Run inside Pyodide (build.mjs): turn the bot's source zip into the browser bundle.
+
+In:  /in/bot/**.py (the bot at one commit)
+Out: /out/pybot.zip with
+  - bot/**.pyc  sourceless bytecode for this Pyodide's CPython (imports skip compiling 33 MB of source),
+                co_filename kept as /pybot/bot/<path>.py so the trace still names the right files;
+  - norm.json   every file's text, letters and digits only, for finding the file that holds a reply.
+"""
+import json
+import os
+import py_compile
+import re
+import zipfile
+
+SRC, OUT = "/in", "/out"
+norm = {}
+with zipfile.ZipFile(f"{OUT}/pybot.zip", "w", zipfile.ZIP_DEFLATED, compresslevel=9) as bundle:
+    for dirpath, _, names in sorted(os.walk(f"{SRC}/bot")):
+        for name in sorted(names):
+            if not name.endswith(".py"):
+                continue
+            path = os.path.join(dirpath, name)
+            rel = os.path.relpath(path, SRC)
+            pyc = py_compile.compile(path, cfile=f"/tmp/{rel}c", dfile=f"/pybot/{rel}", doraise=True,
+                                     invalidation_mode=py_compile.PycInvalidationMode.UNCHECKED_HASH)
+            bundle.write(pyc, rel + "c")
+            with open(path, encoding="utf-8", errors="replace") as handle:
+                norm[rel] = re.sub(r"[^a-z0-9]", "", handle.read().lower())
+    bundle.writestr("norm.json", json.dumps(norm, separators=(",", ":")))
+print(json.dumps({"files": len(norm), "bytes": os.path.getsize(f"{OUT}/pybot.zip")}))

build_progress.py

@@ -1,127 +0,0 @@
-"""Build the local chart snapshot from the leaderboard's cached summaries."""
-
-import argparse
-import json
-import math
-import subprocess
-from pathlib import Path
-
-# The abrupt outage plateau, bounded by the last/first affected commit timestamps.
-# Explicit bounds retain genuine regressions elsewhere; never filter on score.
-OUTAGE_START = "20261107T121247Z"
-OUTAGE_END = "20261107T131511Z"
-MODEL = "HuggingFaceSwarms/SwarmChat-v8"
-
-
-def count_source_tree(repository, commit):
-    """Count a source tree directly for evaluated commits outside the main history."""
-    tree = subprocess.check_output(
-        ["git", "-C", str(repository), "ls-tree", "-r", commit, "--", "bot"], text=True)
-    objects = [line.split()[2] for line in tree.splitlines() if line.endswith(".py")]
-    total = 0
-    with subprocess.Popen(["git", "-C", str(repository), "cat-file", "--batch"],
-                          stdin=subprocess.PIPE, stdout=subprocess.PIPE) as process:
-        for oid in objects:
-            process.stdin.write((oid + "\n").encode())
-            process.stdin.flush()
-            size = int(process.stdout.readline().split()[2])
-            blob = process.stdout.read(size)
-            process.stdout.read(1)
-            total += blob.count(b"\n") + bool(blob and not blob.endswith(b"\n"))
-        process.stdin.close()
-        if process.wait() != 0:
-            raise RuntimeError(f"Could not read source tree for {commit}")
-    return total
-
-
-def source_line_counts(repository):
-    """Physical lines in bot/**/*.py, including blanks/comments, excluding tests.
-
-    Accumulate first-parent diffs, including deletions and merge changes. Disabling
-    rename detection makes moves across the bot/ boundary count correctly.
-    """
-    command = ["git", "-C", str(repository), "log", "--first-parent", "--reverse",
-               "--format=commit:%H", "--numstat", "--no-renames",
-               "--diff-merges=first-parent", "HEAD"]
-    counts, commit, total = {}, None, 0
-    with subprocess.Popen(command, stdout=subprocess.PIPE, text=True) as process:
-        for line in process.stdout:
-            line = line.rstrip("\n")
-            if line.startswith("commit:"):
-                if commit is not None:
-                    counts[commit[:12]] = total
-                commit = line.removeprefix("commit:")
-            elif "\t" in line:
-                added, removed, path = line.split("\t", 2)
-                if path.startswith("bot/") and path.endswith(".py"):
-                    if added == "-" or removed == "-":
-                        raise ValueError(f"Cannot count binary Python file: {path}")
-                    total += int(added) - int(removed)
-        if process.wait() != 0:
-            raise RuntimeError("Could not read source history")
-    if commit is not None:
-        counts[commit[:12]] = total
-    return counts
-
-
-def build_progress(cache):
-    latest = {}
-    for path, summary in cache["summaries"].items():
-        meta = summary.get("metadata", {})
-        metric = summary.get("primary_metric", {})
-        score = metric.get("value")
-        if (meta.get("task") != "smoltalk_bench_gold"
-                or meta.get("status") != "success" or meta.get("task_args")
-                or meta.get("total_samples") != 1000
-                or metric.get("name") != "length_penalised_win_rate"
-                or not isinstance(score, (int, float)) or not math.isfinite(score)):
-            continue
-        key = (meta["model_id"], meta["revision"])
-        order = (meta["completed_at"], path)
-        if key not in latest or order > latest[key][0]:
-            latest[key] = (order, summary)
-    revisions = sorted((revision, summary) for (model, revision), (_, summary)
-                       in latest.items() if model == MODEL)
-    points, excluded = [], 0
-    for index, (revision, summary) in enumerate(revisions, 1):
-        if OUTAGE_START <= revision.split("-")[0] <= OUTAGE_END:
-            excluded += 1
-            continue
-        points.append({"revision": index, "commit": revision.split("-")[1],
-                       "score": summary["primary_metric"]["value"]})
-    baselines = sorted([
-        {"name": model.split("/")[-1], "score": summary["primary_metric"]["value"]}
-        for (model, revision), (_, summary) in latest.items()
-        if model.startswith("HuggingFaceTB/SmolLM-") and revision == "main"
-    ], key=lambda baseline: baseline["score"])
-    if not points or len(baselines) != 3:
-        raise ValueError("Expected v8 evaluations and all three SmolLM baselines")
-    return {"points": points, "baselines": baselines, "excluded": excluded,
-            "fetched_at": cache["fetched_at"]}
-
-
-if __name__ == "__main__":
-    parser = argparse.ArgumentParser(description=__doc__)
-    parser.add_argument("cache", type=Path)
-    parser.add_argument("--source-repo", type=Path, required=True,
-                        help="Git repository containing the evaluated source history")
-    args = parser.parse_args()
-    result = build_progress(json.loads(args.cache.read_text()))
-    counts = source_line_counts(args.source_repo)
-    missing = []
-    for point in result["points"]:
-        if point["commit"] not in counts:
-            exists = subprocess.run(
-                ["git", "-C", str(args.source_repo), "cat-file", "-e", point["commit"]],
-                stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL).returncode == 0
-            counts[point["commit"]] = count_source_tree(args.source_repo, point["commit"]) if exists else None
-        if counts[point["commit"]] is None:
-            missing.append(point["commit"])
-        point["lines_of_code"] = counts[point["commit"]]
-    result["missing_source_commits"] = missing
-    if missing:
-        print(f"No source available for {len(missing)} commits; leaving code-size gaps: {', '.join(missing)}")
-    output = Path(__file__).parent / "scratch" / "progress.json"
-    output.parent.mkdir(exist_ok=True)
-    output.write_text(json.dumps(result, separators=(",", ":")) + "\n")
-    print(f"Wrote {len(result['points'])} points; excluded {result['excluded']} outage evaluations")

index.html

@@ -0,0 +1,232 @@
+<!doctype html>
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<title>PyBot</title>
+<style>
+  @font-face { font-family:"Routed Gothic"; src:url("fonts/routed-gothic.ttf") format("truetype"); }
+  @font-face { font-family:"Letter Gothic"; src:url("fonts/LetterGothicCode-R.ttf") format("truetype"); }
+  @font-face { font-family:"Letter Gothic"; font-weight:700; src:url("fonts/LetterGothicCode-B.ttf") format("truetype"); }
+  /* the swarms landing page's palette and type; one accent, for the module whose reply won */
+  :root { --ink:#000; --paper:#fff; --mute:#5c5c5c; --hair:#e9e9e9; --faint:#d6d6d6; --off:#f4f4f4; --accent:#eb6834;
+          --lead:"Routed Gothic",Helvetica,Arial,sans-serif; --mono:"Letter Gothic",ui-monospace,Menlo,monospace; color-scheme:light; }
+  * { box-sizing:border-box; }
+  html, body { height:100%; }
+  body { margin:0; background:var(--paper); color:var(--ink); font:18px/1.55 var(--lead); }
+  .page { min-height:100%; max-width:1400px; margin:0 auto; padding:clamp(16px,3vw,32px) clamp(16px,3vw,32px) 20px;
+          display:flex; flex-direction:column; gap:16px; }
+  header { display:flex; align-items:baseline; justify-content:space-between; gap:12px 24px; flex-wrap:wrap; }
+  h1 { font:400 clamp(34px,5vw,52px)/1 var(--lead); letter-spacing:.02em; margin:0; }
+  #status { flex:1 1 320px; max-width:520px; font:13px/1.4 var(--mono); color:var(--mute); }
+  #status .bar { height:4px; background:var(--off); border:1px solid var(--ink); margin-top:6px; overflow:hidden; }
+  #status .bar i { display:block; height:100%; width:0; background:var(--ink); transition:width .2s; }
+  #status .bar.busy i { width:30%; animation:slide 1.2s ease-in-out infinite; }
+  #status.done .bar { display:none; }
+  #status.failed { color:var(--accent); }
+  @keyframes slide { from { transform:translateX(-100%); } to { transform:translateX(340%); } }
+  .grid { flex:1; display:grid; grid-template-columns:minmax(0,5fr) minmax(0,6fr); gap:16px; min-height:0; }
+  .app { border:1px solid var(--ink); display:flex; flex-direction:column; min-height:0; }
+  .chat { height:calc(100dvh - 170px); min-height:420px; }
+  .trace { height:calc(100dvh - 170px); min-height:420px; }
+  #log { flex:1; overflow-y:auto; display:flex; flex-direction:column; gap:10px; padding:14px; }
+  .msg { max-width:85%; padding:7px 12px; white-space:pre-wrap; overflow-wrap:anywhere; font-size:17px; }
+  .me { align-self:flex-end; background:var(--ink); color:var(--paper); }
+  .bot { align-self:flex-start; background:var(--off); border:1px solid var(--hair); }
+  .bot.typing { color:var(--mute); }
+  .hint { color:var(--mute); text-align:center; margin:auto 0 0; font-size:16px; }
+  .chips { display:flex; flex-wrap:wrap; justify-content:center; gap:8px; margin:10px 0 auto; }
+  .chips button { font:15px var(--lead); text-transform:none; letter-spacing:0; color:var(--ink); background:var(--paper);
+                  border:1px solid var(--ink); padding:5px 12px; }
+  .chips button:hover { background:var(--off); }
+  form { display:flex; gap:8px; padding:10px 14px; border-top:1px solid var(--ink); }
+  input { flex:1; min-width:0; font:17px var(--lead); color:var(--ink); background:var(--paper); border:1px solid var(--ink); padding:8px 12px; }
+  button { font:700 14px var(--mono); text-transform:uppercase; letter-spacing:.04em; color:var(--paper); background:var(--ink); border:1px solid var(--ink); padding:0 16px; cursor:pointer; }
+  button.ghost { color:var(--ink); background:var(--paper); width:42px; padding:0; font:400 20px var(--lead); }
+  button:disabled, input:disabled { opacity:.45; cursor:default; }
+  input:focus-visible, button:focus-visible { outline:2px solid var(--accent); outline-offset:2px; }
+  #why { font:14px/1.5 var(--mono); color:var(--mute); min-height:44px; padding:10px 14px; border-bottom:1px solid var(--hair); }
+  #why b { color:var(--ink); }
+  #map { flex:1; min-height:0; width:100%; }
+  .legend { display:flex; flex-wrap:wrap; gap:6px 18px; font:13px var(--mono); color:var(--mute); padding:8px 14px 12px; border-top:1px solid var(--hair); }
+  .legend i { display:inline-block; width:16px; height:12px; vertical-align:-1px; margin-right:6px; border:1px solid var(--ink); }
+  .sw-win { background:var(--accent); border-width:2px !important; } .sw-src { background:var(--ink); } .sw-used { background:#8a8a8a; }
+  .sw-prop { background:var(--paper); border-width:2px !important; } .sw-off { background:var(--off); border-color:var(--faint) !important; }
+  .foot { font:13px var(--mono); color:var(--mute); margin:0; } .foot a { color:inherit; }
+  @media (max-width:900px) {
+    .grid { grid-template-columns:minmax(0,1fr); }
+    .chat { height:min(70dvh,560px); min-height:360px; }
+    .trace { height:auto; min-height:0; }
+    #map { flex:none; height:420px; }
+  }
+  @media (max-width:640px) { body { font-size:17px; } .msg { max-width:92%; } #map { height:360px; } }
+  @media (prefers-reduced-motion:reduce) { #status .bar.busy i { animation:none; width:100%; opacity:.3; } }
+</style>
+</head>
+<body>
+<div class="page">
+  <header>
+    <h1>PyBot</h1>
+    <div id="status" role="status" aria-live="polite"><span id="status-text">Starting…</span><div class="bar busy"><i></i></div></div>
+  </header>
+
+  <div class="grid">
+    <section class="app chat" aria-label="Talk to PyBot">
+      <div id="log" aria-live="polite"></div>
+      <form id="f" autocomplete="off">
+        <button type="button" id="reset" class="ghost" title="Start a new conversation" aria-label="New chat">↻</button>
+        <input id="q" maxlength="500" placeholder="Say something to PyBot" aria-label="Message">
+        <button id="send">Send</button>
+      </form>
+    </section>
+
+    <section class="app trace" aria-label="How PyBot made the last reply">
+      <div id="why">Say something to PyBot, then see which of its files made the reply.</div>
+      <div id="map" role="img" aria-label="PyBot's files; the ones used for the last reply are highlighted"></div>
+      <div class="legend">
+        <span><i class="sw-win"></i>module whose reply won</span>
+        <span><i class="sw-src"></i>file holding the reply's text</span>
+        <span><i class="sw-used"></i>files the winner called</span>
+        <span><i class="sw-prop"></i>modules that also proposed a reply</span>
+        <span><i class="sw-off"></i>not used</span>
+      </div>
+    </section>
+  </div>
+
+  <p class="foot">PyBot at commit <a id="commit" target="_blank" rel="noopener"></a>, running in your browser with <a href="https://pyodide.org" target="_blank" rel="noopener">Pyodide</a>. Click a file to open it on the Hub.</p>
+</div>
+
+<script src="https://cdn.jsdelivr.net/npm/plotly.js-dist-min@2.35.2/plotly.min.js"></script>
+<script>
+const INK = "#000000", PAPER = "#ffffff", OFF = "#f4f4f4", FAINT = "#d6d6d6", USED = "#8a8a8a", WIN = "#eb6834", MONO = "'Letter Gothic', ui-monospace, Menlo, monospace";
+const EXAMPLES = ["Hi, I'm Sam", "Tell me a joke", "How do I make a cup of tea?", "What is 17 times 23?", "I feel sad today", "Can you recommend a book?"];
+const $ = (id) => document.getElementById(id);
+const log = $("log"), q = $("q"), send = $("send"), why = $("why"), statusBox = $("status"), statusText = $("status-text"), bar = statusBox.querySelector(".bar");
+
+// ---- the bot, in a Web Worker ----
+const worker = new Worker("worker.js", {type: "module"});
+// The worker script itself failed (e.g. the Pyodide CDN is blocked): report it like a load error.
+worker.onerror = (e) => { e.preventDefault(); worker.onmessage({data: {type: "error", message: e.message || "the Python runtime could not be downloaded"}}); };
+let botReady = false, nextId = 0, turn = 0, pending = null;  // pending: {id, bubble, mine}
+const mb = (n) => (n / 1e6).toFixed(1);
+function setStatus(text, fraction) {
+  statusText.textContent = text;
+  bar.classList.toggle("busy", fraction == null);
+  bar.querySelector("i").style.width = fraction == null ? "" : `${Math.round(fraction * 100)}%`;
+}
+worker.onmessage = ({data: m}) => {
+  if (m.type === "progress") {
+    if (m.stage === "runtime") setStatus("Downloading Python (Pyodide) and PyBot…");
+    else if (m.stage === "cached") setStatus("Downloading Python (Pyodide)… PyBot is cached in your browser");
+    else if (m.stage === "download") setStatus(`Downloading PyBot · ${mb(m.loaded)}${m.total ? ` / ${mb(m.total)}` : ""} MB`, m.total ? m.loaded / m.total : null);
+    else if (m.stage === "import") setStatus(`Loading PyBot's ${TREE ? TREE.files.length : ""} Python files…`);
+  } else if (m.type === "ready") {
+    botReady = true; statusBox.classList.add("done");
+    setStatus(`Ready · ${m.modules} modules loaded in ${m.load_s.toFixed(1)} s · Python ${m.python} in your browser, no server`);
+    if (!pending) { q.disabled = send.disabled = false; q.placeholder = "Say something to PyBot"; }
+  } else if (m.type === "reply") {
+    if (!pending || m.id !== pending.id) return;
+    const {bubble, mine} = pending; pending = null;
+    if (mine === turn) {
+      bubble.textContent = m.reply; bubble.classList.remove("typing");
+      last = m; why.innerHTML = explain(m); draw();
+    }
+    done();
+  } else if (m.type === "error") {
+    if (m.id == null) {  // loading failed
+      statusBox.classList.add("failed"); setStatus(`PyBot could not start: ${m.message}`, 0);
+      if (pending) { pending.bubble.textContent = "PyBot could not start. Reload the page to try again."; pending.bubble.classList.remove("typing"); pending = null; }
+      q.disabled = send.disabled = true; return;
+    }
+    if (pending && m.id === pending.id) { pending.bubble.textContent = "PyBot could not answer that. Try another line."; pending.bubble.classList.remove("typing"); pending = null; }
+    done();
+  }
+};
+function done() { q.disabled = send.disabled = false; q.focus({preventScroll:true}); log.scrollTop = log.scrollHeight; }
+
+// ---- the file map ----
+let TREE = null, dirs, ids, labels, parents, values, last = null;  // last: the last reply's trace
+const url = (p) => `https://huggingface.co/datasets/${TREE.repo}/blob/${TREE.sha}/${p}`;
+function roleOf(id) {
+  if (!last || dirs.has(id)) return null;
+  const mod = (m) => `bot/modules/${m}.py`;
+  if (last.module && id === mod(last.module)) return "won";
+  if (id === last.source) return "text";
+  if (last.files.includes(id)) return "used";
+  if (last.candidates.some((m) => mod(m) === id)) return "proposed";
+  return "off";
+}
+function draw() {
+  if (!TREE || !window.Plotly) return;
+  const roles = ids.map(roleOf);
+  const colors = ids.map((id, i) => dirs.has(id) ? PAPER : !last ? OFF : {won: WIN, text: INK, used: USED, proposed: PAPER, off: OFF}[roles[i]]);
+  const widths = roles.map((r) => (r === "won" || r === "proposed" ? 3 : 1));
+  const lines = ids.map((id, i) => (dirs.has(id) ? INK : roles[i] === "won" || roles[i] === "proposed" || roles[i] === "text" ? INK : roles[i] === "used" ? PAPER : FAINT));
+  const hover = roles.map((r) => ({won: "won: its reply was used", text: "holds the reply's text", used: "called by the winner",
+    proposed: "also proposed a reply", off: "not used for this reply"}[r] || ""));
+  Plotly.react("map", [{type:"treemap", ids, labels, parents, values, branchvalues:"total", maxdepth:3,
+    marker:{colors, line:{color:lines, width:widths}}, customdata:hover,
+    pathbar:{visible:true, thickness:22, textfont:{family:MONO, size:12, color:INK}},
+    texttemplate:"%{label}", textfont:{family:MONO, size:12},
+    hovertemplate:"<b>%{id}</b><br>%{value:,} lines<br>%{customdata}<extra></extra>"}],
+    {margin:{l:0, r:0, t:26, b:0}, paper_bgcolor:PAPER, font:{family:MONO, color:INK},
+     hoverlabel:{bgcolor:PAPER, bordercolor:INK, font:{family:MONO, size:13, color:INK}}},
+    {responsive:true, displaylogo:false, displayModeBar:false});
+}
+fetch("tree.json").then((r) => r.json()).then((tree) => {
+  TREE = tree;
+  const commit = $("commit"); commit.textContent = TREE.sha.slice(0, 7); commit.href = `https://huggingface.co/datasets/${TREE.repo}/tree/${TREE.sha}/bot`;
+  dirs = new Map();
+  for (const [p, n] of TREE.files) { const parts = p.split("/"); for (let i = 1; i < parts.length; i++) { const d = parts.slice(0, i).join("/"); dirs.set(d, (dirs.get(d) || 0) + n); } }
+  ids = [...dirs.keys(), ...TREE.files.map(([p]) => p)];
+  labels = ids.map((id) => (dirs.has(id) ? id.split("/").pop() + "/" : id.split("/").pop()));
+  parents = ids.map((id) => (id.includes("/") ? id.slice(0, id.lastIndexOf("/")) : ""));
+  values = [...dirs.values(), ...TREE.files.map(([, n]) => n)];
+  draw();
+  $("map").on("plotly_treemapclick", (ev) => { const id = ev.points[0].id; if (!dirs.has(id)) { window.open(url(id), "_blank", "noopener"); return false; } });
+});
+
+// ---- the chat ----
+const add = (cls, text) => { const p = document.createElement("div"); p.className = "msg " + cls; p.textContent = text; log.appendChild(p); log.scrollTop = log.scrollHeight; return p; };
+function explain(t) {
+  const ms = ` · ${t.ms < 1000 ? `${t.ms} ms` : `${(t.ms / 1000).toFixed(1)} s`}`;
+  if (!t.module) return "No module had a reply, so PyBot fell back to its default line." + ms;
+  const others = t.candidates.filter((m) => m !== t.module);
+  return `<b>${t.module}</b> won${others.length ? ` over ${others.map((m) => `<b>${m}</b>`).join(", ")}` : " (the only module with a reply)"}`
+    + ` · it called <b>${t.files.length}</b> files` + (t.source ? ` · the reply's text is in <b>${t.source.replace("bot/", "")}</b>` : "") + ms;
+}
+function welcome(text) {
+  log.innerHTML = "";
+  const hint = document.createElement("p"); hint.className = "hint"; hint.textContent = text; log.appendChild(hint);
+  const chips = document.createElement("div"); chips.className = "chips";
+  for (const ex of EXAMPLES) {
+    const b = document.createElement("button"); b.type = "button"; b.textContent = ex;
+    b.addEventListener("click", () => { if (!pending) { q.value = ex; $("f").requestSubmit(); } });
+    chips.appendChild(b);
+  }
+  log.appendChild(chips);
+}
+function ask(text) {
+  log.querySelectorAll(".hint, .chips").forEach((el) => el.remove());
+  add("me", text); q.value = ""; q.disabled = send.disabled = true;
+  const bubble = add("bot typing", botReady ? "…" : "… (PyBot is still loading)");
+  pending = {id: ++nextId, bubble, mine: turn};
+  worker.postMessage({type: "turn", id: pending.id, text});
+}
+$("f").addEventListener("submit", (e) => {
+  e.preventDefault();
+  const text = q.value.trim(); if (!text || pending || statusBox.classList.contains("failed")) return;
+  ask(text);
+});
+$("reset").addEventListener("click", () => {
+  turn++; pending = null; last = null; draw();
+  why.textContent = "New conversation. PyBot has forgotten everything you said.";
+  worker.postMessage({type: "reset"});
+  welcome("New conversation. Say something, or try one of these:");
+  q.value = ""; q.disabled = send.disabled = false; q.focus({preventScroll:true});
+});
+welcome("Say something to PyBot, or try one of these:");
+q.focus({preventScroll:true});
+</script>
+</body>
+</html>

bot/bot-64eb2c844738.zip → pybot-3185a7a9bc50.zip

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6e0650e67c8a4ab9935aac7f0f2f866c15daafed1e835d85ee4744d434a4a91d
-size 9528378
+oid sha256:6c8eb9648a6b10baac097c349a05f2bd6546076125cb4a47ce4873835a03f958
+size 18004369

pybot_web.py

@@ -0,0 +1,121 @@
+"""PyBot in the browser: load the bot once, then answer turns and explain each reply.
+
+Ported from the PyBot Space's server (bot_runner.py's LOADER and traced(), app.py's answer_source()):
+- Random seeds are fixed the way the eval server fixes them, so replies match the benchmark run as
+  closely as a browser allows. The server forks a fresh worker per request; here `reset()` restores the
+  just-loaded random state and starts a new memory instead.
+- For each reply: the module that won, the ones that also proposed a reply, the bot files the winner
+  called (sys.monitoring, one PY_START event per function), and the file holding the reply's text.
+"""
+import json
+import os
+import random
+import re
+import sys
+import time
+
+ROOT = "/pybot"
+SEED = 0
+_norm = lambda s: re.sub(r"[^a-z0-9]", "", s.lower())
+
+_rt = None
+_loaded_state = None
+_memory = {}
+_files = {}
+_sources = None
+
+
+def load():
+    """Import the bot (seconds); returns {"load_s", "modules", "handlers"}."""
+    global _rt, _loaded_state
+    sys.path.insert(0, ROOT)
+    # Modules make their own random.Random(), which seeds itself from the OS: derive those seeds
+    # from the run's seed and a counter instead, so every load starts in the same state.
+    os_seeded, created = random.Random.seed, iter(range(1 << 62))
+
+    def derived_seed(self, a=None, version=2):
+        os_seeded(self, f"{SEED}:{next(created)}" if a is None else a, version)
+
+    random.Random.seed = derived_seed
+    random.seed = random._inst.seed
+    random.seed(SEED)
+    started = time.monotonic()
+    import bot.runtime as rt
+
+    _rt = rt
+    _loaded_state = random.getstate()
+    _watch_handlers()
+    return {"load_s": round(time.monotonic() - started, 2), "modules": len(rt._MODULES), "handlers": len(rt._HANDLERS)}
+
+
+def _watch_handlers():
+    """Wrap every respond() so a turn records the bot files each handler called."""
+    mon, tool = sys.monitoring, 5
+    bot_dir, seen = os.path.join(ROOT, "bot") + os.sep, set()
+    mon.use_tool_id(tool, "pybot-trace")
+
+    def start(code, offset):
+        if code.co_filename.startswith(bot_dir):
+            seen.add("bot/" + code.co_filename[len(bot_dir):])
+        return mon.DISABLE
+
+    mon.register_callback(tool, mon.events.PY_START, start)
+
+    def watch(name, respond):
+        def run(text, mem):
+            seen.clear()
+            mon.restart_events()
+            mon.set_events(tool, mon.events.PY_START)
+            try:
+                return respond(text, mem)
+            finally:
+                mon.set_events(tool, 0)
+                _files[name] = sorted(seen)
+        return run
+
+    _rt._HANDLERS = [(name, priority, watch(name, respond)) for name, priority, respond in _rt._HANDLERS]
+
+
+def reset():
+    """A new conversation: empty memory and the just-loaded random state."""
+    global _memory
+    _memory = {}
+    random.setstate(_loaded_state)
+
+
+def turn(text):
+    """One reply as JSON: {"reply", "module", "candidates", "files", "source", "ms"}."""
+    text = " ".join(str(text).split())
+    _files.clear()
+    started = time.monotonic()
+    decision = {}
+    try:
+        decision = _rt.turn(text, _memory)
+        reply = " ".join(str(decision["reply"]).split()) or "Hello!"
+    except Exception:  # noqa: BLE001 - the fixed __main__'s rule
+        reply = "Hello!"
+    ms = round((time.monotonic() - started) * 1000)
+    module = decision.get("module")
+    files = _files.get(module, []) if module else []
+    return json.dumps({"reply": reply, "module": module,
+                       "candidates": [c["module"] for c in decision.get("candidates", [])],
+                       "files": files, "source": answer_source(reply, files), "ms": ms})
+
+
+def answer_source(reply, used):
+    """The one file that holds the reply verbatim (its first 40+ letters/digits), preferring files the winner called;
+    None when the reply is composed, too short, or found in several files."""
+    global _sources
+    if _sources is None:  # every bot file, letters and digits only (built by build_bundle.py)
+        with open(os.path.join(ROOT, "norm.json"), encoding="utf-8") as handle:
+            _sources = json.load(handle)
+    text = _norm(reply)
+    for n in (60, 40):
+        key = text[:n]
+        if len(key) < 20:  # too short to point at one place
+            continue
+        hits = [f for f, body in _sources.items() if key in body]
+        hits = hits if len(hits) == 1 else [f for f in hits if f in used]
+        if len(hits) == 1:
+            return hits[0]
+    return None

requirements.txt

@@ -1,2 +0,0 @@
-fastapi==0.115.6
-uvicorn==0.32.1

site/progress.js

@@ -1,79 +0,0 @@
-async function drawProgress() {
-  const status = document.getElementById("progress-status");
-  const chart = document.getElementById("progress-chart");
-  const reducedMotion = window.matchMedia("(prefers-reduced-motion: reduce)");
-  const animateOnScroll = !reducedMotion.matches && "IntersectionObserver" in window;
-  if (animateOnScroll) chart.classList.add("awaiting-reveal");
-  try {
-    const response = await fetch("api/progress", {cache: "no-store"});
-    if (!response.ok) throw new Error("Progress data unavailable");
-    const data = await response.json();
-    const points = data.points;
-    const extent = [points[0].revision, points[points.length - 1].revision];
-    const traces = [{
-      type: "scatter", mode: "lines", name: "PyBot",
-      x: points.map(p => p.revision), y: points.map(p => p.score),
-      customdata: points.map(p => p.commit),
-      line: {color: "#eb6834", width: 2},
-      hovertemplate: "PyBot · revision %{x:,}<br>Win rate: %{y:.2f}%<br>Commit %{customdata}<extra></extra>"
-    }, {
-      type: "scatter", mode: "lines", name: "Lines of code", yaxis: "y2",
-      x: points.map(p => p.revision), y: points.map(p => p.lines_of_code),
-      customdata: points.map(p => p.commit),
-      line: {color: "#287a9b", width: 2},
-      hovertemplate: "Revision %{x:,}<br>Lines of code: %{y:,}<br>Python source in bot/ (including blanks and comments)<br>Commit %{customdata}<extra></extra>"
-    }, ...data.baselines.map((baseline, index) => ({
-      type: "scatter", mode: "lines",
-      name: baseline.name,
-      x: extent, y: [baseline.score, baseline.score],
-      line: {color: ["#999999", "#555555", "#000000"][index], width: 1.5,
-             dash: ["dot", "dash", "dashdot"][index]},
-      hovertemplate: `${baseline.name}<br>Win rate: %{y:.2f}%<extra></extra>`
-    }))];
-    await Plotly.newPlot("progress-chart", traces, {
-      margin: {l: 60, r: 72, t: 30, b: 60},
-      paper_bgcolor: "#ffffff", plot_bgcolor: "#ffffff",
-      font: {family: "'Letter Gothic', monospace", size: 12, color: "#000000"},
-      xaxis: {title: {text: "PyBot commit", standoff: 12},
-              range: extent, gridcolor: "#eeeeee", zeroline: false, automargin: true},
-      yaxis: {title: {text: "Win rate (%)", standoff: 10}, rangemode: "tozero",
-              color: "#eb6834",
-              gridcolor: "#eeeeee", zerolinecolor: "#dddddd", automargin: true},
-      yaxis2: {title: {text: "Lines of code", standoff: 10}, overlaying: "y", side: "right",
-               rangemode: "tozero", color: "#287a9b", tickformat: "~s",
-               showgrid: false, zeroline: false, automargin: true},
-      showlegend: false,
-      annotations: data.baselines.map(baseline => ({
-        text: baseline.name, x: 1, xref: "paper", y: baseline.score, yref: "y",
-        xanchor: "right", yanchor: "bottom", yshift: 4, showarrow: false,
-        font: {size: 11, color: "#000000"}, bgcolor: "#ffffff", borderpad: 2
-      })),
-      hoverlabel: {bgcolor: "#ffffff", bordercolor: "#000000"}
-    }, {responsive: true, displayModeBar: false, displaylogo: false});
-    status.hidden = true;
-    if (animateOnScroll) {
-      const observer = new IntersectionObserver(entries => {
-        if (!entries.some(entry => entry.isIntersecting && entry.intersectionRatio >= 0.3)) return;
-        observer.disconnect();
-        // Reveal the rendered SVG traces, keeping both axes and all data fixed.
-        // This avoids redrawing thousands of points on every animation frame.
-        chart.classList.remove("awaiting-reveal");
-        if (reducedMotion.matches) return;
-        const animations = [...chart.querySelectorAll(".scatterlayer .trace")].map(trace =>
-          trace.animate([
-            {clipPath: "inset(-10px 100% -10px 0)"},
-            {clipPath: "inset(-10px 0% -10px 0)"}
-          ], {duration: 1800, easing: "cubic-bezier(0.22, 1, 0.36, 1)"}));
-        const finish = () => animations.forEach(animation => animation.finish());
-        reducedMotion.addEventListener("change", finish, {once: true});
-        Promise.all(animations.map(animation => animation.finished)).finally(() =>
-          reducedMotion.removeEventListener("change", finish));
-      }, {threshold: 0.3});
-      observer.observe(chart);
-    }
-  } catch (error) {
-    document.getElementById("progress-chart").hidden = true;
-    status.textContent = "The progress chart is currently unavailable.";
-  }
-}
-drawProgress();

tree.json

@@ -0,0 +1 @@
+{"repo":"HuggingFaceSwarms/SwarmChat-v8","sha":"3185a7a9bc50491205b19f64839f8c70962c4369","files":[["bot/__init__.py",1],["bot/__main__.py",19],["bot/compose/__init__.py",523],["bot/compose/a_place_nobody_can_see.py",439],["bot/compose/badnews.py",1272],["bot/compose/breakup.py",74],["bot/compose/checking.py",966],["bot/compose/childwords.py",163],["bot/compose/circumstance.py",156],["bot/compose/doubt.py",138],["bot/compose/draft.py",718],["bot/compose/facts.py",96],["bot/compose/fallback.py",15517],["bot/compose/fallback_chain.py",5117],["bot/compose/fallback_data.py",8200],["bot/compose/fallback_pools.py",691],["bot/compose/fallback_words.py",514],["bot/compose/find_a_place.py",352],["bot/compose/finding_out.py",806],["bot/compose/findout.py",526],["bot/compose/fit.py",3892],["bot/compose/gap.py",388],["bot/compose/goodnews.py",1548],["bot/compose/handoff.py",149],["bot/compose/harm.py",334],["bot/compose/how_it_went.py",325],["bot/compose/invitation.py",134],["bot/compose/judgement.py",41],["bot/compose/lead.py",249],["bot/compose/leaving.py",776],["bot/compose/limit.py",248],["bot/compose/liveask.py",93],["bot/compose/loss.py",4955],["bot/compose/missing_half.py",296],["bot/compose/moving.py",119],["bot/compose/no_eyes.py",479],["bot/compose/nottheirbest.py",117],["bot/compose/occasionplan.py",460],["bot/compose/onething.py",462],["bot/compose/only_they_know.py",563],["bot/compose/onscreen.py",504],["bot/compose/own_facts.py",198],["bot/compose/place_facts.py",138],["bot/compose/plans.py",212],["bot/compose/pleased.py",103],["bot/compose/postcode.py",392],["bot/compose/prices.py",262],["bot/compose/quantity.py",1242],["bot/compose/reaction.py",291],["bot/compose/reading.py",1778],["bot/compose/recall.py",112],["bot/compose/receiving_end.py",207],["bot/compose/remark.py",1952],["bot/compose/repair.py",418],["bot/compose/rotten_luck.py",313],["bot/compose/routes.py",538],["bot/compose/rule.py",224],["bot/compose/ruledout.py",680],["bot/compose/safetyask.py",193],["bot/compose/saidby.py",203],["bot/compose/setting_off.py",241],["bot/compose/still.py",290],["bot/compose/talkto.py",170],["bot/compose/taste.py",436],["bot/compose/the_rule_for_it.py",322],["bot/compose/the_turn_after.py",157],["bot/compose/their_day.py",436],["bot/compose/theirperson.py",225],["bot/compose/trouble.py",1203],["bot/compose/unseen.py",208],["bot/compose/urgent.py",646],["bot/compose/verdict.py",5788],["bot/compose/waited_for.py",236],["bot/compose/who.py",329],["bot/compose/who_to_see.py",307],["bot/compose/whole.py",57],["bot/compose/why.py",2072],["bot/compose/worn.py",226],["bot/knowledge/__init__.py",157],["bot/knowledge/affording.py",96],["bot/knowledge/animals.py",900],["bot/knowledge/appointments.py",285],["bot/knowledge/babyfood.py",2136],["bot/knowledge/bake_faults.py",221],["bot/knowledge/bikes.py",1096],["bot/knowledge/body.py",4450],["bot/knowledge/body_energy.py",79],["bot/knowledge/business.py",494],["bot/knowledge/cakes.py",298],["bot/knowledge/caring.py",255],["bot/knowledge/cars.py",4231],["bot/knowledge/chilli.py",267],["bot/knowledge/cleaning.py",4848],["bot/knowledge/clothing.py",4485],["bot/knowledge/comeback.py",188],["bot/knowledge/complaints.py",1066],["bot/knowledge/conditions.py",203],["bot/knowledge/consumer.py",5081],["bot/knowledge/cook_kit.py",493],["bot/knowledge/cooking.py",12280],["bot/knowledge/cooking_entries.py",11145],["bot/knowledge/cooking_gaps.py",2356],["bot/knowledge/cooking_more.py",2197],["bot/knowledge/cooking_tables.py",3171],["bot/knowledge/costs.py",1345],["bot/knowledge/creative.py",96],["bot/knowledge/crisis_replies.py",2054],["bot/knowledge/decorating.py",228],["bot/knowledge/devices.py",319],["bot/knowledge/differences.py",304],["bot/knowledge/dish_verdicts.py",437],["bot/knowledge/dishes.py",5960],["bot/knowledge/drinks.py",240],["bot/knowledge/eithers.py",474],["bot/knowledge/entertainment.py",9526],["bot/knowledge/explains.py",2943],["bot/knowledge/family.py",13174],["bot/knowledge/family_more.py",1057],["bot/knowledge/fitness.py",1669],["bot/knowledge/fitweek.py",240],["bot/knowledge/freshness.py",1127],["bot/knowledge/friends.py",1099],["bot/knowledge/fruit.py",413],["bot/knowledge/fun.py",713],["bot/knowledge/garden.py",6685],["bot/knowledge/garden_tools.py",429],["bot/knowledge/geography.py",1710],["bot/knowledge/habits.py",848],["bot/knowledge/health.py",10567],["bot/knowledge/history.py",2030],["bot/knowledge/home.py",8069],["bot/knowledge/home_craft.py",10052],["bot/knowledge/house_odds.py",438],["bot/knowledge/how_it_works.py",151],["bot/knowledge/idioms.py",465],["bot/knowledge/ingredient_swaps.py",63],["bot/knowledge/joint_time.py",131],["bot/knowledge/joints.py",556],["bot/knowledge/keeping.py",1892],["bot/knowledge/keeping_cooked.py",115],["bot/knowledge/kit_lists.py",69],["bot/knowledge/kitchen.py",146],["bot/knowledge/kitchen_odds.py",529],["bot/knowledge/kitchen_prep.py",43],["bot/knowledge/language.py",3486],["bot/knowledge/left_on.py",266],["bot/knowledge/lifeadmin.py",4448],["bot/knowledge/lookup.py",6012],["bot/knowledge/machine_door.py",90],["bot/knowledge/materials.py",666],["bot/knowledge/maths.py",5766],["bot/knowledge/meals.py",2602],["bot/knowledge/measures.py",636],["bot/knowledge/meat_in_a_card.py",72],["bot/knowledge/microwave_methods.py",80],["bot/knowledge/money.py",10232],["bot/knowledge/money_words.py",362],["bot/knowledge/music.py",398],["bot/knowledge/nature.py",855],["bot/knowledge/no_kit.py",335],["bot/knowledge/notable.py",765],["bot/knowledge/nursing.py",54],["bot/knowledge/nutrition.py",2691],["bot/knowledge/occasions.py",202],["bot/knowledge/odours.py",283],["bot/knowledge/opinions.py",1508],["bot/knowledge/panic_talk.py",211],["bot/knowledge/parking.py",45],["bot/knowledge/pasta.py",303],["bot/knowledge/pensions.py",114],["bot/knowledge/pets.py",11217],["bot/knowledge/pharmacy.py",414],["bot/knowledge/photography.py",186],["bot/knowledge/picking.py",422],["bot/knowledge/picks.py",164],["bot/knowledge/plans.py",2612],["bot/knowledge/power_heat.py",43],["bot/knowledge/pregnancy.py",1302],["bot/knowledge/prices.py",186],["bot/knowledge/quantity.py",37],["bot/knowledge/rice.py",350],["bot/knowledge/routines.py",552],["bot/knowledge/rubbish.py",750],["bot/knowledge/sauces.py",196],["bot/knowledge/schoolmaths.py",217],["bot/knowledge/science.py",11486],["bot/knowledge/seasoning.py",277],["bot/knowledge/shopping.py",4141],["bot/knowledge/sleep.py",1046],["bot/knowledge/spice_heat.py",246],["bot/knowledge/sport.py",3981],["bot/knowledge/still_ok.py",913],["bot/knowledge/study.py",4795],["bot/knowledge/swaps.py",1042],["bot/knowledge/takes.py",329],["bot/knowledge/technology.py",7770],["bot/knowledge/travel.py",11322],["bot/knowledge/units.py",3420],["bot/knowledge/vegetables.py",961],["bot/knowledge/warm_food.py",161],["bot/knowledge/whenbuy.py",403],["bot/knowledge/wording.py",2806],["bot/knowledge/work.py",8132],["bot/memory/__init__.py",1309],["bot/memory/a_year_of_birth.py",122],["bot/memory/aboutthem.py",121],["bot/memory/ack.py",6079],["bot/memory/again.py",866],["bot/memory/already.py",906],["bot/memory/amounts.py",260],["bot/memory/another.py",480],["bot/memory/asleep.py",263],["bot/memory/baking.py",107],["bot/memory/carried_on.py",289],["bot/memory/carry.py",783],["bot/memory/cause.py",672],["bot/memory/change.py",153],["bot/memory/choice.py",663],["bot/memory/company.py",340],["bot/memory/condition.py",364],["bot/memory/corrected.py",1921],["bot/memory/counts.py",882],["bot/memory/deadline.py",103],["bot/memory/decided.py",697],["bot/memory/detail.py",1182],["bot/memory/doubt.py",155],["bot/memory/enough.py",666],["bot/memory/facts.py",10397],["bot/memory/favourite.py",519],["bot/memory/filler.py",36],["bot/memory/guest.py",218],["bot/memory/healing.py",331],["bot/memory/history.py",494],["bot/memory/hope.py",144],["bot/memory/hours.py",150],["bot/memory/household.py",191],["bot/memory/kidcount.py",131],["bot/memory/limit.py",860],["bot/memory/limits.py",288],["bot/memory/lineback.py",445],["bot/memory/matters.py",308],["bot/memory/meant.py",687],["bot/memory/more.py",220],["bot/memory/named.py",2610],["bot/memory/narrow.py",2652],["bot/memory/newborn.py",90],["bot/memory/not_a_trip.py",127],["bot/memory/not_this.py",235],["bot/memory/obstacle.py",644],["bot/memory/onset.py",378],["bot/memory/outfit.py",346],["bot/memory/passing_the_time.py",210],["bot/memory/pending.py",172],["bot/memory/people.py",249],["bot/memory/pile.py",124],["bot/memory/places.py",249],["bot/memory/plan.py",468],["bot/memory/plans.py",450],["bot/memory/prefer.py",190],["bot/memory/price.py",600],["bot/memory/purse.py",173],["bot/memory/quotes.py",71],["bot/memory/rates.py",289],["bot/memory/recall.py",590],["bot/memory/reject.py",681],["bot/memory/relations.py",1766],["bot/memory/remedy.py",184],["bot/memory/reminded.py",155],["bot/memory/rename.py",108],["bot/memory/repair.py",938],["bot/memory/repeat.py",4737],["bot/memory/resize.py",379],["bot/memory/returned.py",215],["bot/memory/returns.py",286],["bot/memory/saving.py",1899],["bot/memory/setback.py",318],["bot/memory/size.py",232],["bot/memory/spent.py",685],["bot/memory/spoken.py",137],["bot/memory/stands.py",235],["bot/memory/store_lines.py",1035],["bot/memory/store_replies.py",12266],["bot/memory/stuck.py",447],["bot/memory/surname.py",321],["bot/memory/swap.py",766],["bot/memory/taking.py",409],["bot/memory/taste.py",507],["bot/memory/the_years.py",213],["bot/memory/thread.py",1786],["bot/memory/times.py",223],["bot/memory/told.py",1552],["bot/memory/too_late.py",174],["bot/memory/topics.py",7810],["bot/memory/tried.py",1665],["bot/memory/trips.py",1518],["bot/memory/warmth.py",479],["bot/memory/when.py",1694],["bot/memory/whereabout.py",140],["bot/memory/whose_body.py",76],["bot/memory/workplace.py",198],["bot/memory/years_in.py",309],["bot/modules/__init__.py",1],["bot/modules/a_count_said.py",40],["bot/modules/a_day_somebody_else_keeps.py",121],["bot/modules/aboutachild.py",39],["bot/modules/already.py",26],["bot/modules/answers.py",11236],["bot/modules/arbiter.py",4029],["bot/modules/attribute_said_again.py",603],["bot/modules/bulky.py",153],["bot/modules/carers_turn.py",422],["bot/modules/chat.py",16948],["bot/modules/done_ask.py",240],["bot/modules/doubt.py",24],["bot/modules/favourites.py",161],["bot/modules/figure_asked_again.py",40],["bot/modules/greeting.py",901],["bot/modules/grief.py",818],["bot/modules/household.py",47],["bot/modules/how_long_a_stay.py",205],["bot/modules/intent.py",38],["bot/modules/invitation.py",48],["bot/modules/judge_ask.py",131],["bot/modules/keeping.py",137],["bot/modules/kid_tally.py",28],["bot/modules/live_source.py",294],["bot/modules/live_times.py",241],["bot/modules/loss.py",140],["bot/modules/memory_store.py",5955],["bot/modules/memory_tables.py",469],["bot/modules/my_work.py",26],["bot/modules/name_taken_back.py",107],["bot/modules/night.py",29],["bot/modules/nudge.py",56],["bot/modules/recap.py",150],["bot/modules/receiving_end.py",23],["bot/modules/record_due.py",171],["bot/modules/ruled_out.py",42],["bot/modules/safety.py",10139],["bot/modules/safety_cards.py",2126],["bot/modules/safety_phrases.py",1169],["bot/modules/safety_tables.py",7231],["bot/modules/said_back.py",28],["bot/modules/saving_followup.py",288],["bot/modules/steps.py",23],["bot/modules/taking_asked_again.py",45],["bot/modules/talkto.py",140],["bot/modules/their_life.py",285],["bot/modules/their_pastime.py",358],["bot/modules/theirperson.py",31],["bot/modules/thing_i_did_with.py",194],["bot/modules/trip_date_said_again.py",88],["bot/modules/understand.py",16],["bot/modules/urgent.py",30],["bot/modules/where_it_answers.py",47],["bot/modules/who_to_see.py",40],["bot/modules/words.py",197],["bot/modules/years_in.py",46],["bot/runtime.py",120],["bot/understanding/__init__.py",1761],["bot/understanding/body.py",566],["bot/understanding/breeds.py",171],["bot/understanding/kinds.py",6582],["bot/understanding/lexicon.py",1949],["bot/understanding/normalize.py",6657],["bot/understanding/references.py",419],["bot/understanding/senses.py",178],["bot/understanding/topics.py",3744]]}

worker.js

@@ -0,0 +1,74 @@
+// PyBot in a Web Worker: Pyodide (CPython compiled to WebAssembly) runs the bot's Python, so the page stays responsive.
+// Messages in:  {type: "turn", id, text} | {type: "reset"}
+// Messages out: {type: "progress", stage, loaded?, total?} | {type: "ready", ...} | {type: "reply", id, ...} | {type: "error", id?, message}
+// A module worker: new Worker("worker.js", {type: "module"}).
+import { loadPyodide } from "https://cdn.jsdelivr.net/pyodide/v314.0.7/full/pyodide.mjs";
+
+const PYODIDE = "https://cdn.jsdelivr.net/pyodide/v314.0.7/full/";
+const BUNDLE = "pybot-3185a7a9bc50.zip";  // bot/**.pyc compiled for this Pyodide's Python, plus norm.json
+const CACHE = "pybot-bundles-v1";
+
+const post = (msg) => self.postMessage(msg);
+
+// Like transformers.js caches model files: the bundle is downloaded once, then read from Cache Storage.
+async function fetchBundle() {
+  const url = new URL(BUNDLE, self.location.href).href;
+  let cache = null;
+  try { cache = await caches.open(CACHE); } catch { /* no Cache Storage (e.g. private window): download every time */ }
+  const hit = cache && await cache.match(url).catch(() => null);
+  if (hit) {
+    post({type: "progress", stage: "cached"});
+    return new Uint8Array(await hit.arrayBuffer());
+  }
+  const res = await fetch(url);
+  if (!res.ok) throw new Error(`Could not download PyBot (HTTP ${res.status}).`);
+  const total = Number(res.headers.get("content-length")) || 0;
+  const reader = res.body.getReader(), chunks = [];
+  let loaded = 0;
+  for (;;) {
+    const {done, value} = await reader.read();
+    if (done) break;
+    chunks.push(value); loaded += value.length;
+    post({type: "progress", stage: "download", loaded, total});
+  }
+  const bytes = new Uint8Array(loaded);
+  let at = 0;
+  for (const c of chunks) { bytes.set(c, at); at += c.length; }
+  if (cache) {
+    // Keep one bundle: drop older commits' bundles, then store this one.
+    for (const req of await cache.keys()) if (req.url !== url) await cache.delete(req);
+    await cache.put(url, new Response(bytes, {headers: {"content-type": "application/zip"}})).catch(() => {});
+  }
+  return bytes;
+}
+
+let web = null;
+const ready = (async () => {
+  post({type: "progress", stage: "runtime"});
+  const [py, bundle, glue] = await Promise.all([
+    loadPyodide({indexURL: PYODIDE, env: {PYTHONHASHSEED: "0"}}),  // the eval server's hash seed, so replies match
+    fetchBundle(),
+    fetch("pybot_web.py").then((r) => { if (!r.ok) throw new Error("Could not download pybot_web.py."); return r.text(); }),
+  ]);
+  post({type: "progress", stage: "import"});
+  py.unpackArchive(bundle, "zip", {extractDir: "/pybot"});
+  py.FS.writeFile("/pybot_web.py", glue);
+  py.runPython("import sys; sys.path.insert(0, '/')");
+  web = py.pyimport("pybot_web");
+  const info = web.load().toJs({dict_converter: Object.fromEntries});
+  web.reset();
+  post({type: "ready", ...info, python: py.runPython("import sys; sys.version.split()[0]"), pyodide: py.version});
+})().catch((err) => post({type: "error", message: String(err && err.message || err)}));
+
+self.onmessage = async ({data}) => {
+  await ready;
+  if (!web) return;
+  if (data.type === "reset") { web.reset(); return; }
+  if (data.type === "turn") {
+    try {
+      post({type: "reply", id: data.id, ...JSON.parse(web.turn(data.text))});
+    } catch (err) {
+      post({type: "error", id: data.id, message: String(err && err.message || err)});
+    }
+  }
+};