lewtun HF Staff Codex commited on
Commit
85bc876
·
1 Parent(s): d99aaa1

Move progress chart before the PyBot introduction to the demo

Browse files
Files changed (1) hide show
  1. site/index.html +10 -6
site/index.html CHANGED
@@ -41,7 +41,7 @@
41
  .sw-win { background:var(--accent); border-width:2px !important; } .sw-src { background:var(--ink); } .sw-used { background:#8a8a8a; }
42
  .sw-prop { background:var(--paper); border-width:2px !important; } .sw-off { background:var(--off); border-color:var(--faint) !important; }
43
  .foot { font:13px var(--mono); color:var(--mute); margin-top:14px; } .foot a { color:inherit; }
44
- .progress { margin-top:48px; border-top:1px solid var(--ink); padding-top:24px; }
45
  .progress h2 { font:400 32px/1.2 var(--lead); margin:0 0 8px; }
46
  .progress p { max-width:680px; }
47
  #progress-chart { width:100%; height:460px; }
@@ -60,6 +60,15 @@
60
  <div class="text">
61
  <p>LLMs drive today’s chatbots and agents, but chatbots long predate them. Sixty years ago, <a href="https://en.wikipedia.org/wiki/ELIZA" target="_blank" rel="noopener">ELIZA</a> held conversations by matching patterns in what the user typed and answering with predefined templates. It could seem insightful, but its repertoire was narrow, and expanding it meant writing more rules by hand. How far could a rule-based chatbot go if a swarm of agents did that work instead?</p>
62
  <p>We challenged a swarm of over 1,000 agents to build a chatbot entirely in Python: no weights, no training, just if-else branches, regex patterns and lookup tables. The agents wrote and evaluated over 5,000 versions, and the chatbot’s win rate on <a href="https://huggingface.co/datasets/HuggingFaceSwarms/SmolTalkBench" target="_blank" rel="noopener">SmolTalkBench</a>, where an LLM judge compares each reply with a reference answer, climbed from 0% to 26%. That puts it on par with SmolLM-360M-Instruct, which just a few years ago was state of the art for models under 1B parameters.</p>
 
 
 
 
 
 
 
 
 
63
  <p>The result is PyBot: around 500k lines of code that can answer everyday questions, backed by 550k lines of tests the agents wrote themselves. Try it below: each reply shows which of PyBot’s modules produced it, and you can click any file to read the code behind it.</p>
64
  </div>
65
 
@@ -81,11 +90,6 @@
81
  </div>
82
  </section>
83
  <p class="foot">PyBot at commit <a id="commit" target="_blank" rel="noopener"></a>. Click a file to open it on the Hub.</p>
84
- <section class="progress" aria-labelledby="progress-title">
85
- <h2 id="progress-title">Programming to chat, one commit at a time</h2>
86
- <div id="progress-chart" role="img" aria-label="PyBot win rate in orange on the left axis and Python lines of code in blue on the right axis, over evaluated revisions, with three SmolLM baselines"></div>
87
- <p class="foot" id="progress-status" aria-live="polite">Loading evaluation history…</p>
88
- </section>
89
  </main>
90
  <script>/**
91
  * plotly.js v2.35.2
 
41
  .sw-win { background:var(--accent); border-width:2px !important; } .sw-src { background:var(--ink); } .sw-used { background:#8a8a8a; }
42
  .sw-prop { background:var(--paper); border-width:2px !important; } .sw-off { background:var(--off); border-color:var(--faint) !important; }
43
  .foot { font:13px var(--mono); color:var(--mute); margin-top:14px; } .foot a { color:inherit; }
44
+ .progress { margin:48px 0 32px; border-top:1px solid var(--ink); padding-top:24px; }
45
  .progress h2 { font:400 32px/1.2 var(--lead); margin:0 0 8px; }
46
  .progress p { max-width:680px; }
47
  #progress-chart { width:100%; height:460px; }
 
60
  <div class="text">
61
  <p>LLMs drive today’s chatbots and agents, but chatbots long predate them. Sixty years ago, <a href="https://en.wikipedia.org/wiki/ELIZA" target="_blank" rel="noopener">ELIZA</a> held conversations by matching patterns in what the user typed and answering with predefined templates. It could seem insightful, but its repertoire was narrow, and expanding it meant writing more rules by hand. How far could a rule-based chatbot go if a swarm of agents did that work instead?</p>
62
  <p>We challenged a swarm of over 1,000 agents to build a chatbot entirely in Python: no weights, no training, just if-else branches, regex patterns and lookup tables. The agents wrote and evaluated over 5,000 versions, and the chatbot’s win rate on <a href="https://huggingface.co/datasets/HuggingFaceSwarms/SmolTalkBench" target="_blank" rel="noopener">SmolTalkBench</a>, where an LLM judge compares each reply with a reference answer, climbed from 0% to 26%. That puts it on par with SmolLM-360M-Instruct, which just a few years ago was state of the art for models under 1B parameters.</p>
63
+ </div>
64
+
65
+ <section class="progress" aria-labelledby="progress-title">
66
+ <h2 id="progress-title">Programming to chat, one commit at a time</h2>
67
+ <div id="progress-chart" role="img" aria-label="PyBot win rate in orange on the left axis and Python lines of code in blue on the right axis, over evaluated revisions, with three SmolLM baselines"></div>
68
+ <p class="foot" id="progress-status" aria-live="polite">Loading evaluation history…</p>
69
+ </section>
70
+
71
+ <div class="text">
72
  <p>The result is PyBot: around 500k lines of code that can answer everyday questions, backed by 550k lines of tests the agents wrote themselves. Try it below: each reply shows which of PyBot’s modules produced it, and you can click any file to read the code behind it.</p>
73
  </div>
74
 
 
90
  </div>
91
  </section>
92
  <p class="foot">PyBot at commit <a id="commit" target="_blank" rel="noopener"></a>. Click a file to open it on the Hub.</p>
 
 
 
 
 
93
  </main>
94
  <script>/**
95
  * plotly.js v2.35.2