Spaces:
Running
Running
Commit History
-
Ignore .serena/ tool metadata directory
e0db8f2 -
Sync leaderboard: EOG + EVA, GitHub README 2026-07-23
eog: old: Claude Fable 5 (with fallback)=51.1, Gemini 3.5 Flash=50.1, GPT-5.5 (xhigh)=46.6, Kimi K3=45.3, Qwen3.7 Max=45 new: Claude Fable 5 (with fallback)=51.1, Gemini 3.5 Flash=50.1, Muse Spark 1.1 (xhigh)=47.2, GPT-5.5 (xhigh)=46.6, Kimi K3=45.39268ae9
-
Sync leaderboard: EOG + EVA, GitHub README 2026-07-17
eog: old: Claude Fable 5 (with fallback)=51.1, Gemini 3.5 Flash=50.1, GPT-5.5 (xhigh)=46.6, Qwen3.7 Max=45, Claude Sonnet 5 (max)=44.7 new: Claude Fable 5 (with fallback)=51.1, Gemini 3.5 Flash=50.1, GPT-5.5 (xhigh)=46.6, Kimi K3=45.3, Qwen3.7 Max=45aec8169
-
Document promote.mjs as the one-command staging->prod promotion
Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>5dd79b5 -
Add guarded staging->prod promote command
promote.mjs performs the two promotion pushes as one interactive, gated action: HF prod Space (fast-forward only) then the GitHub README publish. Refuses unless on main, worktree clean, local main exactly in sync with origin/main, prod/main a fast-forward ancestor, and README.md prod-canonical (guards against shipping staging title/color to prod). Requires a typed "promote" confirmation unless --yes; --dry-run previews both pushes. Never runs from the daily sync. Pure guards live in lib/promote.mjs with tests. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>f28f7a1 -
Wire GitHub README snapshot into sync + prod-promotion flow
Sync now regenerates github-readme/README.md whenever leaderboard data changes and includes it in the staging commit, so the GitHub README is reviewable alongside the HF staging page (never touches prod/GitHub). Adds render-readme.mjs --publish (+ --dry-run) to clone ServiceNow/NOWAI-Bench, copy the README+assets bundle, and push to main — the GitHub half of a staging->prod promotion. Documents the two-push promote flow (HF prod + GitHub README) and the prod-canonical README metadata step in SKILL.md. Adds a regression test that the checked-in snapshot never drifts from data. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>6b03ddb -
Add GitHub README generator for prod-promotion snapshots
Renders the public ServiceNow/NOWAI-Bench README leaderboard block (EOG top 5, EVA-Accuracy/Experience top 3) from data/leaderboard.json between markers, so prod-promotion snapshots stay consistent with the synced data while prose/links remain hand-maintained. Includes the publishable README bundle (reformatted prose, per-benchmark repo/page/paper/dataset/leaderboard links, centered logo) and tests. Publish to GitHub stays a manual step. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>504b178
-
Sync leaderboard: EOG + EVA 2026-07-10
eog: old: Gemini 3.5 Flash (High)=46, Claude Opus 4.6=44.6, GPT-5.5 (High)=41.2, Claude Opus 4.7 (High)=41.2, Claude Sonnet 4.6=40.4 new: Claude Fable 5 (with fallback)=51.1, Gemini 3.5 Flash=50.1, GPT-5.5 (xhigh)=46.6, Qwen3.7 Max=45, Claude Sonnet 5 (max)=44.7cc3c494 -
Repoint EOG sync to live artificialanalysis.ai data
EOG's own script.js source was frozen/abandoned after the project relocated to Artificial Analysis. Now sourcing directly from AA's live evaluation page by parsing the embedded schema.org Dataset (application/ld+json) block, instead of the stale GitHub-hosted script.js. - lib/transform.mjs: parseEogAA() replaces parseEogData(); scales AA's 0-1 scores to our existing 0-100 display convention. Dropped the unused `type` row field the old script.js format provided. - sync.mjs: EOG_URL -> AA eval page. Removed the EOG relocation probe — a format/parse failure on the live page already signals drift, so the separate landing-page redirect check is now redundant for EOG (EVA's probe is unchanged). - lib/healthcheck.mjs: dropped now-unused EOG_HOME. - index.html: EOG leaderboard links now point at the AA eval page. Verified live: top model is now Claude Fable 5 (51.1%), matching AA's page, vs. the previously frozen Gemini 3.5 Flash (46.0%). Co-Authored-By: Claude Sonnet 5 <noreply@a…b15891e -
Add per-source health-check + resilient sync for EOG/EVA targets
EnterpriseOps-Gym's landing page now redirects off-domain to artificialanalysis.ai, but the raw script.js we scrape still returns 200 and parses fine — it's just frozen/abandoned, so the sync stayed silent while our data went stale. - lib/healthcheck.mjs: detectRelocation() flags a landing page that now redirects off its own domain (meta refresh / canonical / JS redirect). - lib/transform.mjs: buildBoardsResilient() builds EOG/EVA boards independently and carries forward the last-good board when one source fails, instead of one bad source blocking both. - sync.mjs: fetches both sources + both landing pages via allSettled so one dead URL can't kill the other; posts a distinct "⚠️ Source content changed" Teams card per unhealthy source; only hard-fails when a source has no fallback to carry forward. Verified live: posted a real EOG relocation warning to Teams, and fault-injected an EVA 404 to confirm EOG still syncs independently. Co-Authored-By: Claude Sonnet 5 <noreply@a…da3bafe
-
fix: restore prod-canonical README metadata before prod push
7910cb8 -
fix: update AGENTS.md with recent context and completion status for blocked-sync Teams notifications feature
a836591 -
fix: update AGENTS.md with recent context and completion status for blocked-sync Teams notifications feature
00f3d9e -
fix: update AGENTS.md with recent context and stats adjustments
1d3bc71 -
fix: rename blocked sync reason to out-of-sync
Co-Authored-By: Codex <noreply@anthropic.com>3dfe2ca -
fix: align blocked sync warnings with exact-sync precheck
Co-Authored-By: Codex <noreply@anthropic.com>e0d4a65 -
fix: handle symlinked notify-blocked cli entry
08db0a7 -
feat: add blocked sync Teams notifier cli
fdb15c0 -
fix: normalize blocked facts and reuse subtitle helper
447d2ec -
feat: add blocked sync notification builder
16ff884 -
Sync leaderboard: header → 2026Q3 2026-07-01
1759d72 -
chore: ignore local worktree directory
1039def -
docs: add blocked sync Teams warning plan
dce9e9f -
docs: add blocked sync Teams warning design
93824b1
-
Add production push utilities and inline help comments to Makefile
e82a5f2 -
Add agents file and skills
cf2f7da -
Sync leaderboard: EOG + EVA 2026-06-30
evaAccuracy: old: Scribe v2.2 Realtime + GPT-5.4 + Eleven Flash v2 (ElevenAgents)=0.67, Grok Voice Think Fast 1.0=0.59, Nova 3 + GPT-5.4 + Sonic 3=0.5 new: Scribe v2.2 Realtime + GPT-5.4 + Eleven Flash v2 (ElevenAgents)=0.67, Universal 3.5 Pro + GPT-5.4 + Sonic 3.5=0.6, Grok Voice Think Fast 1.0=0.5993f60f8
-
Update metadata with huggingface_hub
Brad ClevelandVerified signature9d052fd -
Add quarter label auto-update to leaderboard sync
Sync now detects when index.html header quarter (e.g. 2026Q2) is behind the current calendar quarter and updates it as part of the same commit+push flow. Quarter-only runs get a distinct Teams notification title and a note that no ranking changes were involved. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>5661ed6
-
Add Makefile and update SKILL.md with logging + heartbeat docs
- Makefile: sync (--push), check (--dry-run), logs (tail sync log) - SKILL.md: document Friday heartbeat card behavior; add Logs section Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>0bc5d1a -
Add Friday heartbeat Teams notification when no leaderboard changes
On Fridays when the sync finds no changes, post an info card to Teams showing the last date leaderboard.json was actually changed. Confirms the automation is running even during quiet weeks. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>5e2010f
-
Add nowai-bench-v0.zip to .gitignore
216c813 -
Add Teams Adaptive Card notifications to leaderboard sync
sync.mjs now posts a styled Adaptive Card (via new lib/teams.mjs) on push: green success with an Open-staging button, red failure, amber sanity-abort. Targets a Power Automate "Workflows" webhook (*.powerplatform.com), so the payload is an Adaptive Card in a message/attachments envelope rather than a legacy MessageCard. Header rendered as a bleed/emphasis band over a fact table for connector-card-style hierarchy. No card on "no changes". TEAMS_WEBHOOK_URL is read from the gitignored .env; .env.example documents it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>ca8c06b
-
Sync leaderboard: EOG + EVA 2026-06-10
evaAccuracy: old: Scribe v2.2 Realtime + GPT-5.4 + Eleven Flash v2 (ElevenAgents)=0.67, Nova 3 + GPT-5.4 + Sonic 3=0.5, GPT Realtime 2=0.5 new: Scribe v2.2 Realtime + GPT-5.4 + Eleven Flash v2 (ElevenAgents)=0.67, Grok Voice Think Fast 1.0=0.59, Nova 3 + GPT-5.4 + Sonic 3=0.5 evaExperience: old: Scribe v2.2 Realtime + Claude Haiku 4.5 + Eleven Flash v2 (ElevenAgents)=0.82, Gemini 3.1 Flash Live=0.59, GPT Realtime 1.5=0.57 new: Scribe v2.2 Realtime + Claude Haiku 4.5 + Eleven Flash v2 (ElevenAgents)=0.82, Gemini 3.1 Flash Live=0.59, Grok Voice Think Fast 1.0=0.57ba5565d
-
Restore README.md to prod-canonical values
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>bcfd770 -
Merge remote-tracking branch 'origin/main'
1c9f360 -
Add title animation demo with hover effects and accessibility considerations
df7f292 -
Update metadata with huggingface_hub
Brad ClevelandVerified signaturee539f37 -
Update Space colors: prod indigo, staging red
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>9a74a7d -
Update prod Space colorFrom to purple
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>ec6ce00 -
Restore prod README.md; update push-staging.sh to force-push
metadata_update() creates a commit on staging remote; force-push overwrites it each run so it never needs merging locally. README.md remains prod-canonical (NowAI-Bench / pink) in git. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>a3edfab -
Merge remote-tracking branch 'origin/main'
d1cf091 -
Add push-staging.sh: push + patch staging Space metadata
Keeps README.md prod-canonical in git. After each push to origin, calls huggingface_hub.metadata_update() to set staging title/color independently — no more merge conflicts on prod promotion. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>d6b0a9e -
Update metadata with huggingface_hub
Brad ClevelandVerified signatureb55658f -
Merge prod README: keep NowAI-Bench title for prod Space
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>1d52448 -
Update leaderboard: EVA pass@1 ranking, top-3 cards, footer alignment
- Switch EVA boards to Pass@1 metric (EVA-A_pass / EVA-X_pass) instead of Mean - Show top 3 models for EVA cards (down from 5); EOG unchanged at 5 - Remove v1.1 radar view placeholder text under v1.0 badge - Pin card footers to bottom via flex-col + mt-auto for aligned HR/link - Add pb-6 to EOG runners and EVA content wrapper for consistent spacing - Update transform.mjs and tests to match new metric keys and slice(0,3) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>85f0b3a -
Merge leaderboard auto-sync
- sync-nowai-leaderboard skill: pure transform lib (14 tests), sync.mjs (check/apply/push), SKILL.md - staging-only push guard (refuses unless origin=NowAI-Bench-Staging), fetch timeout, fail-closed gates - data/leaderboard.json generated from live upstream (EOG + EVA) - index.html renders from JSON with static fallback - bars show absolute % of metric max Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>09a2df7 -
Sync leaderboard: EOG + EVA 2026-06-03
30d00f2 -
feat: bars show absolute % of metric max (EOG/100, EVA/1.0)
b384f8e -
fix: verify staging remote before write/commit on --push (no dangling commit)
84f5427