Spaces:
Running
Running
Commit History
14 days ago
-
Update app.py Running
Verified signatureb41db79 -
Update README.md
Verified signature07c1028 -
Update app.py
Verified signature90c6559
-
leaderboard: anonymize Justin1233 references + force English UI
xzhou10Three independent fixes: 1. Drop Justin1233/RobustBench-TC URL and run_eval.py from the header markdown — both leaked the contributor handle and project repo, violating double-blind review. Replaced with a generic 'identifiers withheld during double-blind review' message. 2. gradio's gr.File widget renders the upload placeholder in the visitor's browser locale (zh-CN by default for many users); pin GRADIO_DEFAULT_LANG=en at launch so reviewers always see the English string instead of '将文件拖放到此处或点击上传'. 3. Rename the post-submit Dataframe label from 'leaderboard (refreshed)' to 'Updated leaderboard after this submission' and pre-populate it with the current leaderboard so the table is no longer empty before any submission has been made.b4ffed9
-
leaderboard: include o4-mini full eval (pert=0.502, clean=0.708)
xzhou10o4-mini full eval landed at /projects/bfdz/xzhou10/robustbench/results/openai__o4-mini. Initial scores.json had a stale overall.n (4915 vs canonical 3721) because the aggregator over-counted clean cells from the 6 transition_*/scores.json sub-aggregations; fixed by recomputing overall+by_mdp from by_perturbation. Re-seed leaderboard with the corrected o4-mini row at 21st position.143932c -
leaderboard: drop redundant N column
xzhou10e1b2278 -
leaderboard: add ToolRL-DR-Full/Mixed + o4-mini stub, clarify SFT label
xzhou10Re-seed leaderboard.csv with the two trained checkpoints (ToolRL-DR-Full and ToolRL-DR-Mixed) included. SFT-Clean-4k-Qwen2.5-3B now labeled as '(reproduced baseline)' rather than '(ours)' to make clear that SFT-Clean is a baseline we reproduced from public ToolRL data, not a method contribution. The two ToolRL-DR variants are the only checkpoints we propose as contributions. o4-mini entry is added to the seed list but skipped at seed time because no full scores.json exists yet (only a 5-sample smoke test). Run the full o4-mini eval and rerun seed_with_18_models.py to populate the row.8c2a72f
-
feat: replace 'Overall Acc.' with 'Pert. Acc.' (clean-excluded), explain N
xzhou10User feedback on the live leaderboard: 1. 'n' column was opaque — added a column-guide markdown above the table that defines N (eval), Pert. Acc., Clean explicitly. Also renamed the column to 'N (eval)' for clarity. 2. The headline accuracy was a clean+perturbed average, which inflates the number for any model with high clean accuracy. Replace it with a perturbed-only weighted average (sum of correct/n across all 22 perturbation cells, clean excluded). Now Clean − Pert. Acc. is the sim-to-real gap that the benchmark exists to measure. score_submission(), append_entry(), seed_with_18_models.py all updated to compute and store the perturbed-only Pert. Acc.; load_leaderboard() includes a backward-compat rename for old CSVs (n -> N (eval), Overall Acc. -> Pert. Acc.) so existing rows keep working.bc37764 -
fix: pin starlette<0.40 for gradio 4.44 TemplateResponse compat
xzhou10starlette>=0.40 changed TemplateResponse signature from (name: str, context: dict) to (request, name, context). gradio 4.44 still calls the old form, so the new starlette interprets the second positional arg (the dict context) as 'name', and jinja2 then crashes at cache lookup with 'TypeError: unhashable type: dict' when looking up that dict as a template name. Pinning starlette<0.40 keeps the old signature path working until we upgrade gradio to 5.x (which uses the new signature).cf77118 -
fix: monkey-patch gradio_client get_type for bool additionalProperties
xzhou10show_api=False doesn't help because the schema is generated at app import time (during get_blocks().get_api_info()) — before launch(). The crash: File gradio_client/utils.py:863, in get_type if 'const' in schema: TypeError: argument of type 'bool' is not iterable happens when a tool schema has additionalProperties: false (a bool, not a dict). gradio 4.44 + bundled gradio_client 1.3.0 has this bug; it's fixed in gradio_client 1.5.x but gradio 4.44 pins client to 1.3. Workaround: patch get_type and _json_schema_to_python_type to fall back to 'Any' when given a non-dict, before importing gradio. This is a one-process fix that costs us nothing because we don't expose a programmatic client to consumers.bc9ad31 -
fix: explicit launch config to bypass gradio_client schema crash + bind to 0.0.0.0
xzhou10show_api=False: gradio 4.44 + the bundled gradio_client 0.x crashes when generating the API schema for nested data structures with dict-valued additionalProperties ('TypeError: argument of type bool is not iterable'). We don't expose a programmatic client API, so dropping the schema endpoint is harmless. server_name=0.0.0.0 + server_port=7860: HF Spaces require explicit bind to all interfaces and the canonical port; without it gradio defaults to localhost which the Space proxy cannot reach (the 'When localhost is not accessible, a shareable link must be created' error in the Space build log).a5730fa -
fix: pin huggingface_hub<1.0 (gradio 4.44 imports removed HfFolder)
xzhou10huggingface_hub 1.x removed HfFolder which gradio 4.x oauth module imports unconditionally. Pin hub to 0.x series until we upgrade gradio to >=5 (which would also let us drop the python_version: 3.11 pin since gradio 5+ dropped pydub).147598c -
fix: drop pydub pin (0.25.2 doesn't exist on PyPI)
xzhou10The python_version: 3.11 frontmatter pin alone is enough — Python 3.11 ships the stdlib audioop module that gradio<5 / pydub need. The pydub>=0.25.2 pin was a mistake (max published version is 0.25.1) and caused the build to fail with 'No matching distribution found'.6bc74a6 -
fix: pin python_version=3.11 + audioop-lts shim for HF Space boot
xzhou10Python 3.13 removed the stdlib audioop module that pydub<0.25.2 imports during gradio startup. Two-layer fix: 1. README.md frontmatter pins python_version: "3.11" so HF Space provisions a Python that still ships audioop. 2. requirements.txt adds pydub>=0.25.2 (which ships its own audioop fallback) and audioop-lts as a guard for any Python>=3.13 fallback.224e834
-
Add HF Space metadata header
willchow661519bbf
-
Init RobustBench-TC leaderboard MVP with 18 seeded models
xzhou1037a1a73