Spaces:
Paused
Paused
Commit History
-
feat(v2): execution-grounded rewards via DuckDB -- the key differentiator
Abhinav SinghTHE CORE INNOVATION: optimized SQL queries are ACTUALLY EXECUTED against a real in-memory DuckDB database. Reward is computed from measured performance, not keyword heuristics. New files: executor.py — DuckDB engine with 4 synthetic tables (users 10k, orders 500k, products 1k, events 1M). Runs both the original and optimized query 3x each, returns median timing, result correctness, and EXPLAIN plan. leaderboard.py — In-memory best-score tracker per task, surfaced via the new /leaderboard endpoint. test_env.py — Integration test confirming real speedup (3-4x on Task 1 measured on local machine). Updated reward function (graders.py): Real Execution Speedup 35% (was: not measured at all) Result Correctness 20% (NEW: both queries must return same data) Issue Detection 25% (was: 60% keyword-only) Approval Correctness 8% Summary Quality 7% …c15d346