Big architectural change: submitters no longer host an HTTP
endpoint. They give us (provider, model, api_key) and we run
the agent loop in-process. Task content stays in lab infra
(modulo the LLM provider API call -- that single channel is
the only data egress).
- New eval_agent.py: tool-calling agent loop for Anthropic /
OpenAI / Google / DeepSeek with shared MCP HTTP client and
fasta sequence extraction.
- Rewritten eval_dispatcher.py: dispatch_all_tasks(submission_id)
loads credentials from the row, runs the loop per task,
saves CPU scores, and ALWAYS scrubs the api_key in a finally
block.
- eval_queue.py: new submit() signature (provider/model/key/
custom_mcp_*); adds scrub_credentials() and a unique 16-char
canary token per submission.
- eval_tasks.build_task_payload: embeds canary as an HTML
comment in the task prompt for retrospective leak detection.
- Submit form: provider dropdown / model name / api_key
(password) / advanced custom MCP collapsible. Big green…
Phase C: MCP toggle (reference vs custom) + tighten rate limit to 1/month
- Submit form: replace checkbox with explicit 2-option radio
- Add inline pointer to RomeroLab/protein-design-mcp
- About 'How to submit' rewritten with both modes
- Leaderboard MCP column: badges for both reference and custom
- eval_queue.MAX_SUBMISSIONS_PER_MONTH: 2 -> 1 (judge cost guard)
