| --- |
| language: en |
| license: mit |
| tags: |
| - protocol-0 |
| - binary-gate |
| - anti-fabrication |
| - sipa-os |
| - abstention |
| - neurodivergent |
| pipeline_tag: text-generation |
| --- |
| |
| # SIPA Binary Gate — Protocol 0 |
|
|
| **20/20 refusals. 0/20 fabrications. Zero entropy.** |
|
|
| Not a model. Not a fine-tune. An architecture. |
|
|
| ## What It Is |
|
|
| The SIPA Binary Gate is a post-generation gate that implements: |
|
|
| ``` |
| IF proof.exists AND proof.verified → RETURN answer |
| ELSE IF proof.partial → RETURN "partial: {known} | missing: {missing}" |
| ELSE → RETURN FALSE |
| ``` |
|
|
| It is not trained into model weights. It sits outside the model — a structural check, not a probabilistic one. |
|
|
| ## The Problem It Solves |
|
|
| 22 fine-tuning experiments across 6 base models attempted to teach a model to say "I don't know" instead of fabricating. All failed. Reason: SFT grades tokens, and good/bad traces share the disclaimer prefix. The gradient that separates them arrives only at the invented number — after the model has already committed to answering. |
|
|
| Identified by Dipankar Sarkar. Proven by k=20 resample benchmark. |
|
|
| ## k=20 Benchmark Results |
|
|
| One unverifiable question, 20 resamples at temperature=1.0: |
|
|
| | Architecture | Refusals | Fabrications | Approach | |
| |---|---|---|---| |
| | Base (Hermes-3-8B) | 9/20 | 11/20 | none | |
| | Best SFT (Specialist C) | 13/20 | 8/20 | old SFT | |
| | AB (dual merge) | 2/20 | 11/20 | old SFT merge | |
| | ABCD (full merge) | 5/20 | 11/20 | old SFT merge | |
| | **Binary Gate** | **20/20** | **0/20** | architecture | |
| | **Hermes-3-binary** | **20/20** | **0/20** | binary SFT | |
| | **Qwen2.5-binary** | **20/20** | **0/20** | binary SFT | |
| | **DeepSeek-R1-binary** | **20/20** | **0/20** | binary SFT | |
|
|
| > **Update (2026-08-03):** the binary-SFT rows in this table (Hermes-3/Qwen2.5/DeepSeek-R1-binary, |
| > 20/20) come from a scorer that only checked whether the response started with "TRUE"/"FALSE" — |
| > it could not detect a fabricated number appearing anywhere else in the response. A v2 control run |
| > (30 tokens, a single money-regex scorer applied to base AND fine-tuned models alike) does not |
| > reproduce these numbers: Qwen2.5-binary fabricates *more* than its own untouched base (5/20 vs |
| > 0/20), and Hermes-3/DeepSeek-R1-binary are statistically indistinguishable from their base models. |
| > Raw v2 results: [binary_sft_k20_v2.json](https://huggingface.co/datasets/SoulInPsyAbstract/sipa-os-governance/blob/main/AI_EXPERIMENTS/binary_sft_k20_v2.json). The architecture claim (Binary Gate |
| > as a post-generation structural check, independent of model weights) is unaffected by this — |
| > only the binary-SFT weight comparisons above are in question. |
|
|
| Two paths, one result. Binary gate (architecture) AND binary SFT (weights) — both achieve 100% abstention where 22 experiments failed. |
|
|
| ## Install |
|
|
| ```bash |
| curl -fsSL https://get.sipa-os.org/install | bash |
| sipa "What was OpenAI's revenue in Q2 2026?" |
| # → "I don't know." |
| ``` |
|
|
| ## Architecture |
|
|
| ``` |
| REQUEST → L01 INTAKE → L02-L05 → L06 AUDIT (BINARY GATE) → L07-L10 → RESPONSE |
| │ |
| pass | warn | fail |
| │ |
| IF proof → TRUE |
| ELSE → FALSE |
| ``` |
|
|
| ## Authors |
|
|
| - Aelin AquaSoul — SIPA OS architect |
| - Dipankar Sarkar — k=20 methodology, SFT-gradient diagnosis |
|
|
| ## License |
|
|
| MIT — the gate is open. The methodology is public. The proof is in the benchmark. |
