Commit History
2 days ago
-
2.0.7: the names layer leaves role nouns cut short or glued to a word out
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>4b6a8b0 -
2.0.6: a model span that stops inside a number grows over the whole number
Verified signatureCo-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>ce25bc0
3 days ago
-
2.0.5: the names layer leaves more role nouns, and a number right after one, out
Verified signatureCo-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>f34fc54
4 days ago
-
2.0.4: the names layer leaves procedural role nouns out
Verified signatureb21609b
7 days ago
-
2.0.3: values cut or preceded by a placeholder
Verified signatureee88d62 -
2.0.2: a rule's label wins on its characters
Verified signaturec6b0914
19 days ago
-
2.0.1: a [PERSON] placeholder ends an other-number label
Same weights, threshold and API. Rules SHA c1b924a8: a Kod/NIP/REGON/PESEL/KRS/ISBN label no longer reaches a number across [PERSON], so re-scrubbing names-on output masks a phone after a masked name. Phone cues still cross [PERSON]. A first scrub of raw text gives 2.0.0's rule spans, names on or off. Gates vs 2.0.0 rules: 841-dev + invented 0/845 changed; names-on re-scrub 0/841; bench replay 12/12 cohorts equal; full Dynaword corpus 0/4,144,908 changed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>960c515 -
2.0.0
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>be6377b
20 days ago
-
2.0.0: opt-in FastPDN person names (names/, CC-BY-4.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>dc40291 -
2.0.0: ranked union, person spans
PHONE > PII > PERSON on overlap; person token ranges expand to whole words and merge into spans; scrub_spans counts person. Names off: output unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>7ab6996 -
2.0.0: [PHONE] replaces [Telefon]; [Telefon] stays a rules boundary
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>75130f0 -
1.2.0: phone policy v3 at mask time
Same weights, threshold and API. Rules 1.1.3 (candidate 73c185f7, comments reworded): one span per number of 7+ digits, standalone numbers under 7 digits not masked. Model phone spans are cut by the same _phone_spans before the union (nergal.model_keep). Card basis moves to gold restated to policy v3 plus gold_check_v4 (315 spans). 841-dev, 1.1.2 -> 1.2.0 on that gold: whole 303/315 unchanged, rules FP 83 -> 24, union FP 333 -> 80. No whole value lost in any benchmark cohort. Dynaword reviewed delta: phone whole 57 -> 65 of 70, 0 lost. Full Dynaword: masked characters -10,766 (0.21%). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>bbd1d02
24 days ago
-
1.1.2: email boundary rules
Rules only; same weights, threshold and API. An address match ends before a glued www. host or at a capital glued onto a lowercase domain ending; @-mention lists are not addresses. Card basis moves to amended gold (gold_check_v2), 1.1.1 restated on it. 841-dev union FP 133 -> 80, rules FP 108 -> 24, whole 326/354 unchanged. Full Dynaword: 165 documents change, no mask dropped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>e0caa8b -
1.1.1: grouped national phones without a label
Rules only; same weights, threshold and API. 841-dev 326/354 whole (324), 0 new false characters; cue-less phone test 66/111 phones (37), scored once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>4464d47
26 days ago
-
1.1.0: batch API (predict_many, scrub_many), opt-in float16 (#2)
- Prepare 1.1.0: batch API (predict_many, scrub_many), opt-in float16 (fdc79fca5b4800a95c0eddeb2c18f9fdd1dd59e3) - 1.1.0 card: throughput in TL;DR (7b7f961957267e4439c13c67c51b634fd7fbc76b)78cc85a -
1.0.3: registry and label-note rules, placeholder and card fixes (#1)
- Prepare 1.0.3: registry and label-note rules, placeholder and card fixes (6eee92ef79115a4a40eedb615aacf457bdd54868)2bf5c25
-
Fix public tokenizer batch shape before 1.0.2 release
510b295 -
Release 1.0.2: conservative labelled country-area phone fix
15eb2b4 -
Release 1.0.1: prefix-only glued-email trim.
Same epoch-5 weights; 841-dev union FP 133→123 with no gold loss. Co-authored-by: Cursor <cursoragent@cursor.com>7ce967b
-
Add hybrid PII scripts: nergal.py and frozen scrub_pii.py
Verified signature97bea30