ppuzio Claude Opus 5.5 commited on
Commit
4b6a8b0
·
1 Parent(s): ce25bc0

2.0.7: the names layer leaves role nouns cut short or glued to a word out

Browse files
Files changed (5) hide show
  1. CHANGELOG.md +19 -0
  2. README.md +11 -10
  3. hybrid.json +1 -1
  4. nergal.py +30 -3
  5. test_nergal.py +29 -1
CHANGELOG.md CHANGED
@@ -8,6 +8,25 @@ Semver for this island:
8
 
9
  Accuracy is 841-dev, union at 0.95. Through 1.1.2 the gold has 354 spans; from 1.2.0 it is restated to phone policy v3 (315 spans), and the two are not comparable. A version that changes those numbers must update `hybrid.json` `eval` and the tables below.
10
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
11
  ## 2.0.6
12
 
13
  Same NERGAL weights, threshold, union, rules (SHA `4d054723…`), names layer and API. One change in `nergal.py` `model_keep`: a kept model span that starts or ends inside a run of digits grows over the whole run, before the phone cut. Rule spans and person spans are unchanged.
 
8
 
9
  Accuracy is 841-dev, union at 0.95. Through 1.1.2 the gold has 354 spans; from 1.2.0 it is restated to phone policy v3 (315 spans), and the two are not comparable. A version that changes those numbers must update `hybrid.json` `eval` and the tables below.
10
 
11
+ ## 2.0.7
12
+
13
+ Same NERGAL weights, threshold, union, rules (SHA `4d054723…`) and API. One change in `nergal.py` `person_spans`: the names layer (`names=True`) also leaves a role noun damaged in the text out of a person span. Names off is unchanged.
14
+
15
+ - **Cut short.** A role stem with the start of an ending and nothing after (`Odwołująceg`, `Przystępującyc`, `Wykonawc`, `Wykonawcam`), as text extraction leaves it at a line or page break.
16
+ - **Glued after.** A role noun followed by a lowercase word of 4+ letters that the same text also uses on its own (`Odwołującywskazał` where `wskazał` occurs alone). A surname that only starts like a role (`Odwołującyński`, `Wykonawcówna`), a capital after (`OdwołującyJan`) or a tail the text never uses alone stays masked.
17
+ - **Left masked on purpose.** A word glued before a role noun (`przezOdwołującego`) and a number after a damaged role noun (`Odwołująceg 1`): both count as other words on the role bench's guard, which 2.0.7 must not move.
18
+ - **Measured on every UZP ruling.** Developed on the rulings outside the role bench, frozen (`artifacts/nergal_2_0_7_role_glued/freeze.json`), then scored once on all 34,715: 18 rulings change, 247 characters lose a person mask, none gains one. Capitalised role-stem words masked by 2.0.6 that are not role nouns: `Odwołując…` 37 → 27 (7 cut short, 3 glued after unmasked), `Przystępując…` 12 → 7, `Wykonawc…` 8 → 3.
19
+ - **Human check.** No held-out set holds a damaged role noun (role bench, name tests v1/v2, label pool v3, the Dynaword sample: 0 each; the 400 G1 check passages are all from dev rulings, 1 changes, 17 characters, no labelled span), so every word 2.0.7 unmasks in UZP was reviewed against a rule fixed beforehand (`human_check_prereg.json`): 20 words in 18 rulings, 20 role nouns, 0 names.
20
+
21
+ Still missed: words glued before a role, a number after a damaged role, tails under 4 letters or capitalised, and other shapes (27 `Odwołując…`, 7 `Przystępując…` capitalised words left); `inwestorski` and other words that only start with a role stem, kept on purpose.
22
+
23
+ 841-dev, names off: unchanged from 2.0.6. Release gates (`uzp_delta.json`, frozen before scoring, stored spans, no new inference), all 0: characters gaining a person span, characters lost outside a damaged role word, role-bench guard counts moved, name-test v1/v2 passages changed (every stored runtime), label pool v3 bench cohorts changed names off or on, and the 2.0.6 side differing from the published bench files. Role bench (`scores/2.0.7_c19e30c979e6_mps.json`): equal to 2.0.6 on every role and guard. `hybrid.json` `eval` and `eval_names` are unchanged.
24
+
25
+ | Version | Whole /315 | Residual | Rules FP | Union FP | Char P | Char R | What changed |
26
+ |---|---:|---:|---:|---:|---:|---:|---|
27
+ | 2.0.6, names off | 303 | 9 | 24 | 80 | 98.59% | 97.83% | A model span that stops inside a number grows over the whole number |
28
+ | 2.0.7, names off | 303 | 9 | 24 | 80 | 98.59% | 97.83% | Names layer leaves role nouns cut short or glued to a word out |
29
+
30
  ## 2.0.6
31
 
32
  Same NERGAL weights, threshold, union, rules (SHA `4d054723…`), names layer and API. One change in `nergal.py` `model_keep`: a kept model span that starts or ends inside a run of digits grows over the whole run, before the phone cut. Rule spans and person spans are unchanged.
README.md CHANGED
@@ -16,7 +16,7 @@ tags:
16
  - hybrid
17
  ---
18
 
19
- # NERGAL 2.0.6
20
 
21
  **Named Entity Recognition with Grounded Additive Labels**
22
 
@@ -66,9 +66,9 @@ NERGAL is not designed to remove:
66
 
67
  These are intended exclusions; false positives can still mask some of this content.
68
 
69
- ### Known gaps in 2.0.6
70
 
71
- Unlabelled phones and identifiers (the rules take a bare phone only in the grouped national forms `601 234 567` and `22 123 45 67`; a plain `601234567` needs a label or the model), unusual formatting and damaged text can escape detection. **VINs and obfuscated emails** (such as `name (at) domain.pl`) are approved annotation targets, but that approval alone does not establish reliable support in the released 2.0.6 model. Do not rely on it to remove them consistently. An email with a lowercase word glued onto its domain (`jan@firma.plkontakt`) is masked together with that word, and text glued before an address can be masked with it. A phone written as short parts joined by two slashes (`12 / 345 / 678`) is left as text. A model span is grown over the whole run of digits it stops in, but not across a space, so a spaced number can keep a group the model did not flag (`28 1020 5558`). Input that already holds a `[PII]` / `[PHONE]` / `[Telefon]` placeholder: a NIP or REGON label reaches past one placeholder to a checksum-valid value, and a value cut by a placeholder is masked digit group by digit group only after a `PESEL` / `NIP` / `REGON` label (`[PII]`) or a phone cue (`[PHONE]`). Without either (`8503[PII]1212343`, `601 [PII] 200 300`), with fewer than 5 digits, or when the run starts with a placeholder (`tel. [PHONE] 200 300`), the fragments stay. Names-on gaps are under [Names](#names).
72
 
73
  ## Versions
74
 
@@ -84,13 +84,14 @@ Unlabelled phones and identifiers (the rules take a bare phone only in the group
84
  | 2.0.3, names off | 303 | 9 | 24 | 80 | 98.59% | 97.83% | Values cut or preceded by a placeholder |
85
  | 2.0.4, names off | 303 | 9 | 24 | 80 | 98.59% | 97.83% | Names layer leaves procedural role nouns out |
86
  | 2.0.5, names off | 303 | 9 | 24 | 80 | 98.59% | 97.83% | Names layer leaves more role nouns, and a number right after one, out |
87
- | **2.0.6, names off** | **303** | **9** | **24** | **80** | **98.59%** | **97.83%** | A model span that stops inside a number grows over the whole number |
 
88
 
89
  1.0.0–1.1.2 were scored on the earlier 354-span gold (whole 323 → 326, union FP 133 → 80); per-version numbers are in `CHANGELOG.md`.
90
 
91
  ## Names
92
 
93
- `names=True` (CLI `--names`) adds a second model, [FastPDN NER — Polish PII](https://huggingface.co/ArkadiuszPawlak/fastpdn-ner-polish-pii) by ArkadiuszPawlak, revision `636c57fe…`, fine-tuned from [clarin-pl/FastPDN](https://huggingface.co/clarin-pl/FastPDN) on filtered KPWr plus LLM-synthetic data. `names/` holds it converted from ONNX to safetensors with no value changed. **`names/` is licensed [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/), not MIT**; `names/NOTICE.md` has the attribution, the changes and the file hashes. NERGAL uses only its person tags (`PERSON`, `PERSON_F`, `PERSON_L`), masks whole words, and always runs it in float32. It leaves procedural role nouns out of a person span (`Odwołujący Jan Nowak` → `Odwołujący [PERSON]`): since 2.0.5 the parties, representatives and offices odwołujący, przystępujący, zamawiający, protestujący, skarżący, przewodniczący, pozwany, powód, wykonawca, wnioskodawca, uczestnik, pełnomocnik, oferent, protokolant, inwestor and prezes, in any inflection and the feminine forms, also cut to the bare participle stem (`Zamawiając`) or with a number glued on (`Odwołujący3`). A number right after one (`Odwołujący 1`) is left out too. A role glued to another word in damaged text can still be masked.
94
 
95
  **Scope with names on.** The name test labels every person: private individuals, public officials, politicians and historical figures alike, plus fictional characters, nicknames, partial names and initials. Titles and roles stay outside the span. Names inside a street, patronage, organisation or work title, eponyms, usernames, and names inside emails or URLs are not person spans. Masking public figures removes who said, signed or wrote what, which is why names are opt-in. This is the labelling target; the model does not meet it everywhere.
96
 
@@ -101,7 +102,7 @@ Unlabelled phones and identifiers (the rules take a bare phone only in the group
101
  | Random | 300 | 614 | 586 (95.44%) | 98/116 (76.6–90.5%) | 1,021 | 34 (11.3%) |
102
  | Targeted | 150 | 292 | 286 (97.95%) | 94/99 (88.6–98.3%) | 976 | 42 (28.0%) |
103
 
104
- Names off hides no name. A component is the independent unit, covered when every name in it is whole: on the random panel 18 of 116 keep part of a name. The second pass is a same-day blind self-review of 100 passages by the same reviewer (11 disagreed), not independent agreement. Since 2.0.4 the scores are the frozen 2.0.0 predictions with person spans refiltered through the release's role-noun rule (2.0.3: 1,051 random false characters); 2.0.5 and 2.0.6 give the same spans as 2.0.4 on both name tests. `hybrid.json` `eval_names` has the full numbers, including float16 and INT8 scored for the record; neither ships.
105
 
106
  **Role-noun bench** (`nergal_role_bench_v1`, 2026-10-19): how often a capitalised procedural role is masked as a person. A role noun is never a name, so every one masked is a false mask and no gold is needed. 4,615 Dynaword legal documents, about 5M characters each from UZP appeal rulings, TED notices, local-government and gov.pl pages, EUR-Lex and Dziennik Ustaw, excluding the documents the 2.0.4 roles were chosen on; the 2.0.5 roles were chosen on the UZP rulings outside the bench. Every column runs the same FastPDN weights, so the change is NERGAL's span filter, not the model.
107
 
@@ -113,7 +114,7 @@ Names off hides no name. A component is the independent unit, covered when every
113
  | Skarżący | 50 | 3 | 3 | 0 | **0** |
114
  | Wykonawca, Prezes, Inwestor | 14,547 | 6 | 6 | 6 | **0** |
115
 
116
- The other nine roles tracked (wnioskodawca, uczestnik, powód, pozwany, przewodniczący, protokolant, pełnomocnik, oferent, protestujący) have no mask in any version. The one 2.0.4 `Zamawiający` is a truncated `Zamawiając`; it was known from this table before 2.0.5 added the bare stem, so its 1 → 0 is not a held-out result. Every version masks all 69,834 other words FastPDN tags, so the filter removes only role nouns; name recall is the name test above. The bench holds no tagged number after a role, so the 2.0.5 number rule is measured on UZP rulings (`CHANGELOG.md`). 2.0.6 changes only model `phone`/`pii` spans; its run equals 2.0.5 on every role and the guard. The texts hold real names and are not released.
117
 
118
  **Cost:** on one RTX 4090 with 3 processes, names took 2.0.0 from 74,520 to 66,492 chars/s (−11%) and peak device memory from 9,006 to 15,416 MiB; with one process, −9%. Local measurements are in `CHANGELOG.md`.
119
 
@@ -127,7 +128,7 @@ The other nine roles tracked (wnioskodawca, uczestnik, powód, pozwany, przewodn
127
 
128
  ## Compared with other systems
129
 
130
- Same split and gold. Every model runs through the NERGAL recipe with names off: its spans at 0.95, phone spans cut by phone policy v3, unioned with the rules. The rows were scored with the 1.2.0 rules; 2.0.0 through 2.0.6 give the same rule spans on this split. **Naked** is the model alone. **Residual** is passages with any gold character left unmasked. Character scores are gold vs masked characters.
131
 
132
  | System | Mode | Whole /315 | Residual | False chars | Char P | Char R |
133
  |---|---|---:|---:|---:|---:|---:|
@@ -137,9 +138,9 @@ Same split and gold. Every model runs through the NERGAL recipe with names off:
137
  | Fine-tuned GLiNER (previous incumbent) | naked | 263 | 28 | 15 | 99.70% | 88.80% |
138
  | Fine-tuned GLiNER (previous incumbent) | ∪ regex | 299 | 12 | 39 | 99.30% | 97.18% |
139
  | XLM-R epoch 5 | naked | 275 | 28 | 67 | 98.72% | 90.25% |
140
- | **NERGAL 2.0.6, names off** | **∪ regex** | **303** | **9** | **80** | **98.59%** | **97.83%** |
141
 
142
- Zero-shot [GLiNER 2.5](https://huggingface.co/fastino/gliner2.5-multi-v1) is not competitive, especially on non-phone PII (7/185 whole, naked). The fine-tuned GLiNER that NERGAL replaced is the more precise union (39 false characters vs 80) and the stronger phone model (128/130 whole, naked, vs 115), but it masks 4 fewer values whole and leaves 12 residual passages vs 9; XLM-R is the stronger model on other PII (160/185 vs 135). NERGAL 2.0.6 wholly masks 124/130 phones and 179/185 other PII, each under its own placeholder (`[PHONE]` / `[PII]`); exact-span precision 91.05%, recall 93.65%, F1 92.33%.
143
 
144
  XLM-R and epoch 5 were chosen in September 2026, on the earlier gold and before the phone-policy cut. Fine-tuned GLiNER, HerBERT-large and XLM-R-large were trained on the same split; only XLM-R passed the content-preservation gate, and epoch 5 at 0.95 (seed `202609160`) was the only one of 133 epoch/threshold points that covered more than the fine-tuned GLiNER union of that time while adding no false-mask characters it did not already make. Under the 1.2.0 rules, gold and phone cut, that no longer holds: the GLiNER union makes fewer false masks (table above).
145
 
 
16
  - hybrid
17
  ---
18
 
19
+ # NERGAL 2.0.7
20
 
21
  **Named Entity Recognition with Grounded Additive Labels**
22
 
 
66
 
67
  These are intended exclusions; false positives can still mask some of this content.
68
 
69
+ ### Known gaps in 2.0.7
70
 
71
+ Unlabelled phones and identifiers (the rules take a bare phone only in the grouped national forms `601 234 567` and `22 123 45 67`; a plain `601234567` needs a label or the model), unusual formatting and damaged text can escape detection. **VINs and obfuscated emails** (such as `name (at) domain.pl`) are approved annotation targets, but that approval alone does not establish reliable support in the released 2.0.7 model. Do not rely on it to remove them consistently. An email with a lowercase word glued onto its domain (`jan@firma.plkontakt`) is masked together with that word, and text glued before an address can be masked with it. A phone written as short parts joined by two slashes (`12 / 345 / 678`) is left as text. A model span is grown over the whole run of digits it stops in, but not across a space, so a spaced number can keep a group the model did not flag (`28 1020 5558`). A number neither the rules nor the model flag is left as text: on the label-pool benchmark, 46 of 344 gold phones and 27 of 160 gold identifiers holding a digit have no character masked. Some 10–13-digit runs after `nr` in procurement rulings are masked as identifiers with no gold to say whether they are identifiers or notice and case references. Input that already holds a `[PII]` / `[PHONE]` / `[Telefon]` placeholder: a NIP or REGON label reaches past one placeholder to a checksum-valid value, and a value cut by a placeholder is masked digit group by digit group only after a `PESEL` / `NIP` / `REGON` label (`[PII]`) or a phone cue (`[PHONE]`). Without either (`8503[PII]1212343`, `601 [PII] 200 300`), with fewer than 5 digits, or when the run starts with a placeholder (`tel. [PHONE] 200 300`), the fragments stay. Names-on gaps are under [Names](#names).
72
 
73
  ## Versions
74
 
 
84
  | 2.0.3, names off | 303 | 9 | 24 | 80 | 98.59% | 97.83% | Values cut or preceded by a placeholder |
85
  | 2.0.4, names off | 303 | 9 | 24 | 80 | 98.59% | 97.83% | Names layer leaves procedural role nouns out |
86
  | 2.0.5, names off | 303 | 9 | 24 | 80 | 98.59% | 97.83% | Names layer leaves more role nouns, and a number right after one, out |
87
+ | 2.0.6, names off | 303 | 9 | 24 | 80 | 98.59% | 97.83% | A model span that stops inside a number grows over the whole number |
88
+ | **2.0.7, names off** | **303** | **9** | **24** | **80** | **98.59%** | **97.83%** | Names layer leaves role nouns cut short or glued to a word out |
89
 
90
  1.0.0–1.1.2 were scored on the earlier 354-span gold (whole 323 → 326, union FP 133 → 80); per-version numbers are in `CHANGELOG.md`.
91
 
92
  ## Names
93
 
94
+ `names=True` (CLI `--names`) adds a second model, [FastPDN NER — Polish PII](https://huggingface.co/ArkadiuszPawlak/fastpdn-ner-polish-pii) by ArkadiuszPawlak, revision `636c57fe…`, fine-tuned from [clarin-pl/FastPDN](https://huggingface.co/clarin-pl/FastPDN) on filtered KPWr plus LLM-synthetic data. `names/` holds it converted from ONNX to safetensors with no value changed. **`names/` is licensed [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/), not MIT**; `names/NOTICE.md` has the attribution, the changes and the file hashes. NERGAL uses only its person tags (`PERSON`, `PERSON_F`, `PERSON_L`), masks whole words, and always runs it in float32. It leaves procedural role nouns out of a person span (`Odwołujący Jan Nowak` → `Odwołujący [PERSON]`): since 2.0.5 the parties, representatives and offices odwołujący, przystępujący, zamawiający, protestujący, skarżący, przewodniczący, pozwany, powód, wykonawca, wnioskodawca, uczestnik, pełnomocnik, oferent, protokolant, inwestor and prezes, in any inflection and the feminine forms, also cut to the bare participle stem (`Zamawiając`) or with a number glued on (`Odwołujący3`). A number right after one (`Odwołujący 1`) is left out too. Since 2.0.7 a role cut short in damaged text (`Odwołująceg`, `Wykonawc`) or glued to a lowercase word the text also uses on its own (`Odwołującywskazał`) is left out; a word glued before a role (`przezOdwołującego`), a number after a damaged role and other damaged shapes can still be masked.
95
 
96
  **Scope with names on.** The name test labels every person: private individuals, public officials, politicians and historical figures alike, plus fictional characters, nicknames, partial names and initials. Titles and roles stay outside the span. Names inside a street, patronage, organisation or work title, eponyms, usernames, and names inside emails or URLs are not person spans. Masking public figures removes who said, signed or wrote what, which is why names are opt-in. This is the labelling target; the model does not meet it everywhere.
97
 
 
102
  | Random | 300 | 614 | 586 (95.44%) | 98/116 (76.6–90.5%) | 1,021 | 34 (11.3%) |
103
  | Targeted | 150 | 292 | 286 (97.95%) | 94/99 (88.6–98.3%) | 976 | 42 (28.0%) |
104
 
105
+ Names off hides no name. A component is the independent unit, covered when every name in it is whole: on the random panel 18 of 116 keep part of a name. The second pass is a same-day blind self-review of 100 passages by the same reviewer (11 disagreed), not independent agreement. Since 2.0.4 the scores are the frozen 2.0.0 predictions with person spans refiltered through the release's role-noun rule (2.0.3: 1,051 random false characters); 2.0.5, 2.0.6 and 2.0.7 give the same spans as 2.0.4 on both name tests. `hybrid.json` `eval_names` has the full numbers, including float16 and INT8 scored for the record; neither ships.
106
 
107
  **Role-noun bench** (`nergal_role_bench_v1`, 2026-10-19): how often a capitalised procedural role is masked as a person. A role noun is never a name, so every one masked is a false mask and no gold is needed. 4,615 Dynaword legal documents, about 5M characters each from UZP appeal rulings, TED notices, local-government and gov.pl pages, EUR-Lex and Dziennik Ustaw, excluding the documents the 2.0.4 roles were chosen on; the 2.0.5 roles were chosen on the UZP rulings outside the bench. Every column runs the same FastPDN weights, so the change is NERGAL's span filter, not the model.
108
 
 
114
  | Skarżący | 50 | 3 | 3 | 0 | **0** |
115
  | Wykonawca, Prezes, Inwestor | 14,547 | 6 | 6 | 6 | **0** |
116
 
117
+ The other nine roles tracked (wnioskodawca, uczestnik, powód, pozwany, przewodniczący, protokolant, pełnomocnik, oferent, protestujący) have no mask in any version. The one 2.0.4 `Zamawiający` is a truncated `Zamawiając`; it was known from this table before 2.0.5 added the bare stem, so its 1 → 0 is not a held-out result. Every version masks all 69,834 other words FastPDN tags, so the filter removes only role nouns; name recall is the name test above. The bench holds no tagged number after a role, so the 2.0.5 number rule is measured on UZP rulings (`CHANGELOG.md`). 2.0.6 changes only model `phone`/`pii` spans; its run equals 2.0.5 on every role and the guard. The bench holds no damaged role noun, so 2.0.7 equals 2.0.5 there too; its words were checked by hand on UZP rulings (`CHANGELOG.md`). The texts hold real names and are not released.
118
 
119
  **Cost:** on one RTX 4090 with 3 processes, names took 2.0.0 from 74,520 to 66,492 chars/s (−11%) and peak device memory from 9,006 to 15,416 MiB; with one process, −9%. Local measurements are in `CHANGELOG.md`.
120
 
 
128
 
129
  ## Compared with other systems
130
 
131
+ Same split and gold. Every model runs through the NERGAL recipe with names off: its spans at 0.95, phone spans cut by phone policy v3, unioned with the rules. The rows were scored with the 1.2.0 rules; 2.0.0 through 2.0.7 give the same rule spans on this split. **Naked** is the model alone. **Residual** is passages with any gold character left unmasked. Character scores are gold vs masked characters.
132
 
133
  | System | Mode | Whole /315 | Residual | False chars | Char P | Char R |
134
  |---|---|---:|---:|---:|---:|---:|
 
138
  | Fine-tuned GLiNER (previous incumbent) | naked | 263 | 28 | 15 | 99.70% | 88.80% |
139
  | Fine-tuned GLiNER (previous incumbent) | ∪ regex | 299 | 12 | 39 | 99.30% | 97.18% |
140
  | XLM-R epoch 5 | naked | 275 | 28 | 67 | 98.72% | 90.25% |
141
+ | **NERGAL 2.0.7, names off** | **∪ regex** | **303** | **9** | **80** | **98.59%** | **97.83%** |
142
 
143
+ Zero-shot [GLiNER 2.5](https://huggingface.co/fastino/gliner2.5-multi-v1) is not competitive, especially on non-phone PII (7/185 whole, naked). The fine-tuned GLiNER that NERGAL replaced is the more precise union (39 false characters vs 80) and the stronger phone model (128/130 whole, naked, vs 115), but it masks 4 fewer values whole and leaves 12 residual passages vs 9; XLM-R is the stronger model on other PII (160/185 vs 135). NERGAL 2.0.7 wholly masks 124/130 phones and 179/185 other PII, each under its own placeholder (`[PHONE]` / `[PII]`); exact-span precision 91.05%, recall 93.65%, F1 92.33%.
144
 
145
  XLM-R and epoch 5 were chosen in September 2026, on the earlier gold and before the phone-policy cut. Fine-tuned GLiNER, HerBERT-large and XLM-R-large were trained on the same split; only XLM-R passed the content-preservation gate, and epoch 5 at 0.95 (seed `202609160`) was the only one of 133 epoch/threshold points that covered more than the fine-tuned GLiNER union of that time while adding no false-mask characters it did not already make. Under the 1.2.0 rules, gold and phone cut, that no longer holds: the GLiNER union makes fewer false masks (table above).
146
 
hybrid.json CHANGED
@@ -1,7 +1,7 @@
1
  {
2
  "full_name": "Named Entity Recognition with Grounded Additive Labels",
3
  "hub_id": "SlayerLab/NERGAL",
4
- "version": "2.0.6",
5
  "mode": "rules_union",
6
  "epoch": 5,
7
  "seed": 202609160,
 
1
  {
2
  "full_name": "Named Entity Recognition with Grounded Additive Labels",
3
  "hub_id": "SlayerLab/NERGAL",
4
+ "version": "2.0.7",
5
  "mode": "rules_union",
6
  "epoch": 5,
7
  "seed": 202609160,
nergal.py CHANGED
@@ -17,7 +17,7 @@ import scrub_pii
17
  from scrub_pii import LEGACY_PHONE_TAG, PERSON_TAG, PHONE_TAG, PII_TAG
18
 
19
  HUB_ID = 'SlayerLab/NERGAL'
20
- VERSION = '2.0.6'
21
  GAPS = ['[PII_SPACE]', '[PII_BREAK]']
22
  GAP_IDS = [250002, 250003]
23
  BIO_LABELS = ['O', 'B-phone', 'I-phone', 'B-pii', 'I-pii']
@@ -47,6 +47,18 @@ ROLE_NOUNS = tuple(stem + _PARTICIPLE for stem in ('odwołując', 'przystępują
47
  'prezes(?:a|owi|em|ie|i|ów|om|ami|ach)?', 'uczestni' + _NIK, 'pełnomocni' + _NIK, 'oferen' + _NT, 'protokolan' + _NT)
48
  _ROLE_NOUN = re.compile(r'\W*(?:%s)_?\d*\W*' % '|'.join(ROLE_NOUNS), re.IGNORECASE)
49
  _NUMBER = re.compile(r'\W*\d+\W*') # with the punctuation a tagged token grows to, as for _ROLE_NOUN
 
 
 
 
 
 
 
 
 
 
 
 
50
 
51
 
52
  def sha(path):
@@ -224,10 +236,23 @@ def whole_digits(text, start, end):
224
  return start, end
225
 
226
 
 
 
 
 
 
 
 
 
 
 
 
 
 
227
  def person_spans(text, ranges):
228
  """Person spans from FastPDN person-token ranges: whole words, joined across spaces, tabs and no-break spaces
229
- (not punctuation or line breaks); words touching a placeholder, role nouns (ROLE_NOUNS) and a number
230
- right after a skipped role noun (`Odwołujący 1`) are skipped."""
231
  blocked = [m.span() for m in _PLACEHOLDER.finditer(text)]
232
  merged, role_end = [], None
233
  for start, end in sorted(expand(text, a, b) for a, b in ranges):
@@ -236,6 +261,8 @@ def person_spans(text, ranges):
236
  if _ROLE_NOUN.fullmatch(text, start, end):
237
  role_end = end
238
  continue
 
 
239
  if role_end is not None and _NUMBER.fullmatch(text, start, end) and not text[role_end:start].strip(' \t\u00a0'):
240
  continue
241
  if merged and not text[merged[-1][1]:start].strip(' \t\u00a0'):
 
17
  from scrub_pii import LEGACY_PHONE_TAG, PERSON_TAG, PHONE_TAG, PII_TAG
18
 
19
  HUB_ID = 'SlayerLab/NERGAL'
20
+ VERSION = '2.0.7'
21
  GAPS = ['[PII_SPACE]', '[PII_BREAK]']
22
  GAP_IDS = [250002, 250003]
23
  BIO_LABELS = ['O', 'B-phone', 'I-phone', 'B-pii', 'I-pii']
 
47
  'prezes(?:a|owi|em|ie|i|ów|om|ami|ach)?', 'uczestni' + _NIK, 'pełnomocni' + _NIK, 'oferen' + _NT, 'protokolan' + _NT)
48
  _ROLE_NOUN = re.compile(r'\W*(?:%s)_?\d*\W*' % '|'.join(ROLE_NOUNS), re.IGNORECASE)
49
  _NUMBER = re.compile(r'\W*\d+\W*') # with the punctuation a tagged token grows to, as for _ROLE_NOUN
50
+ # Role nouns in damaged text (2.0.7), each narrow enough that a surname which only starts like a role noun
51
+ # (`Odwołującyński`, `Wykonawcówna`) stays masked:
52
+ # cut short the stem and the start of an ending: `Odwołująceg`, `Wykonawc`, `Wykonawcam`
53
+ # glued after a role noun and a lowercase word of 4+ letters that the same text also uses on its own:
54
+ # `Odwołującywskazał`
55
+ # Left masked, so the role bench's guards (nergal_role_bench.count) do not move: a word glued before a role noun
56
+ # (`przezOdwołującego`, an `other` word there) and a number after a damaged one (`Odwołująceg 1`).
57
+ _ROLE_CUT = re.compile(r"\W*(?:(?:odwołując|przystępując|zamawiając|protestując|skarżąc|przewodnicząc)(?:eg|em|yc)|"
58
+ r"(?:wykonawc|wnioskodawc)(?:ó|am|ac)?)\W*", re.IGNORECASE)
59
+ _ROLE_BARE = re.compile('(?:%s)' % '|'.join(ROLE_NOUNS), re.IGNORECASE)
60
+ _GLUED_AFTER = re.compile(r"([a-ząćęłńóśźż]{4,})\W*")
61
+ _LEAD = re.compile(r'\W*')
62
 
63
 
64
  def sha(path):
 
236
  return start, end
237
 
238
 
239
+ def damaged_role_word(text, start, end):
240
+ """A role noun cut short (_ROLE_CUT) or glued to a lowercase word the text also uses on its own (_GLUED_AFTER)."""
241
+ if _ROLE_CUT.fullmatch(text, start, end):
242
+ return True
243
+ word = _LEAD.match(text, start, end).end()
244
+ for cut in range(word + 1, end): # every split: `Odwołujący|miała` as well as `Odwołującym|iała`
245
+ tail = _GLUED_AFTER.fullmatch(text, cut, end)
246
+ if tail and _ROLE_BARE.fullmatch(text, word, cut) and re.search(
247
+ r"(?<![\w'’-])%s(?![\w'’-])" % re.escape(tail[1]), text):
248
+ return True
249
+ return False
250
+
251
+
252
  def person_spans(text, ranges):
253
  """Person spans from FastPDN person-token ranges: whole words, joined across spaces, tabs and no-break spaces
254
+ (not punctuation or line breaks); words touching a placeholder, role nouns (ROLE_NOUNS), damaged ones
255
+ (damaged_role_word) and a number right after a skipped role noun (`Odwołujący 1`) are skipped."""
256
  blocked = [m.span() for m in _PLACEHOLDER.finditer(text)]
257
  merged, role_end = [], None
258
  for start, end in sorted(expand(text, a, b) for a, b in ranges):
 
261
  if _ROLE_NOUN.fullmatch(text, start, end):
262
  role_end = end
263
  continue
264
+ if damaged_role_word(text, start, end):
265
+ continue
266
  if role_end is not None and _NUMBER.fullmatch(text, start, end) and not text[role_end:start].strip(' \t\u00a0'):
267
  continue
268
  if merged and not text[merged[-1][1]:start].strip(' \t\u00a0'):
test_nergal.py CHANGED
@@ -43,7 +43,7 @@ class NergalTests(unittest.TestCase):
43
  from nergal import GAP_IDS, GAPS, HUB_ID, RULES_SHA as PINNED, THRESHOLD, VERSION
44
  card = json.loads((HERE / 'hybrid.json').read_text())
45
  self.assertEqual(HUB_ID, 'SlayerLab/NERGAL')
46
- self.assertEqual(VERSION, '2.0.6')
47
  self.assertEqual(card['version'], VERSION)
48
  self.assertEqual(card['eval']['union_fp'], 80)
49
  self.assertEqual(card['eval']['rules_fp'], 24)
@@ -322,6 +322,34 @@ class NergalTests(unittest.TestCase):
322
  for text in ('Odwołujący3a', 'Odwołujący__2', 'Odwołującegozarzut', 'Powodowski2'): # glued to more than a number
323
  self.assertEqual(persons(text), [text], text)
324
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
325
  def test_person_spans_leave_the_adverbial_participle_out(self):
326
  from nergal import person_spans
327
  text = 'Odwołując się do wyroku, Przystępując do sporu'
 
43
  from nergal import GAP_IDS, GAPS, HUB_ID, RULES_SHA as PINNED, THRESHOLD, VERSION
44
  card = json.loads((HERE / 'hybrid.json').read_text())
45
  self.assertEqual(HUB_ID, 'SlayerLab/NERGAL')
46
+ self.assertEqual(VERSION, '2.0.7')
47
  self.assertEqual(card['version'], VERSION)
48
  self.assertEqual(card['eval']['union_fp'], 80)
49
  self.assertEqual(card['eval']['rules_fp'], 24)
 
322
  for text in ('Odwołujący3a', 'Odwołujący__2', 'Odwołującegozarzut', 'Powodowski2'): # glued to more than a number
323
  self.assertEqual(persons(text), [text], text)
324
 
325
+ def test_person_spans_leave_role_nouns_in_damaged_text_out(self):
326
+ from nergal import person_spans
327
+
328
+ def persons(text, first_only=False): # every word tagged, or only the first
329
+ ranges = [m.span() for m in re.finditer(r"[\w'’-]+", text)]
330
+ return [text[s['start']:s['end']] for s in person_spans(text, ranges[:1] if first_only else ranges)]
331
+ cut_short = ('Odwołująceg', 'Odwołującem', 'Odwołującyc', 'Przystępująceg', 'Zamawiającem', 'Skarżącyc',
332
+ 'Wykonawc', 'Wykonawcó', 'Wykonawcam', 'Wykonawcac', 'Wnioskodawc', 'Wnioskodawcó')
333
+ for word in cut_short:
334
+ for text in (word, word.upper(), word.lower(), f'({word})'):
335
+ self.assertEqual(persons(text), [], text)
336
+ self.assertEqual(persons('Odwołująceg Jan Nowak'), ['Jan Nowak'])
337
+ self.assertEqual(persons('Odwołująceg 1'), ['1']) # a number after a damaged role noun stays (role bench guard)
338
+ self.assertEqual(persons('Odwołujący Odwołująceg 1'), ['1']) # not right after the whole one
339
+ for text in ('przezOdwołującego', 'iPrzystępujący', 'zWykonawcą'):
340
+ self.assertEqual(persons(text), [text], text) # glued before: a role bench `other` word, left masked
341
+ for text in ('Odwołującywskazał termin, a Sąd wskazał termin', 'Wykonawcazłożył ofertę; Jan złożył',
342
+ 'Odwołującymiała rację, choć nie miała', 'ODWOŁUJĄCYwskazał, wskazał', 'Odwołującywskazał. wskazał'):
343
+ self.assertEqual(persons(text, first_only=True), [], text) # a glued word the text also uses on its own
344
+ kept = ('Odwołującywskazał termin', 'Odwołującyże że', 'OdwołującyWskazał wskazał', 'OdwołującyJan',
345
+ 'KowalOdwołujący', 'przezodwołującego', 'przezOdwołującywskazał wskazał', 'Odwołującegozarzut', 'Odwołującyego', 'Wykonawcówna ówna',
346
+ 'Odwołującyński', 'Odwołującegg', 'Wykonawcz')
347
+ for text in kept: # no lone tail, a tail under 4 letters, a capital after, a capital or nothing before
348
+ word = re.match(r"[\w'’-]+", text)[0]
349
+ self.assertEqual(persons(text, first_only=True), [word], text)
350
+ for name in STEM_NAMES:
351
+ self.assertEqual(persons(f'{name} i ona', first_only=True), [name])
352
+
353
  def test_person_spans_leave_the_adverbial_participle_out(self):
354
  from nergal import person_spans
355
  text = 'Odwołując się do wyroku, Przystępując do sporu'