Instructions to use SlayerLab/NERGAL with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use SlayerLab/NERGAL with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("token-classification", model="SlayerLab/NERGAL")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForTokenClassification tokenizer = AutoTokenizer.from_pretrained("SlayerLab/NERGAL") model = AutoModelForTokenClassification.from_pretrained("SlayerLab/NERGAL", device_map="auto") - Notebooks
- Google Colab
- Kaggle
2.0.7: the names layer leaves role nouns cut short or glued to a word out
Browse files- CHANGELOG.md +19 -0
- README.md +11 -10
- hybrid.json +1 -1
- nergal.py +30 -3
- test_nergal.py +29 -1
CHANGELOG.md
CHANGED
|
@@ -8,6 +8,25 @@ Semver for this island:
|
|
| 8 |
|
| 9 |
Accuracy is 841-dev, union at 0.95. Through 1.1.2 the gold has 354 spans; from 1.2.0 it is restated to phone policy v3 (315 spans), and the two are not comparable. A version that changes those numbers must update `hybrid.json` `eval` and the tables below.
|
| 10 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
## 2.0.6
|
| 12 |
|
| 13 |
Same NERGAL weights, threshold, union, rules (SHA `4d054723…`), names layer and API. One change in `nergal.py` `model_keep`: a kept model span that starts or ends inside a run of digits grows over the whole run, before the phone cut. Rule spans and person spans are unchanged.
|
|
|
|
| 8 |
|
| 9 |
Accuracy is 841-dev, union at 0.95. Through 1.1.2 the gold has 354 spans; from 1.2.0 it is restated to phone policy v3 (315 spans), and the two are not comparable. A version that changes those numbers must update `hybrid.json` `eval` and the tables below.
|
| 10 |
|
| 11 |
+
## 2.0.7
|
| 12 |
+
|
| 13 |
+
Same NERGAL weights, threshold, union, rules (SHA `4d054723…`) and API. One change in `nergal.py` `person_spans`: the names layer (`names=True`) also leaves a role noun damaged in the text out of a person span. Names off is unchanged.
|
| 14 |
+
|
| 15 |
+
- **Cut short.** A role stem with the start of an ending and nothing after (`Odwołująceg`, `Przystępującyc`, `Wykonawc`, `Wykonawcam`), as text extraction leaves it at a line or page break.
|
| 16 |
+
- **Glued after.** A role noun followed by a lowercase word of 4+ letters that the same text also uses on its own (`Odwołującywskazał` where `wskazał` occurs alone). A surname that only starts like a role (`Odwołującyński`, `Wykonawcówna`), a capital after (`OdwołującyJan`) or a tail the text never uses alone stays masked.
|
| 17 |
+
- **Left masked on purpose.** A word glued before a role noun (`przezOdwołującego`) and a number after a damaged role noun (`Odwołująceg 1`): both count as other words on the role bench's guard, which 2.0.7 must not move.
|
| 18 |
+
- **Measured on every UZP ruling.** Developed on the rulings outside the role bench, frozen (`artifacts/nergal_2_0_7_role_glued/freeze.json`), then scored once on all 34,715: 18 rulings change, 247 characters lose a person mask, none gains one. Capitalised role-stem words masked by 2.0.6 that are not role nouns: `Odwołując…` 37 → 27 (7 cut short, 3 glued after unmasked), `Przystępując…` 12 → 7, `Wykonawc…` 8 → 3.
|
| 19 |
+
- **Human check.** No held-out set holds a damaged role noun (role bench, name tests v1/v2, label pool v3, the Dynaword sample: 0 each; the 400 G1 check passages are all from dev rulings, 1 changes, 17 characters, no labelled span), so every word 2.0.7 unmasks in UZP was reviewed against a rule fixed beforehand (`human_check_prereg.json`): 20 words in 18 rulings, 20 role nouns, 0 names.
|
| 20 |
+
|
| 21 |
+
Still missed: words glued before a role, a number after a damaged role, tails under 4 letters or capitalised, and other shapes (27 `Odwołując…`, 7 `Przystępując…` capitalised words left); `inwestorski` and other words that only start with a role stem, kept on purpose.
|
| 22 |
+
|
| 23 |
+
841-dev, names off: unchanged from 2.0.6. Release gates (`uzp_delta.json`, frozen before scoring, stored spans, no new inference), all 0: characters gaining a person span, characters lost outside a damaged role word, role-bench guard counts moved, name-test v1/v2 passages changed (every stored runtime), label pool v3 bench cohorts changed names off or on, and the 2.0.6 side differing from the published bench files. Role bench (`scores/2.0.7_c19e30c979e6_mps.json`): equal to 2.0.6 on every role and guard. `hybrid.json` `eval` and `eval_names` are unchanged.
|
| 24 |
+
|
| 25 |
+
| Version | Whole /315 | Residual | Rules FP | Union FP | Char P | Char R | What changed |
|
| 26 |
+
|---|---:|---:|---:|---:|---:|---:|---|
|
| 27 |
+
| 2.0.6, names off | 303 | 9 | 24 | 80 | 98.59% | 97.83% | A model span that stops inside a number grows over the whole number |
|
| 28 |
+
| 2.0.7, names off | 303 | 9 | 24 | 80 | 98.59% | 97.83% | Names layer leaves role nouns cut short or glued to a word out |
|
| 29 |
+
|
| 30 |
## 2.0.6
|
| 31 |
|
| 32 |
Same NERGAL weights, threshold, union, rules (SHA `4d054723…`), names layer and API. One change in `nergal.py` `model_keep`: a kept model span that starts or ends inside a run of digits grows over the whole run, before the phone cut. Rule spans and person spans are unchanged.
|
README.md
CHANGED
|
@@ -16,7 +16,7 @@ tags:
|
|
| 16 |
- hybrid
|
| 17 |
---
|
| 18 |
|
| 19 |
-
# NERGAL 2.0.
|
| 20 |
|
| 21 |
**Named Entity Recognition with Grounded Additive Labels**
|
| 22 |
|
|
@@ -66,9 +66,9 @@ NERGAL is not designed to remove:
|
|
| 66 |
|
| 67 |
These are intended exclusions; false positives can still mask some of this content.
|
| 68 |
|
| 69 |
-
### Known gaps in 2.0.
|
| 70 |
|
| 71 |
-
Unlabelled phones and identifiers (the rules take a bare phone only in the grouped national forms `601 234 567` and `22 123 45 67`; a plain `601234567` needs a label or the model), unusual formatting and damaged text can escape detection. **VINs and obfuscated emails** (such as `name (at) domain.pl`) are approved annotation targets, but that approval alone does not establish reliable support in the released 2.0.
|
| 72 |
|
| 73 |
## Versions
|
| 74 |
|
|
@@ -84,13 +84,14 @@ Unlabelled phones and identifiers (the rules take a bare phone only in the group
|
|
| 84 |
| 2.0.3, names off | 303 | 9 | 24 | 80 | 98.59% | 97.83% | Values cut or preceded by a placeholder |
|
| 85 |
| 2.0.4, names off | 303 | 9 | 24 | 80 | 98.59% | 97.83% | Names layer leaves procedural role nouns out |
|
| 86 |
| 2.0.5, names off | 303 | 9 | 24 | 80 | 98.59% | 97.83% | Names layer leaves more role nouns, and a number right after one, out |
|
| 87 |
-
|
|
|
|
|
| 88 |
|
| 89 |
1.0.0–1.1.2 were scored on the earlier 354-span gold (whole 323 → 326, union FP 133 → 80); per-version numbers are in `CHANGELOG.md`.
|
| 90 |
|
| 91 |
## Names
|
| 92 |
|
| 93 |
-
`names=True` (CLI `--names`) adds a second model, [FastPDN NER — Polish PII](https://huggingface.co/ArkadiuszPawlak/fastpdn-ner-polish-pii) by ArkadiuszPawlak, revision `636c57fe…`, fine-tuned from [clarin-pl/FastPDN](https://huggingface.co/clarin-pl/FastPDN) on filtered KPWr plus LLM-synthetic data. `names/` holds it converted from ONNX to safetensors with no value changed. **`names/` is licensed [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/), not MIT**; `names/NOTICE.md` has the attribution, the changes and the file hashes. NERGAL uses only its person tags (`PERSON`, `PERSON_F`, `PERSON_L`), masks whole words, and always runs it in float32. It leaves procedural role nouns out of a person span (`Odwołujący Jan Nowak` → `Odwołujący [PERSON]`): since 2.0.5 the parties, representatives and offices odwołujący, przystępujący, zamawiający, protestujący, skarżący, przewodniczący, pozwany, powód, wykonawca, wnioskodawca, uczestnik, pełnomocnik, oferent, protokolant, inwestor and prezes, in any inflection and the feminine forms, also cut to the bare participle stem (`Zamawiając`) or with a number glued on (`Odwołujący3`). A number right after one (`Odwołujący 1`) is left out too.
|
| 94 |
|
| 95 |
**Scope with names on.** The name test labels every person: private individuals, public officials, politicians and historical figures alike, plus fictional characters, nicknames, partial names and initials. Titles and roles stay outside the span. Names inside a street, patronage, organisation or work title, eponyms, usernames, and names inside emails or URLs are not person spans. Masking public figures removes who said, signed or wrote what, which is why names are opt-in. This is the labelling target; the model does not meet it everywhere.
|
| 96 |
|
|
@@ -101,7 +102,7 @@ Unlabelled phones and identifiers (the rules take a bare phone only in the group
|
|
| 101 |
| Random | 300 | 614 | 586 (95.44%) | 98/116 (76.6–90.5%) | 1,021 | 34 (11.3%) |
|
| 102 |
| Targeted | 150 | 292 | 286 (97.95%) | 94/99 (88.6–98.3%) | 976 | 42 (28.0%) |
|
| 103 |
|
| 104 |
-
Names off hides no name. A component is the independent unit, covered when every name in it is whole: on the random panel 18 of 116 keep part of a name. The second pass is a same-day blind self-review of 100 passages by the same reviewer (11 disagreed), not independent agreement. Since 2.0.4 the scores are the frozen 2.0.0 predictions with person spans refiltered through the release's role-noun rule (2.0.3: 1,051 random false characters); 2.0.5 and 2.0.
|
| 105 |
|
| 106 |
**Role-noun bench** (`nergal_role_bench_v1`, 2026-10-19): how often a capitalised procedural role is masked as a person. A role noun is never a name, so every one masked is a false mask and no gold is needed. 4,615 Dynaword legal documents, about 5M characters each from UZP appeal rulings, TED notices, local-government and gov.pl pages, EUR-Lex and Dziennik Ustaw, excluding the documents the 2.0.4 roles were chosen on; the 2.0.5 roles were chosen on the UZP rulings outside the bench. Every column runs the same FastPDN weights, so the change is NERGAL's span filter, not the model.
|
| 107 |
|
|
@@ -113,7 +114,7 @@ Names off hides no name. A component is the independent unit, covered when every
|
|
| 113 |
| Skarżący | 50 | 3 | 3 | 0 | **0** |
|
| 114 |
| Wykonawca, Prezes, Inwestor | 14,547 | 6 | 6 | 6 | **0** |
|
| 115 |
|
| 116 |
-
The other nine roles tracked (wnioskodawca, uczestnik, powód, pozwany, przewodniczący, protokolant, pełnomocnik, oferent, protestujący) have no mask in any version. The one 2.0.4 `Zamawiający` is a truncated `Zamawiając`; it was known from this table before 2.0.5 added the bare stem, so its 1 → 0 is not a held-out result. Every version masks all 69,834 other words FastPDN tags, so the filter removes only role nouns; name recall is the name test above. The bench holds no tagged number after a role, so the 2.0.5 number rule is measured on UZP rulings (`CHANGELOG.md`). 2.0.6 changes only model `phone`/`pii` spans; its run equals 2.0.5 on every role and the guard. The texts hold real names and are not released.
|
| 117 |
|
| 118 |
**Cost:** on one RTX 4090 with 3 processes, names took 2.0.0 from 74,520 to 66,492 chars/s (−11%) and peak device memory from 9,006 to 15,416 MiB; with one process, −9%. Local measurements are in `CHANGELOG.md`.
|
| 119 |
|
|
@@ -127,7 +128,7 @@ The other nine roles tracked (wnioskodawca, uczestnik, powód, pozwany, przewodn
|
|
| 127 |
|
| 128 |
## Compared with other systems
|
| 129 |
|
| 130 |
-
Same split and gold. Every model runs through the NERGAL recipe with names off: its spans at 0.95, phone spans cut by phone policy v3, unioned with the rules. The rows were scored with the 1.2.0 rules; 2.0.0 through 2.0.
|
| 131 |
|
| 132 |
| System | Mode | Whole /315 | Residual | False chars | Char P | Char R |
|
| 133 |
|---|---|---:|---:|---:|---:|---:|
|
|
@@ -137,9 +138,9 @@ Same split and gold. Every model runs through the NERGAL recipe with names off:
|
|
| 137 |
| Fine-tuned GLiNER (previous incumbent) | naked | 263 | 28 | 15 | 99.70% | 88.80% |
|
| 138 |
| Fine-tuned GLiNER (previous incumbent) | ∪ regex | 299 | 12 | 39 | 99.30% | 97.18% |
|
| 139 |
| XLM-R epoch 5 | naked | 275 | 28 | 67 | 98.72% | 90.25% |
|
| 140 |
-
| **NERGAL 2.0.
|
| 141 |
|
| 142 |
-
Zero-shot [GLiNER 2.5](https://huggingface.co/fastino/gliner2.5-multi-v1) is not competitive, especially on non-phone PII (7/185 whole, naked). The fine-tuned GLiNER that NERGAL replaced is the more precise union (39 false characters vs 80) and the stronger phone model (128/130 whole, naked, vs 115), but it masks 4 fewer values whole and leaves 12 residual passages vs 9; XLM-R is the stronger model on other PII (160/185 vs 135). NERGAL 2.0.
|
| 143 |
|
| 144 |
XLM-R and epoch 5 were chosen in September 2026, on the earlier gold and before the phone-policy cut. Fine-tuned GLiNER, HerBERT-large and XLM-R-large were trained on the same split; only XLM-R passed the content-preservation gate, and epoch 5 at 0.95 (seed `202609160`) was the only one of 133 epoch/threshold points that covered more than the fine-tuned GLiNER union of that time while adding no false-mask characters it did not already make. Under the 1.2.0 rules, gold and phone cut, that no longer holds: the GLiNER union makes fewer false masks (table above).
|
| 145 |
|
|
|
|
| 16 |
- hybrid
|
| 17 |
---
|
| 18 |
|
| 19 |
+
# NERGAL 2.0.7
|
| 20 |
|
| 21 |
**Named Entity Recognition with Grounded Additive Labels**
|
| 22 |
|
|
|
|
| 66 |
|
| 67 |
These are intended exclusions; false positives can still mask some of this content.
|
| 68 |
|
| 69 |
+
### Known gaps in 2.0.7
|
| 70 |
|
| 71 |
+
Unlabelled phones and identifiers (the rules take a bare phone only in the grouped national forms `601 234 567` and `22 123 45 67`; a plain `601234567` needs a label or the model), unusual formatting and damaged text can escape detection. **VINs and obfuscated emails** (such as `name (at) domain.pl`) are approved annotation targets, but that approval alone does not establish reliable support in the released 2.0.7 model. Do not rely on it to remove them consistently. An email with a lowercase word glued onto its domain (`jan@firma.plkontakt`) is masked together with that word, and text glued before an address can be masked with it. A phone written as short parts joined by two slashes (`12 / 345 / 678`) is left as text. A model span is grown over the whole run of digits it stops in, but not across a space, so a spaced number can keep a group the model did not flag (`28 1020 5558`). A number neither the rules nor the model flag is left as text: on the label-pool benchmark, 46 of 344 gold phones and 27 of 160 gold identifiers holding a digit have no character masked. Some 10–13-digit runs after `nr` in procurement rulings are masked as identifiers with no gold to say whether they are identifiers or notice and case references. Input that already holds a `[PII]` / `[PHONE]` / `[Telefon]` placeholder: a NIP or REGON label reaches past one placeholder to a checksum-valid value, and a value cut by a placeholder is masked digit group by digit group only after a `PESEL` / `NIP` / `REGON` label (`[PII]`) or a phone cue (`[PHONE]`). Without either (`8503[PII]1212343`, `601 [PII] 200 300`), with fewer than 5 digits, or when the run starts with a placeholder (`tel. [PHONE] 200 300`), the fragments stay. Names-on gaps are under [Names](#names).
|
| 72 |
|
| 73 |
## Versions
|
| 74 |
|
|
|
|
| 84 |
| 2.0.3, names off | 303 | 9 | 24 | 80 | 98.59% | 97.83% | Values cut or preceded by a placeholder |
|
| 85 |
| 2.0.4, names off | 303 | 9 | 24 | 80 | 98.59% | 97.83% | Names layer leaves procedural role nouns out |
|
| 86 |
| 2.0.5, names off | 303 | 9 | 24 | 80 | 98.59% | 97.83% | Names layer leaves more role nouns, and a number right after one, out |
|
| 87 |
+
| 2.0.6, names off | 303 | 9 | 24 | 80 | 98.59% | 97.83% | A model span that stops inside a number grows over the whole number |
|
| 88 |
+
| **2.0.7, names off** | **303** | **9** | **24** | **80** | **98.59%** | **97.83%** | Names layer leaves role nouns cut short or glued to a word out |
|
| 89 |
|
| 90 |
1.0.0–1.1.2 were scored on the earlier 354-span gold (whole 323 → 326, union FP 133 → 80); per-version numbers are in `CHANGELOG.md`.
|
| 91 |
|
| 92 |
## Names
|
| 93 |
|
| 94 |
+
`names=True` (CLI `--names`) adds a second model, [FastPDN NER — Polish PII](https://huggingface.co/ArkadiuszPawlak/fastpdn-ner-polish-pii) by ArkadiuszPawlak, revision `636c57fe…`, fine-tuned from [clarin-pl/FastPDN](https://huggingface.co/clarin-pl/FastPDN) on filtered KPWr plus LLM-synthetic data. `names/` holds it converted from ONNX to safetensors with no value changed. **`names/` is licensed [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/), not MIT**; `names/NOTICE.md` has the attribution, the changes and the file hashes. NERGAL uses only its person tags (`PERSON`, `PERSON_F`, `PERSON_L`), masks whole words, and always runs it in float32. It leaves procedural role nouns out of a person span (`Odwołujący Jan Nowak` → `Odwołujący [PERSON]`): since 2.0.5 the parties, representatives and offices odwołujący, przystępujący, zamawiający, protestujący, skarżący, przewodniczący, pozwany, powód, wykonawca, wnioskodawca, uczestnik, pełnomocnik, oferent, protokolant, inwestor and prezes, in any inflection and the feminine forms, also cut to the bare participle stem (`Zamawiając`) or with a number glued on (`Odwołujący3`). A number right after one (`Odwołujący 1`) is left out too. Since 2.0.7 a role cut short in damaged text (`Odwołująceg`, `Wykonawc`) or glued to a lowercase word the text also uses on its own (`Odwołującywskazał`) is left out; a word glued before a role (`przezOdwołującego`), a number after a damaged role and other damaged shapes can still be masked.
|
| 95 |
|
| 96 |
**Scope with names on.** The name test labels every person: private individuals, public officials, politicians and historical figures alike, plus fictional characters, nicknames, partial names and initials. Titles and roles stay outside the span. Names inside a street, patronage, organisation or work title, eponyms, usernames, and names inside emails or URLs are not person spans. Masking public figures removes who said, signed or wrote what, which is why names are opt-in. This is the labelling target; the model does not meet it everywhere.
|
| 97 |
|
|
|
|
| 102 |
| Random | 300 | 614 | 586 (95.44%) | 98/116 (76.6–90.5%) | 1,021 | 34 (11.3%) |
|
| 103 |
| Targeted | 150 | 292 | 286 (97.95%) | 94/99 (88.6–98.3%) | 976 | 42 (28.0%) |
|
| 104 |
|
| 105 |
+
Names off hides no name. A component is the independent unit, covered when every name in it is whole: on the random panel 18 of 116 keep part of a name. The second pass is a same-day blind self-review of 100 passages by the same reviewer (11 disagreed), not independent agreement. Since 2.0.4 the scores are the frozen 2.0.0 predictions with person spans refiltered through the release's role-noun rule (2.0.3: 1,051 random false characters); 2.0.5, 2.0.6 and 2.0.7 give the same spans as 2.0.4 on both name tests. `hybrid.json` `eval_names` has the full numbers, including float16 and INT8 scored for the record; neither ships.
|
| 106 |
|
| 107 |
**Role-noun bench** (`nergal_role_bench_v1`, 2026-10-19): how often a capitalised procedural role is masked as a person. A role noun is never a name, so every one masked is a false mask and no gold is needed. 4,615 Dynaword legal documents, about 5M characters each from UZP appeal rulings, TED notices, local-government and gov.pl pages, EUR-Lex and Dziennik Ustaw, excluding the documents the 2.0.4 roles were chosen on; the 2.0.5 roles were chosen on the UZP rulings outside the bench. Every column runs the same FastPDN weights, so the change is NERGAL's span filter, not the model.
|
| 108 |
|
|
|
|
| 114 |
| Skarżący | 50 | 3 | 3 | 0 | **0** |
|
| 115 |
| Wykonawca, Prezes, Inwestor | 14,547 | 6 | 6 | 6 | **0** |
|
| 116 |
|
| 117 |
+
The other nine roles tracked (wnioskodawca, uczestnik, powód, pozwany, przewodniczący, protokolant, pełnomocnik, oferent, protestujący) have no mask in any version. The one 2.0.4 `Zamawiający` is a truncated `Zamawiając`; it was known from this table before 2.0.5 added the bare stem, so its 1 → 0 is not a held-out result. Every version masks all 69,834 other words FastPDN tags, so the filter removes only role nouns; name recall is the name test above. The bench holds no tagged number after a role, so the 2.0.5 number rule is measured on UZP rulings (`CHANGELOG.md`). 2.0.6 changes only model `phone`/`pii` spans; its run equals 2.0.5 on every role and the guard. The bench holds no damaged role noun, so 2.0.7 equals 2.0.5 there too; its words were checked by hand on UZP rulings (`CHANGELOG.md`). The texts hold real names and are not released.
|
| 118 |
|
| 119 |
**Cost:** on one RTX 4090 with 3 processes, names took 2.0.0 from 74,520 to 66,492 chars/s (−11%) and peak device memory from 9,006 to 15,416 MiB; with one process, −9%. Local measurements are in `CHANGELOG.md`.
|
| 120 |
|
|
|
|
| 128 |
|
| 129 |
## Compared with other systems
|
| 130 |
|
| 131 |
+
Same split and gold. Every model runs through the NERGAL recipe with names off: its spans at 0.95, phone spans cut by phone policy v3, unioned with the rules. The rows were scored with the 1.2.0 rules; 2.0.0 through 2.0.7 give the same rule spans on this split. **Naked** is the model alone. **Residual** is passages with any gold character left unmasked. Character scores are gold vs masked characters.
|
| 132 |
|
| 133 |
| System | Mode | Whole /315 | Residual | False chars | Char P | Char R |
|
| 134 |
|---|---|---:|---:|---:|---:|---:|
|
|
|
|
| 138 |
| Fine-tuned GLiNER (previous incumbent) | naked | 263 | 28 | 15 | 99.70% | 88.80% |
|
| 139 |
| Fine-tuned GLiNER (previous incumbent) | ∪ regex | 299 | 12 | 39 | 99.30% | 97.18% |
|
| 140 |
| XLM-R epoch 5 | naked | 275 | 28 | 67 | 98.72% | 90.25% |
|
| 141 |
+
| **NERGAL 2.0.7, names off** | **∪ regex** | **303** | **9** | **80** | **98.59%** | **97.83%** |
|
| 142 |
|
| 143 |
+
Zero-shot [GLiNER 2.5](https://huggingface.co/fastino/gliner2.5-multi-v1) is not competitive, especially on non-phone PII (7/185 whole, naked). The fine-tuned GLiNER that NERGAL replaced is the more precise union (39 false characters vs 80) and the stronger phone model (128/130 whole, naked, vs 115), but it masks 4 fewer values whole and leaves 12 residual passages vs 9; XLM-R is the stronger model on other PII (160/185 vs 135). NERGAL 2.0.7 wholly masks 124/130 phones and 179/185 other PII, each under its own placeholder (`[PHONE]` / `[PII]`); exact-span precision 91.05%, recall 93.65%, F1 92.33%.
|
| 144 |
|
| 145 |
XLM-R and epoch 5 were chosen in September 2026, on the earlier gold and before the phone-policy cut. Fine-tuned GLiNER, HerBERT-large and XLM-R-large were trained on the same split; only XLM-R passed the content-preservation gate, and epoch 5 at 0.95 (seed `202609160`) was the only one of 133 epoch/threshold points that covered more than the fine-tuned GLiNER union of that time while adding no false-mask characters it did not already make. Under the 1.2.0 rules, gold and phone cut, that no longer holds: the GLiNER union makes fewer false masks (table above).
|
| 146 |
|
hybrid.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
| 1 |
{
|
| 2 |
"full_name": "Named Entity Recognition with Grounded Additive Labels",
|
| 3 |
"hub_id": "SlayerLab/NERGAL",
|
| 4 |
-
"version": "2.0.
|
| 5 |
"mode": "rules_union",
|
| 6 |
"epoch": 5,
|
| 7 |
"seed": 202609160,
|
|
|
|
| 1 |
{
|
| 2 |
"full_name": "Named Entity Recognition with Grounded Additive Labels",
|
| 3 |
"hub_id": "SlayerLab/NERGAL",
|
| 4 |
+
"version": "2.0.7",
|
| 5 |
"mode": "rules_union",
|
| 6 |
"epoch": 5,
|
| 7 |
"seed": 202609160,
|
nergal.py
CHANGED
|
@@ -17,7 +17,7 @@ import scrub_pii
|
|
| 17 |
from scrub_pii import LEGACY_PHONE_TAG, PERSON_TAG, PHONE_TAG, PII_TAG
|
| 18 |
|
| 19 |
HUB_ID = 'SlayerLab/NERGAL'
|
| 20 |
-
VERSION = '2.0.
|
| 21 |
GAPS = ['[PII_SPACE]', '[PII_BREAK]']
|
| 22 |
GAP_IDS = [250002, 250003]
|
| 23 |
BIO_LABELS = ['O', 'B-phone', 'I-phone', 'B-pii', 'I-pii']
|
|
@@ -47,6 +47,18 @@ ROLE_NOUNS = tuple(stem + _PARTICIPLE for stem in ('odwołując', 'przystępują
|
|
| 47 |
'prezes(?:a|owi|em|ie|i|ów|om|ami|ach)?', 'uczestni' + _NIK, 'pełnomocni' + _NIK, 'oferen' + _NT, 'protokolan' + _NT)
|
| 48 |
_ROLE_NOUN = re.compile(r'\W*(?:%s)_?\d*\W*' % '|'.join(ROLE_NOUNS), re.IGNORECASE)
|
| 49 |
_NUMBER = re.compile(r'\W*\d+\W*') # with the punctuation a tagged token grows to, as for _ROLE_NOUN
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 50 |
|
| 51 |
|
| 52 |
def sha(path):
|
|
@@ -224,10 +236,23 @@ def whole_digits(text, start, end):
|
|
| 224 |
return start, end
|
| 225 |
|
| 226 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 227 |
def person_spans(text, ranges):
|
| 228 |
"""Person spans from FastPDN person-token ranges: whole words, joined across spaces, tabs and no-break spaces
|
| 229 |
-
(not punctuation or line breaks); words touching a placeholder, role nouns (ROLE_NOUNS)
|
| 230 |
-
right after a skipped role noun (`Odwołujący 1`) are skipped."""
|
| 231 |
blocked = [m.span() for m in _PLACEHOLDER.finditer(text)]
|
| 232 |
merged, role_end = [], None
|
| 233 |
for start, end in sorted(expand(text, a, b) for a, b in ranges):
|
|
@@ -236,6 +261,8 @@ def person_spans(text, ranges):
|
|
| 236 |
if _ROLE_NOUN.fullmatch(text, start, end):
|
| 237 |
role_end = end
|
| 238 |
continue
|
|
|
|
|
|
|
| 239 |
if role_end is not None and _NUMBER.fullmatch(text, start, end) and not text[role_end:start].strip(' \t\u00a0'):
|
| 240 |
continue
|
| 241 |
if merged and not text[merged[-1][1]:start].strip(' \t\u00a0'):
|
|
|
|
| 17 |
from scrub_pii import LEGACY_PHONE_TAG, PERSON_TAG, PHONE_TAG, PII_TAG
|
| 18 |
|
| 19 |
HUB_ID = 'SlayerLab/NERGAL'
|
| 20 |
+
VERSION = '2.0.7'
|
| 21 |
GAPS = ['[PII_SPACE]', '[PII_BREAK]']
|
| 22 |
GAP_IDS = [250002, 250003]
|
| 23 |
BIO_LABELS = ['O', 'B-phone', 'I-phone', 'B-pii', 'I-pii']
|
|
|
|
| 47 |
'prezes(?:a|owi|em|ie|i|ów|om|ami|ach)?', 'uczestni' + _NIK, 'pełnomocni' + _NIK, 'oferen' + _NT, 'protokolan' + _NT)
|
| 48 |
_ROLE_NOUN = re.compile(r'\W*(?:%s)_?\d*\W*' % '|'.join(ROLE_NOUNS), re.IGNORECASE)
|
| 49 |
_NUMBER = re.compile(r'\W*\d+\W*') # with the punctuation a tagged token grows to, as for _ROLE_NOUN
|
| 50 |
+
# Role nouns in damaged text (2.0.7), each narrow enough that a surname which only starts like a role noun
|
| 51 |
+
# (`Odwołującyński`, `Wykonawcówna`) stays masked:
|
| 52 |
+
# cut short the stem and the start of an ending: `Odwołująceg`, `Wykonawc`, `Wykonawcam`
|
| 53 |
+
# glued after a role noun and a lowercase word of 4+ letters that the same text also uses on its own:
|
| 54 |
+
# `Odwołującywskazał`
|
| 55 |
+
# Left masked, so the role bench's guards (nergal_role_bench.count) do not move: a word glued before a role noun
|
| 56 |
+
# (`przezOdwołującego`, an `other` word there) and a number after a damaged one (`Odwołująceg 1`).
|
| 57 |
+
_ROLE_CUT = re.compile(r"\W*(?:(?:odwołując|przystępując|zamawiając|protestując|skarżąc|przewodnicząc)(?:eg|em|yc)|"
|
| 58 |
+
r"(?:wykonawc|wnioskodawc)(?:ó|am|ac)?)\W*", re.IGNORECASE)
|
| 59 |
+
_ROLE_BARE = re.compile('(?:%s)' % '|'.join(ROLE_NOUNS), re.IGNORECASE)
|
| 60 |
+
_GLUED_AFTER = re.compile(r"([a-ząćęłńóśźż]{4,})\W*")
|
| 61 |
+
_LEAD = re.compile(r'\W*')
|
| 62 |
|
| 63 |
|
| 64 |
def sha(path):
|
|
|
|
| 236 |
return start, end
|
| 237 |
|
| 238 |
|
| 239 |
+
def damaged_role_word(text, start, end):
|
| 240 |
+
"""A role noun cut short (_ROLE_CUT) or glued to a lowercase word the text also uses on its own (_GLUED_AFTER)."""
|
| 241 |
+
if _ROLE_CUT.fullmatch(text, start, end):
|
| 242 |
+
return True
|
| 243 |
+
word = _LEAD.match(text, start, end).end()
|
| 244 |
+
for cut in range(word + 1, end): # every split: `Odwołujący|miała` as well as `Odwołującym|iała`
|
| 245 |
+
tail = _GLUED_AFTER.fullmatch(text, cut, end)
|
| 246 |
+
if tail and _ROLE_BARE.fullmatch(text, word, cut) and re.search(
|
| 247 |
+
r"(?<![\w'’-])%s(?![\w'’-])" % re.escape(tail[1]), text):
|
| 248 |
+
return True
|
| 249 |
+
return False
|
| 250 |
+
|
| 251 |
+
|
| 252 |
def person_spans(text, ranges):
|
| 253 |
"""Person spans from FastPDN person-token ranges: whole words, joined across spaces, tabs and no-break spaces
|
| 254 |
+
(not punctuation or line breaks); words touching a placeholder, role nouns (ROLE_NOUNS), damaged ones
|
| 255 |
+
(damaged_role_word) and a number right after a skipped role noun (`Odwołujący 1`) are skipped."""
|
| 256 |
blocked = [m.span() for m in _PLACEHOLDER.finditer(text)]
|
| 257 |
merged, role_end = [], None
|
| 258 |
for start, end in sorted(expand(text, a, b) for a, b in ranges):
|
|
|
|
| 261 |
if _ROLE_NOUN.fullmatch(text, start, end):
|
| 262 |
role_end = end
|
| 263 |
continue
|
| 264 |
+
if damaged_role_word(text, start, end):
|
| 265 |
+
continue
|
| 266 |
if role_end is not None and _NUMBER.fullmatch(text, start, end) and not text[role_end:start].strip(' \t\u00a0'):
|
| 267 |
continue
|
| 268 |
if merged and not text[merged[-1][1]:start].strip(' \t\u00a0'):
|
test_nergal.py
CHANGED
|
@@ -43,7 +43,7 @@ class NergalTests(unittest.TestCase):
|
|
| 43 |
from nergal import GAP_IDS, GAPS, HUB_ID, RULES_SHA as PINNED, THRESHOLD, VERSION
|
| 44 |
card = json.loads((HERE / 'hybrid.json').read_text())
|
| 45 |
self.assertEqual(HUB_ID, 'SlayerLab/NERGAL')
|
| 46 |
-
self.assertEqual(VERSION, '2.0.
|
| 47 |
self.assertEqual(card['version'], VERSION)
|
| 48 |
self.assertEqual(card['eval']['union_fp'], 80)
|
| 49 |
self.assertEqual(card['eval']['rules_fp'], 24)
|
|
@@ -322,6 +322,34 @@ class NergalTests(unittest.TestCase):
|
|
| 322 |
for text in ('Odwołujący3a', 'Odwołujący__2', 'Odwołującegozarzut', 'Powodowski2'): # glued to more than a number
|
| 323 |
self.assertEqual(persons(text), [text], text)
|
| 324 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 325 |
def test_person_spans_leave_the_adverbial_participle_out(self):
|
| 326 |
from nergal import person_spans
|
| 327 |
text = 'Odwołując się do wyroku, Przystępując do sporu'
|
|
|
|
| 43 |
from nergal import GAP_IDS, GAPS, HUB_ID, RULES_SHA as PINNED, THRESHOLD, VERSION
|
| 44 |
card = json.loads((HERE / 'hybrid.json').read_text())
|
| 45 |
self.assertEqual(HUB_ID, 'SlayerLab/NERGAL')
|
| 46 |
+
self.assertEqual(VERSION, '2.0.7')
|
| 47 |
self.assertEqual(card['version'], VERSION)
|
| 48 |
self.assertEqual(card['eval']['union_fp'], 80)
|
| 49 |
self.assertEqual(card['eval']['rules_fp'], 24)
|
|
|
|
| 322 |
for text in ('Odwołujący3a', 'Odwołujący__2', 'Odwołującegozarzut', 'Powodowski2'): # glued to more than a number
|
| 323 |
self.assertEqual(persons(text), [text], text)
|
| 324 |
|
| 325 |
+
def test_person_spans_leave_role_nouns_in_damaged_text_out(self):
|
| 326 |
+
from nergal import person_spans
|
| 327 |
+
|
| 328 |
+
def persons(text, first_only=False): # every word tagged, or only the first
|
| 329 |
+
ranges = [m.span() for m in re.finditer(r"[\w'’-]+", text)]
|
| 330 |
+
return [text[s['start']:s['end']] for s in person_spans(text, ranges[:1] if first_only else ranges)]
|
| 331 |
+
cut_short = ('Odwołująceg', 'Odwołującem', 'Odwołującyc', 'Przystępująceg', 'Zamawiającem', 'Skarżącyc',
|
| 332 |
+
'Wykonawc', 'Wykonawcó', 'Wykonawcam', 'Wykonawcac', 'Wnioskodawc', 'Wnioskodawcó')
|
| 333 |
+
for word in cut_short:
|
| 334 |
+
for text in (word, word.upper(), word.lower(), f'({word})'):
|
| 335 |
+
self.assertEqual(persons(text), [], text)
|
| 336 |
+
self.assertEqual(persons('Odwołująceg Jan Nowak'), ['Jan Nowak'])
|
| 337 |
+
self.assertEqual(persons('Odwołująceg 1'), ['1']) # a number after a damaged role noun stays (role bench guard)
|
| 338 |
+
self.assertEqual(persons('Odwołujący Odwołująceg 1'), ['1']) # not right after the whole one
|
| 339 |
+
for text in ('przezOdwołującego', 'iPrzystępujący', 'zWykonawcą'):
|
| 340 |
+
self.assertEqual(persons(text), [text], text) # glued before: a role bench `other` word, left masked
|
| 341 |
+
for text in ('Odwołującywskazał termin, a Sąd wskazał termin', 'Wykonawcazłożył ofertę; Jan złożył',
|
| 342 |
+
'Odwołującymiała rację, choć nie miała', 'ODWOŁUJĄCYwskazał, wskazał', 'Odwołującywskazał. wskazał'):
|
| 343 |
+
self.assertEqual(persons(text, first_only=True), [], text) # a glued word the text also uses on its own
|
| 344 |
+
kept = ('Odwołującywskazał termin', 'Odwołującyże że', 'OdwołującyWskazał wskazał', 'OdwołującyJan',
|
| 345 |
+
'KowalOdwołujący', 'przezodwołującego', 'przezOdwołującywskazał wskazał', 'Odwołującegozarzut', 'Odwołującyego', 'Wykonawcówna ówna',
|
| 346 |
+
'Odwołującyński', 'Odwołującegg', 'Wykonawcz')
|
| 347 |
+
for text in kept: # no lone tail, a tail under 4 letters, a capital after, a capital or nothing before
|
| 348 |
+
word = re.match(r"[\w'’-]+", text)[0]
|
| 349 |
+
self.assertEqual(persons(text, first_only=True), [word], text)
|
| 350 |
+
for name in STEM_NAMES:
|
| 351 |
+
self.assertEqual(persons(f'{name} i ona', first_only=True), [name])
|
| 352 |
+
|
| 353 |
def test_person_spans_leave_the_adverbial_participle_out(self):
|
| 354 |
from nergal import person_spans
|
| 355 |
text = 'Odwołując się do wyroku, Przystępując do sporu'
|