PosterSentry: Multimodal Scientific Poster Classifier
Model Description
PosterSentry is a lightweight, CPU-optimized multimodal classifier that determines whether a PDF is a scientific poster or a non-poster (paper, proceedings, newsletter, abstract book, etc.).
Part of the quality control pipeline for posters.science, a platform for making scientific conference posters Findable, Accessible, Interoperable, and Reusable (FAIR).
Developed by the FAIR Data Innovations Hub at the California Medical Innovations Institute (CalMI²).
Version
| Version | Date | Notes |
|---|---|---|
| 1.4.0 | 2026-10-28 | Trained on the 3,235-document corpus in which every document carries a CC0, CC-BY, or equivalent permissive license, with first-page text read by the poster2json layout engine (vendored in the package). Held-out accuracy 94.4% (95% CI 92.0 to 96.2), nested out-of-fold 94.4%; deployment 80.4% posters. |
| 1.3.0 | 2026-09-21 | License-screened retrain: 84 documents whose source-repository licenses do not permit redistribution were removed, and the head was retrained on the resulting 3,298-document corpus. Held-out accuracy 93.1%. |
| 1.2.0 | 2026-09-01 | Stacked head: the 512-d text embedding is scored by a stage-one logistic regression whose poster probability becomes a single text-score feature for the final classifier over [text_score + 15 visual + 15 structural] = 31 features. Held-out accuracy 92.9%. Head weights now store a zero first column so softmax reproduces sklearn's sigmoid exactly; earlier heads doubled the logit, which sharpened reported confidences (decisions at 0.5 were unaffected). Flat heads remain loadable by the package. |
| 1.1.0 | 2026-08-28 | Feature extraction moved off the AGPL-licensed PyMuPDF to pdfplumber (text and structure) and pypdfium2 (page rendering), both permissively licensed, so the whole stack is MIT-compatible. The head was retrained on the re-extracted features; held-out accuracy is 89.2%. |
| 1.0.0 | 2026-08-18 | Head trained on the human-validated corpus: 3,381 documents labeled by a three-reviewer survey (Krippendorff's alpha 0.79) with blinded adjudication of the 439 contested documents. |
Earlier unversioned weights (April 2026) were trained on heuristically labeled data and are superseded.
Related Models & Tools
| Resource | Description | Link |
|---|---|---|
| PosterSentry | Multimodal poster classifier (this model) | fairdataihub/poster-sentry |
| poster-sentry | Installable classifier package | GitHub |
| poster-sentry-training | Training code and replication | GitHub |
| poster-sentry-training-data | Human-validated training dataset (3,235 samples, with text and feature values) | HuggingFace |
| poster-sentry-evaluation-paper-code | Reproducible analysis for the paper | GitHub |
| Llama-3.1-8B-Poster-Extraction | Poster → structured JSON extraction | fairdataihub/Llama-3.1-8B-Poster-Extraction |
| poster2json | Python library for poster extraction | PyPI · Docs · GitHub |
| poster-json-schema | DataCite-based poster metadata schema | GitHub |
| Platform | posters.science | posters.science |
Pipeline Position
PosterSentry sits at the front of the posters.science pipeline. It screens incoming PDFs before the expensive Llama-based extraction:
PDF Input
│
▼
┌──────────────┐ ┌───────────────────────────────────┐ ┌──────────────┐
│ PosterSentry │ ──► │ Llama-3.1-8B-Poster-Extraction │ ──► │ poster2json │
│ (classify) │ │ (extract structured metadata) │ │ (validate) │
└──────────────┘ └───────────────────────────────────┘ └──────────────┘
poster? ✓ raw text → JSON schema FAIR output
Architecture
Two-stage (stacked) logistic regression. Stage 1 scores the 512-d text embedding alone; its poster probability becomes the single text_score feature of stage 2, which classifies 31 features:
| Channel | Features | Dimension | Signal |
|---|---|---|---|
| Text score | Stage-1 LogisticRegression over the model2vec (potion-base-32M) embedding | 1 | Semantic content, summarized |
| Visual | Color stats, edge density, FFT spatial complexity, whitespace | 15 | Visual layout |
| Structural | Page count, area, font diversity, text blocks, density | 15 | PDF geometry |
Stage 2 is trained on inner 5-fold out-of-fold text scores so it never sees an in-sample-optimistic text score. Both stages (weights and scalers) live in one numpy .npz head (about 20 KB). Inference is pure numpy, no torch required at prediction time. Every stage-2 coefficient is a named, interpretable feature; the raw 542-d concatenation of earlier releases underperformed it by about six points out of fold because the 512 standardized text dimensions crowded out the reliable engineered signal.
PDF backend (speed vs license)
Feature extraction reads PDFs through a selectable backend:
| Backend | Libraries | License | Speed | Notes |
|---|---|---|---|---|
pdfplumber (default) |
pdfplumber + pypdfium2 | MIT + BSD-3-Clause | ~2.0 s/doc | The stack the released model is trained on; reproduces the paper. |
pymupdf |
PyMuPDF | AGPL-3.0 | ~1.2 s/doc (about 1.75x faster) | Faster, but AGPL. Install with pip install poster-sentry[pymupdf]. Its extracted features differ slightly, so retrain with this backend for best accuracy or accept minor prediction drift. |
Select the backend with --backend, the backend= constructor argument, or the POSTER_SENTRY_BACKEND environment variable. The default is pdfplumber so results match this model.
On a 40-document corpus sample, the dominant cost in the default stack is pdfplumber's pure-Python text and structure parsing (about 780 ms and 760 ms per document); page rendering with pypdfium2 adds about 490 ms. A pypdfium2-based text variant cuts the text step to about 75 ms and reaches roughly 1.3 s/doc, but it changes the extracted text, so it would require its own retrain; pymupdf is the recommended fast path when AGPL is acceptable.
Performance
Evaluated against human-validated labels (three-reviewer survey with blinded adjudication of contested documents):
| Metric | Value |
|---|---|
| Held-out accuracy (486 documents) | 94.4% (95% CI 92.0 to 96.2) |
| Macro F1 (held-out) | 0.944 |
| Precision / Recall / F1 (poster) | 0.953 / 0.934 / 0.943 |
| Precision / Recall / F1 (non-poster) | 0.936 / 0.955 / 0.945 |
| Nested out-of-fold accuracy (all 3,235 documents, 5-fold) | 94.4% |
| Panel reliability with the model as a fourth rater | alpha 0.793 -> 0.798 (unchanged) |
| Calibration (out-of-fold) | Brier 0.045, ECE 0.016 |
| Classifier speed | under one second per file on a CPU (PDF parsing is the bottleneck) |
Errors concentrate where the human panel itself divided: out-of-fold agreement is 96.7% on documents the panel rated unanimously and 79.7% on documents decided two to one.
Top Features by Importance
Standardized logistic regression coefficients of the trained head (positive pushes toward poster):
| Rank | Feature | Coefficient | Signal |
|---|---|---|---|
| 1 | page_count |
-3.47 | More pages pushes away from poster |
| 2 | size_per_page_kb |
+2.61 | Dense, high-resolution single pages |
| 3 | line_count |
+1.88 | Posters pack many short text lines |
| 4 | file_size_kb |
-1.76 | Multi-page documents are bigger overall |
| 5 | mean_g |
+1.21 | Colorful, non-white pages |
| 6 | is_landscape |
+1.06 | Many posters are landscape |
| 7 | white_space_ratio |
-0.92 | Sparse, white pages push toward non-poster |
| 8 | page_area_sqin |
-0.83 | Very large page areas push toward non-poster |
| 9 | edge_density |
+0.78 | Visually busy layouts |
| 10 | color_diversity |
+0.73 | Many distinct colors |
Every coefficient of the final classifier is a named feature; the whole text channel enters as the single text_score (+0.60, rank 11 of 31). First-page text is read with the poster2json layout engine, which rebuilds the reading order of multi-column layouts.
Training Data
Trained on 3,235 documents with human-validated labels, zero synthetic data:
| Class | Count | Label provenance |
|---|---|---|
| Poster | 1,605 | Three-reviewer survey; unanimous panel label or blinded adjudication |
| Non-poster | 1,630 | Three-reviewer survey; unanimous panel label or blinded adjudication |
Three reviewers independently rated 3,414 candidate documents, each released by its depositor under CC0, CC-BY, or an equivalent permissive license (inter-rater Krippendorff's alpha 0.79); the 420 documents without a unanimous panel were settled in a blinded adjudication review. After removing 172 near-duplicate documents and 7 with unavailable PDFs, the remaining 3,235 form the training corpus. Applied to the full corpus of 30,139 readable repository PDFs labeled as posters, PosterSentry classifies 80.4% as posters: roughly one in five records labeled as posters is something else.
Training data: fairdataihub/poster-sentry-training-data
Usage
Python API
from poster_sentry import PosterSentry
sentry = PosterSentry()
sentry.initialize()
# Classify a PDF (uses text + visual + structural features)
result = sentry.classify("document.pdf")
print(f"Is poster: {result['is_poster']}, Confidence: {result['confidence']:.2f}")
# {'is_poster': True, 'confidence': 0.97, 'path': 'document.pdf'}
# Batch classification
results = sentry.classify_batch(["poster1.pdf", "paper.pdf", "newsletter.pdf"])
Installation
pip install git+https://github.com/fairdataihub/poster-sentry.git
# Or install from source
git clone https://github.com/fairdataihub/poster-sentry.git
cd poster-sentry
pip install -e ".[dev]"
Training
The as-run training script is scripts/train_poster_sentry.py (also at poster-sentry-training). It needs the source PDFs; the training dataset holds the extracted text and every feature value, enough to refit the head without them. Feature extraction for the 3,235 training PDFs takes about 13 minutes on 12 CPU processes (PDF parsing and rendering are the bottleneck, not the classifier).
Model Specifications
| Attribute | Value |
|---|---|
| Embedding backbone | minishlab/potion-base-32M (model2vec StaticModel) |
| Embedding dimension | 512 |
| Visual features | 15 (color, edge, FFT, whitespace) |
| Structural features | 15 (page geometry, fonts, text blocks) |
| Final-classifier input dimension | 31 (text score + 15 visual + 15 structural) |
| Classifier | Two-stage (stacked) LogisticRegression + StandardScaler |
| Head file size | 20 KB (.npz, both stages) |
| Precision | float32 |
| GPU required | No (CPU-only) |
| License | MIT |
System Requirements
- CPU: Any modern CPU (no GPU needed)
- RAM: ≥4GB
- Python: ≥3.10
- Dependencies: numpy, model2vec, scikit-learn, pdfplumber, pypdfium2, Pillow
Citation
@software{poster_sentry_2026,
title = {PosterSentry: Multimodal Scientific Poster Classifier},
author = {O'Neill, Jamey and Portillo, Dorian and Zeinali, Nahid and Soundarajan, Sanjay and Blake, Gerard and Sarin, Parth and Buttrick, Adam and Patel, Bhavesh},
year = {2026},
version = {1.4.0},
url = {https://huggingface.co/fairdataihub/poster-sentry},
note = {Part of the posters.science initiative}
}
License
This model is released under the MIT License.
Acknowledgments
- FAIR Data Innovations Hub at California Medical Innovations Institute (CalMI²)
- posters.science platform
- MinishLab for the model2vec embedding backbone
- HuggingFace for model hosting infrastructure
- Funded by The Navigation Fund (10.71707/rk36-9x79), "Poster Sharing and Discovery Made Easy"
- Downloads last month
- 40