Pin usage example to the verified model commit

2426e83 verified 8 days ago

4.72 kB

---
language:
- en
license: mit
library_name: sentence-transformers
base_model: nomic-ai/CodeRankEmbed
pipeline_tag: feature-extraction
tags:
- python
- fault-localization
- code-retrieval
- experimental
- safetensors
---
# FailureLens: experimental fault-localization encoder

This is a **trained experimental model**, rebuilt on Hugging Face Jobs from
the FailureLens Stage 2 recipe. Given a Python traceback and candidate function
source texts, it supplies embeddings for cosine-similarity ranking.

**Independent held-out evaluation (Gate C) is pending. No improvement over the
shipped baseline or the base encoder has been established. This release is for
experimentation, not evidence of production readiness or agent adoption.**

## Use

Install PyTorch 2.11.0, transformers 5.22.0, sentence-transformers 6.2.0,
safetensors 0.8.0 and einops 0.8.1. The model contains NomicBERT custom Python
code from the pinned base, with a documented transformers-5 attention-mask fix.
Review that code before enabling `trust_remote_code`. This example pins the
model revision and uses only safetensors weights.

```python
from huggingface_hub import HfApi
from sentence_transformers import SentenceTransformer
revision = "ec8ffde17b5cf5a5bf6f206864fdbe985f373440"
model = SentenceTransformer(
    "LevelCapTech/failurelens",
    revision=revision,
    trust_remote_code=True,
    device="cpu",
    model_kwargs={"use_safetensors": True},
)
error_log = "ZeroDivisionError: division by zero"
candidates = [
    "def average(values): return sum(values) / len(values)",
    "def greet(name): return 'Hello ' + name",
]
# Stage 2 uses the stripped traceback directly, without a query prefix.
query = model.encode([error_log.strip()], normalize_embeddings=True)
code = model.encode(candidates, normalize_embeddings=True)
scores = (query @ code.T)[0]
ranked_indices = sorted(range(len(candidates)), key=lambda i: (-float(scores[i]), i))
print([(i, float(scores[i])) for i in ranked_indices])
```

Scores are similarities, not probabilities. Callers supply the candidate pool;
this encoder does not locate files, invent line numbers, or generate fixes.
The current FailureLens library's generic `from_pretrained` path has a different
query transform and a restricted custom-code allowlist. The standalone example
above is the supported interface for this experimental release.

## Training and limitations

- Base: `nomic-ai/CodeRankEmbed` at `3c4b60807d71f79b43f3c4363786d9493691f8b1` (MIT).
- 136,731,648 parameters; only the final encoder block (9,440,256 parameters)
  is fine-tuned, with per-triplet InfoNCE and cosine similarity.
- 12 private, sanitized synthetic examples: 6 train from `pallets/click` and
  6 validation from `python-attrs/attrs`; source repositories are disjoint.
- Seed 42, 3 epochs, batch size 2, AdamW, learning rate 2e-5, weight decay 0.01,
  temperature 0.07, cosine learning-rate schedule. CLS pooling is retained.
- Effective model sequence limit: 8192 tokens.
  The legacy config's `max_length=512` is not applied by the training function;
  the effective limit and observed feature lengths are recorded in provenance.
- Built on Tesla T4. This cloud
  rebuild has its own checksum; it is not claimed bit-identical to local v3.
- Validation losses only reflect this tiny synthetic set. No natural-bug,
  multilingual, latency-budget, or independent accuracy claim is made.
- Baseline retrieval can omit the actual cause; reranking cannot recover an
  omitted function. Python traceback support in the library remains limited.

## Reproducibility and distribution

The Job fetched pinned public source and base weights, trained, assembled and
tested the model, and uploaded the weights directly to the Hub. The submitting
connection sent only this build script and approximately 6 KB of compressed
training features as an encrypted Job secret. Private records, features,
optimizer checkpoints and held-out labels are not distributed here.

- Job: https://huggingface.co/jobs/LevelCapTech/6ae7d7a8c78863f9c62ff0e8
- Training source: https://github.com/LevelCapTech/FailureLens/tree/2985bfcfbf5bf79bc3b3b0589284f6a4ca481970
- Weights SHA-256: `8e8fb971f8c1dce22ae29ea4f682cb8387790ad608e06d3800b62d3413a0a9e4`
- Data SHA-256: `74909f51b588c311de798bee29b748cfc1ead65fd2948053c8f3a1c729e465c1`
- Exact dependencies, training history, code hashes and smoke checks:
  [failurelens_provenance.json](failurelens_provenance.json).

The weights are distributed under MIT, as declared by the base model.
Upstream code retains its copyright headers and license notices; see
[LICENSE](LICENSE) and [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md).
FailureLens's repository source and build script are Apache-2.0.

	---
	language:
	- en
	license: mit
	library_name: sentence-transformers
	base_model: nomic-ai/CodeRankEmbed
	pipeline_tag: feature-extraction
	tags:
	- python
	- fault-localization
	- code-retrieval
	- experimental
	- safetensors
	---

	# FailureLens: experimental fault-localization encoder

	This is a trained experimental model, rebuilt on Hugging Face Jobs from
	the FailureLens Stage 2 recipe. Given a Python traceback and candidate function
	source texts, it supplies embeddings for cosine-similarity ranking.

	**Independent held-out evaluation (Gate C) is pending. No improvement over the
	shipped baseline or the base encoder has been established. This release is for
	experimentation, not evidence of production readiness or agent adoption.**

	## Use

	Install PyTorch 2.11.0, transformers 5.22.0, sentence-transformers 6.2.0,
	safetensors 0.8.0 and einops 0.8.1. The model contains NomicBERT custom Python
	code from the pinned base, with a documented transformers-5 attention-mask fix.
	Review that code before enabling `trust_remote_code`. This example pins the
	model revision and uses only safetensors weights.

	```python
	from huggingface_hub import HfApi
	from sentence_transformers import SentenceTransformer

	revision = "ec8ffde17b5cf5a5bf6f206864fdbe985f373440"
	model = SentenceTransformer(
	"LevelCapTech/failurelens",
	revision=revision,
	trust_remote_code=True,
	device="cpu",
	model_kwargs={"use_safetensors": True},
	)
	error_log = "ZeroDivisionError: division by zero"
	candidates = [
	"def average(values): return sum(values) / len(values)",
	"def greet(name): return 'Hello ' + name",
	]
	# Stage 2 uses the stripped traceback directly, without a query prefix.
	query = model.encode([error_log.strip()], normalize_embeddings=True)
	code = model.encode(candidates, normalize_embeddings=True)
	scores = (query @ code.T)[0]
	ranked_indices = sorted(range(len(candidates)), key=lambda i: (-float(scores[i]), i))
	print([(i, float(scores[i])) for i in ranked_indices])
	```

	Scores are similarities, not probabilities. Callers supply the candidate pool;
	this encoder does not locate files, invent line numbers, or generate fixes.
	The current FailureLens library's generic `from_pretrained` path has a different
	query transform and a restricted custom-code allowlist. The standalone example
	above is the supported interface for this experimental release.

	## Training and limitations

	- Base: `nomic-ai/CodeRankEmbed` at `3c4b60807d71f79b43f3c4363786d9493691f8b1` (MIT).
	- 136,731,648 parameters; only the final encoder block (9,440,256 parameters)
	is fine-tuned, with per-triplet InfoNCE and cosine similarity.
	- 12 private, sanitized synthetic examples: 6 train from `pallets/click` and
	6 validation from `python-attrs/attrs`; source repositories are disjoint.
	- Seed 42, 3 epochs, batch size 2, AdamW, learning rate 2e-5, weight decay 0.01,
	temperature 0.07, cosine learning-rate schedule. CLS pooling is retained.
	- Effective model sequence limit: 8192 tokens.
	The legacy config's `max_length=512` is not applied by the training function;
	the effective limit and observed feature lengths are recorded in provenance.
	- Built on Tesla T4. This cloud
	rebuild has its own checksum; it is not claimed bit-identical to local v3.
	- Validation losses only reflect this tiny synthetic set. No natural-bug,
	multilingual, latency-budget, or independent accuracy claim is made.
	- Baseline retrieval can omit the actual cause; reranking cannot recover an
	omitted function. Python traceback support in the library remains limited.

	## Reproducibility and distribution

	The Job fetched pinned public source and base weights, trained, assembled and
	tested the model, and uploaded the weights directly to the Hub. The submitting
	connection sent only this build script and approximately 6 KB of compressed
	training features as an encrypted Job secret. Private records, features,
	optimizer checkpoints and held-out labels are not distributed here.

	- Job: https://huggingface.co/jobs/LevelCapTech/6ae7d7a8c78863f9c62ff0e8
	- Training source: https://github.com/LevelCapTech/FailureLens/tree/2985bfcfbf5bf79bc3b3b0589284f6a4ca481970
	- Weights SHA-256: `8e8fb971f8c1dce22ae29ea4f682cb8387790ad608e06d3800b62d3413a0a9e4`
	- Data SHA-256: `74909f51b588c311de798bee29b748cfc1ead65fd2948053c8f3a1c729e465c1`
	- Exact dependencies, training history, code hashes and smoke checks:
	[failurelens_provenance.json](failurelens_provenance.json).

	The weights are distributed under MIT, as declared by the base model.
	Upstream code retains its copyright headers and license notices; see
	[LICENSE](LICENSE) and [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md).
	FailureLens's repository source and build script are Apache-2.0.