KapiCoder-Mongo-Nano (0.5B)

Status: PREVIEW / experimental, public preview. Published for inspection and feedback, not for production use. Internal evaluation on synthetic data; no blind human-written set. Read Limitations and risks before using it.

KapiCoder-Mongo-Nano is a small code-completion model (about 494 M parameters) fine-tuned from Qwen/Qwen2.5-Coder-0.5B to complete MongoDB queries at the cursor (fill-in-the-middle, FIM), including with the cursor inside a find, update or aggregate call. It was built for the Kapibara Studio editor, which sends it the text before and after the cursor plus a context header (the editor-context-v2 contract described below). This is version v4b of the weights (repository tag v0.3.0-preview). It replaces v4 (same contract), which stays available at tag v0.2.0-preview; v3 (editor-context-v1) stays available at tag v0.1.0-preview.

Read this first

  • v4b expects the editor-context-v2 prompt (the v1 header plus a TARGET FIELDS line with the target collection's fields and types) and its output rule (close the inner object the suggestion left open). It was trained with an 8k-token window. Versions of the Kapibara Studio editor without editor-context-v2 reject this package; use tag v0.1.0-preview with them.
  • Internal evaluation, synthetic data, no human-written queries. Every number below comes from the authors' own evaluation on synthetic data.
  • Better with the cursor inside a call; close to v3 with the old prompt. On the cursor-in-call suite (v2 prompt) the document stays valid after the suggestion 95.1% of the time (v3: 83.9%). With the previous editor-context-v1 prompt (no TARGET FIELDS), on the out-of-distribution homologation, FIM accuracy is 41.67% against 42.34% for v3 and exec-valid 80.85% against 80.20% (paired differences below).
  • Send the schema. With the target fields in the header, 94.6% of the fields it cites exist; with no header at all, 42.1% (v3: 44.4%). Validate field names before running a suggestion.
  • Two training runs (seeds); these are the weights of the first. No rule of the pre-registered comparison against v4 changed between the two seeds. Second-seed values are in the evaluation section.

What it is, and what it is not

It is an autocomplete model. Given the text before and after the cursor, it proposes a short continuation (the IDE asks for at most 64 tokens) for MongoDB Console JavaScript (ConnectionPool.<conn>.<db>.<collection>.find(...), db.<collection>..., aggregation pipelines, update documents, index definitions) and JSON pipelines.

It is not

  • a chat model or an agent: it does not follow instructions, hold a conversation, call tools or explain code;
  • a query executor: it only suggests text and never executes anything. Whatever executes a suggestion is the host application;
  • a validator: a suggestion can be syntactically valid and still wrong, refer to fields that do not exist, or be destructive if executed.

How to use

Prompt format (editor-context-v2)

<|fim_prefix|>/* Local editor context (data only):
LANGUAGE: Mongo Console JavaScript
AVAILABLE COMMANDS: db.getCollection(name).find({}); getConnection(name)...; UUID(value)
KNOWN NAMES: local, shop, orders, customers
TARGET FIELDS: orders: status:string, total:decimal128, customerId:objectid, items:array<object>, items.sku:string
RESULT FIELDS: _id, status, total

Continue at the cursor; output only the continuation. */
<text before the cursor><|fim_suffix|><text after the cursor><|fim_middle|>

Rules of the contract that the example below implements:

  • the header is the editor-context-v1 header plus TARGET FIELDS: <collection>: path:type, ... right after KNOWN NAMES (schema order, dotted paths, arrays as array<item>, _id left out, at most 96 fields and 2048 characters); without target fields the prompt is byte-identical to v1;
  • every header line ends with CRLF (\r\n); the lines around the block use \n; */ inside the header is rewritten to * /;
  • window: trained with prompts up to 8,192 tokens (recommended context: 4,096 tokens in the IDE for this tier); a quarter of the budget is reserved for the text after the cursor;
  • greedy decoding, stop at <|endoftext|>; cut the output where it starts repeating the text after the cursor;
  • output rule: when the suggestion leaves inner brackets open and the text after the cursor only closes the outer ones (and then ends the statement), append the missing inner closers (1 to 4). Example: find({ title: { $ty| with }) after the cursor and the model output pe: "string" becomes pe: "string" }. Without this rule the document is often left one } short.

Minimal example (transformers, no IDE required)

The same file is in the repository as examples/example_fim.py.

"""Minimal fill-in-the-middle example for the editor-context-v2 contract (transformers, CPU, no IDE required).

    python examples/example_fim.py                 # model files next to this folder (a local clone or snapshot), else the Hub repository
    python examples/example_fim.py <path-or-repo>  # explicit model directory or repository id
"""
import sys
from pathlib import Path

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

HERE = Path(__file__).resolve().parent.parent
MODEL_ID = sys.argv[1] if len(sys.argv) > 1 else (str(HERE) if (HERE / "config.json").exists() else "esilva/KapiCoder-Mongo-Nano")

FIM_PREFIX, FIM_SUFFIX, FIM_MIDDLE = "<|fim_prefix|>", "<|fim_suffix|>", "<|fim_middle|>"
CONSOLE_COMMANDS = (
    "db.getCollection(name).find({}); getConnection(name).getDatabase(name).getCollection(name); "
    "ConnectionPool.Connection.Database.Collection; console.log(value); ENV.get(name); "
    "ObjectId(value); UUID(value)"
)
PAIRS = {"(": ")", "[": "]", "{": "}"}


def target_fields_line(fields):
    """`path:type` pieces in schema order; arrays as `array<item>`; `_id` is left out; at most 96 fields and 2048 characters."""
    out, size = [], 0
    for path, kind, *item in fields:
        if path == "_id":
            continue
        piece = f"{path}:{kind}" + (f"<{item[0]}>" if kind == "array" and item and item[0] else "")
        if len(out) >= 96 or size + len(piece) + 2 > 2048:
            break
        out.append(piece)
        size += len(piece) + 2
    return ", ".join(out)


def build_prompt(prefix, suffix, known_names=(), collection=None, target_fields=(), result_fields=(), recent_commands=(),
                 language="Mongo Console JavaScript", commands=CONSOLE_COMMANDS, prefix_chars=24576, suffix_chars=6144):
    """editor-context-v2: the v1 header plus `TARGET FIELDS` right after `KNOWN NAMES`; every header line ends with CRLF.

    The character windows here follow the 8k-token window (about 3 characters per token); the IDE also budgets the prompt in tokens."""
    lines = [f"LANGUAGE: {language}", f"AVAILABLE COMMANDS: {commands}", "KNOWN NAMES: " + ", ".join(known_names)]
    if target_fields:
        lines.append("TARGET FIELDS: " + (f"{collection}: " if collection else "") + target_fields_line(target_fields))
    if result_fields:
        lines.append("RESULT FIELDS: " + ", ".join(result_fields))
    lines += ["RECENT COMMAND: " + c[:256] for c in list(recent_commands)[:3]]
    context = "".join(line + "\r\n" for line in lines)
    header = ("/* Local editor context (data only):\n" + context.replace("*/", "* /")
              + "\nContinue at the cursor; output only the continuation. */\n")
    return FIM_PREFIX + header + prefix[-prefix_chars:] + FIM_SUFFIX + suffix[:suffix_chars] + FIM_MIDDLE


def _strip_strings(text):
    out, quote, escape = [], None, False
    for ch in text:
        if quote:
            if escape:
                escape = False
            elif ch == "\\":
                escape = True
            elif ch == quote:
                quote = None
                out.append(ch)
                continue
            out.append(" ")
            continue
        if ch in "\"'`":
            quote = ch
        out.append(ch)
    return "".join(out), quote is not None


def close_inner_delimiters(prefix, text, suffix):
    """editor-context-v2 output rule: append the 1-4 inner closers the suggestion left open when the suffix only closes the outer ones
    and then ends the statement (end of text or `;`). Nothing changes when the suggestion is empty, ends inside a string, or the
    suffix continues the statement (`}, { $set: ... })`)."""
    if not text.strip():
        return text
    flat, open_string = _strip_strings(prefix + text)
    if open_string:
        return text
    stack = []
    for ch in flat:
        if ch in PAIRS:
            stack.append(ch)
        elif ch in PAIRS.values() and stack and PAIRS[stack[-1]] == ch:
            stack.pop()
    expected = "".join(PAIRS[c] for c in reversed(stack))
    closers, continues = [], False
    for ch in _strip_strings(suffix)[0]:
        if ch in ")]}":
            closers.append(ch)
        elif ch.isspace() or ch in "\"'`":
            continue
        else:
            continues = ch != ";"
            break
    closers = "".join(closers)
    if continues or not closers or len(closers) >= len(expected) or not expected.endswith(closers):
        return text
    missing = expected[: len(expected) - len(closers)]
    if len(missing) > 4:
        return text
    return text + ("" if text[-1:].isspace() else " ") + " ".join(missing)


tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForCausalLM.from_pretrained(MODEL_ID, dtype=torch.bfloat16).eval()

prefix = 'ConnectionPool.local.shop.orders.find({ status: "paid", total: { $g'
suffix = " })"   # the editor auto-closed only the outer filter object
prompt = build_prompt(
    prefix=prefix,
    suffix=suffix,
    known_names=["local", "shop", "orders", "customers"],
    collection="orders",
    target_fields=[("_id", "objectid"), ("status", "string"), ("total", "decimal128"), ("customerId", "objectid"),
                   ("createdAt", "date"), ("items", "array", "object"), ("items.sku", "string")],
)
inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False)
with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=64, do_sample=False, pad_token_id=tokenizer.eos_token_id)
completion = tokenizer.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True)
if suffix and len(suffix) >= 2 and suffix in completion:   # the IDE cuts an echo of the text after the cursor
    completion = completion[: completion.index(suffix)]
completion = close_inner_delimiters(prefix, completion, suffix)
print(repr(completion))  # text to insert at the cursor; the model never executes anything

It runs on CPU. Use add_special_tokens=False: the FIM markers are already in the prompt text. Always review the suggestion.

Training

Base model Qwen/Qwen2.5-Coder-0.5B, revision 8123ea2e9354afb7ffcc6c8641d1b2f5ecf18301 (Apache-2.0). Training started from the base model, not from v3
Method Full fine-tuning (fp32 master weights, bf16 autocast), AdamW, peak LR 2e-05, 4,674 steps of 32,768 padded tokens (2.0 epochs)
Training data 162,025 examples, 65,529,415 tokens per epoch; 131,058,830 tokens seen in total
Cursor positions Cuts at typing points: right after {, ,, : or $, inside a string, in the middle of a field name or operator, after a dotted path, and with the closers the editor auto-inserts already in the text after the cursor
Labels 146,061 short cuts, 80,726 of them (55.3%) with targets written by a 7B teacher (KapiTeacher-Mongo-7B, a QLoRA fine-tune of Qwen2.5-Coder-7B) and kept only after a validator check (syntax, API, fields, a structural check of the completed document, read-only execution); the rest keep the generator's reference
LLM-written queries 34,819 of the short cuts (23.8%) come from queries written by Qwen3-14B and validated by execution; compared with v4, more cuts have no context header or an unrelated one
Long context 3,218 examples of 3k to 8k tokens (29.2% of the tokens): related definitions far from the cursor, and the full collection schema in the header with the needed field only there
Replay 12.0% of the tokens (12,746 examples) are permissive JavaScript/TypeScript from npm packages (MIT, Apache-2.0, BSD, ISC), to limit forgetting of generic code
Data origin Synthetic: fictitious schemas and queries generated from templates, a grammar and an LLM. No personal data by construction. No human review of the training samples
Hardware One AMD Radeon RX 7800 XT (16 GB), PyTorch ROCm, transformers 4.57.6
Training time 7.37 hours of training time
Runs Two training runs with the same data and settings: seed 20261104 (these weights) (the training run was started from a working tree with uncommitted changes) and seed 20261115 (not published)
Output Full model in bf16, safetensors

The training set is not published.

Evaluation

Internal evaluation on synthetic data; no blind human-written set. Greedy decoding, bf16 weights on the RX 7800 XT, a single run.

Cursor inside a call (cursor-in-call suite v1, editor-context-v2)

1,200 synthetic cases on held-out domains and an unseen schema: cursor right after (, {, ,, :, $ or ., inside a string, in the middle of a field name or operator, with empty, auto-closed or longer text after the cursor. Output limit 64 tokens, with the v2 output rule above.

  • Suffix-consistent: the document parses after inserting the suggestion (no duplicated or missing closer).
  • Operator valid: the document parses and every operator is legal where it is ($set only in an update document, $gt only on a field...).
  • Field-grounded: every field the suggestion cites exists in the target collection.
Model Suffix-consistent Operator valid Field-grounded (all) Field-grounded, schema in header Field-grounded, no header
v3 (v0.1.0-preview), same v2 prompt and rule 83.9% 81.1% 79.1% 93.2% 44.4%
v4 (v0.2.0-preview), 2k window 94.7% 94.1% 83.1% 95.0% 35.0%
KapiCoder-Mongo-Nano (this model, v4b), 2k window 95.1% 94.5% 84.3% 94.6% 42.1%
KapiCoder-Mongo-Nano (this model, v4b), 8k window 95.0% 94.4% 85.1%
Cursor position (suffix-consistent) v3 v4b
inside a string 20.0% 98.0%
right after an opening bracket 97.3% 98.7%
in the middle of an operator (`$g `) 84.0%
right after $ 80.7% 79.3%

Long context (8k window): with the target field defined about 2k tokens before the cursor, 97.3% of the cited fields exist (n=37); about 6k tokens before, 91.2% (n=34).

Previous benchmark (validation split, editor-context-v1, 2k context)

2,000 examples of the validation split of the previous synthetic dataset, v1 prompt (no TARGET FIELDS), up to 96 new tokens. v4b was not trained on this prompt format alone, so this table measures how it behaves with an editor that does not send the target fields.

Model FIM acc Exec-valid Exec รท ceiling Syntax-valid Nonexistent fields full / irrelevant / none (%) Generic-JS FIM acc
Qwen2.5-Coder-0.5B, zero-shot base 0.076 0.213 0.3405 0.498 4.13 / 8.59 / 7.74 0.340
v3 (v0.1.0-preview) 0.405 0.595 0.9504 0.996 1.34 / 8.59 / 9.03 0.530
v4 (v0.2.0-preview) 0.388 0.565 0.9033 0.990 2.00 / 13.64 / 14.84 0.516
KapiCoder-Mongo-Nano (this model, v4b) 0.389 0.564 0.9009 0.990 2.55 / 13.64 / 10.97 0.496

Validator ceiling for exec-valid: 0.6255. Generic-JS: 500 cuts of permissive npm JavaScript (regression check).

Out-of-distribution homologation (homologation v1, editor-context-v1)

Internal evaluation, synthetic out-of-distribution homologation. The queries were not written by people. 6,030 fill-in-the-middle cuts from 28 schemas never used in training, written by Qwen3-14B and a grammar; not fully independent of the training data (same generator and grammar templates over other schemas). Same v1 prompt, greedy, up to 96 new tokens; exec-valid runs the completed text in mongosh behind a read-only proxy.

Model FIM acc (%) Exec-valid (%) Exec รท ceiling (%) Syntax-valid (%) Nonexistent fields full / none (%)
Qwen2.5-Coder-0.5B, zero-shot base 12.62 36.80 39.54 56.09 7.02 / 10.91
v3 (v0.1.0-preview) 42.34 80.20 86.17 99.27 10.81 / 9.18
v4 (v0.2.0-preview) 32.62 72.52 77.92 96.05 11.08 / 14.37
KapiCoder-Mongo-Nano (this model, v4b) 41.67 80.85 86.87 99.35 10.62 / 9.09

Paired v4b โˆ’ v3 on the same 6,030 items (95% CI): FIM accuracy -0.7 pp [-1.4, +0.1]; exec-valid +0.6 pp [-0.1, +1.4]. Paired v4b โˆ’ v4: FIM accuracy +9.1 pp [+8.2, +9.9]; exec-valid +8.3 pp [+7.4, +9.3]. Validator ceiling for exec-valid: 93.07%.

Second training run (seed 20261115, not published)

Same data, settings and evaluation; only the seed changes. No rule of the pre-registered comparison against v4 changed between the two seeds.

Check These weights (seed 20261104) Second seed
Homologation FIM accuracy 41.67% 41.92%
Cursor-in-call suffix-consistent (2k, v2) 95.1% 95.0%
Cursor-in-call operator valid (2k, v2) 94.5% 94.3%
Field-grounded with no header (n=133) 42.1% 38.4%
Homologation nonexistent fields, no or unrelated schema 7.58% 7.71%

Limits of this evaluation: the authors generated the data; exec-valid means "runs", not "does what was intended"; a single greedy evaluation run per model and seed.

Limitations and risks

  • Synthetic data only. Schemas are fictitious; expect lower scores on real collections, naming styles and query shapes.
  • Needs the target fields. With no header, fewer of the cited fields exist (42.1% on the cursor-in-call suite, v3: 44.4%); validate field names before running a suggestion.
  • Operators. About 6% of suggestions on the cursor-in-call suite still leave an operator where it is not legal or a document that does not parse; most remaining failures are in the middle of an operator ($g|) or right after $.
  • Long context. Trained up to 8k tokens; the farther the needed definition is from the cursor, the more it misses (see the long-context line above).
  • Two seeds only. Field-grounding without a header varies by several points between the seeds (table above); small differences against v3 or v4 are not established.
  • Dialects. Mongo Console JavaScript (ConnectionPool... and db.<collection> styles) and JSON pipelines only; chat is out of scope.
  • No safety guarantee. Destructive operations were neither trained nor tested for; during evaluation only read commands were executed. The host application must keep a human in the loop and must not run suggestions blindly.

Formats

Format Where Notes
safetensors, bf16 this repository Reference weights, loaded with transformers.
ONNX (onnxruntime-genai) esilva/KapiCoder-Mongo-Nano-ONNX CPU builds for the Kapibara Studio editor, tag v0.3.0-preview. No DirectML build in this release.

No GGUF is provided. Neither the teacher model nor the training data are part of this repository.

License, modification notice and attribution

  • License: Apache License 2.0 (see LICENSE).
  • Modification notice: this model is a modified version of Qwen/Qwen2.5-Coder-0.5B (revision above). Changes: full fine-tuning (SFT) for MongoDB fill-in-the-middle autocomplete, with part of the labels distilled from a 7B teacher; export to safetensors bf16. It is not a product of Alibaba Cloud or the Qwen team and is not endorsed by them. See NOTICE.md.
  • Attribution: base model Qwen2.5-Coder by the Qwen team (Hui et al., Qwen2.5-Coder Technical Report, arXiv:2409.12186), Apache-2.0. The teacher was derived from Qwen2.5-Coder-7B (Apache-2.0); its weights are not part of this repository. Part of the training queries was written by Qwen3-14B (Apache-2.0).
  • The generic-code replay used npm packages under MIT, Apache-2.0, BSD-2/3-Clause and ISC licenses. The code itself is not redistributed; the package names, versions and licenses are listed in REPLAY_ATTRIBUTION.md.

Related model: KapiCoder-Mongo-Small (1.5B), same task and contract.


Resumo em portuguรชs (pt-BR)

Status: PREVIEW / experimental, preview pรบblico. KapiCoder-Mongo-Nano รฉ um modelo pequeno (cerca de 494 M de parรขmetros) ajustado a partir do Qwen/Qwen2.5-Coder-0.5B para completar consultas MongoDB no cursor, inclusive com o cursor dentro de find, update e aggregate, no editor Kapibara Studio. Esta รฉ a versรฃo v4b dos pesos (tag v0.3.0-preview), que substitui a v4 (mesmo contrato, tag v0.2.0-preview); a v3 continua na tag v0.1.0-preview.

  • Contrato: editor-context-v2 (o cabeรงalho v1 mais a linha TARGET FIELDS com os campos da coleรงรฃo alvo) e a regra de saรญda que fecha o objeto interno deixado aberto. Versรตes do editor sem editor-context-v2 recusam este pacote; nelas, use a tag v0.1.0-preview.
  • Avaliaรงรฃo: avaliaรงรฃo interna sobre dados sintรฉticos; sem conjunto cego humano. Na suรญte cursor-in-call o documento continua vรกlido em 95,1% dos casos (v3: 83,9%) e, com o schema no cabeรงalho, 94,6% dos campos citados existem. Com o prompt antigo (contrato v1, sem TARGET FIELDS), na homologaรงรฃo, FIM acc 41,67% contra 42,34% da v3 e exec-valid 80,85% contra 80,20%. Sem cabeรงalho, 42,1% dos campos citados existem (v3: 44,4%).
  • Treino: 162.025 exemplos sintรฉticos, 131.058.830 tokens vistos, 7,37 horas em uma RX 7800 XT. Duas execuรงรตes de treino (duas sementes); estes sรฃo os pesos da primeira. Nenhuma regra da comparaรงรฃo prรฉ-registrada contra a v4 mudou entre as duas sementes. Sem dados pessoais por construรงรฃo; nรฃo houve revisรฃo humana das amostras.
  • Limites: sรณ dado sintรฉtico; mande os campos da coleรงรฃo no cabeรงalho e valide os nomes antes de executar; o modelo sรณ sugere texto e nunca executa.
  • Licenรงa: Apache-2.0, com aviso de modificaรงรฃo: derivado e modificado do Qwen2.5-Coder-0.5B (equipe Qwen, Alibaba Cloud), sem endosso deles.
Downloads last month
883
Safetensors
Model size
0.5B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for esilva/KapiCoder-Mongo-Nano

Finetuned
(50)
this model
Quantizations
2 models

Paper for esilva/KapiCoder-Mongo-Nano

Evaluation results

  • Document parses after inserting the suggestion (64 tokens, v2 post-processing) on Internal evaluation on synthetic data, cursor-in-call suite v1 (n=1200, held-out schemas)
    self-reported
    0.951
  • Every operator is legal where it is (64 tokens, v2 post-processing) on Internal evaluation on synthetic data, cursor-in-call suite v1 (n=1200, held-out schemas)
    self-reported
    0.945
  • Cited fields exist in the target collection, schema in the header (context_kind=full) on Internal evaluation on synthetic data, cursor-in-call suite v1 (n=1200, held-out schemas)
    self-reported
    0.946
  • FIM accuracy (normalized exact match) on KapiCoder v2c validation split, internal and synthetic (n=2000)
    self-reported
    0.389
  • Exec-valid (completion parses and runs read-only) on KapiCoder v2c validation split, internal and synthetic (n=2000)
    self-reported
    0.564
  • FIM accuracy (normalized exact match), homologation on Internal evaluation, synthetic out-of-distribution homologation v1 (n=6030, LLM- and grammar-written; not fully independent of the training data)
    self-reported
    0.417
  • Exec-valid (completion parses and runs read-only), homologation on Internal evaluation, synthetic out-of-distribution homologation v1 (n=6030, LLM- and grammar-written; not fully independent of the training data)
    self-reported
    0.808