KapiCoder-Mongo-Nano-ONNX (0.5B)

Status: PREVIEW / experimental, public preview, not qualified. ONNX builds of esilva/KapiCoder-Mongo-Nano (version v4b, tag v0.3.1-preview) for onnxruntime-genai, made for the Kapibara Studio editor. CPU builds and a DirectML build (dml-fp16); the DirectML build was validated only on an AMD Radeon RX 7800 XT (16 GB) under Windows 11. Read the model card of the safetensors repository first: the model, its limits and its evaluation are described there. The previous builds stay available by tag: the same v4b weights with CPU builds only at v0.3.0-preview, v4 (same contract, with a DirectML build) at v0.2.0-preview, v3 (editor-context-v1) at v0.1.0-preview.

What is here

One self-contained folder per variant (each can be downloaded and used on its own). Parity with the bf16 weights was measured on 420 cases of the cursor-in-call suite with the editor-context-v2 prompt (300 short, 120 long prompts up to 8k tokens), greedy, 64 tokens:

Folder Build Greedy tokens identical to bf16 Suffix-consistent bf16 / ONNX Field-grounded bf16 / ONNX
cpu-int4/ INT4, CPU (onnxruntime-genai, cpu provider) 63.3% 94.5% / 94.3% 86.9% / 86.7%
cpu-int8/ INT8, CPU 88.6% 94.5% / 94.8% 86.9% / 87.3%
dml-fp16/ FP16, DirectML (onnxruntime-genai, dml provider) 90.5% 94.5% / 94.5% 86.9% / 86.9%

Internal evaluation on synthetic data. No difference in these metrics was statistically significant (paired McNemar); token-level differences grow on long prompts. Each folder holds model.onnx (with model.onnx.data when the weights are external), genai_config.json (search.max_length 8192), the tokenizer files, kapibarastudio-model.json, kapicoder_manifest.json (SHA-256 of every file) and export_manifest.json (tool, version, options; local paths were removed). Builds were produced with onnxruntime-genai 0.15.2 (model.type=qwen2) from the safetensors weights of the main repository.

Notes before using

  • The kapi metadata serves the Kapibara Studio application. kapibarastudio-model.json (schema kapilab-kapi-model-v1) declares contextContract: editor-context-v2, the recommended window (4,096 tokens) and output (64 tokens), and the inner-closer output rule. Editors without editor-context-v2 reject these packages; use tag v0.1.0-preview with them.
  • EOS integration requirement (GenAI 0.15.2): EOS may finish generation without appending a token. The IDE consumes a token only when GetSequence(0) grows after GenerateNextToken; checking IsDone() alone after generating can discard a valid token.
  • Quantised and converted builds can differ from the bf16 weights. Measure on your own prompts before relying on one; latency depends on the host.
  • DirectML picks the GPU through Windows. On a machine with an integrated and a discrete GPU, device_id did not change the adapter in our tests; the Windows graphics preference of the host process did. Set the host process to "High performance" or the build may run on the integrated GPU. DirectML latency was not measured for this release.
  • The model only suggests text and never executes anything; validate field names before running a suggestion.

License, modification notice and attribution

  • License: Apache License 2.0 (see LICENSE).
  • Modification notice: derived from Qwen/Qwen2.5-Coder-0.5B (revision 8123ea2e9354afb7ffcc6c8641d1b2f5ecf18301) by fine-tuning for MongoDB fill-in-the-middle autocomplete (see the main repository) and converted to ONNX (and, where the folder name says so, quantised). Not a product of Alibaba Cloud or the Qwen team and not endorsed by them. See NOTICE.md.
  • Attribution: base model Qwen2.5-Coder by the Qwen team (Hui et al., arXiv:2409.12186), Apache-2.0. Replay and data attribution are in the main repository (REPLAY_ATTRIBUTION.md).

Resumo em português (pt-BR)

Status: PREVIEW / experimental, preview público, não qualificado. Builds ONNX de CPU e DirectML do esilva/KapiCoder-Mongo-Nano (versão v4b, tag v0.3.1-preview) para o onnxruntime-genai, feitos para o editor Kapibara Studio, com o contrato editor-context-v2. A mesma v4b só com CPU fica na tag v0.3.0-preview. A v4 continua na tag v0.2.0-preview e a v3 na v0.1.0-preview. Na paridade com os pesos bf16 (420 casos da suíte cursor-in-call, prompts de até 8k), a qualidade não mudou de forma significativa. Editores sem editor-context-v2 recusam estes pacotes. Licença Apache-2.0 com aviso de modificação (derivado do Qwen2.5-Coder-0.5B, sem endosso da equipe Qwen).

Downloads last month
157
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for esilva/KapiCoder-Mongo-Nano-ONNX

Quantized
(2)
this model

Paper for esilva/KapiCoder-Mongo-Nano-ONNX