Instructions to use esilva/KapiCoder-Mongo-Nano-ONNX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- ONNX Runtime GenAI
How to use esilva/KapiCoder-Mongo-Nano-ONNX with ONNX Runtime GenAI:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
KapiCoder-Mongo-Nano-ONNX (0.5B)
Status: PREVIEW / experimental, public preview, not qualified. ONNX builds of
esilva/KapiCoder-Mongo-Nano(version v4b, tagv0.3.1-preview) foronnxruntime-genai, made for the Kapibara Studio editor. CPU builds and a DirectML build (dml-fp16); the DirectML build was validated only on an AMD Radeon RX 7800 XT (16 GB) under Windows 11. Read the model card of the safetensors repository first: the model, its limits and its evaluation are described there. The previous builds stay available by tag: the same v4b weights with CPU builds only atv0.3.0-preview, v4 (same contract, with a DirectML build) atv0.2.0-preview, v3 (editor-context-v1) atv0.1.0-preview.
What is here
One self-contained folder per variant (each can be downloaded and used on its own). Parity with the bf16 weights was measured on 420 cases of the
cursor-in-call suite with the editor-context-v2 prompt (300 short, 120 long prompts up to 8k tokens), greedy, 64 tokens:
| Folder | Build | Greedy tokens identical to bf16 | Suffix-consistent bf16 / ONNX | Field-grounded bf16 / ONNX |
|---|---|---|---|---|
cpu-int4/ |
INT4, CPU (onnxruntime-genai, cpu provider) |
63.3% | 94.5% / 94.3% | 86.9% / 86.7% |
cpu-int8/ |
INT8, CPU | 88.6% | 94.5% / 94.8% | 86.9% / 87.3% |
dml-fp16/ |
FP16, DirectML (onnxruntime-genai, dml provider) |
90.5% | 94.5% / 94.5% | 86.9% / 86.9% |
Internal evaluation on synthetic data. No difference in these metrics was statistically significant (paired McNemar); token-level differences grow on
long prompts. Each folder holds model.onnx (with model.onnx.data when the weights are external), genai_config.json (search.max_length 8192),
the tokenizer files, kapibarastudio-model.json, kapicoder_manifest.json (SHA-256 of every file) and export_manifest.json (tool, version, options;
local paths were removed). Builds were produced with onnxruntime-genai 0.15.2 (model.type=qwen2) from the safetensors weights of the main repository.
Notes before using
- The
kapimetadata serves the Kapibara Studio application.kapibarastudio-model.json(schemakapilab-kapi-model-v1) declarescontextContract: editor-context-v2, the recommended window (4,096 tokens) and output (64 tokens), and the inner-closer output rule. Editors withouteditor-context-v2reject these packages; use tagv0.1.0-previewwith them. - EOS integration requirement (GenAI 0.15.2): EOS may finish generation without appending a token. The IDE consumes a token only when
GetSequence(0)grows afterGenerateNextToken; checkingIsDone()alone after generating can discard a valid token. - Quantised and converted builds can differ from the bf16 weights. Measure on your own prompts before relying on one; latency depends on the host.
- DirectML picks the GPU through Windows. On a machine with an integrated and a discrete GPU,
device_iddid not change the adapter in our tests; the Windows graphics preference of the host process did. Set the host process to "High performance" or the build may run on the integrated GPU. DirectML latency was not measured for this release. - The model only suggests text and never executes anything; validate field names before running a suggestion.
License, modification notice and attribution
- License: Apache License 2.0 (see
LICENSE). - Modification notice: derived from
Qwen/Qwen2.5-Coder-0.5B(revision8123ea2e9354afb7ffcc6c8641d1b2f5ecf18301) by fine-tuning for MongoDB fill-in-the-middle autocomplete (see the main repository) and converted to ONNX (and, where the folder name says so, quantised). Not a product of Alibaba Cloud or the Qwen team and not endorsed by them. SeeNOTICE.md. - Attribution: base model Qwen2.5-Coder by the Qwen team (Hui et al., arXiv:2409.12186), Apache-2.0. Replay and data attribution are in the main repository (
REPLAY_ATTRIBUTION.md).
Resumo em português (pt-BR)
Status: PREVIEW / experimental, preview público, não qualificado. Builds ONNX de CPU e DirectML do esilva/KapiCoder-Mongo-Nano (versão v4b, tag v0.3.1-preview) para o onnxruntime-genai,
feitos para o editor Kapibara Studio, com o contrato editor-context-v2. A mesma v4b só com CPU fica na tag v0.3.0-preview. A v4 continua na tag v0.2.0-preview e a v3 na v0.1.0-preview. Na paridade com os pesos bf16 (420 casos da
suíte cursor-in-call, prompts de até 8k), a qualidade não mudou de forma significativa. Editores sem editor-context-v2 recusam estes pacotes.
Licença Apache-2.0 com aviso de modificação (derivado do Qwen2.5-Coder-0.5B, sem endosso da equipe Qwen).
- Downloads last month
- 157
Model tree for esilva/KapiCoder-Mongo-Nano-ONNX
Base model
Qwen/Qwen2.5-0.5B