The published build rejected seq=1 with "Size (1) of dimension (1) is not in
allowed range (2..74)", which surfaced in React Native ExecuTorch as
Error::DelegateInvalidHandle for any single-word input. See
software-mansion/react-native-executorch#1164.
torch.export keeps the declared min of 1; to_edge_transform_and_lower is what
widened the token dim to 2..74. Exporting under
torch.fx.experimental._config.backed_size_oblivious keeps 1 in the range.
schema.json already declared min 1, so the file now matches the contract it
publishes rather than the other way round.
Verified on macOS (ExecuTorch 1.4.1) against the eager model, cosine at every
length from 1 to the maximum:
seq=1 0.99998
seq=2 0.99997
seq=16 0.99990
seq=74 0.99991
and on an iPhone 16 in the demo app, where a single word now embeds instead of
failing.
The file name is unchanged, so nothing moves in the model registry.
Stop restating quantized, and drop the unused default
quantized duplicated precisions.json, which is authoritative for the
quantized/non-quantized partition. default expressed a per-repository
recommendation nothing consumed: the library resolves its own per-platform
default. Both are gone from the schema.
model.json points at {backend}/config.json, but this repo had no coreml/config.json at all, so the Core ML artifacts were published without a contract. Added, following the same shape as the published xnnpack config: a single variant with `components` mapping the image and text towers to their files.
The image fp32 variant is removed. Its text counterpart was already dropped, so the pair was mismatched, and fp16 is equivalent: relative L2 0.0026 and cosine 0.999997 against fp32 across 8 images. Core ML fp32 also cannot run on the ANE and measured slower everywhere.
Note the Core ML text tower takes two inputs (ids and mask) at [1,74] where the xnnpack text tower takes one at [1,77]; that difference is pre-existing and preserved here.
v0.9.0 and below are untouched.
Core ML: pin compute_unit=CPU_ONLY, drop the fp32 variant
These graphs fail the Apple Neural Engine path (coremltools#2825 masked mean-pool over a dynamic sequence dim; clip-text additionally ANECCompile FAILED). Core ML still returned correct output because it silently fell back to CPU, but the failed ANE compile was paid on every call. Pinning compute_unit=CPU_ONLY skips the attempt.
Measured on an iPhone 16, fp16, 3 warmup + 15 timed runs, median:
all-MiniLM 13.7 ms -> 2.5 ms 5.5x
multi-qa 14.1 ms -> 2.5 ms 5.6x
paraphrase-L12 26.9 ms -> 4.2 ms 6.4x
distiluse 33.0 ms -> 8.0 ms 4.1x
clip-text 18.1 ms -> 4.9 ms 3.7x
The fp32 variant is removed. Once fp16 is pinned it is dominated on both axes: slower (all-MiniLM 4.5 vs 2.5 ms, clip-text 9.5 vs 4.9 ms) and twice the size. Output fidelity against fp32 was gated first: relative L2 4e-6 (all-MiniLM, multi-qa), 0.0010 (paraphrase), 0.0046 (distiluse).
v0.9.0 and below are untouched.
Add clip text coreml export (CPU_AND_NE, ExecuTorch 1.4.1)
