software-mansion/react-native-executorch-clip-vit-base-patch32

clip-vit-base-patch32

This repository hosts the clip-vit-base-patch32 models exported for the React Native ExecuTorch library as ExecuTorch .pte programs, ready to run on device.

Upstream model: clip-vit-base-patch32

Variants

Path	Backend	Precision
coreml/clip_vit_base_patch32_image_coreml_fp16.pte	coreml	fp16
coreml/clip_vit_base_patch32_text_coreml_fp16.pte	coreml	fp16
coreml/clip_vit_base_patch32_text_coreml_int8.pte	coreml	int8
mlx/clip_vit_base_patch32_image_mlx_int8.pte	mlx	int8
vulkan/clip_vit_base_patch32_image_vulkan_fp16.pte	vulkan	fp16
vulkan/clip_vit_base_patch32_text_vulkan_fp16.pte	vulkan	fp16
xnnpack/clip_vit_base_patch32_image_xnnpack_fp32.pte	xnnpack	fp32
xnnpack/clip_vit_base_patch32_image_xnnpack_int8.pte	xnnpack	int8
xnnpack/clip_vit_base_patch32_text_xnnpack_fp32.pte	xnnpack	fp32
xnnpack/clip_vit_base_patch32_text_xnnpack_int8.pte	xnnpack	int8

XNNPACK int8

The xnnpack/*_int8.pte towers are 8da8w (int8 dynamic activations, int8 per-channel weights) on the Linear layers; the text tower also has an int8 token embedding table. ImageNetV2 zero-shot top-1 (1000 images, one per class, "a photo of a {class}."):

image	text	top-1
fp32	fp32	51.4
int8	fp32	51.5
fp32	int8	51.2
int8	int8	51.3

Repository structure

config.json                                           43 B
coreml/clip_vit_base_patch32_image_coreml_fp16.pte    168 MB
coreml/clip_vit_base_patch32_text_coreml_fp16.pte     122 MB
coreml/clip_vit_base_patch32_text_coreml_int8.pte     61.0 MB
coreml/config.json                                    1.7 kB
mlx/clip_vit_base_patch32_image_mlx_int8.pte          93.7 MB
mlx/config.json                                       840 B
tokenizer.json                                        2.1 MB
vulkan/clip_vit_base_patch32_image_vulkan_fp16.pte    168 MB
vulkan/clip_vit_base_patch32_text_vulkan_fp16.pte     121 MB
vulkan/config.json                                    1.7 kB
xnnpack/clip_vit_base_patch32_image_xnnpack_fp32.pte  335 MB
xnnpack/clip_vit_base_patch32_image_xnnpack_int8.pte  91.5 MB
xnnpack/clip_vit_base_patch32_text_xnnpack_fp32.pte   242 MB
xnnpack/clip_vit_base_patch32_text_xnnpack_int8.pte   61.4 MB
xnnpack/config.json                                   2.9 kB

Compatibility

These files are published for the ExecuTorch v1.4.1 runtime. ExecuTorch gives no forward compatibility guarantee, so an older runtime may fail to load them.

To use them in React Native ExecuTorch, pass the model constant shipped in the library's model registry to the corresponding task pipeline. See the documentation.

To load these files in your own ExecuTorch runtime, read the compatibility note first.