ai.onnx.Unique

ai.onnx · standard ONNX operator · ONNX opset ≥ 11

Description

Finds unique values or subtensors along an optional axis. Without an axis, X is flattened; results are sorted or retain first-occurrence order. Sub-32-bit integers and booleans use lossless widened 32-bit storage. Metadata outputs remain logical int64 but use lossless uint32 storage because all values are bounded by an addressable tensor extent. Exact data-dependent output shapes must be supplied. ONNX-permitted uint16, 64-bit, string, and complex inputs are unsupported by this package.

See the ONNX Unique spec for the reference semantics.

Inputs

Outputs

Attributes

Attributes and default values (overridable per request):

Type constraints

Implementation variants

One implementation is selected per call from the device capabilities, the request shapes and the dtypes; these notes say what each one covers.

• single_class_y — Copy the first representative and materialize requested metadata when the exact output contract proves there is one distinct class.
• scalar_hash_parallel_y_unsorted — Hash first occurrences, scan flag blocks in parallel, and scatter scalar representatives in input order with only the requested metadata.
• scalar_hash_parallel_y_sorted — Hash and compact scalar representatives in parallel, sort their order with shared and global bitonic stages, and materialize only the requested metadata.
• single_class_indices — Copy the first representative and materialize requested metadata when the exact output contract proves there is one distinct class.
• scalar_hash_parallel_indices_unsorted — Hash first occurrences, scan flag blocks in parallel, and scatter scalar representatives in input order with only the requested metadata.
• scalar_hash_parallel_indices_sorted — Hash and compact scalar representatives in parallel, sort their order with shared and global bitonic stages, and materialize only the requested metadata.
• single_class_inverse — Copy the first representative and materialize requested metadata when the exact output contract proves there is one distinct class.
• scalar_hash_parallel_inverse_unsorted — Hash first occurrences, scan flag blocks in parallel, and scatter scalar representatives in input order with only the requested metadata.
• scalar_hash_parallel_inverse_sorted — Hash and compact scalar representatives in parallel, sort their order with shared and global bitonic stages, and materialize only the requested metadata.
• single_class_indices_inverse — Copy the first representative and materialize requested metadata when the exact output contract proves there is one distinct class.
• scalar_hash_parallel_indices_inverse_unsorted — Hash first occurrences, scan flag blocks in parallel, and scatter scalar representatives in input order with only the requested metadata.
• scalar_hash_parallel_indices_inverse_sorted — Hash and compact scalar representatives in parallel, sort their order with shared and global bitonic stages, and materialize only the requested metadata.
• single_class_counts — Copy the first representative and materialize requested metadata when the exact output contract proves there is one distinct class.
• scalar_hash_parallel_counts_unsorted — Hash first occurrences, scan flag blocks in parallel, and scatter scalar representatives in input order with only the requested metadata.
• scalar_hash_parallel_counts_sorted — Hash and compact scalar representatives in parallel, sort their order with shared and global bitonic stages, and materialize only the requested metadata.
• single_class_indices_counts — Copy the first representative and materialize requested metadata when the exact output contract proves there is one distinct class.
• scalar_hash_parallel_indices_counts_unsorted — Hash first occurrences, scan flag blocks in parallel, and scatter scalar representatives in input order with only the requested metadata.
• scalar_hash_parallel_indices_counts_sorted — Hash and compact scalar representatives in parallel, sort their order with shared and global bitonic stages, and materialize only the requested metadata.
• single_class_inverse_counts — Copy the first representative and materialize requested metadata when the exact output contract proves there is one distinct class.
• scalar_hash_parallel_inverse_counts_unsorted — Hash first occurrences, scan flag blocks in parallel, and scatter scalar representatives in input order with only the requested metadata.
• scalar_hash_parallel_inverse_counts_sorted — Hash and compact scalar representatives in parallel, sort their order with shared and global bitonic stages, and materialize only the requested metadata.
• single_class_indices_inverse_counts — Copy the first representative and materialize requested metadata when the exact output contract proves there is one distinct class.
• scalar_hash_parallel_indices_inverse_counts_unsorted — Hash first occurrences, scan flag blocks in parallel, and scatter scalar representatives in input order with only the requested metadata.
• scalar_hash_parallel_indices_inverse_counts_sorted — Hash and compact scalar representatives in parallel, sort their order with shared and global bitonic stages, and materialize only the requested metadata.
• hash_dedup — Hashes flattened values and records each key's first input index. Long, low-cardinality f32 inputs coalesce duplicates in workgroup memory when the device's storage budget permits it.
• axis_hash_bounded — Hash axis slices, sort compacted representative indices in one workgroup, and scatter the exact output.
• axis_hash_bounded_unsorted — Hash axis slices and compact representative indices in first-occurrence order.
• axis_hash_bounded_nan_safe — Hash and sort finite axis slices, with an exact ordered fallback when any floating-point slice contains a NaN.
• axis_hash_bounded_unsorted_nan_safe — Hash finite axis slices in first-occurrence order, with an exact ordered fallback when any floating-point slice contains a NaN.
• axis_hash_bounded_merge — Hash axis slices, sort device-sized shared tiles, and merge by exact lexicographic ranks.
• axis_hash_bounded_nan_safe_merge — Hash axis slices, sort device-sized shared tiles, and merge by exact lexicographic ranks. Preserve the exact ordered fallback when a floating-point slice contains NaN.
• flat_parallel_metadata_indices — Builds the unique values with the parallel dedup and compaction passes, then resolves every metadata output from the finished result: one thread per input element for the bucket and the first-occurrence store, one thread per unique value for the tally.
• flat_hash_metadata_indices — Builds the unique values with the hash-set dedup passes, then resolves every metadata output from the finished result: one thread per input element for the bucket and the first-occurrence store, one thread per unique value for the tally.
• flat_parallel_metadata_inverse — Builds the unique values with the parallel dedup and compaction passes, then resolves every metadata output from the finished result: one thread per input element for the bucket and the first-occurrence store, one thread per unique value for the tally.
• flat_hash_metadata_inverse — Builds the unique values with the hash-set dedup passes, then resolves every metadata output from the finished result: one thread per input element for the bucket and the first-occurrence store, one thread per unique value for the tally.
• flat_parallel_metadata_indices_inverse — Builds the unique values with the parallel dedup and compaction passes, then resolves every metadata output from the finished result: one thread per input element for the bucket and the first-occurrence store, one thread per unique value for the tally.
• flat_hash_metadata_indices_inverse — Builds the unique values with the hash-set dedup passes, then resolves every metadata output from the finished result: one thread per input element for the bucket and the first-occurrence store, one thread per unique value for the tally.
• flat_parallel_metadata_counts — Builds the unique values with the parallel dedup and compaction passes, then resolves every metadata output from the finished result: one thread per input element for the bucket and the first-occurrence store, one thread per unique value for the tally.
• flat_hash_metadata_counts — Builds the unique values with the hash-set dedup passes, then resolves every metadata output from the finished result: one thread per input element for the bucket and the first-occurrence store, one thread per unique value for the tally.
• flat_parallel_metadata_indices_counts — Builds the unique values with the parallel dedup and compaction passes, then resolves every metadata output from the finished result: one thread per input element for the bucket and the first-occurrence store, one thread per unique value for the tally.
• flat_hash_metadata_indices_counts — Builds the unique values with the hash-set dedup passes, then resolves every metadata output from the finished result: one thread per input element for the bucket and the first-occurrence store, one thread per unique value for the tally.
• flat_parallel_metadata_inverse_counts — Builds the unique values with the parallel dedup and compaction passes, then resolves every metadata output from the finished result: one thread per input element for the bucket and the first-occurrence store, one thread per unique value for the tally.
• flat_hash_metadata_inverse_counts — Builds the unique values with the hash-set dedup passes, then resolves every metadata output from the finished result: one thread per input element for the bucket and the first-occurrence store, one thread per unique value for the tally.
• flat_parallel_metadata_all — Builds the unique values with the parallel dedup and compaction passes, then resolves every metadata output from the finished result: one thread per input element for the bucket and the first-occurrence store, one thread per unique value for the tally.
• flat_hash_metadata_all — Builds the unique values with the hash-set dedup passes, then resolves every metadata output from the finished result: one thread per input element for the bucket and the first-occurrence store, one thread per unique value for the tally.

Files

• metadata.json — kernel metadata (id, digests, per-variant templates, provenance)
• manifest.json — the op contract (source of truth)
• test.json — correctness cases
• bench.json — benchmark cases
• scan-block-prefix-u32.wgsl.jinja
• unique-axis-compact-sort.wgsl.jinja
• unique-axis-dedup.wgsl.jinja
• unique-axis-hash.wgsl.jinja
• unique-axis-merge.wgsl.jinja
• unique-axis-scalar-inverse.wgsl.jinja
• unique-axis-scalar-ranks.wgsl.jinja
• unique-axis-scatter.wgsl.jinja
• unique-axis.wgsl.jinja
• unique-compact-sort.wgsl.jinja
• unique-dedup.wgsl.jinja
• unique-flag-block-scan.wgsl.jinja
• unique-flat-metadata.wgsl.jinja
• unique-global-sort-exchange.wgsl.jinja
• unique-global-sort-output.wgsl.jinja
• unique-global-sort-shared.wgsl.jinja
• unique-hash-build.wgsl.jinja
• unique-hash-collect.wgsl.jinja
• unique-hash-init.wgsl.jinja
• unique-hash-mark.wgsl.jinja
• unique-hash-sort-collected-key-only.wgsl.jinja
• unique-scalar-compact.wgsl.jinja
• unique-scalar-metadata.wgsl.jinja
• unique-scalar-output.wgsl.jinja
• unique-single-class.wgsl.jinja
• unique.wgsl.jinja

Use with @huggingface/kernels

npm install --save-exact @huggingface/kernels@0.0.1-preview.3

Outputs with inferable metadata are allocated automatically. Explicit outputs entries request optional results or provide metadata that cannot be inferred from the supplied inputs and attributes.

This example supplies explicit metadata for:

• y

The version: 1 option selects the published kernel contract; it is independent of any operator opset, contrib since_version, or model version. It follows the v1 branch as fixes land. To pin exact artifact bytes, pass a 40-character commit revision instead of version.

Replace each *Data placeholder with a typed array containing the corresponding input data.

import { getKernel } from "@huggingface/kernels";

const kernel = await getKernel("webgpu-kernels/ai.onnx.Unique", { version: 1 });
// Explicit destinations request optional results or supply metadata that cannot be inferred.
const { y } = await kernel({ x: { data: xData, shape: [1] } }, {
  outputs: { y: { shape: [1], dtype: "float32" } },
});