com.microsoft.BiasSoftmax

com.microsoft · ONNX Runtime contrib operator · contrib since_version 1

Description

Computes softmax(data + bias) over the flattened suffix beginning at axis. The required is_inner_broadcast attribute selects how bias rows are reused: consecutive groups for inner broadcast or cyclic groups for outer broadcast. This specializes the softmax(scores + additive_mask) pattern used by transformer attention. Float16 and float32 are supported; the schema's double type is not.

See the ONNX Runtime BiasSoftmax contrib-operator spec for the reference semantics.

Inputs

Outputs

Attributes

Attributes and default values (overridable per request):

Type constraints

Implementation variants

One implementation is selected per call from the device capabilities, the request shapes and the dtypes; these notes say what each one covers.

• online_subgroup_rows_vec4 — Subgroup-per-row fused bias and softmax with register-resident vec4 values, f32 addition and reduction, and the existing zero-output convention for fully masked rows.
• online_workgroup_vec4 — Vectorized workgroup online bias-softmax with f32 addition and reduction; portable fallback below the register-resident subgroup-row path.
• online_workgroup_scalar_vec4 — Logical vec4 online reduction with scalar storage for unaligned row strides. Invalid tail components contribute zero denominator and are never stored.

Device requirements

Some implementation variants require subgroups. These are route-specific capabilities, not package-wide requirements; availability also depends on the request shape and dtype.

Files

• metadata.json — kernel metadata (id, digests, per-variant templates, provenance)
• manifest.json — the op contract (source of truth)
• test.json — correctness cases
• bench.json — benchmark cases
• bias-softmax-longrow-normalize.wgsl.jinja
• bias-softmax.wgsl.jinja
• softmax-longrow-stats.wgsl.jinja
• softmax-online.wgsl.jinja
• softmax-subgroup-rows.wgsl.jinja

Use with @huggingface/kernels

npm install --save-exact @huggingface/kernels@0.0.1-preview.3

Required output shapes and logical data types are inferred from the supplied inputs and attributes; result tensors are allocated automatically.

The version: 1 option selects the published kernel contract; it is independent of any operator opset, contrib since_version, or model version. It follows the v1 branch as fixes land. To pin exact artifact bytes, pass a 40-character commit revision instead of version.

Replace each *Data placeholder with a typed array containing the corresponding input data.

import { getKernel } from "@huggingface/kernels";

const kernel = await getKernel("webgpu-kernels/com.microsoft.BiasSoftmax", { version: 1 });
const { output } = await kernel({
  data: { data: dataData, shape: [1, 2, 2] },
  bias: { data: biasData, shape: [1, 2, 2] },
}, {
  attrs: { is_inner_broadcast: 1 },
});