torch.Tensor.Tensor.heaviside
Heaviside step function (generalized step function).
Element-wise step function: Heaviside(x, y) returns y where x ≤ 0, and 1 where x > 0. The second argument (y, called "values") is the return value for non-positive inputs. Creates a sharp step-like transition, useful for conditional logic and gating.
Definition:
- For x > 0: returns 1
- For x ≤ 0: returns the corresponding value from the
valuestensor - Commonly: y = 0 (standard step) or y = 0.5 (symmetric, at discontinuity)
Use Cases:
- Hard gating (on/off decision)
- Step activation functions
- Conditional logic in neural networks
- Implementing ReLU variants with custom behavior
- Signal processing and control flow
- Hard cutoff: Discontinuous at x=0, creates sharp step.
- No gradient at x=0: Derivative is zero except at discontinuity (problematic for backprop).
- Backward propagation: Gradients only flow where input 0.
- Broadcasting: Values tensor broadcasts with input shape.
- GPU efficient: Implemented as GPU shader (WebGPU).
- Zero gradient below threshold prevents learning in that region.
- Not differentiable at x=0 (step discontinuity).
- May cause vanishing gradients in networks.
- Use sigmoid or smooth alternatives if gradients needed for x ≤ 0.
Parameters
valuesTensor- Tensor specifying the value for non-positive inputs (x ≤ 0) Must be broadcastable with input shape
Returns
Tensor<S, D, Dev>– Tensor with same shape as input, containing 1 or valuesExamples
// Standard step function (y=0)
const x = torch.tensor([-2, -1, 0, 1, 2]);
const step = x.heaviside(torch.tensor(0)); // [0, 0, 0, 1, 1]
// Symmetric step (y=0.5 at discontinuity)
const sym = x.heaviside(torch.tensor(0.5)); // [0.5, 0.5, 0.5, 1, 1]
// Gating with Heaviside
const scores = torch.randn(16, 10);
const gated = scores.heaviside(torch.zeros_like(scores)); // Zero where < 0
// Hard decision boundary
const predictions = model(input); // Continuous output
const binary = predictions.heaviside(torch.zeros_like(predictions)); // Threshold at 0
// Broadcasting values
const x = torch.randn(8, 16, 16);
const threshold = torch.tensor(0.5); // Scalar broadcasts to all
const y = torch.zeros(8, 16, 16);
const stepped = x.heaviside(y);See Also
- PyTorch torch.heaviside()
- relu - Similar hard cutoff at x=0 with y=0
- threshold - Similar with different semantics (x t)
- sigmoid - Smooth step alternative
- tanh - Smooth approximation with centered range