torch.isclose
function isclose<S1 extends Shape, S2 extends Shape>(input: Tensor<S1>, other: Tensor<S2>, options?: IscloseOptions): Tensor<BroadcastShape<S1, S2>>function isclose<S1 extends Shape, S2 extends Shape>(input: Tensor<S1>, other: Tensor<S2>, rtol: number, atol: number, equal_nan: boolean, options?: IscloseOptions): Tensor<BroadcastShape<S1, S2>>Element-wise tolerance-based comparison: returns true where input and other are close.
Compares tensors with a tolerance for floating-point numbers. Critical for:
- Unit testing neural networks (comparing model outputs to expected values)
- Numerical validation (detecting floating-point precision issues)
- Gradient checking (comparing analytical vs numerical gradients)
- Convergence detection (checking if iterations have stabilized)
- Approximate equality: handling inherent floating-point imprecision
- Cross-framework testing: comparing torch.js results with PyTorch
Returns a boolean tensor where each element indicates if input and other are "close".
Two values are considered close if: |input[i] - other[i]| <= (atol + rtol * |other[i]|)
The tolerance formula has two components:
- Absolute tolerance (atol): Fixed threshold, important for values near zero
- Relative tolerance (rtol): Percentage of the other value, important for large values
- Tolerance formula breakdown: Check both absolute AND relative components
- Relative tolerance scales: rtol=1e-5 means ±0.001% tolerance on large values
- Absolute tolerance for near-zero: atol=1e-8 critical for values like 1e-9
- Default tolerances: rtol=1e-5, atol=1e-8 work well for float32 precision
- Broadcasting: Automatically broadcasts shapes for element-wise comparison
- Non-symmetric behavior: Uses |other| in tolerance (order matters slightly!)
- Common pair: Often combined with all() for full tensor comparison
- Not symmetric: isclose(x, y, rtol=r) may differ from isclose(y, x, rtol=r)
- NaN default behavior: Without equal_nan=true, NaN values always return false
- Tolerance ordering: atol + rtol*|other| - if atol is 0, must rely on rtol
- Precision loss: float16 needs larger tolerance than float32 (consider 1e-3, 1e-4)
Parameters
inputTensor<S1>- The first tensor (any shape and dtype)
otherTensor<S2>- The second tensor (must be broadcastable with input)
optionsIscloseOptionsoptional- Tolerance options: -
rtol: Relative tolerance (default: 1e-5). Proportion of |other| value allowed to differ -atol: Absolute tolerance (default: 1e-8). Maximum absolute difference allowed -equal_nan: If true, treat NaN as equal to NaN (default: false)
Returns
Tensor<BroadcastShape<S1, S2>>– Boolean tensor with shape matching broadcast of input/other. True where values are close.Examples
// Unit test: comparing model output to expected
const model_output = torch.tensor([0.123, 0.456, 0.789]);
const expected = torch.tensor([0.1231, 0.4561, 0.7889]);
const correct = torch.isclose(model_output, expected, { rtol: 1e-3 });
// [true, true, true] - all values match within 0.1% relative tolerance
// Floating-point precision: default tolerances work well for float32
const x = torch.tensor([1.0, 1.0000001, 2.0]);
const y = torch.tensor([1.0, 1.0000002, 2.1]);
torch.isclose(x, y); // Uses default rtol=1e-5, atol=1e-8
// [true, true, false]
// Gradient checking: comparing analytical vs numerical gradients
const analytical = computeGradient(model, input);
const numerical = computeNumericalGradient(model, input);
const grad_match = torch.isclose(analytical, numerical, { rtol: 1e-5, atol: 1e-7 });
if (grad_match.all().item()) console.log("Gradients are correct!");
// Handling near-zero values: absolute tolerance is crucial
const small_x = torch.tensor([0.0, 1e-9, 1e-6]);
const small_y = torch.tensor([1e-10, 1e-8, 1e-5]);
// Without enough absolute tolerance, near-zero comparisons fail
torch.isclose(small_x, small_y, { rtol: 1e-5, atol: 0 }); // [false, false, false]
torch.isclose(small_x, small_y, { rtol: 1e-5, atol: 1e-7 }); // [false, true, false]
// Handling NaN values: depends on equal_nan flag
const x = torch.tensor([1.0, NaN, 3.0]);
const y = torch.tensor([1.0, NaN, 3.0]);
torch.isclose(x, y, { equal_nan: false }); // [true, false, true] - NaN != NaN
torch.isclose(x, y, { equal_nan: true }); // [true, true, true] - NaN == NaN
// Element-wise checking: find which elements differ significantly
const pred = torch.randn(100);
const actual = torch.randn(100);
const significant_diff = torch.isclose(pred, actual, { rtol: 0.1 }).logical_not();
const num_mismatches = significant_diff.sum().item();
console.log(`${num_mismatches} elements differ by > 10%`);
// Selecting close values: combining with where for conditional logic
const values1 = torch.randn(50);
const values2 = torch.randn(50);
const close = torch.isclose(values1, values2, { atol: 0.5 });
const avg_if_close = torch.where(close, values1.add(values2).div(2), values1);See Also
- PyTorch torch.isclose(input, other, rtol=1e-05, atol=1e-08, equal_nan=False)
- allclose - Single boolean result for full tensor comparison
- equal - Exact equality check (no tolerance)
- where - Conditional selection based on isclose results
- abs - For manual tolerance checking with different formulas