torch.nn.SoftMarginLossOptions

Soft Margin Loss: smooth hinge loss for two-class classification.

Computes a soft (smooth) version of margin loss for binary classification. Similar to hinge loss but uses logarithm for smoothness and better numerical behavior. Takes raw unbounded outputs (not probabilities), making it more numerically stable than hinge loss. Less common than BCE or CrossEntropy but useful for margin-based binary classification.

When to use SoftMarginLoss:

Binary classification with raw outputs (not probabilities)
Smooth hinge loss for binary targets (-1, 1)
Margin-based binary classification
When you want properties between hinge and logistic loss
Rarely used; BCEWithLogitsLoss or CrossEntropyLoss preferred

Trade-offs:

vs BCEWithLogitsLoss: Similar but different formulation; BCE more common
vs Hinge Loss: Soft version, numerically better behaved
vs CrossEntropyLoss: Binary vs multi-class; different formulation
Smoothness: Better gradients than hard hinge loss
Popularity: Less common; mainly for compatibility

Algorithm: For each output with target y ∈ {-1, 1}:

loss_i = log(1 + exp(-y * output_i))

This is equivalent to: log(1 + exp(-margin_quantity)) Where margin is y * output (target-weighted output).

Definition

export interface SoftMarginLossOptions {
  /** How to reduce the loss ('none' | 'mean' | 'sum', default: 'mean') */
  reduction?: Reduction;
}

reduction(Reduction)optional: – How to reduce the loss ('none' | 'mean' | 'sum', default: 'mean')

Examples

// Binary classification with soft margin
const soft_margin = new torch.nn.SoftMarginLoss();

// Raw outputs from model (not sigmoid-ed)
const outputs = torch.randn([32, 1]);

// Binary targets (-1 or 1)
const targets = torch.randint(0, 2, [32, 1]).mul(2).sub(1);  // 0,1 -> -1,1

const loss = soft_margin.forward(outputs, targets);
// Smooth loss for margin-based binary classification

// Binary classification network
class BinaryClassifier extends torch.nn.Module {
  fc1: torch.nn.Linear;
  fc2: torch.nn.Linear;

  constructor() {
    super();
    this.fc1 = new torch.nn.Linear(100, 64);
    this.fc2 = new torch.nn.Linear(64, 1);
  }

  forward(x: torch.Tensor): torch.Tensor {
    let h = torch.nn.functional.relu(this.fc1.forward(x));
    // Return raw output, not sigmoid
    return this.fc2.forward(h);
  }
}

const model = new BinaryClassifier();
const loss_fn = new torch.nn.SoftMarginLoss();

const batch_x = torch.randn([32, 100]);
const batch_y = torch.tensor([1, -1, 1, -1, ...]);  // Binary targets

const outputs = model.forward(batch_x);
const loss = loss_fn.forward(outputs, batch_y);