torch.nn.functional.hinge_embedding_loss
function hinge_embedding_loss(input: Tensor, target: Tensor): Tensorfunction hinge_embedding_loss(input: Tensor, target: Tensor, margin: number, size_average: boolean | null, reduce: boolean | null, reduction: 'none' | 'mean' | 'sum', options: HingeEmbeddingLossFunctionalOptions): TensorHinge embedding loss for learning embeddings with margin constraints.
Learns embeddings where similar samples are attracted and dissimilar samples are repelled. Given an input x and target y ∈ {+1, -1}, optimizes for either small (similar) or large (dissimilar) distances/scores. Essential for:
- Learning nonlinear embeddings with margin constraints
- Semi-supervised learning (push similar, pull dissimilar)
- Siamese networks (embedding similarity learning)
- Contrastive learning (metric learning with explicit margins)
- One-class SVM inspired objectives
Core idea: Pull similar samples (y=+1) toward zero, push dissimilar (y=-1) beyond margin.
- When y=+1: penalizes large x (want x near 0 for similarity)
- When y=-1: penalizes x < margin (want x beyond margin threshold)
Loss formula:
- When target = +1 (similar): loss = x (want x small/near 0)
- When target = -1 (dissimilar): loss = max(0, margin - x) (want x ≥ margin)
Margin interpretation: Minimum acceptable dissimilarity score. Larger margin means dissimilar samples must be pushed further away.
- Asymmetric loss: Different behavior for y=+1 vs y=-1
- Linear for similar: When y=+1, loss is unbounded above (want x → 0)
- Hinge for dissimilar: When y=-1, loss has margin threshold (standard hinge)
- Distance interpretation: For distances, y=+1 wants small x, y=-1 wants large x
- Similarity interpretation: For similarities, may need different sign conventions
- Target validity: Must contain only +1 or -1; other values produce wrong results
- Margin size: Very large margin makes dissimilar loss hard to satisfy
- Input scaling: Loss depends on absolute scale of input; normalize if needed
- Similarity vs distance: Sign convention matters; be careful with input meaning
Parameters
inputTensor- Embedding distance/similarity scores, shape [batch_size] or [...]. Typically values like Euclidean distance or negative similarity. The loss penalizes based on whether y=+1 or y=-1.
targetTensor- Target similarity labels, shape [...] matching input. Values must be +1 (similar) or -1 (dissimilar).
Returns
Tensor– Loss tensor, shape [] (scalar) if reduction='mean'|'sum', else [...]Examples
// Siamese network: learn embeddings with similarity labels
const batch_size = 32;
const distances = torch.randn([batch_size]); // Pairwise distances in embedding space
const targets = torch.ones([batch_size]); // +1: similar pairs, -1: dissimilar
targets.sub_(2 * torch.bernoulli(torch.full([batch_size], 0.5))); // Random ±1
const loss = torch.nn.functional.hinge_embedding_loss(distances, targets, 1.0);// Semi-supervised learning: similar → collapse, dissimilar → separate
const embeddings_a = torch.randn([64, 128]); // Embedding A
const embeddings_b = torch.randn([64, 128]); // Embedding B
const distances = embeddings_a.sub(embeddings_b).pow(2).sum(-1).sqrt(); // L2 distance
const labels = torch.ones([64]); // Mix of similar/dissimilar pairs
labels.masked_fill_(torch.rand([64]).lt(0.5), -1); // Random ±1 labels
const loss = torch.nn.functional.hinge_embedding_loss(distances, labels, 1.0);// Metric learning with similarity scores instead of distances
const similarity_scores = torch.randn([32]); // Cosine similarity [−1, 1]
const pair_labels = torch.ones([32]); // Whether pairs are similar
pair_labels.masked_fill_(torch.rand([32]).lt(0.3), -1); // 30% dissimilar
const loss = torch.nn.functional.hinge_embedding_loss(similarity_scores, pair_labels, 0.5);See Also
- PyTorch torch.nn.functional.hinge_embedding_loss
- torch.nn.functional.margin_ranking_loss - Margin loss for pairwise ranking
- torch.nn.functional.triplet_margin_loss - Margin loss for triplets (anchor, pos, neg)
- torch.nn.functional.contrastive_loss - Alternative contrastive learning loss