torch.nn.CosineEmbeddingLoss

class CosineEmbeddingLoss extends Module

new CosineEmbeddingLoss(options?: CosineEmbeddingLossOptions)

readonlymargin(number)
readonlyreduction(Reduction)

Cosine Embedding Loss: metric learning loss for comparing embeddings via cosine similarity.

Measures the cosine distance between two vectors with a margin. Designed for similarity learning tasks where you want similar pairs to have high cosine similarity and dissimilar pairs to have low cosine similarity. Common in:

Siamese networks (comparing two inputs)
Face recognition (matching faces)
Person re-identification
Metric learning where relative similarity matters

When to use CosineEmbeddingLoss:

Siamese/Triplet networks (pair-wise comparison of embeddings)
Learning embeddings where cosine distance is the metric
Face recognition, person re-ID, similarity learning
When you want to push similar pairs together and dissimilar pairs apart
Learning representations where direction matters, not magnitude

Trade-offs:

vs TripletMarginLoss: Both metric learning; triplet is harder mining (3 samples), cosine is pair-wise (2 samples)
vs MSELoss: Cosine similarity is invariant to magnitude, better for embeddings
Pair-wise: Only takes 2 embeddings vs triplet's 3 (anchor, positive, negative)
Margin interpretation: In cosine space (-1 to 1), margin controls separation

Algorithm: For each pair (x1, x2) with label y ∈ {1, -1}:

similarity = cosine_similarity(x1, x2) = dot(x1, x2) / (||x1|| * ||x2||)
If y == 1: loss = max(0, margin - similarity) (push together)
If y == -1: loss = max(0, similarity - margin) (push apart)

Negative target means dissimilar, positive means similar.

\begin{aligned} \text{similarity} = \frac{\langle \mathbf{x}_1, \mathbf{x}_2 \rangle}{\|\mathbf{x}_1\| \cdot \|\mathbf{x}_2\|} \\ \text{loss}(y) = \begin{cases} \max(0, \text{margin} - \text{similarity}) & \text{if } y = 1 \\ \max(0, \text{similarity} - \text{margin}) & \text{if } y = -1 \end{cases} \end{aligned}

Metric learning: Directly optimizes cosine similarity for ranking tasks
Embedding invariance: Loss is invariant to embedding magnitude (only direction matters)
Pair-wise comparison: Only 2 embeddings per sample (vs triplet's 3)
Margin effect: margin 0 creates buffer between similar and dissimilar
Binary target: target must be 1 (similar) or -1 (dissimilar)
Cosine range: Similarity output is in [-1, 1] (perfect opposite to perfect match)
Common in ranking: Good for search/retrieval where relative ordering matters

Examples

// Siamese network: learn embeddings of paired inputs
const cosine_loss = new torch.nn.CosineEmbeddingLoss(0.0);

// Two embeddings from same class (similar)
const embedding1 = torch.randn([32, 128]);  // 32 pairs, 128-dim embeddings
const embedding2 = torch.randn([32, 128]);
const target = torch.ones([32]);  // 1 means similar

const loss = cosine_loss.forward(embedding1, embedding2, target);
// Encourages cosine_similarity(embedding1, embedding2) to be high

// With negative pairs (dissimilar)
const pos_emb1 = torch.randn([16, 128]);
const pos_emb2 = torch.randn([16, 128]);
const pos_target = torch.ones([16]);  // Similar (y=1)

const neg_emb1 = torch.randn([16, 128]);
const neg_emb2 = torch.randn([16, 128]);
const neg_target = torch.ones([16]).mul(-1);  // Dissimilar (y=-1)

// Concatenate into batch
const all_emb1 = torch.cat([pos_emb1, neg_emb1], 0);
const all_emb2 = torch.cat([pos_emb2, neg_emb2], 0);
const all_target = torch.cat([pos_target, neg_target], 0);

const cosine_loss = new torch.nn.CosineEmbeddingLoss(0.0);
const loss = cosine_loss.forward(all_emb1, all_emb2, all_target);
// Simultaneously pulls similar pairs together and pushes dissimilar apart

// Face recognition with margin
const face_extractor = new FaceFeatureExtractor();  // Outputs embeddings
const cosine_loss = new torch.nn.CosineEmbeddingLoss(0.25);  // Margin = 0.25

const face1 = face_extractor.forward(image1);  // [1, 512] embedding
const face2 = face_extractor.forward(image2);  // [1, 512] embedding
const is_same = torch.tensor([1]);  // 1 if same person, -1 if different

const loss = cosine_loss.forward(face1, face2, is_same);
// With margin, requires high similarity for same person, low for different

// Person re-identification: matching gallery to query
const query_embedding = torch.randn([256]);
const gallery_embeddings = torch.randn([1000, 256]);

// Expand query to match batch
const query_batch = query_embedding.unsqueeze(0).expand([1000, 256]);

// Assume first 100 gallery samples are same person (target=1), others different (target=-1)
const target = torch.cat([torch.ones([100]), torch.ones([900]).mul(-1)]);

const cosine_loss = new torch.nn.CosineEmbeddingLoss(0.1);
const loss = cosine_loss.forward(query_batch, gallery_embeddings, target);

torch.nn.CosineEmbeddingLoss

Examples

See Also

torch.nn.CosineEmbeddingLoss

Examples

See Also