Problems

1 / 1

Implement InfoNCE Loss

Loss Functions

Hard

Implement InfoNCE Loss for contrastive learning. This is the core loss function used in SimCLR, CLIP, and many other self-supervised learning methods.

InfoNCE Loss Formula:

Similarity matrix:

S = \frac{Z_1 Z_2^{T}}{\tau}

InfoNCE Loss:

L = -\frac{1}{N} \sum_{i=1}^{N} \log \left( \frac{\exp(S_{i,i})}{\sum_{j=1}^{N} \exp(S_{i,j})} \right)

where τ is temperature, S_i,iare positive pairs

Function Arguments

Z1: array-like - First embedding batch (N, D)
Z2: array-like - Second embedding batch (N, D)
temperature: float = 0.1 - Temperature parameter (τ)

Loading visualization...

Examples

Input: Z1=[[1,0],[0,1]], Z2=[[1,0],[0,1]], temperature=0.1

Output: ~0.0 (low loss)

Perfect alignment: positive pairs have highest similarity

Input: Z1=[[1,0],[0,1]], Z2=[[0,1],[1,0]], temperature=0.1

Output: ~10.0 (high loss)

Misaligned: positive pairs have low similarity

Input: Z1=[[1,0],[0,1]], Z2=[[1,0],[0,1]], temperature=1.0

Output: ~0.31 (moderate loss)

Higher temperature reduces contrast between similarities

Hint 1

Compute similarity matrix: S = np.dot() / temperature.

Hint 2

For numerical stability: S_stable = S - np.max() before applying exp.

Hint 3

Extract diagonal elements (positive pairs): np.diag() and compute cross-entropy loss.

Requirements

Compute similarity matrix using dot product
Apply numerically stable softmax (subtract max before exp)
Return scalar mean loss across batch
Must be vectorized (no Python loops)

Constraints

N ≤ 256 batch size, D ≤ 512 dimensions
temperature > 0
NumPy only; time limit: 400ms

Try Similar Problems

Contrastive Loss Triplet Loss Cosine Embedding Loss Cross Entropy Loss Softmax Function

Prev Next