torch.nn.functional.instance_norm
function instance_norm(input: Tensor, options?: InstanceNormFunctionalOptions): TensorInstance Normalization: normalizes each channel independently for each sample.
Applies instance normalization by treating each channel as its own group (equivalent to GroupNorm with num_groups = num_channels). Normalizes activations independently per sample and per channel, ignoring batch and spatial information. Particularly effective for style transfer, image generation, and other perceptual tasks. Essential for:
- Style transfer and artistic style applications (ADAIN, CycleGAN)
- Image generation tasks (GANs, diffusion models)
- Domain adaptation and unsupervised learning
- Video processing where frame independence matters
- Perceptual loss computation in generative models
- Tasks where content preservation is critical
- Real-time inference (no batch dependency)
Instance Norm vs Batch Norm: Instance norm is particularly useful in generative tasks because it normalizes out style/appearance information while preserving content. Unlike BatchNorm which normalizes across the batch (batch statistics), InstanceNorm normalizes per sample per channel, making it invariant to style variations within the batch.
When to use InstanceNorm:
- Style transfer networks (removes content-independent statistics)
- Image generation and GANs (preserves semantic content)
- Domain adaptation (aligns feature distributions)
- Real-time processing (no batch statistics needed)
- When batch dependency is undesirable
- Video frame processing (frame-level normalization)
- Perceptual tasks where appearance invariance helps
Comparison with alternatives:
- Batch Norm: Uses batch statistics; sensitive to batch size and content
- Layer Norm: Single normalization over all channels; treats all equally
- Group Norm: Flexible groups; InstanceNorm is extreme case (one channel per group)
- Local Response Norm: Different approach (channel-wise competition)
Implementation detail: InstanceNorm is mathematically equivalent to GroupNorm with num_groups = num_channels. Each channel is normalized independently, using only that channel's spatial statistics.
- Per-channel normalization: Each channel normalized independently
- Batch-independent: No reliance on batch size or batch statistics
- Style removal: Effective at removing style information (useful for transfer)
- Real-time friendly: Works with batch_size=1
- Learnable parameters: weight and bias trained like other normalizations
- Content preservation: Preserves spatial structure within channels
- Equivalence: InstanceNorm = GroupNorm with num_groups = num_channels
- Different statistics: Each channel uses only its own spatial statistics
- Minimal 3D: Requires at least 3D input (batch, channels, spatial)
- May lose information: Per-channel normalization can be too aggressive for some tasks
- Not for all tasks: Use BatchNorm for classification; InstanceNorm for generation
Parameters
inputTensor- Input tensor [batch, channels, spatial...]. Minimum 3D required. Example: [N, C, H, W] for images, [N, C, length] for 1D signals Note: All spatial dimensions are normalized over (not per-spatial-location)
optionsInstanceNormFunctionalOptionsoptional
Returns
Tensor– Normalized tensor with same shape as inputExamples
// Style transfer: normalize each channel independently
const batch = 1, channels = 64, height = 256, width = 256;
const x = torch.randn([batch, channels, height, width]); // Input image
const normalized = torch.nn.functional.instance_norm(x);
// Each channel normalized independently; removes style information// Style transfer network (CycleGAN style)
class StyleTransferEncoder extends torch.nn.Module {
private conv1: torch.nn.Conv2d;
private in1: torch.nn.Parameter; // InstanceNorm weight
private conv2: torch.nn.Conv2d;
private in2: torch.nn.Parameter;
constructor() {
super();
this.conv1 = new torch.nn.Conv2d(3, 64, { kernel_size: 7, padding: 3 });
this.in1 = torch.ones([64]);
this.conv2 = new torch.nn.Conv2d(64, 128, { kernel_size: 3, padding: 1 });
this.in2 = torch.ones([128]);
}
forward(x: torch.Tensor): torch.Tensor {
x = this.conv1.forward(x);
x = torch.nn.functional.instance_norm(x, undefined, undefined, this.in1);
x = torch.relu(x);
x = this.conv2.forward(x);
x = torch.nn.functional.instance_norm(x, undefined, undefined, this.in2);
x = torch.relu(x);
return x;
}
}// GAN discriminator with InstanceNorm
class Discriminator extends torch.nn.Module {
private convs: torch.nn.Conv2d[] = [];
private ins: torch.nn.Parameter[] = [];
constructor() {
super();
const channels = [3, 64, 128, 256, 512];
for (let i = 0; i < channels.length - 1; i++) {
this.convs.push(new torch.nn.Conv2d(channels[i], channels[i+1], { kernel_size: 4, stride: 2, padding: 1 }));
this.ins.push(torch.ones([channels[i+1]]));
}
}
forward(x: torch.Tensor): torch.Tensor {
for (let i = 0; i < this.convs.length; i++) {
x = this.convs[i].forward(x);
x = torch.nn.functional.instance_norm(x, undefined, undefined, this.ins[i]);
x = torch.nn.functional.leaky_relu(x, 0.2);
}
return x;
}
}// Domain adaptation: normalize style away
const source_batch = torch.randn([8, 256, 32, 32]); // Source domain
const target_batch = torch.randn([8, 256, 32, 32]); // Target domain
// InstanceNorm removes domain-specific style
const source_normalized = torch.nn.functional.instance_norm(source_batch);
const target_normalized = torch.nn.functional.instance_norm(target_batch);
// Both now have similar statistics; domain gap reduced// Comparison: InstanceNorm vs BatchNorm
const x = torch.randn([4, 64, 32, 32]);
const weight = torch.ones([64]);
const inst = torch.nn.functional.instance_norm(x, undefined, undefined, weight);
const batch_mean = x.mean([0, 2, 3], true);
const batch_var = x.var([0, 2, 3], true);
// InstanceNorm: each (sample, channel) normalized independently
// BatchNorm: statistics computed across all samplesSee Also
- PyTorch torch.nn.functional.instance_norm
- torch.nn.functional.group_norm - Generalized version (flexible grouping)
- torch.nn.functional.layer_norm - Single group (all channels together)
- torch.nn.functional.batch_norm - Normalizes over batch (for classification)
- torch.nn.InstanceNorm2d - Module version for 4D inputs specifically