DL0013 Instance Normalization

Instance Normalization (IN) normalizes each individual sample and each channel independently: for every (instance, channel) pair it subtracts the mean and divides by the standard deviation computed over that feature map’s spatial dimensions only. Because statistics never cross instance boundaries, IN is unaffected by mini-batch composition and works with batch size 1. This per-instance normalization removes sample-specific contrast and style, which is why IN became the standard in style transfer and image-generation models.

(1) Per-Instance, Per-Channel Statistics: Mean and variance are computed over the spatial positions of each channel of each sample separately, so no information leaks across the batch.
(2) Batch-Size Independent: Statistics do not depend on other samples, so IN behaves identically at training and test time and remains stable with small batches.
(3) Removes Instance-Specific Style: Normalizing each map’s contrast discards style-like appearance information while preserving content structure, which suits style transfer, GANs, and domain adaptation.

Figure 1: BN pools statistics per channel across the whole batch (red dashed groups), while IN normalizes each (instance, channel) map alone (orange dashed boxes): the source of IN’s batch independence.

Mathematical Formulation:

Where:

• is the input activation at batch index , channel , spatial position .
• and are the mean and variance over the spatial extent of instance , channel .
• is the normalized activation; is a small constant for numerical stability.
• and are learnable per-channel scale and shift parameters; is the output.

Contrast with Batch Normalization: BN computes per-channel statistics over the entire mini-batch (), which couples a sample’s output to its batchmates and forces a switch to running averages at inference. IN’s per-sample statistics are identical in both phases, and discarding per-instance contrast is precisely what removes style from content images.

Feature	Instance Normalization (IN)	Batch Normalization (BN)
Scope of stats	Per instance, per channel (over )	Per channel (over )
Batch size	Independent; works with batch = 1	Dependent; needs stable batch stats
Primary use	Style transfer, GANs, domain adaptation	Image classification, general CNNs
Effect	Removes instance-specific style/contrast	Stabilizes training, speeds convergence
Inference	Same per-sample stats at test time	Uses running stats from training

Bottom line: IN removes per-instance contrast (style); BN aligns feature scales across the batch. IN trades BN’s cross-sample regularization for batch independence and style removal.