DL0010 Receptive Field

The receptive field (RF) of a neuron is the region of the input image that can influence that neuron’s activation. It grows with network depth, so deeper layers see larger context and learn more hierarchical features. The RF is computed layer by layer: each layer expands the field according to its kernel size, scaled by the cumulative stride of all preceding layers.

(1) Definition: The RF is the input region that affects one activation in a given layer; a neuron with a large RF integrates global context.
(2) Growth Rule: Each layer adds to the RF, where is the kernel size and the product runs over all previous strides.
(3) Design Implication: Stacked small kernels grow the RF parameter-efficiently; strided and dilated convolutions grow it much faster.

Mathematical Formulation:

Where:

• is the receptive field size after layer , with at the input layer.
• is the kernel size of layer .
• is the stride of layer , and the product accumulates all strides before layer .

Figure 1: With per layer, one neuron’s RF grows from 3 to 5 to 7 input cells as layers stack.

Stride and Dilation Effects: A layer with stride 2 doubles the jump between adjacent neurons, so every subsequent layer adds twice as much to the RF. Dilated convolutions enlarge the kernel’s span by inserting gaps, growing the RF without reducing resolution, which is useful in segmentation.

Figure 2: Worked example: a stride-2 layer at doubles the increment contributed by every later layer, jumping the RF from 7 to 11.

Real CNN Stacks: In practice, convolutions, pooling, and dilation mix freely: pooling multiplies the jump between neurons, and dilated kernels widen the span, so the RF can reach 22 within just five layers.

Figure 3: In a realistic stack, max pooling doubles the jump and dilation (D=2) widens the kernel, pushing the RF from 1 to 22 in five layers.