What are dilated convolutions? When would you use them?
Answer
Dilated convolutions (also called atrous convolutions) insert gaps between the elements of a convolutional kernel: with dilation rate 
, the kernel samples every -th input position instead of adjacent ones. This expands the receptive field without adding parameters and without reducing spatial resolution, a combination that pooling cannot offer. A dilation rate of 1 is just a standard convolution.
(1) Larger Receptive Field, Same Weights: A 3×3 kernel with dilation 2 covers a 5×5 area but still has only 9 weights.
(2) Resolution Preserved: Unlike pooling, dilation grows the receptive field while keeping the output the same size as a standard convolution, which is critical for dense prediction.
(3) Multi-Scale Context: Stacking or mixing dilation rates lets one network aggregate both fine local detail and broad context.
Figure 1: Same 3 weights, wider view: dilation 3 spreads the taps across a receptive field of 7 instead of 3.
Mathematical Formulation:
Where:
is the dilation rate, the spacing between tapped positions (
is standard convolution).- is the nominal kernel size;
is the effective span of the dilated kernel (3×3 with 
behaves like 5×5 with holes). 
are the kernel weights; the parameter count is independent of .
is standard convolution).
is the nominal kernel size; 
is the effective span of the dilated kernel (3×3 with 
behaves like 5×5 with holes).
are the kernel weights; the parameter count is independent of In Two Dimensions: The same idea applies to images: the kernel’s taps spread over a checkerboard-like stencil, expanding the covered span from 3×3 to 5×5 at dilation 2 while the channel count and output resolution stay unchanged.
Figure 2: Dilation 2 turns a 3×3 stencil into a 5×5 receptive field (red dashed box), still only 9 multiply-accumulates.
When to Use Them: Any task needing large context at full resolution: semantic segmentation (DeepLab), audio generation (WaveNet models long-range temporal structure with exponentially growing dilation), and dense tasks like super-resolution or depth estimation.
Figure 3: Gridding artifacts: stacking the same dilation leaves a checkerboard of unattended cells. Mitigate with hybrid dilation rates (e.g., 1, 2, 5) that overlap coverage.
Leave a Reply