DL0060 Depthwise Separable Convolution

Written by

What is depthwise separable convolution, and where is it commonly used?

Answer

Depthwise separable convolution factorizes a standard convolution into a depthwise spatial operation and a pointwise channel-mixing operation. The depthwise stage applies one $k\times k$ filter per input channel, so it learns spatial patterns independently; the $1\times1$ pointwise stage combines those channel responses into new output channels. This factorization can reduce multiply-accumulates and parameters by almost the kernel area when channel counts are large. It is common in MobileNet, Xception, EfficientNet-style mobile blocks, and edge models where latency, power, and model size matter.

Standard convolution compared with depthwise and pointwise factorization.

Figure 1: Tensor-level view of spatial filtering per channel followed by 1×1 channel mixing, with parameters and MACs.

(1) Depthwise Stage: With depth multiplier one, $C_{in}$ independent $k\times k$ filters produce $C_{in}$ spatially filtered maps and do not mix channels.
(2) Pointwise Stage: A $1\times1$ convolution applies a learned $C_{in}\times C_{out}$ channel transformation independently at every spatial position.
(3) Real Hardware: Lower arithmetic cost does not guarantee proportional speedup because memory traffic, kernel launch overhead, vectorization, and accelerator support can dominate.

Decision flowchart for choosing standard, grouped, or depthwise separable convolution.

Figure 2: Architecture decision based on efficiency target, channel interaction, hardware kernels, and measured latency.

Mathematical Formulation:
$\mathrm{MACs}_{std}=HWk^2C_{in}C_{out}$
$\mathrm{MACs}_{sep}=HW\left(k^2C_{in}+C_{in}C_{out}\right)$
$\frac{\mathrm{MACs}_{sep}}{\mathrm{MACs}_{std}}=\frac{1}{C_{out}}+\frac{1}{k^2}$

Where:

$\mathrm{MACs}_{std}$ and $\mathrm{MACs}_{sep}$ count multiply-accumulates for standard and depthwise-separable convolution.
$H$ and $W$ are output height and width, and $k$ is the square spatial-kernel width.
$C_{in}$ and $C_{out}$ are input and output channel counts; the formula assumes depth multiplier $1$ .
For $k=3$ and large $C_{out}$ , the ratio approaches $1/9$ .