DL0060 Depthwise Separable Convolution

Written by

What is depthwise separable convolution, and where is it commonly used?

Answer

Depthwise separable convolution factorizes a standard convolution into a depthwise spatial operation and a pointwise channel-mixing operation. The depthwise stage applies one $k\times k$ filter per input channel, so it learns spatial patterns independently; the $1\times1$ pointwise stage combines those channel responses into new output channels. This factorization can reduce multiply-accumulates and parameters by almost the kernel area when channel counts are large. It is common in MobileNet, Xception, EfficientNet-style mobile blocks, and edge models where latency, power, and model size matter.

(1) Depthwise Stage: With depth multiplier one, $C_{in}$ independent $k\times k$ filters produce $C_{in}$ spatially filtered maps and do not mix channels.
(2) Pointwise Stage: A $1\times1$ convolution applies a learned $C_{in}\times C_{out}$ channel transformation independently at every spatial position.
(3) Real Hardware: Lower arithmetic cost does not guarantee proportional speedup because memory traffic, kernel launch overhead, vectorization, and accelerator support can dominate.

Standard convolution compared with depthwise and pointwise factorization.

Figure 1: Tensor-level view of spatial filtering per channel followed by 1×1 channel mixing, with parameters and MACs.

Mathematical Formulation:
$\mathrm{MACs}_{std}=HWk^2C_{in}C_{out}$
$\mathrm{MACs}_{sep}=HW\left(k^2C_{in}+C_{in}C_{out}\right)$
$\frac{\mathrm{MACs}_{sep}}{\mathrm{MACs}_{std}}=\frac{1}{C_{out}}+\frac{1}{k^2}$

Where:

$\mathrm{MACs}_{std}$ and $\mathrm{MACs}_{sep}$ count multiply-accumulates for standard and depthwise-separable convolution.
$H$ and $W$ are output height and width, and $k$ is the square spatial-kernel width.
$C_{in}$ and $C_{out}$ are input and output channel counts; the formula assumes depth multiplier $1$ .
For $k=3$ and large $C_{out}$ , the ratio approaches $1/9$ .