DL0073 EfficientNet Compound Scaling

EfficientNet starts from a small baseline, EfficientNet-B0, produced by a multi-objective architecture search that rewards accuracy and FLOPs together, and then grows that fixed topology into the family B1 through B7 by scaling three dimensions simultaneously: depth (number of layers), width (number of channels), and input resolution. The empirical observation behind the method is that scaling any single dimension saturates: beyond a point, extra layers, extra channels, or extra pixels buy almost no accuracy while still costing compute. Compound scaling ties the three together through one user-chosen coefficient and three fixed exponents obtained by a small grid search on B0, subject to so that each unit of roughly doubles the FLOPs budget. With , , , the family climbs from 77.1% ImageNet top-1 at 0.39B FLOPs (B0) to 84.3% at 37B FLOPs (B7), matching the best accuracy of its era with about 8.4x fewer parameters than GPipe.

(1) The Baseline Is a Prerequisite: compound scaling only multiplies an existing topology, so a weak baseline yields a weak family; B0 is itself a search result built from MBConv blocks with squeeze-and-excitation, and the same scaling rule applied to MobileNet or ResNet gives smaller gains.
(2) Single-Dimension Scaling Saturates: very deep networks hit optimization and degradation limits, very wide shallow networks capture fine-grained patterns but few high-level ones, and resolution alone raises cost quadratically for shrinking returns.
(3) The Balance Rule: a larger input needs more layers to grow the receptive field and more channels to encode the finer patterns those extra pixels expose, which is why the three factors should move in a fixed ratio rather than one at a time.
(4) Two-Step Search: fix and grid-search once on the cheap baseline, then freeze them and sweep to get B1 through B7, which avoids re-searching the architecture at every model size.

Figure 1: Illustrative accuracy-versus-compute curves from the same B0 baseline: depth-only, width-only, and resolution-only scaling flatten near 80% top-1, while compound scaling keeps converting FLOPs into accuracy.

The constraint has a direct cost interpretation. A standard convolution’s FLOPs scale linearly with the number of layers and quadratically with both channel count and spatial size, so total compute grows like . Forcing therefore makes a clean compute dial: each additional unit costs about 2x the FLOPs, and the exponents decide how that doubled budget is split across the three dimensions. The exponents are searched once, on a model cheap enough that a small grid over is affordable.

Mathematical Formulation:

Where:

• , , and are the multipliers applied to the baseline’s layer count per stage, channel count per layer, and input side length.
• is the user-chosen compound coefficient that sets the resource budget; recovers the baseline B0.
• are constants from a small grid search on B0 with , , ; the published values are , , and .
• Convolution cost scales as , so the product constraint is what turns into an approximate doubling of FLOPs per unit.

Figure 2: The same topology at two budgets: a bigger input square (resolution), taller blocks (width), and more blocks (depth) all grow together instead of one dimension racing ahead.

Two caveats matter in practice. The released coefficients are rounded rather than exact powers of a single , so treat the formula as the design principle and the published table as the shipped configuration. More importantly, the objective is FLOPs, not latency or memory: depthwise separable convolutions have low arithmetic intensity and underuse GPU and TPU matrix units, and activation memory grows with , so the largest variants train slowly and can exhaust device memory. EfficientNetV2 addressed exactly this by replacing early MBConv stages with Fused-MBConv, capping the maximum image size, and adding training-aware search plus progressive resizing.