Can you explain what a fully connected layer is?
Answer
A fully connected (FC) layer, also called a dense layer, is a layer in which every neuron connects to all neurons of the previous layer. It computes a weighted sum of its inputs, adds a bias, and applies an activation function, letting the network learn global combinations of features. FC layers typically appear at the end of a network, mapping extracted features to class scores.
(1) Dense Connectivity: With 
inputs and 
neurons, the layer holds 
weights plus biases.
(2) Parameter Heavy: FC layers often dominate model size (e.g., flattening a 
map into 1000 classes needs ~25M parameters), which raises overfitting risk.
(3) Loses Spatial Structure: Inputs must be flattened, discarding spatial layout; Global Average Pooling (GAP) is a parameter-free alternative.
Mathematical Formulation:
Where:
is the flattened input vector and is the output vector.- is the weight matrix and is the bias vector.
- is a non-linear activation such as ReLU (or softmax at the classifier output).
is the flattened input vector and 
is the output vector.
is the weight matrix and 
is the bias vector.
is a non-linear activation such as ReLU (or softmax at the classifier output).
Figure 1: In an FC layer, every input neuron connects to every output neuron; each output is 
.
Flatten + FC vs GAP + FC: For a CNN head with 
feature maps and 6 classes, flattening feeds 392 values to the FC layer, while GAP feeds only 8, a 44× parameter reduction with less overfitting:
| Approach | FC Input Size | FC Parameters | Spatial Info |
|---|---|---|---|
| Flatten + FC |  |  | Discarded by flattening |
| GAP + FC |  (one per channel) |  | Summarized per channel |
Figure 2: GAP collapses each feature map to one value before the FC layer, cutting parameters from 2358 to 54 in this example.
Leave a Reply