ML0046 Forward Propagation

Forward propagation is the process by which a neural network takes an input and generates a prediction: the input data is passed systematically through each layer of the network. At every neuron, a weighted sum of the previous layer’s outputs (plus a bias) is computed, then a non-linear activation is applied; this repeats layer by layer until the output layer emits the final prediction. It is essentially the prediction phase of the network: information flows in one direction, from input to output, using the learned weights and biases.

(1) Input Layer: The network receives the raw input data.
(2) Layer-Wise Processing: Each neuron computes a linear combination (weighted sum plus bias), then applies a non-linear activation (ReLU, sigmoid, tanh) to introduce non-linearity.
(3) Propagation Through Layers: The output of one layer becomes the input to the next, progressing through all hidden layers.
(4) Output Generation: The final layer applies a task-appropriate function (softmax for classification, a linear function for regression) to produce the prediction.

Figure 1: One direction only: each layer transforms activations into pre-activations () and back into activations () until the output layer produces the prediction. No gradient information flows here; that is backpropagation’s job.

Mathematical Formulation:

Where:

• is the previous layer’s activation vector (, the input).
• are layer ‘s learned weights and biases; is the pre-activation.
• is the activation function; is the number of layers and the prediction.

Figure 2: A worked example with concrete numbers: inputs flow through fixed weights and biases; each hidden node shows its computed and sigmoid activation , ending in the scalar prediction. Every number on the diagram follows from the two formulas above.