ML0030 Sigmoid

The sigmoid activation function transforms input values into a range between 0 and 1, making it useful in various applications like binary classification.

Advantages:
(1) Smooth, Bounded Gradient: The sigmoid’s S‑shape yields a continuous derivative, preventing abrupt changes in backpropagation and aiding stable training on shallow networks.
(2) Probability interpretation: Since the output is between 0 and 1, it can be useful for problems where predictions need to represent probabilities.

Disadvantages:
(1) Vanishing gradient problem: For very large or small inputs, the gradient becomes almost zero, slowing down training in deep networks.
(2) Not zero-centered: The outputs are always positive, which can lead to inefficient weight updates and slower convergence.