ML0041 Concept of NN

Written by

Please explain the concept of a Neural Network.

Answer

A neural network (NN) is a machine learning model composed of layers of interconnected neurons. It learns patterns in data by adjusting weights through training, enabling tasks like classification, regression, and more. Neural networks are inspired by biology (they are computer systems modeled after the human brain’s network of neurons), and they excel at identifying complex, non-linear patterns, which makes them suitable for image recognition, natural language processing, and data classification.

(1) Layered Structure: A neural network consists of an input layer, one or more hidden layers, and an output layer; data flows from input to output through the hidden layers.
(2) Neurons And Activation: Each neuron computes a weighted sum of its inputs, adds a bias, and applies an activation function. Weights and biases are learnable parameters adjusted during training, and activation functions (e.g., ReLU, sigmoid) introduce the non-linearity that lets the network model complex relationships.
(3) Learning Process: The network learns by adjusting weights and biases through training algorithms such as backpropagation, minimizing the error between its predictions and the actual results.

Feedforward neural network with a three node input layer, a five node hidden layer, and a two node output layer, fully connected

Figure 1: A feedforward neural network: every neuron in one layer connects to every neuron in the next. Each connection carries a learned weight; each hidden and output neuron adds a bias and applies an activation function.

Mathematical Formulation:
$a_j = f\Big(\sum_{i} w_{ji}\, x_i + b_j\Big)$

Where:

$x_i$ are the inputs to the neuron (raw features for the input layer, previous-layer activations otherwise).
$w_{ji}$ is the weight on the connection from input $i$ to neuron $j$ ; $b_j$ is the neuron’s bias; both are learned during training.
$f(\cdot)$ is the activation function (ReLU, sigmoid, tanh, …); $a_j$ is the neuron’s output passed to the next layer.