10.3. Working of a Neural Network

• The input layer receives data, e.g., numbers representing an image or a sentence.

In the first step of a neural network's functioning, the input layer plays a critical role. This layer is the gateway through which data enters the neural network. The data can take many forms—numbers, pixel values representing images, words in a sentence, etc. Essentially, this step prepares and formats the data so that it can be processed further in the neural network.

Think of the input layer like the entry point to a library. Just as visitors bring in books to read, the input layer receives information that will be analyzed by the neural network.

• Each input is multiplied by a weight, and the weighted sum is calculated.

Once the data is received, the next step involves multiplying each input by a corresponding weight. Weights are numerical values assigned to each connection in the network, reflecting the importance of the input in relation to what the network is trying to learn. This operation—multiplying the inputs by weights and summing them up—results in a single numerical value known as the weighted sum, which serves as the basis for further processing in the network.

Imagine you’re ordering ingredients for a recipe. Each ingredient (input) is multiplied by how much you need (weight) to find out the total amount needed for the dish (weighted sum).

• A bias is added to the weighted sum to fine-tune the output.

In this step, a bias value is added to the weighted sum calculated in the previous step. The purpose of the bias is to adjust the output and provide a degree of freedom to the model. This adjustment helps the neural network better fit the data it learns from, allowing it to make predictions that are less rigidly bound to the input data alone. Essentially, the bias acts as a way to shift the activation function, influencing how the network interprets the data.

Think of the bias like adding salt to your meal after cooking. While the main ingredients create the dish (weighted sum), the salt adjusts the flavor to your liking (fine-tuning the output).

• The result goes through an activation function like: o Sigmoid: Output between 0 and 1. o ReLU (Rectified Linear Unit): Outputs 0 if negative, otherwise the input. o Tanh: Output between -1 and 1.

After adding the bias, the resultant value is processed through an activation function. This step determines whether a neuron should be activated or not based on its input. Different types of activation functions are used, such as Sigmoid, which compresses the output between 0 and 1; ReLU, which outputs the input directly if it's positive and zero if it’s negative; and Tanh, which gives an output between -1 and 1. The activation function introduces non-linearity into the network, enabling it to learn complex patterns.

Consider the activation function like a light switch. If the input (current) is strong enough (like a switch being flipped), the light (output) can turn on; if not, it stays off. Each type of activation function represents a different type of switch or control mechanism.

• The final result is passed to the next layer or shown as output.

In the final step, the processed result from the previous operations is either passed to the next layer in the neural network or presented as the final output if it is the last layer. This output can take various forms such as a numerical value, a class label for a classification task, or any other representation that indicates what the network has learned from the input data. This step is crucial as it fulfills the network's purpose of making predictions or providing insights based on the data it has processed.

This step can be likened to a teacher providing feedback after grading a student’s exam. The output is the final assessment of the student’s performance based on all the information processed throughout the test (like a neural network processing inputs).

--