Artificial Neural Networks Illustrated From Scratch. No Code. No Complicated Math. Part 2.
No knowledge of calculus, programming, or linear algebra required. Conceptualizing neural networks and building a useful framework and mental model on what they are and how they work.
In part 2, we will first cover some terminologies, then we will use these terminologies to help us understand the beauty and efficiency of neural networks in design and in mathematics.
We will use one of the neural networks from Part 1. Please read part 1 first.
For the simple neural network above, we can illustrate everything easily, but to efficiently build and design very large neural networks with potentially millions of input numbers at once, and sometimes millions of output numbers at once, we need to introduce a few terms.
For a neural network, we have input number(s). They will go through some mathematical processing. Then we get output number(s).
We will introduce a concept called layers. We will have an input layer that are the input numbers, an output layer that will give the output numbers. And any layers in between would be called 'hidden' layers.
Another example, with two hidden layers.
When designing a neural network, you can have as many nodes in any layer as you need, depending on the task at hand. And you can have as many hidden layers as you want.
Connecting the Layers
Now as you can perhaps see from the arrows, the numbers and calculations flow from left to right, in a sequential manner.
Number(s) are first given to the input layer.
They are then multiplied by some weight parameters (not shown, refer to Pt. 1; each arrow is a separate multiplication).
After multiplications, they converge at the first hidden layer, to be added with the nodes' bias parameters (not shown, refer to Pt. 1; each node has an inherent bias parameter).
Then they go on to repeat 2 and 3 until reaching the output layer, where the result of the final computations would be given as output number(s).
Now, there are many different ways that we can name a layer and define what it can do to the numbers given to it, but we will just focus on one way right now; that is, fully connected.
If a layer is fully connected, that means that every one of its nodes is connected to every single node from the preceding layer.
The input layer is not fully connected because it has no preceding layer.
The first hidden layer is fully connected, as each of its 4 nodes is connected to each of the three node in the preceding layer (input layer). 3*4=12 connections.
The second hidden layer is also fully connected, as each of its 3 nodes is connected to each of the 4 nodes in the preceding layer (first hidden layer). 4*3=12 connections.
The output layer is also fully connected. 3*1=3 connections.
The fully connected layers are also known as dense layers, precisely because everything is connected to everything in the previous layer. Imagine, if a preceding layer has just 1000 nodes, and we have a fully connected layer right after it, we would have 1000*1000 = 1 million connections.
But WHY these Terms?
If you are wondering why we should care about these terminologies, it is because they make it easy to conceptualize and build extremely complicated neural networks. For the example above, I only have to describe in a few words how the neural network is setup and how the layers are connected, for someone else to replicate or understand what neural network I am building. This one has an input layer with 3 nodes, followed by a fully connected layer with 4 nodes, followed by a fully connected layer with 3 nodes, and an output layer with 1 node.
I could easily tell you, I have designed this big neural network that works great for some particular task, here is how to build it: an input layer with 1 million parameters (one for each pixel of a 1000*1000 image), followed 5 hidden layers in sequence, each of which are fully connected layers with 5,000 nodes, then an output layer with 1 node as the output.
Illustrations are useful in demonstrating the concepts of how neural networks are built, but to actually build the neural networks with billions of numbers and mathematical operations, we will need to familiarize ourselves with the basic building blocks, organize them into bigger blocks represented certain terms so that we can understand and design their blueprints without drawing out every little 'brick'.
If you are interested in how Artificial Intelligence works — how the data is structured, how networks are trained, and how learning works — checkout this book at Amazon, titled Artificial Intelligence for Peanut Brains: Illustrated. An AI book that doesn’t make you feel peanut-brained.
Books on AI make us feel like we have peanuts for brains. But it’s not our fault.
Artificial Intelligence is a complicated subject but anything should and can be learned intuitively.
This book strives to introduce any ordinary folk to the field of AI, how AI programs work, how they have intelligence, how they learn, what are needed to build them, and so on.
Things not included in this book: Complicated math, Calculus, Code, and Abstract Concepts
This would be the first of a series of books I have on artificial intelligence and neural networks. Follow for updates on EasyLearn.AI or Twitter (@EasyLearnAI).