Artificial Neural Networks Illustrated From Scratch. No Code. No Complicated Math. Part 1.
No knowledge of calculus, programming, or linear algebra required. Conceptualizing neural networks and building a useful framework and mental model on what they are and how they work.
This will be a series of articles that will use illustrations and animations to help you conceptualize artificial neural networks (ANNs), understand what they are, how they work, and how to build them. For the sake of simplicity, I will, from now on, simply refer to these artificial neural networks as neural networks.
If you pick up a technical book or a programming book, you will perhaps see an image like this:
For a beginner, it's quite difficult to even think about what these seemingly simple circles and lines mean. That's why we will start from scratch: without code or advanced math to complicate things further.
What do they do?
Neural networks are simply made from a bunch of mathematical operations and numbers. That's all there is to them. But phrased this way, I might as well be saying houses are just wood and concrete. There are a lot of intricacies to neural networks, but in essence, yes, they are just math operations and numbers.
And what do they do? A neural network takes in number(s), process them using its math operations and numbers, and outputs the resulting number(s).
Sounds simple enough. That is, until I tell you that you need to build the neural networks with billions of mathematical operations and numbers that are capable of processing millions of numbers at once.
(If you want more details on what these input numbers are, how they are obtained from images or texts, or what outputted numbers mean, I explain these things in much greater detail in this book). As far as mathematicians are concerned, everything, including texts, audio, video, and images, can be turned into numbers; and therefore be able to be processed by the neural network machinery. And this series of articles will explore only the design of the architecture, not the data.
That's why we have neural networks. Neural networks are an efficient way to design and conceptualize these billions of mathematical operations that make up neural networks. They are simple and succinct, like the blueprints that architects draw to conceptualize incredibly complex buildings and architectures. Once you learn how they are drawn or designed, it will give you much less of a headache later on when you learn about the more complex neural networks architectures, such as CNN or Transformers.
1. The Simplest It Can Be.
This will be the simplest possible neural network. (This may not be even considered a neural network but we will surely build on this to make a real one.)
A neural network takes in number(s) and outputs number(s). Here, one number goes in, and one number will go out. The question mark stands for parameters. Here we have two parameters, the input number will be multiplied by a weight parameter, then added to a bias parameter. These parameters will usually be randomized at the start, before the learning process, and would update to fit a specific task as learning progresses. (Again, the focus here is purely on designing neural networks from the simplest to the more complex; the meaning of parameters and the specifics of the learning process is not covered in this series; perhaps in a separate series.)
Easy enough... A number will be given to the neural network, it will be multiplied by a parameter, then added to another parameter, then the neural network will yield a number.
2. Simplest Simplified
Parameters are almost always randomized before the learning process for the neural network. Even for the same neural network that might be trained separately, they will be randomized. Their final values will be learned through the learning process, so there is no need to explicitly draw them when we are sketching our neural network 'blueprint'. So here is the same one simplified.
The parameters are implicit or hidden now. But each arrow that points from one node to another will have one weight (multiplication) parameter implied. And each white circle/node is a bias parameter, implicit. Arrow from one node to another means that the former node's value will be multiplied by a hidden parameter, and then added to the node that the arrow points to.
Make sure you can see that the above illustration is exactly the same as the previous illustration, just with a the parameters now hidden.
3. Warming Up
Can you see the multiplication operations and the addition operations in the above illustration? How many parameters do you see?
There is a total of 4 weight parameters (4 node-to-node arrows) and 3 bias parameters (3 white nodes), making up a total of 7 parameters in this simply neural network.
If you are confused what it implies when two arrows points to the same node, here is an illustration to clarify things up:
Again, arrow means multiplying by some parameter. And when two arrows points to the same node, it just means that the result from the multiplications will be added to the bias parameter, one by one; so the latter node will end up being the sum of the parameter value, the first multiplication's result, and the second multiplication's result.
4. Starting to look like a network
A little practice for you. How many parameters in total do you see?
(You should count 8 multiplication (weight) parameters, and 5 (bias) parameters, for a total of 13.)
5. More input numbers
Now the complexity of the neural networks comes from having lots and lots of input numbers, and that's where the 'network' is most prominent. In above examples, we have been playing with single number input examples. Now we try two input numbers.
Let's introduce a second number in the input.
Now we connect the second number to the same nodes as the first number did.
Voilà, this is starting to look a lot like the neural network we saw in the beginning!
Instead of 1 number in and 1 number out, now we have 2 numbers in and 1 number out.
Each number is connected to all 4 nodes in the middle, making it 2x4=8 connections/arrows. The 4 nodes then each connects to the single node on the right, 4x1=4 connections/arrows. This totals to 12 connections/arrows, or 12 implicit weight parameters. Then we have the 5 white circle/nodes, each with an implicit bias parameter. This totals to 17 parameters.
6. Going Crazy
You figure out how many arrows and parameters there are in the above illustration. You should be able to find a shortcut to calculating the number of connections now, using basic multiplications and addition.
This is a good place to end Part 1. You have now seen how we go from the simplest neural network to a quite complex one, relatively quickly. And it’s about to get crazier in the Part 2 once I introduce you to some terminologies.
If you are interested in my take on the current landscape of Artificial Intelligence, checkout this book at Amazon, titled Artificial Intelligence for Peanut Brains: Illustrated. An AI book that doesn’t make you feel peanut-brained.
Books on AI make us feel like we have peanuts for brains. But it’s not our fault.
Artificial Intelligence is a complicated subject but anything should and can be learned intuitively.
This book strives to introduce any ordinary folk to the field of AI, how AI programs work, how they have intelligence, how they learn, what are needed to build them, and so on.
Things not included in this book:
This would be the first of a series of books I have on artificial intelligence and neural networks. Follow for updates on EasyLearn.AI or Twitter (@EasyLearnAI).