Machine learning and statistics with python

 Introduction: Activation functions are one of the most important components of a neural network model. Activation functions instill non-linearity into neural networks and also helps to suppress noisy data and improve signals where needed. The early idea of activation functions came into the engineering field via biological censoring of informations and how neurons stay inactive on some signal level and then activate on more than that. This phenomenon motivated early scientists to let the artificial neurons also behave in certain ways which leads it to emphasis or suppress based on different signal values.  But in current trends, activation functions are mostly motivated from mathematical point of views or even optimization or purely technical reasons. We will explore some of the most famous activation functions in this blog; will try to see why they were motivated and how/where they can become important. Summary of the article: In this article, we will explore activation fu...

 Introduction: All of us, when start out with neural network, go through pictures of networks, try and understand complex equations; get baffled by the back-propagation equations and take our time to eventually assimilate the knowledge of what all that stands for. Recently while mentoring students for our new effort mentorbruh , one of my students asked some pretty interesting yet very basic questions. This content is an effort to write down those so that it can help other students who are going through neural networks first time and have these doubts. But before that, let's brush up the basics once pretty quickly.   Abstract of this article: In this post, we are going to take small steps in explaining what is neural network, what are input,output and hidden layers; how does a node calculate its values. We will also briefly touch the concepts of bias, activation, hidden layer number count and all the related artifacts. In the more looser second part of this neural network b...