Recently, Mate Labs co-founder and CTO wrote "Everything you need to know about Neural Networks" in Medium, from neurons to Epoch, to introduce the main core terms of neural networks.
Understanding what artificial intelligence is and how machine learning and deep learning affect it is an extraordinary experience. At Mate Labs, we have a group of self-taught engineers who hope that this article will share some of the learning experiences and shortcuts to help machine learners understand the meaning of some core terms.
Neuron (node)—The basic unit of a neural network that includes a specific number of inputs and an offset value. When a signal (value) is entered, it is multiplied by a weight value. If a neuron has 4 inputs, there are 4 weight values ​​that can be adjusted during training.
Operation of a neuron in a neural network
Connection—It is responsible for connecting neurons between the same layer or between layers. A connection always has a weight value. The goal of training is to update this weight value to reduce the loss (error).
Offset - This is an extra input to the neuron, always 1 and has its own connection weight. This ensures that there is an activation function in the neuron even when all inputs are zero.
Activation Function (Migration Function)—The activation function is responsible for introducing nonlinear features into the neural network. It compresses the value to a smaller range, ie the value range of a Sigmoid activation function is [0,1]. There are many activation functions in deep learning, and ReLU, SeLU, and TanH are more commonly used than Sigmoid. For more activation functions, see the article "Activate Function in Deep Learning".
Various activation functions
Basic neural network design
Input layer - the first layer of the neural network. It receives the input signal (value) and passes it to the next layer, but does not perform any operations on the input signal (value). It does not have its own weight value and offset value. There are 4 input signals x1, x2, x3, x4 in our network.
Hidden Layer—The hidden layer of neurons (nodes) transform the input data in different ways. A hidden layer is a set of vertically stacked neurons. The image below has 5 hidden layers, the first hidden layer has 4 neurons (nodes), the 2nd 5 neurons, the 3rd 6 neurons, the 4th 4 neurons, the 5th 3 neurons. The last hidden layer passes the value to the output layer. All the neurons in the hidden layer are connected to each other, and the same is true for each neuron in the next layer, so that we get a fully connected hidden layer.
Output layer—This is the last layer of the neural network that receives input from the last hidden layer. Through it we can get the ideal value within a reasonable range. The output layer of the neural network has three neurons, and outputs y1, y2, and y3, respectively.
Input shape—it is the shape of the input matrix we pass to the input layer. The input layer of our neural network has 4 neurons, which predicts 4 values ​​in 1 sample. The ideal input shape for this network is (1, 4, 1) if we feed it one sample at a time. If we feed 100 samples, the input shape will be (100, 4, 1). Different libraries are expected to have different format shapes.
Weight (parameter)—The weight characterizes the strength of the connection between different units. If the weight from node 1 to node 2 is of a larger magnitude, it means that God has a greater influence on neuron 2. A weight reduces the importance of the input value. A weight close to 0 means that changing this input will not change the output. Negative weight means that increasing this input will reduce the output. The weight determines the influence of the input on the output.
Forward propagation
Forward Propagation—This is the process of feeding input values ​​to a neural network and getting an output we call a predictor. Sometimes we also refer to forward propagation as inference. When we feed the input value to the first layer of the neural network, it does not perform any operations. The second layer receives the value of the first layer, then performs multiplication, addition, and activation operations, and then passes to the next layer. Subsequent layers repeat the same process, and finally we get the output value from the last layer.
iDealTek-Electronics' high-quality, high-precision DC power supply product line is built on years of production, design and technological innovation with DC power supplies. We have a wide range of DC power supply products, from portable Laboratory AC - DC power supplies to Desktop High-precision DC power supplies, to Rack-mounted and Floor-standing Cabinet-type High-power DC power supplies, as well as Intelligent Programmable DC Power Supplies. The output power covers a wide range from 300W to 2000KW, and the output voltage can reach up to 2000VDC. iDealTek-Electronics` DC power supplies are widely used in laboratories, capacitor testing, battery charging, inverter testing, DC motor testing and various cutting-edge applications.
Thanks to the mature and stable AC/DC topology matched with the rapid and efficient control loop and monitoring feedback loop, our DC power supplies can provide high-precision, low-ripple and fast-response DC output with high stability. Through the flexible and friendly man-machine operation panel equipped on the front panel of the power supplies, setting and programming control of the output voltage and current can be easily realized and the high-precision LED or LCD ensures the intuitive and high-precision output measurement function. At the same time, our DC The power supplies can be equipped with RS232 / RS485 / analog signal interfaces to realize remote control programming and status monitoring of the power supplies.
Our DC power supplies have complete built-in protection functions, such as over-voltage protection, over-current protection, over-temperature protection, etc., to ensure the long-term and reliable operation of the power supplies.
If you have a demand for DC power products, we believe that you can definitely find the suitable DC power products among our DC power supplies family.
AC - DC Power Supplies, AC DC Supplies, AC DC Power Supplies, AC to DC Power Supplies, Switching DC Power Supplies
Yangzhou IdealTek Electronics Co., Ltd. , https://www.idealtekpower.com