AI Formula

Basic Concepts

1. Loss Function
2. Gradient Descent Algorithm
3. Activation Function in Deep Learning
4. MLP (Multi-Layer Perceptron) Neural Network
5. CNN Convolutional Neural Network
6. Image (RGB and Grayscale)

Data Processing

Data Dimensionless

https://github.com/fengdu78/Coursera-ML-AndrewNg-Notes

• Prompt optimization

Loss Function

1. Loss Function for Regression Tasks (Used for predicting continuous values)

$$MSE = \frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2$$

1. Loss Function for Classification Tasks (Used for predicting discrete categories)
2. Loss Function for Object Detection/Segmentation Tasks

Gradient Descent Algorithm

Activation Function

$$Swish: f(x)=x⋅δ(x)=x⋅\frac{1}{1 + e^{-x}}$$

MLP Principle

Linear Transformation

$z=Wx+b$

Where:

• $W$ is weight matrix,
• $x$ is input vector,
• $b$ is bias term.

Activation Function

$ReLU:f(z)=max(0,z)$

$Sigmoid:f(z)=\frac{1}{1 + e^{-z}}$

$Tanh:f(z)=\frac{e^z - e^{-z}}{e^z + e^{-z}}f(z)$

SoftMax omitted, converts data to probability distribution

Backpropagation

$$W=W-α⋅\frac{∂L}{∂W}$$

Where α is learning rate

CNN Principle

Includes four layers

• Convolutional Layer
• Pooling Layer
• Fully Connected Layer
• Output Layer

Other Neural Networks

• Recurrent Neural Network (RNN)
• Long Short-Term Memory (LSTM)
• Gated Recurrent Unit (GRU)
• Autoencoder
• Generative Adversarial Network (GAN)
• Transformer Network
• Graph Neural Network (GNN)
• Reinforcement Learning (RL)
• Attention Mechanism

https://chatgpt.com/share/67a7ee4e-d198-8009-996d-cd7cb5e11c65

Agreement

The code part of this work is licensed under Apache License 2.0 . You may freely modify and redistribute the code, and use it for commercial purposes, provided that you comply with the license. However, you are required to:

• Attribution: Retain the original author's signature and code source information in the original and derivative code.
• Preserve License: Retain the Apache 2.0 license file in the original and derivative code.

The documentation part of this work is licensed under Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License . You may freely share, including copying and distributing this work in any medium or format, and freely adapt, remix, transform, and build upon the material. However, you are required to:

• Attribution: Give appropriate credit, provide a link to the license, and indicate if changes were made.
• NonCommercial: You may not use the material for commercial purposes. For commercial use, please contact the author.
• ShareAlike: If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.