Essential Mathematics Foundations

Reading modern artificial intelligence and deep learning research papers requires being comfortable with standard mathematical notations and symbolic representations. Let's break down the most essential notations and step-by-step expansions:

• Sigma Summation ($\sum_{i=1}^n x_i$): Represents the sum of a sequence of numbers starting from index $i=1$ up to $n$. Let's trace a concrete example: if we evaluate the expression $\sum_{i=1}^3 2^i$, we expand it by substituting $i$ with $1$, then $2$, then $3$, and summing the terms: $$\sum_{i=1}^3 2^i = 2^1 + 2^2 + 2^3 = 2 + 4 + 8 = 14$$

• Pi Product ($\prod_{i=1}^n x_i$): Represents multiplying a sequence of numbers starting from index $i=1$ up to $n$. For example, the product $\prod_{i=1}^3 (i + 1)$ expands as: $$\prod_{i=1}^3 (i + 1) = (1+1) \times (2+1) \times (3+1) = 2 \times 3 \times 4 = 24$$

• Set Notation ($\in, \subset, \mathbb{R}, \mathbb{R}^d$): - $x \in S$ means '$x$ is an element of set $S$'. - $\mathbb{R}$ represents the set of all real continuous numbers. - $\mathbb{R}^d$ represents a $d$-dimensional coordinate vector space of real numbers. *Example:* In deep learning, an image or word is often represented as a flat vector of $512$ numbers, meaning the vector lies in the $\mathbb{R}^{512}$ space.

Logarithmic and exponential functions are central to machine learning loss functions (like cross-entropy loss) and probability updates.

• Exponential Function ($e^x$): The base $e$ (Euler's number $\approx 2.71828$) is used in calculus and machine learning because its derivative is also $e^x$. This unique mathematical property makes calculating gradient updates incredibly elegant. Exponential functions are used to convert arbitrary real numbers into positive numbers.

• Logarithmic Function ($\ln(x)$ or $\log_e(x)$): The natural logarithm is the inverse of $e^x$ (i.e., $\ln(e^x) = x$). Some key properties used repeatedly in ML calculations include: - Multiplication inside: $\ln(ab) = \ln(a) + \ln(b)$. In probabilistic models, multiplying many small probabilities together causes floating-point numerical underflow (rounding to zero on computers). Taking the logarithm converts these multiplications into additions of negative numbers, preserving numerical stability. - Exponents inside: $\ln(a^b) = b \ln(a)$. This allows us to pull down exponential powers inside probability equations, simplifying partial derivative calculations.

A mathematical function is a rule that maps an input to a unique output ($y = f(x)$). In machine learning, neural networks are represented as compound Composite Functions (e.g., $f(g(h(x)))$), where the output of one layer ($h(x)$) serves as the input to the next layer ($g(x)$).

A System of Linear Equations ($Ax = b$) represents multiple linear relationships that must be satisfied simultaneously. We use matrices to write and solve these systems compactly, which serves as the core mathematical framework of linear regressions, projection matrices, and deep network weight layers.