Unit 05

Essential Mathematics Foundations

reviewing the mathematical concepts required before advanced machine learning mathematics

Core Concepts Covered

Mathematical notations and symbols used in AI research papers
Functions, systems of equations, and exponential behaviors
Algebraic manipulation and notation

1. Demystifying Mathematical Notation

Research papers often compress a full calculation into a few symbols. Once you learn the common notation, those expressions become instructions you can unpack one step at a time:

• Sigma Summation ( $\sum_{i=1}^n x_i$ ): Adds a sequence of terms while an index moves from its starting value to its ending value.

• Pi Product ( $\prod_{i=1}^n x_i$ ): Multiplies a sequence of terms over the stated index range.

• Set Notation ( $\in, \subset, \mathbb{R}, \mathbb{R}^d$ ): - $x \in S$ means ' $x$ is an element of set $S$ '. - $\mathbb{R}$ represents the set of all real continuous numbers. - $\mathbb{R}^d$ represents a $d$ -dimensional coordinate vector space of real numbers. Example: In deep learning, an image or word is often represented as a flat vector of $512$ numbers, meaning the vector lies in the $\mathbb{R}^{512}$ space.

Jupyter Code Notebook Cell

Python 3 (ipykernel)

In [1]:

# Standard programmatic implementation of Summation and Product notations
import numpy as np

elements = np.array([2, 4, 6])

# Summation: Sigma sum
sigma_sum = np.sum(elements)
print("Sigma sum of [2, 4, 6] is:", sigma_sum)

# Product: Pi product
pi_product = np.prod(elements)
print("Pi product of [2, 4, 6] is:", pi_product)

Out [1]:

Sigma sum of [2, 4, 6] is: 12
Pi product of [2, 4, 6] is: 48

Worked Example 1

Expand a Sum and a Product

Problem

Evaluate $\sum_{i=1}^{3}2^i$ and $\prod_{i=1}^{3}(i+1)$ by writing out every term.

Step-by-step solution

1. $\sum_{i=1}^{3}2^i=2^1+2^2+2^3=2+4+8=14$ .

2. $\prod_{i=1}^{3}(i+1)=(1+1)(2+1)(3+1)=2\cdot3\cdot4=24$ .

Final answer and interpretation

The summation equals $14$ , while the product equals $24$ .

Always substitute the index values before trying to simplify sigma or pi notation mentally.

Worked Example 2

Check Matrix and Vector Shapes

Problem

Let $W\in\mathbb{R}^{2\times3}$ and $x\in\mathbb{R}^{3}$ . Is $Wx$ defined, and what space contains the result?

Step-by-step solution

1.The inner dimensions match: $W$ has three columns and $x$ has three entries.

2.The multiplication combines each of the two rows of $W$ with the three entries of $x$ .

Final answer and interpretation

Therefore $Wx\in\mathbb{R}^{2}$ .

For matrix multiplication, the inner dimensions must match and the outer dimensions determine the result.

2. Logarithms and Exponential Behaviors

Logarithmic and exponential functions are central to machine learning loss functions (like cross-entropy loss) and probability updates.

• Exponential Function ( $e^x$ ): The base $e$ (Euler's number $\approx 2.71828$ ) is used in calculus and machine learning because its derivative is also $e^x$ . This unique mathematical property makes calculating gradient updates incredibly elegant. Exponential functions are used to convert arbitrary real numbers into positive numbers.

• Logarithmic Function ( $\ln(x)$ or $\log_e(x)$ ): The natural logarithm is the inverse of $e^x$ (i.e., $\ln(e^x) = x$ ). Some key properties used repeatedly in ML calculations include: - Multiplication inside: $\ln(ab) = \ln(a) + \ln(b)$ . In probabilistic models, multiplying many small probabilities together causes floating-point numerical underflow (rounding to zero on computers). Taking the logarithm converts these multiplications into additions of negative numbers, preserving numerical stability. - Exponents inside: $\ln(a^b) = b \ln(a)$ . This allows us to pull down exponential powers inside probability equations, simplifying partial derivative calculations.

Jupyter Code Notebook Cell

Python 3 (ipykernel)

In [1]:

import math

# Demonstrating Logarithmic properties
a = 5.0
b = 3.0

# 1. log(ab) == log(a) + log(b)
log_ab = math.log(a * b)
log_a_plus_log_b = math.log(a) + math.log(b)
print("log(a * b) equals log(a) + log(b):", math.isclose(log_ab, log_a_plus_log_b))

# 2. log(a ** b) == b * log(a)
log_a_pow_b = math.log(a ** b)
b_times_log_a = b * math.log(a)
print("log(a ** b) equals b * log(a):", math.isclose(log_a_pow_b, b_times_log_a))

Out [1]:

log(a * b) equals log(a) + log(b): True
log(a ** b) equals b * log(a): True

Worked Example 1

Turn a Probability Product into a Log Sum

Problem

Three independent observations have probabilities $0.1$ , $0.01$ , and $0.5$ . Compute their joint likelihood and log-likelihood.

Step-by-step solution

1.The likelihood is $L=0.1\cdot0.01\cdot0.5=0.0005$ .

2. $\ln L=\ln(0.1)+\ln(0.01)+\ln(0.5)\approx-2.3026-4.6052-0.6931=-7.6009$ .

Final answer and interpretation

Exponentiating returns the original likelihood: $e^{-7.6009}\approx0.0005$ .

Adding log-probabilities represents the same model as multiplying probabilities, but it is safer numerically for long products.

3. Systems of Linear Equations & Functions

A mathematical function is a rule that maps an input to a unique output ( $y = f(x)$ ). In machine learning, neural networks are represented as compound Composite Functions (e.g., $f(g(h(x)))$ ), where the output of one layer ( $h(x)$ ) serves as the input to the next layer ( $g(x)$ ).

A System of Linear Equations ( $Ax = b$ ) represents multiple linear relationships that must be satisfied simultaneously. We use matrices to write and solve these systems compactly, which serves as the core mathematical framework of linear regressions, projection matrices, and deep network weight layers.

Worked Example 1

Solve a Two-Equation System

Problem

Solve $2x+y=7$ and $x-y=2$ .

Step-by-step solution

1.Add the equations to eliminate $y$ : $(2x+y)+(x-y)=7+2$ , so $3x=9$ .

2. $x=3$ . Substitute into $x-y=2$ : $3-y=2$ , so $y=1$ .

Final answer and interpretation

The solution is $(x,y)=(3,1)$ , which satisfies both original equations.

Elimination removes one unknown, then substitution recovers the other.

Worked Example 2

Evaluate a Composite Function

Problem

Let $h(x)=2x$ , $g(u)=u+3$ , and $f(v)=v^2$ . Evaluate $f(g(h(4)))$ .

Step-by-step solution

1.Start with the innermost function: $h(4)=2(4)=8$ .

2.Feed that result into the next function: $g(8)=8+3=11$ .

Final answer and interpretation

Apply the outer function last: $f(11)=11^2=121$ .

A composite function is evaluated from the inside outward, just like data moving forward through model layers.

Interactive Practice Quiz

Test your understanding with instant feedback

QUESTION 01

What is the value of the Pi Product expression $\prod_{i=2}^4 (i - 1)$ ?

QUESTION 02

Which of the following describes the set representation $\mathbb{R}^{512}$ in machine learning context?

QUESTION 03

Why is the logarithmic property $\ln(ab) = \ln(a) + \ln(b)$ highly valuable in probabilistic machine learning algorithms?

QUESTION 04

Simplify the natural log expression $\ln(e^{-x^2})$ ?

QUESTION 05

If a deep learning model has three consecutive layers represented by functions $f(x)$ , $g(x)$ , and $h(x)$ , how is the final output expressed as a composite function of input $x$ ?

QUESTION 06

What is the value of the Sigma Summation expression: $\sum_{i=1}^3 (2^i)$ ?

QUESTION 07

Which of the following describes the set notation $x \in S$ ?

QUESTION 08

Why is Euler's number $e \approx 2.71828$ favored as the base for exponential activation and loss derivatives?

QUESTION 09

Apply logarithmic properties to simplify the expression: $\ln(x^3 y)$ ?

QUESTION 10

Essential Mathematics Foundations

Core Concepts Covered

1. Demystifying Mathematical Notation

Expand a Sum and a Product

Check Matrix and Vector Shapes

2. Logarithms and Exponential Behaviors

Turn a Probability Product into a Log Sum

3. Systems of Linear Equations & Functions

Solve a Two-Equation System

Evaluate a Composite Function

Interactive Practice Quiz

What is the value of the Pi Product expression $\prod_{i=2}^4 (i - 1)$ ?

Which of the following describes the set representation $\mathbb{R}^{512}$ in machine learning context?

Why is the logarithmic property $\ln(ab) = \ln(a) + \ln(b)$ highly valuable in probabilistic machine learning algorithms?

Simplify the natural log expression $\ln(e^{-x^2})$ ?

If a deep learning model has three consecutive layers represented by functions $f(x)$ , $g(x)$ , and $h(x)$ , how is the final output expressed as a composite function of input $x$ ?

What is the value of the Sigma Summation expression: $\sum_{i=1}^3 (2^i)$ ?

Which of the following describes the set notation $x \in S$ ?

Why is Euler's number $e \approx 2.71828$ favored as the base for exponential activation and loss derivatives?

Apply logarithmic properties to simplify the expression: $\ln(x^3 y)$ ?

What is the compact matrix notation used to represent a system of multiple linear equations simultaneously?

Further Readings

Essential Mathematics Foundations

Core Concepts Covered

1. Demystifying Mathematical Notation

Expand a Sum and a Product

Check Matrix and Vector Shapes

2. Logarithms and Exponential Behaviors

Turn a Probability Product into a Log Sum

3. Systems of Linear Equations & Functions

Solve a Two-Equation System

Evaluate a Composite Function

Interactive Practice Quiz

What is the value of the Pi Product expression ∏i=24(i−1)\prod_{i=2}^4 (i - 1)∏i=24​(i−1)?

Which of the following describes the set representation R512\mathbb{R}^{512}R512 in machine learning context?

Why is the logarithmic property ln⁡(ab)=ln⁡(a)+ln⁡(b)\ln(ab) = \ln(a) + \ln(b)ln(ab)=ln(a)+ln(b) highly valuable in probabilistic machine learning algorithms?

Simplify the natural log expression ln⁡(e−x2)\ln(e^{-x^2})ln(e−x2)?

If a deep learning model has three consecutive layers represented by functions f(x)f(x)f(x), g(x)g(x)g(x), and h(x)h(x)h(x), how is the final output expressed as a composite function of input xxx?

What is the value of the Sigma Summation expression: ∑i=13(2i)\sum_{i=1}^3 (2^i)∑i=13​(2i)?

Which of the following describes the set notation x∈Sx \in Sx∈S?

Why is Euler's number e≈2.71828e \approx 2.71828e≈2.71828 favored as the base for exponential activation and loss derivatives?

Apply logarithmic properties to simplify the expression: ln⁡(x3y)\ln(x^3 y)ln(x3y)?

What is the compact matrix notation used to represent a system of multiple linear equations simultaneously?

Further Readings

What is the value of the Pi Product expression $\prod_{i=2}^4 (i - 1)$ ?

Which of the following describes the set representation $\mathbb{R}^{512}$ in machine learning context?

Why is the logarithmic property $\ln(ab) = \ln(a) + \ln(b)$ highly valuable in probabilistic machine learning algorithms?

Simplify the natural log expression $\ln(e^{-x^2})$ ?

If a deep learning model has three consecutive layers represented by functions $f(x)$ , $g(x)$ , and $h(x)$ , how is the final output expressed as a composite function of input $x$ ?

What is the value of the Sigma Summation expression: $\sum_{i=1}^3 (2^i)$ ?

Which of the following describes the set notation $x \in S$ ?

Why is Euler's number $e \approx 2.71828$ favored as the base for exponential activation and loss derivatives?

Apply logarithmic properties to simplify the expression: $\ln(x^3 y)$ ?