Unit 04

Data Fundamentals for Machine Learning

understanding how data is collected, represented, processed, and prepared for AI models

Dataset Splitting Architecture & Hygiene

TRAINING SET70% of dataset

Model learns weights

VAL15%

Hyperparameter tuning

TEST15%

Final checks

← Learn parameters

Tuning loop feedback ←

✕ Leakage Barrier (Keep Isolated) ✕

Core Concepts Covered

Data types, structured vs unstructured data
Preprocessing pipelines: cleaning, normalizing, scaling, and feature engineering
Dataset partitioning: Training, Validation, and Testing sets

Local Setup Recommendation

To execute and experiment with the code cells below on your local machine, ensure you have set up your isolated virtual environments and scientific libraries by following the detailed protocols in Unit 03: Environment Setup or run them in Google Colab.

1. Data Types: Structured vs. Unstructured

A machine learning model can calculate only with numbers, vectors, and matrices. Before we train one, we need to decide how the information we care about will take one of those numerical forms. That starts with recognizing whether the raw data is structured or unstructured.

• Structured Data: This data already fits a consistent layout, such as a table, schema, or relational database. In a CSV file of houses, for example, each row might describe one house while columns hold its age, size, and price.

• Unstructured Data: Text, images, audio, and video do not arrive as neat rows of named features. A model needs a representation that turns their useful patterns into numbers. A Convolutional Neural Network (CNN), for example, can turn an image's pixel grid into a vector that describes the visual features it found.

Jupyter Code Notebook Cell

Python 3 (ipykernel)

In [1]:

import pandas as pd

# Load, structure, and explore tabular datasets
raw_data = {
    'Feature_Age': [25, 47, 31, 19, 52],
    'Feature_Income': [52000, 115000, 71000, 24000, 142000],
    'Target_Purchased': [0, 1, 1, 0, 1]
}

df = pd.DataFrame(raw_data)
print("Dataframe Shape:", df.shape)
print("Summary Statistics:\n", df.describe().round(1))

Out [1]:

Dataframe Shape: (5, 3)
Summary Statistics:
        Feature_Age  Feature_Income  Target_Purchased
count          5.0             5.0               5.0
mean          34.8         80800.0               0.6
std           13.4         49660.9               0.5
min           19.0         24000.0               0.0
max           52.0        142000.0               1.0

Worked Example 1

Separate Features from the Target

Problem

Three houses have $(\text{area},\text{bedrooms},\text{price in thousands})$ values $(1000,2,200)$ , $(1500,3,300)$ , and $(2000,4,400)$ . Build the feature matrix $X$ and target vector $y$ , then state their shapes.

Step-by-step solution

1.Each house is one row. Area and bedroom count are the input features, so $X=\begin{bmatrix}1000&2\\1500&3\\2000&4\end{bmatrix}$ .

2.Price is the value to predict, so $y=\begin{bmatrix}200\\300\\400\end{bmatrix}$ .

Final answer and interpretation

$X$ has shape $3\times2$ and $y$ has shape $3\times1$ : three examples, two input features, and one target per example.

Choosing which columns belong in $X$ and which belong in $y$ is part of turning a real table into a supervised-learning problem.

2. Feature Engineering: Normalization vs. Standardization

In raw tabular datasets, features often span completely different mathematical scales. For instance, in a real estate dataset, the number of bedrooms spans $1-5$ , while annual household income ranges from 20,000 to 200,000 dollars.

Why Scaling Matters: When one feature is numerically much larger than another, it can stretch the loss contours into a long, narrow shape. Gradient Descent may then bounce across the narrow direction instead of moving cleanly toward the minimum. Scaling puts the features on comparable ranges, which usually makes the optimization path more direct.

We solve this using two primary scaling techniques:

• Normalization (Min-Max Scaling): Linearly shifts and scales all values so they lie strictly within a bounded interval, typically $[0, 1]$ . It is ideal when you know your data boundaries have no extreme outliers. The formula is:

$x_{\text{scaled}} = \frac{x - x_{\text{min}}}{x_{\text{max}} - x_{\text{min}}}$

• Standardization (Standard Scaling): Centers the feature values to have a mean ( $\mu$ ) of $0$ and a standard deviation ( $\sigma$ ) of $1$ . Unlike normalization, it does not restrict values to a hard range, making it highly robust to extreme outliers. The formula is:

$x_{\text{standardized}} = \frac{x - \mu}{\sigma}$

Jupyter Code Notebook Cell

Python 3 (ipykernel)

In [1]:

from sklearn.preprocessing import MinMaxScaler, StandardScaler
import numpy as np

# Sample incomes
incomes = np.array([[52000], [115000], [71000], [24000], [142000]])

# Apply Min-Max Normalization [0, 1]
norm_scaler = MinMaxScaler()
normalized = norm_scaler.fit_transform(incomes)

# Apply Standard Scaling (Mean=0, Std=1)
std_scaler = StandardScaler()
standardized = std_scaler.fit_transform(incomes)

print("Original incomes:\n", incomes.flatten())
print("Min-Max Normalized:\n", normalized.flatten().round(2))
print("Standardized (z-score):\n", standardized.flatten().round(2))

Out [1]:

Original incomes:
 [ 52000 115000  71000  24000 142000]
Min-Max Normalized:
 [0.24 0.77 0.4  0.   1.  ]
Standardized (z-score):
 [-0.65  0.77 -0.22 -1.28  1.38]

Worked Example 1

Scale One Value Two Ways

Problem

A feature value is $x=100$ . The training feature has minimum $40$ , maximum $120$ , mean $80$ , and standard deviation $20$ . Calculate its Min-Max normalized value and standardized value.

Step-by-step solution

1.Min-Max scaling gives $x_{\text{scaled}}=\frac{100-40}{120-40}=\frac{60}{80}=0.75$ .

2.Standardization gives $x_{\text{standardized}}=\frac{100-80}{20}=1$ .

Final answer and interpretation

The value lies 75% of the way across the observed training range and one training standard deviation above the mean.

Fit the minimum, maximum, mean, and standard deviation on the training set, then reuse those same values for validation, testing, and inference.

3. Training, Validation, and Testing Split Hygiene

To find out whether a model can handle data it has not seen before, we split the dataset into three separate parts. Each part has a different job:

1. Training Set ( $70\%$ - $80\%$ ): The data passed directly to our optimization algorithms to fit the model parameters (weights and biases).

2. Validation Set ( $10\%$ - $15\%$ ): Used during development to monitor model generalization, tune hyperparameters (like learning rate or model architecture size), and trigger early stopping if the model begins to overfit.

3. Testing Set ( $10\%$ - $15\%$ ): Kept completely isolated in a vault until the end of development. It serves as an uncorrupted final benchmark to measure the model's true real-world generalization performance on novel inputs.

• The Threat of Data Leakage: Data leakage occurs when information from the validation or testing set is unintentionally exposed to the model during training. A common source of leakage is scaling: if you compute the global mean ( $\mu$ ) and variance ( $\sigma$ ) using the entire dataset before splitting, the model 'leaks' properties of the test set into its training phase, yielding artificially optimistic testing scores that collapse when deployed to actual production environments.

Jupyter Code Notebook Cell

Python 3 (ipykernel)

In [1]:

from sklearn.model_selection import train_test_split
import numpy as np

# Generate a synthetic dataset (100 samples)
X, y = np.arange(200).reshape(100, 2), np.arange(100)

# Correct split workflow: 1st split isolates test partition (20%)
X_train_val, X_test, y_train_val, y_test = train_test_split(X, y, test_size=0.20, random_state=42)

# 2nd split divides train_val partition into train (75%) and val (25% of 80% = 20% overall)
X_train, X_val, y_train, y_val = train_test_split(X_train_val, y_train_val, test_size=0.25, random_state=42)

print(f"Total dataset: 100% (100 samples)")
print(f"Training split: {len(X_train)}% ({len(X_train)} samples)")
print(f"Validation split: {len(X_val)}% ({len(X_val)} samples)")
print(f"Testing split: {len(X_test)}% ({len(X_test)} samples)")

Out [1]:

Total dataset: 100% (100 samples)
Training split: 60% (60 samples)
Validation split: 20% (20 samples)
Testing split: 20% (20 samples)

Worked Example 1

Calculate a 60/20/20 Split

Problem

A dataset has $1000$ examples. First reserve $20\%$ for testing. Then use $25\%$ of the remaining examples for validation. How many examples are in each split?

Step-by-step solution

1.The test set receives $0.20(1000)=200$ examples, leaving $800$ examples.

2.Validation receives $0.25(800)=200$ examples.

Final answer and interpretation

Training receives the remaining $800-200=600$ examples, so the final split is $600/200/200$ , or $60\%/20\%/20\%$ .

Split before fitting preprocessors. A scaler fitted on all 1000 examples would leak information from the 400 validation and test examples.

Interactive Practice Quiz

Test your understanding with instant feedback

QUESTION 01

Which of the following scenarios represents unstructured data?

QUESTION 02

Which feature scaling formula converts values into a bounded interval strictly between 0 and 1?

QUESTION 03

Why is 'Standardization' (Standard Scaling) often preferred over 'Min-Max Normalization' in the presence of extreme outliers?

QUESTION 04

Which of the following defines 'Data Leakage' in machine learning workflows?

QUESTION 05

To prevent data leakage, when should feature scalers (like StandardScaler) be 'fit'?

QUESTION 06

Which of the following describes structured data compared to unstructured data?

QUESTION 07

What is the mathematical difference between Normalization (Min-Max Scaling) and Standardization (Standard Scaling)?

QUESTION 08

Why does unscaled feature data distort the 'Loss Contour' landscape during gradient descent?

QUESTION 09

Which dataset partition is used during development to compare model architectures, tune human-set hyperparameters, and trigger early stopping?

QUESTION 10

Data Fundamentals for Machine Learning

Core Concepts Covered

Local Setup Recommendation

1. Data Types: Structured vs. Unstructured

Separate Features from the Target

2. Feature Engineering: Normalization vs. Standardization

Scale One Value Two Ways

3. Training, Validation, and Testing Split Hygiene

Calculate a 60/20/20 Split

Interactive Practice Quiz

Which of the following scenarios represents unstructured data?

Which feature scaling formula converts values into a bounded interval strictly between 0 and 1?

Why is 'Standardization' (Standard Scaling) often preferred over 'Min-Max Normalization' in the presence of extreme outliers?

Which of the following defines 'Data Leakage' in machine learning workflows?

To prevent data leakage, when should feature scalers (like StandardScaler) be 'fit'?

Which of the following describes structured data compared to unstructured data?

What is the mathematical difference between Normalization (Min-Max Scaling) and Standardization (Standard Scaling)?

Why does unscaled feature data distort the 'Loss Contour' landscape during gradient descent?

Which dataset partition is used during development to compare model architectures, tune human-set hyperparameters, and trigger early stopping?

In dataset hygiene, why must the 'Testing Set' be kept completely separated in a 'vault' until final model compilation?

Further Readings