Programming Fundamentals for AI
learning the programming skills required to build, train, and understand AI systems
Core Concepts Covered
- Python programming foundations for machine learning and deep learning
- Control flow, standard libraries, and essential algorithms
- Object-oriented programming (OOP) and structural coding in AI models
Local Setup Recommendation
To execute and experiment with the code cells below on your local machine, ensure you have set up your isolated virtual environments and scientific libraries by following the detailed protocols in Unit 03: Environment Setup or run them in Google Colab.
1. Core Python Structures & List Comprehensions
Python is widely used in AI because its syntax is readable and its scientific libraries are mature. Before you build a machine learning pipeline, you need a firm grasp of the data structures that hold and organize its inputs:
• Lists ([]): Ordered, mutable collections of elements. Ideal for storing dynamic arrays of training sample paths or progressive loss metrics during training.
• Dictionaries ({}): Unordered collections of key-value pairs. Crucial for managing model configurations, mapping class labels to index integers (e.g., {'cat': 0, 'dog': 1}), or organizing hyperparameter dictionaries.
• Tuples (()): Ordered, immutable sequences of elements. Because they cannot be modified after instantiation, they are used to represent structural properties that must remain constant throughout execution, such as the dimensions of an input tensor (e.g., shape = (3, 224, 224) for a 3-channel image).
List Comprehensions Demystified: A list comprehension is a highly optimized, Pythonic construct used to map or filter iterables. Standard Python for loops have considerable execution overhead due to the dynamic interpreter looking up list append methods on each iteration. In contrast, list comprehensions execute at compiled C-speed under the hood.
Step-by-Step Trace: Let's look at the task of filtering a range of numbers ( to ), selecting only the odd values, and squaring them. In a standard loop, the interpreter creates an empty list in memory, runs a loop, evaluates the condition, and manually calls .append() repeatedly. In a list comprehension, the entire operation is evaluated as a single optimized bytecode instruction, generating the list in place and significantly reducing execution time.
# Standard looping approach
squares = []
for x in range(10):
if x % 2 != 0:
squares.append(x ** 2)
print("Standard loop:", squares)
# Pythonic List Comprehension
py_squares = [x ** 2 for x in range(10) if x % 2 != 0]
print("Comprehension:", py_squares)Standard loop: [1, 9, 25, 49, 81]
Comprehension: [1, 9, 25, 49, 81]2. Object-Oriented Programming (OOP) in Deep Learning
Frameworks such as PyTorch and TensorFlow represent neural networks with objects and classes. A class gives the model one organized place to keep its layers, weights, and behavior instead of scattering those pieces across unrelated variables.
Class Inheritance in Deep Learning: When building a custom neural network, we do not write weight tracking or backpropagation logic from scratch. Instead, our model class inherits from a base library module (like PyTorch's torch.nn.Module). This base class automatically registers all internal parameters (weights and biases) so they can be optimized by gradient descent.
Constructors and super() Overloads:
• Constructor (__init__): The initialization method that runs when a new model object is created. Here, we instantiate the structural layers of our network (e.g., fully-connected layers or convolutions) and store them as instance attributes (using self).
• super().__init__(name) Overload: This statement is critical. It invokes the constructor of the parent base class (SimpleModelBase or nn.Module), ensuring that the parent's internal state, parameter registers, and parameter tracking hooks are fully and correctly initialized in memory before our subclass adds its custom layers.
class SimpleModelBase:
def __init__(self, name):
self.name = name
def run_inference(self, x):
raise NotImplementedError("Subclasses must implement run_inference")
# Inheriting base functionality
class LinearClassifier(SimpleModelBase):
def __init__(self, name, weight, bias):
# Call the parent constructor first to initialize 'name'
super().__init__(name)
self.weight = weight
self.bias = bias
def run_inference(self, x):
# Calculate: f(x) = wx + b
return self.weight * x + self.bias
model = LinearClassifier("LinearRegressionNode", 2.5, 1.0)
print(f"Model: {model.name} | Prediction for x=4: {model.run_inference(4)}")Model: LinearRegressionNode | Prediction for x=4: 11.03. Vectorization vs. Loop Iteration (NumPy Memory Layout)
Standard Python works well for coordinating a program, but it is slow at repeating the same calculation over a large collection of numbers. The reason starts with how Python lists are laid out in memory.
Standard Python Lists (Array of Pointers): A Python list stores references to separate Python objects rather than packing the raw values into one continuous block. Each object carries information such as its type and reference count. As a loop walks through the list, the CPU follows those references from place to place, which adds overhead and can lead to cache misses.
NumPy Arrays (Contiguous Homogeneous Memory): A NumPy array keeps values of one data type, such as float64, together in a contiguous block of memory. The CPU can load and process that regular block much more efficiently through its cache.
Vectorization: NumPy takes advantage of this layout by handing whole-array operations to optimized C and C++ routines. Instead of asking the Python interpreter to inspect one element at a time, a vectorized operation can apply parallel SIMD instructions across a block of values, often producing a large speedup.
import time
# List-based multiplication loop (Slow pointers)
size = 1000000
python_list = list(range(size))
start_time = time.time()
multiplied_list = [x * 2 for x in python_list]
list_duration = time.time() - start_time
print(f"Python list loop duration: {list_duration:.4f} seconds")
# NumPy Vectorized array multiplication (Fast contiguous memory)
import numpy as np
numpy_array = np.arange(size)
start_time = time.time()
multiplied_array = numpy_array * 2 # Vectorized calculation
array_duration = time.time() - start_time
print(f"NumPy vectorized duration: {array_duration:.4f} seconds")
print(f"Vectorized speedup: {list_duration / array_duration:.1f}x")Python list loop duration: 0.0824 seconds
NumPy vectorized duration: 0.0011 seconds
Vectorized speedup: 74.9xInteractive Practice Quiz
Test your understanding with instant feedback
What will be the output of the following list comprehension: [x for x in range(5) if x % 2 == 0]?
Why is Object-Oriented Programming (OOP) so crucial in modern Deep Learning frameworks like PyTorch?
Which of the following describes why 'Vectorization' in NumPy is significantly faster than standard Python loops?
What is the purpose of calling the super().__init__() statement inside a subclass constructor?
Which of the following standard Python collection types is immutable (its contents cannot be modified after creation)?
In Python, which of the following is an example of an ordered, mutable collection?
Why does iterating over a standard Python list with a 'for' loop incur substantial computational overhead compared to NumPy arrays?
How does a NumPy array's memory layout differ from a standard Python list?
Which term describes performing mathematical calculations on entire arrays simultaneously using optimized C backends instead of element-by-element loops?
What is the purpose of the constructor (__init__) method in Python OOP?
Further Readings
Explore these highly recommended external references to deepen your understanding