Introduction to Artificial Intelligence and Machine Learning

Artificial Intelligence (AI) is the field of building computer systems that can do things we usually associate with human intelligence, including reasoning through a problem, recognizing what is in an image, understanding language, and deciding what to do next.

AI became a formal field of study in the middle of the twentieth century. At the 1956 Dartmouth Workshop, researchers including John McCarthy, Marvin Minsky, and Claude Shannon asked a bold question: could learning and intelligence be described clearly enough for a machine to simulate them?

The first wave of AI relied on Symbolic AI and hand-coded Expert Systems. In this paradigm, human specialists manually mapped out databases of static rules, logical statements, and 'if-then' paths. For instance, to build a medical diagnostic expert system, doctors had to manually write rules like: IF patient_has_fever AND patient_has_cough THEN suspect_flu.

The Self-Driving Analogy: Imagine programming an autonomous car strictly using an expert system. You would have to write explicit instructions for every single possible scenario: if pedestrian_distance < 5 meters AND speed > 20: apply_brakes(). Now, consider the chaos of a real-world city intersection: rain blurring the camera lens, a bicycle swerving, or a shadow on the road. A rule-based program is too brittle and collapses instantly when faced with ambiguous, real-world inputs because humans cannot manually hand-code an infinite number of rules.

Machine learning takes a different approach. Instead of writing a rule for every situation, we show an algorithm many examples, such as thousands of hours of driving footage, and let it find useful patterns in the data. That idea sits at the heart of modern Machine Learning.

These terms are easy to mix up. It helps to picture them as nested categories rather than competing technologies:

• Artificial Intelligence (AI): The broadest category. It includes any system that behaves intelligently, whether it follows hard-coded rules (like a chess search tree) or learns through mathematical algorithms.

• Machine Learning (ML): A statistical subset of AI. Rather than relying only on rules written by a person, an ML algorithm studies historical data and improves at a particular task. Example: A spam filter can compare word frequencies in known spam and inbox messages, then adjust the probability it assigns to a new email.

• Deep Learning (DL): A subset of ML built with Artificial Neural Networks that have many layers, which is where the word 'deep' comes from. Those layers learn useful features directly from raw data. Example: Traditional facial-recognition systems depended on measurements chosen by people, such as eye width or nose length. A deep network can start with pixels, detect edges in early layers, combine them into shapes, and eventually recognize a face.

• Generative AI (GenAI): A modern subset of Deep Learning that creates new content. Instead of only classifying an input, such as deciding whether a picture shows a dog, a generative model learns patterns in a large collection of examples and uses those patterns to produce something new. Example: Large Language Models (LLMs) generate text one token at a time, while Diffusion Models can turn a text prompt into an image.

So what does learning look like in practice? A model works through data, adjusts its internal parameters when it makes mistakes, checks its progress, and repeats the process. The rest of this section breaks that cycle into its main parts.

• Dataset Partitioning (The Train-Val-Test Hygiene): Learning begins with data, but we must split our dataset into three separate, uncorrupted sets to evaluate our model's true generalization capacity: 1. Training Set ($70\%$-$80\%$): The data our optimization algorithm repeatedly looks at to learn patterns and adjust internal weights. 2. Validation Set ($10\%$-$15\%$): The data used to test the model during development. We use its performance to tune hyperparameters and decide when to stop training to prevent overfitting. 3. Testing Set ($10\%$-$15\%$): Held back completely in a 'vault'. It is used exactly once at the very end to check how our final model performs on novel, unseen data before deploying it to production.

• Parameters vs. Hyperparameters: - Parameters ($\theta$): The internal mathematical variables (like weights $w$ and biases $b$) that the model adjusts on its own during training to minimize error. Analogy: The self-adjusting cruise control dials in an autonomous car. - Hyperparameters: The external configurations set by the human developer before the training loop begins (such as the learning rate $\eta$, model depth, or batch size). They dictate how the model is allowed to learn. Analogy: The maximum speed limit or safety margin set by the human driver before turning on autopilot.

• Iterative Learning Loop: Training is not one calculation. The model repeats a feedback loop over many small batches of the training set: 1. Forward pass: Use the current parameters to turn inputs $x$ into predictions $\hat{y} = f_{\theta}(x)$. 2. Measure loss: Compare predictions with the correct targets $y$ using a loss function $\mathcal{L}(\hat{y}, y)$; a lower loss means the predictions are closer to the desired answers. 3. Backpropagation: Calculate how each parameter contributed to the loss by finding its gradient $\nabla_{\theta}\mathcal{L}$. 4. Optimizer update: Move the parameters a small step in the direction that reduces loss: $$\theta \leftarrow \theta - \eta\nabla_{\theta}\mathcal{L}$$ 5. Repeat and validate: Process the next batch, then repeat for many passes through the dataset (epochs). Monitor validation performance so the model learns general patterns instead of memorizing the training set.

• Training vs. Inference: - Training: The computationally intensive phase where the model iteratively processes the training set, measures its errors (loss), and adjusts its parameters ($\theta$) to minimize those errors. - Inference: The practical deployment phase. The trained model takes a brand-new, real-world input (like a live camera feed at an intersection) and outputs a prediction instantly using its fixed, learned parameters.