2025 Fall UVA CS - Machine Learning



TuTh 3:30pm-4:45PM

Introduction

1Basic

Att: the following markdown text was generated from the corresponding powerpoint lecture file automatically. Errors and misformatting, therefore, do exist (a lot!)!

Lecture 1 Study Guide

Course Logistics and Overview

Course Staff and Communication

Course Materials

Background Needed

Assessment Breakdown

Course Format (2025F Flipped Classroom - Starting Week 3)

In-Class Activities

Emphasis

Key Objectives

Machine Learning History and Concepts

Artificial Intelligence (AI)

History of AI/ML (Timeline)

Reasons for 2010s Deep Learning Breakthroughs

  1. Plenty of good quality (labeled) data (text, visual, audio, user activity, knowledge graphs, genomics, medical imaging)
  2. Advanced computer architecture that fits Deep Learning (e.g., GPUs for faster, more accurate results)
  3. Powerful, well-engineered machine learning libraries (easy to learn, use, extend)

Recent Advances (2023-Now) on Generative AI

Machine Learning Basics

Goal

Build computer systems that learn and adapt from “experience” (data examples)

Traditional Programming vs. Machine Learning

Common Notations

Supervised Learning

Find a function f() to map input space X to output space Y such that the difference between true y and f(x) is small

Example: Linear Binary Classifier

f(x,w,b) = sign(wᵀx + b)

Training

Learning parameters w, b by minimizing a loss/cost function L() on the training set (available x and y pairs)

Testing

Evaluating performance on “future” (unseen) points by comparing true y to predicted f(x). Key: testing examples are not in the training set

Basic Concepts

Two Modes of ML

  1. Training: Input-output pairs (X, Y) are fed into a model to learn f()
  2. Testing/Production: Unseen input X’ is fed into the learned model f() to produce f(X’) (predicted output)

Three Aimed Features of ML Models

  1. Robustness
  2. Computation
  3. Prediction

General Lessons for Excellence

  1. Master Algorithm by Pedro Domingos (explores different “tribes” of ML: Symbolists, Connectionists, Evolutionists, Bayesians, Analogizers)
  2. Homo Deus: A Brief History of Tomorrow by Yuval Noah Harari

Quiz: UVA CS 4774: Machine Learning - Introduction

Instructions: Answer each question in 2-3 sentences.

  1. What is the main distinction between “Traditional Programming” and the “Machine Learning (training phase)” as illustrated in the lecture?
  2. Name three specific reasons cited in the lecture for the significant breakthroughs in Deep Learning during the 2010s.
  3. Define “Generative AI” and explain how it differs from “discriminative AI” as presented in the source material.
  4. In the context of supervised learning, what is the purpose of the “training phase” for a linear binary classifier, and what is being minimized?
  5. What is the concept of “generalization” in machine learning, and why is it crucial for model performance?

Quiz Answer Key

  1. Machine Learning Goal
    The primary goal of machine learning is to build computer systems that can learn and adapt from their “experience.” In this context, “experience” refers to the available data examples, also known as instances or samples, which are described with various properties.

  2. Traditional vs ML Programming
    Traditional Programming involves a computer executing a pre-defined program on data to produce an output. In contrast, the Machine Learning training phase takes input data (X) and corresponding outputs (Y) to learn or create the program/model f().

  3. Deep Learning Breakthroughs
    The significant breakthroughs in Deep Learning during the 2010s were primarily due to the availability of plenty of good quality labeled data, advanced computer architectures suitable for Deep Learning (like GPUs), and powerful, well-engineered machine learning libraries.

  4. Generative AI
    Generative AI refers to AI systems that are capable of creating new content, such as text, images, or code, powered by foundation models. This differs from discriminative AI, which primarily focuses on tasks like classification rather than content generation.

  5. Training Phase Purpose
    In supervised learning, the training phase for a linear binary classifier aims to learn the parameters w and b. This is achieved by minimizing a loss or cost function L(), which quantifies the difference between the true output y and the model’s prediction f(x) on the available training examples.

  6. Generalization
    Generalization in machine learning is the ability of a model to apply the knowledge learned from past, observed data to effectively “explain,” “predict,” or “model” new, unseen data examples. It is crucial because it indicates whether the model has truly learned the underlying patterns rather than just memorizing the training data.