2022 Spring UVA CS - Machine Learning



TuTh 2pm-3:15PM

2022 Spring UVa CS Machine Learning Lectures Organized by Given Order

---- ----

Introduction

1Basic

Algebra Review

1Basic

Section 1 - Basics Supervised & On Tabular Input Type


Scikit-learn

1Basic platform

Machine Learning in a Nutshell

1Basic

Linear Regression

2Regression Linear 1Basic

GD and SGD for LR

2Regression Optimization

LR with basis

2Regression Nonlinear ModelSelection Local

Workflow for model selection

2Regression Nonlinear ModelSelection Local

Linear Prediction with Regularization

2Regression Optimization Regularization ModelSelection

KNN and Theory

3Classification 5Theory Local ModelSelection

Bias Variance Tradeoff

5Theory Local

Lasso and Elastic Net

2Regression Optimization Regularization ModelSelection

Section 2 - Deep on 2D Grid Type (e.g. Imaging)


ProbReview + MLE

1Basic

Logistic and NN

3Classification Nonlinear Deep Linear Discriminative

NN and Deep Learning

3Classification Nonlinear Deep Discriminative

CNN

3Classification Nonlinear Deep Discriminative

PCA, Feature Selection

DimenReduct

Feature Selection

DimenReduct ModelSelection

Section 3 - Deep on 1D Sequence Type (e.g. Language Text)


Prob Review

1Basic

pyTorch + Keras

platform

Recent deep learning on Text

Nonlinear Deep Discriminative 4Unsupervised Generative

In this lecture, we cover:

  • What is NLP?
  • Typical NLP tasks / Challenges / Pipeline
  • f() on natural language
    • Before Deep NLP (Pre 2012) • (BOW / LSI / Topic Modeling LDA )
    • Word2Vec (2013-2016) • (GloVe/ FastText)
    • Recurrent NN (2014-2016) • LSTM
    • Seq2Seq
    • Attention
    • Self-Attention (2016 – now )
    • Transformer (attention only Seq2Seq)
    • BERT / RoBERTa/ XLNet/ GPT-2 / …

Huggingface Invited Lecture

platform

Generative Classification

3Classification Generative

NaiveBC on Text

3Classification Generative

Gaussian GBC

3Classification Gaussian Generative

Section 4 - More Advanced Supervsied on Tabular Type


Buffer


Quiz reviews


SVM

3Classification Linear Discriminative Regularization Optimization 5Theory

SVM, Kernel

3Classification Discriminative Regularization Optimization 5Theory

DecisionTree and Bagging

3Classification Ensemble 5Theory
  • Module1: Basic Tree4Classifier @ https://youtu.be/JKcTiyvIpp8
  • Module2: How2Learn Tree https://youtu.be/iKTxnJU0L1E
  • Module3: Bagged DT https://youtu.be/WaWTw07Luzs

RF and Boosting

3Classification Ensemble Discriminative
  • Module1: Committee of Models and Analysis https://youtu.be/NuxS9SycZG8
  • Module2: Random Forest and Analysis https://youtu.be/m52bovS_eNA
  • Module3: Stacking https://youtu.be/sRLyzg5hmuM
  • Module4: Boosting https://youtu.be/VwFTEW_NM4o

SVM, Dual

3Classification Discriminative Regularization Optimization 5Theory

convex optim with Dual

3Classification Discriminative Regularization Optimization 5Theory

More on Boosting

3Classification Ensemble Optimization 5Theory

Section 5 - Not Supervised


Clustering Hier

4Unsupervised

Hierarchical:

  • Module1: Basics of Unsupervised Clustering @ https://youtu.be/mxrcPHWCZAI
  • Module2: Hierarchical Clustering @ https://youtu.be/2K6gXLf4Em4

K-means

  • Module1: Basic K-Means Algorithm @ https://youtu.be/239r6zlZYhY
  • Module2: Theory/Complexity of K-Means @ https://youtu.be/OFKKeVhCQfA
  • Extra Module3: Gaussian Mixture Models @ https://youtu.be/pmNPXpj0eK4

Clustering Partition

4Unsupervised Generative
  • Module1: Basic K-Means Algorithm @ https://youtu.be/239r6zlZYhY
  • Module2: Theory/Complexity of K-Means @ https://youtu.be/OFKKeVhCQfA
  • Extra Module3: Gaussian Mixture Models @ https://youtu.be/pmNPXpj0eK4

Reinforcement Learning

notSupervised RL 5Theory

deep RL Gym

notSupervised

Clustering GMM

4Unsupervised Generative 5Theory

Clustering GMM

4Unsupervised Generative 5Theory

Section 6 - Wrap Up


Quick survey of recent deep learning

3Classification Nonlinear Deep Discriminative 4Unsupervised Generative

This lecture covers 10 deep learning trends that go beyond classic machine learning:

    1. Popular CNN, RNN, Transformer models are not covered much here
    1. DNN on graphs / trees / sets
    1. NTM 4program induction
    1. Deep Generative models/ DeepFake
    1. Deep reinforcement learning

  • 5 . Few-shots / Meta learning / AGI?

    1. pretraining workflow / Autoencoder / self-supervised training
    1. Generative Adversarial Networks (GAN) workflow
    1. AutoML workflow / Learning to optimize /to search architecture
    1. Validate / Evade / Test / Verify / Understand DNNs
    1. Model Compression / Efficient Net

Disclaimer: it is quite hard to make important topics of deep learning fit on a one-session schedule. We aim to make the content reasonably digestible in an introductory manner. We try to focus on a modularity view by introducing important variables to digest deep learning into chunks regarding data/ model architecture /tasks / training workflows and model characteristics. We think this teaching style provides students with context concerning those choices and helps them build a much deeper understanding.

Review

1Basic
BackTop