20Fall UVA CS - Machine Learning



TuTh 3:30PM - 4:45PM

2020 Fall UVa CS Machine Learning Lectures Organized by Given Order

---- ----

Introduction

1Basic

Algebra Review

1Basic

Scikit-learn

1Basic platform

Section 1 - Basics Supervised & On Tabular Input Type


Machine Learning in a Nutshell

1Basic

Linear Regression

2Regression Linear 1Basic

GD and SGD for LR

2Regression Optimization

LR with basis

2Regression Nonlinear ModelSelection Local

Workflow for model selection

2Regression Nonlinear ModelSelection Local

Linear Prediction with Regularization

2Regression Optimization Regularization ModelSelection

KNN and Theory

3Classification 5Theory Local ModelSelection

Bias Variance Tradeoff

5Theory Local

machine leanring in the AWS cloud

platform

Friday, September 18, 2020 at 3:00 PM - 4:00 PM.

AI/ML on Amazon Web Services (AWS)

Amazon SageMaker is a fully managed service that provides every developer and data scientist with the ability to build, train, and deploy machine learning (ML) models quickly. SageMaker removes the heavy lifting of building and setting up hardware and infrastructure from each step of the machine learning process, so you can focus on your models.

In this session, we will provide you with an overview of the AI/ML stacks on AWS, how machine learning is done on AWS using Amazon SageMaker Studio, and built-in algorithms and new features of SageMaker and SageMaker Studio.

Speaker: Jianjun Xu, PhD

Speaker Bio: Dr. Jianjun Xu was an astrophysicist and a senior software development executive in higher education before he joined AWS as a senior solutions architect working with several large US universities and colleges. He specializes in software development, big data, artificial intelligence (AI) and machine learning (ML), and high performance computing (HPC). Jianjun enjoys interacting with researchers, instructors, students and higher education administrators to discuss challenges, opportunities, and technology solutions in higher education.

Lasso and Elastic Net

2Regression Optimization Regularization ModelSelection

Section 2 - Deep on 2D Grid Type (e.g. Imaging)


ProbReview + MLE

1Basic

Logistic and NN

3Classification Nonlinear Deep Linear Discriminative

NN and Deep Learning

3Classification Nonlinear Deep Discriminative

CNN

3Classification Nonlinear Deep Discriminative

PCA, Feature Selection

DimenReduct

auto differentiation

Optimization

Section 3 - Deep ang Generative on 1D Sequence Type (e.g. Language Text)


Feature Selection

DimenReduct ModelSelection

pyTorch + Keras

platform

Prob Review

1Basic

Generative Classification

3Classification Generative

NaiveBC on Text

3Classification Generative

Recent deep learning on Text

Nonlinear Deep Discriminative 4Unsupervised Generative

In this lecture, we cover:

  • What is NLP?
  • Typical NLP tasks / Challenges / Pipeline
  • f() on natural language
    • Before Deep NLP (Pre 2012) • (BOW / LSI / Topic Modeling LDA )
    • Word2Vec (2013-2016) • (GloVe/ FastText)
    • Recurrent NN (2014-2016) • LSTM
    • Seq2Seq
    • Attention
    • Self-Attention (2016 – now )
    • Transformer (attention only Seq2Seq)
    • BERT / RoBERTa/ XLNet/ GPT-2 / …

Quick survey of recent deep learning

3Classification Nonlinear Deep Discriminative 4Unsupervised Generative

This lecture covers 10 deep learning trends that go beyond classic machine learning:

    1. Popular CNN, RNN, Transformer models are not covered much here
    1. DNN on graphs / trees / sets
    1. NTM 4program induction
    1. Deep Generative models/ DeepFake
    1. Deep reinforcement learning

  • 5 . Few-shots / Meta learning / AGI?

    1. pretraining workflow / Autoencoder / self-supervised training
    1. Generative Adversarial Networks (GAN) workflow
    1. AutoML workflow / Learning to optimize /to search architecture
    1. Validate / Evade / Test / Verify / Understand DNNs
    1. Model Compression / Efficient Net

Disclaimer: it is quite hard to make important topics of deep learning fit on a one-session schedule. We aim to make the content reasonably digestible in an introductory manner. We try to focus on a modularity view by introducing important variables to digest deep learning into chunks regarding data/ model architecture /tasks / training workflows and model characteristics. We think this teaching style provides students with context concerning those choices and helps them build a much deeper understanding.

adversarial text

platform

deep RL Gym

notSupervised

Gaussian GBC

3Classification Gaussian Generative

Learning to Generate

3Classification

probabilistic programming

Optimization Generative

Section 4 - More Advanced Supervsied on Tabular Type


SVM

3Classification Linear Discriminative Regularization Optimization 5Theory

SVM, Kernel

3Classification Discriminative Regularization Optimization 5Theory

DecisionTree and Bagging

3Classification Ensemble 5Theory
  • Module1: Basic Tree4Classifier @ https://youtu.be/JKcTiyvIpp8
  • Module2: How2Learn Tree https://youtu.be/iKTxnJU0L1E
  • Module3: Bagged DT https://youtu.be/WaWTw07Luzs

RF and Boosting

3Classification Ensemble Discriminative
  • Module1: Committee of Models and Analysis https://youtu.be/NuxS9SycZG8
  • Module2: Random Forest and Analysis https://youtu.be/m52bovS_eNA
  • Module3: Stacking https://youtu.be/sRLyzg5hmuM
  • Module4: Boosting https://youtu.be/VwFTEW_NM4o

SVM, Dual

3Classification Discriminative Regularization Optimization 5Theory

convex optim with Dual

3Classification Discriminative Regularization Optimization 5Theory

More on Boosting

3Classification Ensemble Optimization 5Theory

Section 5 - Not Supervised


Clustering Hier

4Unsupervised

  • Module1: Basics of Unsupervised Clustering @ https://youtu.be/mxrcPHWCZAI

  • Module2: Hierarchical Clustering @ https://youtu.be/2K6gXLf4Em4

Clustering Partition

4Unsupervised Generative
  • Module1: Basic K-Means Algorithm @ https://youtu.be/239r6zlZYhY
  • Module2: Theory/Complexity of K-Means @ https://youtu.be/OFKKeVhCQfA
  • Extra Module3: Gaussian Mixture Models @ https://youtu.be/pmNPXpj0eK4

Clustering GMM

4Unsupervised Generative 5Theory

Clustering GMM

4Unsupervised Generative 5Theory

self/semi-supervised

notSupervised

Section 6 - Wrap Up


Review

1Basic

Project Presentations (on Dec7 and Dec8)

1Basic
BackTop