Introduction

• SlideDeck: W1.1-deepNNtext
• Version: current
• Lead team: instructor
• Notes: deep nlp basics

BasicLLM

Readings:

Basics of ML and DL:

• URL

Basics of NLP

• URL
• Typical NLP tasks / Challenges / Pipeline
• f() on natural language
  • Before Deep NLP (Pre 2012) • (BOW / LSI / Topic Modeling LDA )
  • Word2Vec (2013-2016) • (GloVe/ FastText)
  • Recurrent NN (2014-2016) • LSTM
  • Seq2Seq
  • Attention
  • Self-Attention (2016 – now )
  • Transformer (attention only Seq2Seq)
  • BERT / RoBERTa/ XLNet/ GPT / …
• A good code walk through on transformer at URL

∂