Recent deep learning on Text

• Lecture: S3-deepNNtext
• Version: current
• Please to Read: Keras Notebook on DNN text
• Recorded Videos: M1 + M2 + M3

Nonlinear Deep Discriminative 4Unsupervised Generative

Att: the following markdown text was generated from the corresponding powerpoint lecture file automatically. Errors and misformatting, therefore, do exist (a lot!)!

In this lecture, we cover:

• What is NLP?
• Typical NLP tasks / Challenges / Pipeline
• f() on natural language
  • Before Deep NLP (Pre 2012) • (BOW / LSI / Topic Modeling LDA )
  • Word2Vec (2013-2016) • (GloVe/ FastText)
  • Recurrent NN (2014-2016) • LSTM
  • Seq2Seq
  • Attention
  • Self-Attention (2016 – now )
  • Transformer (attention only Seq2Seq)
  • BERT / RoBERTa/ XLNet/ GPT-2 / …