LLM basics
- SlideDeck: S0-Intro
- Version: current
- Lead team: on llm basics
- Blog team: instructor
Required Readings:
Emergent Abilities of Large Language Models
- URL
-
“an ability to be emergent if it is not present in smaller models but is present in larger models. Thus, emergent abilities cannot be predicted simply by extrapolating the performance of smaller models.”
Language Models are Few-Shot Learners
- URL
-
“GPT-3, 175B autoregerssive LLM; show that scaling up language models greatly improves task-agnostic, few-shot performance, sometimes even reaching competitiveness with prior state-of-the-art fine-tuning approaches.”
Extra Readings:
A survey of Generative AI Applications
- https://arxiv.org/abs/2306.02781
- Generative AI has experienced remarkable growth in recent years, leading to a wide array of applications across diverse domains. In this paper, we present a comprehensive survey of more than 350 generative AI applications, providing a structured taxonomy and concise descriptions of various unimodal and even multimodal generative AIs. The survey is organized into sections, covering a wide range of unimodal generative AI applications such as text, images, video, gaming and brain information. Our survey aims to serve as a valuable resource for researchers and practitioners to navigate the rapidly expanding landscape of generative AI, facilitating a better understanding of the current state-of-the-art and fostering further innovation in the field.
Generative AI: Perspectives from Stanford HAI
- https://hai.stanford.edu/generative-ai-perspectives-stanford-hai