Section outline

  • April 10th, Friday (10:30-12:30)

    Lab Session II: Hugging Face Transformer

    • General overview of the library
    • Importing and using BERT
    • Using Gemma and chat templates

    Large language models and pretraining

    • Training corpora
    • Scaling laws for LLMs
    • Overview of LLMs
    • Multi-lingual LLMs
    • Training of MLLMs
    • Evaluation of LLM
    • Emergent abilities
    • Mixture of Experts

    References

    • Jurafsky and Martin, chapter 7
    • Jurafsky and Martin, chapter 8
    • Slides from lecture

    Resources