Section outline

  • March 27th, Wednesday (10:30-12:30)

    Large language models

    • Multi-lingual large language models (continued)
    • Sentence BERT
    • Miscellanea: emergent abilities, hallucinations, mixture of experts
    • Research papers

    Fine-tuning

    • Adaptation: feature extraction vs. fine-tuning; catastrophic forgetting
    • Adapters
    • LoRA
    • Transfer learning

    Exercises

    • Positive pointwise mutual information (PPMI)

    References

    • Jurafsky and Martin, chapter 11
    • Voita, NLP Course | For You (web course): Transfer Learning
    • Slides from lecture