Section outline

  • March 24th, Monday (16:30-18:30)

    Contextualised word embeddings

    • GPT-n decoder-only model
    • Sentence-BERT

    Large language models

    • Language modeling head
    • Text completion and decoder-only model
    • Casting NLP tasks as text completion
    • Sampling
    • Pretraining

    References

    • Jurafsky and Martin, section 9.5
    • Jurafsky and Martin, chapter 10
    • Slides from lecture