Section outline

  • April 2nd, Thursday (10:30-12:30)

    Lab Session I: Static word embeddings

    • Introduction to the gensim library
    • Common operations with word embeddings: lookup, similarity, NN retrieval
    • Visualizing word embeddings: dimensionality reduction with PCA
    • Intrisic evaluation of word embeddings: word similarity and word analogy benchmarks

    Large language models and pretraining

    • Pretraining and transfer learning
    • Large language models
    • Language modeling head
    • Text completion and decoder-only model
    • Casting NLP tasks as text completion

    References

    • Jurafsky and Martin, chapter 7
    • Jurafsky and Martin, chapter 8

    Resources