Section outline

  • March 20th, Friday (10:30-12:30)

    Statistical language models

    • Out-of-vocabulary words
    • Limitations of N-gram model
    • Research papers

    Neural language models (NLM)

    • General architecture for NLM
    • Feedforward NLM: inference
    • Feedforward NLM: training
    • Recurrent NLM: inference

    Exercises

    • Subword tokenization: BPE algorithm

    References

    • Jurafsky and Martin, chapter 3
    • Jurafsky and Martin, section 6.5
    • Voita, NLP Course | For You (web course): Language Modeling