Section outline

  • March 12th, Wednesday (16:30-18:30)

    Language models

    • N-gram probabilities and bias-variance trade-off
    • Practical issues
    • Evaluation: perplexity measure
    • Sampling sentences
    • Smoothing: Laplace and add-k smoothing
    • Stupid backoff and linear interpolation
    • Out-of-vocabulary words
    • Limitations of N-gram model
    • Research papers

    Exercises

    • Subword tokenization: BPE algorithm

    References

    • Jurafsky and Martin, chapter 3