Advanced Language Modelling Methods
Teaching Assistant, University of Oxford, Oxford Internet Institute, 2024
This course is offered to MSc and DPhil students in computer science and social sciences. It covers the following topics:
- Introduction to Language Modelling
- N-gram Language Models
- Neural Language Models
- Transformer Models
- Attention Mechanism
- Self-attention Mechanism
- Positional Encoding
- Multi-head Attention
- Masked Self-attention
- Encoder-Decoder Architecture
- BERT and GPT
- Training Language Models
- Fine-tuning Language Models
- Language Model Evaluation
- Applications of Language Models