| Title | Model Compression |
| Duration | 150 min |
| Module | C |
| Lesson Type | Practical |
| Focus | Future AI/Learning – Technical |
| Topic | Model compression |
Administrative Information
Keywords
Model Compression, Pruning, Quantization, Knowledge Distillation
Learning Goals
Expected Preparation
Learning Events to be Completed Before
- Lecture: Model Compression – Edge Computing
Obligatory for Students
- Basic understanding of model compression concepts and techniques
- Basic understanding of how the performance of machine and deep learning models can be evaluated (e.g. accuracy, precision and recall, F score)
- Knowledge of the Python programming language
Optional for Students
- Knowledge of the TensorFlow framework
Background for Students
- Knowledge of machine learning and neural networks theory
Recommended for Teachers
- Recall knowledge of the TensorFlow framework and Python programming language
- Provide a practical view on the implementations needed to leverage model compression techniques
- Propose pop-up quizzes
Instructions for Teachers
- Give a brief overview of Tensorflow 2.x
- Use Google Colab as a working Jupyter Notebook for practical application
- Students must use the indicated time allocated for each task.
- Task 1 to Task 4 should be completed before the remaining tasks are assigned.
Topics to cover
- Introduction to tools used and how to make hands dirty in a second (10 mins)
- [Task 1 – Task 3] Training a model and then? How to apply pruning and quantization to working models and compare performances. (70 mins)
- [Task 4 – Task 6] When could be knowledge distillation useful? How to distil knowledge from teacher to student. (40 mins)
- Conclusion, questions and answers (10 mins)
Acknowledgements
The Human-Centered AI Masters programme was Co-Financed by the Connecting Europe Facility of the European Union Under Grant №CEF-TC-2020-1 Digital Skills 2020-EU-IA-0068.
The materials of this learning event are available under CC BY-NC-ND 4.0