|Focus||Future AI/Learning – Technical|
Model Compression, Pruning, Quantization, Knowledge Distillation
Learner understands how to implement techniques of model compression.
Learner has a grasp the advantages of pruning, quantization, and knowledge distillation.
Learner is familiar with a high-level framework such as TensorFlow.
Learning Events to be Completed Before
- Lecture: Model Compression – Edge Computing
Obligatory for Students
- Basic understanding of model compression concepts and techniques
- Basic understanding of how the performance of machine and deep learning models can be evaluated (e.g. accuracy, precision and recall, F score)
- Knowledge of the Python programming language
Optional for Students
- Knowledge of the TensorFlow framework
Background for Students
- Knowledge of machine learning and neural networks theory
Recommended for Teachers
- Recall knowledge of the TensorFlow framework and Python programming language
- Provide a practical view on the implementations needed to leverage model compression techniques
- Propose pop-up quizzes
Instructions for Teachers
- Give a brief overview of Tensorflow 2.x
- Use Google Colab as a working Jupyter Notebook for practical application
- Students must use the indicated time allocated for each task.
- Task 1 to Task 4 should be completed before the remaining tasks are assigned.
Topics to cover
- Introduction to tools used and how to make hands dirty in a second (10 mins)
- [Task 1 – Task 3] Training a model and then? How to apply pruning and quantization to working models and compare performances. (70 mins)
- [Task 4 – Task 6] When could be knowledge distillation useful? How to distil knowledge from teacher to student. (40 mins)
- Conclusion, questions and answers (10 mins)
The Human-Centered AI Masters programme was Co-Financed by the Connecting Europe Facility of the European Union Under Grant №CEF-TC-2020-1 Digital Skills 2020-EU-IA-0068.
The materials of this learning event are available under CC BY-NC-ND 4.0