Administrative Information
| Title | Model Compression |
| Duration | 150 min |
| Module | C |
| Lesson Type | Practical |
| Focus | Technical - Future AI |
| Topic | Advances in ML models through a HC lens - A result Oriented Study |
Keywords
model compression, pruning, quantization, knowledge distillation,
Learning Goals
- Understand how to implement techniques of model compression
- Grasp the advantages of pruning, quantization and knowledge distillation
- Becoming familiar with a high-level framework like TensorFlow
Expected Preparation
Learning Events to be Completed Before
Obligatory for Students
- Basic understanding of model compression concepts and techniques
- Basic understanding of how the performance of machine and deep learning models can be evaluated (e.g. accuracy, precision and recall, F score)
- Knowledge of the Python programming language
Optional for Students
- Knowledge of the TensorFlow framework
References and background for students
- Knowledge of machine learning and neural networks theory
Recommended for Teachers
- Recall knowledge of the TensorFlow framework and Python programming language
- Provide a practical view on the implementations needed to leverage model compression techniques
- Propose pop-up quizzes
Lesson materials
Instructions for Teachers
- Give a brief overview of Tensorflow 2.x
- Use Google Colab as working Jupyter Notebook for practical application
- Students must use the indicated time allocated for each task.
- Task 1 to Task 4 should be completed before the remaining tasks are assigned.
Outline of the lesson
| Duration | Description | Concepts | Activity | Material |
|---|---|---|---|---|
| 0-10 min | Introduction to tools used and how to make hands dirty in a second | Tools introduction | Introduction to main tools | |
| 10-80 min | [Task 1 - Task 3] Training a model and then? How to apply pruning and quantization to working models and compare performances | Pruning & Quantization | Practical session and working examples | Colab Notebook |
| 80-140 min | [Task 4 - Task 6] When could be knowledge distillation useful? How to distill knowledge from teacher to student | Knowledge Distillation | Practical session and working examples | Colab Notebook |
| 140-150 min | Conclusion, questions and answers | Summary | Conclusions |
Acknowledgements
- Rosario Catelli [CNR]
- Google Colab materials on Pruning
- Google Colab materials on Distillation
The Human-Centered AI Masters programme was Co-Financed by the Connecting Europe Facility of the European Union Under Grant №CEF-TC-2020-1 Digital Skills 2020-EU-IA-0068.