Administrative Information
| Title | Regularization Techniques |
| Duration | 60 |
| Module | B |
| Lesson Type | Lecture |
| Focus | Technical - Deep Learning |
| Topic | Regularization Techniques |
Keywords
Regularization, Callbacks, Gridsearch,
Learning Goals
- Examine Weight initializers
- Investigate bias
- Apply dropout and noise
- Impliment callbacks
- Undertsand and implement a gridsearch
- Apply non traditional overfitting techniques
Expected Preparation
Learning Events to be Completed Before
None.
Obligatory for Students
None.
Optional for Students
None.
References and background for students
- John D Kelleher and Brain McNamee. (2018), Fundamentals of Machine Learning for Predictive Data Analytics, MIT Press.
- Michael Nielsen. (2015), Neural Networks and Deep Learning, 1. Determination press, San Francisco CA USA.
- Charu C. Aggarwal. (2018), Neural Networks and Deep Learning, 1. Springer
- Antonio Gulli,Sujit Pal. Deep Learning with Keras, Packt, [ISBN: 9781787128422].
Recommended for Teachers
None.
Lesson materials
Instructions for Teachers
This lecture will introduce students to the fundamentals of the hyperparameter tuning. We will use the Census Dataset as the examples of the use and outcomes from the regularisation techniques. The Adult Census dataset is a binary classification problem. The goal of this lecture is to introduce several forms of regularisation, starting with weight initialisers, bias, co-adaption, callbacks, a grid search for automatic hyperparameter tunning, and additional regularisation checking techniques. The goal is to identify techniques to support the development of generalisable models with limited co-adaption to learn the function and not the data. Some of these techniques also improve training time, thus can reduce the computation needed for larger models.
Outline
| Duration (Min) | Description |
|---|---|
| 10 | Weight initialisers and bias |
| 10 | Co-adaption |
| 10 | Callbacks |
| 20 | GridSearch |
| 10 | Additional checks |
Acknowledgements
The Human-Centered AI Masters programme was Co-Financed by the Connecting Europe Facility of the European Union Under Grant №CEF-TC-2020-1 Digital Skills 2020-EU-IA-0068.