Administrative Information
| Title | Semi-supervised and Unsupervised Learning |
| Duration | 45 - 60 |
| Module | C |
| Lesson Type | Lecture |
| Focus | Technical - Future AI |
| Topic | Advances in ML models through a HC lens - A result Oriented Study |
Keywords
supervised,unsupervised,semi-supervised,self-supervised learning,
Learning Goals
- Understand the supervised and unsupervised learning methods
- Be able to distinguish between semi-supervised learning and self-supervised learning
Expected Preparation
Learning Events to be Completed Before
Obligatory for Students
- Introduction to machine learning and deep learning concepts given in previous lectures
Optional for Students
References and background for students
- Machine Learning Methods - Computerphile
- Semi-supervised Learning explained
- FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
- Wav2vec: State-of-the-art speech recognition through self-supervision
- SALnet text classifier
- From Vision to Language: Semi-supervised Learning in Action at Scale
- Self-Training for End-to-End Speech Recognition
- What Is Self-Supervised Learning?
- Yann LeCun: Dark Matter of Intelligence and Self-Supervised Learning - Lex Fridman Podcast
- Jure Žbontar - Barlow Twins: Self-Supervised Learning via Redundancy Reduction
- Self-Supervised Learning: Self-Prediction and Contrastive Learning - Tutorial, NeurIPS 2021
Recommended for Teachers
None.
Lesson materials
Instructions for Teachers
The goal of this lecture is to focus on the learning techniques that allow us to build models in the absence of labelled training data. In other words, building systems that learn more like humans. The lecture should focus on new approaches in semi-supervised and self-supervised learning techniques that reduce or remove the requirement for labelled data sets. The lecture should:
- Summarise supervised and unsupervised machine learning models and their limitations
- Explain the concepts behind semi-supervised learning and give some examples
- Explain the concepts behind self-supervised learning and give some examples
- Identify and describe suitable application areas and problem types for semi-supervised and self-supervised learning
Outline
| Duration | Description | Concepts | Activity | Material |
|---|---|---|---|---|
| 10 min | Review of supervised and unsupervised learning | Labelled data, unlabelled data, classificaiton, clustering, dimensionality reduction, limitations and problems (cost of labelling data) | Taught session and examples | Lecture materials |
| 10 min | Semi-supervised learning | Definition of semi-supervised learning (learning with limited labelled data), self-training model, pseudo-labelling, confidence levels, co-training, graph based label propagation | Taught session and examples | Lecture materials |
| 10 min | Self-supervised learning | Definition of self-supervised learning (learning without labelled data), pre-text task, down-stream task, contrastive learning | Taught session and examples | Lecture materials |
| 10 min | Use cases and application areas | Semi-supervised learning (labelling audio, web content classification, text document classification), Self-supervised learning (patch localisation, content-aware pixel predication, next sentence predication, Auto-regressive language modelling, hate-speech detection) | Taught session and examples | Lecture materials |
| 5 min | Conclusion, questions and answers | Summary | Conclusions | Lecture materials |
Acknowledgements
The Human-Centered AI Masters programme was Co-Financed by the Connecting Europe Facility of the European Union Under Grant №CEF-TC-2020-1 Digital Skills 2020-EU-IA-0068.