Administrative Information
| Title | Building computational graphs, modern architectures |
| Duration | 60 minutes |
| Module | B |
| Lesson Type | Lecture |
| Focus | Technical - Deep Learning |
| Topic | Computational Graphs |
Keywords
neural networks, computational graph, residual connection, skip connection, deep learning,
Learning Goals
- Understanding the fundamentals of Computational Graphs, residual connections, highway connections and skip connections
Expected Preparation
Learning Events to be Completed Before
Obligatory for Students
- The Functional API
- He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 770-778).
- Srivastava, R. K., Greff, K., & Schmidhuber, J. (2015). Highway networks. arXiv preprint arXiv:1505.00387.
Optional for Students
None.
References and background for students
None.
Recommended for Teachers
None.
Lesson materials
Instructions for Teachers
In this lecture one of the main goals is to show, that deep neural networks are computational graphs, that can scale well. In the lecture modern architectures, including residual, highway and skip connections, that are widely applied in neural networks are introduced.
Outline
- Computational graphs
- Building computational graphs with Keras
- Residual connections
- Highway connections
- Skip connections
| Duration (Min) | Description |
|---|---|
| 10 | Introduction to computational graphs of neural networks |
| 15 | Introduction of functional API of Keras with an example |
| 10 | Description of residual connections with example source code |
| 10 | Description of highway connections with example source code |
| 10 | Description of skip connections with example source code |
| 5 | Summary and conclusions |
Acknowledgements
Balint Gyires-Tóth (Budapest University of Technology and Economics)
The Human-Centered AI Masters programme was Co-Financed by the Connecting Europe Facility of the European Union Under Grant №CEF-TC-2020-1 Digital Skills 2020-EU-IA-0068.