Administrative Information
| Title | Serving Production Models |
| Duration | 60 minutes |
| Module | B |
| Lesson Type | Lecture |
| Focus | Practical - Organisational AI |
| Topic | Serving a production model |
Keywords
containarization,
Learning Goals
- Overview of Containerisation
- Introduction to TFX Serving
- Serving models Locally and Serving on Azure Container Instances
Expected Preparation
Learning Events to be Completed Before
None.
Obligatory for Students
Optional for Students
None.
References and background for students
None.
Recommended for Teachers
None.
Lesson materials
Instructions for Teachers
This Lecture will provide an overview/foundation for Serving Tensorflow models. The lecture will provide some foundations and background (including some code snippets) that will be required for the following tutorial that will put into practice the MLOps process of Serving a model for production purposes. Specifically the lecture will cover:
- The development of a exemplar model using the Stanford Dogs dataset
- Saving the model and the format of the saved model
- Tensorflow TFX Serving overview
- The APIs TFX Serving uses (gRPC and Restful)
- Containerisation
- Docker containers overview
- Tensorflow Serving Docker Image
- Serving a model to a local IP address
- Taking a Docker image and creating and running a Docker container binding the Tensorflow model in transit
- Serving a Model using Azure Container Instances (ACI)
- Taking a Docker image and creating and running a Docker container, and binding a Tensorflow model.
- Committing the Docker Container with the model saved as a new Docker Image.
- Running the new image as a Docker Container in ACI - using the Docker CLI.
Outline
| Duration (Min) | Description |
|---|---|
| 10 | Introduction to the exemplar model used in the lecture and tutorial |
| 15 | Saving the model, and overview of TFX serving toolkit and the APIs that TFX serving uses |
| 10 | An overview of Containerisation |
| 10 | Serving the model locally |
| 15 | Serving the model on ACI |
Acknowledgements
The Human-Centered AI Masters programme was Co-Financed by the Connecting Europe Facility of the European Union Under Grant №CEF-TC-2020-1 Digital Skills 2020-EU-IA-0068.