Administrative Information
| Title | MLOPs Life Cycle |
| Duration | 60 |
| Module | B |
| Lesson Type | Practical |
| Focus | Practical - Organisational AI |
| Topic | End-to-end overview of the MLOPs lifecycle |
Keywords
MLOPs,Organizational AI,Ethical,Design,
Learning Goals
- Understanding MLOPs end-to-end
- Understanding Data Ingestion
- Getting familiar with Algorithm Development
- Getting familiar with Model Development and Deployment
- Understanding Continuous Monitoring
Expected Preparation
Learning Events to be Completed Before
Obligatory for Students
Optional for Students
- Data Preparation and Management: Before diving into MLOps, it's beneficial to understand the initial phases of the machine learning process, especially data collection, cleaning, and preprocessing
- Model Training and Validation: A grasp of how models are trained, validated, and evaluated will provide a solid foundation for understanding the operational aspects of ML.
- Hyperparameter Tuning: While not always covered in depth in MLOps courses, understanding hyperparameter tuning can be beneficial as it's a crucial step in model optimization.
- MLOps Tools and Platforms: Familiarity with tools like Kubeflow, Azure ML, and others can give students a head start.
- Documentation Practices in ML: Proper documentation is essential in MLOps for reproducibility and collaboration. Understanding best practices in ML documentation can be advantageous.
- CRISP-DM, CRISP-ML, ML Canvas: These are methodologies and frameworks for ML project management. Having a basic understanding can be beneficial for the operational side of ML projects.
References and background for students
- Students are advised to familiarize with the following tools and techniques
- JupyterNotebooks
- CUE
- Docker
- Papermill
- Streamlit
- Shell Scripting
Recommended for Teachers
- JupyterNotebooks
- CUE
- Docker
- Papermill
- Streamlit
- Shell Scripting
Lesson materials
- Please refer to full package here RADIX-Practical-Package
Instructions for Teachers
Introduction
Before teaching the practical MLOps course, it's essential for teachers to have a deep understanding of the tools and technologies mentioned in the course outline. The course is structured around a 3-part demo, each highlighting a different approach to MLOps. Here are the steps to familiarize yourself with these tools:
1. The Shell (Manual) Way
Objective: Understand the basics of setting up a Python environment and running a Jupyter notebook.
Steps:
Python Environment: Install Python on your system. Learn how to create a virtual environment using venv or conda. Practice activating and deactivating the environment.
Package Installation: Understand the structure and purpose of a requirements.txt file. Practice installing packages using pip install -r requirements.txt.
Jupyter Notebook: Install Jupyter Notebook. Learn the basics of starting a Jupyter server. Practice creating, running, and saving notebooks.
Streamlit Server: Understand the purpose of Streamlit and how it can be used to create web applications. Practice using artifacts generated from a Jupyter notebook in a Streamlit application.
Considerations: Understand the benefits of this approach, such as building foundational knowledge and initial prototyping. Be aware of its limitations, like the potential messiness, challenges in knowledge transfer, and difficulties in replicating setups.
2. The Docker + Make (Inheritance) Way
Objective: Grasp the concepts of containerization and automation using Docker and Make.
Steps:
Docker: Install Docker on your system. Understand the structure and purpose of a Dockerfile. Practice building Docker images and running containers. Familiarize yourself with common Docker commands.
Makefile:Understand the purpose of a Makefile in automating tasks. Learn the basic syntax of a Makefile. Practice writing and executing simple Make commands.
Considerations: Understand the benefits of this approach, such as reproducibility. Recognize its limitations, like the need for replication across projects, context switching, and its focus on images over artifacts.
3. The Radix (Compositional) Way
Objective: Dive into advanced MLOps practices using compositional workflows.
Steps:
Papermill: Understand the purpose of Papermill in parameterizing and executing Jupyter notebooks. Practice creating and running notebooks with Papermill. Explore available packages or consider creating a simple one.
Streamlit (Advanced): Dive deeper into advanced Streamlit functionalities. Explore available packages or consider creating a simple one.
Workflow Creation: Understand the concept of compositional workflows in MLOps. Practice creating workflows that utilize the Papermill and Streamlit packages.
Considerations: Understand the benefits of this approach, such as reproducibility, easy parameterization, validation, and composability. Be aware of its limitations, like being on the bleeding edge and potential gaps in documentation.
Conclusion
By following these steps, teachers will be well-equipped to deliver the practical MLOps course effectively. It's crucial to not only understand the technical aspects but also the underlying reasons for choosing each approach. This will enable teachers to provide students with a comprehensive understanding of MLOps practices.
Outline
| Duration (Min) | Description |
|---|---|
| 30 | The Manual Way - Shell Scripting |
| 30 | The Docker + Make (Inheritance) way |
| 30 | The Radix (Compositional) way |
| 30 | Runtime, Tooling and Performance considerations |
| 10 | Summarize and evaluate results |
Acknowledgements
Tarry Singh. (Real AI B.V., Assen, The Netherlands) https://realai.eu
The Human-Centered AI Masters programme was Co-Financed by the Connecting Europe Facility of the European Union Under Grant №CEF-TC-2020-1 Digital Skills 2020-EU-IA-0068.