AI Workloads on Kubernetes: Deploying Machine Learning Models at Scale Training Course
Kubernetes stands as a robust, scalable platform designed for the deployment, serving, and management of machine learning models in production settings.
This instructor-led live training, available both online and onsite, is tailored for intermediate-level professionals looking to reliably operationalize ML models on Kubernetes.
By the end of this training, participants will be equipped with the skills to:
- Containerize and prepare ML models for efficient deployment.
- Serve models leveraging modern inference frameworks.
- Optimize workloads through autoscaling, GPU configuration, and resource tuning.
- Implement model rollout strategies, including A/B testing and canary deployments.
Course Format
- A blended approach combining lectures, architectural analysis, and guided discussions.
- Extensive practical exercises based on real-world deployment scenarios.
- Hands-on implementation within a live Kubernetes environment.
Course Customization Options
- If you require this course to be tailored to your specific environment or toolchain, please reach out to discuss customization possibilities.
Course Outline
Preparing Machine Learning Models for Deployment
- Packaging models using Docker
- Exporting models from TensorFlow and PyTorch
- Considerations for versioning and storage
Serving Models on Kubernetes
- Overview of inference servers
- Deploying TensorFlow Serving and TorchServe
- Configuring model endpoints
Optimizing Inference Performance
- Batching strategies
- Managing concurrent requests
- Tuning for latency and throughput
Autoscaling ML Workloads
- Horizontal Pod Autoscaler (HPA)
- Vertical Pod Autoscaler (VPA)
- Kubernetes Event-Driven Autoscaling (KEDA)
GPU Provisioning and Resource Management
- Configuring GPU nodes
- Overview of the NVIDIA device plugin
- Setting resource requests and limits for ML workloads
Model Rollout and Release Strategies
- Blue/green deployments
- Canary rollout patterns
- A/B testing for model evaluation
Monitoring and Observability for Production ML
- Key metrics for inference workloads
- Best practices for logging and tracing
- Setting up dashboards and alerts
Security and Reliability Considerations
- Securing model endpoints
- Implementing network policies and access control
- Ensuring high availability
Summary and Next Steps
Requirements
- A solid understanding of containerized application workflows
- Experience working with Python-based machine learning models
- Familiarity with Kubernetes fundamentals
Target Audience
- ML engineers
- DevOps engineers
- Platform engineering teams
Need help picking the right course?
southafrica@nobleprog.co.za or +27 (0)10 005 5793
AI Workloads on Kubernetes: Deploying Machine Learning Models at Scale Training Course - Enquiry
Testimonials (1)
I like the Kubernetes part of the training as I had some tasks with K8s pods and ports and it was nice to get a better understanding now. Thank you!
Laura Miron - Betfair
Course - Docker and Kubernetes
Related Courses
Advanced Platform Engineering: Scaling with Microservices and Kubernetes
35 HoursThis instructor-led, live training in Nigeria (online or onsite) is aimed at advanced-level platform engineers and DevOps professionals who wish to master scaling applications using microservices and Kubernetes.
By the end of this training, participants will be able to:
- Design and implement scalable microservices architectures.
- Deploy and manage applications on Kubernetes clusters.
- Utilize Helm charts for efficient service deployment.
- Monitor and maintain the health of microservices in production.
- Apply best practices for security and compliance in a Kubernetes environment.
AI-Driven Kubernetes Operations: Autoscaling, Scheduling & Resource Optimization
21 HoursCertified Kubernetes Administrator (CKA) - exam preparation
21 HoursCertified Kubernetes Application Developer (CKAD) - exam preparation
21 HoursCertified Kubernetes Security Specialist (CKS)
21 HoursThis instructor-led live training, offered in Nigeria (either online or onsite), targets Kubernetes practitioners aiming to prepare for the CKS exam.
By the end of the program, participants will understand how to secure Kubernetes environments and container-based applications throughout the application life cycle, covering build, deployment, and runtime stages.
Cloud‑Native DevOps Mastery: Designing, Deploying & Operating Scalable Kubernetes Microservices
49 HoursThis 7-day programme provides a comprehensive, hands-on journey into designing, deploying, and operating cloud-native applications using modern DevOps practices.
Participants will explore how to design scalable microservices architectures, optimise container environments, and manage production workloads using Kubernetes. The course covers advanced deployment strategies, GitOps-based automation, and observability practices to ensure system reliability and performance.
A strong focus is placed on real-world operational challenges, including incident response, failure simulation, and root cause analysis. The programme concludes with the use of AI-powered tools to support troubleshooting and accelerate operational decision-making.
By the end of the training, participants will have a clear understanding of how to build, deploy, monitor, and maintain resilient distributed systems in a Kubernetes-based environment.
Introduction to Containers, Kubernetes & OpenShift
35 HoursLearn the fundamentals of containers, Kubernetes and OpenShift in a practical, hands-on training designed for developers, DevOps engineers and IT professionals. Participants will learn how to build containerized applications, deploy workloads, manage Kubernetes resources and use OpenShift to streamline modern application delivery in cloud and hybrid environments.
Container Platforms Administration with Docker, Kubernetes and OpenShift
35 HoursDiscover how to build, deploy, and administer containerized environments using Docker, Kubernetes, and OpenShift. This five-day, hands-on training covers container images, Kubernetes workloads, cluster networking, storage, security, monitoring, and practical OpenShift administration. Participants will acquire the necessary skills to operate modern container platforms and troubleshoot applications across both development and production settings.
Docker and Kubernetes: Building and Scaling a Containerized Application
21 HoursIn this instructor-led, live training in Nigeria (onsite or remote), participants will learn how to create and manage Docker containers, then deploy a sample application inside a container. Participants will also learn how to automate, scale, and manage their containerized applications within a Kubernetes cluster. Finally, the training goes on to more advanced topics, walking participants through the process of securing, scaling and monitoring a Kubernetes cluster.
By the end of this training, participants will be able to:
- Set up and run a Docker container.
- Deploy a containerized server and web application.
- Build and manage Docker images.
- Set up a Docker and Kubernetes cluster.
- Use Kubernetes to deploy and manage a clustered web application.
- Secure, scale and monitor a Kubernetes cluster.
Docker and Kubernetes
21 HoursTraining Objectives: Acquire theoretical and practical skills in Docker and Kubernetes.
Docker and Kubernetes advanced
35 HoursThe "Advanced Docker and Kubernetes" course is a comprehensive training covering the Kubernetes platform and its ecosystem. It caters to participants with diverse experience levels, ensuring a full range of both basic and advanced Kubernetes functionalities. Key concepts covered include Containers, Labels, Controllers, Services, Secrets, Persistent Volumes, Namespaces, Quotas, Container Networking Model, Service Discovery, Scaling, Load Balancing, Cluster Management, Kubernetes Installation, Cluster Security, Access Control, Controller High Availability, Monitoring and Logging, Application Auto-composition, Advanced Scheduling, Microservices-based Applications, Application Design Patterns, and Deploying Applications on a Kubernetes Cluster.
Edge AI on Kubernetes: Deploying Intelligent Applications at the Edge
21 HoursDeploying Kubernetes Applications with Helm
7 HoursThis instructor-led, live training in Nigeria (online or onsite) is designed for engineers who want to leverage Helm to simplify the installation and management of Kubernetes applications.
Upon completion of this training, participants will be equipped to:
- Install and configure Helm.
- Build reproducible Kubernetes application packages.
- Distribute applications as Helm charts.
- Execute third-party applications packaged as Helm charts.
- Manage Helm package releases.
Introduction to Minikube and Kubernetes
21 HoursThis instructor-led, live training in Nigeria (online or onsite) is aimed at beginner-level to intermediate-level software developers and DevOps professionals who wish to learn how to set up and manage a local Kubernetes environment using Minikube.
By the end of this training, participants will be able to:
- Install and configure Minikube on their local machine.
- Understand the basic concepts and architecture of Kubernetes.
- Deploy and manage containers using kubectl and the Minikube dashboard.
- Set up persistent storage and networking solutions for Kubernetes.
- Utilize Minikube for developing, testing, and debugging applications.
Minikube for Developers
14 HoursThis instructor-led, live training in Nigeria (online or onsite) is designed for intermediate-level developers and DevOps engineers who want to incorporate Minikube into their development workflow.
Upon completion of this training, participants will be able to:
- Set up and manage a local Kubernetes environment using Minikube.
- Gain insight into deploying, managing, and debugging applications on Minikube.
- Integrate Minikube into their continuous integration and deployment pipelines.
- Enhance their development process by leveraging Minikube's advanced features.
- Apply best practices for local Kubernetes development.