Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs Training Course
Parameter-Efficient Fine-Tuning (PEFT) represents a suite of strategies designed to adapt large language models (LLMs) effectively by adjusting only a small fraction of their parameters.
This live, instructor-led training, available online or onsite, targets intermediate data scientists and AI engineers looking to fine-tune LLMs with greater cost-effectiveness and efficiency using techniques such as LoRA, Adapter Tuning, and Prefix Tuning.
Upon completing this program, participants will be equipped to:
- Grasp the theoretical foundations of parameter-efficient fine-tuning methods.
- Apply LoRA, Adapter Tuning, and Prefix Tuning using Hugging Face PEFT.
- Analyze the performance versus cost implications of PEFT techniques compared to full fine-tuning.
- Deploy and scale fine-tuned LLMs while minimizing computational and storage demands.
Course Structure
- Engaging lectures and interactive discussions.
- Extensive exercises and practical drills.
- Live laboratory sessions for hands-on implementation.
Customization Possibilities
- To tailor this training to your specific needs, please reach out to us to discuss arrangements.
Course Outline
Overview of Parameter-Efficient Fine-Tuning (PEFT)
- Drivers and constraints associated with full fine-tuning.
- Introduction to PEFT: objectives and advantages.
- Industrial applications and real-world use cases.
LoRA (Low-Rank Adaptation)
- Theoretical concepts and intuitive understanding of LoRA.
- Building LoRA implementations with Hugging Face and PyTorch.
- Practical exercise: Fine-tuning a model using LoRA.
Adapter Tuning
- Mechanisms of adapter modules.
- Integrating adapters into transformer-based architectures.
- Practical exercise: Implementing Adapter Tuning on a transformer model.
Prefix Tuning
- Leveraging soft prompts for fine-tuning processes.
- Comparing strengths and limitations against LoRA and adapters.
- Practical exercise: Applying Prefix Tuning to an LLM task.
Assessing and Contrasting PEFT Techniques
- Key metrics for judging performance and efficiency.
- Balancing training speed, memory consumption, and accuracy.
- Conducting benchmark tests and interpreting outcomes.
Rolling Out Fine-Tuned Models
- Procedures for saving and retrieving fine-tuned models.
- Key considerations for deploying PEFT-based models.
- Integration into broader applications and pipelines.
Optimal Practices and Advanced Extensions
- Synergizing PEFT with quantization and distillation.
- Application in resource-constrained and multilingual contexts.
- Emerging trends and current research frontiers.
Requirements
- Foundational knowledge of machine learning.
- Practical experience handling large language models (LLMs).
- Proficiency in Python and PyTorch.
Target Audience
- Data scientists.
- AI engineers.
Need help picking the right course?
southafrica@nobleprog.co.za or +27 (0)10 005 5793
Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs Training Course - Enquiry
Related Courses
Advanced LangGraph: Optimization, Debugging, and Monitoring Complex Graphs
35 HoursBuilding Coding Agents with Devstral: From Agent Design to Tooling
14 HoursGain expertise in Devstral through this live training conducted in Nigeria. The curriculum dives into architecture, agentic workflow design, and tool integration. Tailored for senior engineers, the course utilizes hands-on labs to construct coding agents that elevate productivity and security within contemporary development ecosystems.
Open-Source Model Ops: Self-Hosting, Fine-Tuning and Governance with Devstral & Mistral Models
14 HoursThis instructor-led training in Nigeria is designed for ML engineers looking to master the self-hosting and fine-tuning of Devstral and Mistral models. Learners acquire practical skills in governance, monitoring, and compliance to deploy open-source AI solutions with confidence in production settings.
LangGraph Applications in Finance
35 HoursLangGraph Foundations: Graph-Based LLM Prompting and Chaining
14 HoursLangGraph in Healthcare: Workflow Orchestration for Regulated Environments
35 HoursLangGraph for Legal Applications
35 HoursBuilding Dynamic Workflows with LangGraph and LLM Agents
14 HoursLangGraph for Marketing Automation
14 HoursLe Chat Enterprise: Private ChatOps, Integrations & Admin Controls
14 HoursThis live training in Nigeria covers Le Chat Enterprise, a private ChatOps solution for secure, governed conversational AI. Learners will deploy, configure, and integrate it with enterprise apps, enabling RBAC, SSO, and compliance-driven controls for scalable, secure AI workflows.
Cost-Effective LLM Architectures: Mistral at Scale (Performance / Cost Engineering)
14 HoursGain mastery over cost-efficient Mistral LLM architectures through this 14-hour live training in Nigeria. Discover how to deploy, optimize, and scale Mistral Medium 3 to maximize throughput while minimizing costs, leveraging batching, quantization, and efficient serving strategies for enterprise applications.
Productizing Conversational Assistants with Mistral Connectors & Integrations
14 HoursThis live session in Nigeria focuses on embedding Mistral conversational assistants into enterprise workflows. Attendees will master connecting SaaS tools, applying retrieval-augmented generation, and crafting effective UX patterns for both customer and internal chat interfaces.
Enterprise-Grade Deployments with Mistral Medium 3
14 HoursThis instructor-led course in Nigeria empowers senior AI engineers to deploy and secure Mistral Medium 3 effectively. You will learn to optimize inference costs, build multimodal applications, and implement enterprise-grade compliance measures. The program includes hands-on labs for both API and self-hosted configurations.
Mistral for Responsible AI: Privacy, Data Residency & Enterprise Controls
14 HoursThis live Mistral Responsible AI training in Nigeria is specifically tailored for compliance and security professionals. You will gain expertise in privacy-preserving techniques, data residency strategies, and enterprise controls such as RBAC and SSO, ensuring secure and compliant AI deployment.
Multimodal Applications with Mistral Models (Vision, OCR, & Document Understanding)
14 HoursGain the skills to develop multimodal applications utilizing Mistral models, with a specific focus on OCR, document intelligence, and vision-text search. This practical training covers pipeline implementation, NLP integration, and enterprise-grade architecture, available online or in Nigeria.
