Get in Touch

Course Outline

Introduction to Multimodal LLMs in Vertex AI

  • Overview of multimodal capabilities within Vertex AI.
  • Understanding Gemini models and their supported modalities.
  • Exploring use cases in enterprise and research settings.

Setting Up the Development Environment

  • Configuring Vertex AI for multimodal workflows.
  • Managing datasets across various modalities.
  • Hands-on lab: Environment setup and dataset preparation.

Long Context Windows and Advanced Reasoning

  • Comprehending long-context workflows.
  • Examining use cases in planning and decision-making.
  • Hands-on lab: Implementing long-context analysis.

Cross-Modal Workflow Design

  • Integrating text, audio, and image analysis.
  • Chaining multimodal steps within pipelines.
  • Hands-on lab: Designing a multimodal pipeline.

Working with Gemini API Parameters

  • Configuring multimodal inputs and outputs.
  • Optimizing inference processes and efficiency.
  • Hands-on lab: Tuning Gemini API parameters.

Advanced Applications and Integrations

  • Developing interactive multimodal agents and assistants.
  • Integrating external APIs and tools.
  • Hands-on lab: Building a multimodal application.

Evaluation and Iteration

  • Testing multimodal performance.
  • Establishing metrics for accuracy, alignment, and drift.
  • Hands-on lab: Evaluating multimodal workflows.

Summary and Next Steps

Requirements

  • Proficiency in Python programming.
  • Experience in developing machine learning models.
  • Familiarity with multimodal data types (text, audio, and image).

Audience

  • AI researchers.
  • Senior developers.
  • Machine learning scientists.
 14 Hours

Related Categories