Get in Touch

Course Outline

Introduction:

  • Apache Spark within the Hadoop Ecosystem
  • Brief introduction to Python and Scala

Foundations (Theory):

  • Architecture Overview
  • Resilient Distributed Datasets (RDD)
  • Transformations and Actions
  • Stages, Tasks, and Dependencies

Mastering the Basics via Databricks (Hands-on Workshop):

  • Exercises using the RDD API
  • Essential action and transformation functions
  • PairRDDs
  • Join Operations
  • Caching Strategies
  • Exercises using the DataFrame API
  • SparkSQL
  • DataFrame Operations: select, filter, group, sort
  • UDFs (User Defined Functions)
  • Introduction to the DataFrame API
  • Streaming

Understanding Deployment in AWS (Hands-on Workshop):

  • Basics of AWS Glue
  • Key differences between AWS EMR and AWS Glue
  • Example jobs on both environments
  • Pros and Cons analysis

Additional Topics:

  • Introduction to Apache Airflow for orchestration

Requirements

Programming skills (preferably in Python or Scala)

Basic knowledge of SQL

 21 Hours

Testimonials (3)

Related Categories