Real-Time Ingestion & Processing Using Kafka & Spark

The Real-Time Ingestion & Processing Using Kafka & Spark training course focuses on Data Ingestion and Processing using Kafka and Spark Streaming. The course covers Kafka Fundamentals, Architecture, API, Kafka Connect, Kafka Streams, Spark Micro-batch processing and Structured Streaming Processing. Students will gain an understanding of Kafka fundamentals and internals, Zookeeper, integrations and the API. The course then moves onto Spark Streaming and Kafka Streams. The course utilizes a mixture of lecture and hands-on exercises to reinforce learning. Hands-on exercises related to Kafka APIs will be in Java/Scala and Scala language will be used for Spark related exercises.

Course Summary

Purpose: 
This course will provide the knowledge and skills needed to utilize Data Ingestion and Processing using Kafka and Spark Streaming.
Audience: 
Developers and developer teams needing to learn Real-Time Ingestion and Processing using Kafka and Spark.
Skill Level: 
Learning Style: 

Workshops are instructor-led lab-intensives focused on the practical application of technologies through the facilitation of a project-related lab. Workshops are just the opposite of Seminars. They deliver the highest level of knowledge transfer of any format. Think wide (breadth) and deep (depth).

Workshop help
Duration: 
2 Days
Productivity Objectives: 
  • Use Kafka Fundamentals & Architecture, Producer & Consumer API, Kafka Connect, Kafka Streams
  • Perform Spark Micro-batch processing
  • Perform Structured Streaming Processing

What You'll Learn

In the Real-Time Ingestion & Processing Using Kafka & Spark training course you’ll learn:

  • Introduction to Apache Kafka
    • Kafka Overview
    • Salient Features of Kafka
    • Kafka Use cases
    • Comparing Kafka with other Key tools
  • Kafka Fundamentals & Internals
    • Logical Architecture of Kafka
    • Physical Architecture of Kafka
      • Partitions
      • Topics
      • Replicas
      • Producers & Consumers
      • Brokers
    • Roles and Responsibilities of various components
    • Replication mechanism
    • Message Delivery Semantic
    • Key Terminologies
    • Key configurations settings of Brokers, Producers, Consumers etc.
  • Zookeeper
    • Role of Zookeeper
    • Zookeeper Basic Operations
    • Apache Kafka – Zookeeper Role
  • Kafka Integrations
    • End to End Data Pipeline using Kafka
    • Kafka Connect
    • Explain Spark Components
    • Integrate Kafka with Spark
  • Kafka API
    • Overview
    • Producer API
      • Sync Producers
      • Async Producers
      • Message Acknowledgement
      • Batching Messages
      • Keyed and Non-Keyed Messages
      • Compression
      • Batching
    • Consumer API
  • Spark Streaming
    • Analyzing streaming data using Spark
    • Stateless Streaming
    • Stateful Streaming
    • Structured Streaming
    • Kafka Streams
    • Overview
    • What is Kafka Streams
    • Why Kafka Streams

Get Custom Training Quote

We'll work with you to design a custom Real-Time Ingestion & Processing Using Kafka & Spark training program that meets your specific needs. A 100% guaranteed plan that works for you, your team, and your budget.

Learn More

Chat with one of our Program Managers from our Boulder, Colorado office to discuss various training options.

DevelopIntelligence has been in the technical/software development learning and training industry for nearly 20 years. We’ve provided learning solutions to more than 48,000 engineers, across 220 organizations worldwide.

About Develop Intelligence
Di Clients
Need help finding the right learning solution?   Call us: 877-629-5631