Skip to content

Contact sales

By filling out this form and clicking submit, you acknowledge our privacy policy.

Real-Time Ingestion & Processing Using Kafka & Spark

Course Summary

The Real-Time Ingestion & Processing Using Kafka & Spark training course focuses on Data Ingestion and Processing using Kafka and Spark Streaming. The course covers Kafka Fundamentals, Architecture, API, Kafka Connect, Kafka Streams, Spark Micro-batch processing, and Structured Streaming Processing.

The course begins with students gaining an understanding of Kafka fundamentals and internals. The course then covers Zookeeper, integrations, and the API. The course concludes with lessons on Spark Streaming and Kafka Streams.

The course utilizes a mixture of lectures and hands-on exercises to reinforce learning. Hands-on exercises related to Kafka APIs will be in Java/Scala and Scala language will be used for Spark related exercises.

Purpose
 Learn the knowledge and skills needed to utilize Data Ingestion and Processing using Kafka and Spark Streaming.
Audience
 Developers and developer teams needing to learn Real-Time Ingestion and Processing using Kafka and Spark.
Role
Software Developer - System Administrator
Skill Level
Intermediate
Style
Workshops
Duration
2 Days
Related Technologies
Apache Spark | Scala | Java | Apache Kafka

 

Productivity Objectives
  • Use Kafka Fundamentals & Architecture, Producer & Consumer API, Kafka Connect, Kafka Streams
  • Perform Spark Micro-batch processing
  • Perform Structured Streaming processing

What You'll Learn:

In the Real-Time Ingestion & Processing Using Kafka & Spark training course, you'll learn:
  • Introduction to Apache Kafka
    • Kafka Overview
    • Salient Features of Kafka
    • Kafka Use cases
    • Comparing Kafka with other Key tools
  • Kafka Fundamentals & Internals
    • Logical Architecture of Kafka
    • Physical Architecture of Kafka
      • Partitions
      • Topics
      • Replicas
      • Producers & Consumers
      • Brokers
    • Roles and Responsibilities of various components
    • Replication mechanism
    • Message Delivery Semantic
    • Key Terminologies
    • Key configuration settings of Brokers, Producers, Consumers etc.
  • Zookeeper
    • Role of Zookeeper
    • Zookeeper Basic Operations
    • Apache Kafka - Zookeeper Role
  • Kafka Integrations
    • End to End Data Pipeline using Kafka
    • Kafka Connect
    • Explain Spark Components
    • Integrate Kafka with Spark
  • Kafka API
    • Overview
    • Producer API
      • Sync Producers
      • Async Producers
      • Message Acknowledgement
      • Batching Messages
      • Keyed and Non-Keyed Messages
      • Compression
      • Batching
    • Consumer API
  • Spark Streaming
    • Analyzing streaming data using Spark
    • Stateless Streaming
    • Stateful Streaming
    • Structured Streaming
    • Kafka Streams
    • Overview
    • What is Kafka Streams
    • Why Kafka Streams
real world content

Real-World Content

Project-focused demos and labs using your tool stack and environment, not some canned "training room" lab.
expert practitioners

Expert Practitioners

Industry experts that bring their battle scars into the classroom.
experiential learning

Experiential Learning

More coding than lecture, coupled with architectural and design discussions.
tailored outlines

Tailored Outlines

One-size-fits-all doesn't apply to training teams. That's where we come in!
“I appreciated the instructor's technique of writing live code examples rather than using fixed slide decks to present the material.”

VMware

Dive in and learn more

When transforming your workforce, it's important to have expert advice and tailored solutions. We can help. Tell us your unique needs and we'll explore ways to address them.

Let's chat

By filling out this form and clicking submit, you acknowledge our privacy policy.