Introduction to Administering Hadoop Clusters Course

Course Summary

The Introduction to Administering Hadoop Clusters training course includes one or more labs to reinforce and extend the topics under discussion, including a review of example configurations and run-time reports. The Hadoop administration course focuses on the key aspects of installing and maintaining a Hadoop cluster in various forms.

The course begins by teaching students how to operate the Hadoop Distributed File System (HDFS) file system and MapReduce I/O framework as complementary technologies. Next the course dives into configuring and monitoring processes to manage storage and job tasks, adding network topology awareness to a cluster and configuring a federated or highly-available storage system. The course concludes with supplementing clusters with enhanced storage features and client tools.

This course can be extended to five days if additional coverage in the following areas as needed:
a) writing MapReduce jobs
b) managing job properties
c) overviews on ecosystem projects such as Hive, Pig, Impala, and HBase
d) lab integration with an existing in-house cluster

Purpose

Learn how to set, configure, and administer Hadoop.

Audience

System adminstrators, developers, and DevOps engineers creating Big Data solutions using Hadoop.

Role

Software Developer - System Administrator

Skill Level

Intermediate

Style

Hack-a-thon - Learning Spikes - Workshops

Duration

4 Days

Related Technologies

Hadoop

Java

Apache

Download

Productivity Objectives:

Describe the HDFS file system and MapReduce I/O frameworks.
Configure and monitor storage management processes and tasks.
Add network topology awareness to a cluster.
Configure a highly-available storage system.
Supplement clusters with enhanced storage features and client tools.

Get Quote

Get your team upskilled or reskilled today. Chat with one of our experts to create a custom training proposal. Fully customized at no additional cost.

If you are not completely satisfied with your training class, we'll give you your money back.

What You'll Learn

In the Introduction to Administering Hadoop Clusters training course, you'll learn:

Hadoop Concepts
- Operating on Large Data Sets
- Parallelizing to Improve Performance
- Using Large Block Sizes
- Distributing & Replicating Data
- Assigning Code to Data
- Compensating for Node Failures and Recoveries
- Adding Nodes for Better Performance
- Using Virtualization for Rapid Deployment
Installing a Hadoop Cluster
- Understanding the NameNode
- Understanding the Secondary NameNode
- Understanding the Data Node
- Understanding the JobTracker
- Understanding the TaskTracker
Understanding the MapReduce Flow
- Mapping Data
- Shuffling and Sorting
- Reducing Data
- Using the Write Once, Read Many Approach
Reviewing Job Performance
- Interpreting Console Output
- Navigating the JobTracker UI
- Using TaskTracker Logs
Configuring Nodes
- Understanding Hadoop Property Management
- Managing Core Properties
- Managing HDFS Properties
- Managing MapReduce Properties
- Managing Worker Properties
- Restricting Job Property Changes
Supporting Federated & HA File Systems
- Restoring NameNode Services
- Protecting NameNode Metadata
- Using Federated NameNodes
- Understanding the NameNode HA Model
  - Configure a Federated HDFS system
  - Create defensive copies of NameNode metadata
  - Alt: Configure an HA NameNode for manual failover
Controlling Jobs and Resources
- Scheduling Jobs
- Understanding the FairScheduler
- Orchestrating Workflows
Importing Legacy and Continuous Data
- Using Sqoop
- Using Flume
- Understanding Hive, Impala, and HBase
Maintaining HDFS
- Checking block integrity
- Balancing data across nodes
- Using HDFS Safe Mode
- Addressing Other HDFS Systems
- Restricting Node Additions
Installing Ecosystem Packages
- Installing Pig
- Installing Hive
- Reviewing HBase Requirements
Improving Hadoop Security
- Reviewing Authentication & Authorization
- Understanding Hadoop’s Authorization Model
- Understanding Kerberos Architecture
- Reviewing Kerberos Implementation Options