Curriculum

1 Introduction to Big Data and Hadoop
Learning Objective :
Understanding what is Big Data and its solution for traditional Problems. You will learn about  Hadoop and its core components and you will know how to read and write happens in HDFS. You will also know the roles and responsibilities of a Hadoop Administrator.

Topics:

• Introduction to big data
• Limitations of existing solutions
• Common Big Data domain scenarios
• Hadoop Architecture
• Hadoop Components and Ecosystem
• Data loading & Reading from HDFS
• Replication Rules
• Rack Awareness theory
• Hadoop cluster Administrator: Roles and Responsibilities.

Hands-on:
Writing and Reading the Data from hdfs, how to submit the job in Hadoop 1.0 and YARN.

2 Hadoop Cluster and its Architecture
Learning Objectives:
Understanding different Configuration files and building Hadoop Multi Node Cluster. Differences in Hadoop 1.0 and Hadoop 2.0. You will also get to know the architecture of Hadoop 1.0 and Hadoop2.0(YARN).

Topics:

• Working of HDFS and its internals
• Hadoop Server roles and their usage
• Hadoop Installation and Initial configuration
• Different Modes of Hadoop Cluster.
• Deploying Hadoop in a Pseudo-distributed mode
• Deploying a Multi-node Hadoop cluster
• Installing Hadoop Clients
• Understanding the working of HDFS and resolving simulated problems.
• Hadoop 1 and its Core Components.
• Hadoop 2 and its Core Components.

Hands-on:
Creating Pseudo and Fully Distributed Hadoop Cluster. Changing different configuration Properties while submitting the Jobs and different hdfs admin commands.

3 Hadoop Cluster Administration and Understanding Different Processing Frameworks on Hadoop
Learning Objectives:
Understanding the various properties of Namenode, Data node, and Secondary Namenode. You will also learn how to add and decommission the data node to the cluster. You will also learn Various Processing frameworks in Hadoop and its Architecture in the context of Hadoop administrator and schedulers.

Topics:

• Properties of NameNode, DataNode and Secondary Namenode
• OS Tuning for Hadoop Performance
• Understanding Secondary Namenode
• Log Files in Hadoop
• Working with Hadoop distributed cluster
• Decommissioning or commissioning of nodes
• Different Processing Frameworks
• Understanding MapReduce
• Spark and its Features
• Application Workflow in YARN
• YARN Metrics
• YARN Capacity Scheduler and Fair Scheduler
• Understanding Schedulers and enabling them.

Hands-on:
Changing the configuration files of Secondary Namenode. Add and remove the data nodes in a Distributed Cluster. And also Changes Schedulers in run time while submitting the jobs to YARN.

4 Hadoop Cluster Administration and Maintenance
Learning Objectives:
You will learn regular  Cluster Administration tasks like balancing data in the cluster, protecting data by enabling trash, attempting a manual failover, creating backup within or across clusters

Topics:

• Namenode Federation in Hadoop
• HDFS Balancer
• High Availability in Hadoop
• Enabling Trash Functionality
• Checkpointing in Hadoop
• DistCP and Disk Balancer.

Hands-on:
Works with Cluster Administration and Maintenance tasks. Runs DistCP and HDFS Balancer Commands to get even distribution of the data.

5 Backup, Recovery, And Maintenance
Learning Objectives:
You will learn how to take Backup and recovery of data in master and slaves. You will also learn about allocating Quota to the master and slaves files.

Topics:

• Key Admin commands like DFSADMIN
• Safemode
• Importing Check Point
• MetaSave command
• Data backup and recovery
• Backup vs Disaster recovery
• Namespace count quota or space quota
• Manual failover or metadata recovery.

Hands-on:
Do regular backup using MetaSave commands. You will also run commands to do data Recovery using Checkpoints.

6 Hadoop 2.0 Cluster: Planning and Management
Learning Objective:
You will understand about Cluster Planning and Managing, what are the aspects you need to think about when planning a setup of a new cluster.

Topics:

• Planning a Hadoop 2.0 cluster
• Cluster sizing
• Hardware
• Network and Software considerations
• Popular Hadoop distributions
• Workload and usage patterns
• Industry recommendations.

Hands-on:
Setting up a new Cluster and scaling Dynamically. Login to different Hadoop distributions online.

7 Hadoop Security and Cluster Monitoring
Learning Objectives:
You will get to know about the Hadoop cluster monitoring and security concepts. You will also learn how to secure a Hadoop cluster with Kerberos.

Topics:

• Monitoring Hadoop Clusters
• Authentication & Authorization
• Nagios and Ganglia
• Hadoop Security System Concepts
• Securing a Hadoop Cluster With Kerberos
• Common Misconfigurations
• Overview on Kerberos
• Checking log files to understand Hadoop clusters for troubleshooting.

Hands-on:
Monitor the cluster and also authorization of Hadoop resource by granting tickets using Kerberos.

8 Hadoop 2.0 With High Availability And Upgrading
Learning Objectives:
You will learn how to configure Hadoop2 with high availability and upgrading. You will also learn how to work with the Hadoop ecosystem.

Topics:

• Configuring Hadoop 2 with high availability
• Upgrading to Hadoop 2
• Working with Sqoop
• Understanding Oozie
• Working with Hive.
• Working with Pig.

Hands-on:
Login to the Hive and Pig shell with their respective commands. You will also schedule OOZIE Job.

9 Cloudera Manager And Cluster Setup
Learning Objectives:
You will see how to work with CDH and its administration tool Cloudera Manager. You will also learn ecosystem administration and its optimization.

Topics:

• Cloudera Manager and cluster setup
• Hive administration
• HBase architecture
• HBase setup
• Hadoop/Hive/Hbase performance optimization.
• Pig setup and working with a grunt.

Hands-on:
Install CDH and works with Cloudera Manager. Install new parcel in CDH machine.