Installing CDH5 on a Single Linux (Ubuntu) Node in Pseudo-distributed Mode on Amazon EC2

Cloudera has good article here on how to install CDH5 on a Single Linux Node in Pseudo-distributed Mode:

But I have added more information here for Ubuntu which will help you getting it up and running quickly.
I hope you know how to launch instance on Amazon EC2. We are starting with Ubuntu 14.04 and java pre-installed on this instance.
1) First find out which version of ubuntu you are running by using following command:
root@ip-172-30-0-84:/home/ubuntu# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.2 LTS
Release: 14.04
Codename: trusty

2)Download the CDH 5 “1-click Install” package according to your ubuntu version: for us, it is trusty as highlighted in step 1.
wget http://archive.cloudera.com/cdh5/one-click-install/trusty/amd64/cdh5-repository_1.0_all.deb

3)Install the package
root@ip-172-30-0-84:/home/ubuntu# sudo dpkg -i cdh5-repository_1.0_all.deb
Selecting previously unselected package cdh5-repository.
(Reading database … 70808 files and directories currently installed.)
Preparing to unpack cdh5-repository_1.0_all.deb …
Unpacking cdh5-repository (1.0) …
Setting up cdh5-repository (1.0) …
gpg: keyring `/etc/apt/secring.gpg’ created
gpg: keyring `/etc/apt/trusted.gpg.d/cloudera-cdh5.gpg’ created
gpg: /etc/apt/trustdb.gpg: trustdb created
gpg: key 02A818DD: public key “Cloudera Apt Repository” imported
gpg: Total number processed: 1
gpg: imported: 1

4)Install hadoop in pseudo-distributed mode:
4a) root@ip-172-30-0-84:/home/ubuntu# sudo apt-get update
Ign http://us-east-1.ec2.archive.ubuntu.com trusty InRelease
Ign http://us-east-1.ec2.archive.ubuntu.com trusty-updates InRelease
Hit http://us-east-1.ec2.archive.ubuntu.com trusty Release.gpg
Get:1 http://archive.cloudera.com trusty-cdh5 InRelease [1,930 B]
Get:2 http://us-east-1.ec2.archive.ubuntu.com trusty-updates Release.gpg [933 B]
Ign http://security.ubuntu.com trusty-security InRelease
Ign http://download.draios.com stable-amd64/ InRelease
Ign http://ppa.launchpad.net trusty InRelease
Ign http://ppa.launchpad.net trusty InRelease
Hit http://us-east-1.ec2.archive.ubuntu.com trusty Release
Get:3 http://us-east-1.ec2.archive.ubuntu.com trusty-updates Release [63.5 kB]
Get:4 http://download.draios.com stable-amd64/ Release.gpg [490 B]
Get:5 http://security.ubuntu.com trusty-security Release.gpg [933 B]
Get:6 http://download.draios.com stable-amd64/ Release [753 B]
Get:7 http://security.ubuntu.com trusty-security Release [63.5 kB]
Hit http://ppa.launchpad.net trusty Release.gpg
Get:8 http://archive.cloudera.com trusty-cdh5/contrib Sources [11.0 kB]
Hit http://ppa.launchpad.net trusty Release.gpg
Hit http://us-east-1.ec2.archive.ubuntu.com trusty/main Sources
Hit http://ppa.launchpad.net trusty Release
Hit http://us-east-1.ec2.archive.ubuntu.com trusty/universe Sources
Hit http://us-east-1.ec2.archive.ubuntu.com trusty/main amd64 Packages
Hit http://us-east-1.ec2.archive.ubuntu.com trusty/universe amd64 Packages
Hit http://us-east-1.ec2.archive.ubuntu.com trusty/main Translation-en
Hit http://us-east-1.ec2.archive.ubuntu.com trusty/universe Translation-en
Get:9 http://us-east-1.ec2.archive.ubuntu.com trusty-updates/main Sources [229 kB]
Get:10 http://archive.cloudera.com trusty-cdh5/contrib amd64 Packages [26.6 kB]
Get:11 http://us-east-1.ec2.archive.ubuntu.com trusty-updates/universe Sources [133 kB]
Get:12 http://us-east-1.ec2.archive.ubuntu.com trusty-updates/main amd64 Packages [600 kB]
Get:13 http://us-east-1.ec2.archive.ubuntu.com trusty-updates/universe amd64 Packages [307 kB]
Get:14 http://us-east-1.ec2.archive.ubuntu.com trusty-updates/main Translation-en [289 kB]
Get:15 http://us-east-1.ec2.archive.ubuntu.com trusty-updates/universe Translation-en [163 kB]
Get:16 http://download.draios.com stable-amd64/ Packages [1,451 B]
Hit http://ppa.launchpad.net trusty Release
Ign http://us-east-1.ec2.archive.ubuntu.com trusty/main Translation-en_US
Ign http://us-east-1.ec2.archive.ubuntu.com trusty/universe Translation-en_US
Ign http://archive.cloudera.com trusty-cdh5/contrib Translation-en_US
Ign http://archive.cloudera.com trusty-cdh5/contrib Translation-en
Get:17 http://security.ubuntu.com trusty-security/main Sources [91.1 kB]
Get:18 http://security.ubuntu.com trusty-security/universe Sources [29.4 kB]
Hit http://ppa.launchpad.net trusty/main amd64 Packages
Get:19 http://security.ubuntu.com trusty-security/main amd64 Packages [326 kB]
Ign http://download.draios.com stable-amd64/ Translation-en_US
Ign http://download.draios.com stable-amd64/ Translation-en
Hit http://ppa.launchpad.net trusty/main Translation-en
Get:20 http://security.ubuntu.com trusty-security/universe amd64 Packages [113 kB]
Get:21 http://security.ubuntu.com trusty-security/main Translation-en [178 kB]
Hit http://ppa.launchpad.net trusty/main amd64 Packages
Get:22 http://security.ubuntu.com trusty-security/universe Translation-en [66.3 kB]
Hit http://ppa.launchpad.net trusty/main Translation-en
Fetched 2,695 kB in 2s (981 kB/s)
Reading package lists… Done
————————————————————————
4b)root@ip-172-30-0-84:/home/ubuntu# sudo apt-get install hadoop-conf-pseudo
Reading package lists… Done
Building dependency tree
Reading state information… Done
The following extra packages will be installed:
avro-libs bigtop-jsvc bigtop-utils hadoop hadoop-hdfs hadoop-hdfs-datanode
hadoop-hdfs-namenode hadoop-hdfs-secondarynamenode hadoop-mapreduce
hadoop-mapreduce-historyserver hadoop-yarn hadoop-yarn-nodemanager
hadoop-yarn-resourcemanager parquet parquet-format zookeeper
The following NEW packages will be installed:
avro-libs bigtop-jsvc bigtop-utils hadoop hadoop-conf-pseudo hadoop-hdfs
hadoop-hdfs-datanode hadoop-hdfs-namenode hadoop-hdfs-secondarynamenode
hadoop-mapreduce hadoop-mapreduce-historyserver hadoop-yarn
hadoop-yarn-nodemanager hadoop-yarn-resourcemanager parquet parquet-format
zookeeper
0 upgraded, 17 newly installed, 0 to remove and 18 not upgraded.
Need to get 196 MB of archives.
After this operation, 238 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://archive.cloudera.com/cdh5/ubuntu/trusty/amd64/cdh/ trusty-cdh5/contrib avro-libs all 1.7.6+cdh5.4.4+91-1.cdh5.4.4.p0.6~trusty-cdh5.4.4 [55.9 MB]
Get:2 http://archive.cloudera.com/cdh5/ubuntu/trusty/amd64/cdh/ trusty-cdh5/contrib bigtop-utils all 0.7.0+cdh5.4.4+0-1.cdh5.4.4.p0.6~trusty-cdh5.4.4 [52.2 kB]
Get:3 http://archive.cloudera.com/cdh5/ubuntu/trusty/amd64/cdh/ trusty-cdh5/contrib bigtop-jsvc amd64 0.6.0+cdh5.4.4+680-1.cdh5.4.4.p0.6~trusty-cdh5.4.4 [72.8 kB]
Get:4 http://archive.cloudera.com/cdh5/ubuntu/trusty/amd64/cdh/ trusty-cdh5/contrib zookeeper all 3.4.5+cdh5.4.4+91-1.cdh5.4.4.p0.6~trusty-cdh5.4.4 [4,186 kB]
Get:5 http://archive.cloudera.com/cdh5/ubuntu/trusty/amd64/cdh/ trusty-cdh5/contrib parquet-format all 2.1.0+cdh5.4.4+12-1.cdh5.4.4.p0.6~trusty-cdh5.4.4 [470 kB]
Get:6 http://archive.cloudera.com/cdh5/ubuntu/trusty/amd64/cdh/ trusty-cdh5/contrib hadoop-yarn all 2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4 [15.2 MB]
Get:7 http://archive.cloudera.com/cdh5/ubuntu/trusty/amd64/cdh/ trusty-cdh5/contrib hadoop-mapreduce all 2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4 [28.5 MB]
Get:8 http://archive.cloudera.com/cdh5/ubuntu/trusty/amd64/cdh/ trusty-cdh5/contrib hadoop-hdfs all 2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4 [18.5 MB]
Get:9 http://archive.cloudera.com/cdh5/ubuntu/trusty/amd64/cdh/ trusty-cdh5/contrib parquet all 1.5.0+cdh5.4.4+96-1.cdh5.4.4.p0.6~trusty-cdh5.4.4 [28.5 MB]
Get:10 http://archive.cloudera.com/cdh5/ubuntu/trusty/amd64/cdh/ trusty-cdh5/contrib hadoop all 2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4 [43.6 MB]
Get:11 http://archive.cloudera.com/cdh5/ubuntu/trusty/amd64/cdh/ trusty-cdh5/contrib hadoop-hdfs-namenode all 2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4 [121 kB]
Get:12 http://archive.cloudera.com/cdh5/ubuntu/trusty/amd64/cdh/ trusty-cdh5/contrib hadoop-hdfs-datanode all 2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4 [121 kB]
Get:13 http://archive.cloudera.com/cdh5/ubuntu/trusty/amd64/cdh/ trusty-cdh5/contrib hadoop-hdfs-secondarynamenode all 2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4 [121 kB]
Get:14 http://archive.cloudera.com/cdh5/ubuntu/trusty/amd64/cdh/ trusty-cdh5/contrib hadoop-yarn-resourcemanager all 2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4 [121 kB]
Get:15 http://archive.cloudera.com/cdh5/ubuntu/trusty/amd64/cdh/ trusty-cdh5/contrib hadoop-yarn-nodemanager all 2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4 [121 kB]
Get:16 http://archive.cloudera.com/cdh5/ubuntu/trusty/amd64/cdh/ trusty-cdh5/contrib hadoop-mapreduce-historyserver all 2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4 [121 kB]
Get:17 http://archive.cloudera.com/cdh5/ubuntu/trusty/amd64/cdh/ trusty-cdh5/contrib hadoop-conf-pseudo all 2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4 [124 kB]
Fetched 196 MB in 5s (34.0 MB/s)
Selecting previously unselected package avro-libs.
(Reading database … 70813 files and directories currently installed.)
Preparing to unpack …/avro-libs_1.7.6+cdh5.4.4+91-1.cdh5.4.4.p0.6~trusty-cdh5.4.4_all.deb …
Unpacking avro-libs (1.7.6+cdh5.4.4+91-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
Selecting previously unselected package bigtop-utils.
Preparing to unpack …/bigtop-utils_0.7.0+cdh5.4.4+0-1.cdh5.4.4.p0.6~trusty-cdh5.4.4_all.deb …
Unpacking bigtop-utils (0.7.0+cdh5.4.4+0-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
Selecting previously unselected package bigtop-jsvc.
Preparing to unpack …/bigtop-jsvc_0.6.0+cdh5.4.4+680-1.cdh5.4.4.p0.6~trusty-cdh5.4.4_amd64.deb …
Unpacking bigtop-jsvc (0.6.0+cdh5.4.4+680-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
Selecting previously unselected package zookeeper.
Preparing to unpack …/zookeeper_3.4.5+cdh5.4.4+91-1.cdh5.4.4.p0.6~trusty-cdh5.4.4_all.deb …
Unpacking zookeeper (3.4.5+cdh5.4.4+91-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
Selecting previously unselected package parquet-format.
Preparing to unpack …/parquet-format_2.1.0+cdh5.4.4+12-1.cdh5.4.4.p0.6~trusty-cdh5.4.4_all.deb …
Unpacking parquet-format (2.1.0+cdh5.4.4+12-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
Selecting previously unselected package hadoop-yarn.
Preparing to unpack …/hadoop-yarn_2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4_all.deb …
Unpacking hadoop-yarn (2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
Selecting previously unselected package hadoop-mapreduce.
Preparing to unpack …/hadoop-mapreduce_2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4_all.deb …
Unpacking hadoop-mapreduce (2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
Selecting previously unselected package hadoop-hdfs.
Preparing to unpack …/hadoop-hdfs_2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4_all.deb …
Unpacking hadoop-hdfs (2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
Selecting previously unselected package parquet.
Preparing to unpack …/parquet_1.5.0+cdh5.4.4+96-1.cdh5.4.4.p0.6~trusty-cdh5.4.4_all.deb …
Unpacking parquet (1.5.0+cdh5.4.4+96-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
Selecting previously unselected package hadoop.
Preparing to unpack …/hadoop_2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4_all.deb …
Unpacking hadoop (2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
Selecting previously unselected package hadoop-hdfs-namenode.
Preparing to unpack …/hadoop-hdfs-namenode_2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4_all.deb …
Unpacking hadoop-hdfs-namenode (2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
Selecting previously unselected package hadoop-hdfs-datanode.
Preparing to unpack …/hadoop-hdfs-datanode_2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4_all.deb …
Unpacking hadoop-hdfs-datanode (2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
Selecting previously unselected package hadoop-hdfs-secondarynamenode.
Preparing to unpack …/hadoop-hdfs-secondarynamenode_2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4_all.deb …
Unpacking hadoop-hdfs-secondarynamenode (2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
Selecting previously unselected package hadoop-yarn-resourcemanager.
Preparing to unpack …/hadoop-yarn-resourcemanager_2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4_all.deb …
Unpacking hadoop-yarn-resourcemanager (2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
Selecting previously unselected package hadoop-yarn-nodemanager.
Preparing to unpack …/hadoop-yarn-nodemanager_2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4_all.deb …
Unpacking hadoop-yarn-nodemanager (2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
Selecting previously unselected package hadoop-mapreduce-historyserver.
Preparing to unpack …/hadoop-mapreduce-historyserver_2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4_all.deb …
Unpacking hadoop-mapreduce-historyserver (2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
Selecting previously unselected package hadoop-conf-pseudo.
Preparing to unpack …/hadoop-conf-pseudo_2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4_all.deb …
Unpacking hadoop-conf-pseudo (2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
Processing triggers for man-db (2.6.7.1-1ubuntu1) …
Processing triggers for ureadahead (0.100.0-16) …
Setting up avro-libs (1.7.6+cdh5.4.4+91-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
Setting up bigtop-utils (0.7.0+cdh5.4.4+0-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
Setting up bigtop-jsvc (0.6.0+cdh5.4.4+680-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
Setting up zookeeper (3.4.5+cdh5.4.4+91-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
update-alternatives: using /etc/zookeeper/conf.dist to provide /etc/zookeeper/conf (zookeeper-conf) in auto mode
Setting up parquet-format (2.1.0+cdh5.4.4+12-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
Setting up parquet (1.5.0+cdh5.4.4+96-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
Setting up hadoop (2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
update-alternatives: using /etc/hadoop/conf.empty to provide /etc/hadoop/conf (hadoop-conf) in auto mode
Setting up hadoop-yarn (2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
Setting up hadoop-yarn-resourcemanager (2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
starting resourcemanager, logging to /var/log/hadoop-yarn/yarn-yarn-resourcemanager-ip-172-30-0-84.out
* Started Hadoop resourcemanager:
Setting up hadoop-yarn-nodemanager (2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
starting nodemanager, logging to /var/log/hadoop-yarn/yarn-yarn-nodemanager-ip-172-30-0-84.out
* Started Hadoop nodemanager:
Setting up hadoop-mapreduce (2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
Setting up hadoop-hdfs (2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
Setting up hadoop-hdfs-namenode (2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
starting namenode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-namenode-ip-172-30-0-84.out
* Failed to start Hadoop namenode. Return value: 1
invoke-rc.d: initscript hadoop-hdfs-namenode, action “start” failed.
Setting up hadoop-hdfs-datanode (2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
starting datanode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-datanode-ip-172-30-0-84.out
* Failed to start Hadoop datanode. Return value: 1
invoke-rc.d: initscript hadoop-hdfs-datanode, action “start” failed.
Setting up hadoop-hdfs-secondarynamenode (2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
starting secondarynamenode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-secondarynamenode-ip-172-30-0-84.out
* Failed to start Hadoop secondarynamenode. Return value: 1
invoke-rc.d: initscript hadoop-hdfs-secondarynamenode, action “start” failed.
Setting up hadoop-mapreduce-historyserver (2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
chown: changing ownership of ‘/var/log/hadoop-mapreduce’: Operation not permitted
starting historyserver, logging to /var/log/hadoop-mapreduce/mapred-mapred-historyserver-ip-172-30-0-84.out
15/08/13 19:18:51 INFO hs.JobHistoryServer: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting JobHistoryServer
STARTUP_MSG: host = ip-172-30-0-84.ec2.internal/172.30.0.84
STARTUP_MSG: args = []
STARTUP_MSG: version = 2.6.0-cdh5.4.4
STARTUP_MSG: classpath = /etc/hadoop/conf:/usr/lib/hadoop/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop/lib/jetty-util-6.1.26.cloudera.4.jar:/usr/lib/hadoop/lib/guava-11.0.2.jar:/usr/lib/hadoop/lib/slf4j-api-1.7.5.jar:/usr/lib/hadoop/lib/curator-framework-2.7.1.jar:/usr/lib/hadoop/lib/paranamer-2.3.jar:/usr/lib/hadoop/lib/logredactor-1.0.3.jar:/usr/lib/hadoop/lib/xz-1.0.jar:/usr/lib/hadoop/lib/aws-java-sdk-1.7.4.jar:/usr/lib/hadoop/lib/jersey-core-1.9.jar:/usr/lib/hadoop/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop/lib/httpcore-4.2.5.jar:/usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop/lib/jettison-1.1.jar:/usr/lib/hadoop/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop/lib/apacheds-i18n-2.0.0-M15.jar:/usr/lib/hadoop/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop/lib/java-xmlbuilder-0.4.jar:/usr/lib/hadoop/lib/log4j-1.2.17.jar:/usr/lib/hadoop/lib/jsp-api-2.1.jar:/usr/lib/hadoop/lib/commons-codec-1.4.jar:/usr/lib/hadoop/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop/lib/commons-digester-1.8.jar:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop/lib/commons-el-1.0.jar:/usr/lib/hadoop/lib/avro.jar:/usr/lib/hadoop/lib/httpclient-4.2.5.jar:/usr/lib/hadoop/lib/commons-lang-2.6.jar:/usr/lib/hadoop/lib/hamcrest-core-1.3.jar:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop/lib/jets3t-0.9.0.jar:/usr/lib/hadoop/lib/commons-net-3.1.jar:/usr/lib/hadoop/lib/commons-io-2.4.jar:/usr/lib/hadoop/lib/jersey-json-1.9.jar:/usr/lib/hadoop/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop/lib/jetty-6.1.26.cloudera.4.jar:/usr/lib/hadoop/lib/curator-recipes-2.7.1.jar:/usr/lib/hadoop/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop/lib/commons-configuration-1.6.jar:/usr/lib/hadoop/lib/jersey-server-1.9.jar:/usr/lib/hadoop/lib/zookeeper.jar:/usr/lib/hadoop/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop/lib/jasper-compiler-5.5.23.jar:/usr/lib/hadoop/lib/htrace-core-3.0.4.jar:/usr/lib/hadoop/lib/junit-4.11.jar:/usr/lib/hadoop/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop/lib/stax-api-1.0-2.jar:/usr/lib/hadoop/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/lib/hadoop/lib/jsr305-3.0.0.jar:/usr/lib/hadoop/lib/commons-cli-1.2.jar:/usr/lib/hadoop/lib/servlet-api-2.5.jar:/usr/lib/hadoop/lib/api-asn1-api-1.0.0-M20.jar:/usr/lib/hadoop/lib/commons-math3-3.1.1.jar:/usr/lib/hadoop/lib/asm-3.2.jar:/usr/lib/hadoop/lib/activation-1.1.jar:/usr/lib/hadoop/lib/jsch-0.1.42.jar:/usr/lib/hadoop/lib/api-util-1.0.0-M20.jar:/usr/lib/hadoop/lib/curator-client-2.7.1.jar:/usr/lib/hadoop/lib/slf4j-log4j12.jar:/usr/lib/hadoop/lib/gson-2.2.4.jar:/usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop/.//parquet-thrift.jar:/usr/lib/hadoop/.//hadoop-nfs.jar:/usr/lib/hadoop/.//parquet-column.jar:/usr/lib/hadoop/.//hadoop-common-2.6.0-cdh5.4.4-tests.jar:/usr/lib/hadoop/.//parquet-test-hadoop2.jar:/usr/lib/hadoop/.//parquet-tools.jar:/usr/lib/hadoop/.//hadoop-aws.jar:/usr/lib/hadoop/.//hadoop-aws-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop/.//parquet-hadoop-bundle.jar:/usr/lib/hadoop/.//hadoop-nfs-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop/.//parquet-encoding.jar:/usr/lib/hadoop/.//parquet-format.jar:/usr/lib/hadoop/.//parquet-scala_2.10.jar:/usr/lib/hadoop/.//parquet-avro.jar:/usr/lib/hadoop/.//parquet-cascading.jar:/usr/lib/hadoop/.//parquet-scrooge_2.10.jar:/usr/lib/hadoop/.//hadoop-annotations.jar:/usr/lib/hadoop/.//parquet-pig-bundle.jar:/usr/lib/hadoop/.//parquet-format-javadoc.jar:/usr/lib/hadoop/.//hadoop-auth.jar:/usr/lib/hadoop/.//parquet-pig.jar:/usr/lib/hadoop/.//parquet-hadoop.jar:/usr/lib/hadoop/.//hadoop-common-tests.jar:/usr/lib/hadoop/.//hadoop-common-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop/.//parquet-generator.jar:/usr/lib/hadoop/.//hadoop-common.jar:/usr/lib/hadoop/.//hadoop-auth-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop/.//hadoop-annotations-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop/.//parquet-common.jar:/usr/lib/hadoop/.//parquet-jackson.jar:/usr/lib/hadoop/.//parquet-protobuf.jar:/usr/lib/hadoop/.//parquet-format-sources.jar:/usr/lib/hadoop-hdfs/./:/usr/lib/hadoop-hdfs/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-hdfs/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop-hdfs/lib/jetty-util-6.1.26.cloudera.4.jar:/usr/lib/hadoop-hdfs/lib/guava-11.0.2.jar:/usr/lib/hadoop-hdfs/lib/jersey-core-1.9.jar:/usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop-hdfs/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-hdfs/lib/log4j-1.2.17.jar:/usr/lib/hadoop-hdfs/lib/jsp-api-2.1.jar:/usr/lib/hadoop-hdfs/lib/commons-codec-1.4.jar:/usr/lib/hadoop-hdfs/lib/xmlenc-0.52.jar:/usr/lib/hadoop-hdfs/lib/commons-el-1.0.jar:/usr/lib/hadoop-hdfs/lib/commons-lang-2.6.jar:/usr/lib/hadoop-hdfs/lib/commons-io-2.4.jar:/usr/lib/hadoop-hdfs/lib/jetty-6.1.26.cloudera.4.jar:/usr/lib/hadoop-hdfs/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-hdfs/lib/jersey-server-1.9.jar:/usr/lib/hadoop-hdfs/lib/htrace-core-3.0.4.jar:/usr/lib/hadoop-hdfs/lib/jsr305-3.0.0.jar:/usr/lib/hadoop-hdfs/lib/commons-daemon-1.0.13.jar:/usr/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/usr/lib/hadoop-hdfs/lib/servlet-api-2.5.jar:/usr/lib/hadoop-hdfs/lib/asm-3.2.jar:/usr/lib/hadoop-hdfs/.//hadoop-hdfs-2.6.0-cdh5.4.4-tests.jar:/usr/lib/hadoop-hdfs/.//hadoop-hdfs-nfs-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-hdfs/.//hadoop-hdfs.jar:/usr/lib/hadoop-hdfs/.//hadoop-hdfs-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-hdfs/.//hadoop-hdfs-tests.jar:/usr/lib/hadoop-hdfs/.//hadoop-hdfs-nfs.jar:/usr/lib/hadoop-yarn/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-yarn/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop-yarn/lib/jetty-util-6.1.26.cloudera.4.jar:/usr/lib/hadoop-yarn/lib/guava-11.0.2.jar:/usr/lib/hadoop-yarn/lib/guice-3.0.jar:/usr/lib/hadoop-yarn/lib/xz-1.0.jar:/usr/lib/hadoop-yarn/lib/jersey-core-1.9.jar:/usr/lib/hadoop-yarn/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-yarn/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-yarn/lib/jettison-1.1.jar:/usr/lib/hadoop-yarn/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-yarn/lib/log4j-1.2.17.jar:/usr/lib/hadoop-yarn/lib/commons-codec-1.4.jar:/usr/lib/hadoop-yarn/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop-yarn/lib/commons-lang-2.6.jar:/usr/lib/hadoop-yarn/lib/commons-io-2.4.jar:/usr/lib/hadoop-yarn/lib/jersey-json-1.9.jar:/usr/lib/hadoop-yarn/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop-yarn/lib/jetty-6.1.26.cloudera.4.jar:/usr/lib/hadoop-yarn/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-yarn/lib/jersey-server-1.9.jar:/usr/lib/hadoop-yarn/lib/zookeeper.jar:/usr/lib/hadoop-yarn/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop-yarn/lib/jersey-client-1.9.jar:/usr/lib/hadoop-yarn/lib/jline-2.11.jar:/usr/lib/hadoop-yarn/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop-yarn/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop-yarn/lib/stax-api-1.0-2.jar:/usr/lib/hadoop-yarn/lib/jsr305-3.0.0.jar:/usr/lib/hadoop-yarn/lib/commons-cli-1.2.jar:/usr/lib/hadoop-yarn/lib/javax.inject-1.jar:/usr/lib/hadoop-yarn/lib/aopalliance-1.0.jar:/usr/lib/hadoop-yarn/lib/servlet-api-2.5.jar:/usr/lib/hadoop-yarn/lib/asm-3.2.jar:/usr/lib/hadoop-yarn/lib/activation-1.1.jar:/usr/lib/hadoop-yarn/lib/jersey-guice-1.9.jar:/usr/lib/hadoop-yarn/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-yarn/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-nodemanager.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-registry-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-applications-distributedshell.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-registry.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-resourcemanager-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-applications-unmanaged-am-launcher-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-api-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-nodemanager-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-api.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-applications-unmanaged-am-launcher.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-web-proxy-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-applicationhistoryservice.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-resourcemanager.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-tests-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-tests.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-applicationhistoryservice-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-common.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-common-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-client-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-web-proxy.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-client.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-common-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-applications-distributedshell-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-common.jar:/usr/lib/hadoop-mapreduce/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-mapreduce/lib/guice-3.0.jar:/usr/lib/hadoop-mapreduce/lib/paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/lib/xz-1.0.jar:/usr/lib/hadoop-mapreduce/lib/jersey-core-1.9.jar:/usr/lib/hadoop-mapreduce/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-mapreduce/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-mapreduce/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-mapreduce/lib/log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/lib/avro.jar:/usr/lib/hadoop-mapreduce/lib/hamcrest-core-1.3.jar:/usr/lib/hadoop-mapreduce/lib/commons-io-2.4.jar:/usr/lib/hadoop-mapreduce/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-mapreduce/lib/jersey-server-1.9.jar:/usr/lib/hadoop-mapreduce/lib/junit-4.11.jar:/usr/lib/hadoop-mapreduce/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop-mapreduce/lib/javax.inject-1.jar:/usr/lib/hadoop-mapreduce/lib/aopalliance-1.0.jar:/usr/lib/hadoop-mapreduce/lib/asm-3.2.jar:/usr/lib/hadoop-mapreduce/lib/jersey-guice-1.9.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-examples.jar:/usr/lib/hadoop-mapreduce/.//hadoop-azure.jar:/usr/lib/hadoop-mapreduce/.//commons-logging-1.1.3.jar:/usr/lib/hadoop-mapreduce/.//hadoop-extras.jar:/usr/lib/hadoop-mapreduce/.//hadoop-distcp-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//jetty-util-6.1.26.cloudera.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs.jar:/usr/lib/hadoop-mapreduce/.//guava-11.0.2.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-shuffle.jar:/usr/lib/hadoop-mapreduce/.//curator-framework-2.7.1.jar:/usr/lib/hadoop-mapreduce/.//hadoop-datajoin.jar:/usr/lib/hadoop-mapreduce/.//hadoop-sls.jar:/usr/lib/hadoop-mapreduce/.//paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/.//xz-1.0.jar:/usr/lib/hadoop-mapreduce/.//hadoop-streaming.jar:/usr/lib/hadoop-mapreduce/.//metrics-core-3.0.1.jar:/usr/lib/hadoop-mapreduce/.//jersey-core-1.9.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-plugins.jar:/usr/lib/hadoop-mapreduce/.//jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-mapreduce/.//jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-mapreduce/.//hadoop-streaming-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//httpcore-4.2.5.jar:/usr/lib/hadoop-mapreduce/.//commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop-mapreduce/.//hadoop-datajoin-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//jackson-annotations-2.2.3.jar:/usr/lib/hadoop-mapreduce/.//jettison-1.1.jar:/usr/lib/hadoop-mapreduce/.//jasper-runtime-5.5.23.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient-tests.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient-2.6.0-cdh5.4.4-tests.jar:/usr/lib/hadoop-mapreduce/.//snappy-java-1.0.4.1.jar:/usr/lib/hadoop-mapreduce/.//apacheds-i18n-2.0.0-M15.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient.jar:/usr/lib/hadoop-mapreduce/.//hadoop-extras-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//java-xmlbuilder-0.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-nativetask.jar:/usr/lib/hadoop-mapreduce/.//log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/.//jsp-api-2.1.jar:/usr/lib/hadoop-mapreduce/.//commons-codec-1.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-core-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-examples-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//commons-digester-1.8.jar:/usr/lib/hadoop-mapreduce/.//xmlenc-0.52.jar:/usr/lib/hadoop-mapreduce/.//commons-el-1.0.jar:/usr/lib/hadoop-mapreduce/.//avro.jar:/usr/lib/hadoop-mapreduce/.//hadoop-gridmix.jar:/usr/lib/hadoop-mapreduce/.//jackson-databind-2.2.3.jar:/usr/lib/hadoop-mapreduce/.//httpclient-4.2.5.jar:/usr/lib/hadoop-mapreduce/.//jackson-core-2.2.3.jar:/usr/lib/hadoop-mapreduce/.//commons-lang-2.6.jar:/usr/lib/hadoop-mapreduce/.//hadoop-azure-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//microsoft-windowsazure-storage-sdk-0.6.0.jar:/usr/lib/hadoop-mapreduce/.//hamcrest-core-1.3.jar:/usr/lib/hadoop-mapreduce/.//commons-beanutils-1.7.0.jar:/usr/lib/hadoop-mapreduce/.//jets3t-0.9.0.jar:/usr/lib/hadoop-mapreduce/.//hadoop-ant-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-plugins-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//commons-net-3.1.jar:/usr/lib/hadoop-mapreduce/.//commons-io-2.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-core.jar:/usr/lib/hadoop-mapreduce/.//jersey-json-1.9.jar:/usr/lib/hadoop-mapreduce/.//commons-collections-3.2.1.jar:/usr/lib/hadoop-mapreduce/.//jetty-6.1.26.cloudera.4.jar:/usr/lib/hadoop-mapreduce/.//curator-recipes-2.7.1.jar:/usr/lib/hadoop-mapreduce/.//hadoop-auth.jar:/usr/lib/hadoop-mapreduce/.//protobuf-java-2.5.0.jar:/usr/lib/hadoop-mapreduce/.//commons-configuration-1.6.jar:/usr/lib/hadoop-mapreduce/.//jersey-server-1.9.jar:/usr/lib/hadoop-mapreduce/.//zookeeper.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//mockito-all-1.8.5.jar:/usr/lib/hadoop-mapreduce/.//jaxb-api-2.2.2.jar:/usr/lib/hadoop-mapreduce/.//jasper-compiler-5.5.23.jar:/usr/lib/hadoop-mapreduce/.//htrace-core-3.0.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-common-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//junit-4.11.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-app.jar:/usr/lib/hadoop-mapreduce/.//commons-compress-1.4.1.jar:/usr/lib/hadoop-mapreduce/.//jackson-xc-1.8.8.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-shuffle-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//stax-api-1.0-2.jar:/usr/lib/hadoop-mapreduce/.//apacheds-kerberos-codec-2.0.0-M15.jar:/usr/lib/hadoop-mapreduce/.//hadoop-auth-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-sls-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//jsr305-3.0.0.jar:/usr/lib/hadoop-mapreduce/.//commons-cli-1.2.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-nativetask-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-archives.jar:/usr/lib/hadoop-mapreduce/.//servlet-api-2.5.jar:/usr/lib/hadoop-mapreduce/.//api-asn1-api-1.0.0-M20.jar:/usr/lib/hadoop-mapreduce/.//hadoop-distcp.jar:/usr/lib/hadoop-mapreduce/.//hadoop-gridmix-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-ant.jar:/usr/lib/hadoop-mapreduce/.//hadoop-rumen-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-rumen.jar:/usr/lib/hadoop-mapreduce/.//commons-math3-3.1.1.jar:/usr/lib/hadoop-mapreduce/.//asm-3.2.jar:/usr/lib/hadoop-mapreduce/.//activation-1.1.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-app-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-archives-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//jsch-0.1.42.jar:/usr/lib/hadoop-mapreduce/.//api-util-1.0.0-M20.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-common.jar:/usr/lib/hadoop-mapreduce/.//curator-client-2.7.1.jar:/usr/lib/hadoop-mapreduce/.//gson-2.2.4.jar:/usr/lib/hadoop-mapreduce/.//jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-mapreduce/.//commons-httpclient-3.1.jar:/usr/lib/hadoop-mapreduce/.//activation-1.1.jar:/usr/lib/hadoop-mapreduce/.//apacheds-i18n-2.0.0-M15.jar:/usr/lib/hadoop-mapreduce/.//apacheds-kerberos-codec-2.0.0-M15.jar:/usr/lib/hadoop-mapreduce/.//api-asn1-api-1.0.0-M20.jar:/usr/lib/hadoop-mapreduce/.//api-util-1.0.0-M20.jar:/usr/lib/hadoop-mapreduce/.//asm-3.2.jar:/usr/lib/hadoop-mapreduce/.//avro.jar:/usr/lib/hadoop-mapreduce/.//commons-beanutils-1.7.0.jar:/usr/lib/hadoop-mapreduce/.//commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop-mapreduce/.//commons-cli-1.2.jar:/usr/lib/hadoop-mapreduce/.//commons-codec-1.4.jar:/usr/lib/hadoop-mapreduce/.//commons-collections-3.2.1.jar:/usr/lib/hadoop-mapreduce/.//commons-compress-1.4.1.jar:/usr/lib/hadoop-mapreduce/.//commons-configuration-1.6.jar:/usr/lib/hadoop-mapreduce/.//commons-digester-1.8.jar:/usr/lib/hadoop-mapreduce/.//commons-el-1.0.jar:/usr/lib/hadoop-mapreduce/.//commons-httpclient-3.1.jar:/usr/lib/hadoop-mapreduce/.//commons-io-2.4.jar:/usr/lib/hadoop-mapreduce/.//commons-lang-2.6.jar:/usr/lib/hadoop-mapreduce/.//commons-logging-1.1.3.jar:/usr/lib/hadoop-mapreduce/.//commons-math3-3.1.1.jar:/usr/lib/hadoop-mapreduce/.//commons-net-3.1.jar:/usr/lib/hadoop-mapreduce/.//curator-client-2.7.1.jar:/usr/lib/hadoop-mapreduce/.//curator-framework-2.7.1.jar:/usr/lib/hadoop-mapreduce/.//curator-recipes-2.7.1.jar:/usr/lib/hadoop-mapreduce/.//gson-2.2.4.jar:/usr/lib/hadoop-mapreduce/.//guava-11.0.2.jar:/usr/lib/hadoop-mapreduce/.//hadoop-ant-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-ant.jar:/usr/lib/hadoop-mapreduce/.//hadoop-archives-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-archives.jar:/usr/lib/hadoop-mapreduce/.//hadoop-auth-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-auth.jar:/usr/lib/hadoop-mapreduce/.//hadoop-azure-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-azure.jar:/usr/lib/hadoop-mapreduce/.//hadoop-datajoin-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-datajoin.jar:/usr/lib/hadoop-mapreduce/.//hadoop-distcp-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-distcp.jar:/usr/lib/hadoop-mapreduce/.//hadoop-extras-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-extras.jar:/usr/lib/hadoop-mapreduce/.//hadoop-gridmix-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-gridmix.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-app-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-app.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-common-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-common.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-core-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-core.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-plugins-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-plugins.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient-2.6.0-cdh5.4.4-tests.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient-tests.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-nativetask-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-nativetask.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-shuffle-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-shuffle.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-examples-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-examples.jar:/usr/lib/hadoop-mapreduce/.//hadoop-rumen-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-rumen.jar:/usr/lib/hadoop-mapreduce/.//hadoop-sls-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-sls.jar:/usr/lib/hadoop-mapreduce/.//hadoop-streaming-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-streaming.jar:/usr/lib/hadoop-mapreduce/.//hamcrest-core-1.3.jar:/usr/lib/hadoop-mapreduce/.//htrace-core-3.0.4.jar:/usr/lib/hadoop-mapreduce/.//httpclient-4.2.5.jar:/usr/lib/hadoop-mapreduce/.//httpcore-4.2.5.jar:/usr/lib/hadoop-mapreduce/.//jackson-annotations-2.2.3.jar:/usr/lib/hadoop-mapreduce/.//jackson-core-2.2.3.jar:/usr/lib/hadoop-mapreduce/.//jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-mapreduce/.//jackson-databind-2.2.3.jar:/usr/lib/hadoop-mapreduce/.//jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop-mapreduce/.//jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-mapreduce/.//jackson-xc-1.8.8.jar:/usr/lib/hadoop-mapreduce/.//jasper-compiler-5.5.23.jar:/usr/lib/hadoop-mapreduce/.//jasper-runtime-5.5.23.jar:/usr/lib/hadoop-mapreduce/.//java-xmlbuilder-0.4.jar:/usr/lib/hadoop-mapreduce/.//jaxb-api-2.2.2.jar:/usr/lib/hadoop-mapreduce/.//jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-mapreduce/.//jersey-core-1.9.jar:/usr/lib/hadoop-mapreduce/.//jersey-json-1.9.jar:/usr/lib/hadoop-mapreduce/.//jersey-server-1.9.jar:/usr/lib/hadoop-mapreduce/.//jets3t-0.9.0.jar:/usr/lib/hadoop-mapreduce/.//jettison-1.1.jar:/usr/lib/hadoop-mapreduce/.//jetty-6.1.26.cloudera.4.jar:/usr/lib/hadoop-mapreduce/.//jetty-util-6.1.26.cloudera.4.jar:/usr/lib/hadoop-mapreduce/.//jsch-0.1.42.jar:/usr/lib/hadoop-mapreduce/.//jsp-api-2.1.jar:/usr/lib/hadoop-mapreduce/.//jsr305-3.0.0.jar:/usr/lib/hadoop-mapreduce/.//junit-4.11.jar:/usr/lib/hadoop-mapreduce/.//log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/.//metrics-core-3.0.1.jar:/usr/lib/hadoop-mapreduce/.//microsoft-windowsazure-storage-sdk-0.6.0.jar:/usr/lib/hadoop-mapreduce/.//mockito-all-1.8.5.jar:/usr/lib/hadoop-mapreduce/.//paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/.//protobuf-java-2.5.0.jar:/usr/lib/hadoop-mapreduce/.//servlet-api-2.5.jar:/usr/lib/hadoop-mapreduce/.//snappy-java-1.0.4.1.jar:/usr/lib/hadoop-mapreduce/.//stax-api-1.0-2.jar:/usr/lib/hadoop-mapreduce/.//xmlenc-0.52.jar:/usr/lib/hadoop-mapreduce/.//xz-1.0.jar:/usr/lib/hadoop-mapreduce/.//zookeeper.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-api-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-api.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-applications-distributedshell-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-applications-distributedshell.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-applications-unmanaged-am-launcher-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-applications-unmanaged-am-launcher.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-client-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-client.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-common-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-common.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-registry-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-registry.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-applicationhistoryservice-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-applicationhistoryservice.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-common-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-common.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-nodemanager-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-nodemanager.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-resourcemanager-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-resourcemanager.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-tests-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-tests.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-web-proxy-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-web-proxy.jar:/usr/lib/hadoop-mapreduce/lib/aopalliance-1.0.jar:/usr/lib/hadoop-mapreduce/lib/asm-3.2.jar:/usr/lib/hadoop-mapreduce/lib/avro.jar:/usr/lib/hadoop-mapreduce/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop-mapreduce/lib/commons-io-2.4.jar:/usr/lib/hadoop-mapreduce/lib/guice-3.0.jar:/usr/lib/hadoop-mapreduce/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-mapreduce/lib/hamcrest-core-1.3.jar:/usr/lib/hadoop-mapreduce/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-mapreduce/lib/javax.inject-1.jar:/usr/lib/hadoop-mapreduce/lib/jersey-core-1.9.jar:/usr/lib/hadoop-mapreduce/lib/jersey-guice-1.9.jar:/usr/lib/hadoop-mapreduce/lib/jersey-server-1.9.jar:/usr/lib/hadoop-mapreduce/lib/junit-4.11.jar:/usr/lib/hadoop-mapreduce/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-mapreduce/lib/log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-mapreduce/lib/paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-mapreduce/lib/xz-1.0.jar:/usr/lib/hadoop-mapreduce/modules/*.jar
STARTUP_MSG: build = http://github.com/cloudera/hadoop -r b739cd891f6269da5dd22766d7e75bd2c9db73b6 ; compiled by ‘jenkins’ on 2015-07-06T23:58Z
STARTUP_MSG: java = 1.7.0_80
************************************************************/
* Started Hadoop historyserver:
Processing triggers for ureadahead (0.100.0-16) …
Setting up hadoop-conf-pseudo (2.6.0+cdh5.4.4+597-1.cdh5.4.4.p0.6~trusty-cdh5.4.4) …
update-alternatives: using /etc/hadoop/conf.pseudo to provide /etc/hadoop/conf (hadoop-conf) in auto mode
Processing triggers for libc-bin (2.19-0ubuntu6.6) …
root@ip-172-30-0-84:/home/ubuntu#

For YARN, a pseudo-distributed Hadoop installation consists of one node running all five Hadoop daemons: namenode, secondarynamenode, resourcemanager, datanode, and nodemanager.
5) To view files on Ubuntu system:
ubuntu@ip-172-30-0-142:~$ dpkg -L hadoop-conf-pseudo
/.
/etc
/etc/hadoop
/etc/hadoop/conf.pseudo
/etc/hadoop/conf.pseudo/hadoop-env.sh
/etc/hadoop/conf.pseudo/hadoop-metrics.properties
/etc/hadoop/conf.pseudo/core-site.xml
/etc/hadoop/conf.pseudo/README
/etc/hadoop/conf.pseudo/hdfs-site.xml
/etc/hadoop/conf.pseudo/mapred-site.xml
/etc/hadoop/conf.pseudo/yarn-site.xml
/etc/hadoop/conf.pseudo/log4j.properties
/usr
/usr/share
/usr/share/doc
/usr/share/doc/hadoop-conf-pseudo
/usr/share/doc/hadoop-conf-pseudo/copyright
/usr/share/doc/hadoop-conf-pseudo/changelog.Debian.gz
/usr/share/lintian
/usr/share/lintian/overrides
/usr/share/lintian/overrides/hadoop-conf-pseudo

6) Format Namenode
root@ip-172-30-0-84:/home/ubuntu# sudo -u hdfs hdfs namenode -format
15/08/13 19:35:00 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = ip-172-30-0-84.ec2.internal/172.30.0.84
STARTUP_MSG: args = [-format]
STARTUP_MSG: version = 2.6.0-cdh5.4.4
STARTUP_MSG: classpath = /etc/hadoop/conf:/usr/lib/hadoop/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop/lib/jetty-util-6.1.26.cloudera.4.jar:/usr/lib/hadoop/lib/guava-11.0.2.jar:/usr/lib/hadoop/lib/slf4j-api-1.7.5.jar:/usr/lib/hadoop/lib/curator-framework-2.7.1.jar:/usr/lib/hadoop/lib/paranamer-2.3.jar:/usr/lib/hadoop/lib/logredactor-1.0.3.jar:/usr/lib/hadoop/lib/xz-1.0.jar:/usr/lib/hadoop/lib/aws-java-sdk-1.7.4.jar:/usr/lib/hadoop/lib/jersey-core-1.9.jar:/usr/lib/hadoop/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop/lib/httpcore-4.2.5.jar:/usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop/lib/jettison-1.1.jar:/usr/lib/hadoop/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop/lib/apacheds-i18n-2.0.0-M15.jar:/usr/lib/hadoop/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop/lib/java-xmlbuilder-0.4.jar:/usr/lib/hadoop/lib/log4j-1.2.17.jar:/usr/lib/hadoop/lib/jsp-api-2.1.jar:/usr/lib/hadoop/lib/commons-codec-1.4.jar:/usr/lib/hadoop/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop/lib/commons-digester-1.8.jar:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop/lib/commons-el-1.0.jar:/usr/lib/hadoop/lib/avro.jar:/usr/lib/hadoop/lib/httpclient-4.2.5.jar:/usr/lib/hadoop/lib/commons-lang-2.6.jar:/usr/lib/hadoop/lib/hamcrest-core-1.3.jar:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop/lib/jets3t-0.9.0.jar:/usr/lib/hadoop/lib/commons-net-3.1.jar:/usr/lib/hadoop/lib/commons-io-2.4.jar:/usr/lib/hadoop/lib/jersey-json-1.9.jar:/usr/lib/hadoop/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop/lib/jetty-6.1.26.cloudera.4.jar:/usr/lib/hadoop/lib/curator-recipes-2.7.1.jar:/usr/lib/hadoop/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop/lib/commons-configuration-1.6.jar:/usr/lib/hadoop/lib/jersey-server-1.9.jar:/usr/lib/hadoop/lib/zookeeper.jar:/usr/lib/hadoop/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop/lib/jasper-compiler-5.5.23.jar:/usr/lib/hadoop/lib/htrace-core-3.0.4.jar:/usr/lib/hadoop/lib/junit-4.11.jar:/usr/lib/hadoop/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop/lib/stax-api-1.0-2.jar:/usr/lib/hadoop/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/lib/hadoop/lib/jsr305-3.0.0.jar:/usr/lib/hadoop/lib/commons-cli-1.2.jar:/usr/lib/hadoop/lib/servlet-api-2.5.jar:/usr/lib/hadoop/lib/api-asn1-api-1.0.0-M20.jar:/usr/lib/hadoop/lib/commons-math3-3.1.1.jar:/usr/lib/hadoop/lib/asm-3.2.jar:/usr/lib/hadoop/lib/activation-1.1.jar:/usr/lib/hadoop/lib/jsch-0.1.42.jar:/usr/lib/hadoop/lib/api-util-1.0.0-M20.jar:/usr/lib/hadoop/lib/curator-client-2.7.1.jar:/usr/lib/hadoop/lib/slf4j-log4j12.jar:/usr/lib/hadoop/lib/gson-2.2.4.jar:/usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop/.//parquet-thrift.jar:/usr/lib/hadoop/.//hadoop-nfs.jar:/usr/lib/hadoop/.//parquet-column.jar:/usr/lib/hadoop/.//hadoop-common-2.6.0-cdh5.4.4-tests.jar:/usr/lib/hadoop/.//parquet-test-hadoop2.jar:/usr/lib/hadoop/.//parquet-tools.jar:/usr/lib/hadoop/.//hadoop-aws.jar:/usr/lib/hadoop/.//hadoop-aws-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop/.//parquet-hadoop-bundle.jar:/usr/lib/hadoop/.//hadoop-nfs-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop/.//parquet-encoding.jar:/usr/lib/hadoop/.//parquet-format.jar:/usr/lib/hadoop/.//parquet-scala_2.10.jar:/usr/lib/hadoop/.//parquet-avro.jar:/usr/lib/hadoop/.//parquet-cascading.jar:/usr/lib/hadoop/.//parquet-scrooge_2.10.jar:/usr/lib/hadoop/.//hadoop-annotations.jar:/usr/lib/hadoop/.//parquet-pig-bundle.jar:/usr/lib/hadoop/.//parquet-format-javadoc.jar:/usr/lib/hadoop/.//hadoop-auth.jar:/usr/lib/hadoop/.//parquet-pig.jar:/usr/lib/hadoop/.//parquet-hadoop.jar:/usr/lib/hadoop/.//hadoop-common-tests.jar:/usr/lib/hadoop/.//hadoop-common-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop/.//parquet-generator.jar:/usr/lib/hadoop/.//hadoop-common.jar:/usr/lib/hadoop/.//hadoop-auth-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop/.//hadoop-annotations-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop/.//parquet-common.jar:/usr/lib/hadoop/.//parquet-jackson.jar:/usr/lib/hadoop/.//parquet-protobuf.jar:/usr/lib/hadoop/.//parquet-format-sources.jar:/usr/lib/hadoop-hdfs/./:/usr/lib/hadoop-hdfs/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-hdfs/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop-hdfs/lib/jetty-util-6.1.26.cloudera.4.jar:/usr/lib/hadoop-hdfs/lib/guava-11.0.2.jar:/usr/lib/hadoop-hdfs/lib/jersey-core-1.9.jar:/usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop-hdfs/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-hdfs/lib/log4j-1.2.17.jar:/usr/lib/hadoop-hdfs/lib/jsp-api-2.1.jar:/usr/lib/hadoop-hdfs/lib/commons-codec-1.4.jar:/usr/lib/hadoop-hdfs/lib/xmlenc-0.52.jar:/usr/lib/hadoop-hdfs/lib/commons-el-1.0.jar:/usr/lib/hadoop-hdfs/lib/commons-lang-2.6.jar:/usr/lib/hadoop-hdfs/lib/commons-io-2.4.jar:/usr/lib/hadoop-hdfs/lib/jetty-6.1.26.cloudera.4.jar:/usr/lib/hadoop-hdfs/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-hdfs/lib/jersey-server-1.9.jar:/usr/lib/hadoop-hdfs/lib/htrace-core-3.0.4.jar:/usr/lib/hadoop-hdfs/lib/jsr305-3.0.0.jar:/usr/lib/hadoop-hdfs/lib/commons-daemon-1.0.13.jar:/usr/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/usr/lib/hadoop-hdfs/lib/servlet-api-2.5.jar:/usr/lib/hadoop-hdfs/lib/asm-3.2.jar:/usr/lib/hadoop-hdfs/.//hadoop-hdfs-2.6.0-cdh5.4.4-tests.jar:/usr/lib/hadoop-hdfs/.//hadoop-hdfs-nfs-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-hdfs/.//hadoop-hdfs.jar:/usr/lib/hadoop-hdfs/.//hadoop-hdfs-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-hdfs/.//hadoop-hdfs-tests.jar:/usr/lib/hadoop-hdfs/.//hadoop-hdfs-nfs.jar:/usr/lib/hadoop-yarn/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-yarn/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop-yarn/lib/jetty-util-6.1.26.cloudera.4.jar:/usr/lib/hadoop-yarn/lib/guava-11.0.2.jar:/usr/lib/hadoop-yarn/lib/guice-3.0.jar:/usr/lib/hadoop-yarn/lib/xz-1.0.jar:/usr/lib/hadoop-yarn/lib/jersey-core-1.9.jar:/usr/lib/hadoop-yarn/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-yarn/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-yarn/lib/jettison-1.1.jar:/usr/lib/hadoop-yarn/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-yarn/lib/log4j-1.2.17.jar:/usr/lib/hadoop-yarn/lib/commons-codec-1.4.jar:/usr/lib/hadoop-yarn/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop-yarn/lib/commons-lang-2.6.jar:/usr/lib/hadoop-yarn/lib/commons-io-2.4.jar:/usr/lib/hadoop-yarn/lib/jersey-json-1.9.jar:/usr/lib/hadoop-yarn/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop-yarn/lib/jetty-6.1.26.cloudera.4.jar:/usr/lib/hadoop-yarn/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-yarn/lib/jersey-server-1.9.jar:/usr/lib/hadoop-yarn/lib/zookeeper.jar:/usr/lib/hadoop-yarn/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop-yarn/lib/jersey-client-1.9.jar:/usr/lib/hadoop-yarn/lib/jline-2.11.jar:/usr/lib/hadoop-yarn/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop-yarn/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop-yarn/lib/stax-api-1.0-2.jar:/usr/lib/hadoop-yarn/lib/jsr305-3.0.0.jar:/usr/lib/hadoop-yarn/lib/commons-cli-1.2.jar:/usr/lib/hadoop-yarn/lib/javax.inject-1.jar:/usr/lib/hadoop-yarn/lib/aopalliance-1.0.jar:/usr/lib/hadoop-yarn/lib/servlet-api-2.5.jar:/usr/lib/hadoop-yarn/lib/asm-3.2.jar:/usr/lib/hadoop-yarn/lib/activation-1.1.jar:/usr/lib/hadoop-yarn/lib/jersey-guice-1.9.jar:/usr/lib/hadoop-yarn/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-yarn/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-nodemanager.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-registry-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-applications-distributedshell.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-registry.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-resourcemanager-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-applications-unmanaged-am-launcher-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-api-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-nodemanager-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-api.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-applications-unmanaged-am-launcher.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-web-proxy-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-applicationhistoryservice.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-resourcemanager.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-tests-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-tests.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-applicationhistoryservice-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-common.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-common-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-client-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-web-proxy.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-client.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-common-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-applications-distributedshell-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-common.jar:/usr/lib/hadoop-mapreduce/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-mapreduce/lib/guice-3.0.jar:/usr/lib/hadoop-mapreduce/lib/paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/lib/xz-1.0.jar:/usr/lib/hadoop-mapreduce/lib/jersey-core-1.9.jar:/usr/lib/hadoop-mapreduce/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-mapreduce/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-mapreduce/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-mapreduce/lib/log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/lib/avro.jar:/usr/lib/hadoop-mapreduce/lib/hamcrest-core-1.3.jar:/usr/lib/hadoop-mapreduce/lib/commons-io-2.4.jar:/usr/lib/hadoop-mapreduce/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-mapreduce/lib/jersey-server-1.9.jar:/usr/lib/hadoop-mapreduce/lib/junit-4.11.jar:/usr/lib/hadoop-mapreduce/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop-mapreduce/lib/javax.inject-1.jar:/usr/lib/hadoop-mapreduce/lib/aopalliance-1.0.jar:/usr/lib/hadoop-mapreduce/lib/asm-3.2.jar:/usr/lib/hadoop-mapreduce/lib/jersey-guice-1.9.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-examples.jar:/usr/lib/hadoop-mapreduce/.//hadoop-azure.jar:/usr/lib/hadoop-mapreduce/.//commons-logging-1.1.3.jar:/usr/lib/hadoop-mapreduce/.//hadoop-extras.jar:/usr/lib/hadoop-mapreduce/.//hadoop-distcp-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//jetty-util-6.1.26.cloudera.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs.jar:/usr/lib/hadoop-mapreduce/.//guava-11.0.2.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-shuffle.jar:/usr/lib/hadoop-mapreduce/.//curator-framework-2.7.1.jar:/usr/lib/hadoop-mapreduce/.//hadoop-datajoin.jar:/usr/lib/hadoop-mapreduce/.//hadoop-sls.jar:/usr/lib/hadoop-mapreduce/.//paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/.//xz-1.0.jar:/usr/lib/hadoop-mapreduce/.//hadoop-streaming.jar:/usr/lib/hadoop-mapreduce/.//metrics-core-3.0.1.jar:/usr/lib/hadoop-mapreduce/.//jersey-core-1.9.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-plugins.jar:/usr/lib/hadoop-mapreduce/.//jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-mapreduce/.//jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-mapreduce/.//hadoop-streaming-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//httpcore-4.2.5.jar:/usr/lib/hadoop-mapreduce/.//commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop-mapreduce/.//hadoop-datajoin-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//jackson-annotations-2.2.3.jar:/usr/lib/hadoop-mapreduce/.//jettison-1.1.jar:/usr/lib/hadoop-mapreduce/.//jasper-runtime-5.5.23.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient-tests.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient-2.6.0-cdh5.4.4-tests.jar:/usr/lib/hadoop-mapreduce/.//snappy-java-1.0.4.1.jar:/usr/lib/hadoop-mapreduce/.//apacheds-i18n-2.0.0-M15.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient.jar:/usr/lib/hadoop-mapreduce/.//hadoop-extras-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//java-xmlbuilder-0.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-nativetask.jar:/usr/lib/hadoop-mapreduce/.//log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/.//jsp-api-2.1.jar:/usr/lib/hadoop-mapreduce/.//commons-codec-1.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-core-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-examples-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//commons-digester-1.8.jar:/usr/lib/hadoop-mapreduce/.//xmlenc-0.52.jar:/usr/lib/hadoop-mapreduce/.//commons-el-1.0.jar:/usr/lib/hadoop-mapreduce/.//avro.jar:/usr/lib/hadoop-mapreduce/.//hadoop-gridmix.jar:/usr/lib/hadoop-mapreduce/.//jackson-databind-2.2.3.jar:/usr/lib/hadoop-mapreduce/.//httpclient-4.2.5.jar:/usr/lib/hadoop-mapreduce/.//jackson-core-2.2.3.jar:/usr/lib/hadoop-mapreduce/.//commons-lang-2.6.jar:/usr/lib/hadoop-mapreduce/.//hadoop-azure-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//microsoft-windowsazure-storage-sdk-0.6.0.jar:/usr/lib/hadoop-mapreduce/.//hamcrest-core-1.3.jar:/usr/lib/hadoop-mapreduce/.//commons-beanutils-1.7.0.jar:/usr/lib/hadoop-mapreduce/.//jets3t-0.9.0.jar:/usr/lib/hadoop-mapreduce/.//hadoop-ant-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-plugins-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//commons-net-3.1.jar:/usr/lib/hadoop-mapreduce/.//commons-io-2.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-core.jar:/usr/lib/hadoop-mapreduce/.//jersey-json-1.9.jar:/usr/lib/hadoop-mapreduce/.//commons-collections-3.2.1.jar:/usr/lib/hadoop-mapreduce/.//jetty-6.1.26.cloudera.4.jar:/usr/lib/hadoop-mapreduce/.//curator-recipes-2.7.1.jar:/usr/lib/hadoop-mapreduce/.//hadoop-auth.jar:/usr/lib/hadoop-mapreduce/.//protobuf-java-2.5.0.jar:/usr/lib/hadoop-mapreduce/.//commons-configuration-1.6.jar:/usr/lib/hadoop-mapreduce/.//jersey-server-1.9.jar:/usr/lib/hadoop-mapreduce/.//zookeeper.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//mockito-all-1.8.5.jar:/usr/lib/hadoop-mapreduce/.//jaxb-api-2.2.2.jar:/usr/lib/hadoop-mapreduce/.//jasper-compiler-5.5.23.jar:/usr/lib/hadoop-mapreduce/.//htrace-core-3.0.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-common-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//junit-4.11.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-app.jar:/usr/lib/hadoop-mapreduce/.//commons-compress-1.4.1.jar:/usr/lib/hadoop-mapreduce/.//jackson-xc-1.8.8.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-shuffle-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//stax-api-1.0-2.jar:/usr/lib/hadoop-mapreduce/.//apacheds-kerberos-codec-2.0.0-M15.jar:/usr/lib/hadoop-mapreduce/.//hadoop-auth-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-sls-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//jsr305-3.0.0.jar:/usr/lib/hadoop-mapreduce/.//commons-cli-1.2.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-nativetask-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-archives.jar:/usr/lib/hadoop-mapreduce/.//servlet-api-2.5.jar:/usr/lib/hadoop-mapreduce/.//api-asn1-api-1.0.0-M20.jar:/usr/lib/hadoop-mapreduce/.//hadoop-distcp.jar:/usr/lib/hadoop-mapreduce/.//hadoop-gridmix-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-ant.jar:/usr/lib/hadoop-mapreduce/.//hadoop-rumen-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-rumen.jar:/usr/lib/hadoop-mapreduce/.//commons-math3-3.1.1.jar:/usr/lib/hadoop-mapreduce/.//asm-3.2.jar:/usr/lib/hadoop-mapreduce/.//activation-1.1.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-app-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//hadoop-archives-2.6.0-cdh5.4.4.jar:/usr/lib/hadoop-mapreduce/.//jsch-0.1.42.jar:/usr/lib/hadoop-mapreduce/.//api-util-1.0.0-M20.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-common.jar:/usr/lib/hadoop-mapreduce/.//curator-client-2.7.1.jar:/usr/lib/hadoop-mapreduce/.//gson-2.2.4.jar:/usr/lib/hadoop-mapreduce/.//jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-mapreduce/.//commons-httpclient-3.1.jar
STARTUP_MSG: build = http://github.com/cloudera/hadoop -r b739cd891f6269da5dd22766d7e75bd2c9db73b6 ; compiled by ‘jenkins’ on 2015-07-06T23:58Z
STARTUP_MSG: java = 1.7.0_80
************************************************************/
15/08/13 19:35:00 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
15/08/13 19:35:00 INFO namenode.NameNode: createNameNode [-format]
Formatting using clusterid: CID-b2904cb5-935f-4a49-855f-e135a07344bf
15/08/13 19:35:01 INFO namenode.FSNamesystem: No KeyProvider found.
15/08/13 19:35:01 INFO namenode.FSNamesystem: fsLock is fair:true
15/08/13 19:35:01 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit=1000
15/08/13 19:35:01 INFO blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=true
15/08/13 19:35:01 INFO blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
15/08/13 19:35:01 INFO blockmanagement.BlockManager: The block deletion will start around 2015 Aug 13 19:35:01
15/08/13 19:35:01 INFO util.GSet: Computing capacity for map BlocksMap
15/08/13 19:35:01 INFO util.GSet: VM type = 64-bit
15/08/13 19:35:01 INFO util.GSet: 2.0% max memory 889 MB = 17.8 MB
15/08/13 19:35:01 INFO util.GSet: capacity = 2^21 = 2097152 entries
15/08/13 19:35:01 INFO blockmanagement.BlockManager: dfs.block.access.token.enable=false
15/08/13 19:35:01 INFO blockmanagement.BlockManager: defaultReplication = 1
15/08/13 19:35:01 INFO blockmanagement.BlockManager: maxReplication = 512
15/08/13 19:35:01 INFO blockmanagement.BlockManager: minReplication = 1
15/08/13 19:35:01 INFO blockmanagement.BlockManager: maxReplicationStreams = 2
15/08/13 19:35:01 INFO blockmanagement.BlockManager: shouldCheckForEnoughRacks = false
15/08/13 19:35:01 INFO blockmanagement.BlockManager: replicationRecheckInterval = 3000
15/08/13 19:35:01 INFO blockmanagement.BlockManager: encryptDataTransfer = false
15/08/13 19:35:01 INFO blockmanagement.BlockManager: maxNumBlocksToLog = 1000
15/08/13 19:35:01 INFO namenode.FSNamesystem: fsOwner = hdfs (auth:SIMPLE)
15/08/13 19:35:01 INFO namenode.FSNamesystem: supergroup = supergroup
15/08/13 19:35:01 INFO namenode.FSNamesystem: isPermissionEnabled = true
15/08/13 19:35:01 INFO namenode.FSNamesystem: HA Enabled: false
15/08/13 19:35:01 INFO namenode.FSNamesystem: Append Enabled: true
15/08/13 19:35:01 INFO util.GSet: Computing capacity for map INodeMap
15/08/13 19:35:01 INFO util.GSet: VM type = 64-bit
15/08/13 19:35:01 INFO util.GSet: 1.0% max memory 889 MB = 8.9 MB
15/08/13 19:35:01 INFO util.GSet: capacity = 2^20 = 1048576 entries
15/08/13 19:35:01 INFO namenode.NameNode: Caching file names occuring more than 10 times
15/08/13 19:35:01 INFO util.GSet: Computing capacity for map cachedBlocks
15/08/13 19:35:01 INFO util.GSet: VM type = 64-bit
15/08/13 19:35:01 INFO util.GSet: 0.25% max memory 889 MB = 2.2 MB
15/08/13 19:35:01 INFO util.GSet: capacity = 2^18 = 262144 entries
15/08/13 19:35:01 INFO namenode.FSNamesystem: dfs.namenode.safemode.threshold-pct = 0.9990000128746033
15/08/13 19:35:01 INFO namenode.FSNamesystem: dfs.namenode.safemode.min.datanodes = 0
15/08/13 19:35:01 INFO namenode.FSNamesystem: dfs.namenode.safemode.extension = 0
15/08/13 19:35:01 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.window.num.buckets = 10
15/08/13 19:35:01 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.num.users = 10
15/08/13 19:35:01 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.windows.minutes = 1,5,25
15/08/13 19:35:01 INFO namenode.FSNamesystem: Retry cache on namenode is enabled
15/08/13 19:35:01 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
15/08/13 19:35:01 INFO util.GSet: Computing capacity for map NameNodeRetryCache
15/08/13 19:35:01 INFO util.GSet: VM type = 64-bit
15/08/13 19:35:01 INFO util.GSet: 0.029999999329447746% max memory 889 MB = 273.1 KB
15/08/13 19:35:01 INFO util.GSet: capacity = 2^15 = 32768 entries
15/08/13 19:35:01 INFO namenode.NNConf: ACLs enabled? false
15/08/13 19:35:01 INFO namenode.NNConf: XAttrs enabled? true
15/08/13 19:35:01 INFO namenode.NNConf: Maximum size of an xattr: 16384
15/08/13 19:35:01 INFO namenode.FSImage: Allocated new BlockPoolId: BP-1416750624-172.30.0.84-1439494501674
15/08/13 19:35:01 INFO common.Storage: Storage directory /var/lib/hadoop-hdfs/cache/hdfs/dfs/name has been successfully formatted.
15/08/13 19:35:01 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0
15/08/13 19:35:01 INFO util.ExitUtil: Exiting with status 0
15/08/13 19:35:01 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at ip-172-30-0-84.ec2.internal/172.30.0.84
************************************************************/
root@ip-172-30-0-84:/home/ubuntu#

7) Start HDFS
ubuntu@ip-172-30-0-178:~$ for x in `cd /etc/init.d ; ls hadoop-hdfs-*` ; do sudo service $x start ; done
starting datanode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-datanode-ip-172-30-0-178.out
* Started Hadoop datanode (hadoop-hdfs-datanode):
starting namenode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-namenode-ip-172-30-0-178.out
* Started Hadoop namenode:
starting secondarynamenode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-secondarynamenode-ip-172-30-0-178.out
* Started Hadoop secondarynamenode:

8) Create directories needed for Hadoop processes
ubuntu@ip-172-30-0-142:~$ sudo /usr/lib/hadoop/libexec/init-hdfs.sh
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -mkdir -p /tmp’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -chmod -R 1777 /tmp’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -mkdir -p /var’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -mkdir -p /var/log’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -chmod -R 1775 /var/log’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -chown yarn:mapred /var/log’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -mkdir -p /tmp/hadoop-yarn’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -chown -R mapred:mapred /tmp/hadoop-yarn’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -mkdir -p /tmp/hadoop-yarn/staging/history/done_intermediate’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -chown -R mapred:mapred /tmp/hadoop-yarn/staging’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -chmod -R 1777 /tmp’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -mkdir -p /var/log/hadoop-yarn/apps’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -chmod -R 1777 /var/log/hadoop-yarn/apps’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -chown yarn:mapred /var/log/hadoop-yarn/apps’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -mkdir -p /hbase’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -chown hbase /hbase’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -mkdir -p /benchmarks’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -chmod -R 777 /benchmarks’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -mkdir -p /user’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -mkdir -p /user/history’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -chown mapred /user/history’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -mkdir -p /user/jenkins’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -chmod -R 777 /user/jenkins’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -chown jenkins /user/jenkins’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -mkdir -p /user/hive’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -chmod -R 777 /user/hive’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -chown hive /user/hive’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -mkdir -p /user/root’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -chmod -R 777 /user/root’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -chown root /user/root’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -mkdir -p /user/hue’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -chmod -R 777 /user/hue’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -chown hue /user/hue’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -mkdir -p /user/oozie’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -mkdir -p /user/oozie/share’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -mkdir -p /user/oozie/share/lib’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -mkdir -p /user/oozie/share/lib/hive’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -mkdir -p /user/oozie/share/lib/mapreduce-streaming’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -mkdir -p /user/oozie/share/lib/distcp’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -mkdir -p /user/oozie/share/lib/pig’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -mkdir -p /user/oozie/share/lib/sqoop’
+ ls ‘/usr/lib/hive/lib/*.jar’
+ ls /usr/lib/hadoop-mapreduce/hadoop-streaming-2.6.0-cdh5.4.4.jar /usr/lib/hadoop-mapreduce/hadoop-streaming.jar
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -put /usr/lib/hadoop-mapreduce/hadoop-streaming*.jar /user/oozie/share/lib/mapreduce-streaming’
+ ls /usr/lib/hadoop-mapreduce/hadoop-distcp-2.6.0-cdh5.4.4.jar /usr/lib/hadoop-mapreduce/hadoop-distcp.jar
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -put /usr/lib/hadoop-mapreduce/hadoop-distcp*.jar /user/oozie/share/lib/distcp’
+ ls ‘/usr/lib/pig/lib/*.jar’ ‘/usr/lib/pig/*.jar’
+ ls ‘/usr/lib/sqoop/lib/*.jar’ ‘/usr/lib/sqoop/*.jar’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -chmod -R 777 /user/oozie’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -chown -R oozie /user/oozie’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -mkdir -p /var/lib/hadoop-hdfs/cache/mapred/mapred/staging’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -chmod 1777 /var/lib/hadoop-hdfs/cache/mapred/mapred/staging’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -chown -R mapred /var/lib/hadoop-hdfs/cache/mapred’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -mkdir -p /user/spark/applicationHistory’
+ su -s /bin/bash hdfs -c ‘/usr/bin/hadoop fs -chown spark /user/spark/applicationHistory’

9) Verify the HDFS file structure:
ubuntu@ip-172-30-0-178:~$ sudo -u hdfs hadoop fs -ls -R /
drwxrwxrwx – hdfs supergroup 0 2015-08-13 20:11 /benchmarks
drwxr-xr-x – hbase supergroup 0 2015-08-13 20:11 /hbase
drwxrwxrwt – hdfs supergroup 0 2015-08-13 20:11 /tmp
drwxrwxrwt – mapred mapred 0 2015-08-13 20:11 /tmp/hadoop-yarn
drwxrwxrwt – mapred mapred 0 2015-08-13 20:11 /tmp/hadoop-yarn/staging
drwxrwxrwt – mapred mapred 0 2015-08-13 20:11 /tmp/hadoop-yarn/staging/history
drwxrwxrwt – mapred mapred 0 2015-08-13 20:11 /tmp/hadoop-yarn/staging/history/done_intermediate
drwxr-xr-x – hdfs supergroup 0 2015-08-13 20:12 /user
drwxr-xr-x – mapred supergroup 0 2015-08-13 20:11 /user/history
drwxrwxrwx – hive supergroup 0 2015-08-13 20:11 /user/hive
drwxrwxrwx – hue supergroup 0 2015-08-13 20:12 /user/hue
drwxrwxrwx – jenkins supergroup 0 2015-08-13 20:11 /user/jenkins
drwxrwxrwx – oozie supergroup 0 2015-08-13 20:12 /user/ooze
drwxrwxrwx – oozie supergroup 0 2015-08-13 20:12 /user/oozie/share
drwxrwxrwx – oozie supergroup 0 2015-08-13 20:12 /user/oozie/share/lib
drwxrwxrwx – oozie supergroup 0 2015-08-13 20:12 /user/oozie/share/lib/distcp
-rwxrwxrwx 1 oozie supergroup 98630 2015-08-13 20:12 /user/oozie/share/lib/distcp/hadoop-distcp-2.6.0-cdh5.4.4.jar
-rwxrwxrwx 1 oozie supergroup 98630 2015-08-13 20:12 /user/oozie/share/lib/distcp/hadoop-distcp.jar
drwxrwxrwx – oozie supergroup 0 2015-08-13 20:12 /user/oozie/share/lib/hive
drwxrwxrwx – oozie supergroup 0 2015-08-13 20:12 /user/oozie/share/lib/mapreduce-streaming
-rwxrwxrwx 1 oozie supergroup 109997 2015-08-13 20:12 /user/oozie/share/lib/mapreduce-streaming/hadoop-streaming-2.6.0-cdh5.4.4.jar
-rwxrwxrwx 1 oozie supergroup 109997 2015-08-13 20:12 /user/oozie/share/lib/mapreduce-streaming/hadoop-streaming.jar
drwxrwxrwx – oozie supergroup 0 2015-08-13 20:12 /user/oozie/share/lib/pig
drwxrwxrwx – oozie supergroup 0 2015-08-13 20:12 /user/oozie/share/lib/sqoop
drwxrwxrwx – root supergroup 0 2015-08-13 20:11 /user/root
drwxr-xr-x – hdfs supergroup 0 2015-08-13 20:12 /user/spark
drwxr-xr-x – spark supergroup 0 2015-08-13 20:12 /user/spark/applicationHistory
drwxr-xr-x – hdfs supergroup 0 2015-08-13 20:12 /var
drwxr-xr-x – hdfs supergroup 0 2015-08-13 20:12 /var/lib
drwxr-xr-x – hdfs supergroup 0 2015-08-13 20:12 /var/lib/hadoop-hdfs
drwxr-xr-x – hdfs supergroup 0 2015-08-13 20:12 /var/lib/hadoop-hdfs/cache
drwxr-xr-x – mapred supergroup 0 2015-08-13 20:12 /var/lib/hadoop-hdfs/cache/mapred
drwxr-xr-x – mapred supergroup 0 2015-08-13 20:12 /var/lib/hadoop-hdfs/cache/mapred/mapred
drwxrwxrwt – mapred supergroup 0 2015-08-13 20:12 /var/lib/hadoop-hdfs/cache/mapred/mapred/staging
drwxrwxr-t – yarn mapred 0 2015-08-13 20:11 /var/log
drwxr-xr-x – hdfs mapred 0 2015-08-13 20:11 /var/log/hadoop-yarn
drwxrwxrwt – yarn mapred 0 2015-08-13 20:11 /var/log/hadoop-yarn/apps

10) Run ps -ef at your command prompt to see if it has started all required deamons or not:
You will see following:

ubuntu@ip-172-30-0-178:~$ ps -ef
ubuntu 1484 1483 0 19:43 pts/0 00:00:00 -bash
yarn 2365 1 0 19:46 ? 00:00:17 /usr/lib/jvm/java-7-oracle/bin/java -Dproc_resourcemanager -Xmx1000m -Dhadoop.log.dir=/var/log/hadoop-yarn -Dyarn.log.dir=/var/log/hadoop-yarn -Dhadoop.lo
yarn 2453 1 0 19:46 ? 00:00:12 /usr/lib/jvm/java-7-oracle/bin/java -Dproc_nodemanager -Xmx1000m -Dhadoop.log.dir=/var/log/hadoop-yarn -Dyarn.log.dir=/var/log/hadoop-yarn -Dhadoop.log.fi
mapred 3128 1 0 19:47 ? 00:00:09 /usr/lib/jvm/java-7-oracle/bin/java -Dproc_historyserver -Xmx1000m -Dhadoop.log.dir=/var/log/hadoop-mapreduce -Dhadoop.log.file=hadoop.log -Dhadoop.home.d
root 3217 2 0 19:49 ? 00:00:00 [kworker/u30:2]
hdfs 3399 1 0 20:01 ? 00:00:09 /usr/lib/jvm/java-7-oracle/bin/java -Dproc_datanode -Xmx1000m -Dhadoop.log.dir=/var/log/hadoop-hdfs -Dhadoop.log.file=hadoop-hdfs-datanode-ip-172-30-0-178
hdfs 3497 1 0 20:01 ? 00:00:08 /usr/lib/jvm/java-7-oracle/bin/java -Dproc_namenode -Xmx1000m -Dhadoop.log.dir=/var/log/hadoop-hdfs -Dhadoop.log.file=hadoop-hdfs-namenode-ip-172-30-0-178
hdfs 3613 1 0 20:01 ? 00:00:05 /usr/lib/jvm/java-7-oracle/bin/java -Dproc_secondarynamenode -Xmx1000m -Dhadoop.log.dir=/var/log/hadoop-hdfs -Dhadoop.log.file=hadoop-hdfs-secondarynameno
ubuntu 5923 1484 0 20:32 pts/0 00:00:00 ps -ef

11) Some useful commands:
Start YARN
$ sudo service hadoop-yarn-resourcemanager start
Start Node Manager
$ sudo service hadoop-yarn-nodemanager start
Start History Server
$ sudo service hadoop-mapreduce-historyserver start

12) Check Resource Manager at: http://yourmachineDNS:8088
13) Check your security group rules: at-least following port should have inbound access

RangeSource

TypeProtocolPort
SSHTCP22Anywhere
Custom TCP RuleTCP50070Anywhere
Custom TCP RuleTCP8088Anywhere

14) Set HADOOP_MAPREDUCE_HOME using following command and then check if you can run hadoop command from anywhere using command line.
ubuntu@ip-172-30-0-178:~$ export HADOOP_MAPRED_HOME=/usr/lib/hadoop-mapreduce
ubuntu@ip-172-30-0-178:~$ hadoop
Usage: hadoop [--config confdir] COMMAND
where COMMAND is one of:
fs run a generic filesystem user client
version print the version
jar run a jar file
checknative [-a|-h] check native hadoop and compression libraries availability
distcp copy file or directories recursively
archive -archiveName NAME -p * create a hadoop archive
classpath prints the class path needed to get the
credential interact with credential providers
Hadoop jar and the required libraries
daemonlog get/set the log level for each daemon
trace view and modify Hadoop tracing settings
or
CLASSNAME run the class named CLASSNAME

Most commands print help when invoked w/o parameters.

15) check hadoop version
ubuntu@ip-172-30-0-178:~$ hadoop version
Hadoop 2.6.0-cdh5.4.4
Subversion http://github.com/cloudera/hadoop -r b739cd891f6269da5dd22766d7e75bd2c9db73b6
Compiled by jenkins on 2015-07-06T23:58Z
Compiled with protoc 2.5.0
From source with checksum 4acea6ac185376e0b48b33695e88e7a7
This command was run using /usr/lib/hadoop/hadoop-common-2.6.0-cdh5.4.4.jar

Running an example application with YARN
1) Add user Pankaj
ubuntu@ip-172-30-0-178:~$ sudo useradd -d /home/pankaj -m pankaj
ubuntu@ip-172-30-0-178:~$ sudo passwd pankaj
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
ubuntu@ip-172-30-0-178:~$ ls -la /home/pankaj
total 20
drwxr-xr-x 2 pankaj pankaj 4096 Aug 14 02:56 .
drwxr-xr-x 4 root root 4096 Aug 14 02:56 ..
-rw-r–r– 1 pankaj pankaj 220 Apr 9 2014 .bash_logout
-rw-r–r– 1 pankaj pankaj 3637 Apr 9 2014 .bashrc
-rw-r–r– 1 pankaj pankaj 675 Apr 9 2014 .profile

2) create a home directory on HDFS for the user who will be running the job
root@ip-172-30-0-142:/home/ubuntu# sudo -u hdfs hadoop fs -mkdir /user/pankaj
root@ip-172-30-0-142:/home/ubuntu# sudo -u hdfs hadoop fs -chown pankaj /user/pankaj
root@ip-172-30-0-142:/home/ubuntu# ubuntu# sudo -u hdfs hadls /user/
Found 9 items
drwxr-xr-x – mapred supergroup 0 2015-08-14 14:41 /user/history
drwxrwxrwx – hive supergroup 0 2015-08-14 14:41 /user/hive
drwxrwxrwx – hue supergroup 0 2015-08-14 14:41 /user/hue
drwxrwxrwx – jenkins supergroup 0 2015-08-14 14:41 /user/jenkins
drwxrwxrwx – oozie supergroup 0 2015-08-14 14:41 /user/oozie
drwxr-xr-x – pankaj supergroup 0 2015-08-14 16:25 /user/pankaj
drwxrwxrwx – root supergroup 0 2015-08-14 14:41 /user/root
drwxr-xr-x – hdfs supergroup 0 2015-08-14 14:42 /user/spark

3) change user to newly created user
ubuntu@ip-172-30-0-142:/home/ubuntu# su - pankaj
Password: …

4) Make a directory in HDFS for input file
pankaj@ip-172-30-0-142:~$ hadoop fs -mkdir input

pankaj@ip-172-30-0-142:~$ hadoop fs -ls
Found 1 items
drwxr-xr-x – pankaj supergroup 0 2015-08-14 16:26 input

5) get your input file (this is just an example, you can get file from sources)
pankaj@ip-172-30-0-142:~$ wget http://pthakkar.com/plaintext.txt ~/
pankaj@ip-172-30-0-142:~$ ls
plaintext.txt

6) copy from local directory to HDFS
pankaj@ip-172-30-0-142:~$ hadoop fs -copyFromLocal ~/plaintext.txt input
pankaj@ip-172-30-0-142:~$ hadoop fs -ls
Found 1 items
drwxr-xr-x – pankaj supergroup 0 2015-08-14 16:33 input
pankaj@ip-172-30-0-142:~$ hadoop fs -ls input
Found 1 items
-rw-r–r– 1 pankaj supergroup 4538523 2015-08-14 16:33 input/plaintext.txt

You can do the same thing with put command e.g hdfs dfs -put localfile /user/hadoop/hadoopfile
pankaj@ip-172-30-0-142:~$ hadoop fs -put ~/plaintext.txt input

7) Just note of caution: do not create output directory.
8) Run the map reduce example job that comes with distribution. It takes three arguments: first-> name of the example job which is wordcount in our case, second-> input directory, third-> name of the output directory which will be created the example job.
pankaj@ip-172-30-0-142:~$ hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar wordcount input output
15/08/14 16:37:15 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
15/08/14 16:37:15 INFO input.FileInputFormat: Total input paths to process : 1
15/08/14 16:37:15 INFO mapreduce.JobSubmitter: number of splits:1
15/08/14 16:37:16 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1439567484890_0002
15/08/14 16:37:16 INFO impl.YarnClientImpl: Submitted application application_1439567484890_0002
15/08/14 16:37:16 INFO mapreduce.Job: The url to track the job: http://ip-172-30-0-142:8088/proxy/application_1439567484890_0002/
15/08/14 16:37:16 INFO mapreduce.Job: Running job: job_1439567484890_0002
15/08/14 16:37:21 INFO mapreduce.Job: Job job_1439567484890_0002 running in uber mode : false
15/08/14 16:37:21 INFO mapreduce.Job: map 0% reduce 0%
15/08/14 16:37:27 INFO mapreduce.Job: map 100% reduce 0%
15/08/14 16:37:33 INFO mapreduce.Job: map 100% reduce 100%
15/08/14 16:37:33 INFO mapreduce.Job: Job job_1439567484890_0002 completed successfully
15/08/14 16:37:33 INFO mapreduce.Job: Counters: 49
File System Counters
FILE: Number of bytes read=483860
FILE: Number of bytes written=1189105
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=4538643
HDFS: Number of bytes written=356409
HDFS: Number of read operations=6
HDFS: Number of large read operations=0
HDFS: Number of write operations=2
Job Counters
Launched map tasks=1
Launched reduce tasks=1
Data-local map tasks=1
Total time spent by all maps in occupied slots (ms)=4141
Total time spent by all reduces in occupied slots (ms)=2994
Total time spent by all map tasks (ms)=4141
Total time spent by all reduce tasks (ms)=2994
Total vcore-seconds taken by all map tasks=4141
Total vcore-seconds taken by all reduce tasks=2994
Total megabyte-seconds taken by all map tasks=4240384
Total megabyte-seconds taken by all reduce tasks=3065856
Map-Reduce Framework
Map input records=129107
Map output records=980637
Map output bytes=8406347
Map output materialized bytes=483860
Input split bytes=120
Combine input records=980637
Combine output records=33505
Reduce input groups=33505
Reduce shuffle bytes=483860
Reduce input records=33505
Reduce output records=33505
Spilled Records=67010
Shuffled Maps =1
Failed Shuffles=0
Merged Map outputs=1
GC time elapsed (ms)=54
CPU time spent (ms)=5270
Physical memory (bytes) snapshot=524533760
Virtual memory (bytes) snapshot=2764136448
Total committed heap usage (bytes)=570425344
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
File Input Format Counters
Bytes Read=4538523
File Output Format Counters
Bytes Written=356409

9) check the output in output directory
pankaj@ip-172-30-0-142:~$ hadoop fs -ls output
Found 2 items
-rw-r–r– 1 pankaj supergroup 0 2015-08-14 16:37 output/_SUCCESS
-rw-r–r– 1 pankaj supergroup 356409 2015-08-14 16:37 output/part-r-00000

10) copy output files from hdfs output directory to local directory so you can open with your favourite editor.
pankaj@ip-172-30-0-142:~$ hadoop fs -copyToLocal output/* ~/
pankaj@ip-172-30-0-142:~$ ls
part-r-00000 plaintext.txt _SUCCESS
pankaj@ip-172-30-0-142:~$ less part-r-00000

Comments are closed.