How to Install Hadoop Cluster on Ubuntu – Part 1

Install SSH

Hadoop use SSH for control Data Node

sudo apt-get install ssh

Tip – how to login root

If you want to login root by Ubuntu you must have to set password of root
Becuase Ubuntu root password is null, so can’t login self.

sudo passwd root
<Input new Password>
<Retype new Password>

add host on /etc/hosts

we have to edit the /etc/hosts for machine know about each hadoop cluster node. and also need to change to hostname.

And if have 127.0.0.1 <hostname> you have to delete

sudo nano /etc/hosts

192.168.24.130 hadoop1
192.168.24.131 hadoop2
192.168.24.132 hadoop3

sudo nano /etc/hostname
hadoop1

Install java-8-openjdk

apt-get -y remove "java-*"
sudo apt-get install openjdk-8-jdk

Check JAVA_HOME

Check Java install path. it will use JAVA_HOME environment settings

update-alternatives --display java

Downlad to hadoop

download the hadoop-3.2.1.tar.gz and decompress the Hadoop package.

wget http://mirror.intergrid.com.au/apache/hadoop/common/hadoop-3.2.1/hadoop-3.2.1.tar.gz
tar -xzvf hadoop-3.2.1.tar.gz
sudo mv hadoop-3.2.1 /opt/hadoop

Create hadoop account

 dedicated Hadoop user account for running Hadoop applications

sudo addgroup hadoopgroup
sudo adduser --ingroup hadoopgroup hadoop
sudo adduser hadoop sudo
sudo chown hadoop:root -R /opt/hadoop
sudo chmod g+rwx -R /opt/hadoop

need to login as Hadoopuser and generating a key value pair using SSH

su - hadoop
ssh-keygen -t rsa -P "" -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub>> ~/.ssh/authorized_keys
chmod 0600 ~/.ssh/authorized_keys

ssh connection check without password

ssh localhost

If you can’t connect ssh localhost, you try to install ssh

input hadoop Config Path

Environment settings that will be used by Hadoop.To centralize these settings in /etc/profile

sudo vi /etc/profile

export HADOOP_HOME="/opt/hadoop"
export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
export JAVA_HOME="/usr/lib/jvm/java-8-openjdk-amd64/jre"
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export HADOOP_HDFS_HOME=$HADOOP_HOME
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_YARN_HOME=$HADOOP_HOME
Facebook Comments

Leave A Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.