How To Install Hadoop On Kali Linux Quickly

🛠 Step 1: Install Java (OpenJDK 11)

Hadoop requires Java to function. Follow these steps to install Java.

1️⃣ Update the Package List

sudo apt update && sudo apt upgrade -y

2️⃣ Install OpenJDK 11

sudo apt install openjdk-11-jdk -y

3️⃣ Verify Java Installation

java -version
javac -version

Expected output:

openjdk version "11.0.xx"
OpenJDK Runtime Environment...

🔐 Step 2: Setup SSH for Hadoop

Hadoop requires passwordless SSH access.

1️⃣ Install SSH Server

sudo apt install openssh-server -y

2️⃣ Start and Enable SSH Service

sudo systemctl start ssh
sudo systemctl enable ssh

3️⃣ Create SSH Key for Passwordless Authentication

ssh-keygen -t rsa -P "" -f ~/.ssh/id_rsa

4️⃣ Add Key to Authorized Keys

cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

5️⃣ Test SSH

ssh localhost

If prompted, type yes and press Enter.

📂 Step 3: Download and Install Hadoop

1️⃣ Download Hadoop

wget https://downloads.apache.org/hadoop/common/hadoop-3.3.6/hadoop-3.3.6.tar.gz

2️⃣ Extract and Move Hadoop

tar -xvzf hadoop-3.3.6.tar.gz
sudo mv hadoop-3.3.6 /usr/local/hadoop
sudo chown -R $USER:$USER /usr/local/hadoop

📝 Step 4: Configure Hadoop Environment Variables

1️⃣ Edit ~/.bashrc

nano ~/.bashrc

2️⃣ Add the Following Lines

export HADOOP_HOME=/usr/local/hadoop
export HADOOP_INSTALL=$HADOOP_HOME
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/native"
export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64

3️⃣ Apply Changes

source ~/.bashrc

⚙️ Step 5: Configure Hadoop XML Files

1️⃣ Edit core-site.xml

nano $HADOOP_HOME/etc/hadoop/core-site.xml

Add:

<property>
    <name>fs.defaultFS</name>
    <value>hdfs://localhost:9000</value>
</property>

2️⃣ Edit hdfs-site.xml

nano $HADOOP_HOME/etc/hadoop/hdfs-site.xml

Add:

<property>
    <name>dfs.replication</name>
    <value>1</value>
</property>
<property>
    <name>dfs.namenode.name.dir</name>
    <value>file:///usr/local/hadoop/data/namenode</value>
</property>
<property>
    <name>dfs.datanode.data.dir</name>
    <value>file:///usr/local/hadoop/data/datanode</value>
</property>

3️⃣ Configure mapred-site.xml

cp $HADOOP_HOME/etc/hadoop/mapred-site.xml.template $HADOOP_HOME/etc/hadoop/mapred-site.xml
nano $HADOOP_HOME/etc/hadoop/mapred-site.xml

Add:

<property>
    <name>mapreduce.framework.name</name>
    <value>yarn</value>
</property>

4️⃣ Configure yarn-site.xml

nano $HADOOP_HOME/etc/hadoop/yarn-site.xml

Add:

<property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
</property>
<property>
    <name>yarn.nodemanager.env-whitelist</name>
    <value>JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,HADOOP_MAPRED_HOME</value>
</property>

🖥️ Step 6: Format HDFS and Start Hadoop

1️⃣ Format the Hadoop Namenode

hdfs namenode -format

2️⃣ Start Hadoop Services

start-dfs.sh
start-yarn.sh

3️⃣ Check Running Hadoop Processes

jps

Expected output:

NameNode
DataNode
SecondaryNameNode
ResourceManager
NodeManager
Jps

🌐 Step 7: Verify Hadoop Web Interfaces

• NameNode UI → http://localhost:9870/
• ResourceManager UI → http://localhost:8088/

If these pages load successfully, Hadoop is running correctly. 🎉

🛠 Step 8: Run HDFS Commands (Test Hadoop)

1️⃣ Create a Directory in HDFS

hdfs dfs -mkdir /test

2️⃣ Upload a File to HDFS

echo "Hello Hadoop" > testfile.txt
hdfs dfs -put testfile.txt /test/

3️⃣ List Files in HDFS

hdfs dfs -ls /test/

Expected output:

Found 1 items
-rw-r--r--   1 hadoop supergroup        12 2025-03-07 10:00 /test/testfile.txt

🔄 Restarting Hadoop

If needed, restart Hadoop services:

stop-dfs.sh
stop-yarn.sh
start-dfs.sh
start-yarn.sh

Check if all services are running:

jps

🎉 Congratulations!

You have successfully installed and configured Hadoop on Kali Linux in a single-node mode! 🚀🔥 Now, you can start experimenting with MapReduce, HDFS, and YARN.