Hadoop FS Shell copyToLocal/put

Introduction

Imagine a mystical forest filled with ancient trees and magical creatures. Within this forest lives a dark sorcerer who has the power to manipulate the elements. Your quest is to navigate through the enchanted forest and retrieve a precious artifact guarded by the dark sorcerer. To achieve this, you must harness the power of Hadoop's Hadoop HDFS skills, specifically focusing on the FS Shell copyToLocal/put commands.

Hadoop FS Shell CopyToLocal

In this step, you will use the Hadoop FS Shell command copyToLocal to copy a file from HDFS to the local file system.

Open the terminal and follow the steps below to get started.

1. Switch to the Hadoop user for proper permissions:

   su - hadoop
   

2. Create a sample file in HDFS:

   echo "Magic artifact content" | hdfs dfs -put - /home/hadoop/magic_artifact.txt
   

3. Use FS Shell copyToLocal to copy the file to the local system:

   hdfs dfs -copyToLocal /home/hadoop/magic_artifact.txt /home/hadoop/magic_artifact.txt
   

4. Verify the file copy by checking the local file existence:

   ls /home/hadoop/magic_artifact.txt
   

Hadoop FS Shell Put

In this step, you will utilize the FS Shell command put to upload a file from the local system to HDFS.

1. Create a local sample file:

   echo "Dark sorcerer's spellbook" > /home/hadoop/spellbook.txt
   

2. Use FS Shell put to upload the file to HDFS:

   hdfs dfs -put /home/hadoop/spellbook.txt /home/hadoop/spellbook.txt
   

3. Verify the file upload by checking its existence in HDFS:

   hdfs dfs -ls /home/hadoop/spellbook.txt
   

Summary

In this lab, we designed a magical scenario within an enchanted forest to engage users in learning the FS Shell copyToLocal and put commands in Hadoop. By embarking on a quest to retrieve a precious artifact from a dark sorcerer, users were guided through practical steps to copy files between HDFS and the local system. Through this immersive experience, users gained hands-on experience in managing files within the Hadoop ecosystem and honed their HDFS skills.