Hadoop FS Shell mkdir

Introduction

Imagine yourself as a communications officer stationed at an extraterrestrial base on a distant planet. Your task is to manage the communication data stored on the Hadoop Distributed File System (HDFS). Today, you will learn how to use the Hadoop FS Shell mkdir command to create directories to organize your data effectively.

Skills Graph

Create a New Directory

In this step, you will create a new directory named communications in the HDFS.

1. Switch to the hadoop user:

   su - hadoop

2. Check the current directory in HDFS:

   hdfs dfs -ls /

   If you execute the above command at this time, you will find that HDFS does not have any directories.

3. Create a new directory named communications:

   hdfs dfs -mkdir /communications

4. Explain the purpose of the command and interpret the output:

   The command hdfs dfs -mkdir /communications is used to create a new directory named "communications" in the Hadoop Distributed File System (HDFS). Let's break down the command:

   • hdfs: This is the command-line tool used to interact with the HDFS.
   • dfs: It stands for "Distributed File System" and is a subcommand of the hdfs tool.
   • -mkdir: This is an option that specifies that we want to create a new directory.
   • communications: This is the name of the directory we want to create.

5. Verify that the directory was created successfully:

   hdfs dfs -ls /

After this command, if you have found a directory named "communications", which means you have completed this step.

Create Subdirectories

In this step, you will create subdirectories within the communications directory to further organize your communication data.

1. Create a subdirectory named communications:

   hdfs dfs -mkdir /communications/reports

2. Create another subdirectory named messages:

   hdfs dfs -mkdir /communications/messages

3. Verify that the subdirectories were created successfully:

   hdfs dfs -ls /communications

If the subdirectories "reports" and "messages" are displayed under the "communications" directory after executing this command, it indicates that the experiment has been successfully completed.

Summary

In this lab, you have learned how to use the Hadoop FS Shell mkdir command to create directories in HDFS. By organising your data into structured directories, you can efficiently manage and access communication files within the Hadoop ecosystem. This lab was designed to enhance your skills in Hadoop HDFS file management and directory structuring.

Additionally, during the learning process, you can also draw an analogy between this command and the mkdir command in Linux.