LabEx
Log InJoin Free

Perform a Basic Directory Scan in Gobuster

Beginner
Practice Now

Introduction

In this lab, you will learn the fundamentals of using Gobuster for directory scanning. Gobuster is a powerful tool used in penetration testing and ethical hacking to discover hidden directories and files on web servers. By brute-forcing common directory names against a target URL, Gobuster can reveal sensitive information or overlooked entry points. This lab will guide you through selecting a target, choosing an appropriate wordlist, constructing the Gobuster command, executing the scan, and interpreting the results.

Select a Target URL for the Scan

In this step, you will select a target URL for your Gobuster scan. For the purpose of this lab, we will use a deliberately vulnerable web application hosted locally. This ensures that you can perform the scan without impacting external systems and observe typical results.

First, let's ensure the target web server is running. We will use curl to check if the web server is accessible.

Open your terminal in the ~/project directory.

curl http://localhost:8080

You should see an HTML output, indicating the web server is active. The target URL for our scan will be http://localhost:8080.

<!DOCTYPE html>
<html>
<head>
<title>Welcome to Nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to Nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

Choose a Basic Wordlist

In this step, you will choose a basic wordlist for your Gobuster scan. A wordlist is a file containing a list of common directory and file names that Gobuster will attempt to find on the target server. The quality and comprehensiveness of your wordlist directly impact the effectiveness of your scan.

For this lab, we will use a small, pre-installed wordlist that is suitable for demonstration purposes. This wordlist is located at /usr/share/wordlists/dirb/common.txt.

You can inspect the first few lines of this wordlist using the head command to understand its content:

head /usr/share/wordlists/dirb/common.txt

This command will display the beginning of the wordlist, showing examples of the directory names Gobuster will test.

admin/
backup/
bin/
cgi-bin/
conf/
css/
data/
db/
dev/
doc/

This wordlist contains common directory names that are often found on web servers. Using a smaller wordlist like this helps to complete the scan quickly for the purpose of this lab.

Construct the Basic gobuster dir Command

In this step, you will construct the basic gobuster dir command. The gobuster dir command is used for directory and file brute-forcing. It requires at least two main flags: -u for the target URL and -w for the wordlist.

The basic syntax for the command is:

gobuster dir -u <target_url> -w <wordlist_path>

Based on our previous steps:

  • The target URL is http://localhost:8080.
  • The wordlist path is /usr/share/wordlists/dirb/common.txt.

So, the command you will construct is:

gobuster dir -u http://localhost:8080 -w /usr/share/wordlists/dirb/common.txt

This command tells Gobuster to perform a directory scan on http://localhost:8080 using the entries in common.txt as potential directory names. You will execute this command in the next step.

Execute the Scan with -u and -w Flags

In this step, you will execute the gobuster dir command you constructed in the previous step. This will initiate the directory scan against our target URL using the specified wordlist.

Execute the following command in your terminal:

gobuster dir -u http://localhost:8080 -w /usr/share/wordlists/dirb/common.txt

As the scan runs, Gobuster will display its progress and any directories or files it discovers. The output will show the status code (e.g., 200 for OK, 301 for Moved Permanently) and the size of the response for each found entry.

===============================================================
Gobuster vX.X.X-XXXXXX Linux/amd64
===============================================================
[+] Url:                     http://localhost:8080
[+] Threads:                 10
[+] Wordlist:                /usr/share/wordlists/dirb/common.txt
[+] Status codes:            200,204,301,302,307,401,403
[+] User Agent:              gobuster/X.X.X
[+] Timeout:                 10s
===============================================================
XXXX/XX/XX XX:XX:XX Starting gobuster in directory enumeration mode
===============================================================
/css                  (Status: 301)
/js                   (Status: 301)
/images               (Status: 301)
/admin                (Status: 301)
/icons                (Status: 301)
/server-status        (Status: 403)
===============================================================
XXXX/XX/XX XX:XX:XX Finished
===============================================================

The output above shows several directories found, along with their HTTP status codes. For example, /css returned a 301 status, indicating a permanent redirect.

Analyze the Output for Found Directories

In this step, you will analyze the output from your Gobuster scan to understand the results. The output provides valuable information about the directories and files discovered on the target web server.

Review the output from the previous step. You should see lines similar to these:

/css                  (Status: 301)
/js                   (Status: 301)
/images               (Status: 301)
/admin                (Status: 301)
/icons                (Status: 301)
/server-status        (Status: 403)

Each line represents a potential directory or file found by Gobuster.

  • The first part, e.g., /css, is the path that Gobuster successfully identified.
  • (Status: XXX) indicates the HTTP status code returned by the server for that path.
    • 200 OK: The request was successful, and the content was returned. This often means a valid directory or file.
    • 301 Moved Permanently: The resource has been permanently moved to a new URL. This still indicates the existence of the directory.
    • 403 Forbidden: The server understood the request but refuses to authorize it. This means the directory exists but you don't have permission to access it directly.
    • 401 Unauthorized: Similar to 403, but typically requires authentication.

In our example, /css, /js, /images, /admin, and /icons all returned 301 status codes, meaning these directories exist and the server is redirecting requests to them. The /server-status path returned a 403 status, indicating it exists but access is forbidden.

This analysis helps you identify potential areas for further investigation, such as administrative panels (/admin) or configuration files, which could be vulnerable.

Summary

In this lab, you successfully performed a basic directory scan using Gobuster. You learned how to select a target URL, choose an appropriate wordlist, construct the gobuster dir command with the -u and -w flags, execute the scan, and analyze the output to identify existing directories and their HTTP status codes. This fundamental skill is crucial for reconnaissance in cybersecurity, helping you discover hidden web assets that might contain sensitive information or vulnerabilities. You can now apply these techniques with different wordlists and targets to expand your understanding of web server structures.