Control Output Verbosity in Gobuster

Introduction

Gobuster is a powerful tool used for directory and file brute-forcing, DNS subdomain brute-forcing, and S3 bucket enumeration. When performing scans, the amount of information displayed in the terminal can vary significantly. Understanding how to control this output verbosity is crucial for efficient analysis and to avoid overwhelming your terminal with unnecessary data.

In this lab, you will learn how to manage Gobuster's output verbosity. You will perform scans using the default settings, then explore the quiet mode (-q) to minimize output, and finally, the verbose mode (-v) to maximize detail. By comparing these different levels, you will gain practical experience in tailoring Gobuster's output to suit various scenarios, from quick checks to in-depth investigations.

Run a Standard Scan and Observe the Default Output

In this step, you will perform a standard Gobuster scan without any specific verbosity flags. This will allow you to observe the default output behavior of Gobuster, which typically shows found directories and files as they are discovered.

First, ensure you are in your home directory's project folder.

cd ~/project

Now, execute the Gobuster command. We will use a local target for this lab to avoid scanning external websites. We will use http://localhost as the target and a small wordlist for demonstration purposes.

gobuster dir -u http://localhost -w /usr/share/wordlists/dirb/common.txt -x php,html,txt -z

Let's break down the command:

• gobuster dir: Specifies that we are performing a directory/file brute-forcing scan.
• -u http://localhost: Sets the target URL to http://localhost.
• -w /usr/share/wordlists/dirb/common.txt: Specifies the wordlist to use for enumeration. This wordlist contains common directory and file names.
• -x php,html,txt: Specifies file extensions to look for.
• -z: Hides the status bar, which can sometimes interfere with capturing clean output for comparison.

Observe the output in your terminal. You should see lines indicating the directories and files found, along with their HTTP status codes.

===============================================================
Gobuster v3.1.0
by OJ <ojob.dev>
===============================================================
[+] Url: http://localhost
[+] Threads: 10
[+] Wordlist: /usr/share/wordlists/dirb/common.txt
[+] Extensions: php, html, txt
[+] Status codes: 200,204,301,302,307,401,403
[+] User Agent: gobuster/3.1.0
[+] Timeout: 10s
===============================================================
2024/07/29 08:00:00 Starting gobuster in directory enumeration mode
/index.html (Status: 200)
/server-status (Status: 403)
/manual (Status: 301)
/icons (Status: 301)
/phpmyadmin (Status: 301)
/test.php (Status: 200)
/info.php (Status: 200)
/robots.txt (Status: 200)
/license.txt (Status: 200)
/README.txt (Status: 200)
===============================================================
2024/07/29 08:00:00 Finished
===============================================================

This is the default output, providing a balance between information and conciseness.

Rerun the Scan in Quiet Mode with the -q Flag

In this step, you will rerun the Gobuster scan using the quiet mode. The -q flag suppresses most of the output, showing only the essential findings. This mode is useful when you want to quickly check for specific resources without being distracted by verbose information, or when piping output to another tool.

Execute the same Gobuster command as before, but this time add the -q flag:

gobuster dir -u http://localhost -w /usr/share/wordlists/dirb/common.txt -x php,html,txt -z -q

Observe the output. You should notice that the initial banner, statistics, and progress updates are no longer displayed. Only the found directories and files are printed to the console.

/index.html (Status: 200)
/server-status (Status: 403)
/manual (Status: 301)
/icons (Status: 301)
/phpmyadmin (Status: 301)
/test.php (Status: 200)
/info.php (Status: 200)
/robots.txt (Status: 200)
/license.txt (Status: 200)
/README.txt (Status: 200)

This quiet output is ideal for scripting or when you only care about the discovered paths.

Rerun the Scan with Verbose Mode using the -v Flag

In this step, you will rerun the Gobuster scan using the verbose mode. The -v flag provides more detailed information about the scanning process, including every request made and its corresponding response status. This mode is particularly useful for debugging or when you need to understand why certain paths are not being found or are returning unexpected responses.

Execute the same Gobuster command, but this time add the -v flag:

gobuster dir -u http://localhost -w /usr/share/wordlists/dirb/common.txt -x php,html,txt -z -v

Observe the output. You will see a significant increase in the amount of information displayed. For each word in the wordlist, Gobuster will show the URL being tested and the HTTP status code received, even for paths that do not exist or return errors.

===============================================================
Gobuster v3.1.0
by OJ <ojob.dev>
===============================================================
[+] Url: http://localhost
[+] Threads: 10
[+] Wordlist: /usr/share/wordlists/dirb/common.txt
[+] Extensions: php, html, txt
[+] Status codes: 200,204,301,302,307,401,403
[+] User Agent: gobuster/3.1.0
[+] Timeout: 10s
===============================================================
2024/07/29 08:00:00 Starting gobuster in directory enumeration mode
Testing url: http://localhost/index.html (Status: 200)
Testing url: http://localhost/server-status (Status: 403)
Testing url: http://localhost/manual (Status: 301)
Testing url: http://localhost/icons (Status: 301)
Testing url: http://localhost/phpmyadmin (Status: 301)
Testing url: http://localhost/test.php (Status: 200)
Testing url: http://localhost/info.php (Status: 200)
Testing url: http://localhost/robots.txt (Status: 200)
Testing url: http://localhost/license.txt (Status: 200)
Testing url: http://localhost/README.txt (Status: 200)
Testing url: http://localhost/admin (Status: 404)
Testing url: http://localhost/login (Status: 404)
... (many more lines for each tested path)
===============================================================
2024/07/29 08:00:00 Finished
===============================================================

This verbose output provides a complete trace of the scanning process, which can be invaluable for troubleshooting or deep analysis.

Compare the Three Levels of Output Detail

In this step, you will compare the outputs from the three different Gobuster scans you performed: default, quiet (-q), and verbose (-v). Understanding the differences will help you choose the most appropriate output level for various situations.

Recall the characteristics of each output:

• Default Output: Provides a summary banner, progress updates (if -z is not used), and lists only the found directories/files with their status codes. It's a good balance for general use.
• Quiet Output (-q): Suppresses all non-essential information, showing only the found directories/files. This is useful for clean, parsable output, especially when piping to other tools or when you only need the results.
• Verbose Output (-v): Displays extensive details, including every URL tested and its response status, regardless of whether it's a "found" path or an error. This is invaluable for debugging, understanding network interactions, or when you need a full audit trail of the scan.

Consider the following scenarios and how each output level might be best suited:

• Quick check for common paths: Quiet mode (-q) would be efficient.
• General reconnaissance: Default output provides enough context.
• Troubleshooting a scan that isn't finding expected paths: Verbose mode (-v) would help identify why requests are failing or what responses are being received.
• Integrating Gobuster into a script for automated processing: Quiet mode (-q) ensures clean output that's easy to parse.

By comparing the terminal outputs you observed in the previous steps, you can clearly see how each flag alters the verbosity.

Choose the Appropriate Verbosity for Different Scenarios

In this final step, you will solidify your understanding of when to use each Gobuster output verbosity level. The choice of verbosity depends heavily on your objective and the context of your penetration testing or security assessment.

Here's a summary of when to use each mode:

• Default Mode (no flags):

  • When to use: For general-purpose scanning, initial reconnaissance, or when you need a balanced view of the scan's progress and findings. It's the most common starting point.
  • Example: You're exploring a new target and want to quickly see what common directories exist without too much noise.

• Quiet Mode (-q):

  • When to use: When you only care about the successful findings and want to suppress all other informational messages, banners, and errors. This is ideal for scripting, piping output to other tools (like grep or awk), or when you're running many scans in parallel and want minimal terminal clutter.
  • Example: You're writing a script to automate directory enumeration and want to feed the found URLs directly into another tool for further analysis.

• Verbose Mode (-v):

  • When to use: For debugging, troubleshooting, or when you need a detailed understanding of every request and response. This mode shows all attempts, including those that result in 404s or other errors, which can be crucial for identifying subtle issues or understanding server behavior.
  • Example: Your Gobuster scan isn't finding expected directories, and you suspect a network issue, a WAF blocking requests, or an unusual server response. The verbose output will show you exactly what's happening for each request.

By mastering these verbosity options, you can make your Gobuster scans more efficient and tailored to your specific needs, saving time and improving your analysis capabilities.

Summary

In this lab, you have successfully learned how to control the output verbosity of Gobuster. You started by performing a standard scan to observe the default output, which provides a balanced view of the enumeration process. Then, you explored the quiet mode using the -q flag, which minimizes output to only show the discovered paths, making it ideal for scripting and clean results. Finally, you utilized the verbose mode with the -v flag to gain a detailed insight into every request and response, which is invaluable for debugging and in-depth analysis.

By understanding and applying these different verbosity levels, you can effectively tailor Gobuster's output to suit various scenarios, from quick checks to detailed troubleshooting, thereby enhancing your efficiency in directory and file enumeration tasks.