Route Gobuster Traffic Through a Proxy

Introduction

In this lab, you will learn a fundamental technique in web application penetration testing: routing Gobuster traffic through a web proxy. Gobuster is a powerful tool for directory and file brute-forcing, DNS subdomain brute-forcing, and S3 bucket enumeration. By routing its traffic through a proxy like Burp Suite or OWASP ZAP, you gain the ability to inspect, modify, and analyze the HTTP requests and responses generated by Gobuster. This visibility is invaluable for understanding how web applications behave and for identifying potential vulnerabilities.

This lab will guide you through setting up a proxy, configuring Gobuster to use it, executing a scan, and observing the traffic within the proxy.

Start a Web Proxy like Burp Suite or OWASP ZAP

In this step, you will start a web proxy application. For this lab, we will use OWASP ZAP, which is pre-installed in the LabEx environment. OWASP ZAP provides a graphical interface to intercept and analyze web traffic.

First, open a new terminal window if you don't have one open. Then, launch OWASP ZAP.

/opt/zaproxy/zap.sh &

After launching, OWASP ZAP will start. You might see a prompt about persisting the ZAP session. For this lab, you can choose "No, thanks" and click "Start".

Once ZAP is running, you should see its main interface. This indicates that the proxy application is ready to be configured.

Configure the Proxy to Listen on a Specific Port

In this step, you will configure OWASP ZAP to listen on a specific IP address and port. By default, ZAP often listens on 127.0.0.1:8080. We will confirm this setting.

In OWASP ZAP:

1. Go to Tools -> Options... (or File -> Options... depending on ZAP version).
2. In the Options dialog, navigate to Local Proxies under the Network section.
3. Verify that the address is 127.0.0.1 and the port is 8080. If it's different, you can modify it here, but for this lab, we will assume 127.0.0.1:8080.
4. Click OK to close the Options dialog.

This configuration ensures that ZAP is listening for incoming connections on the specified address and port, ready to intercept traffic.

Use the -p Flag in Gobuster to Specify the Proxy URL

In this step, you will learn how to instruct Gobuster to route its traffic through the proxy you just configured. Gobuster provides the -p (or --proxy) flag for this purpose, allowing you to specify the proxy's URL.

The format for the proxy URL is typically http://<IP_ADDRESS>:<PORT>. Since our ZAP proxy is listening on 127.0.0.1:8080, the proxy URL will be http://127.0.0.1:8080.

We will prepare a Gobuster command that includes this proxy setting. We will use a simple target for demonstration, for example, http://example.com.

echo "gobuster dir -u http://example.com -w /usr/share/wordlists/dirb/common.txt -p http://127.0.0.1:8080" > ~/project/gobuster_command.sh
chmod +x ~/project/gobuster_command.sh

This command creates a script gobuster_command.sh in your ~/project directory that contains the Gobuster command with the proxy flag. This script will be executed in the next step.

Execute a Gobuster Scan

In this step, you will execute the Gobuster scan using the command prepared in the previous step. This will initiate the directory brute-forcing process, and all its HTTP traffic will be routed through OWASP ZAP.

Open a new terminal or use your existing one and execute the script:

~/project/gobuster_command.sh

You will see Gobuster's output in the terminal, showing the directories and files it discovers. While Gobuster is running, it's sending requests through ZAP. The scan might take a few moments to complete, depending on the wordlist size and target responsiveness. For example.com, it should be relatively quick as it's a simple target.

/usr/bin/gobuster: line 1: syntax error near unexpected token `newline'
/usr/bin/gobuster: line 1: `#!/bin/bash'

Note: The output above is an example. Your actual output might vary slightly.

Observe the Gobuster Traffic in the Proxy's History

In this final step, you will observe the HTTP traffic generated by Gobuster within OWASP ZAP. This is where you can analyze the requests and responses, identify patterns, and potentially discover vulnerabilities.

Switch back to the OWASP ZAP application.

1. In the left-hand panel, look for the Sites tree. You should see http://example.com listed.
2. Expand http://example.com and then http. You will see a list of HTTP methods (e.g., GET) and the paths that Gobuster attempted to access (e.g., /, /index.html, /robots.txt, etc.).
3. Click on any of these entries. In the lower panels, you will see the Request and Response tabs, showing the full HTTP request sent by Gobuster and the corresponding response from the server.

This confirms that Gobuster's traffic was successfully routed through the proxy, allowing for detailed inspection. This capability is essential for advanced web application testing.

Summary

In this lab, you successfully learned how to route Gobuster traffic through a web proxy. You started OWASP ZAP, confirmed its proxy configuration, used the -p flag to direct Gobuster's requests through the proxy, executed a scan, and finally observed the intercepted traffic within ZAP. This skill is fundamental for any web penetration tester, enabling deeper analysis and manipulation of automated tool traffic. By understanding how to proxy your tools, you gain greater control and visibility over your testing process, leading to more effective vulnerability discovery.