Introduction
In this lab, you will explore how to manage the scan speed of Gobuster, a popular directory and file brute-forcing tool, by manipulating its thread count. Understanding how to adjust the number of threads is crucial for efficient scanning and for minimizing the impact on target servers. You will learn to run scans with default, increased, and decreased thread counts, observing the practical implications of each setting. This hands-on experience will provide insights into optimizing your reconnaissance efforts while being mindful of network etiquette and target stability.
Run a Scan with the Default Thread Count (10)
In this step, you will initiate a Gobuster scan using its default thread count, which is typically 10. This will give you a baseline understanding of its performance and the output format. Gobuster will attempt to discover directories and files on a local web server that has been set up for this lab.
Open your terminal and execute the following command:
gobuster dir -u http://127.0.0.1:8080 -w /tmp/wordlist.txt
gobuster dir: Specifies that you want to perform a directory/file brute-forcing scan.-u http://127.0.0.1:8080: Sets the target URL for the scan. In this lab, we are targeting a local HTTP server running on port 8080.-w /tmp/wordlist.txt: Specifies the wordlist to use for the brute-forcing process. This file contains common directory and file names.
Observe the output. You will see Gobuster starting the scan, indicating the number of threads being used (defaulting to 10), and then listing any discovered directories or files. Pay attention to how quickly the results appear.
===============================================================
Gobuster vX.X.X-XXXXXX
===============================================================
[+] Url: http://127.0.0.1:8080
[+] Threads: 10
[+] Wordlist: /tmp/wordlist.txt
[+] Status codes: 200,204,301,302,307,401,403
[+] User Agent: gobuster/X.X.X
[+] Timeout: 10s
===============================================================
XXXX/XX/XX XX:XX:XX Starting gobuster in directory enumeration mode
===============================================================
/admin (Status: 200)
/login (Status: 200)
/secret (Status: 200)
/index (Status: 200)
===============================================================
XXXX/XX/XX XX:XX:XX Finished
===============================================================
The output shows the configuration used for the scan, including the URL, wordlist, and importantly, the number of threads. The discovered paths like /admin, /login, /secret, and /index are listed with their respective HTTP status codes.
Increase the Thread Count to 50 with the -t Flag
In this step, you will increase the number of threads used by Gobuster to 50. This is done using the -t flag, which allows you to specify the desired number of concurrent threads. A higher thread count generally leads to a faster scan, as more requests are sent simultaneously.
Execute the following command in your terminal:
gobuster dir -u http://127.0.0.1:8080 -w /tmp/wordlist.txt -t 50
-t 50: This new flag sets the number of threads to 50.
Observe the output again. You should notice that the scan completes much faster than with the default 10 threads. The initial output will confirm that 50 threads are being used.
===============================================================
Gobuster vX.X.X-XXXXXX
===============================================================
[+] Url: http://127.0.0.1:8080
[+] Threads: 50
[+] Wordlist: /tmp/wordlist.txt
[+] Status codes: 200,204,301,302,307,401,403
[+] User Agent: gobuster/X.X.X
[+] Timeout: 10s
===============================================================
XXXX/XX/XX XX:XX:XX Starting gobuster in directory enumeration mode
===============================================================
/admin (Status: 200)
/login (Status: 200)
/secret (Status: 200)
/index (Status: 200)
===============================================================
XXXX/XX/XX XX:XX:XX Finished
===============================================================
Notice the [+] Threads: 50 line in the output, confirming that your specified thread count is being used. The scan should complete almost instantly due to the small wordlist and local target.
Observe the Increase in Scan Speed
In this step, we will explicitly compare the scan times to visually confirm the impact of increasing the thread count. While the previous steps provided a qualitative observation, this step aims to reinforce the concept.
Although the previous scans were very fast due to the small wordlist and local server, you should have noticed a significant difference in the time it took for the output to appear. When you increased the threads from 10 to 50, the scan completed almost instantaneously.
To better illustrate the speed difference, imagine if the wordlist contained thousands or millions of entries, and the target server was remote with higher latency. In such scenarios, the difference between 10 and 50 threads would be dramatically more apparent, potentially reducing scan time from hours to minutes.
The key takeaway here is that more threads allow Gobuster to send more requests concurrently, which can drastically reduce the total scan time, especially for large wordlists and responsive targets.
No specific command is required for this step, as it's an observation based on the previous two steps.
Decrease the Thread Count to 5 for a Slower Scan
Now, let's reverse the process and decrease the thread count to 5. This will demonstrate how fewer threads lead to a slower scan. While a slower scan might seem counterproductive, it can be necessary in certain situations, such as when dealing with rate-limited targets or to be less intrusive.
Execute the following command in your terminal:
gobuster dir -u http://127.0.0.1:8080 -w /tmp/wordlist.txt -t 5
-t 5: This sets the number of threads to 5, which is half of the default.
Observe the output. You should notice that the scan takes slightly longer to complete compared to the default 10 threads, and noticeably longer than the 50-thread scan.
===============================================================
Gobuster vX.X.X-XXXXXX
===============================================================
[+] Url: http://127.0.0.1:8080
[+] Threads: 5
[+] Wordlist: /tmp/wordlist.txt
[+] Status codes: 200,204,301,302,307,401,403
[+] User Agent: gobuster/X.X.X
[+] Timeout: 10s
===============================================================
XXXX/XX/XX XX:XX:XX Starting gobuster in directory enumeration mode
===============================================================
/admin (Status: 200)
/login (Status: 200)
/secret (Status: 200)
/index (Status: 200)
===============================================================
XXXX/XX/XX XX:XX:XX Finished
===============================================================
The [+] Threads: 5 line confirms the reduced thread count. Even with our small wordlist, you should perceive a slight delay compared to the previous runs. This illustrates the direct relationship between thread count and scan duration.
Understand the Impact of Threads on the Target Server
In this final step, we will discuss the broader implications of adjusting thread counts, particularly concerning the target server. While increasing threads can speed up your scan, it also increases the load on the target server.
High Thread Count (e.g., 50 or more):
- Pros: Faster scan completion.
- Cons:
- Increased Server Load: Sending many requests simultaneously can overwhelm the target server, potentially leading to denial of service (DoS) or performance degradation.
- Rate Limiting/Blocking: Many web servers and WAFs (Web Application Firewalls) implement rate limiting to prevent brute-force attacks. A high thread count can quickly trigger these defenses, leading to your IP being temporarily or permanently blocked.
- Network Congestion: Can consume significant network bandwidth, especially if the target is remote.
- Suspicion: High request rates are often indicative of malicious activity and can alert security teams.
Low Thread Count (e.g., 1-5):
- Pros:
- Reduced Server Load: Less intrusive and less likely to cause performance issues on the target.
- Evading Rate Limiting: Slower request rates are less likely to trigger rate-limiting mechanisms, allowing the scan to proceed undetected for longer.
- Stealth: Appears more like legitimate user traffic, reducing the chance of detection.
- Cons:
- Slower Scan: Takes significantly longer to complete, especially with large wordlists.
Choosing the Right Thread Count: The optimal thread count depends on several factors:
- Target Server's Capacity: How much load can the server handle?
- Network Latency: Higher latency might require more threads to maintain a decent speed.
- Rate Limiting Policies: Are there any known rate limits or WAFs in place?
- Your Goal: Do you prioritize speed or stealth?
For ethical hacking and penetration testing, it's crucial to be mindful of the target's stability and to avoid causing any disruption. Always start with a lower thread count and gradually increase it if the target can handle it and your goals require faster results.
No specific command is required for this step, as it's a conceptual understanding.
Summary
In this lab, you have successfully learned how to control the scan speed of Gobuster by adjusting the number of threads using the -t flag. You started by running a scan with the default 10 threads, then increased it to 50 to observe a significant speed improvement, and finally decreased it to 5 to see the scan slow down.
More importantly, you gained an understanding of the critical impact that thread count has not only on scan duration but also on the target server's performance and security mechanisms. You now know that while more threads mean faster scans, they also increase server load and the risk of detection or blocking. Conversely, fewer threads offer stealth and reduce server impact at the cost of speed. This knowledge is vital for conducting responsible and effective reconnaissance.