Introduction
In web application penetration testing or security auditing, it's common to encounter areas of a website that are only accessible after a user has authenticated. Standard directory scanning tools like Gobuster might miss these hidden paths because they don't maintain a session. This lab will guide you through the process of using Gobuster's cookie functionality to perform authenticated directory scans. You will learn how to obtain a session cookie from a web application and then use that cookie with Gobuster to discover directories and files within authenticated sections, providing a more comprehensive view of the application's attack surface.
Log into a Web Application and Obtain a Session Cookie
In this step, you will simulate logging into a web application and capture the session cookie that is set upon successful authentication. This cookie is crucial for Gobuster to access authenticated areas. We will use curl to perform the login and extract the cookie.
First, let's try to access the dashboard without authentication to see that it redirects us:
curl -v http://localhost:8080/dashboard
You should see a 302 Found redirecting you back to /. Now, let's log in. The web application has a simple login form at http://localhost:8080/ with username user and password password. We will send a POST request to /login with these credentials.
curl -v -X POST -d "username=user&password=password" http://localhost:8080/login
In the output, look for the Set-Cookie header. It should look something like Set-Cookie: session=authenticated_session_id_12345; Path=/. The value authenticated_session_id_12345 is your session cookie. Make a note of this value.
Now, let's try to access the dashboard using the obtained cookie. Replace YOUR_COOKIE_VALUE with the actual cookie value you found.
curl -v --cookie "session=authenticated_session_id_12345" http://localhost:8080/dashboard
You should now see the content of the dashboard page, indicating successful authenticated access.
Construct a gobuster dir Command for an Authenticated Area
In this step, you will prepare the basic gobuster dir command. We will target the authenticated area of the web application. The base URL for our authenticated scan will be http://localhost:8080/authenticated/. We will use a common wordlist for directory brute-forcing.
First, let's ensure Gobuster is installed.
gobuster version
You should see the version information. If not, please refer to the setup section.
Now, let's construct the basic gobuster dir command. We'll use the -u flag for the URL and -w for the wordlist. For this lab, we'll use a small, built-in wordlist for demonstration purposes, or you can specify a common one like common.txt if available on your system. If common.txt is not found, you can create a small custom wordlist for testing.
Let's create a small wordlist for this lab:
echo -e "secret_dir\nadmin\nconfig\nbackup\nusers" > ~/project/wordlist.txt
Now, the basic command without the cookie would look like this:
gobuster dir -u http://localhost:8080/authenticated/ -w ~/project/wordlist.txt
If you run this command now, it will likely not find the secret_dir because it requires authentication. The next step will add the cookie to this command.
Use the -c Flag to Provide the Session Cookie
In this step, you will integrate the session cookie you obtained in Step 1 into your Gobuster command using the -c flag. This flag allows Gobuster to include the specified cookie in its requests, enabling it to access authenticated areas.
Recall the cookie value you obtained in Step 1, which was session=authenticated_session_id_12345.
The -c flag expects the cookie in the format key=value. So, our cookie string will be "session=authenticated_session_id_12345".
Now, combine this with the command from Step 2:
gobuster dir -u http://localhost:8080/authenticated/ -w ~/project/wordlist.txt -c "session=authenticated_session_id_12345"
This command tells Gobuster to perform a directory scan on http://localhost:8080/authenticated/, using the provided wordlist, and crucially, including the session=authenticated_session_id_12345 cookie in every request. This will allow Gobuster to bypass the authentication barrier and discover resources within the protected area.
Before executing, double-check that your cookie value is correct.
Execute the Scan
Now that you have constructed the full Gobuster command, it's time to execute it and observe the results. This scan will attempt to find directories and files within the authenticated section of the web application.
Execute the command you prepared in the previous step:
gobuster dir -u http://localhost:8080/authenticated/ -w ~/project/wordlist.txt -c "session=authenticated_session_id_12345"
Gobuster will start the scan and display its progress. Pay close attention to the output. You should see entries indicating directories or files found with a Status: 200 (OK) or Status: 301 (Moved Permanently) or Status: 302 (Found), which means the resource was successfully accessed or redirected.
Example output:
===============================================================
Gobuster vX.X.X
===============================================================
[+] Url: http://localhost:8080/authenticated/
[+] Wordlist: /home/labex/project/wordlist.txt
[+] Threads: 10
[+] Timeout: 10s
[+] User Agent: gobuster/X.X.X
[+] Cookies: session=authenticated_session_id_12345
===============================================================
2024/01/01 12:00:00 Starting gobuster in directory enumeration mode
===============================================================
/secret_dir (Status: 200) [Size: 100]
===============================================================
2024/01/01 12:00:05 Finished
===============================================================
Notice how /secret_dir is found with a Status: 200. This indicates that Gobuster successfully accessed this directory because it was sending the authentication cookie. Without the cookie, this directory would likely not be found or would return a redirect/unauthorized status.
Analyze the Results to Find Pages Accessible Only When Authenticated
In this final step, you will analyze the output from your Gobuster scan to identify resources that were successfully discovered within the authenticated area. The key is to look for entries that returned a Status: 200 (OK) or other success codes, which would typically be inaccessible without the session cookie.
From the output of the previous step, you should have seen something similar to:
/secret_dir (Status: 200) [Size: 100]
This line indicates that Gobuster successfully found the /secret_dir path within the http://localhost:8080/authenticated/ URL, and it returned an HTTP 200 OK status. This is a strong indicator that this directory is accessible when authenticated.
To confirm, you can try to access this path directly using curl without the cookie:
curl http://localhost:8080/authenticated/secret_dir
You should be redirected to the login page or receive an unauthorized message.
Now, try accessing it with the cookie:
curl --cookie "session=authenticated_session_id_12345" http://localhost:8080/authenticated/secret_dir/hidden_file.html
You should see the content of hidden_file.html, confirming that this path is indeed accessible only with the correct authentication cookie.
This process demonstrates how using cookies with Gobuster can reveal hidden parts of a web application that are only visible to authenticated users, significantly expanding the scope of your security assessment.
Summary
In this lab, you successfully learned how to perform authenticated directory scans using Gobuster. You started by logging into a simulated web application and extracting a session cookie. Then, you constructed a Gobuster command that included this cookie using the -c flag. By executing this command, you were able to discover a hidden directory (/secret_dir) that was only accessible to authenticated users. This technique is invaluable for security professionals and developers who need to thoroughly audit web applications, ensuring that all accessible paths, even those behind authentication, are properly identified and secured.