Gobuster Basics for Penetration Testing

   ______      __               __           
  / ____/___  / /_  __  _______/ /____  _____
 / / __/ __ \/ __ \/ / / / ___/ __/ _ \/ ___/
/ /_/ / /_/ / /_/ / /_/ (__  ) /_/  __/ /    
\____/\____/_.___/\__,_/____/\__/\___/_/     
                                             

Gobuster is a open-source command-line tool for web content discovery and vulnerability assessment. Security professionals and ethical hackers utilize Gobuster for reconnaissance, efficiently brute-forcing directories, files, DNS subdomains, and virtual hosts on web servers. This powerful enumeration tool, written in Go, helps uncover hidden resources, misconfigurations, and potential attack vectors during penetration testing.

By leveraging custom wordlists and flexible operational modes, Gobuster allows for targeted and efficient website scanning and subdomain enumeration. It's an indispensable hacking tool for identifying overlooked administration panels, sensitive files, and forgotten development environments, significantly enhancing the intelligence-gathering phase of any security audit or bug bounty hunting. 

Let's start off by learning about directory enumeration using Gobuster, which can help us discover hidden directories and files within a web server. These files can be vital to us, and may contain backup files, configuration files, temporary files, and even something as sensitive as a password list in a text file. Let's see an example command below and break down its structure:

gobuster dir -u https://examplesite.com -w /usr/share/dirb/wordlists/common.txt 

We can use the "dir" argument to let Gobuster know we want to scan for directories, the "-u" option to mark that we want to scan the URL that follows in the command, and the "-w" option to mark that we want to use a wordlist, followed by the path to the wordlist we want to use.

Wordlists can be downloaded off the web but most versions of Kali Linux come with some common wordlists usually located at /usr/share/wordlists. Gobuster's GUI older brother, Dirbuster, comes installed with it's own wordlists which you can utilize with Gobuster as well. I also highly suggest you check out SecLists on GitHub. It's listed on my "Resources:" link list for a reason.

Results of a directory scan using Gobuster will look like the following:

===============================================================
Gobuster v3.6
by OJ Reeves (@TheColonial) & Christian Mehlmauer (@firefart)
===============================================================
[+] Url:                     https://examplesite.com/
[+] Method:                  GET
[+] Threads:                 50
[+] Wordlist:                /usr/share/dirb/wordlists/common.txt
[+] Negative Status codes:   404
[+] User Agent:              gobuster/3.6
[+] Extensions:              php,txt
[+] Timeout:                 10s
===============================================================
Starting gobuster in directory enumeration mode
===============================================================
/index.php            (Status: 200) [Size: 6690]
/help.php             (Status: 200) [Size: 6689]
/themes               (Status: 301) [Size: 0] [--> https://examplesite.com/themes/]
/stats.php            (Status: 200) [Size: 6690]
/css                  (Status: 301) [Size: 0] [--> https://examplesite.com/css/]
/edit.php             (Status: 200) [Size: 6689]
/includes             (Status: 301) [Size: 0] [--> https://examplesite.com/includes/]
/license.php          (Status: 200) [Size: 6692]
/system.php           (Status: 200) [Size: 6691]
/status.php           (Status: 200) [Size: 6691]
/javascript           (Status: 301) [Size: 0] [--> https://examplesite.com/javascript/]
/changelog.txt        (Status: 200) [Size: 271]
/classes              (Status: 301) [Size: 0] [--> https://examplesite.com/classes/]
/exec.php             (Status: 200) [Size: 6689]
/widgets              (Status: 301) [Size: 0] [--> https://examplesite.com/widgets/]
/graph.php            (Status: 200) [Size: 6690]
/tree                 (Status: 301) [Size: 0] [--> https://examplesite.com/tree/]
/wizard.php           (Status: 200) [Size: 6691]
/shortcuts            (Status: 301) [Size: 0] [--> https://examplesite.com/shortcuts/]
/pkg.php              (Status: 200) [Size: 6688]
/installer            (Status: 301) [Size: 0] [--> https://examplesite.com/installer/]
/wizards              (Status: 301) [Size: 0] [--> https://examplesite.com/wizards/]
/xmlrpc.php           (Status: 200) [Size: 384]
/reboot.php           (Status: 200) [Size: 6691]
/interfaces.php       (Status: 200) [Size: 6695]
/csrf                 (Status: 301) [Size: 0] [--> https://examplesite.com/csrf/]
/system-users.txt     (Status: 200) [Size: 106]
/filebrowser          (Status: 301) [Size: 0] [--> https://examplesite.com/filebrowser/]

We receive a list of directories and files on the web server. These discoveries are flagged with HTTP status codes that signify whether the directory or file has been successfully found. Status codes of 200 mean the directory or file has been found while 301 means it has been moved and will be followed with the correct URL of the moved directory. Other common HTTP status codes include 4xx (Client Error), 5xx (Server Error), and 1xx (Informational).

Using this information, we can download discovered files by using wget:

wget http://examplesite.com/path/to/discovered_file.ext

We can also continue using Gobuster on discovered directories to do some deeper digging i.e. https://examplesite.com/filebrowser/.

Next we will learn about scanning for subdomains. Scanning for subdomains expands the attack surface by uncovering hidden or less-secured subdomains, which can host vulnerable applications or services often missed during initial recon. By brute-forcing common prefixes, we can find subdomains not found in public sources, aiding in the identification of potential attack targets and providing a more comprehensive understanding of a target. We can scan for subdomains by simply swapping in the "dns" argument into our command:

gobuster dns -d examplesite.com -w /usr/share/dirb/wordlists/common.txt 

We have added the "-d" option to our command to mark the target domain. Notice we removed the "https://" from our domain. We do not need to specify the protocol when executing a subdomain scan. Following the target domain we will use the "-w" option followed by the path to our wordlist. There are many common subdomain wordlists that can be found on the web which will help with acquiring better results. Below is an example of subdomain scan results:

===============================================================
Gobuster v3.6
by OJ Reeves (@TheColonial) & Christian Mehlmauer (@firefart)
===============================================================
[+] Domain:     examplesite.com
[+] Threads:    10
[+] Timeout:    1s
[+] Wordlist:   /usr/share/dirb/wordlists/common.txt
===============================================================
2025/07/17 16:20:35 Starting gobuster in DNS enumeration mode
===============================================================
Found: www.examplesite.com
Found: nagios.examplesite.com
Found: admin.examplesite.com
Found: about.examplesite.com
Found: dev.examplesite.com   
Found: auto.examplesite.com  
Found: login.examplesite.com

Similar to scanning for subdomains, Gobuster has the ability to scan for virtual hosts. Virtual host (vhost) scanning is a technique used to detect additional websites that reside on the same physical server, but are accessible through different virtual host configurations or domain names. To find vhosts, Gobuster sends HTTP requests to the target IP, changing the 'Host' header in each with names from a wordlist. If a virtual host exists, the web server replies with its content. By performing vhost scanning, you can find unlisted websites or applications that reside on the same server as a familiar site, even if they're not publicly discoverable through DNS.

We can scan for vhosts by using the following command:

gobuster vhost -u examplesite.com -w /usr/share/dirb/wordlists/common.txt 

Results of the scan will appear similar to the following: 

===============================================================
Gobuster v3.6
by OJ Reeves (@TheColonial) & Christian Mehlmauer (@firefart)
===============================================================
[+] Url:          examplesite.com
[+] Threads:      10
[+] Wordlist:     /usr/share/dirb/wordlists/common.txt 
[+] User Agent:   gobuster/3.6
[+] Timeout:      10s
===============================================================
2025/07/17 8:11:38 Starting gobuster in VHOST enumeration mode
===============================================================
Found: auto.examplesite.com (Status: 200) [Size: 162]
Found: mail.examplesite.com (Status: 200) [Size: 162]
Found: beta.examplesite.com (Status: 200) [Size: 162]
Found: api.examplesite.com (Status: 200) [Size: 162]
Found: apache.examplesite.com (Status: 200) [Size: 162]

Gobuster has many more options that can be added to our basic scans. For example, we can use the "-d" flag to set a delay in our scanning process which is beneficial for stability, performance, and stealth. Or the "-x" flag which is used for filtering file extensions during the scan. For instance, if you want to specifically search for images, you might use the following command:

gobuster dir -u https://examplesite.com -w /usr/share/dirb/wordlists/common.txt -x jpg,jpeg,png,gif,ico

Be sure to check out additional flags by using "--help".

Create a Honeypot Using T-Pot

  ,,     ,,     ,,
  xx    _xx_   ,xx,
 /==\   /==\   /==\
(/==\) (/==\) (/==\)
  \/     \/     \/

Honeypots are virtual traps to lure threat actors. An intentionally compromised system allows attackers to exploit vulnerabilities so you can study them to improve security policies. Setting up your own honeypot is a quick and easy project especially when utilizing T-Pot. T-Pot is a free, all-in-one multi honeypot platform that uses 20+ honeypots on various ports and features countless visualization options to display data obtained from attackers.

We will be creating our honeypot on a cloud server hosted through Vultr. It is not suggested to host honeypots using your home internet. You can sign up for a Vultr account using my referral link here. This will give you $300 in credits which will allow you to host your honeypot for free for around 6 months when following this tutorial.

After creating your Vultr account, navigate to the top right of the dashboard and click the "Deploy +" button. You will be directed to the server deployment screen. For the type, select "Cloud Computed - Shared CPU". For the server location, select a region closest to your home. It is worth noting that selecting other regions may produce different interesting results as far as the number of threat actors attempting to access your server and their location. Feel free to experiment. When asked to select an image for your server, click "Ubuntu" and select the most recent version. When selecting a plan, click the "Regular Cloud Compute" option and select the 160GB SSD option. This option gives us the base requirements for running T-Pot. You may turn off all additional features for your server including Auto Backups and IPv6. You will then be asked to name your server under "Server Hostname & Label". Once done, click "Deploy Now".

On your Compute dashboard, once the status of the server is listed as "Running", click the server name to go to the server dashboard. Click the top right "View Console" icon to launch the console for your server where you will be greeted with the server login screen. Use the Username and Password credentials listed on the server dashboard. This will give you root access to your server.

T-Pot requires you to install the service using a non-root user. We will create a new user using the following command where "*username*" is your desired username:

adduser *username*

You will be prompted through the user creation process where you will set a password and other information regarding the user. Once the user is created you will need to give them sudo privileges using the following command:

usermod -aG sudo *username*

Once given sudo permission to the new user, switch to that user by entering the following command:

su *username*

You will now switch to your new user. If you have been following this tutorial you will be in the root folder and will need to enter the following command to move back one directory:

cd ..

Change directories using the command below where "*username*" is the name of your user:

cd home/*username*

This directory is where we will clone the T-Pot Github repository. We will first need to install Git by entering the following command:

sudo apt install git

Clone the Github repository using the command below:

git clone https://github.com/telekom-security/tpotce

Change into the tpotce/ directory:

cd tpotce

Run the installer:

./install.sh

Follow the T-Pot installer prompts. Installation will take a few minutes. When asked which type of T-Pot you would like to use, select Hive by typing "h" into the command prompt.

You will be asked to setup the web user information. This information will be used to log into the T-Pot web dashboard/command center where you will be able to access the collected data. Complete the prompt by entering in a username and password.

You will eventually be told the installation is complete and you may reboot your server. Reboot by using the following command:

sudo reboot

Once your server has been rebooted, you may close out of the Vultr console. Congratulations, your honeypot is up and running!

To access the web dashboard/command center for your new honeypot enter https://<your.ip>:64297 into your web browser where "<your.ip>" is the ip address of your server, which can be found on your server dashboard on Vultr. You will be prompted to log in. Use the username and password created during the installation of T-Pot for the web dashboard. You may receive a warning from your browser. You can ignore this. Once logged in, you will be greeted with your T-Pot web dashboard:

One of the most popular tools in T-Pot is the real-time Attack Map. This map displays attacks in real-time on your server. The interface displays the world map along with information on each attack including the service the attacker is utilizing and the location/ip address of the attacker.

The Kibana tool on T-Pot is a great way to see more in-depth details about the attacks being done on your server. When accessing Kibana, you will be asked to select a dashboard. Dashboards are broken up into the honeypot type. For example, selecting the Cowrie dashboard displays attacks using ssh/telnet on ports 22 and 23. 

To see the complete list of honeypots on T-Pot and their ports, click here. The T-Pot Github also has links to all the honeypots under Technical Concept where you can learn more about each one.