Gobuster Basics for Penetration Testing

   ______      __               __           
  / ____/___  / /_  __  _______/ /____  _____
 / / __/ __ \/ __ \/ / / / ___/ __/ _ \/ ___/
/ /_/ / /_/ / /_/ / /_/ (__  ) /_/  __/ /    
\____/\____/_.___/\__,_/____/\__/\___/_/     
                                             

Gobuster is a open-source command-line tool for web content discovery and vulnerability assessment. Security professionals and ethical hackers utilize Gobuster for reconnaissance, efficiently brute-forcing directories, files, DNS subdomains, and virtual hosts on web servers. This powerful enumeration tool, written in Go, helps uncover hidden resources, misconfigurations, and potential attack vectors during penetration testing.

By leveraging custom wordlists and flexible operational modes, Gobuster allows for targeted and efficient website scanning and subdomain enumeration. It's an indispensable hacking tool for identifying overlooked administration panels, sensitive files, and forgotten development environments, significantly enhancing the intelligence-gathering phase of any security audit or bug bounty hunting. 

Let's start off by learning about directory enumeration using Gobuster, which can help us discover hidden directories and files within a web server. These files can be vital to us, and may contain backup files, configuration files, temporary files, and even something as sensitive as a password list in a text file. Let's see an example command below and break down its structure:

gobuster dir -u https://examplesite.com -w /usr/share/dirb/wordlists/common.txt 

We can use the "dir" argument to let Gobuster know we want to scan for directories, the "-u" option to mark that we want to scan the URL that follows in the command, and the "-w" option to mark that we want to use a wordlist, followed by the path to the wordlist we want to use.

Wordlists can be downloaded off the web but most versions of Kali Linux come with some common wordlists usually located at /usr/share/wordlists. Gobuster's GUI older brother, Dirbuster, comes installed with it's own wordlists which you can utilize with Gobuster as well. I also highly suggest you check out SecLists on GitHub. It's listed on my "Resources:" link list for a reason.

Results of a directory scan using Gobuster will look like the following:

===============================================================
Gobuster v3.6
by OJ Reeves (@TheColonial) & Christian Mehlmauer (@firefart)
===============================================================
[+] Url:                     https://examplesite.com/
[+] Method:                  GET
[+] Threads:                 50
[+] Wordlist:                /usr/share/dirb/wordlists/common.txt
[+] Negative Status codes:   404
[+] User Agent:              gobuster/3.6
[+] Extensions:              php,txt
[+] Timeout:                 10s
===============================================================
Starting gobuster in directory enumeration mode
===============================================================
/index.php            (Status: 200) [Size: 6690]
/help.php             (Status: 200) [Size: 6689]
/themes               (Status: 301) [Size: 0] [--> https://examplesite.com/themes/]
/stats.php            (Status: 200) [Size: 6690]
/css                  (Status: 301) [Size: 0] [--> https://examplesite.com/css/]
/edit.php             (Status: 200) [Size: 6689]
/includes             (Status: 301) [Size: 0] [--> https://examplesite.com/includes/]
/license.php          (Status: 200) [Size: 6692]
/system.php           (Status: 200) [Size: 6691]
/status.php           (Status: 200) [Size: 6691]
/javascript           (Status: 301) [Size: 0] [--> https://examplesite.com/javascript/]
/changelog.txt        (Status: 200) [Size: 271]
/classes              (Status: 301) [Size: 0] [--> https://examplesite.com/classes/]
/exec.php             (Status: 200) [Size: 6689]
/widgets              (Status: 301) [Size: 0] [--> https://examplesite.com/widgets/]
/graph.php            (Status: 200) [Size: 6690]
/tree                 (Status: 301) [Size: 0] [--> https://examplesite.com/tree/]
/wizard.php           (Status: 200) [Size: 6691]
/shortcuts            (Status: 301) [Size: 0] [--> https://examplesite.com/shortcuts/]
/pkg.php              (Status: 200) [Size: 6688]
/installer            (Status: 301) [Size: 0] [--> https://examplesite.com/installer/]
/wizards              (Status: 301) [Size: 0] [--> https://examplesite.com/wizards/]
/xmlrpc.php           (Status: 200) [Size: 384]
/reboot.php           (Status: 200) [Size: 6691]
/interfaces.php       (Status: 200) [Size: 6695]
/csrf                 (Status: 301) [Size: 0] [--> https://examplesite.com/csrf/]
/system-users.txt     (Status: 200) [Size: 106]
/filebrowser          (Status: 301) [Size: 0] [--> https://examplesite.com/filebrowser/]

We receive a list of directories and files on the web server. These discoveries are flagged with HTTP status codes that signify whether the directory or file has been successfully found. Status codes of 200 mean the directory or file has been found while 301 means it has been moved and will be followed with the correct URL of the moved directory. Other common HTTP status codes include 4xx (Client Error), 5xx (Server Error), and 1xx (Informational).

Using this information, we can download discovered files by using wget:

wget http://examplesite.com/path/to/discovered_file.ext

We can also continue using Gobuster on discovered directories to do some deeper digging i.e. https://examplesite.com/filebrowser/.

Next we will learn about scanning for subdomains. Scanning for subdomains expands the attack surface by uncovering hidden or less-secured subdomains, which can host vulnerable applications or services often missed during initial recon. By brute-forcing common prefixes, we can find subdomains not found in public sources, aiding in the identification of potential attack targets and providing a more comprehensive understanding of a target. We can scan for subdomains by simply swapping in the "dns" argument into our command:

gobuster dns -d examplesite.com -w /usr/share/dirb/wordlists/common.txt 

We have added the "-d" option to our command to mark the target domain. Notice we removed the "https://" from our domain. We do not need to specify the protocol when executing a subdomain scan. Following the target domain we will use the "-w" option followed by the path to our wordlist. There are many common subdomain wordlists that can be found on the web which will help with acquiring better results. Below is an example of subdomain scan results:

===============================================================
Gobuster v3.6
by OJ Reeves (@TheColonial) & Christian Mehlmauer (@firefart)
===============================================================
[+] Domain:     examplesite.com
[+] Threads:    10
[+] Timeout:    1s
[+] Wordlist:   /usr/share/dirb/wordlists/common.txt
===============================================================
2025/07/17 16:20:35 Starting gobuster in DNS enumeration mode
===============================================================
Found: www.examplesite.com
Found: nagios.examplesite.com
Found: admin.examplesite.com
Found: about.examplesite.com
Found: dev.examplesite.com   
Found: auto.examplesite.com  
Found: login.examplesite.com

Similar to scanning for subdomains, Gobuster has the ability to scan for virtual hosts. Virtual host (vhost) scanning is a technique used to detect additional websites that reside on the same physical server, but are accessible through different virtual host configurations or domain names. To find vhosts, Gobuster sends HTTP requests to the target IP, changing the 'Host' header in each with names from a wordlist. If a virtual host exists, the web server replies with its content. By performing vhost scanning, you can find unlisted websites or applications that reside on the same server as a familiar site, even if they're not publicly discoverable through DNS.

We can scan for vhosts by using the following command:

gobuster vhost -u examplesite.com -w /usr/share/dirb/wordlists/common.txt 

Results of the scan will appear similar to the following: 

===============================================================
Gobuster v3.6
by OJ Reeves (@TheColonial) & Christian Mehlmauer (@firefart)
===============================================================
[+] Url:          examplesite.com
[+] Threads:      10
[+] Wordlist:     /usr/share/dirb/wordlists/common.txt 
[+] User Agent:   gobuster/3.6
[+] Timeout:      10s
===============================================================
2025/07/17 8:11:38 Starting gobuster in VHOST enumeration mode
===============================================================
Found: auto.examplesite.com (Status: 200) [Size: 162]
Found: mail.examplesite.com (Status: 200) [Size: 162]
Found: beta.examplesite.com (Status: 200) [Size: 162]
Found: api.examplesite.com (Status: 200) [Size: 162]
Found: apache.examplesite.com (Status: 200) [Size: 162]

Gobuster has many more options that can be added to our basic scans. For example, we can use the "-d" flag to set a delay in our scanning process which is beneficial for stability, performance, and stealth. Or the "-x" flag which is used for filtering file extensions during the scan. For instance, if you want to specifically search for images, you might use the following command:

gobuster dir -u https://examplesite.com -w /usr/share/dirb/wordlists/common.txt -x jpg,jpeg,png,gif,ico

Be sure to check out additional flags by using "--help".

Nmap Basics for Penetration Testing

          ___.-------.___
      _.-' ___.--;--.___ `-._
   .-' _.-'  /  .+.  \  `-._ `-.
 .' .-'      |-|-o-|-|      `-. `.
(_ <O__      \  `+'  /      __O> _)
  `--._``-..__`._|_.'__..-''_.--'
        ``--._________.--''

Nmap, short for Network Mapper, is a free and open-source tool widely used for network discovery and security auditing. Nmap utilizes raw IP packets in innovative ways to gather information about a network. It can identify available hosts, running services (including application name and version), operating systems, firewalls or packet filters in use, and many other details. Designed for rapid scanning of large networks, Nmap also works effectively for single hosts.

Network administrators rely on Nmap for various tasks, including creating network inventories, managing service upgrades, and monitoring uptime of hosts and services, while hackers may utilize it for enumeration i.e. identifying open ports running services that may be exploitable.

Nmap is available for most operating systems but is included with the Kali Linux distro. Remember to replace the placeholder ip address with your target ip address. You can also test this out by scanning the url: scanme.nmap.org. Let's start off with a basic Nmap scan. We can scan using an ip address or a host name i.e. scanme.nmap.org:

nmap 192.168.1.1

We can also scan a list of network/host targets within a text file using the following commands where "ip_list.txt" is the path to the text file.

nmap -iL ip_list.txt 

When performing a penetration test on a network, we may want to perform a ping sweep scan, which pings all available hosts on a network by sending ICMP packets and returns the live hosts. Conducting a ping sweep is a crucial part of identifying active hosts on a network and lays the groundwork for a penetration test.

nmap -sn 192.168.0.0/24

Once we have the list of live hosts, we can then do port scans on the individual hosts. Note the "/24" addition to the ip address. This sets the ip address range. It will send an ICMP echo request to every ip address in the network from 192.168.0.1 to 192.168.0.255.

Nmap defaults to scanning the 1000 most commonly used ports. Port specification and scan order refer to the process of selecting which ports to scan during network reconnaissance. This is crucial for efficient scanning as it allows you to exclude irrelevant ports and prioritize the scan based on port usage frequency.

We can specify ports using the following command:

nmap -p 22,80,443 192.168.1.1

We can also scan for the operating system. This information helps pinpoint vulnerabilities specific to the operating system, enabling more effective attacks on the target system. We can use "-O" to enable OS detection.

nmap -O 192.168.1.1

What if we want to detect the services being run on open ports on the network? The service/version detection feature in Nmap provides valuable insights into the target system, enabling you to identify vulnerabilities and weaknesses on those ports. This option can be enabled by using "-sV".

nmap -sV 192.168.1.1

With the returned information, we can run a query using something like  searchsploit for possible exploits for these services.

One of the most popular scans used is the SYN scan. A SYN scan involves sending an SYN packet to the target host and monitoring for a response. If the target responds with an SYN/ACK packet, the port is open; if it responds with an RST packet, the port is closed. This half-open scan technique, which doesn't complete the full TCP three-way handshake, is fast and, most importantly, stealthy, ideal for mapping large networks.

nmap -sS 192.168.1.1

Being stealthy is an important part of penetration testing. Some firewalls and Intrusion Detection System (IDS) solutions may temporarily block ip addresses that exhibit unusual network activity, such as high traffic volumes or sending network packets to multiple hosts in a systematic manner. We can mitigate these risks by using certain commands that effect the scans order and timing.

We can add the "-T" option to effect the scans timing template. By default, Nmap uses the "-T2" timing template but we can slow the scan down by using the "-T1" or "-T0" timing templates. These will slow down the scan process but will minimize the impact on the network and prevent triggering any alerts.

nmap -T1 192.168.0.0/24

You can also slow down an Nmap scan using delay. Use the "--scan-delay" option and specify the desired delay time in seconds.

nmap --scan-delay 3s 192.168.0.0/24

Let's change the order of targets we scan. Nmap's --randomize-hosts option can help you randomize your scans, making them less predictable and harder to detect by security.

nmap --randomize-hosts 192.168.0.0/24

One of the most powerful features of Nmap is it's ability to run scripts. The Nmap Script Engine (NSE) are scripts written in the programming language, Lua.

For example, you can run the following command that runs the "http-headers" script which pulls the HTTP headers configured on the target webserver.

nmap --script http-headers scanme.nmap.org

As a penetration tester, you may want to run the "vulners" script which automatically lists the vulnerabilities on a target using a CVE database.

nmap --script vulners 192.168.1.1

There are hundreds of NSE scripts available which you can view here.