Gobuster Basics for Penetration Testing

   ______      __               __           
  / ____/___  / /_  __  _______/ /____  _____
 / / __/ __ \/ __ \/ / / / ___/ __/ _ \/ ___/
/ /_/ / /_/ / /_/ / /_/ (__  ) /_/  __/ /    
\____/\____/_.___/\__,_/____/\__/\___/_/     
                                             

Gobuster is a open-source command-line tool for web content discovery and vulnerability assessment. Security professionals and ethical hackers utilize Gobuster for reconnaissance, efficiently brute-forcing directories, files, DNS subdomains, and virtual hosts on web servers. This powerful enumeration tool, written in Go, helps uncover hidden resources, misconfigurations, and potential attack vectors during penetration testing.

By leveraging custom wordlists and flexible operational modes, Gobuster allows for targeted and efficient website scanning and subdomain enumeration. It's an indispensable hacking tool for identifying overlooked administration panels, sensitive files, and forgotten development environments, significantly enhancing the intelligence-gathering phase of any security audit or bug bounty hunting. 

Let's start off by learning about directory enumeration using Gobuster, which can help us discover hidden directories and files within a web server. These files can be vital to us, and may contain backup files, configuration files, temporary files, and even something as sensitive as a password list in a text file. Let's see an example command below and break down its structure:

gobuster dir -u https://examplesite.com -w /usr/share/dirb/wordlists/common.txt 

We can use the "dir" argument to let Gobuster know we want to scan for directories, the "-u" option to mark that we want to scan the URL that follows in the command, and the "-w" option to mark that we want to use a wordlist, followed by the path to the wordlist we want to use.

Wordlists can be downloaded off the web but most versions of Kali Linux come with some common wordlists usually located at /usr/share/wordlists. Gobuster's GUI older brother, Dirbuster, comes installed with it's own wordlists which you can utilize with Gobuster as well. I also highly suggest you check out SecLists on GitHub. It's listed on my "Resources:" link list for a reason.

Results of a directory scan using Gobuster will look like the following:

===============================================================
Gobuster v3.6
by OJ Reeves (@TheColonial) & Christian Mehlmauer (@firefart)
===============================================================
[+] Url:                     https://examplesite.com/
[+] Method:                  GET
[+] Threads:                 50
[+] Wordlist:                /usr/share/dirb/wordlists/common.txt
[+] Negative Status codes:   404
[+] User Agent:              gobuster/3.6
[+] Extensions:              php,txt
[+] Timeout:                 10s
===============================================================
Starting gobuster in directory enumeration mode
===============================================================
/index.php            (Status: 200) [Size: 6690]
/help.php             (Status: 200) [Size: 6689]
/themes               (Status: 301) [Size: 0] [--> https://examplesite.com/themes/]
/stats.php            (Status: 200) [Size: 6690]
/css                  (Status: 301) [Size: 0] [--> https://examplesite.com/css/]
/edit.php             (Status: 200) [Size: 6689]
/includes             (Status: 301) [Size: 0] [--> https://examplesite.com/includes/]
/license.php          (Status: 200) [Size: 6692]
/system.php           (Status: 200) [Size: 6691]
/status.php           (Status: 200) [Size: 6691]
/javascript           (Status: 301) [Size: 0] [--> https://examplesite.com/javascript/]
/changelog.txt        (Status: 200) [Size: 271]
/classes              (Status: 301) [Size: 0] [--> https://examplesite.com/classes/]
/exec.php             (Status: 200) [Size: 6689]
/widgets              (Status: 301) [Size: 0] [--> https://examplesite.com/widgets/]
/graph.php            (Status: 200) [Size: 6690]
/tree                 (Status: 301) [Size: 0] [--> https://examplesite.com/tree/]
/wizard.php           (Status: 200) [Size: 6691]
/shortcuts            (Status: 301) [Size: 0] [--> https://examplesite.com/shortcuts/]
/pkg.php              (Status: 200) [Size: 6688]
/installer            (Status: 301) [Size: 0] [--> https://examplesite.com/installer/]
/wizards              (Status: 301) [Size: 0] [--> https://examplesite.com/wizards/]
/xmlrpc.php           (Status: 200) [Size: 384]
/reboot.php           (Status: 200) [Size: 6691]
/interfaces.php       (Status: 200) [Size: 6695]
/csrf                 (Status: 301) [Size: 0] [--> https://examplesite.com/csrf/]
/system-users.txt     (Status: 200) [Size: 106]
/filebrowser          (Status: 301) [Size: 0] [--> https://examplesite.com/filebrowser/]

We receive a list of directories and files on the web server. These discoveries are flagged with HTTP status codes that signify whether the directory or file has been successfully found. Status codes of 200 mean the directory or file has been found while 301 means it has been moved and will be followed with the correct URL of the moved directory. Other common HTTP status codes include 4xx (Client Error), 5xx (Server Error), and 1xx (Informational).

Using this information, we can download discovered files by using wget:

wget http://examplesite.com/path/to/discovered_file.ext

We can also continue using Gobuster on discovered directories to do some deeper digging i.e. https://examplesite.com/filebrowser/.

Next we will learn about scanning for subdomains. Scanning for subdomains expands the attack surface by uncovering hidden or less-secured subdomains, which can host vulnerable applications or services often missed during initial recon. By brute-forcing common prefixes, we can find subdomains not found in public sources, aiding in the identification of potential attack targets and providing a more comprehensive understanding of a target. We can scan for subdomains by simply swapping in the "dns" argument into our command:

gobuster dns -d examplesite.com -w /usr/share/dirb/wordlists/common.txt 

We have added the "-d" option to our command to mark the target domain. Notice we removed the "https://" from our domain. We do not need to specify the protocol when executing a subdomain scan. Following the target domain we will use the "-w" option followed by the path to our wordlist. There are many common subdomain wordlists that can be found on the web which will help with acquiring better results. Below is an example of subdomain scan results:

===============================================================
Gobuster v3.6
by OJ Reeves (@TheColonial) & Christian Mehlmauer (@firefart)
===============================================================
[+] Domain:     examplesite.com
[+] Threads:    10
[+] Timeout:    1s
[+] Wordlist:   /usr/share/dirb/wordlists/common.txt
===============================================================
2025/07/17 16:20:35 Starting gobuster in DNS enumeration mode
===============================================================
Found: www.examplesite.com
Found: nagios.examplesite.com
Found: admin.examplesite.com
Found: about.examplesite.com
Found: dev.examplesite.com   
Found: auto.examplesite.com  
Found: login.examplesite.com

Similar to scanning for subdomains, Gobuster has the ability to scan for virtual hosts. Virtual host (vhost) scanning is a technique used to detect additional websites that reside on the same physical server, but are accessible through different virtual host configurations or domain names. To find vhosts, Gobuster sends HTTP requests to the target IP, changing the 'Host' header in each with names from a wordlist. If a virtual host exists, the web server replies with its content. By performing vhost scanning, you can find unlisted websites or applications that reside on the same server as a familiar site, even if they're not publicly discoverable through DNS.

We can scan for vhosts by using the following command:

gobuster vhost -u examplesite.com -w /usr/share/dirb/wordlists/common.txt 

Results of the scan will appear similar to the following: 

===============================================================
Gobuster v3.6
by OJ Reeves (@TheColonial) & Christian Mehlmauer (@firefart)
===============================================================
[+] Url:          examplesite.com
[+] Threads:      10
[+] Wordlist:     /usr/share/dirb/wordlists/common.txt 
[+] User Agent:   gobuster/3.6
[+] Timeout:      10s
===============================================================
2025/07/17 8:11:38 Starting gobuster in VHOST enumeration mode
===============================================================
Found: auto.examplesite.com (Status: 200) [Size: 162]
Found: mail.examplesite.com (Status: 200) [Size: 162]
Found: beta.examplesite.com (Status: 200) [Size: 162]
Found: api.examplesite.com (Status: 200) [Size: 162]
Found: apache.examplesite.com (Status: 200) [Size: 162]

Gobuster has many more options that can be added to our basic scans. For example, we can use the "-d" flag to set a delay in our scanning process which is beneficial for stability, performance, and stealth. Or the "-x" flag which is used for filtering file extensions during the scan. For instance, if you want to specifically search for images, you might use the following command:

gobuster dir -u https://examplesite.com -w /usr/share/dirb/wordlists/common.txt -x jpg,jpeg,png,gif,ico

Be sure to check out additional flags by using "--help".

Password Cracking With Hashcat

 _._     _,-'""`-._
(,-.`._,'(       |\`-/|
    `-.-' \ )-`( , x x)
          `-    \`_`"'-

Hashcat is a command line interface password cracking tool for over 300 hashing algorithms. Hashcat can utilize five possible attack modes including brute-force, dictionary attacks, and more. We will be focusing on dictionary attacks, a more simple and faster attack mode that compares the hashed password to a wordlist rather than brute-force methods which can take a large amount of time and computer resources due to the attacks method of running combinations of random characters through trial and error.

We will be using the rockyou.txt password list. This password list was compiled back in 2009 after a major data breach at RockYou, a social media app and advertising network. The list contains over 14 million unique passwords that are commonly used amongst the general public and is a staple wordlist in the hacker community. You can view and download other password lists from here.

Passwords are usually discovered as hashes. Password hashing is the practice of algorithmically turning a plain-text password into ciphertext, or an irreversibly obfuscated version of itself, as a means of blocking against the threat of password breaches. This process helps secure passwords from the naked eye. There are many types of hashing algorithms including MD5, SHA-1, SHA-256, and hundreds of others. As mentioned earlier, Hashcat can crack over 300 different types of hashing algorithm. Below is a collection of downloadable hashed passwords which can be utilized for practice with Hashcat and the rockyou.txt password list. You can also hash your own passwords using online resources such as the All Hash Generator on Browserling.com.

Hashed Password Samples:
MD5Password.txt
SHA1Password.txt
SHA256Password.txt
CRC32Password.txt
NTLMPassword.txt

Hashcat can be downloaded on various operating systems but is included in the Kali Linux distro. We can crack our first MD5 hashed password in the samples above using the below command in our terminal:

hashcat –m 0 *path to MD5Password.txt* *path to rockyou.txt*

The first option of the command, "-m 0", sets the type of hash you are attempting to crack, 0 being MD5. You can view all the Hashcat hashing algorithms and their dedicated #'s using the command "hashcat --help". For example, 100 is to be used for SHA-1 hashes. If you do not know the hashing algorithm used in the password you are attempting to crack, you can use a hash identifying tool like hash-identifier which is included in Kali Linux distros. 

The second option, "*path to MD5Password.txt*", directs Hashcat to the hashed password file path. For example /home/USERNAME/Downloads/MD5Password.txt, where USERNAME is your username. The final option of the command is the path to the wordlist you are running the attack against. Kali Linux includes many default wordlists, including rockyou.txt, in the /usr/share/wordlists path. Please note that you will need to unzip the rockyou.txt.gz directory on your machine if you haven't already by using the gunzip command.

Let's run the command and see what Hashcat comes up with:

hashcat -m 0 /root/Downloads/MD5Password.txt /usr/share/wordlists/rockyou.txt
hashcat (v6.2.6) starting

OpenCL API (OpenCL 3.0 PoCL 5.0+debian  Linux, None+Asserts, RELOC, SPIR, LLVM 16.0.6, SLEEF, DISTRO, POCL_DEBUG) - Platform #1 [The pocl project]
==================================================================================================================================================
* Device #1: cpu-haswell-AMD Ryzen 5 4600H with Radeon Graphics, 1425/2914 MB (512 MB allocatable), 2MCU

Minimum password length supported by kernel: 0
Maximum password length supported by kernel: 256

Hashes: 1 digests; 1 unique digests, 1 unique salts
Bitmaps: 16 bits, 65536 entries, 0x0000ffff mask, 262144 bytes, 5/13 rotates
Rules: 1

Optimizers applied:
* Zero-Byte
* Early-Skip
* Not-Salted
* Not-Iterated
* Single-Hash
* Single-Salt
* Raw-Hash

ATTENTION! Pure (unoptimized) backend kernels selected.
Pure kernels can crack longer passwords, but drastically reduce performance.
If you want to switch to optimized kernels, append -O to your commandline.
See the above message to find out about the exact limits.

Watchdog: Temperature abort trigger set to 90c

Host memory required for this attack: 0 MB

Dictionary cache built:
* Filename..: /usr/share/wordlists/rockyou.txt
* Passwords.: 14344392
* Bytes.....: 139921507
* Keyspace..: 14344385
* Runtime...: 1 sec

6104df369888589d6dbea304b59a32d4:blink182                 
                                                          
Session..........: hashcat
Status...........: Cracked
Hash.Mode........: 0 (MD5)
Hash.Target......: 6104df369888589d6dbea304b59a32d4
Time.Started.....: Tue Sep 24 15:54:28 2024 (0 secs)
Time.Estimated...: Tue Sep 24 15:54:28 2024 (0 secs)
Kernel.Feature...: Pure Kernel
Guess.Base.......: File (/usr/share/wordlists/rockyou.txt)
Guess.Queue......: 1/1 (100.00%)
Speed.#1.........:     9780 H/s (0.06ms) @ Accel:256 Loops:1 Thr:1 Vec:8
Recovered........: 1/1 (100.00%) Digests (total), 1/1 (100.00%) Digests (new)
Progress.........: 512/14344385 (0.00%)
Rejected.........: 0/512 (0.00%)
Restore.Point....: 0/14344385 (0.00%)
Restore.Sub.#1...: Salt:0 Amplifier:0-1 Iteration:0-1
Candidate.Engine.: Device Generator
Candidates.#1....: 123456 -> letmein
Hardware.Mon.#1..: Util: 66%

Started: Tue Sep 24 15:53:52 2024
Stopped: Tue Sep 24 15:54:30 2024

Success! After a few seconds we can see the hashed password, blink182, has been cracked at the line highlighted in orange. Blink-182 fans beware, your password isn't fooling even the most amateur of hackers out there.

Try using Hashcat with the other sample hashed passwords in this post, or hash your own passwords using the All Hash Generator on Browserling.com and see if your passwords are vulnerable to dictionary attacks using the rockyou.txt or other common wordlists. To learn more about Hashcat and it's other features, visit their Wiki here.