Explore and Create Nmap Scripts

Introduction

In this lab, you will learn about Nmap's Scripting Engine (NSE) and how to categorize and update Nmap scripts. Nmap, a powerful open - source network discovery and security auditing tool, has its functionality extended by the NSE. The NSE enables users to write and share scripts for automating various networking tasks.

This lab will guide you through exploring Nmap's script categories, creating custom script directories, adding your own scripts, and updating the Nmap script database. These skills are crucial for network administrators and security professionals to efficiently scan networks and identify vulnerabilities.

Skills Graph

Exploring Nmap Script Categories

In this step, we're going to explore the categories of Nmap scripts that come pre - installed on your system. Nmap scripts are powerful tools in the field of cybersecurity. They are organized according to their functionality. For example, some scripts are used for discovery, which means they can find devices and services on a network. Others are for vulnerability assessment, which helps in identifying security weaknesses in systems.

First, we need to open a terminal. A terminal is a text - based interface where you can enter commands to interact with your system. Once the terminal is open, we need to make sure we are in the correct directory. The directory is like a folder on your computer where files and other folders are stored. Run the following command in the terminal:

cd /home/labex/project

This command changes the current working directory to /home/labex/project.

Now, let's check the version of Nmap installed on our system. Knowing the version is important because different versions may have different features and compatibility. Run this command in the terminal:

nmap --version

The output will look similar to this:

Nmap version 7.80 ( https://nmap.org )
Platform: x86_64-pc-linux-gnu
Compiled with: liblua-5.3.3 openssl-1.1.1f libpcre-8.39 libpcap-1.9.1 nmap-libdnet-1.12 ipv6
Compiled without:
Available nsock engines: epoll poll select

Next, we'll examine the available Nmap script categories. Nmap scripts are stored in the /usr/share/nmap/scripts/ directory. To get an idea of what scripts are available, we can list them and sort them by their names. Run this command in the terminal:

ls /usr/share/nmap/scripts/ | grep -v .lua | sort | head -10

This command first lists all the files in the /usr/share/nmap/scripts/ directory. Then, it uses grep -v .lua to exclude files with the .lua extension. After that, it sorts the remaining files by their names and shows the first 10 results. The output will be similar to:

address-info.nse
afp-brute.nse
afp-ls.nse
afp-path-vuln.nse
afp-serverinfo.nse
afp-showmount.nse
ajp-auth.nse
ajp-brute.nse
ajp-headers.nse
ajp-methods.nse

To see all the script categories, we can use the following command:

grep -r categories /usr/share/nmap/scripts/*.nse | grep -o "categories = {[^}]*}" | sort | uniq | head -10

This command searches through all .nse script files in the /usr/share/nmap/scripts/ directory for the "categories" string. It then extracts the category information, sorts it, removes any duplicate entries, and shows the first 10 results. The output will look something like:

categories = {"auth", "brute", "intrusive"}
categories = {"auth", "default", "discovery", "safe"}
categories = {"auth", "discovery", "safe"}
categories = {"auth", "intrusive"}
categories = {"auth", "safe"}
categories = {"broadcast", "discovery"}
categories = {"broadcast", "discovery", "safe"}
categories = {"default", "discovery"}
categories = {"default", "discovery", "safe"}
categories = {"default", "discovery", "safe", "version"}

For better organization, we'll create a directory structure to categorize scripts based on their functionality. This will make it easier to find and use the scripts later. We'll create separate directories for different categories. Run these commands in the terminal:

mkdir -p /home/labex/project/NmapScripts/vulnerability
mkdir -p /home/labex/project/NmapScripts/discovery
mkdir -p /home/labex/project/NmapScripts/authentication

The mkdir -p command creates directories. If the parent directories don't exist, it creates them as well.

Now, let's copy some vulnerability - related scripts to our newly created vulnerability directory. Run this command in the terminal:

cp /usr/share/nmap/scripts/smb-vuln* /home/labex/project/NmapScripts/vulnerability/

This command copies all the scripts in the /usr/share/nmap/scripts/ directory whose names start with smb - vuln to the /home/labex/project/NmapScripts/vulnerability/ directory.

Let's verify the files were copied correctly. Run this command in the terminal:

ls -la /home/labex/project/NmapScripts/vulnerability/

The output will show the copied vulnerability scripts:

total 88
drwxr-xr-x 2 labex labex  4096 Mar 15 12:30 .
drwxr-xr-x 4 labex labex  4096 Mar 15 12:30 ..
-rw-r--r-- 1 labex labex  3355 Mar 15 12:30 smb-vuln-conficker.nse
-rw-r--r-- 1 labex labex  8045 Mar 15 12:30 smb-vuln-cve2009-3103.nse
-rw-r--r-- 1 labex labex  5100 Mar 15 12:30 smb-vuln-cve-2017-7494.nse
-rw-r--r-- 1 labex labex  9595 Mar 15 12:30 smb-vuln-ms06-025.nse
-rw-r--r-- 1 labex labex 11645 Mar 15 12:30 smb-vuln-ms07-029.nse
-rw-r--r-- 1 labex labex 12558 Mar 15 12:30 smb-vuln-ms08-067.nse
-rw-r--r-- 1 labex labex  9719 Mar 15 12:30 smb-vuln-ms10-054.nse
-rw-r--r-- 1 labex labex  7326 Mar 15 12:30 smb-vuln-ms10-061.nse
-rw-r--r-- 1 labex labex  8091 Mar 15 12:30 smb-vuln-ms17-010.nse
-rw-r--r-- 1 labex labex  4245 Mar 15 12:30 smb-vuln-regsvc-dos.nse

Similarly, let's copy some discovery and authentication scripts to their respective directories. Run these commands in the terminal:

cp /usr/share/nmap/scripts/dns-* /home/labex/project/NmapScripts/discovery/
cp /usr/share/nmap/scripts/ssh-* /home/labex/project/NmapScripts/authentication/

Now you have organized some Nmap scripts into categories based on their functionality, which makes it easier to find and use them for specific tasks.

Understanding Script Functionality

In this step, we're going to learn how to examine and understand Nmap scripts to figure out what they can do. Knowing this is really important because it helps you pick the right scripts for different network scanning jobs.

Let's start by looking closely at one of the vulnerability scripts we copied in the previous step. We'll use the cat command to display the content of the script, and then use head -20 to show only the first 20 lines. This way, we can quickly see the important information at the beginning of the script.

cat /home/labex/project/NmapScripts/vulnerability/smb-vuln-ms17-010.nse | head -20

The output will show the start of the script. This part usually has details about what the script is for and how to use it.

local smb = require "smb"
local vulns = require "vulns"
local stdnse = require "stdnse"
local string = require "string"

description = [[
Attempts to detect if a Microsoft SMBv1 server is vulnerable to a remote code
execution vulnerability (ms17-010, a.k.a. EternalBlue). The vulnerability affects
Windows Vista, 7, 8.1, 10, Server 2008, Server 2008 R2, Server 2012 Gold and R2, and Server 2016.

The script connects to the $IPC tree, executes a transaction on FID 0 and
checks if the error "STATUS_INSUFF_SERVER_RESOURCES" is returned to
determine if the target is not patched against ms17-010. Additionally it checks
for a matching error code on EternalChampion.

References:
* https://technet.microsoft.com/en-us/library/security/ms17-010.aspx
* https://blogs.technet.microsoft.com/msrc/2017/05/12/customer-guidance-for-wannacrypt-attacks/
* https://msrc-blog.microsoft.com/2017/05/12/customer-guidance-for-wannacrypt-attacks/
* https://github.com/rapid7/metasploit-framework/pull/8654/files

This script is designed to check if a Microsoft SMBv1 server has the MS17-010 vulnerability, which is also known as EternalBlue.

Now, let's look at the structure of an Nmap script to understand its different parts. Most Nmap scripts have these key components:

1. A description section: This tells you what the script does. It's like a short summary that helps you quickly understand the script's purpose.
2. An author section: This gives credit to the person who created the script. It's important to know who made the script, especially if you need to contact them or look for more scripts from the same author.
3. Categories: These group similar scripts together. For example, scripts related to vulnerability scanning might be in the "vuln" category. This makes it easier to find and manage scripts.
4. Rules: These determine when the script should run. There are different types of rules, like portrule and hostrule. For example, a hostrule can specify that the script should only run on certain hosts.
5. The main action function: This is where the actual work of the script happens. It contains the code that performs the task, like checking for a vulnerability or getting information about a host.

Let's create our own basic Nmap script to see this structure in action. We'll use the cat command to create a new file and write the script into it.

cat << 'EOF' > /home/labex/project/my-ping-check.nse
description = [[
A simple script that checks if a host responds to ICMP echo requests (ping).
]]

author = "LabEx User"

categories = {"discovery", "safe"}

-- The rule section determines when the script should be run
hostrule = function(host)
  return true  -- Run the script for all hosts
end

-- The action section contains the main function of the script
action = function(host)
  local output = "Host status: "
  
  if host.pingresponse then
    output = output .. "Responds to ping"
  else
    output = output .. "Does not respond to ping"
  end
  
  return output
end
EOF

Let's break down this script:

• The description section clearly explains that the script checks if a host responds to ping requests.
• The author section says that the script was created by a LabEx User.
• The categories section groups this script with other discovery and safe scripts. Discovery scripts are used to find information about hosts, and safe scripts are those that don't cause any harm to the target.
• The hostrule function is set to return true, which means the script will run for all hosts.
• The action function contains the main logic. It checks if the host has a ping response and then returns a message indicating whether the host responds to ping or not.

Now, let's run our custom script against the localhost (127.0.0.1). We'll use the nmap command with the --script option to specify our custom script.

nmap --script /home/labex/project/my-ping-check.nse 127.0.0.1

You should see output similar to this:

Starting Nmap 7.80 ( https://nmap.org )
Nmap scan report for localhost (127.0.0.1)
Host is up (0.000091s latency).
Not shown: 998 closed ports
PORT     STATE SERVICE
22/tcp   open  ssh
| my-ping-check: Host status: Responds to ping
3001/tcp open  nessus
|_my-ping-check: Host status: Responds to ping

Nmap done: 1 IP address (1 host up) scanned in 0.24 seconds

This output shows that our script is working correctly. It detected that the localhost responds to ping requests. This is a simple example, but it shows you the basic structure and function of an Nmap script.

Creating and Adding Custom Scripts to Nmap

In this step, we're going to learn how to create a custom Nmap script and add it to the Nmap script database. Nmap is a powerful network scanning tool, and by adding custom scripts, you can extend its functionality to meet your specific needs. This means you can perform unique tasks during network scans that aren't covered by the default Nmap scripts.

First, let's create a new, slightly more complex script. This script will check if a specific port is open. Ports are like doors in a network; they allow different types of network traffic to enter or leave a device. By checking if a port is open, we can find out if a particular service is running on a device.

cat << 'EOF' > /usr/share/nmap/scripts/port-check.nse
description = [[
A script that checks if a specific port is open and reports its status.
]]

author = "LabEx User"
license = "Same as Nmap--See https://nmap.org/book/man-legal.html"
categories = {"discovery", "safe"}

-- The portrule determines when the script should be run
portrule = function(host, port)
  return port.protocol == "tcp" and port.state == "open"
end

-- The action section contains the main function of the script
action = function(host, port)
  local output = string.format("Port %d is open on host %s", port.number, host.ip)
  return output
end
EOF

In the code above, we first define a description of what the script does. Then we set the author and license information. The portrule function decides when the script should run. Here, it checks if the port uses the TCP protocol and is open. The action function is the main part of the script. It creates a message indicating that a specific port is open on a particular host.

Notice that we've created this script directly in the Nmap scripts directory (/usr/share/nmap/scripts/). This is important because Nmap looks for scripts in this directory. However, just creating the script isn't enough. Nmap needs to know about this new script, so we need to update the Nmap script database.

To update the Nmap script database, run the following command:

sudo nmap --script-updatedb

The output should be similar to:

Starting Nmap 7.80 ( https://nmap.org )
NSE: Updating rule database.
NSE: Script Database updated successfully.
Nmap done: 0 IP addresses (0 hosts up) scanned in 0.36 seconds

This output shows that the script database has been updated successfully. Now, let's verify that our script has been added to the Nmap script database. We can do this by searching for our script in the script.db file.

grep port-check /usr/share/nmap/scripts/script.db

You should see output similar to:

Entry { filename = "port-check.nse", categories = { "discovery", "safe", } }

This confirms that our script has been successfully added to the Nmap script database.

Now, let's create one more script that demonstrates how to use libraries within Nmap scripts. Libraries are pre - written code that we can use to make our scripts more powerful. This script will check for HTTP headers on web servers. HTTP headers contain important information about a web page, like the type of server, the date it was last modified, etc.

cat << 'EOF' > /usr/share/nmap/scripts/http-headers-check.nse
description = [[
A script that retrieves and displays HTTP headers from web servers.
]]

author = "LabEx User"
license = "Same as Nmap--See https://nmap.org/book/man-legal.html"
categories = {"discovery", "safe"}

local http = require "http"
local shortport = require "shortport"
local stdnse = require "stdnse"

-- The portrule determines when the script should be run
portrule = function(host, port)
  return shortport.http(host, port)
end

-- The action section contains the main function of the script
action = function(host, port)
  local response = http.get(host, port, "/")
  
  if not response or not response.status then
    return "Failed to retrieve HTTP headers"
  end
  
  local output = stdnse.output_table()
  output["Status"] = response.status
  
  if response.header then
    output["Headers"] = {}
    for name, value in pairs(response.header) do
      output["Headers"][name] = value
    end
  end
  
  return output
end
EOF

This script uses several Nmap libraries:

• http for making HTTP requests. This library allows our script to communicate with web servers and get information from them.
• shortport for determining if a port is likely running an HTTP service. It helps us figure out if a particular port on a host is used for web traffic.
• stdnse for formatting output. This makes the information we get from the web server easier to read.

Let's update the script database again so that Nmap knows about our new script:

sudo nmap --script-updatedb

Now, let's run our new script against a webserver. For demonstration purposes, we'll use a local webserver on port 3001 (which is often used by development servers).

nmap --script http-headers-check -p 3001 127.0.0.1

If there's no web server running on port 3001, the output will indicate that the port is closed or filtered. If there is a web server, you'll see the HTTP headers that were retrieved.

Finally, let's run both of our custom scripts together. This way, we can see the results of both scripts at once.

nmap --script "port-check,http-headers-check" -p 22,3001 127.0.0.1

This command will run both our custom scripts against ports 22 and 3001 on localhost. Port 22 is commonly used for SSH (Secure Shell) connections.

The output will show whether these ports are open and, if port 3001 is running an HTTP service, it will display the HTTP headers.

By creating and adding custom scripts to Nmap, you can extend its functionality to perform specific tasks needed for your network scanning and security assessment requirements.

Running Script Categories and Managing Script Output

In this step, we'll explore how to run Nmap scripts by category and manage the output of these script scans. This is a very useful skill because it allows you to run multiple related scripts without having to specify each one separately.

Nmap scripts are grouped into different categories based on what they do. Understanding these categories helps you quickly find and run the scripts you need. Here are some common categories:

• auth: These are scripts related to authentication. They can help you check if authentication mechanisms on a target are secure.
• broadcast: Scripts in this category send broadcasts on the local network. They're useful for discovering devices and services on the same network.
• brute: These are brute force password auditing scripts. They try different passwords to see if they can gain unauthorized access to a service.
• default: When you use the -sC option in Nmap, these are the scripts that run by default. They provide a good starting point for a basic scan.
• discovery: These scripts are used for host and service discovery. They can find out what hosts are available on a network and what services they're running.
• dos: Denial of Service scripts are designed to overload a target system and make it unavailable. Be careful when using these, as they can cause real damage.
• exploit: Exploitation scripts are used to take advantage of known vulnerabilities in a system. They can be used for security testing but should be used responsibly.
• external: These scripts may send data to third - party databases. They can provide additional information about the target, but you need to be aware of the privacy implications.
• fuzzer: Fuzzer scripts use fuzzing techniques. They send random or malformed data to a target to see if it can cause errors or vulnerabilities to be exposed.
• intrusive: Scripts in this category might crash services or be considered intrusive. Use them with caution, especially in a production environment.
• malware: Malware detection scripts are used to check if a target system is infected with malware.
• safe: These scripts are considered safe and non - intrusive. They won't cause any harm to the target system.
• version: Version detection scripts help you find out the version of software running on a target service. This information can be useful for identifying potential vulnerabilities.
• vuln: Vulnerability detection scripts are used to find security weaknesses in a target system.

Now, let's see how to run scripts from a specific category. For example, if you want to run all scripts in the "discovery" category, you can use the following command:

nmap --script discovery 127.0.0.1

This command tells Nmap to run all the discovery scripts against the localhost (127.0.0.1). The output will be quite long because it shows the results of all the discovery scripts.

Sometimes, you may want to run scripts from multiple categories. You can do this by using a comma to separate the category names. Here's an example:

nmap --script "discovery,safe" -p 1-100 127.0.0.1

This command runs all the scripts in both the discovery and safe categories against ports 1 - 100 on the localhost.

You can also exclude specific categories using the not operator. For instance:

nmap --script "discovery and not broadcast" 127.0.0.1

This command runs the discovery scripts but excludes any that are also in the broadcast category.

Next, let's look at different ways to format and save the output of Nmap script scans. Nmap offers several output formats, which are useful for different purposes:

1. Normal output (default): This is the standard text output that Nmap shows on the screen.
2. XML output: XML is a structured format that can be easily parsed by other tools.
3. Grepable output: This format is designed to be easily searched using tools like grep.
4. JSON output (with the appropriate script): JSON is another structured format that's widely used for data exchange.

To save the output in XML format, you can use the following command:

nmap --script vuln -p 1-1000 127.0.0.1 -oX /home/labex/project/scan_results.xml

This command runs all the vulnerability scripts against ports 1 - 1000 on the localhost and saves the results in XML format to the file /home/labex/project/scan_results.xml.

Let's check the content of the XML file:

head -20 /home/labex/project/scan_results.xml

This command shows the first 20 lines of the XML - formatted output. You'll see information about the scan, the options used, and the scan results.

For grepable output, you can use this command:

nmap --script "default" 127.0.0.1 -oG /home/labex/project/scan_results.gnmap

This command saves the results in a format that's easy to parse with tools like grep.

Let's check the content of the gnmap file:

cat /home/labex/project/scan_results.gnmap

You'll see that each host is on a single line, which makes it easy to search for specific information.

If you want to save the output in both normal and XML formats at the same time, you can use this command:

nmap --script "default" 127.0.0.1 -oX /home/labex/project/scan_results2.xml -oN /home/labex/project/scan_results.txt

This command saves the output in both normal text format and XML format simultaneously.

Finally, you can use the -oA option to save the output in all three formats (normal, XML, and grepable) at once:

nmap --script "default" 127.0.0.1 -oA /home/labex/project/all_formats

This command creates three files:

• /home/labex/project/all_formats.nmap (normal output)
• /home/labex/project/all_formats.xml (XML output)
• /home/labex/project/all_formats.gnmap (grepable output)

By mastering these techniques for running script categories and managing script output, you can efficiently conduct network scans and organize the results for further analysis.

Summary

In this lab, you have learned essential skills for working with Nmap's Scripting Engine (NSE). You explored how to categorize and organize Nmap scripts according to their functionality, which is vital for efficient network scanning and security assessments.

You gained hands - on experience in multiple aspects, including exploring script categories, creating organized directories, understanding script structures, creating custom scripts, adding them to the database, running scripts by category with logical operators, and managing output in different formats. These skills enable you to extend Nmap's functionality for specific needs. By mastering script creation, categorization, and updating, you can develop more targeted and efficient scanning workflows. The ability to organize and format scan results is also useful for documentation and communication with team members and stakeholders. As you continue using Nmap, you can build on these skills for more complex scripts and scanning strategies.