Learn Nmap Scanning and Output Analysis

Introduction

In this lab, you will learn how to use Nmap, a powerful network scanning tool widely used by security professionals to discover hosts and services on a computer network. You'll explore various output formats provided by Nmap, which are vital for interpreting scan results and conducting further analysis.

Nmap, short for Network Mapper, is a free and open - source utility for network discovery and security auditing. It uses raw IP packets to gather information about network hosts, such as available services and operating systems. This lab will guide you through setting up a test service and scanning it with different Nmap output options, which is crucial for effective network analysis and documentation.

Setting Up a Test Environment for Nmap Scanning

In this step, we're going to create a simple environment where you can practice Nmap scanning. Nmap is a powerful tool used for network exploration and security auditing. By setting up a basic web server on your local machine and then scanning it with Nmap, you'll be able to learn about network scanning in a safe and controlled environment. This way, you can experiment without affecting real - world networks.

Setting Up a Local Web Server

First, we need to set up a simple web server that we can scan with Nmap. Python provides an easy way to create a basic HTTP server. An HTTP server is a software that can serve web pages to clients, like browsers.

Open a terminal and navigate to your project directory:
```
cd /home/labex/project
```
The cd command stands for "change directory". It allows you to move around in your file system. In this case, we're moving to the /home/labex/project directory where we'll set up our web server.
Create a simple HTML file to serve:
```
echo "Welcome to Nmap Testing Environment" > index.html
```
The echo command prints text to the terminal. The > symbol redirects that output into a file. So, we're creating a file named index.html with the text "Welcome to Nmap Testing Environment". This will be the web page our server serves.
Start a Python HTTP server on port 8080:
```
python3 -m http.server 8080 &
```
The python3 -m http.server command starts a simple HTTP server using Python. The 8080 specifies the port number on which the server will listen. A port is like a door through which network traffic enters or leaves a computer. The & at the end of the command runs the server in the background, allowing you to continue using the terminal for other commands.
Verify that your web server is running by accessing it:
```
curl http://localhost:8080
```
The curl command is used to transfer data from or to a server. Here, we're trying to access the web server we just set up at http://localhost:8080. localhost refers to the current computer.

You should see the following output:
```
Welcome to Nmap Testing Environment
```
If you see this output, it means your web server is running correctly.

Basic Nmap Scanning

Now that you have a web server running, you can scan it using Nmap. Scanning a server helps you find out which ports are open and what services might be running on them.

Run a basic Nmap scan on your local web server:
```
nmap -p 8080 localhost
```
The -p option in the nmap command specifies the port to scan. In this case, we're telling Nmap to scan only port 8080 on localhost (which is the IP address 127.0.0.1).

You should see output similar to this:
```
Starting Nmap 7.80 ( https://nmap.org ) at 2023-11-08 12:34 UTC
Nmap scan report for localhost (127.0.0.1)
Host is up (0.000097s latency).

PORT     STATE SERVICE
8080/tcp open  http-proxy

Nmap done: 1 IP address (1 host up) scanned in 0.05 seconds
```
This output is in Nmap's default format, which is human - readable and displays the basic information about the scan.

Understanding the Default Nmap Output

Let's understand what the default output shows:

Starting Nmap: This line shows the version of Nmap you're using and the time when the scan started. It helps you keep track of when the scan was initiated.
Nmap scan report: This indicates the target of the scan. In our case, the target is localhost.
Host is up: This confirms that the target is online. The latency time shows how long it took for Nmap to get a response from the target.
PORT: This shows the port number that was scanned. In our scan, we scanned port 8080.
STATE: This indicates whether the port is open, closed, or filtered. An open port means that a service is listening on that port and accepting connections.
SERVICE: Nmap tries to guess what service might be running on that port. Here, it guessed http - proxy.
Nmap done: This is a summary of the scan. It tells you how many IP addresses were scanned and how long the scan took.

Now you have successfully set up a test environment and performed a basic Nmap scan. In the next step, you will explore different output formats that Nmap offers.

Exploring Different Nmap Output Formats

In this step, we're going to learn about the various output formats that Nmap offers. Output formats are crucial in cybersecurity because they allow us to present the scan results in different ways, depending on our needs. For instance, some formats are easy for humans to read, while others are designed for machines to parse, which is useful when integrating with other tools.

Overview of Nmap Output Formats

Nmap supports several output formats, each with its own unique characteristics and use - cases:

Normal Output (default): This is a human - readable format. It presents the scan results in a way that is easy for us to understand at a glance, making it great for quick manual analysis.
XML Output (-oX): XML stands for Extensible Markup Language. It's a structured format, which means the data is organized in a hierarchical way. This makes it easy for programs and scripts to parse the data, and it's commonly used when integrating Nmap with other security tools.
Grepable Output (-oG): This is a line - based format. It's designed to be easily processed with Unix tools like grep, awk, and cut. This format is useful when you want to quickly extract specific information from the scan results.
Script Kiddie Output (-oS): This format is similar to the normal output, but it includes ASCII art. However, it's rarely used in practical scenarios.
All Formats (-oA): This option allows you to save the scan results in normal, XML, and grepable formats simultaneously. It's a convenient way to have all types of outputs available at once.

Let's explore these formats by scanning your local web server again.

XML Output Format

XML, or Extensible Markup Language, is a structured format that is widely used for data storage and exchange. Its hierarchical structure makes it easy for programs to parse the data, which is why it's commonly used for integration with other security tools.

Run an Nmap scan and save the output in XML format:
```
nmap -p 8080 localhost -oX /home/labex/project/scan_results.xml
```
In this command, -p 8080 specifies that we're scanning port 8080, localhost is the target we're scanning (our local machine), and -oX tells Nmap to save the output in XML format to the specified file path.

View the XML output:

cat /home/labex/project/scan_results.xml

The cat command is used to display the contents of a file. When you run this command, you'll see XML - formatted output that looks something like this (abbreviated):

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE nmaprun>
<nmaprun scanner="nmap" args="nmap -p 8080 localhost -oX /home/labex/project/scan_results.xml" ...>
  <scaninfo type="connect" protocol="tcp" .../>
  <verbose level="0"/>
  <debugging level="0"/>
  <host>
    <status state="up" reason="conn - refused" reason_ttl="0"/>
    <address addr="127.0.0.1" addrtype="ipv4"/>
    <hostnames>
      <hostname name="localhost" type="user"/>
      <hostname name="localhost" type="PTR"/>
    </hostnames>
    <ports>
      <port protocol="tcp" portid="8080">
        <state state="open" reason="syn - ack" reason_ttl="0"/>
        <service name="http - proxy" method="table" conf="3"/>
      </port>
    </ports>
    <times srtt="97" rttvar="5000" to="100000"/>
  </host>
  <runstats>...</runstats>
</nmaprun>

The XML format includes detailed information organized in a hierarchical structure. Let's extract the specific port information:
```
grep -A5 "<port " /home/labex/project/scan_results.xml > /home/labex/project/port_details.txt
```
The grep command is used to search for a specific pattern in a file. -A5 means to display 5 lines after the line that matches the pattern. We're searching for lines that contain <port in the XML file and saving the results to a new file called port_details.txt.
View the extracted port details:
```
cat /home/labex/project/port_details.txt
```
Running this command will show you the port information section from the XML.

Grepable Output Format

Grepable output is specifically designed to be easily processed with tools like grep, awk, and cut. These Unix tools are very powerful for text processing and can help us quickly extract the information we need from the scan results.

Run an Nmap scan and save the output in grepable format:
```
nmap -p 8080 localhost -oG /home/labex/project/scan_results.grep
```
Here, -oG tells Nmap to save the output in grepable format to the specified file.

View the grepable output:

cat /home/labex/project/scan_results.grep

The output will look similar to this:

## Nmap 7.80 scan initiated Wed Nov 8 12:40:00 2023 as: nmap -p 8080 localhost -oG /home/labex/project/scan_results.grep
Host: 127.0.0.1 (localhost)   Status: Up
Host: 127.0.0.1 (localhost)   Ports: 8080/open/tcp//http - proxy///
## Nmap done at Wed Nov 8 12:40:00 2023 -- 1 IP address (1 host up) scanned in 0.05 seconds

The grepable format puts all host information on a single line, making it easy to use with text processing tools. For example, you can extract just the line containing port information:
```
grep "Ports:" /home/labex/project/scan_results.grep > /home/labex/project/ports_info.txt
```
This command searches for lines containing the word "Ports:" in the grepable output file and saves the matching line to a new file called ports_info.txt.
View the extracted ports information:
```
cat /home/labex/project/ports_info.txt
```

Saving in All Formats Simultaneously

Nmap provides a convenient option to save the output in multiple formats at once using the -oA option. This can save you time if you need different types of outputs for different purposes.

Run an Nmap scan and save the output in all formats:
```
nmap -p 8080 localhost -oA /home/labex/project/all_formats
```
This command scans port 8080 on the local machine and saves the results in normal, XML, and grepable formats.
Check the files created:
```
ls -la /home/labex/project/all_formats.*
```
The ls -la command lists all files in a directory with detailed information. When you run this command, you should see three files:
- all_formats.nmap (normal output)
- all_formats.xml (XML output)
- all_formats.gnmap (grepable output)

Now you understand the different output formats that Nmap offers and how to use them. Each format has its own advantages:

Normal output is easy for humans to read
XML output is structured for machine parsing and integration with other tools
Grepable output is designed for quick analysis with Unix text processing tools

In the next step, you will learn how to analyze these outputs in more detail.

Analyzing Nmap Output and Extracting Useful Information

In this step, you'll learn how to extract and analyze specific information from Nmap scan results. This is a crucial skill for network administrators and security professionals. When dealing with network scans, you often get a large amount of data. Being able to extract the relevant parts efficiently helps you understand the network's security status and identify potential issues.

Analyzing XML Output

XML output is very useful for detailed analysis and can be easily integrated with other tools. XML is a structured format that organizes data in a hierarchical way, making it easier to locate and extract specific information. Let's explore how to extract specific information from the XML output.

First, let's look at the structure of the XML output to understand what information is available. We'll use the grep command to find the start of the host section in the XML file. The grep command searches for a specified pattern in a file.
```
grep "<host" -A2 /home/labex/project/scan_results.xml
```
This command searches for the <host> tag in the scan_results.xml file and shows the next two lines after the match. This helps us see the beginning of the host section in the XML file.
Now, let's extract information about the port state. We'll use grep again to find all lines containing the <state tag and save the results to a new file.
```
grep "<state " /home/labex/project/scan_results.xml > /home/labex/project/port_state.txt
```
View the extracted port state. We'll use the cat command, which is used to display the contents of a file.
```
cat /home/labex/project/port_state.txt
```
You should see something like:
```
<state state="open" reason="syn-ack" reason_ttl="0"/>
```
This tells us that the port is open. Nmap determined this because it received a SYN - ACK packet in response to its SYN packet. This is part of the TCP three - way handshake, which is how a TCP connection is established between two devices on a network.

Using Regular Expressions for More Complex Extraction

For more complex data extraction, you can use regular expressions with tools like grep, sed, or awk. Regular expressions are a powerful way to search for and match patterns in text. They allow you to define complex search criteria.

Let's extract both the port number and its state. We'll use grep with a regular expression to find the relevant information in the XML file and save it to a new file.
```
grep -o 'portid="[0-9]*".*state="[^"]*"' /home/labex/project/scan_results.xml > /home/labex/project/port_and_state.txt
```
View the extracted information using the cat command.
```
cat /home/labex/project/port_and_state.txt
```
You should see output like:
```
portid="8080" state="open"
```

Analyzing Grepable Output

The grepable output format is designed to be easily processed with standard Unix tools. It's a text - based format that makes it straightforward to search for specific information using commands like grep.

Let's extract just the open ports from the grepable output. We'll use grep to find all lines containing the word "open" and save the results to a new file.
```
grep "open" /home/labex/project/scan_results.grep > /home/labex/project/open_ports.txt
```
View the extracted information using the cat command.
```
cat /home/labex/project/open_ports.txt
```
This should show you the line containing all open ports.
You can further process this to extract just the port numbers. We'll use a combination of grep and cut commands. The cut command is used to extract specific parts of a line based on a delimiter.
```
grep "open" /home/labex/project/scan_results.grep | grep -o "8080/open" | cut -d "/" -f1 > /home/labex/project/port_numbers.txt
```
View the extracted port numbers using the cat command.
```
cat /home/labex/project/port_numbers.txt
```
You should see:
```
8080
```

Creating a Simple Summary Report

Now, let's create a simple summary report that combines information from different parts of the scan. A summary report helps you quickly understand the key findings of the scan.

Create a summary report. We'll use the echo command to write text to a file. The >> operator appends the text to the end of the file.

echo "Nmap Scan Summary" > /home/labex/project/scan_summary.txt
echo "----------------" >> /home/labex/project/scan_summary.txt
echo "Target: localhost (127.0.0.1)" >> /home/labex/project/scan_summary.txt
echo "Open ports:" >> /home/labex/project/scan_summary.txt
grep "open" /home/labex/project/scan_results.grep | grep -o "[0-9]*/open/tcp//[^/]*" >> /home/labex/project/scan_summary.txt

View the summary report using the cat command.

cat /home/labex/project/scan_summary.txt

The output should look like:

Nmap Scan Summary
----------------
Target: localhost (127.0.0.1)
Open ports:
8080/open/tcp//http - proxy

By learning how to extract and analyze specific information from Nmap scan results, you can efficiently process and interpret network reconnaissance data. This skill is particularly valuable when scanning large networks with numerous hosts and services.

In the next step, you will learn about more advanced Nmap scanning techniques.

Advanced Nmap Scanning Techniques

In this step, we'll explore more advanced Nmap scanning techniques. These techniques are crucial as they can provide additional and detailed information about network services. Specifically, we'll focus on service version detection and script scanning. Understanding these techniques will help you gain a deeper insight into the network services you're scanning, which is essential for security assessments, network inventory, and troubleshooting.

Service Version Detection

Nmap has a powerful feature that allows it to attempt to determine the specific version of a service running on a port. This is done using the -sV option. Knowing the service version can be very useful, as different versions may have different security vulnerabilities or features.

Run an Nmap scan with service version detection:
```
nmap -p 8080 -sV localhost -oN /home/labex/project/version_scan.txt
```
In this command, the -sV option tells Nmap to probe the open ports. By doing so, Nmap can gather information about the services running on those ports, including their versions. The -oN option is used to save the output of the scan in a normal format to the specified file /home/labex/project/version_scan.txt. Saving the output in a file allows you to review the results later.

View the scan results:

cat /home/labex/project/version_scan.txt

After running the cat command, you should see output similar to this:

Starting Nmap 7.80 ( https://nmap.org ) at 2023-11-08 13:20:00 UTC
Nmap scan report for localhost (127.0.0.1)
Host is up (0.000097s latency).

PORT     STATE SERVICE VERSION
8080/tcp open  http    Python/3.10 aiohttp/3.8.1 (Python httpd)

Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 6.52 seconds

Notice that Nmap now shows not just the service name, but also version information. In this case, it detected that the HTTP server is running with Python 3.10. This detailed information can be used to identify potential security risks or to understand the capabilities of the service.

Using Nmap Scripts (NSE)

The Nmap Script Engine (NSE) is a powerful tool that allows you to run scripts to perform various tasks. These tasks range from service detection to vulnerability scanning. By using NSE scripts, you can extend the functionality of Nmap and gather more information about the target network.

Run a basic HTTP information script on your web server:
```
nmap -p 8080 --script=http-title localhost -oN /home/labex/project/script_scan.txt
```
The http-title script is designed to fetch the title of HTTP web pages. This can be useful for quickly identifying the content or purpose of a web page. The -oN option saves the output of the script scan in a normal format to the specified file /home/labex/project/script_scan.txt.

View the script scan results:

cat /home/labex/project/script_scan.txt

The output should include something like:

Starting Nmap 7.80 ( https://nmap.org ) at 2023-11-08 13:25:00 UTC
Nmap scan report for localhost (127.0.0.1)
Host is up (0.000097s latency).

PORT     STATE SERVICE
8080/tcp open  http-proxy
|_http-title: Directory listing for /

Nmap done: 1 IP address (1 host up) scanned in 0.42 seconds

The http-title script retrieved the title of the web page, which in this case is a directory listing. This information can help you understand the structure and content of the web server.

Let's try another script that provides more information about the HTTP server:
```
nmap -p 8080 --script=http-server-header localhost -oN /home/labex/project/server_header_scan.txt
```
The http-server-header script is used to retrieve the HTTP server header. The server header often contains information about the server software, such as the version and type. This information can be valuable for security assessments and network inventory. The -oN option saves the output of the scan in a normal format to the specified file /home/labex/project/server_header_scan.txt.

View the server header scan results:

cat /home/labex/project/server_header_scan.txt

The output should include information about the server software:

Starting Nmap 7.80 ( https://nmap.org ) at 2023-11-08 13:30:00 UTC
Nmap scan report for localhost (127.0.0.1)
Host is up (0.000097s latency).

PORT     STATE SERVICE
8080/tcp open  http-proxy
|_http-server-header: SimpleHTTP/0.6 Python/3.10.12

Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds

Combining Multiple Techniques

You can combine multiple scanning techniques in a single Nmap command to perform a more comprehensive scan. This allows you to gather a wide range of information about the target network in one go.

Run a comprehensive scan that includes port scanning, service version detection, and script scanning:
```
nmap -p 8080 -sV --script=http-title,http-server-header localhost -oA /home/labex/project/comprehensive_scan
```
This command does several things:
- It scans port 8080 (-p 8080), which means it checks if this port is open on the target host.
- It detects service versions (-sV), providing information about the specific versions of the services running on the port.
- It runs the http-title and http-server-header scripts (--script=http-title,http-server-header), which fetch the web page title and the server header information respectively.
- It saves the results in all formats (-oA). Saving in all formats ensures that you have different representations of the scan results for different purposes.
View the comprehensive scan results:
```
cat /home/labex/project/comprehensive_scan.nmap
```
The output will include all the information from the previous scans combined. This gives you a holistic view of the target network service running on port 8080.

Create a final analysis report that summarizes all your findings:

echo "Final Nmap Analysis Report" > /home/labex/project/final_report.txt
echo "------------------------" >> /home/labex/project/final_report.txt
echo "Target: localhost (127.0.0.1)" >> /home/labex/project/final_report.txt
echo "Port: 8080/tcp (open)" >> /home/labex/project/final_report.txt
echo "Service:" >> /home/labex/project/final_report.txt
grep "SERVICE VERSION" -A1 /home/labex/project/version_scan.txt | tail -1 >> /home/labex/project/final_report.txt
echo "HTTP Server Header:" >> /home/labex/project/final_report.txt
grep "http-server-header:" /home/labex/project/server_header_scan.txt >> /home/labex/project/final_report.txt
echo "HTTP Page Title:" >> /home/labex/project/final_report.txt
grep "http-title:" /home/labex/project/script_scan.txt >> /home/labex/project/final_report.txt

This set of commands creates a text file named final_report.txt and populates it with a summary of all the information gathered from the previous scans. It includes details about the target, the port, the service version, the server header, and the web page title.

View your final report:
```
cat /home/labex/project/final_report.txt
```
Your final report should contain a comprehensive summary of all the information you've gathered about the web server running on port 8080. This report can be used for further analysis, security assessments, or documentation purposes.

By learning these advanced Nmap scanning techniques, you can gather detailed information about network services that is valuable for security assessments, network inventory, and troubleshooting.

Summary

In this lab, you have learned how to use Nmap, a powerful network scanning tool for network discovery and security auditing. You gained practical experience in various Nmap operations, including setting up a test environment with a local web server, performing basic port scans, exploring different output formats, extracting and analyzing scan results, and using advanced techniques like service version detection and script scanning.

These skills are essential for network administrators, security analysts, and penetration testers. They enable you to discover network hosts and services, document network infrastructure, identify security issues, and automate scanning and reporting. Mastering Nmap and its output formats allows for effective network reconnaissance, integration with other security tools, and building comprehensive network inventories. The command - line skills you practiced are also transferable to other system administration and security tasks.