Introduction
Network traffic analysis often requires extracting specific pieces of information from thousands of packets. While reading packet summaries is helpful for humans, automated tools and scripts require structured data.
This is where packet analysis starts to overlap with data engineering. A human can scan a few packet lines visually, but computers work best when information is arranged into stable fields and columns. Once network traffic is converted into structured output, it can be searched, counted, graphed, imported into spreadsheets, or processed by Python and SIEM pipelines.
In this lab, you will learn how to transform raw network traffic into structured, easy-to-parse formats using tshark, the command-line version of Wireshark. You will practice disabling the default human-readable output, selecting specific protocol fields (like IP addresses, DNS queries, and HTTP User-Agents), and formatting the results into Comma-Separated Values (CSV) for further analysis.
Disable Default Output
In this step, you will learn how to change the default output behavior of tshark.
By default, when you read a packet capture file using tshark, it prints a summary line for every single packet. This summary is designed for human reading but is very difficult to parse using scripts or import into databases. To extract specific data, we first need to tell tshark to stop printing this default summary and instead prepare to output specific fields.
The key idea is that summaries are presentation-oriented, while fields are analysis-oriented. A summary line may look convenient to a person, but small formatting changes can break downstream automation. Extracting named fields gives you predictable output that other tools can reliably consume.
We achieve this by using the -T fields option. The -T flag stands for "Text output format", and setting it to fields tells tshark that we only want to see the exact data fields we specify.
However, if you only use -T fields without specifying which fields you want, tshark will output nothing. You must pair it with the -e flag, which specifies the exact field to extract.
Let's start by extracting just the frame number from our sample capture file. Ensure you are in your default working directory /home/labex/project.
Run the following command to read the sample.pcap file and extract only the frame numbers:
tshark -r sample.pcap -T fields -e frame.number | head -n 5
The -r flag tells tshark to read from a file instead of capturing live traffic. The | head -n 5 part of the command limits the output to the first 5 lines so your terminal doesn't get flooded with numbers.
You should see an output similar to this:
1
2
3
4
5
Notice how clean the output is compared to the standard tshark summary. This is the foundation for building structured datasets from network traffic.
Use Custom Column Formatting
In this step, you will learn how to extract multiple fields simultaneously and format them as Comma-Separated Values (CSV).
When you use multiple -e flags, tshark will extract multiple fields and place them in columns. By default, tshark separates these columns using a "Tab" character. While tab-separated values are useful, CSV is the universal standard for importing data into spreadsheets, databases, and analysis tools like Python's Pandas.
This may feel like a formatting detail, but format choice shapes what you can do next. Security work often involves moving data between packet analyzers, shell pipelines, spreadsheets, notebooks, and dashboards. The easier your output is to import elsewhere, the easier it becomes to scale a one-off command into a repeatable workflow.
To change the character that separates the columns, we use the -E separator= option.
Let's extract the frame number, the source IP address, and the destination IP address, and format them as a CSV. The field names for source and destination IPs are ip.src and ip.dst.
To make sure we are looking at packets that actually have IP addresses (and skip things like low-level ARP packets which do not), we will use the -Y "ip" filter alongside our field extraction.
Run the following command in your terminal:
tshark -r sample.pcap -Y "ip" -T fields -e frame.number -e ip.src -e ip.dst -E separator=, | head -n 5
You should see an output similar to this:
12,127.0.0.1,127.0.0.11
13,127.0.0.11,127.0.0.1
14,127.0.0.1,127.0.0.11
15,127.0.0.11,127.0.0.1
16,172.60.1.88,104.18.26.120
Because we filtered out the initial ARP packets, you will notice the frame numbers might not start at 1. Sometimes, a packet might not contain a specific field, and in such cases, tshark will simply leave that column blank, ensuring your CSV structure remains intact.
Extract DNS Queries
In this step, you will combine field extraction with display filters to isolate and extract specific information from DNS traffic.
When analyzing network traffic, you rarely want to extract fields from all packets. Usually, you are interested in a specific protocol. You can use the -Y flag to apply a display filter before extracting the fields.
This is another example of narrowing scope before interpreting results. If you mix unrelated protocols together in one export, the output becomes sparse and confusing because many packets will not contain the fields you requested. Filtering first keeps the resulting dataset semantically coherent.
For example, if we want to see which websites a user is trying to visit, we can look at their DNS queries. The display filter for DNS traffic is simply dns. The specific field that contains the domain name being queried is dns.qry.name.
Let's extract the source IP address and the DNS query name, but only for packets that are actually DNS requests or responses.
Execute the following command:
tshark -r sample.pcap -Y "dns" -T fields -e ip.src -e dns.qry.name -E separator=, | head -n 5
Your output will look similar to this:
10.0.2.15,labex.io
168.63.129.16,labex.io
10.0.2.15,google.com
168.63.129.16,google.com
By combining -Y "dns" with -T fields, we ensure that our CSV output only contains relevant data, filtering out all the background noise of other network protocols.
Extract HTTP User Agents
In this final step, you will extract detailed information from HTTP traffic, specifically focusing on the User-Agent string.
The User-Agent is a string sent by a web browser or client to the server, identifying the software being used (e.g., Chrome, Firefox, or a custom script). Security analysts often extract User-Agents to identify malicious automated scripts or outdated browsers on a network.
Metadata like User-Agent is useful because it gives clues about behavior without requiring you to inspect every byte of content. In network defense, many useful judgments come from context fields such as user agent, host header, TLS server name, or DNS query name. They help analysts separate normal application traffic from suspicious automation.
To isolate HTTP requests, we can use the display filter http.request. The fields we want to extract are the source IP (ip.src), the requested host domain (http.host), the specific URI path (http.request.uri), and the User-Agent string (http.user_agent).
Run the following command to extract this HTTP metadata into a CSV format:
tshark -r sample.pcap -Y "http.request" -T fields -e ip.src -e http.host -e http.request.uri -e http.user_agent -E separator=,
You should see an output similar to this:
10.0.2.15,example.com,/,curl/7.81.0
10.0.2.15,neverssl.com,/,LabEx-Custom-Agent/1.0
Notice how we successfully extracted the custom User-Agent LabEx-Custom-Agent/1.0 that was generated during the lab setup. This technique is incredibly powerful for creating automated reports or feeding network data into security monitoring dashboards.
Summary
In this lab, you learned how to transform raw network packet captures into structured, analyzable data using tshark. You started by disabling the default human-readable output using -T fields. You then learned how to select specific data points using the -e flag and format them into CSV using -E separator=,. Finally, you combined these formatting techniques with display filters (-Y) to extract highly specific information, such as DNS query names and HTTP User-Agent strings. These skills are essential for automating network analysis and integrating packet data with other security tools.
