Introduction
As a system administrator or security analyst, you will often encounter massive log files generated by web servers like Apache or Nginx. Manually reading thousands of lines of logs to find malicious activity is unsustainable and prone to human error.
In this lab, you will step into the shoes of a security responder. You will write a Python script designed to ingest Apache web server access logs. By utilizing Python's Regular Expressions (re) library, you will automatically parse out IP addresses and detect known SQL injection (SQLi) attack signatures. Finally, you will export these findings into a structured CSV report for further analysis.
This module is where Python starts becoming an analyst multiplier rather than just a programming language. Logs are one of the clearest examples of work that is too repetitive for careful manual review but structured enough for automation to help immediately.
Read Log Files in Python
In this step, you will learn how to use Python to open and read a log file. The environment has already been prepared with a sample Apache log file named access.log in your working directory.
Reading files line by line is a foundational automation pattern. It scales better than copying data by hand, and it prepares you to add filtering, parsing, counting, and reporting logic without changing the basic ingestion approach.
First, let's view the contents of the sample log file to understand what we are working with. Use the cat command in your terminal to print the file contents.
cat access.log
You should see several lines of standard Apache access logs. Some of these lines contain normal web requests, while others contain suspicious SQL injection attempts.
Now, let's create a Python script to read this file programmatically. We will use the nano text editor to create a file named log_analyzer.py.
nano log_analyzer.py
In Python, the open() function is used to open a file, and the with statement ensures that the file is properly closed after we are done reading it. Copy and paste the following Python code into your nano editor:
def analyze_logs(file_path):
print(f"Opening log file: {file_path}")
## Open the file in read mode ('r')
with open(file_path, 'r') as file:
## Read the file line by line
for line in file:
## Print each line, stripping the extra newline character
print(line.strip())
if __name__ == "__main__":
log_file = "access.log"
analyze_logs(log_file)
Save the file in nano by pressing Ctrl+O, then Enter to confirm the filename. If you want to continue building the script before testing it, you can stay in nano and move on to the next step. Use Ctrl+X only when you are ready to return to the terminal.
Run your newly created Python script using the python3 command:
python3 log_analyzer.py
You will see the script outputting the exact same log lines you saw earlier with the cat command. This confirms your Python script can successfully ingest the log data line by line.
Apply Regex for IP Extraction
In this step, you will enhance your Python script to automatically extract the IP address from each log line. To do this, we will use Regular Expressions (Regex). Regex is a powerful tool for matching patterns in text.
Regular expressions are useful because logs are semi-structured: they follow patterns, but not usually in the form of clean Python objects. Regex gives you a way to describe the text pattern you want and let the interpreter do the repetitive matching work.
Python provides a built-in module called re for working with regular expressions. We need to import this module and define a pattern that matches an IPv4 address. An IPv4 address consists of four numbers (ranging from 0 to 255) separated by dots.
If log_analyzer.py is still open in nano, continue editing it there. Otherwise, reopen it with:
nano log_analyzer.py
Update your script to include the re module and the logic to search for IP addresses. Replace the entire contents of the file with the following code:
import re
def analyze_logs(file_path):
print(f"Analyzing log file: {file_path}\n")
## Define a Regex pattern for an IPv4 address
## \b ensures word boundaries, \d{1,3} matches 1 to 3 digits, \. matches a literal dot
ip_pattern = r'\b(?:\d{1,3}\.){3}\d{1,3}\b'
with open(file_path, 'r') as file:
for line in file:
## re.findall returns a list of all matches in the string
ips_found = re.findall(ip_pattern, line)
if ips_found:
## We take the first IP found in the line
source_ip = ips_found[0]
print(f"Extracted IP: {source_ip}")
if __name__ == "__main__":
log_file = "access.log"
analyze_logs(log_file)
Save the file with Ctrl+O and Enter. If you want to test this intermediate version now, exit with Ctrl+X and run the script. If not, you can continue directly to the next step.
python3 log_analyzer.py
You should see an output similar to this:
Analyzing log file: access.log
Extracted IP: 192.168.1.10
Extracted IP: 10.0.0.15
Extracted IP: 172.16.0.5
Extracted IP: 192.168.1.22
Extracted IP: 10.0.0.25
The script is now successfully ignoring the rest of the log text and isolating just the source IP addresses using the power of Regex.
Detect SQLi Signatures
In this step, you will add a security detection feature to your script. SQL Injection (SQLi) is a common web attack where malicious SQL statements are inserted into entry fields.
This turns the script from parser into detector. Instead of only extracting fields, you are now applying security logic to decide which lines deserve attention. That shift from collection to interpretation is what makes automation operationally valuable.
Attackers often use specific keywords like UNION SELECT to steal data, or ' OR '1'='1 to bypass authentication. We can use Regex to scan the log lines for these known attack signatures.
If you are still editing the same file in nano, keep working there. Otherwise, reopen it:
nano log_analyzer.py
We will define a new Regex pattern to catch common SQLi phrases. We will use (?i) to make the search case-insensitive, meaning it will catch UNION, union, or uNiOn. Update your script to match the following code:
import re
def analyze_logs(file_path):
print(f"Scanning log file for SQLi attacks: {file_path}\n")
ip_pattern = r'\b(?:\d{1,3}\.){3}\d{1,3}\b'
## Define a Regex pattern for SQL Injection signatures
## (?i) makes it case-insensitive
## It looks for 'union select' OR 'or 1=1' variations
sqli_pattern = r"(?i)(union.*select|or\s+1\s*=\s*1|'.*or.*')"
with open(file_path, 'r') as file:
for line in file:
ips_found = re.findall(ip_pattern, line)
source_ip = ips_found[0] if ips_found else "Unknown IP"
## re.search returns a match object if the pattern is found, otherwise None
if re.search(sqli_pattern, line):
print(f"[ALERT] SQL Injection detected from IP: {source_ip}")
print(f"Raw Log: {line.strip()}\n")
if __name__ == "__main__":
log_file = "access.log"
analyze_logs(log_file)
Save the file with Ctrl+O and Enter. Exit nano with Ctrl+X only when you are ready to run the script from the terminal.
python3 log_analyzer.py
Your output should now specifically flag the malicious requests:
Scanning log file for SQLi attacks: access.log
[ALERT] SQL Injection detected from IP: 10.0.0.15
Raw Log: 10.0.0.15 - - [10/Oct/2026:13:55:45 -0700] "GET /login.php?user=admin' OR '1'='1 HTTP/1.1" 200 5120
[ALERT] SQL Injection detected from IP: 192.168.1.22
Raw Log: 192.168.1.22 - - [10/Oct/2026:13:56:12 -0700] "GET /products.php?id=1 UNION SELECT username, password FROM users HTTP/1.1" 200 4012
Your script is now acting as a basic Intrusion Detection System (IDS), automatically identifying the suspicious requests in the log file.
Output to CSV Report
In this final step, you will format the results and save them into a CSV (Comma-Separated Values) file. Printing to the terminal is great for testing, but security teams need structured reports that can be opened in spreadsheet software or ingested by other tools.
Python has a built-in csv module that makes writing CSV files very easy.
If you kept the file open, continue editing it. Otherwise, reopen it:
nano log_analyzer.py
We will import the csv module, open a new file called report.csv in write mode ('w'), and write our findings into it. Replace your code with this final version:
import re
import csv
def analyze_logs(file_path, output_csv):
print(f"Analyzing logs and generating report: {output_csv}...")
ip_pattern = r'\b(?:\d{1,3}\.){3}\d{1,3}\b'
sqli_pattern = r"(?i)(union.*select|or\s+1\s*=\s*1|'.*or.*')"
## Open the output CSV file in write mode
with open(output_csv, 'w', newline='') as csv_file:
writer = csv.writer(csv_file)
## Write the header row
writer.writerow(['Source IP', 'SQLi Detected', 'Raw Log'])
with open(file_path, 'r') as file:
for line in file:
clean_line = line.strip()
ips_found = re.findall(ip_pattern, clean_line)
source_ip = ips_found[0] if ips_found else "Unknown IP"
## Check if SQLi is detected
sqli_detected = "Yes" if re.search(sqli_pattern, clean_line) else "No"
## Write the data row to the CSV
writer.writerow([source_ip, sqli_detected, clean_line])
print("Report generation finished.")
if __name__ == "__main__":
log_file = "access.log"
report_file = "report.csv"
analyze_logs(log_file, report_file)
Save the file and execute the script:
python3 log_analyzer.py
You will see a message indicating the report generation is complete. Now, let's verify that the CSV file was created and check its contents using the cat command:
cat report.csv
You should see a neatly formatted CSV output:
Source IP,SQLi Detected,Raw Log
192.168.1.10,No,192.168.1.10 - - [10/Oct/2026:13:55:36 -0700] "GET /index.html HTTP/1.1" 200 2326
10.0.0.15,Yes,10.0.0.15 - - [10/Oct/2026:13:55:45 -0700] "GET /login.php?user=admin' OR '1'='1 HTTP/1.1" 200 5120
172.16.0.5,No,172.16.0.5 - - [10/Oct/2026:13:56:01 -0700] "POST /api/data HTTP/1.1" 404 134
192.168.1.22,Yes,192.168.1.22 - - [10/Oct/2026:13:56:12 -0700] "GET /products.php?id=1 UNION SELECT username, password FROM users HTTP/1.1" 200 4012
10.0.0.25,No,10.0.0.25 - - [10/Oct/2026:13:57:00 -0700] "GET /about.html HTTP/1.1" 200 1024
Summary
In this lab, you successfully transitioned from manual log reading to automated log analysis using Python. You learned how to read and process text files line by line using Python's built-in file handling. You also utilized the re module to apply Regular Expressions for extracting specific data formats, such as IP addresses. Furthermore, you implemented security detection logic by matching known SQL injection attack signatures against raw log data. Finally, you practiced exporting structured data into a CSV report using the csv module, making the results easy to share and analyze further.
These skills form the foundation of security automation and data parsing, allowing you to handle massive datasets efficiently.
