Introduction
In this lab, we will explore how to determine if the Linux kernel's powerful tracing utility, ftrace, is enabled and accessible on your system. We will begin by examining the /sys/kernel/debug/tracing directory, which serves as the primary interface for interacting with ftrace.
Following the initial check, we will utilize the trace-cmd tool to further verify ftrace functionality and list available tracing options. Finally, we will inspect relevant configuration parameters within the /proc/sys/kernel file system to gain a deeper understanding of the ftrace setup.
Check ftrace status with ls /sys/kernel/debug/tracing
In this step, we will start exploring ftrace, a powerful tracing utility built into the Linux kernel. ftrace allows developers and system administrators to understand the runtime behavior of the kernel.
The primary interface for interacting with ftrace is through the debug file system, specifically located at /sys/kernel/debug/tracing.
First, let's check if the debug file system is mounted and if the tracing directory exists. We can use the ls command to list the contents of this directory.
Type the following command in your terminal and press Enter:
ls /sys/kernel/debug/tracing
You should see a list of files and directories related to ftrace. The exact output may vary depending on the kernel version and configuration, but it should look something like this:
README events options
available_events free_buffer per_cpu
available_filter_functions function_graph_max_depth printk_formats
available_tracers function_graph_tracer set_event
buffer_size_kb function_profile_dynamic set_filter
buffer_total_size_kb function_trace set_ftrace_filter
current_tracer instances set_ftrace_notrace
dynamic_filter kprobe_events set_graph_function
enabled kprobe_profile set_graph_notrace
error_log max_graph_depth set_tracer
event_sources options snapshot
tracing_cpumask tracing_max_latency tracing_on
tracing_thresh trace trace_clock
trace_marker trace_options trace_pipe
trace_stat
If you see a similar output, it means the ftrace interface is available on your system. If you get an error like "No such file or directory", it might indicate that the debug file system is not mounted or ftrace is not enabled in the kernel. However, in the LabEx environment, it should be available.
This directory contains various files that control ftrace behavior and provide tracing data. We will explore some of these files in later steps.
For now, simply verifying the existence of this directory and its contents confirms that you can interact with ftrace.
Click Continue to proceed to the next step.
Verify ftrace with trace-cmd list
In the previous step, we confirmed the presence of the ftrace interface in the debug file system. Now, let's use a command-line tool called trace-cmd to interact with ftrace. trace-cmd is a user-space utility that simplifies the process of configuring and collecting data from ftrace.
One of the useful functions of trace-cmd is listing available tracing events and tracers. This helps us understand what we can monitor with ftrace.
Let's use trace-cmd list to see the available tracers. Tracers are different modes of operation for ftrace, allowing you to trace different aspects of the kernel.
Type the following command in your terminal and press Enter:
trace-cmd list -t
The -t option tells trace-cmd to list available tracers. You should see output similar to this:
List of available tracers:
blk
function
function_graph
irqsoff
nop
preemptirqsoff
wakeup
wakeup_rt
This output shows the different tracers available on this system. For example, function traces function calls, and function_graph traces function calls and their call graph.
Next, let's list the available events that can be traced. Events are specific points in the kernel where tracing information can be collected.
Type the following command and press Enter:
trace-cmd list -e
The -e option tells trace-cmd to list available events. This command will produce a long list of events, categorized by subsystems (e.g., sched, syscalls, ext4). The output will look something like this (only a small portion is shown):
List of available events:
ftrace:
ftrace_buffer_overrun
ftrace_inject_event
ftrace_print_event
kmem:
kmem_cache_alloc
kmem_cache_alloc_node
kmem_cache_free
kmem_cache_free_page
kmem_cache_free_pages
kmem_cache_grow
kmem_cache_shrink
mm_page_alloc
mm_page_alloc_zone_locked
mm_page_free
mm_page_free_batched
mm_page_pcpu_drain
... (many more events)
You can scroll through the output to see the vast number of events available for tracing. This demonstrates the detailed visibility ftrace provides into kernel operations.
Using trace-cmd list is a good way to get an overview of what ftrace can monitor on your specific system.
Click Continue to move on.
Inspect ftrace config in /proc/sys/kernel
In addition to the debug file system interface at /sys/kernel/debug/tracing, some ftrace related configurations can also be found in the /proc/sys/kernel directory. The /proc file system is a virtual file system that provides information about processes and other system information. The /proc/sys directory contains files that allow you to view and modify kernel parameters at runtime.
Let's inspect some files in /proc/sys/kernel that are related to ftrace. We can use the cat command to view the content of these files.
First, let's check the ftrace_enabled file. This file indicates whether ftrace is currently enabled in the kernel.
Type the following command in your terminal and press Enter:
cat /proc/sys/kernel/ftrace_enabled
You should see an output like this:
1
A value of 1 means ftrace is enabled, and 0 means it is disabled.
Next, let's look at the ftrace_dump_on_oops file. This file controls whether the ftrace buffer is dumped to the console when a kernel oops (a kernel error) occurs.
Type the following command and press Enter:
cat /proc/sys/kernel/ftrace_dump_on_oops
The output will likely be:
1
A value of 1 means the ftrace buffer will be dumped on a kernel oops, which can be helpful for debugging.
Finally, let's check the ftrace_filter_notrace_regex file. This file contains a regular expression that can be used to filter which functions are not traced when using function tracing.
Type the following command and press Enter:
cat /proc/sys/kernel/ftrace_filter_notrace_regex
The output might be empty or contain a regular expression, depending on the system's configuration:
These files in /proc/sys/kernel provide a glimpse into some global ftrace settings. While most detailed configuration and data access happen through /sys/kernel/debug/tracing, these files offer a quick way to check the overall status and some basic behaviors of ftrace.
You have now explored the main interfaces for interacting with ftrace: the debug file system and the /proc/sys/kernel directory.
Click Continue to complete this lab.
Summary
In this lab, we learned how to check if the kernel ftrace utility is enabled and accessible on a Linux system. We started by exploring the primary ftrace interface located within the debug file system at /sys/kernel/debug/tracing. By listing the contents of this directory using the ls command, we verified the presence of ftrace-related files and directories, indicating that the ftrace interface is available.
This initial step confirmed the fundamental requirement for using ftrace: the debug file system must be mounted and the tracing directory must exist. Seeing the various ftrace configuration and control files in the output of ls /sys/kernel/debug/tracing provided a visual confirmation that the ftrace infrastructure is present and ready for further interaction.
