How to check if kernel ftrace is enabled in Linux

Introduction

In this lab, we will explore how to determine if the Linux kernel's powerful tracing utility, ftrace, is enabled and accessible on your system. We will begin by examining the /sys/kernel/debug/tracing directory, which serves as the primary interface for interacting with ftrace.

Following the initial check, we will utilize the trace-cmd tool to further verify ftrace functionality and list available tracing options. Finally, we will inspect relevant configuration parameters within the /proc/sys/kernel file system to gain a deeper understanding of the ftrace setup.

Skills Graph

Check ftrace status with ls /sys/kernel/debug/tracing

In this step, we will start exploring ftrace, a powerful tracing utility built into the Linux kernel. ftrace allows developers and system administrators to understand the runtime behavior of the kernel.

The primary interface for interacting with ftrace is through the debug file system, specifically located at /sys/kernel/debug/tracing.

First, let's check if the debug file system is mounted and if the tracing directory exists. We can use the ls command to list the contents of this directory.

Type the following command in your terminal and press Enter:

ls /sys/kernel/debug/tracing

You should see a list of files and directories related to ftrace. The exact output may vary depending on the kernel version and configuration, but it should look something like this:

README                       events                       options
available_events             free_buffer                  per_cpu
available_filter_functions   function_graph_max_depth     printk_formats
available_tracers            function_graph_tracer        set_event
buffer_size_kb               function_profile_dynamic     set_filter
buffer_total_size_kb         function_trace               set_ftrace_filter
current_tracer               instances                    set_ftrace_notrace
dynamic_filter               kprobe_events                set_graph_function
enabled                      kprobe_profile               set_graph_notrace
error_log                    max_graph_depth              set_tracer
event_sources                options                      snapshot
tracing_cpumask              tracing_max_latency          tracing_on
tracing_thresh               trace                        trace_clock
trace_marker                 trace_options                trace_pipe
trace_stat

If you see a similar output, it means the ftrace interface is available on your system. If you get an error like "No such file or directory", it might indicate that the debug file system is not mounted or ftrace is not enabled in the kernel. However, in the LabEx environment, it should be available.

This directory contains various files that control ftrace behavior and provide tracing data. We will explore some of these files in later steps.

For now, simply verifying the existence of this directory and its contents confirms that you can interact with ftrace.

Click Continue to proceed to the next step.

Verify ftrace with trace-cmd list

In the previous step, we confirmed the presence of the ftrace interface in the debug file system. Now, let's use a command-line tool called trace-cmd to interact with ftrace. trace-cmd is a user-space utility that simplifies the process of configuring and collecting data from ftrace.

One of the useful functions of trace-cmd is listing available tracing events and tracers. This helps us understand what we can monitor with ftrace.

Let's use trace-cmd list to see the available tracers. Tracers are different modes of operation for ftrace, allowing you to trace different aspects of the kernel.

Type the following command in your terminal and press Enter:

trace-cmd list -t

The -t option tells trace-cmd to list available tracers. You should see output similar to this:

List of available tracers:
 blk
 function
 function_graph
 irqsoff
 nop
 preemptirqsoff
 wakeup
 wakeup_rt

This output shows the different tracers available on this system. For example, function traces function calls, and function_graph traces function calls and their call graph.

Next, let's list the available events that can be traced. Events are specific points in the kernel where tracing information can be collected.

Type the following command and press Enter:

trace-cmd list -e

The -e option tells trace-cmd to list available events. This command will produce a long list of events, categorized by subsystems (e.g., sched, syscalls, ext4). The output will look something like this (only a small portion is shown):

List of available events:
  ftrace:
    ftrace_buffer_overrun
    ftrace_inject_event
    ftrace_print_event
  kmem:
    kmem_cache_alloc
    kmem_cache_alloc_node
    kmem_cache_free
    kmem_cache_free_page
    kmem_cache_free_pages
    kmem_cache_grow
    kmem_cache_shrink
    mm_page_alloc
    mm_page_alloc_zone_locked
    mm_page_free
    mm_page_free_batched
    mm_page_pcpu_drain
  ... (many more events)

You can scroll through the output to see the vast number of events available for tracing. This demonstrates the detailed visibility ftrace provides into kernel operations.

Using trace-cmd list is a good way to get an overview of what ftrace can monitor on your specific system.

Click Continue to move on.

Inspect ftrace config in /proc/sys/kernel

In addition to the debug file system interface at /sys/kernel/debug/tracing, some ftrace related configurations can also be found in the /proc/sys/kernel directory. The /proc file system is a virtual file system that provides information about processes and other system information. The /proc/sys directory contains files that allow you to view and modify kernel parameters at runtime.

Let's inspect some files in /proc/sys/kernel that are related to ftrace. We can use the cat command to view the content of these files.

First, let's check the ftrace_enabled file. This file indicates whether ftrace is currently enabled in the kernel.

Type the following command in your terminal and press Enter:

cat /proc/sys/kernel/ftrace_enabled

You should see an output like this:

1

A value of 1 means ftrace is enabled, and 0 means it is disabled.

Next, let's look at the ftrace_dump_on_oops file. This file controls whether the ftrace buffer is dumped to the console when a kernel oops (a kernel error) occurs.

Type the following command and press Enter:

cat /proc/sys/kernel/ftrace_dump_on_oops

The output will likely be:

1

A value of 1 means the ftrace buffer will be dumped on a kernel oops, which can be helpful for debugging.

Finally, let's check the ftrace_filter_notrace_regex file. This file contains a regular expression that can be used to filter which functions are not traced when using function tracing.

Type the following command and press Enter:

cat /proc/sys/kernel/ftrace_filter_notrace_regex

The output might be empty or contain a regular expression, depending on the system's configuration:

These files in /proc/sys/kernel provide a glimpse into some global ftrace settings. While most detailed configuration and data access happen through /sys/kernel/debug/tracing, these files offer a quick way to check the overall status and some basic behaviors of ftrace.

You have now explored the main interfaces for interacting with ftrace: the debug file system and the /proc/sys/kernel directory.

Click Continue to complete this lab.

Summary

In this lab, we learned how to check if the kernel ftrace utility is enabled and accessible on a Linux system. We started by exploring the primary ftrace interface located within the debug file system at /sys/kernel/debug/tracing. By listing the contents of this directory using the ls command, we verified the presence of ftrace-related files and directories, indicating that the ftrace interface is available.

This initial step confirmed the fundamental requirement for using ftrace: the debug file system must be mounted and the tracing directory must exist. Seeing the various ftrace configuration and control files in the output of ls /sys/kernel/debug/tracing provided a visual confirmation that the ftrace infrastructure is present and ready for further interaction.