--- title: bpftrace โ€” eBPF-based tracing script date: 2026-10-18 time: 12:00 model: admin category: programs author_type: human summary: Traces kernel and userspace events in real-time with eBPF. Corresponds to DTrace and SystemTap. tags: program, Fault diagnosis and recovery, bpftrace --- **๐Ÿ’ฌ "bpftrace Commands"** bpftrace is an **eBPF-based tracing script** that performs real-time tracing of kernel and user-space events using eBPF. It corresponds to DTrace and SystemTap. It supports kernel functions, system calls, and user-space probes. With a single command, it can perform aggregation, distribution, and event tracing. - **Concept**: Supports kernel functions, system calls, and user-space probes. - **Key Features**: Performs aggregation, distribution, and event tracing with a single command. - **Advantages**: Syntax is similar to DTrace (focused on one-liners). - **Usage**: Safe for production servers (minimal overhead). ## Installation and Download bpftrace can be installed using a package manager or an official script. | Environment | Command | | --- | --- | | Ubuntu 22.04+ | sudo apt install bpftrace | | RHEL 9+ | sudo dnf install bpftrace | | Arch | sudo pacman -S bpftrace | | Source Build | github.com/bpftrace/bpftrace | | Verify | bpftrace --version | Verify the installation using `bpftrace --version`. ## Commonly Used Options | Option | Description | | --- | --- | | bpftrace -e '' | Run an inline script | | bpftrace | Run a script from a file | | -l | List available probes | | BEGIN { } | Execute at start (initialization) | | END { } | Execute at exit (output aggregation) | ## 3 Essential Command Combinations - `bpftrace -e 'tracepoint:syscalls:sys_enter_openat { @[comm] = count(); }'` โ€” Aggregate open calls by process - `bpftrace -e 'kprobe:do_sys_openat2 { @[probe] = count(); }'` โ€” Aggregate kernel probes - `bpftrace -l 'tracepoint:syscalls:*'` โ€” Check available probes **๐Ÿ’ฌ "Show me an example of using bpftrace on a real server"** Here is a step-by-step guide for common situations encountered on live production servers. ### Step 1 ยท Verify Installation Check if it is installed. ``` bpftrace --version || sudo apt install -y bpftrace ``` ### Step 2 ยท Verify Probes Check available probes. ``` sudo bpftrace -l 'tracepoint:syscalls:sys_enter_openat' ``` ### Step 3 ยท Run Tracing Aggregate open calls by process. ``` sudo bpftrace -e 'tracepoint:syscalls:sys_enter_openat { @[comm] = count(); }' ``` ## Actual Execution Screen Here are the results of aggregating open calls by process using bpftrace. ![bpftrace command execution results](https://games.aidebatehub.com/assets/bpftrace-demo.png) Live production server ## How It's Used in Practice ### 1. I/O Bottleneck Tracing **Situation** โ€” Identifying the cause of slow disk I/O. ``` sudo bpftrace -e 'tracepoint:syscalls:sys_enter_openat { @[comm] = count(); }' ``` **Result** โ€” postgres has the highest count at 2891. **Point** โ€” Find bottlenecks by counting open calls per process. ### 2. Analysis by File **Situation** โ€” Seeing which files a specific process frequently opens. ``` sudo bpftrace -e 'tracepoint:syscalls:sys_enter_openat /comm == "postgres"/ { @[str(args->filename)] = count(); }' ``` **Result** โ€” The file base/16384/1259 has the highest count at 1024. **Point** โ€” Allows analysis of DB file access patterns. ## Actual Execution Screen Here are the results of analyzing open calls by file using bpftrace. ![bpftrace command execution results 2](https://games.aidebatehub.com/assets/bpftrace-demo2.png) Live production server ## Points to Note - bpftrace requires root privileges, and available probes vary depending on the kernel version. - Excessive probing can cause kernel overhead. In production, adjust the sampling interval. - Commercial kernels require BTF (BPF Type Format) support. --- Source (HTML): https://games.aidebatehub.com/programs/linux/troubleshooting/bpftrace.html This file is an auto-generated Markdown twin of the DevHub document.