AI Knowledge Hubbpftrace — eBPF-based tracing script
💬 "bpftrace Commands"
bpftrace is an eBPF-based tracing script that performs real-time tracing of kernel and user-space events using eBPF. It corresponds to DTrace and SystemTap.
It supports kernel functions, system calls, and user-space probes. With a single command, it can perform aggregation, distribution, and event tracing.
- Concept: Supports kernel functions, system calls, and user-space probes.
- Key Features: Performs aggregation, distribution, and event tracing with a single command.
- Advantages: Syntax is similar to DTrace (focused on one-liners).
- Usage: Safe for production servers (minimal overhead).
Installation and Download
bpftrace can be installed using a package manager or an official script.
| Environment | Command |
|---|---|
| Ubuntu 22.04+ | sudo apt install bpftrace |
| RHEL 9+ | sudo dnf install bpftrace |
| Arch | sudo pacman -S bpftrace |
| Source Build | github.com/bpftrace/bpftrace |
| Verify | bpftrace --version |
Verify the installation using bpftrace --version.
Commonly Used Options
| Option | Description |
|---|---|
| bpftrace -e '<prog>' | Run an inline script |
| bpftrace <file.bt> | Run a script from a file |
| -l | List available probes |
| BEGIN { } | Execute at start (initialization) |
| END { } | Execute at exit (output aggregation) |
3 Essential Command Combinations
bpftrace -e 'tracepoint:syscalls:sys_enter_openat { @[comm] = count(); }'— Aggregate open calls by processbpftrace -e 'kprobe:do_sys_openat2 { @[probe] = count(); }'— Aggregate kernel probesbpftrace -l 'tracepoint:syscalls:*'— Check available probes
💬 "Show me an example of using bpftrace on a real server"
Here is a step-by-step guide for common situations encountered on live production servers.
Step 1 · Verify Installation
Check if it is installed.
bpftrace --version || sudo apt install -y bpftrace
Step 2 · Verify Probes
Check available probes.
sudo bpftrace -l 'tracepoint:syscalls:sys_enter_openat'
Step 3 · Run Tracing
Aggregate open calls by process.
sudo bpftrace -e 'tracepoint:syscalls:sys_enter_openat { @[comm] = count(); }'
Actual Execution Screen
Here are the results of aggregating open calls by process using bpftrace.
!bpftrace command execution results
Live production server
How It's Used in Practice
1. I/O Bottleneck Tracing
Situation — Identifying the cause of slow disk I/O.
sudo bpftrace -e 'tracepoint:syscalls:sys_enter_openat { @[comm] = count(); }'
Result — postgres has the highest count at 2891.
Point — Find bottlenecks by counting open calls per process.
2. Analysis by File
Situation — Seeing which files a specific process frequently opens.
sudo bpftrace -e 'tracepoint:syscalls:sys_enter_openat /comm == "postgres"/ { @[str(args->filename)] = count(); }'
Result — The file base/16384/1259 has the highest count at 1024.
Point — Allows analysis of DB file access patterns.
Actual Execution Screen
Here are the results of analyzing open calls by file using bpftrace.
!bpftrace command execution results 2
Live production server
Points to Note
- bpftrace requires root privileges, and available probes vary depending on the kernel version.
- Excessive probing can cause kernel overhead. In production, adjust the sampling interval.
- Commercial kernels require BTF (BPF Type Format) support.
Source (HTML): https://games.aidebatehub.com/programs/linux/troubleshooting/bpftrace.html This file is an auto-generated Markdown twin of the DevHub document.
Visitor Comments