SystemTap provides an infrastructure to simplify the gathering of information about the running Linux kernel or userspace programs. SystemTap scripts are compiled into loadable kernel modules and can instrument the execution of functions or statements in the kernel or user-space.

Quick start

Install systemtap as well as the kernel headers for your currently running kernel:

apt install systemtap linux-headers-$(uname -r)

Try out this hello world one-liner:

stap -v -e 'probe oneshot { println("hello world") }'

Inspect some user-space code, for example coreutils, by installing the relevant debug symbols:

apt install coreutils-dbgsym

If the debug symbols package cannot be found, add the debug packages archive to your apt sources, run apt update and then try to install the package again.

List the probe points matching a certain pattern:

sudo stap -L 'process("/bin/ls").function("*user*")'

See when a given function gets called:

stap -e 'probe process("/bin/ls").function("format_user") { printf("format_user(uid=%d)\n", $u) }'

Now try running ls -l /etc/passwd, and you should see output from your SystemTap probe.

Finally, to instrument the Linux kernel, install the debug symbols (warning! that's a big package):

apt install systemtap linux-image-$(uname -r)-dbg

You can now instrument kernel functions, for example:

stap -ve 'probe kernel.function("icmp_reply") { println("icmp reply") }'

See also

There is a guide on how to configure SystemTap on a Debian system in SystemTap's Wiki.