bpftrace-gethostlatency.bt

title: bpftrace-gethostlatency.bt(8)

text:
# bpftrace-gethostlatency.bt(8)

## NAME
gethostlatency.bt - Show latency for getaddrinfo/gethostbyname[2] calls. Uses bpftrace/eBPF.

## SYNOPSIS
gethostlatency.bt

## DESCRIPTION
This traces and prints when getaddrinfo(), gethostbyname(), and gethostbyname2() are called, system wide, and shows the responsible PID and command name, latency of the call (duration) in milliseconds, and the host string.

This tool can be useful for identifying DNS latency, by identifying which remote host name lookups were slow, and by how much.

This tool currently uses dynamic tracing of user-level functions and registers, and may need modifications to match your software and processor architecture.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bcc.

## EXAMPLES
Trace host lookups (getaddrinfo/gethostbyname[2]) system wide:

\# gethostlatency.bt

## FIELDS
TIME

Time of the command (HH:MM:SS).

PID

Process ID of the client performing the call.

COMM

Process (command) name of the client performing the call.

LATms

Latency of the call, in milliseconds.

HOST

Host name string: the target of the lookup.

## OVERHEAD
The rate of lookups should be relatively low, so the overhead is not expected to be a problem.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool provides command line options.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
tcpdump(8)


bpftrace-killsnoop.bt

package: bpftrace

title: bpftrace-killsnoop.bt(8)

text:
# bpftrace-killsnoop.bt(8)

## NAME
killsnoop.bt - Trace signals issued by the kill() syscall. Uses bpftrace/eBPF.

## SYNOPSIS
killsnoop.bt

## DESCRIPTION
killsnoop traces the kill() syscall, to show signals sent via this method. This may be useful to troubleshoot failing applications, where an unknown mechanism is sending signals.

This works by tracing the kill() syscall tracepoints.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace all kill() syscalls:

\# killsnoop.bt

## FIELDS
TIME

Time of the kill call.

PID

Source process ID

COMM

Source process name

SIG

Signal number. See signal(7).

TPID

Target process ID

RES

Result. 0 == success, a negative value (of the error code) for failure.

## OVERHEAD
This traces the kernel kill function and prints output for each event. As the rate of this is generally expected to be low (< 100/s), the overhead is also expected to be negligible. If you have an application that is calling a very high rate of kill()s for some reason, then test and understand overhead before use.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool may provide more options and customizations.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
opensnoop.bt(8)


bpftrace-loads.bt

title: bpftrace-loads.bt(8)

text:
# bpftrace-loads.bt(8)

## NAME
loads.bt - Prints load averages. Uses bpftrace/eBPF.

## SYNOPSIS
loads.bt

## DESCRIPTION
These are the same load averages printed by "uptime", but to three decimal places instead of two (not that it really matters). This is really a demonstration of fetching and processing a kernel structure from bpftrace.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Print system load averages every second:

\# loads.bt

## FIELDS
HH:MM:SS

Each output line includes time of printing in "HH:MM:SS" format.

load averages:

These are exponentially-damped moving sum averages of the system loads. Load is a measurement of demand on system resources, which include CPUs and other resources that are accessed with the kernel in an uninterruptible state (TASK_UNINTERRUPTIBLE), which includes types of disk I/O and lock accesses. Linux load averages originally reflected CPU demand only, as it does in other OSes, but this was changed in Linux 0.99.14. This demand measurement reflects not just the utilized resource, but also the queued demand (a saturation measurement). Finally, the three numbers are called the "one-", "five-", and "fifteen-minute" load averages, however these times are constants used in the exponentially-damping equation, and the load averages reflect load beyond these times. Were you expecting an accurate description of load averages in the man page of a bpftrace tool?

## OVERHEAD
Other than bpftrace startup time, negligible.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

## REFERENCE
For more on load averages, see:

http://www.brendangregg.com/blog/2017-08-08/linux-load-averages.html

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
uptime(1)


bpftrace-mdflush.bt

title: bpftrace-mdflush.bt(8)

text:
# bpftrace-mdflush.bt(8)

## NAME
mdflush.bt - Trace md flush events. Uses bpftrace/eBPF.

## SYNOPSIS
mdflush.bt

## DESCRIPTION
This tool traces flush events by md, the Linux multiple device driver (software RAID). The timestamp and md device for the flush are printed. Knowing when these flushes happen can be useful for correlation with unexplained spikes in disk latency.

This works by tracing the kernel md_flush_request() function using kernel dynamic tracing, and will need updating to match any changes to this function.

Note that the flushes themselves are likely to originate from higher in the I/O stack, such as from the file systems.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace md flush events:

\# mdflush.bt

## FIELDS
TIME

Time of the flush event (HH:MM:SS).

PID

The process ID that was on-CPU when the event was issued. This may identify the cause of the flush (eg, the "sync" command), but will often identify a kernel worker thread that was managing I/O.

COMM

The command name for the PID.

DEVICE

The md device name.

## OVERHEAD
Expected to be negligible.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool may provide more options and customizations.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
biosnoop.bt(8)


bpftrace-naptime.bt

title: bpftrace-naptime.bt(8)

text:
# bpftrace-naptime.bt(8)

## NAME
naptime.bt - Trace voluntary sleep calls. Uses bpftrace/eBPF.

## SYNOPSIS
naptime.bt

## DESCRIPTION
This tool traces application sleeps, and can be used for debugging high latency that may be caused by deliberate sleeps placed in application routines, especially administration scripts.

This tool works by tracing the nanosleep(2) syscall using the syscall tracepoints.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace application sleeps via nanosleep(2):

\# naptime.bt

## FIELDS
TIME

A timestamp in HH:MM:SS format.

PPID

Parent process ID.

PCOMM

Parent process name.

PID

The sleeping process ID.

COMM

The sleeping process name.

SECONDS

The requested duration of the sleep.

## OVERHEAD
nanosleep(2) calls are expected to be low frequency (<< 100/s), so the overhead of this tool is expected to be negligible.

## SOURCE
This tool originated from the book "BPF Performance Tools", published by Addison Wesley (2019):

http://www.brendangregg.com/bpf-performance-tools-book.html

See the book for more documentation on this tool.

This version is in the bpftrace repository:

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
capable.bt(8)


bpftrace-oomkill.bt

title: bpftrace-oomkill.bt(8)

summary: Trace OOM killer. Uses bpftrace/eBPF.

text:
# bpftrace-oomkill.bt(8)

## NAME
oomkill.bt - Trace OOM killer. Uses bpftrace/eBPF.

## SYNOPSIS
oomkill.bt

## DESCRIPTION
This traces the kernel out-of-memory killer, and prints basic details, including the system load averages at the time of the OOM kill. This can provide more context on the system state at the time: was it getting busier or steady, based on the load averages? This tool may also be useful to customize for investigations; for example, by adding other task_struct details at the time of OOM, or by adding other commands to run at the shell.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace OOM kill events:

\# oomkill.bt

## FIELDS
Triggered by ...

The process ID and process name of the task that was running when another task was OOM killed.

OOM kill of ...

The process ID and name of the target process that was OOM killed.

loadavg

Contents of /proc/loadavg. The first three numbers are 1, 5, and 15 minute load averages (where the average is an exponentially damped moving sum, and those numbers are constants in the equation); then there is the number of running tasks, a slash, and the total number of tasks; and then the last number is the last PID to be created.

## OVERHEAD
Negligible.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool may provide more options and customizations.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
dmesg(1)


bpftrace-opensnoop.bt

title: bpftrace-opensnoop.bt(8)

text:
# bpftrace-opensnoop.bt(8)

## NAME
opensnoop.bt - Trace open() syscalls. Uses bpftrace/eBPF.

## SYNOPSIS
opensnoop.bt

## DESCRIPTION
opensnoop traces the open() syscall, showing which processes are attempting to open which files. This can be useful for determining the location of config and log files, or for troubleshooting applications that are failing, specially on startup.

This works by tracing the open() syscall tracepoint.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bcc.

## EXAMPLES
Trace all open() syscalls:

\# opensnoop.bt

## FIELDS
PID Process ID

TID

Thread ID

COMM

Process name

FD

File descriptor (if success), or -1 (if failed)

ERR

Error number (see the system's errno.h)

PATH

Open path

## OVERHEAD
This traces the open tracepoint and prints output for each event. As the rate of this is generally expected to be low (< 1000/s), the overhead is also expected to be negligible. If you have an application that is calling a high rate of open()s, then test and understand overhead before use.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool may provide more options and customizations.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
statsnoop.bt(8), execsnoop.bt(8)


bpftrace-pidpersec.bt

title: bpftrace-pidpersec.bt(8)

text:
# bpftrace-pidpersec.bt(8)

## NAME
pidpersec.bt - Count new processes (via fork()). Uses bpftrace/eBPF.

## SYNOPSIS
pidpersec.bt

## DESCRIPTION
pidpersec shows how many new processes were created each second. There can be performance issues caused by many short-lived processes, which may not be visible in sampling tools like top(1). pidpersec provides one way to investigate this behavior.

This works by tracing the tracepoint:sched:sched_process_fork tracepoint.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Count new processes, printing per-second summaries until Ctrl-C is hit:

\# pidpersec.bt

## FIELDS
1st

Count of processes (after "@")

## OVERHEAD
This traces kernel forks, and maintains an in-kernel count which is read asynchronously from user-space. As the rate of this is generally expected to be low (<< 1000/s), the overhead is also expected to be negligible.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool may provide more options and customizations.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
top(1)


bpftrace-runqlat.bt

title: bpftrace-runqlat.bt(8)

text:
# bpftrace-runqlat.bt(8)

## NAME
runqlat.bt - CPU scheduler run queue latency as a histogram. Uses bpftrace/eBPF.

## SYNOPSIS
runqlat.bt

## DESCRIPTION
This traces time spent waiting in the CPU scheduler for a turn on-CPU. This metric is often called run queue latency, or scheduler latency. This tool shows this latency as a power-of-2 histogram in nanoseconds, allowing multimodal distributions to be studied, as well as latency outliers.

This tool uses the sched tracepoints.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace CPU run queue latency system wide, printing a histogram on Ctrl-C:

\# runqlat.bt

## FIELDS
1st, 2nd

This is a range of latency, in microseconds (shown in "[...)" set notation).

3rd

A column showing the count of scheduler events in this range.

4th

This is an ASCII histogram representing the count column.

## OVERHEAD
This traces scheduler functions, which can become very frequent. While eBPF has very low overhead, and this tool uses in-kernel maps for efficiency, the frequency of scheduler events for some workloads may be high enough that the overhead of this tool becomes significant. Measure in a lab environment to quantify the overhead before use.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool may provide more options and customizations.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
runqlen.bt(8), mpstat(1), pidstat(1), uptime(1)


bpftrace-runqlen.bt

title: bpftrace-runqlen.bt(8)

text:
# bpftrace-runqlen.bt(8)

## NAME
runqlen.bt - CPU scheduler run queue length as a histogram. Uses bpftrace/eBPF.

## SYNOPSIS
runqlen.bt

## DESCRIPTION
This program summarizes scheduler queue length as a histogram, and can also show run queue occupancy. It works by sampling the run queue length on all CPUs at 99 Hertz.

This tool can be used to identify imbalances, eg, when processes are bound to CPUs causing queueing, or interrupt mappings causing the same.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace CPU run queue length system wide, printing a histogram on Ctrl-C:

\# runqlen.bt

## FIELDS
1st, 2nd

The run queue length is shown in the first field (after "[").

3rd

A column showing the count of samples in for that length.

4th

This is an ASCII histogram representing the count column.

## OVERHEAD
This samples scheduler structs at 99 Hertz across all CPUs. Relatively, this is a low rate of events, and the overhead of this tool is expected to be near zero.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool may provide more options and customizations.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
runqlat.bt(8), mpstat(1), pidstat(1), uptime(1)


bpftrace-setuids.bt

title: bpftrace-setuids.bt(8)

text:
# bpftrace-setuids.bt(8)

## NAME
setuids.bt - Trace setuid family of syscalls. Uses bpftrace/eBPF.

## SYNOPSIS
setuids.bt

## DESCRIPTION
This tool traces privilege escalation via setuid syscalls, and can be used for debugging, whitelist creation, and intrusion detection.

It works by tracing the setuid(2), setfsuid(2), and retresuid(2) syscalls using the syscall tracepoints.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace setuid syscalls:

\# setuids.bt

## FIELDS
PID

The calling process ID.

COMM

The calling process (thread) name.

UID

The UID of the caller.

SYSCALL

The syscall name.

ARGS

The arguments to the syscall

(RET)

The return value for the syscall: 0 == success, other numbers indicate an error code.

## OVERHEAD
setuid calls are expected to be low frequency (<< 100/s), so the overhead of this tool is expected to be negligible.

## SOURCE
This tool originated from the book "BPF Performance Tools", published by Addison Wesley (2019):

http://www.brendangregg.com/bpf-performance-tools-book.html

See the book for more documentation on this tool.

This version is in the bpftrace repository:

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
capable.bt(8)


bpftrace-ssllatency.bt

title: bpftrace-ssllatency.bt(8)

text:
# bpftrace-ssllatency.bt(8)

## NAME
ssllatency.bt - Show SSL/TLS handshake latency histogram. Uses bpftrace/eBPF.

## SYNOPSIS
ssllatency.bt

## DESCRIPTION
ssllatency shows latency distribution for OpenSSL handshake functions. This is useful for performance analysis with different crypto cipher suite, async SSL acceleration by CPU or offload device, etc.

This tool works by dynamic tracing the uprobes in OpenSSL and related crypto libs, and may need updating to match future changes to these functions.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace SSL/TLS handshake latency, and print a histogram on Ctrl-C:

\# ssllatency.bt

## FIELDS
0th

A function name is shown in "@hist[...]" followed by latency histogram and "@stat[...]" followed by total call count, average, total latency in microseconds. Non-zero failed calls are traced separately (in "@histF[]" and "@statF[]") for some functions.

1st, 2nd

This is a range of latency, in microseconds (shown in "[...)" set notation).

3rd

A column showing the count of operations in this range.

4th

This is an ASCII histogram representing the count column.

## OVERHEAD
SSL/TLS handshake usually contains network latency and the traced crypto functions are CPU intensive tasks, so call frequency should be low and the overhead of this tool is expected to be negligible.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

There is a bcc tool sslsniff that can show SSL/TLS handshake event latency before sniffing the plaintext in SSL_read/write. This tool provides more detailed crypto latency distribution during the handshake event.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Tao Xu

## SEE ALSO
sslsnoop.bt(8)


bpftrace-sslsnoop.bt

title: bpftrace-sslsnoop.bt(8)

text:
# bpftrace-sslsnoop.bt(8)

## NAME
sslsnoop.bt - Show SSL/TLS handshake events. Uses bpftrace/eBPF.

## SYNOPSIS
sslsnoop.bt

## DESCRIPTION
sslsnoop traces OpenSSL handshake functions, and shows latency and return value. This can be used to analyze SSL/TLS performance.

This tool works by dynamic tracing the uprobes in OpenSSL and related crypto libs, and may need updating to match future changes to these functions.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace SSL/TLS handshake events, printing per-line summaries:

\# sslsnoop.bt

## FIELDS
TIME(us)

Time of the call completion, in microseconds since program start.

TID

Thread ID.

COMM

Process name.

LAT(us)

Latency of the call, in microseconds.

RET

Return value of the call.

FUNC

Function name.

## OVERHEAD
SSL/TLS handshake usually contains network latency and the traced crypto functions are CPU intensive tasks, so call frequency should be low and the overhead of this tool is expected to be negligible.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

There is a bcc tool sslsniff that can show SSL/TLS handshake event latency before sniffing the plaintext in SSL_read/write. This tool provides more detailed crypto latency distribution during the handshake event.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Tao Xu

## SEE ALSO
biosnoop.bt(8)


bpftrace-statsnoop.bt

title: bpftrace-statsnoop.bt(8)

text:
# bpftrace-statsnoop.bt(8)

## NAME
statsnoop.bt - Trace stat() syscalls. Uses bpftrace/eBPF.

## SYNOPSIS
statsnoop.bt

## DESCRIPTION
statsnoop traces the stat() syscall, showing which processes are attempting to stat which files. This can be useful for determining the location of config and log files, or for troubleshooting applications that are failing, specially on startup.

This traces the tracepoints for statfs(), statx(), newstat(), and newlstat(). These aren't the only the stat syscalls: if you are missing activity, you may need to add more variants.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bcc.

## EXAMPLES
Trace all stat() syscalls:

\# statsnoop.bt

## FIELDS
PID Process ID

TID

Thread ID

COMM

Process name

FD

File descriptor (if success), or -1 (if failed)

ERR

Error number (see the system's errno.h)

PATH

Stat path

## OVERHEAD
This traces the stat tracepoints and prints output for each event. As the rate of this is generally expected to be low (< 1000/s), the overhead is also expected to be negligible. If you have an application that is calling a high rate of stat()s, then test and understand overhead before use.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool may provide more options and customizations.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
opensnoop.bt(8), execsnoop.bt(8)


bpftrace-swapin.bt

title: bpftrace-swapin.bt(8)

text:
# bpftrace-swapin.bt(8)

## NAME
swapin.bt - Count swapins by process. Uses bpftrace/eBPF.

## SYNOPSIS
swapin

## DESCRIPTION
This tool counts swapins by process, to show which process is affected by swapping (if swap devices are in use). This can explain a significant source of application latency, if it has began swapping due to memory pressure on the system.

This works by tracing the swap_readpage() kernel function using dynamic instrumentation. This tool may need maintenance to keep working if that function changes in later kernels.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Count swapins by process, showing per-second summaries.

\# swapin.bt

## FIELDS
1st

The process name.

2nd

The process ID.

3rd

The count of swapins during that interval.

## OVERHEAD
The rate of swapins should be low (bounded by swapin device IOPS), such that the overhead of this tool is expected to be negligible.

## SOURCE
This tool originated from the book "BPF Performance Tools", published by Addison Wesley (2019):

http://www.brendangregg.com/bpf-performance-tools-book.html

See the book for more documentation on this tool.

This version is in the bpftrace repository:

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
swapon(8)


bpftrace-syncsnoop.bt

title: bpftrace-syncsnoop.bt(8)

text:
# bpftrace-syncsnoop.bt(8)

## NAME
syncsnoop.bt - Trace the sync() variety of syscalls. Uses bpftrace/eBPF.

## SYNOPSIS
syncsnoop.bt

## DESCRIPTION
syncsnoop traces calls to sync() syscalls (sync(), fsync(), msync(), etc), which flushes file system cache and buffers to storage devices. These calls can cause performance perturbations, and it can be useful to know if they are happening, when they happen, and how frequently.

This works by tracing the sync() variety of syscalls via tracepoints.

This program is also a basic example of eBPF/bcc.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace calls to sync() syscalls:

\# syncsnoop.bt

## FIELDS
TIME

A timestamp on the output, in "HH:MM:SS" format.

PID

The process ID that was on-CPU during the event.

COMM

The process name that was on-CPU during the event.

EVENT

The tracepoint name for the sync event.

## OVERHEAD
This traces sync syscalls and prints output for each event. As the rate of this is generally expected to be low (<< 100/s), the overhead is also expected to be negligible.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
iostat(1)


bpftrace-syscount.bt

title: bpftrace-syscount.bt(8)

summary: Count system calls. Uses bpftrace/eBPF.

text:
# bpftrace-syscount.bt(8)

## NAME
syscount.bt - Count system calls. Uses bpftrace/eBPF.

## SYNOPSIS
syscount.bt

## DESCRIPTION
This counts system calls (syscalls), printing a summary of the top ten syscall IDs, and the top ten process names making syscalls. This can be helpful for characterizing the kernel and resource workload, and finding applications who are using syscalls inefficiently.

This works by using the tracepoint:raw_syscalls:sys_enter tracepoint.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Count system calls until Ctrl-C is hit:

\# syscount.bt

## OUTPUT
Top 10 syscalls IDs:

This shows the syscall ID number (in @syscall[]) followed by a count for this syscall during tracing. To see the syscall name for that ID, you can use "ausyscall --dump", or the bcc version of this tool that does translations.

Top 10 processes:

This shows the process name (in @process[]) followed by a count of syscalls during tracing.

## OVERHEAD
For most applications, the overhead should be manageable if they perform 1000's or even 10,000's of syscalls per second. For higher rates, the overhead may become considerable. For example, tracing a microbenchmark loop of 4 million calls to geteuid(), slows it down by 2.4x. However, this represents tracing a workload that has a syscall rate of over 4 million syscalls per second per CPU, which should not be typical (in one large cloud production environment, rates of between 10k and 50k are typical, where the application overhead is expected to be closer to 1%).

For comparison, strace(1) in its current ptrace-based implementation (which it has had for decades) runs the same geteuid() workload 102x slower (that's one hundred and two times slower).

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc version provides different command line options, and translates the syscall IDs to their syscall names.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
strace(1)


bpftrace-tcpaccept.bt

title: bpftrace-tcpaccept.bt(8)

text:
# bpftrace-tcpaccept.bt(8)

## NAME
tcpaccept.bt - Trace TCP passive connections (accept()). Uses bpftrace/eBPF

## SYNOPSIS
tcpaccept.bt

## DESCRIPTION
This tool traces passive TCP connections (eg, via an accept() syscall; connect() are active connections). This can be useful for general troubleshooting to see what new connections the local server is accepting.

This uses dynamic tracing of the kernel inet_csk_accept() socket function (from tcp_prot.accept), and will need to be modified to match kernel changes.

This tool only traces successful TCP accept()s. Connection attempts to closed ports will not be shown (those can be traced via other functions).

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace all passive TCP connections (accept()s):

\# tcpaccept.bt

## FIELDS
TIME(s)

Time of the call, in HH:MM:SS format.

PID

Process ID

COMM

Process name

RADDR

Remote IP address.

RPORT

Remote port.

LADDR

Local IP address.

LPORT

Local port

BL

Current accept backlog vs maximum backlog

## OVERHEAD
This traces the kernel inet_csk_accept function and prints output for each event. The rate of this depends on your server application. If it is a web or proxy server accepting many tens of thousands of connections per second, then the overhead of this tool may be measurable (although, still a lot better than tracing every packet). If it is less than a thousand a second, then the overhead is expected to be negligible. Test and understand this overhead before use.

## SOURCE
This is from bpftrace

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool may provide more options and customizations.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg, adapted for bpftrace by Dale Hamel

## SEE ALSO
tcpconnect.bt(8), funccount(8), tcpdump(8)


bpftrace-tcpconnect.bt

title: bpftrace-tcpconnect.bt(8)

text:
# bpftrace-tcpconnect.bt(8)

## NAME
tcpconnect.bt - Trace TCP active connections (connect()). Uses Linux bpftrace/eBPF

## SYNOPSIS
tcpconnect.bt

## DESCRIPTION
This tool traces active TCP connections (eg, via a connect() syscall; accept() are passive connections). This can be useful for general troubleshooting to see what connections are initiated by the local server.

All connection attempts are traced, even if they ultimately fail.

This works by tracing the kernel tcp_v4_connect() and tcp_v6_connect() functions using dynamic tracing, and will need updating to match any changes to these functions.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace all active TCP connections:

\# tcpconnect.bt

## FIELDS
TIME(s)

Time of the call, in HH:MM:SS format.

PID

Process ID

COMM

Process name

SADDR

Source IP address.

SPORT

Source port.

DADDR

Destination IP address.

DPORT

Destination port

## OVERHEAD
This traces the kernel tcp_v[46]_connect functions and prints output for each event. As the rate of this is generally expected to be low (< 1000/s), the overhead is also expected to be negligible. If you have an application that is calling a high rate of connects()s, such as a proxy server, then test and understand this overhead before use.

## SOURCE
This is from bpftrace

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool may provide more options and customizations.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg, adapted for bpftrace by Dale Hamel

## SEE ALSO
tcpaccept.bt(8), funccount(8), tcpdump(8)


bpftrace-tcpdrop.bt

title: bpftrace-tcpdrop.bt(8)

text:
# bpftrace-tcpdrop.bt(8)

## NAME
tcpdrop.bt - Trace kernel-based TCP packet drops with details. Uses Linux bpftrace/eBPF

## SYNOPSIS
tcpdrop.bt

## DESCRIPTION
This tool traces TCP packets or segments that were dropped by the kernel, and shows details from the IP and TCP headers, the socket state, and the kernel stack trace. This is useful for debugging cases of high kernel drops, which can cause timer-based retransmits and performance issues.

This tool works using dynamic tracing of the tcp_drop() kernel function, which requires a recent kernel version.

This tool is limited to ipv4, and cannot parse tcpflags as bpftrace currently cannot parse socket buffers in the way that bcc can.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace all tcp drops:

\# tcpdrop.bt

## FIELDS
TIME

Time of the call, in HH:MM:SS format.

PID

Process ID that was on-CPU during the drop. This may be unrelated, as drops can occur on the receive interrupt and be unrelated to the PID that was interrupted.

COMM

Process name

SADDR

Source IP address.

SPORT

Source TCP port.

DADDR

Destination IP address.

DPORT

Destination TCP port.

STATE

TCP session state ("ESTABLISHED", etc).

## OVERHEAD
This traces the kernel tcp_drop() function, which should be low frequency, and therefore the overhead of this tool should be negligible.

As always, test and understand this tools overhead for your types of workloads before production use.

## SOURCE
This is from bpftrace

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool may provide more options and customizations.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg, adapted for bpftrace by Dale Hamel

## SEE ALSO
tcplife.bt(8), tcpaccept.bt(8), tcpconnect.bt(8), tcptop(8)


bpftrace-tcplife.bt

title: bpftrace-tcplife.bt(8)

text:
# bpftrace-tcplife.bt(8)

## NAME
tcplife.bt - Trace TCP session lifespans with connection details. Uses bpftrace/eBPF.

## SYNOPSIS
tcplife

## DESCRIPTION
This tool shows the lifespan of TCP sessions that open and close while tracing, and shows the duration and throughput statistics. For efficiency, this tool only instruments TCP state changes, rather than all packets.

This tool works by using the sock:inet_sock_set_state tracepoint, which was added in Linux 4.16.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF, bpftrace, and the sock:inet_sock_set_state tracepoint (Linux 4.16+).

## EXAMPLES
Show TCP sessions with details:

\# tcplife.bt

## FIELDS
PID

Process ID

COMM

Process name

LADDR

Local IP address.

DADDR

Remote IP address.

LPORT

Local port.

RPORT

Remote port.

TX_KB

Total transmitted Kbytes.

RX_KB

Total received Kbytes.

MS

Lifespan of the session, in milliseconds.

## OVERHEAD
This traces the kernel TCP set state function, which should be called much less often than send/receive tracing, and therefore have lower overhead. The overhead of the tool is relative to the rate of new TCP sessions: if this is high, over 10,000 per second, then there may be noticeable overhead just to print out 10k lines of formatted output per second.

You can find out the rate of new TCP sessions using "sar -n TCP 1", and adding the active/s and passive/s columns.

As always, test and understand this tools overhead for your types of workloads before production use.

## SOURCE
This tool originated from BCC:

https://github.com/iovisor/bcc

The BCC version has many command line options for customizing the output.

This bpftrace version originated from the book "BPF Performance Tools", published by Addison Wesley (2019):

http://www.brendangregg.com/bpf-performance-tools-book.html

See the book for more documentation on this tool.

This bpftrace version is in the bpftrace repository:

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
tcptop(8)


bpftrace-tcpretrans.bt

title: bpftrace-tcpretrans.bt(8)

text:
# bpftrace-tcpretrans.bt(8)

## NAME
tcpretrans.bt - Trace or count TCP retransmits. Uses Linux bpftrace/eBPF

## SYNOPSIS
tcpretrans.bt

## DESCRIPTION
This traces TCP retransmits, showing address, port, and TCP state information, and sometimes the PID (although usually not, since retransmits are usually sent by the kernel on timeouts). To keep overhead very low, only the TCP retransmit functions are traced. This does not trace every packet (like tcpdump(8) or a packet sniffer). Optionally, it can count retransmits over a user signalled interval to spot potentially dropping network paths the flows are traversing.

This uses dynamic tracing of the kernel tcp_retransmit_skb() and tcp_send_loss_probe() functions, and will need to be updated to match kernel changes to these functions.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bcc. CONFIG_BPF and bpftrace.

## EXAMPLES
Trace TCP retransmits:

\# tcpretrans.bt

## FIELDS
TIME

Time of the call, in HH:MM:SS format.

PID

Process ID that was on-CPU. This is less useful than it might sound, as it may usually be 0, for the kernel, for timer-based retransmits.

LADDR

Local IP address.

LPORT

Local port.

RADDR

Remote IP address.

RPORT

Remote port.

STATE

TCP session state.

## OVERHEAD
Should be negligible: TCP retransmit events should be low (<1000/s), and the low overhead this tool adds to each event should make the cost negligible.

## SOURCE
This is from bpftrace

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool may provide more options and customizations.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg, adapted for bpftrace by Dale Hamel

## SEE ALSO
tcpconnect.bt(8), tcpaccept.bt(8)


bpftrace-tcpsynbl.bt

title: bpftrace-tcpsynbl.bt(8)

text:
# bpftrace-tcpsynbl.bt(8)

## NAME
tcpsynbl.bt - Show the TCP SYN backlog as a histogram. Uses bpftrace/eBPF.

## SYNOPSIS
tcpsynbl

## DESCRIPTION
This tool shows the TCP SYN backlog size during SYN arrival as a histogram. This lets you see how close your applications are to hitting the backlog limit and dropping SYNs (causing performance issues with SYN retransmits), and is a measure of workload saturation. The histogram shown is measured at the time of SYN received, and a separate histogram is shown for each backlog limit.

This works by tracing the tcp_v4_syn_recv_sock() and tcp_v6_syn_recv_sock() kernel functions using dynamic instrumentation. Since these functions may change in future kernels, this tool may need maintenance to keep working.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Show the TCP SYN backlog as a histogram.

\# tcpsynbl.bt

## FIELDS
backlog

The backlog size when a SYN was received.

count

The number of times this backlog size was encountered.

distribution

An ASCII visualization of the count column.

## OVERHEAD
Inbound SYNs should be relatively low compared to packets and other events, so the overhead of this tool is expected to be negligible.

## SOURCE
This tool originated from the book "BPF Performance Tools", published by Addison Wesley (2019):

http://www.brendangregg.com/bpf-performance-tools-book.html

See the book for more documentation on this tool.

This version is in the bpftrace repository:

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
tcptop(8)


bpftrace-threadsnoop.bt

title: bpftrace-threadsnoop.bt(8)

text:
# bpftrace-threadsnoop.bt(8)

## NAME
threadsnoop.bt - Trace thread creation via pthread_create(). Uses bpftrace/eBPF.

## SYNOPSIS
threadsnoop.bt

## DESCRIPTION
threadsnoop traces calls to pthread_create(), showing this path of thread creation. This can be used for workload characterization and discovery, and is a companion to execsnoop.bt(8) which traces execve(2).

This works by tracing the pthread_create() from libpthread.so.0. The path to this library may need adjusting in the tool source to match your system.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace calls pthread_create():

\# threadsnoop.bt

## FIELDS
TIME(ms)

Elapsed time since the tool began tracing (in milliseconds).

PID

The process ID.

COMM

The process (thread) name.

FUNC

The name of the start routine, if the symbol is available, else a hex address for the start routine address.

## OVERHEAD
Thread creation is expected to be low (<< 1000/s), so the overhead of this tool is expected to be negligible.

## SOURCE
This tool originated from the book "BPF Performance Tools", published by Addison Wesley (2019):

http://www.brendangregg.com/bpf-performance-tools-book.html

See the book for more documentation on this tool.

This version is in the bpftrace repository:

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
execsnoop.bt(8)


bpftrace-undump.bt

title: bpftrace-undump.bt(8)

text:
# bpftrace-undump.bt(8)

## NAME
undump.bt - Catch UNIX domain socket packages. Uses bpftrace/eBPF.

## SYNOPSIS
undump.bt

## DESCRIPTION
undump.bt tracked reception of UNIX domain sockets.

This program is also a basic example of bpftrace and kprobes.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace reception of UNIX domain sockets:

\# undump.bt

## FIELDS
TIME

A timestamp on the output, in "HH:MM:SS" format.

COMM

The process COMM.

PID

The process ID.

SIZE

The size of the received packet, in bytes.

DATA

Display received packets in hex or string.

## OVERHEAD
The overhead of this program mainly comes from the data packets received by the terminal output.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc examples/tracing of the same name. The bcc tool may provide more options and customizations.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Rong Tao

## SEE ALSO
opensnoop.bt(8)


bpftrace-vfscount.bt

title: bpftrace-vfscount.bt(8)

summary: Count VFS calls (vfs_*). Uses bpftrace/eBPF.

text:
# bpftrace-vfscount.bt(8)

## NAME
vfscount.bt - Count VFS calls ("vfs_*"). Uses bpftrace/eBPF.

## SYNOPSIS
vfscount.bt

## DESCRIPTION
This counts VFS calls. This can be useful for general workload characterization of these operations.

This works by tracing all kernel functions beginning with "vfs_" using dynamic tracing. This may match more functions than you are interested in measuring: Edit the script to customize which functions to trace.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Count all VFS calls until Ctrl-C is hit:

\# vfscount.bt

## FIELDS
1st

Kernel function name (in @[])

2nd

Number of calls while tracing

## OVERHEAD
This traces kernel vfs functions and maintains in-kernel counts, which are asynchronously copied to user-space. While the rate of VFS operations can be very high (>1M/sec), this is a relatively efficient way to trace these events, and so the overhead is expected to be small for normal workloads. Measure in a test environment, and if overheads are an issue, edit the script to reduce the types of vfs functions traced (currently all beginning with "vfs_").

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool may provide more options and customizations.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
vfsstat.bt(8)


bpftrace-vfsstat.bt

title: bpftrace-vfsstat.bt(8)

text:
# bpftrace-vfsstat.bt(8)

## NAME
vfsstat.bt - Count key VFS calls. Uses bpftrace/eBPF.

## SYNOPSIS
vfsstat.bt

## DESCRIPTION
This traces some common VFS calls and prints per-second summaries. This can be useful for general workload characterization, and looking for patterns in operation usage over time.

This works by tracing some kernel vfs functions using dynamic tracing, and will need updating to match any changes to these functions. Edit the script to customize which functions are traced. Also see vfscount, which is more easily customized to trace multiple functions.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Count some VFS calls, printing per-second summaries until Ctrl-C is hit:

\# vfsstat.bt

## FIELDS
HH:MM:SS

Each output summary is prefixed by the time of printing in "HH:MM:SS" format.

1st

Kernel function name (in @[])

2nd

Number of calls while tracing

## OVERHEAD
This traces various kernel vfs functions and maintains in-kernel counts, which are asynchronously copied to user-space. While the rate of VFS operations can be very high (>1M/sec), this is a relatively efficient way to trace these events, and so the overhead is expected to be small for normal workloads. Measure in a test environment.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool may provide more options and customizations.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
vfscount.bt(8)


bpftrace-writeback.bt

title: bpftrace-writeback.bt(8)

text:
# bpftrace-writeback.bt(8)

## NAME
writeback.bt - Trace file system writeback events with details. Uses bpftrace/eBPF.

## SYNOPSIS
writeback.bt

## DESCRIPTION
This traces when file system dirtied pages are flushed to disk by kernel writeback, and prints details including when the event occurred, and the duration of the event. This can be useful for correlating these times with other performance problems, and if there is a match, it would be a clue that the problem may be caused by writeback. How quickly the kernel does writeback can be tuned: see the kernel docs, eg, vm.dirty_writeback_centisecs.

This uses the tracepoint:writeback:writeback_start and tracepoint:writeback:writeback_written tracepoints.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace all writeback events with timestamps and latency details:

\# writeback.bt

## FIELDS
TIME

Time that the writeback event completed, in %H:%M:%S format.

DEVICE

Device name in major:minor number format.

PAGES

Pages written during writeback.

REASON

Reason for the writeback event. This may be "background", "vmscan", "sync", "periodic", etc.

ms

Duration of the writeback event in milliseconds.

## OVERHEAD
Since writeback events are expected to be infrequent (<10/sec), the overhead of this tool is expected to be negligible (near 0%).

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
biosnoop.bt(8)


bpftrace-xfsdist.bt

title: bpftrace-xfsdist.bt(8)

text:
# bpftrace-xfsdist.bt(8)

## NAME
xfsdist.bt - Summarize XFS operation latency. Uses bpftrace/eBPF.

## SYNOPSIS
xfsdist.bt

## DESCRIPTION
This tool summarizes time (latency) spent in common XFS file operations: reads, writes, opens, and syncs, and presents it as a power-of-2 histogram. It uses an in-kernel eBPF map to store the histogram for efficiency.

Since this works by tracing the xfs_file_operations interface functions, it will need updating to match any changes to these functions.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace XFS operation time, and print a summary on Ctrl-C:

\# xfsdist.bt

## FIELDS
0th

The operation name (shown in "@[...]") is printed before each I/O histogram.

1st, 2nd

This is a range of latency, in microseconds (shown in "[...)" set notation).

3rd

A column showing the count of operations in this range.

4th

This is an ASCII histogram representing the count column.

## OVERHEAD
This adds low-overhead instrumentation to these XFS operations, including reads and writes from the file system cache. Such reads and writes can be very frequent (depending on the workload; eg, 1M/sec), at which point the overhead of this tool may become noticeable. Measure and quantify before use.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool may provide more options and customizations.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
biolatency.bt(8)


compsize

package: btrfs-compsize

title: compsize(8)

text:
# compsize(8)

## NAME
compsize - calculate compression ratio of a set of files on btrfs

## SYNOPSIS
compsize file-or-dir [ file-or-dir ... ]

## DESCRIPTION
compsize takes a list of files on a btrfs filesystem (recursing directories) and measures used compression types and the effective compression ratio. Besides compression, compsize shows the effect of reflinks (cp --reflink, snapshots, deduplication), and certain types of btrfs waste.

The program gives a report similar to:

Processed 90319 files.

Type Perc Disk Usage Uncompressed Referenced

TOTAL 79% 1.4G 1.8G 1.9G

none 100% 1.0G 1.0G 1.0G

lzo 53% 446M 833M 843M

The fields above are:

Type

compression algorithm

Perc

disk usage/uncompressed (compression ratio)

Disk Usage

blocks on the disk; this is what storing these files actually costs you (save for RAID considerations)

Uncompressed

uncompressed extents; what you would need without compression - includes deduplication savings and pinned extent waste

Referenced

apparent file sizes (sans holes); this is what a traditional filesystem that supports holes and efficient tail packing, or tar -S, would need to store these files

Let's see this on an example: a file 128K big is stored as a single extent A which compressed to a single 4K page. It then receives a write of 32K at offset 32K, which also compressed to a single 4K page, stored as extent B.

The file now appears as:

+-------+-------+---------------+

extent A | used | waste | used |

+-------+-------+---------------+

extent B | used |

+-------+

The "waste" inside extent A can't be gotten rid until the whole extent is rewritten (for example by defrag). If compressed, the whole extent needs to be read every time that part of the file is being read, thus the "waste" is still required.

In this case, we have: Disk Usage: 8KB, Uncompressed: 160K, Referenced: 128K.

## OPTIONS
-b/--bytes

Show raw byte counts rather than human-friendly sizes.

-x/--one-file-system

Skip files and directories on different file systems.

## SIGNALS
USR1

Displays partial data for files processed so far.

## CAVEATS
Recently written files may show as not taking any space until they're actually allocated and compressed; this happens once they're synced or on natural writeout, typically on the order of 30 seconds.

The ioctls used by this program require root.

Inline extents are considered to be always unique, even if they share the same bytes on the disk.

This program doesn't currently support filesystems above 8TB on 32-bit machines but neither do other btrfs tools.


btrfs

package: btrfs-progs

title: btrfs(8)

summary: a toolbox to manage btrfs filesystems

text:
# btrfs(8)

## NAME
btrfs - a toolbox to manage btrfs filesystems

## SYNOPSIS
btrfs [global] <group> [<group>...] <command> [<args>]

## DESCRIPTION
The btrfs utility is a toolbox for managing btrfs filesystems. There are command groups to work with subvolumes, devices, for whole filesystem or other specific actions. See section COMMANDS.

There are also standalone tools for some tasks like btrfs-convert(8) or btrfstune(8) that were separate historically and/or haven't been merged to the main utility. See section STANDALONE TOOLS for more details.

For other topics (mount options, etc) please refer to the separate manual page btrfs(5).

## COMMAND SYNTAX
Any command name can be shortened so long as the shortened form is unambiguous, however, it is recommended to use full command names in scripts. All command groups have their manual page named btrfs-<group>.

For example: it is possible to run btrfs sub snaps instead of btrfs subvolume snapshot. But btrfs file s is not allowed, because file s may be interpreted both as filesystem show and as filesystem sync.

If the command name is ambiguous, the list of conflicting options is printed.

Sizes, both upon input and output, can be expressed in either SI or IEC-I units (see numfmt(1)) with the suffix B appended. All numbers will be formatted according to the rules of the C locale (ignoring the shell locale, see locale(7)).

For an overview of a given command use btrfs command --help or btrfs [command...] --help --full to print all available options.

There are global options that are passed between btrfs and the group name and affect behaviour not specific to the command, e.g. verbosity or the type of the output.

--format <format>

if supported by the command, print subcommand output in that format (text, json)

-v|--verbose

increase verbosity of the subcommand

-q|--quiet

print only errors

--log <level>

set log level (default, info, verbose, debug, quiet)

The remaining options are relevant only for the main tool:

--help

print condensed help for all subcommands

--version

print version string

## COMMANDS
balance

Balance btrfs filesystem chunks across single or several devices. See btrfs-balance(8) for details.

check

Do off-line check on a btrfs filesystem. See btrfs-check(8) for details.

device

Manage devices managed by btrfs, including add/delete/scan and so on. See btrfs-device(8) for details.

filesystem

Manage a btrfs filesystem, including label setting/sync and so on. See btrfs-filesystem(8) for details.

inspect-internal

Debug tools for developers/hackers. See btrfs-inspect-internal(8) for details.

property

Get/set a property from/to a btrfs object. See btrfs-property(8) for details.

qgroup

Manage quota group(qgroup) for btrfs filesystem. See btrfs-qgroup(8) for details.

quota

Manage quota on btrfs filesystem like enabling/rescan and etc. See btrfs-quota(8) and btrfs-qgroup(8) for details.

receive

Receive subvolume data from stdin/file for restore and etc. See btrfs-receive(8) for details.

replace

Replace btrfs devices. See btrfs-replace(8) for details.

rescue

Try to rescue damaged btrfs filesystem. See btrfs-rescue(8) for details.

restore

Try to restore files from a damaged btrfs filesystem. See btrfs-restore(8) for details.

scrub

Scrub a btrfs filesystem. See btrfs-scrub(8) for details.

send

Send subvolume data to stdout/file for backup and etc. See btrfs-send(8) for details.

subvolume

Create/delete/list/manage btrfs subvolume. See btrfs-subvolume(8) for details.

## STANDALONE TOOLS
New functionality could be provided using a standalone tool. If the functionality proves to be useful, then the standalone tool is declared obsolete and its functionality is copied to the main tool. Obsolete tools are removed after a long (years) depreciation period.

Tools that are still in active use without an equivalent in btrfs:

btrfs-convert

in-place conversion from ext2/3/4 filesystems to btrfs

btrfstune

tweak some filesystem properties on a unmounted filesystem

btrfs-select-super

rescue tool to overwrite primary superblock from a spare copy

btrfs-find-root

rescue helper to find tree roots in a filesystem

For space-constrained environments, it's possible to build a single binary with functionality of several standalone tools. This is following the concept of busybox where the file name selects the functionality. This works for symlinks or hardlinks. The full list can be obtained by btrfs help --box.

## EXIT STATUS
btrfs returns a zero exit status if it succeeds. Non zero is returned in case of failure.

## AVAILABILITY
btrfs is part of btrfs-progs. Please refer to the documentation at https://btrfs.readthedocs.io.

## SEE ALSO
btrfs(5), btrfs-balance(8), btrfs-check(8), btrfs-convert(8), btrfs-device(8), btrfs-filesystem(8), btrfs-inspect-internal(8), btrfs-property(8), btrfs-qgroup(8), btrfs-quota(8), btrfs-receive(8), btrfs-replace(8), btrfs-rescue(8), btrfs-restore(8), btrfs-scrub(8), btrfs-send(8), btrfs-subvolume(8), btrfstune(8), mkfs.btrfs(8)


btrfs-balance

title: btrfs-balance(8)

text:
# btrfs-balance(8)

## NAME
btrfs-balance - balance block groups on a btrfs filesystem

## SYNOPSIS
btrfs balance <subcommand> <args>

## DESCRIPTION
The primary purpose of the balance feature is to spread block groups across all devices so they match constraints defined by the respective profiles. See mkfs.btrfs(8) section PROFILES for more details. The scope of the balancing process can be further tuned by use of filters that can select the block groups to process. Balance works only on a mounted filesystem. Extent sharing is preserved and reflinks are not broken. Files are not defragmented nor recompressed, file extents are preserved but the physical location on devices will change.

The balance operation is cancellable by the user. The on-disk state of the filesystem is always consistent so an unexpected interruption (e.g. system crash, reboot) does not corrupt the filesystem. The progress of the balance operation is temporarily stored as an internal state and will be resumed upon mount, unless the mount option skip_balance is specified.

WARNING:

Running balance without filters will take a lot of time as it basically move data/metadata from the whole filesystem and needs to update all block pointers.

The filters can be used to perform following actions:

- convert block group profiles (filter convert)

- make block group usage more compact (filter usage)

- perform actions only on a given device (filters devid, drange)

The filters can be applied to a combination of block group types (data, metadata, system). Note that changing only the system type needs the force option. Otherwise system gets automatically converted whenever metadata profile is converted.

When metadata redundancy is reduced (e.g. from RAID1 to single) the force option is also required and it is noted in system log.

NOTE:

The balance operation needs enough work space, i.e. space that is completely unused in the filesystem, otherwise this may lead to ENOSPC reports. See the section ENOSPC for more details.

## COMPATIBILITY
NOTE:

The balance subcommand also exists under the btrfs filesystem namespace. This still works for backward compatibility but is deprecated and should not be used any more.

NOTE:

A short syntax btrfs balance <path> works due to backward compatibility but is deprecated and should not be used any more. Use btrfs balance start command instead.

## PERFORMANCE IMPLICATIONS
Balancing operations are very IO intensive and can also be quite CPU intensive, impacting other ongoing filesystem operations. Typically large amounts of data are copied from one location to another, with corresponding metadata updates.

Depending upon the block group layout, it can also be seek heavy. Performance on rotational devices is noticeably worse compared to SSDs or fast arrays.

## SUBCOMMAND
cancel <path>

cancels a running or paused balance, the command will block and wait until the current block group being processed completes Since kernel 5.7 the response time of the cancellation is significantly improved, on older kernels it might take a long time until currently processed chunk is completely finished.

pause <path>

pause running balance operation, this will store the state of the balance progress and used filters to the filesystem

resume <path>

resume interrupted balance, the balance status must be stored on the filesystem from previous run, e.g. after it was paused or forcibly interrupted and mounted again with skip_balance

start [options] <path>

start the balance operation according to the specified filters, without any filters the data and metadata from the whole filesystem are moved. The process runs in the foreground. NOTE:

The balance command without filters will basically move everything in the filesystem to a new physical location on devices (i.e. it does not affect the logical properties of file extents like offsets within files and extent sharing). The run time is potentially very long, depending on the filesystem size. To prevent starting a full balance by accident, the user is warned and has a few seconds to cancel the operation before it starts. The warning and delay can be skipped with --full-balance option.

Please note that the filters must be written together with the -d, -m and -s options, because they're optional and bare -d and -m also work and mean no filters.

NOTE:

When the target profile for conversion filter is raid5 or raid6, there's a safety timeout of 10 seconds to warn users about the status of the feature

Options

-d[<filters>]

act on data block groups, see section FILTERS for details about filters

-m[<filters>]

act on metadata chunks, see FILTERS for details about filters

-s[<filters>]

act on system chunks (requires -f), see FILTERS for details about filters.

-f

force a reduction of metadata integrity, e.g. when going from raid1 to single, or skip safety timeout when the target conversion profile is raid5 or raid6

--background|--bg

run the balance operation asynchronously in the background, uses fork(2) to start the process that calls the kernel ioctl

--enqueue

wait if there's another exclusive operation running, otherwise continue

-v

(deprecated) alias for global '-v' option

status [-v] <path>

Show status of running or paused balance. Options

-v

(deprecated) alias for global -v option

## FILTERS
From kernel 3.3 onwards, BTRFS balance can limit its action to a subset of the whole filesystem, and can be used to change the replication configuration (e.g. convert data from single to RAID1).

Balance can be limited to a block group profile with the following options:

- -d for data block groups

- -m for metadata block groups (also implicitly applies to -s)

- -s for system block groups

The options have an optional parameter which means that the parameter must start right after the option without a space (this is mandatory getopt syntax), like -dusage=10. Options for all block group types can be specified in one command.

A filter has the following structure: filter[=params][,filter=...]

To combine multiple filters use ,, without spaces. Example: -dconvert=raid1,soft

BTRFS can have different profiles on a single device or the same profile on multiple device.

The main reason why you want to have different profiles for data and metadata is to provide additional protection of the filesystem's metadata when devices fail, since a single sector of unrecoverable metadata will break the filesystem, while a single sector of lost data can be trivially recovered by deleting the broken file.

Before changing profiles, make sure there is enough unallocated space on existing drives to create new metadata block groups (for filesystems over 50GiB, this is 1GB * (number_of_devices + 2)).

Default profiles on BTRFS are:

- data: single

-

metadata:

- single devices: dup

- multiple devices: raid1

The available filter types are:

### Filter types
profiles=<profiles>

Balances only block groups with the given profiles. Parameters are a list of profile names separated by | (pipe).

usage=<percent>, usage=<range>

Balances only block groups with usage under the given percentage. The value of 0 is allowed and will clean up completely unused block groups, this should not require any new work space allocated. You may want to use usage=0 in case balance is returning ENOSPC and your filesystem is not too full. The argument may be a single value or a range. The single value N means at most N percent used, equivalent to ..N range syntax. Kernels prior to 4.4 accept only the single value format. The minimum range boundary is inclusive, maximum is exclusive.

devid=<id>

Balances only block groups which have at least one chunk on the given device. To list devices with ids use btrfs filesystem show.

drange=<range>

Balance only block groups which overlap with the given byte range on any device. Use in conjunction with devid to filter on a specific device. The parameter is a range specified as start..end.

vrange=<range>

Balance only block groups which overlap with the given byte range in the filesystem's internal virtual address space. This is the address space that most reports from btrfs in the kernel log use. The parameter is a range specified as start..end.

convert=<profile>

Convert each selected block group to the given profile name identified by parameters. NOTE:

Starting with kernel 4.5, the data chunks can be converted to/from the DUP profile on a single device. Starting with kernel 4.6, all profiles can be converted to/from DUP on multi-device filesystems.

WARNING:

Bad or missing device are not detected immediately during runtime and this depends on some later event like failed write or failed transaction commit. If there's a known failing device, or a device deleted by /sys/block/<dev>/device/delete interface, the device will be still accessed and written to. In such case, one should not convert to a profile with lower redundancy (e.g. from RAID1 to SINGLE), as attempts to create new chunks on the new devices will cause various problems.

The proper action is to use btrfs replace or btrfs device remove to handle the failing/missing device first. Then convert will work with all devices correctly.

limit=<number>, limit=<range>

Process only given number of chunks, after all filters are applied. This can be used to specifically target a chunk in connection with other filters (drange, vrange) or just simply limit the amount of work done by a single balance run. The argument may be a single value or a range. The single value N means at most N chunks, equivalent to ..N range syntax. Kernels prior to 4.4 accept only the single value format. The range minimum and maximum are inclusive.

stripes=<range>

Balance only block groups which have the given number of stripes. The parameter is a range specified as start..end. Makes sense for block group profiles that utilize striping, i.e. RAID0/10/5/6. The range minimum and maximum are inclusive.

soft

Takes no parameters. Only has meaning when converting between profiles, or When doing convert from one profile to another and soft mode is on, chunks that already have the target profile are left untouched. This is useful e.g. when half of the filesystem was converted earlier but got cancelled. The soft mode switch is (like every other filter) per-type. For example, this means that we can convert metadata chunks the "hard" way while converting data chunks selectively with soft switch.

Profile names, used in profiles and convert are one of:

- raid0

- raid1

- raid1c3

- raid1c4

- raid10

- raid5

- raid6

- dup

- single

The mixed data/metadata profiles can be converted in the same way, but conversion between mixed and non-mixed is not implemented. For the constraints of the profiles please refer to mkfs.btrfs(8) section PROFILES.

## ENOSPC
The way balance operates, it usually needs to temporarily create a new block group and move the old data there, before the old block group can be removed. For that it needs the work space, otherwise it fails for ENOSPC reasons. This is not the same ENOSPC as if the free space is exhausted. This refers to the space on the level of block groups, which are bigger parts of the filesystem that contain many file extents.

The free work space can be calculated from the output of the btrfs filesystem show command:

```
Label: 'BTRFS'  uuid: 8a9d72cd-ead3-469d-b371-9c7203276265

        Total devices 2 FS bytes used 77.03GiB

        devid    1 size 53.90GiB used 51.90GiB path /dev/sdc2

        devid    2 size 53.90GiB used 51.90GiB path /dev/sde1
```

size - used = free work space

53.90GiB - 51.90GiB = 2.00GiB

An example of a filter that does not require workspace is usage=0. This will scan through all unused block groups of a given type and will reclaim the space. After that it might be possible to run other filters.

CONVERSIONS ON MULTIPLE DEVICES

Conversion to profiles based on striping (RAID0, RAID5/6) require the work space on each device. An interrupted balance may leave partially filled block groups that consume the work space.

## EXAMPLES
A more comprehensive example when going from one to multiple devices, and back, can be found in section TYPICAL USECASES of btrfs-device(8).

### MAKING BLOCK GROUP LAYOUT MORE COMPACT
The layout of block groups is not normally visible; most tools report only summarized numbers of free or used space, but there are still some hints provided.

Let's use the following real life example and start with the output:

```
$ btrfs filesystem df /path
Data, single: total=75.81GiB, used=64.44GiB
System, RAID1: total=32.00MiB, used=20.00KiB
Metadata, RAID1: total=15.87GiB, used=8.84GiB
GlobalReserve, single: total=512.00MiB, used=0.00B
```

Roughly calculating for data, 75G - 64G = 11G, the used/total ratio is about 85%. How can we can interpret that:

- chunks are filled by 85% on average, i.e. the usage filter with anything smaller than 85 will likely not affect anything

- in a more realistic scenario, the space is distributed unevenly, we can assume there are completely used chunks and the remaining are partially filled

Compacting the layout could be used on both. In the former case it would spread data of a given chunk to the others and removing it. Here we can estimate that roughly 850 MiB of data have to be moved (85% of a 1 GiB chunk).

In the latter case, targeting the partially used chunks will have to move less data and thus will be faster. A typical filter command would look like:

```
# btrfs balance start -dusage=50 /path
Done, had to relocate 2 out of 97 chunks
$ btrfs filesystem df /path
Data, single: total=74.03GiB, used=64.43GiB
System, RAID1: total=32.00MiB, used=20.00KiB
Metadata, RAID1: total=15.87GiB, used=8.84GiB
GlobalReserve, single: total=512.00MiB, used=0.00B
```

As you can see, the total amount of data is decreased by just 1 GiB, which is an expected result. Let's see what will happen when we increase the estimated usage filter.

```
# btrfs balance start -dusage=85 /path
Done, had to relocate 13 out of 95 chunks
$ btrfs filesystem df /path
Data, single: total=68.03GiB, used=64.43GiB
System, RAID1: total=32.00MiB, used=20.00KiB
Metadata, RAID1: total=15.87GiB, used=8.85GiB
GlobalReserve, single: total=512.00MiB, used=0.00B
```

Now the used/total ratio is about 94% and we moved about 74G - 68G = 6G of data to the remaining block groups, i.e. the 6GiB are now free of filesystem structures, and can be reused for new data or metadata block groups.

We can do a similar exercise with the metadata block groups, but this should not typically be necessary, unless the used/total ratio is really off. Here the ratio is roughly 50% but the difference as an absolute number is "a few gigabytes", which can be considered normal for a workload with snapshots or reflinks updated frequently.

```
# btrfs balance start -musage=50 /path
Done, had to relocate 4 out of 89 chunks
$ btrfs filesystem df /path
Data, single: total=68.03GiB, used=64.43GiB
System, RAID1: total=32.00MiB, used=20.00KiB
Metadata, RAID1: total=14.87GiB, used=8.85GiB
GlobalReserve, single: total=512.00MiB, used=0.00B
```

Just 1 GiB decrease, which possibly means there are block groups with good utilization. Making the metadata layout more compact would in turn require updating more metadata structures, i.e. lots of IO. As running out of metadata space is a more severe problem, it's not necessary to keep the utilization ratio too high. For the purpose of this example, let's see the effects of further compaction:

```
# btrfs balance start -musage=70 /path
Done, had to relocate 13 out of 88 chunks
$ btrfs filesystem df .
Data, single: total=68.03GiB, used=64.43GiB
System, RAID1: total=32.00MiB, used=20.00KiB
Metadata, RAID1: total=11.97GiB, used=8.83GiB
GlobalReserve, single: total=512.00MiB, used=0.00B
```

### GETTING RID OF COMPLETELY UNUSED BLOCK GROUPS
Normally the balance operation needs a work space, to temporarily move the data before the old block groups gets removed. If there's no work space, it ends with no space left.

There's a special case when the block groups are completely unused, possibly left after removing lots of files or deleting snapshots. Removing empty block groups is automatic since 3.18. The same can be achieved manually with a notable exception that this operation does not require the work space. Thus it can be used to reclaim unused block groups to make it available.

```
# btrfs balance start -dusage=0 /path
```

This should lead to decrease in the total numbers in the btrfs filesystem df output.

## EXIT STATUS
Unless indicated otherwise below, all btrfs balance subcommands return a zero exit status if they succeed, and non zero in case of failure.

The pause, cancel, and resume subcommands exit with a status of 2 if they fail because a balance operation was not running.

The status subcommand exits with a status of 0 if a balance operation is not running, 1 if the command-line usage is incorrect or a balance operation is still running, and 2 on other errors.

## AVAILABILITY
btrfs is part of btrfs-progs. Please refer to the documentation at https://btrfs.readthedocs.io.

## SEE ALSO
mkfs.btrfs(8), btrfs-device(8)


btrfs-check

title: btrfs-check(8)

summary: check or repair a btrfs filesystem

text:
# btrfs-check(8)

## NAME
btrfs-check - check or repair a btrfs filesystem

## SYNOPSIS
btrfs check [options] <device>

## DESCRIPTION
The filesystem checker is used to verify structural integrity of a filesystem and attempt to repair it if requested. It is recommended to unmount the filesystem prior to running the check, but it is possible to start checking a mounted filesystem (see --force).

By default, btrfs check will not modify the device but you can reaffirm that by the option --readonly.

btrfsck is an alias of btrfs check command and is now deprecated.

NOTE:

Even though the filesystem checker requires a device argument, it scans for all devices belonging to the same filesystem, thus it should not cause a difference using different devices of the same filesystem. Furthermore btrfs-check(8) will automatically choose the good mirror, thus as long as there is a good copy for metadata, it will not report such case as an error.

WARNING:

Do not use --repair unless you are advised to do so by a developer or an experienced user, and then only after having accepted that no fsck successfully repair all types of filesystem corruption. E.g. some other software or hardware bugs can fatally damage a volume.

The structural integrity check verifies if internal filesystem objects or data structures satisfy the constraints, point to the right objects or are correctly connected together.

There are several cross checks that can detect wrong reference counts of shared extents, backreferences, missing extents of inodes, directory and inode connectivity etc.

The amount of memory required can be high, depending on the size of the filesystem, similarly the run time. Check the modes that can also affect that.

## SAFE OR ADVISORY OPTIONS
-b|--backup

use the first valid set of backup roots stored in the superblock This can be combined with --super if some of the superblocks are damaged.

--check-data-csum

verify checksums of data blocks This expects that the filesystem is otherwise OK, and is basically an offline scrub that does not repair data from spare copies.

--chunk-root <bytenr>

use the given offset bytenr for the chunk tree root

-E|--subvol-extents <subvolid>

show extent state for the given subvolume

--mode <MODE>

select mode of operation regarding memory and IO The MODE can be one of:

original

The metadata are read into memory and verified, thus the requirements are high on large filesystems and can even lead to out-of-memory conditions. The possible workaround is to export the block device over network to a machine with enough memory.

lowmem

This mode is supposed to address the high memory consumption at the cost of increased IO when it needs to re-read blocks. This may increase run time.

-p|--progress

indicate progress at various checking phases

-Q|--qgroup-report

verify qgroup accounting and compare against filesystem accounting

-r|--tree-root <bytenr>

use the given offset 'bytenr' for the tree root

--readonly

(default) run in read-only mode, this option exists to calm potential panic when users are going to run the checker

-s|--super <N>

use Nth superblock copy, valid values are 0, 1 or 2 if the respective superblock offset is within the device size This can be used to use a different starting point if some of the primary superblock is damaged.

## DANGEROUS OPTIONS
--repair

enable the repair mode and attempt to fix problems where possible NOTE:

There's a warning and 10 second delay when this option is run without --force to give users a chance to think twice before running repair, the warnings in documentation have shown to be insufficient.

--init-csum-tree

create a new checksum tree and recalculate checksums in all files WARNING:

Do not blindly use this option to fix checksum mismatch problems.

--init-extent-tree

build the extent tree from scratch WARNING:

Do not use unless you know what you're doing.

--force

allow work on a mounted filesystem and skip mount checks. Note that this should work fine on a quiescent or read-only mounted filesystem but may crash if the device is changed externally, e.g. by the kernel module. NOTE:

It is possible to run with --repair but on a mounted filesystem that will most likely lead to a corruption unless the filesystem is in a quiescent state which may not be possible to guarantee.

This option also skips the delay and warning in the repair mode (see --repair).

## DEPRECATED OR REMOVED OPTIONS
--clear-space-cache v1|v2

completely remove the free space cache of the given version See also the clear_cache mount option.

WARNING:

This option is deprecated, please use btrfs rescue clear-space-cache instead, this option would be removed in the future eventually.

## EXIT STATUS
btrfs check returns a zero exit status if it succeeds. Non zero is returned in case of failure.

## AVAILABILITY
btrfs is part of btrfs-progs. Please refer to the documentation at https://btrfs.readthedocs.io.

## SEE ALSO
mkfs.btrfs(8), btrfs-scrub(8), btrfs-rescue(8)


btrfs-convert

title: btrfs-convert(8)

text:
# btrfs-convert(8)

## NAME
btrfs-convert - convert from ext2/3/4 or reiserfs filesystem to btrfs in-place

## SYNOPSIS
btrfs-convert [options] <device>

## DESCRIPTION
The btrfs-convert tool can be used to convert existing source filesystem image to a btrfs filesystem in-place. The original filesystem image is accessible in subvolume named like ext2_saved as file image.

Supported filesystems:

- ext2, ext3, ext4 -- original feature, always built in

- reiserfs -- since version 4.13, optionally built, requires libreiserfscore 3.6.27

- NTFS -- external tool https://github.com/maharmstone/ntfs2btrfs

The list of supported source filesystem by a given binary is listed at the end of help (option --help).

WARNING:

If you are going to perform rollback to the original filesystem, you should not execute btrfs balance command on the converted filesystem. This will change the extent layout and make btrfs-convert unable to rollback.

The conversion utilizes free space of the original filesystem. The exact estimate of the required space cannot be foretold. The final btrfs metadata might occupy several gigabytes on a hundreds-gigabyte filesystem.

If the ability to rollback is no longer important, the it is recommended to perform a few more steps to transition the btrfs filesystem to a more compact layout. This is because the conversion inherits the original data blocks' fragmentation, and also because the metadata blocks are bound to the original free space layout.

Due to different constraints, it is only possible to convert filesystems that have a supported data block size (i.e. the same that would be valid for mkfs.btrfs). This is typically the system page size (4KiB on x86_64 machines).

NOTE:

Always consider if a mkfs and file copy would not be a better option than the in-place conversion given what was said above. The conversion depends on 3rd party libraries and the other filesystems could still evolve and add new features. Not all combinations are covered or tested.

BEFORE YOU START

The source filesystem must be clean, e.g. no journal to replay or no repairs needed. The respective fsck utility must be run on the source filesystem prior to conversion. Please refer to the manual pages in case you encounter problems.

For ext2/3/4:

```
# e2fsck -fvy /dev/sdx
```

For reiserfs:

```
# reiserfsck -fy /dev/sdx
```

Skipping that step could lead to incorrect results on the target filesystem, but it may work.

REMOVE THE ORIGINAL FILESYSTEM METADATA

By removing the subvolume named like ext2_saved or reiserfs_saved, all metadata of the original filesystem will be removed:

```
# btrfs subvolume delete /mnt/ext2_saved
```

At this point it is not possible to do a rollback. The filesystem is usable but may be impacted by the fragmentation inherited from the original filesystem.

MAKE FILE DATA MORE CONTIGUOUS

An optional but recommended step is to run defragmentation on the entire filesystem. This will attempt to make file extents more contiguous.

```
# btrfs filesystem defrag -v -r -f -t 32M /mnt/btrfs
```

Verbose recursive defragmentation (-v, -r), flush data per-file (-f) with target extent size 32MiB (-t).

ATTEMPT TO MAKE BTRFS METADATA MORE COMPACT

Optional but recommended step.

The metadata block groups after conversion may be smaller than the default size (256MiB or 1GiB). Running a balance will attempt to merge the block groups. This depends on the free space layout (and fragmentation) and may fail due to lack of enough work space. This is a soft error leaving the filesystem usable but the block group layout may remain unchanged.

Note that balance operation takes a lot of time, please see also btrfs-balance(8).

```
# btrfs balance start -m /mnt/btrfs
```

## OPTIONS
--csum <type>, --checksum <type>

Specify the checksum algorithm. Default is crc32c. Valid values are crc32c, xxhash, sha256 or blake2. To mount such filesystem kernel must support the checksums as well.

-d|--no-datasum

disable data checksum calculations and set the NODATASUM file flag, this can speed up the conversion

-i|--no-xattr

ignore xattrs and ACLs of files

-n|--no-inline

disable inlining of small files to metadata blocks, this will decrease the metadata consumption and may help to convert a filesystem with low free space

-N|--nodesize <SIZE>

set filesystem nodesize, the tree block size in which btrfs stores its metadata. The default value is 16KiB (16384) or the page size, whichever is bigger. Must be a multiple of the sectorsize, but not larger than 65536. See mkfs.btrfs(8) for more details.

-r|--rollback

rollback to the original ext2/3/4 filesystem if possible

-l|--label <LABEL>

set filesystem label during conversion

-L|--copy-label

use label from the converted filesystem

-O|--features <feature1>[,<feature2>...]

A list of filesystem features enabled the at time of conversion. Not all features are supported by old kernels. To disable a feature, prefix it with ^. Description of the features is in section FILESYSTEM FEATURES of mkfs.btrfs(8). To see all available features that btrfs-convert supports run:

```
btrfs-convert -O list-all
```

-p|--progress

show progress of conversion (a heartbeat indicator and number of inodes processed), on by default

--no-progress

disable progress and show only the main phases of conversion

--uuid <SPEC>

set the FSID of the new filesystem based on 'SPEC':

- new - (default) generate UUID for the FSID of btrfs

- copy - copy UUID from the source filesystem

- UUID - a conforming UUID value, the 36 byte string representation

--version

Print the btrfs-convert version, builtin features and exit.

## EXIT STATUS
btrfs-convert will return 0 if no error happened. If any problems happened, 1 will be returned.

## SEE ALSO
mkfs.btrfs(8)