bpftool-gen

title: bpftool-gen(8)

summary: tool for BPF code-generation

text:
# bpftool-gen(8)

## NAME
bpftool-gen - tool for BPF code-generation

## SYNOPSIS
bpftool [OPTIONS] gen COMMAND OPTIONS := { { -j | --json } [{ -p | --pretty }] | { -d | --debug } | { -l | --legacy } | { -L | --use-loader } }

COMMAND := { object | skeleton | help }

## GEN COMMANDS

```
bpftool gen object OUTPUT_FILE INPUT_FILE [INPUT_FILE...]
bpftool gen skeleton FILE [name OBJECT_NAME]
bpftool gen subskeleton FILE [name OBJECT_NAME]
bpftool gen min_core_btf INPUT OUTPUT OBJECT [OBJECT...]
bpftool gen help
```

## DESCRIPTION
bpftool gen object OUTPUT_FILE INPUT_FILE [INPUT_FILE...]

Statically link (combine) together one or more INPUT_FILE's into a single resulting OUTPUT_FILE. All the files involved are BPF ELF object files. The rules of BPF static linking are mostly the same as for user-space object files, but in addition to combining data and instruction sections, .BTF and .BTF.ext (if present in any of the input files) data are combined together. .BTF data is deduplicated, so all the common types across INPUT_FILE's will only be represented once in the resulting BTF information.

BPF static linking allows to partition BPF source code into individually compiled files that are then linked into a single resulting BPF object file, which can be used to generated BPF skeleton (with gen skeleton command) or passed directly into libbpf (using bpf_object__open() family of APIs).

bpftool gen skeleton FILE

Generate BPF skeleton C header file for a given FILE. BPF skeleton is an alternative interface to existing libbpf APIs for working with BPF objects. Skeleton code is intended to significantly shorten and simplify code to load and work with BPF programs from userspace side. Generated code is tailored to specific input BPF object FILE, reflecting its structure by listing out available maps, program, variables, etc. Skeleton eliminates the need to lookup mentioned components by name. Instead, if skeleton instantiation succeeds, they are populated in skeleton structure as valid libbpf types (e.g., struct bpf_map pointer) and can be passed to existing generic libbpf APIs.

In addition to simple and reliable access to maps and programs, skeleton provides a storage for BPF links (struct bpf_link) for each BPF program within BPF object. When requested, supported BPF programs will be automatically attached and resulting BPF links stored for further use by user in pre-allocated fields in skeleton struct. For BPF programs that can't be automatically attached by libbpf, user can attach them manually, but store resulting BPF link in per-program link field. All such set up links will be automatically destroyed on BPF skeleton destruction. This eliminates the need for users to manage links manually and rely on libbpf support to detach programs and free up resources.

Another facility provided by BPF skeleton is an interface to global variables of all supported kinds: mutable, read-only, as well as extern ones. This interface allows to pre-setup initial values of variables before BPF object is loaded and verified by kernel. For non-read-only variables, the same interface can be used to fetch values of global variables on userspace side, even if they are modified by BPF code.

During skeleton generation, contents of source BPF object FILE is embedded within generated code and is thus not necessary to keep around. This ensures skeleton and BPF object file are matching 1-to-1 and always stay in sync. Generated code is dual-licensed under LGPL-2.1 and BSD-2-Clause licenses.

It is a design goal and guarantee that skeleton interfaces are interoperable with generic libbpf APIs. User should always be able to use skeleton API to create and load BPF object, and later use libbpf APIs to keep working with specific maps, programs, etc.

As part of skeleton, few custom functions are generated. Each of them is prefixed with object name. Object name can either be derived from object file name, i.e., if BPF object file name is example.o, BPF object name will be example. Object name can be also specified explicitly through name OBJECT_NAME parameter. The following custom functions are provided (assuming example as the object name):

- example__open and example__open_opts. These functions are used to instantiate skeleton. It corresponds to libbpf's bpf_object__open() API. _opts variants accepts extra bpf_object_open_opts options.

- example__load. This function creates maps, loads and verifies BPF programs, initializes global data maps. It corresponds to libppf's bpf_object__load() API.

- example__open_and_load combines example__open and example__load invocations in one commonly used operation.

- example__attach and example__detach This pair of functions allow to attach and detach, correspondingly, already loaded BPF object. Only BPF programs of types supported by libbpf for auto-attachment will be auto-attached and their corresponding BPF links instantiated. For other BPF programs, user can manually create a BPF link and assign it to corresponding fields in skeleton struct. example__detach will detach both links created automatically, as well as those populated by user manually.

- example__destroy Detach and unload BPF programs, free up all the resources used by skeleton and BPF object.

If BPF object has global variables, corresponding structs with memory layout corresponding to global data data section layout will be created. Currently supported ones are: .data, .bss, .rodata, and .kconfig structs/data sections. These data sections/structs can be used to set up initial values of variables, if set before example__load. Afterwards, if target kernel supports memory-mapped BPF arrays, same structs can be used to fetch and update (non-read-only) data from userspace, with same simplicity as for BPF side.

bpftool gen subskeleton FILE

Generate BPF subskeleton C header file for a given FILE. Subskeletons are similar to skeletons, except they do not own the corresponding maps, programs, or global variables. They require that the object file used to generate them is already loaded into a bpf_object by some other means.

This functionality is useful when a library is included into a larger BPF program. A subskeleton for the library would have access to all objects and globals defined in it, without having to know about the larger program.

Consequently, there are only two functions defined for subskeletons:

- example__open(bpf_object*) Instantiates a subskeleton from an already opened (but not necessarily loaded) bpf_object.

- example__destroy() Frees the storage for the subskeleton but does not unload any BPF programs or maps.

bpftool gen min_core_btf INPUT OUTPUT OBJECT [OBJECT...]

Generate a minimum BTF file as OUTPUT, derived from a given INPUT BTF file, containing all needed BTF types so one, or more, given eBPF objects CO-RE relocations may be satisfied. When kernels aren't compiled with CONFIG_DEBUG_INFO_BTF, libbpf, when loading an eBPF object, has to rely on external BTF files to be able to calculate CO-RE relocations.

Usually, an external BTF file is built from existing kernel DWARF data using pahole. It contains all the types used by its respective kernel image and, because of that, is big.

The min_core_btf feature builds smaller BTF files, customized to one or multiple eBPF objects, so they can be distributed together with an eBPF CO-RE based application, turning the application portable to different kernel versions.

Check examples bellow for more information how to use it.

bpftool gen help

Print short help message.

## OPTIONS
-h, --help

Print short help message (similar to bpftool help).

-V, --version

Print bpftool's version number (similar to bpftool version), the number of the libbpf version in use, and optional features that were included when bpftool was compiled. Optional features include linking against libbfd to provide the disassembler for JIT-ted programs (bpftool prog dump jited) and usage of BPF skeletons (some features like bpftool prog profile or showing pids associated to BPF objects may rely on it).

-j, --json

Generate JSON output. For commands that cannot produce JSON, this option has no effect.

-p, --pretty

Generate human-readable JSON output. Implies -j.

-d, --debug

Print all logs available, even debug-level information. This includes logs from libbpf as well as from the verifier, when attempting to load programs.

-l, --legacy

Use legacy libbpf mode which has more relaxed BPF program requirements. By default, bpftool has more strict requirements about section names, changes pinning logic and doesn't support some of the older non-BTF map declarations. See https://github.com/libbpf/libbpf/wiki/Libbpf:-the-road-to-v1.0 for details.

-L, --use-loader

For skeletons, generate a "light" skeleton (also known as "loader" skeleton). A light skeleton contains a loader eBPF program. It does not use the majority of the libbpf infrastructure, and does not need libelf.

## EXAMPLES
$ cat example1.bpf.c

```
#include <stdbool.h>
#include <linux/ptrace.h>
#include <linux/bpf.h>
#include <bpf/bpf_helpers.h>
const volatile int param1 = 42;
bool global_flag = true;
struct { int x; } data = {};
SEC("raw_tp/sys_enter")
int handle_sys_enter(struct pt_regs *ctx)
{

      static long my_static_var;

      if (global_flag)

              my_static_var++;

      else

              data.x += param1;

      return 0;
}
```

$ cat example2.bpf.c

```
#include <linux/ptrace.h>
#include <linux/bpf.h>
#include <bpf/bpf_helpers.h>
struct {

      __uint(type, BPF_MAP_TYPE_HASH);

      __uint(max_entries, 128);

      __type(key, int);

      __type(value, long);
} my_map SEC(".maps");
SEC("raw_tp/sys_exit")
int handle_sys_exit(struct pt_regs *ctx)
{

      int zero = 0;

      bpf_map_lookup_elem(&my_map, &zero);

      return 0;
}
```

This is example BPF application with two BPF programs and a mix of BPF maps and global variables. Source code is split across two source code files.

$ clang -target bpf -g example1.bpf.c -o example1.bpf.o

$ clang -target bpf -g example2.bpf.c -o example2.bpf.o

$ bpftool gen object example.bpf.o example1.bpf.o example2.bpf.o

This set of commands compiles example1.bpf.c and example2.bpf.c individually and then statically links respective object files into the final BPF ELF object file example.bpf.o.

$ bpftool gen skeleton example.bpf.o name example | tee example.skel.h

```
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
/* THIS FILE IS AUTOGENERATED! */
#ifndef __EXAMPLE_SKEL_H__
#define __EXAMPLE_SKEL_H__
#include <stdlib.h>
#include <bpf/libbpf.h>
struct example {

      struct bpf_object_skeleton *skeleton;

      struct bpf_object *obj;

      struct {

              struct bpf_map *rodata;

              struct bpf_map *data;

              struct bpf_map *bss;

              struct bpf_map *my_map;

      } maps;

      struct {

              struct bpf_program *handle_sys_enter;

              struct bpf_program *handle_sys_exit;

      } progs;

      struct {

              struct bpf_link *handle_sys_enter;

              struct bpf_link *handle_sys_exit;

      } links;

      struct example__bss {

              struct {

                      int x;

              } data;

      } *bss;

      struct example__data {

              _Bool global_flag;

              long int handle_sys_enter_my_static_var;

      } *data;

      struct example__rodata {

              int param1;

      } *rodata;
};
static void example__destroy(struct example *obj);
static inline struct example *example__open_opts(

              const struct bpf_object_open_opts *opts);
static inline struct example *example__open();
static inline int example__load(struct example *obj);
static inline struct example *example__open_and_load();
static inline int example__attach(struct example *obj);
static inline void example__detach(struct example *obj);
#endif /* __EXAMPLE_SKEL_H__ */
```

$ cat example.c

```
#include "example.skel.h"
int main()
{

      struct example *skel;

      int err = 0;

      skel = example__open();

      if (!skel)

              goto cleanup;

      skel->rodata->param1 = 128;

      err = example__load(skel);

      if (err)

              goto cleanup;

      err = example__attach(skel);

      if (err)

              goto cleanup;

      /* all libbpf APIs are usable */

      printf("my_map name: %s\n", bpf_map__name(skel->maps.my_map));

      printf("sys_enter prog FD: %d\n",

             bpf_program__fd(skel->progs.handle_sys_enter));

      /* detach and re-attach sys_exit program */

      bpf_link__destroy(skel->links.handle_sys_exit);

      skel->links.handle_sys_exit =

              bpf_program__attach(skel->progs.handle_sys_exit);

      printf("my_static_var: %ld\n",

             skel->bss->handle_sys_enter_my_static_var);
cleanup:

      example__destroy(skel);

      return err;
}
```

\# ./example

```
my_map name: my_map
sys_enter prog FD: 8
my_static_var: 7
```

This is a stripped-out version of skeleton generated for above example code.

### min_core_btf
$ bpftool btf dump file 5.4.0-example.btf format raw

```
[1] INT 'long unsigned int' size=8 bits_offset=0 nr_bits=64 encoding=(none)
[2] CONST '(anon)' type_id=1
[3] VOLATILE '(anon)' type_id=1
[4] ARRAY '(anon)' type_id=1 index_type_id=21 nr_elems=2
[5] PTR '(anon)' type_id=8
[6] CONST '(anon)' type_id=5
[7] INT 'char' size=1 bits_offset=0 nr_bits=8 encoding=(none)
[8] CONST '(anon)' type_id=7
[9] INT 'unsigned int' size=4 bits_offset=0 nr_bits=32 encoding=(none)
<long output>
```

$ bpftool btf dump file one.bpf.o format raw

```
[1] PTR '(anon)' type_id=2
[2] STRUCT 'trace_event_raw_sys_enter' size=64 vlen=4

      'ent' type_id=3 bits_offset=0

      'id' type_id=7 bits_offset=64

      'args' type_id=9 bits_offset=128

      '__data' type_id=12 bits_offset=512
[3] STRUCT 'trace_entry' size=8 vlen=4

      'type' type_id=4 bits_offset=0

      'flags' type_id=5 bits_offset=16

      'preempt_count' type_id=5 bits_offset=24
<long output>
```

$ bpftool gen min_core_btf 5.4.0-example.btf 5.4.0-smaller.btf one.bpf.o

$ bpftool btf dump file 5.4.0-smaller.btf format raw

```
[1] TYPEDEF 'pid_t' type_id=6
[2] STRUCT 'trace_event_raw_sys_enter' size=64 vlen=1

      'args' type_id=4 bits_offset=128
[3] STRUCT 'task_struct' size=9216 vlen=2

      'pid' type_id=1 bits_offset=17920

      'real_parent' type_id=7 bits_offset=18048
[4] ARRAY '(anon)' type_id=5 index_type_id=8 nr_elems=6
[5] INT 'long unsigned int' size=8 bits_offset=0 nr_bits=64 encoding=(none)
[6] TYPEDEF '__kernel_pid_t' type_id=8
[7] PTR '(anon)' type_id=3
[8] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
<end>
```

Now, the "5.4.0-smaller.btf" file may be used by libbpf as an external BTF file when loading the "one.bpf.o" object into the "5.4.0-example" kernel. Note that the generated BTF file won't allow other eBPF objects to be loaded, just the ones given to min_core_btf.

```
LIBBPF_OPTS(bpf_object_open_opts, opts, .btf_custom_path = "5.4.0-smaller.btf");
struct bpf_object *obj;
obj = bpf_object__open_file("one.bpf.o", &opts);
...
```

## SEE ALSO
bpf(2), bpf-helpers(7), bpftool(8), bpftool-btf(8), bpftool-cgroup(8), bpftool-feature(8), bpftool-iter(8), bpftool-link(8), bpftool-map(8), bpftool-net(8), bpftool-perf(8), bpftool-prog(8), bpftool-struct_ops(8)


bpftool-iter

package: bpftool

title: bpftool-iter(8)

summary: tool to create BPF iterators

text:
# bpftool-iter(8)

## NAME
bpftool-iter - tool to create BPF iterators

## SYNOPSIS
bpftool [OPTIONS] iter COMMAND OPTIONS := { { -j | --json } [{ -p | --pretty }] | { -d | --debug } | { -l | --legacy } }

COMMANDS := { pin | help }

## ITER COMMANDS

```
bpftool iter pin OBJ PATH [map MAP]
bpftool iter help
OBJ := /a/file/of/bpf_iter_target.o
MAP := { id MAP_ID | pinned FILE }
```

## DESCRIPTION
bpftool iter pin OBJ PATH [map MAP]

A bpf iterator combines a kernel iterating of particular kernel data (e.g., tasks, bpf_maps, etc.) and a bpf program called for each kernel data object (e.g., one task, one bpf_map, etc.). User space can read kernel iterator output through read() syscall. The pin command creates a bpf iterator from OBJ, and pin it to PATH. The PATH should be located in bpffs mount. It must not contain a dot character ('.'), which is reserved for future extensions of bpffs.

Map element bpf iterator requires an additional parameter MAP so bpf program can iterate over map elements for that map. User can have a bpf program in kernel to run with each map element, do checking, filtering, aggregation, etc. without copying data to user space.

User can then cat PATH to see the bpf iterator output.

bpftool iter help

Print short help message.

## OPTIONS
-h, --help

Print short help message (similar to bpftool help).

-V, --version

Print bpftool's version number (similar to bpftool version), the number of the libbpf version in use, and optional features that were included when bpftool was compiled. Optional features include linking against libbfd to provide the disassembler for JIT-ted programs (bpftool prog dump jited) and usage of BPF skeletons (some features like bpftool prog profile or showing pids associated to BPF objects may rely on it).

-j, --json

Generate JSON output. For commands that cannot produce JSON, this option has no effect.

-p, --pretty

Generate human-readable JSON output. Implies -j.

-d, --debug

Print all logs available, even debug-level information. This includes logs from libbpf as well as from the verifier, when attempting to load programs.

-l, --legacy

Use legacy libbpf mode which has more relaxed BPF program requirements. By default, bpftool has more strict requirements about section names, changes pinning logic and doesn't support some of the older non-BTF map declarations. See https://github.com/libbpf/libbpf/wiki/Libbpf:-the-road-to-v1.0 for details.

## EXAMPLES
\# bpftool iter pin bpf_iter_netlink.o /sys/fs/bpf/my_netlink

```
Create a file-based bpf iterator from bpf_iter_netlink.o and pin it
to /sys/fs/bpf/my_netlink
```

\# bpftool iter pin bpf_iter_hashmap.o /sys/fs/bpf/my_hashmap map id 20

```
Create a file-based bpf iterator from bpf_iter_hashmap.o and map with
id 20, and pin it to /sys/fs/bpf/my_hashmap
```

## SEE ALSO
bpf(2), bpf-helpers(7), bpftool(8), bpftool-btf(8), bpftool-cgroup(8), bpftool-feature(8), bpftool-gen(8), bpftool-link(8), bpftool-map(8), bpftool-net(8), bpftool-perf(8), bpftool-prog(8), bpftool-struct_ops(8)


bpftool-link

title: bpftool-link(8)

text:
# bpftool-link(8)

## NAME
bpftool-link - tool for inspection and simple manipulation of eBPF links

## SYNOPSIS
bpftool [OPTIONS] link COMMAND OPTIONS := { { -j | --json } [{ -p | --pretty }] | { -d | --debug } | { -l | --legacy } | { -f | --bpffs } | { -n | --nomount } }

COMMANDS := { show | list | pin | help }

## LINK COMMANDS

```
bpftool link { show | list } [LINK]
bpftool link pin LINK FILE
bpftool link detach LINK
bpftool link help
LINK := { id LINK_ID | pinned FILE }
```

## DESCRIPTION
bpftool link { show | list } [LINK]

Show information about active links. If LINK is specified show information only about given link, otherwise list all links currently active on the system. Output will start with link ID followed by link type and zero or more named attributes, some of which depend on type of link.

Since Linux 5.8 bpftool is able to discover information about processes that hold open file descriptors (FDs) against BPF links. On such kernels bpftool will automatically emit this information as well.

bpftool link pin LINK FILE

Pin link LINK as FILE. Note: FILE must be located in bpffs mount. It must not contain a dot character ('.'), which is reserved for future extensions of bpffs.

bpftool link detach LINK

Force-detach link LINK. BPF link and its underlying BPF program will stay valid, but they will be detached from the respective BPF hook and BPF link will transition into a defunct state until last open file descriptor for that link is closed.

bpftool link help

Print short help message.

## OPTIONS
-h, --help

Print short help message (similar to bpftool help).

-V, --version

Print bpftool's version number (similar to bpftool version), the number of the libbpf version in use, and optional features that were included when bpftool was compiled. Optional features include linking against libbfd to provide the disassembler for JIT-ted programs (bpftool prog dump jited) and usage of BPF skeletons (some features like bpftool prog profile or showing pids associated to BPF objects may rely on it).

-j, --json

Generate JSON output. For commands that cannot produce JSON, this option has no effect.

-p, --pretty

Generate human-readable JSON output. Implies -j.

-d, --debug

Print all logs available, even debug-level information. This includes logs from libbpf as well as from the verifier, when attempting to load programs.

-l, --legacy

Use legacy libbpf mode which has more relaxed BPF program requirements. By default, bpftool has more strict requirements about section names, changes pinning logic and doesn't support some of the older non-BTF map declarations. See https://github.com/libbpf/libbpf/wiki/Libbpf:-the-road-to-v1.0 for details.

-f, --bpffs

When showing BPF links, show file names of pinned links.

-n, --nomount

Do not automatically attempt to mount any virtual file system (such as tracefs or BPF virtual file system) when necessary.

## EXAMPLES
\# bpftool link show

```
10: cgroup  prog 25

        cgroup_id 614  attach_type egress

        pids test_progs(223)
```

\# bpftool --json --pretty link show

```
[{

        "type": "cgroup",

        "prog_id": 25,

        "cgroup_id": 614,

        "attach_type": "egress",

        "pids": [{

                "pid": 223,

                "comm": "test_progs"

            }

        ]

    }
]
```

```
# bpftool link pin id 10 /sys/fs/bpf/link
# ls -l /sys/fs/bpf/
```

```
-rw------- 1 root root 0 Apr 23 21:39 link
```

## SEE ALSO
bpf(2), bpf-helpers(7), bpftool(8), bpftool-btf(8), bpftool-cgroup(8), bpftool-feature(8), bpftool-gen(8), bpftool-iter(8), bpftool-map(8), bpftool-net(8), bpftool-perf(8), bpftool-prog(8), bpftool-struct_ops(8)


bpftool-map

title: bpftool-map(8)

text:
# bpftool-map(8)

## NAME
bpftool-map - tool for inspection and simple manipulation of eBPF maps

## SYNOPSIS
bpftool [OPTIONS] map COMMAND OPTIONS := { { -j | --json } [{ -p | --pretty }] | { -d | --debug } | { -l | --legacy } | { -f | --bpffs } | { -n | --nomount } }

COMMANDS := { show | list | create | dump | update | lookup | getnext | delete | pin | help }

## MAP COMMANDS

```
bpftool map { show | list }   [MAP]
bpftool map create     FILE type TYPE key KEY_SIZE value VALUE_SIZE

entries MAX_ENTRIES name NAME [flags FLAGS] [inner_map MAP]
[dev NAME]

bpftool map dump       MAP
bpftool map update     MAP [key DATA] [value VALUE] [UPDATE_FLAGS]
bpftool map lookup     MAP [key DATA]
bpftool map getnext    MAP [key DATA]
bpftool map delete     MAP  key DATA
bpftool map pin        MAP  FILE
bpftool map event_pipe MAP [cpu N index M]
bpftool map peek       MAP
bpftool map push       MAP value VALUE
bpftool map pop        MAP
bpftool map enqueue    MAP value VALUE
bpftool map dequeue    MAP
bpftool map freeze     MAP
bpftool map help
MAP := { id MAP_ID | pinned FILE | name MAP_NAME }
DATA := { [hex] BYTES }
PROG := { id PROG_ID | pinned FILE | tag PROG_TAG | name PROG_NAME }
VALUE := { DATA | MAP | PROG }
UPDATE_FLAGS := { any | exist | noexist }
TYPE := { hash | array | prog_array | perf_event_array | percpu_hash

| percpu_array | stack_trace | cgroup_array | lru_hash
| lru_percpu_hash | lpm_trie | array_of_maps | hash_of_maps
| devmap | devmap_hash | sockmap | cpumap | xskmap | sockhash
| cgroup_storage | reuseport_sockarray | percpu_cgroup_storage
| queue | stack | sk_storage | struct_ops | ringbuf | inode_storage
| task_storage | bloom_filter | user_ringbuf }
```

## DESCRIPTION
bpftool map { show | list } [MAP]

Show information about loaded maps. If MAP is specified show information only about given maps, otherwise list all maps currently loaded on the system. In case of name, MAP may match several maps which will all be shown. Output will start with map ID followed by map type and zero or more named attributes (depending on kernel version).

Since Linux 5.8 bpftool is able to discover information about processes that hold open file descriptors (FDs) against BPF maps. On such kernels bpftool will automatically emit this information as well.

bpftool map create FILE type TYPE key KEY_SIZE value VALUE_SIZE entries MAX_ENTRIES name NAME [flags FLAGS] [inner_map MAP] [dev NAME]

Create a new map with given parameters and pin it to bpffs as FILE. FLAGS should be an integer which is the combination of desired flags, e.g. 1024 for BPF_F_MMAPABLE (see bpf.h UAPI header for existing flags).

To create maps of type array-of-maps or hash-of-maps, the inner_map keyword must be used to pass an inner map. The kernel needs it to collect metadata related to the inner maps that the new map will work with.

Keyword dev expects a network interface name, and is used to request hardware offload for the map.

bpftool map dump MAP

Dump all entries in a given MAP. In case of name, MAP may match several maps which will all be dumped.

bpftool map update MAP [key DATA] [value VALUE] [UPDATE_FLAGS]

Update map entry for a given KEY. UPDATE_FLAGS can be one of: any update existing entry or add if doesn't exit; exist update only if entry already exists; noexist update only if entry doesn't exist.

If the hex keyword is provided in front of the bytes sequence, the bytes are parsed as hexadecimal values, even if no "0x" prefix is added. If the keyword is not provided, then the bytes are parsed as decimal values, unless a "0x" prefix (for hexadecimal) or a "0" prefix (for octal) is provided.

bpftool map lookup MAP [key DATA]

Lookup key in the map.

bpftool map getnext MAP [key DATA]

Get next key. If key is not specified, get first key.

bpftool map delete MAP key DATA

Remove entry from the map.

bpftool map pin MAP FILE

Pin map MAP as FILE. Note: FILE must be located in bpffs mount. It must not contain a dot character ('.'), which is reserved for future extensions of bpffs.

bpftool map event_pipe MAP [cpu N index M]

Read events from a BPF_MAP_TYPE_PERF_EVENT_ARRAY map. Install perf rings into a perf event array map and dump output of any bpf_perf_event_output() call in the kernel. By default read the number of CPUs on the system and install perf ring for each CPU in the corresponding index in the array.

If cpu and index are specified, install perf ring for given cpu at index in the array (single ring).

Note that installing a perf ring into an array will silently replace any existing ring. Any other application will stop receiving events if it installed its rings earlier.

bpftool map peek MAP

Peek next value in the queue or stack.

bpftool map push MAP value VALUE

Push VALUE onto the stack.

bpftool map pop MAP

Pop and print value from the stack.

bpftool map enqueue MAP value VALUE

Enqueue VALUE into the queue.

bpftool map dequeue MAP

Dequeue and print value from the queue.

bpftool map freeze MAP

Freeze the map as read-only from user space. Entries from a frozen map can not longer be updated or deleted with the bpf() system call. This operation is not reversible, and the map remains immutable from user space until its destruction. However, read and write permissions for BPF programs to the map remain unchanged.

bpftool map help

Print short help message.

## OPTIONS
-h, --help

Print short help message (similar to bpftool help).

-V, --version

Print bpftool's version number (similar to bpftool version), the number of the libbpf version in use, and optional features that were included when bpftool was compiled. Optional features include linking against libbfd to provide the disassembler for JIT-ted programs (bpftool prog dump jited) and usage of BPF skeletons (some features like bpftool prog profile or showing pids associated to BPF objects may rely on it).

-j, --json

Generate JSON output. For commands that cannot produce JSON, this option has no effect.

-p, --pretty

Generate human-readable JSON output. Implies -j.

-d, --debug

Print all logs available, even debug-level information. This includes logs from libbpf as well as from the verifier, when attempting to load programs.

-l, --legacy

Use legacy libbpf mode which has more relaxed BPF program requirements. By default, bpftool has more strict requirements about section names, changes pinning logic and doesn't support some of the older non-BTF map declarations. See https://github.com/libbpf/libbpf/wiki/Libbpf:-the-road-to-v1.0 for details.

-f, --bpffs

Show file names of pinned maps.

-n, --nomount

Do not automatically attempt to mount any virtual file system (such as tracefs or BPF virtual file system) when necessary.

## EXAMPLES
\# bpftool map show

```
10: hash  name some_map  flags 0x0

      key 4B  value 8B  max_entries 2048  memlock 167936B

      pids systemd(1)
```

The following three commands are equivalent:

```
# bpftool map update id 10 key hex   20   c4   b7   00 value hex   0f   ff   ff   ab   01   02   03   4c
# bpftool map update id 10 key     0x20 0xc4 0xb7 0x00 value     0x0f 0xff 0xff 0xab 0x01 0x02 0x03 0x4c
# bpftool map update id 10 key       32  196  183    0 value       15  255  255  171    1    2    3   76
```

\# bpftool map lookup id 10 key 0 1 2 3

```
key: 00 01 02 03 value: 00 01 02 03 04 05 06 07
```

\# bpftool map dump id 10

```
key: 00 01 02 03  value: 00 01 02 03 04 05 06 07
key: 0d 00 07 00  value: 02 00 00 00 01 02 03 04
Found 2 elements
```

\# bpftool map getnext id 10 key 0 1 2 3

```
key:
00 01 02 03
next key:
0d 00 07 00
```

```
# mount -t bpf none /sys/fs/bpf/
# bpftool map pin id 10 /sys/fs/bpf/map
# bpftool map del pinned /sys/fs/bpf/map key 13 00 07 00
```

Note that map update can also be used in order to change the program references hold by a program array map. This can be used, for example, to change the programs used for tail-call jumps at runtime, without having to reload the entry-point program. Below is an example for this use case: we load a program defining a prog array map, and with a main function that contains a tail call to other programs that can be used either to "process" packets or to "debug" processing. Note that the prog array map MUST be pinned into the BPF virtual file system for the map update to work successfully, as kernel flushes prog array maps when they have no more references from user space (and the update would be lost as soon as bpftool exits).

```
# bpftool prog loadall tail_calls.o /sys/fs/bpf/foo type xdp
# bpftool prog --bpffs
```

```
545: xdp  name main_func  tag 674b4b5597193dc3  gpl

        loaded_at 2018-12-12T15:02:58+0000  uid 0

        xlated 240B  jited 257B  memlock 4096B  map_ids 294

        pinned /sys/fs/bpf/foo/xdp
546: xdp  name bpf_func_process  tag e369a529024751fc  gpl

        loaded_at 2018-12-12T15:02:58+0000  uid 0

        xlated 200B  jited 164B  memlock 4096B

        pinned /sys/fs/bpf/foo/process
547: xdp  name bpf_func_debug  tag 0b597868bc7f0976  gpl

        loaded_at 2018-12-12T15:02:58+0000  uid 0

        xlated 200B  jited 164B  memlock 4096B

        pinned /sys/fs/bpf/foo/debug
```

\# bpftool map

```
294: prog_array  name jmp_table  flags 0x0

        key 4B  value 4B  max_entries 1  memlock 4096B

        owner_prog_type xdp  owner jited
```

```
# bpftool map pin id 294 /sys/fs/bpf/bar
# bpftool map dump pinned /sys/fs/bpf/bar
```

```
Found 0 elements
```

```
# bpftool map update pinned /sys/fs/bpf/bar key 0 0 0 0 value pinned /sys/fs/bpf/foo/debug
# bpftool map dump pinned /sys/fs/bpf/bar
```

```
key: 00 00 00 00  value: 22 02 00 00
Found 1 element
```

## SEE ALSO
bpf(2), bpf-helpers(7), bpftool(8), bpftool-btf(8), bpftool-cgroup(8), bpftool-feature(8), bpftool-gen(8), bpftool-iter(8), bpftool-link(8), bpftool-net(8), bpftool-perf(8), bpftool-prog(8), bpftool-struct_ops(8)


bpftool-net

title: bpftool-net(8)

text:
# bpftool-net(8)

## NAME
bpftool-net - tool for inspection of netdev/tc related bpf prog attachments

## SYNOPSIS
bpftool [OPTIONS] net COMMAND OPTIONS := { { -j | --json } [{ -p | --pretty }] | { -d | --debug } | { -l | --legacy } }

COMMANDS := { show | list | attach | detach | help }

## NET COMMANDS

```
bpftool net { show | list } [ dev NAME ]
bpftool net attach ATTACH_TYPE PROG dev NAME [ overwrite ]
bpftool net detach ATTACH_TYPE dev NAME
bpftool net help
PROG := { id PROG_ID | pinned FILE | tag PROG_TAG }
ATTACH_TYPE := { xdp | xdpgeneric | xdpdrv | xdpoffload }
```

## DESCRIPTION
bpftool net { show | list } [ dev NAME ]

List bpf program attachments in the kernel networking subsystem. Currently, only device driver xdp attachments and tc filter classification/action attachments are implemented, i.e., for program types BPF_PROG_TYPE_SCHED_CLS, BPF_PROG_TYPE_SCHED_ACT and BPF_PROG_TYPE_XDP. For programs attached to a particular cgroup, e.g., BPF_PROG_TYPE_CGROUP_SKB, BPF_PROG_TYPE_CGROUP_SOCK, BPF_PROG_TYPE_SOCK_OPS and BPF_PROG_TYPE_CGROUP_SOCK_ADDR, users can use bpftool cgroup to dump cgroup attachments. For sk_{filter, skb, msg, reuseport} and lwt/seg6 bpf programs, users should consult other tools, e.g., iproute2.

The current output will start with all xdp program attachments, followed by all tc class/qdisc bpf program attachments. Both xdp programs and tc programs are ordered based on ifindex number. If multiple bpf programs attached to the same networking device through tc filter, the order will be first all bpf programs attached to tc classes, then all bpf programs attached to non clsact qdiscs, and finally all bpf programs attached to root and clsact qdisc.

bpftool net attach ATTACH_TYPE PROG dev NAME [ overwrite ]

Attach bpf program PROG to network interface NAME with type specified by ATTACH_TYPE. Previously attached bpf program can be replaced by the command used with overwrite option. Currently, only XDP-related modes are supported for ATTACH_TYPE. ATTACH_TYPE can be of: xdp - try native XDP and fallback to generic XDP if NIC driver does not support it; xdpgeneric - Generic XDP. runs at generic XDP hook when packet already enters receive path as skb; xdpdrv - Native XDP. runs earliest point in driver's receive path; xdpoffload - Offload XDP. runs directly on NIC on each packet reception;

bpftool net detach ATTACH_TYPE dev NAME

Detach bpf program attached to network interface NAME with type specified by ATTACH_TYPE. To detach bpf program, same ATTACH_TYPE previously used for attach must be specified. Currently, only XDP-related modes are supported for ATTACH_TYPE.

bpftool net help

Print short help message.

## OPTIONS
-h, --help

Print short help message (similar to bpftool help).

-V, --version

Print bpftool's version number (similar to bpftool version), the number of the libbpf version in use, and optional features that were included when bpftool was compiled. Optional features include linking against libbfd to provide the disassembler for JIT-ted programs (bpftool prog dump jited) and usage of BPF skeletons (some features like bpftool prog profile or showing pids associated to BPF objects may rely on it).

-j, --json

Generate JSON output. For commands that cannot produce JSON, this option has no effect.

-p, --pretty

Generate human-readable JSON output. Implies -j.

-d, --debug

Print all logs available, even debug-level information. This includes logs from libbpf as well as from the verifier, when attempting to load programs.

-l, --legacy

Use legacy libbpf mode which has more relaxed BPF program requirements. By default, bpftool has more strict requirements about section names, changes pinning logic and doesn't support some of the older non-BTF map declarations. See https://github.com/libbpf/libbpf/wiki/Libbpf:-the-road-to-v1.0 for details.

## EXAMPLES

```
# bpftool net
```

```
xdp:
eth0(2) driver id 198
tc:
eth0(2) htb name prefix_matcher.o:[cls_prefix_matcher_htb] id 111727 act []
eth0(2) clsact/ingress fbflow_icmp id 130246 act []
eth0(2) clsact/egress prefix_matcher.o:[cls_prefix_matcher_clsact] id 111726
eth0(2) clsact/egress cls_fg_dscp id 108619 act []
eth0(2) clsact/egress fbflow_egress id 130245
```

```
# bpftool -jp net
```

```
[{

        "xdp": [{

                "devname": "eth0",

                "ifindex": 2,

                "mode": "driver",

                "id": 198

            }

        ],

        "tc": [{

                "devname": "eth0",

                "ifindex": 2,

                "kind": "htb",

                "name": "prefix_matcher.o:[cls_prefix_matcher_htb]",

                "id": 111727,

                "act": []

            },{

                "devname": "eth0",

                "ifindex": 2,

                "kind": "clsact/ingress",

                "name": "fbflow_icmp",

                "id": 130246,

                "act": []

            },{

                "devname": "eth0",

                "ifindex": 2,

                "kind": "clsact/egress",

                "name": "prefix_matcher.o:[cls_prefix_matcher_clsact]",

                "id": 111726,

            },{

                "devname": "eth0",

                "ifindex": 2,

                "kind": "clsact/egress",

                "name": "cls_fg_dscp",

                "id": 108619,

                "act": []

            },{

                "devname": "eth0",

                "ifindex": 2,

                "kind": "clsact/egress",

                "name": "fbflow_egress",

                "id": 130245,

            }

        ]

    }
]
```

```
# bpftool net attach xdpdrv id 16 dev enp6s0np0
# bpftool net
```

```
xdp:
enp6s0np0(4) driver id 16
```

```
# bpftool net attach xdpdrv id 16 dev enp6s0np0
# bpftool net attach xdpdrv id 20 dev enp6s0np0 overwrite
# bpftool net
```

```
xdp:
enp6s0np0(4) driver id 20
```

```
# bpftool net attach xdpdrv id 16 dev enp6s0np0
# bpftool net detach xdpdrv dev enp6s0np0
# bpftool net
```

```
xdp:
```

## SEE ALSO
bpf(2), bpf-helpers(7), bpftool(8), bpftool-btf(8), bpftool-cgroup(8), bpftool-feature(8), bpftool-gen(8), bpftool-iter(8), bpftool-link(8), bpftool-map(8), bpftool-perf(8), bpftool-prog(8), bpftool-struct_ops(8)


bpftool-perf

title: bpftool-perf(8)

text:
# bpftool-perf(8)

## NAME
bpftool-perf - tool for inspection of perf related bpf prog attachments

## SYNOPSIS
bpftool [OPTIONS] perf COMMAND OPTIONS := { { -j | --json } [{ -p | --pretty }] | { -d | --debug } | { -l | --legacy } }

COMMANDS := { show | list | help }

## PERF COMMANDS

```
bpftool perf { show | list }
bpftool perf help
```

## DESCRIPTION
bpftool perf { show | list }

List all raw_tracepoint, tracepoint, kprobe attachment in the system. Output will start with process id and file descriptor in that process, followed by bpf program id, attachment information, and attachment point. The attachment point for raw_tracepoint/tracepoint is the trace probe name. The attachment point for k[ret]probe is either symbol name and offset, or a kernel virtual address. The attachment point for u[ret]probe is the file name and the file offset.

bpftool perf help

Print short help message.

## OPTIONS
-h, --help

Print short help message (similar to bpftool help).

-V, --version

Print bpftool's version number (similar to bpftool version), the number of the libbpf version in use, and optional features that were included when bpftool was compiled. Optional features include linking against libbfd to provide the disassembler for JIT-ted programs (bpftool prog dump jited) and usage of BPF skeletons (some features like bpftool prog profile or showing pids associated to BPF objects may rely on it).

-j, --json

Generate JSON output. For commands that cannot produce JSON, this option has no effect.

-p, --pretty

Generate human-readable JSON output. Implies -j.

-d, --debug

Print all logs available, even debug-level information. This includes logs from libbpf as well as from the verifier, when attempting to load programs.

-l, --legacy

Use legacy libbpf mode which has more relaxed BPF program requirements. By default, bpftool has more strict requirements about section names, changes pinning logic and doesn't support some of the older non-BTF map declarations. See https://github.com/libbpf/libbpf/wiki/Libbpf:-the-road-to-v1.0 for details.

## EXAMPLES

```
# bpftool perf
```

```
pid 21711  fd 5: prog_id 5  kprobe  func __x64_sys_write  offset 0
pid 21765  fd 5: prog_id 7  kretprobe  func __x64_sys_nanosleep  offset 0
pid 21767  fd 5: prog_id 8  tracepoint  sys_enter_nanosleep
pid 21800  fd 5: prog_id 9  uprobe  filename /home/yhs/a.out  offset 1159
```

```
# bpftool -j perf
```

```
[{"pid":21711,"fd":5,"prog_id":5,"fd_type":"kprobe","func":"__x64_sys_write","offset":0}, \

 {"pid":21765,"fd":5,"prog_id":7,"fd_type":"kretprobe","func":"__x64_sys_nanosleep","offset":0}, \

 {"pid":21767,"fd":5,"prog_id":8,"fd_type":"tracepoint","tracepoint":"sys_enter_nanosleep"}, \

 {"pid":21800,"fd":5,"prog_id":9,"fd_type":"uprobe","filename":"/home/yhs/a.out","offset":1159}]
```

## SEE ALSO
bpf(2), bpf-helpers(7), bpftool(8), bpftool-btf(8), bpftool-cgroup(8), bpftool-feature(8), bpftool-gen(8), bpftool-iter(8), bpftool-link(8), bpftool-map(8), bpftool-net(8), bpftool-prog(8), bpftool-struct_ops(8)


bpftool-prog

title: bpftool-prog(8)

text:
# bpftool-prog(8)

## NAME
bpftool-prog - tool for inspection and simple manipulation of eBPF progs

## SYNOPSIS
bpftool [OPTIONS] prog COMMAND OPTIONS := { { -j | --json } [{ -p | --pretty }] | { -d | --debug } | { -l | --legacy } | { -f | --bpffs } | { -m | --mapcompat } | { -n | --nomount } | { -L | --use-loader } }

COMMANDS := { show | list | dump xlated | dump jited | pin | load | loadall | help }

## PROG COMMANDS

```
bpftool prog { show | list } [PROG]
bpftool prog dump xlated PROG [{file FILE | opcodes | visual | linum}]
bpftool prog dump jited  PROG [{file FILE | opcodes | linum}]
bpftool prog pin PROG FILE
bpftool prog { load | loadall } OBJ PATH [type TYPE] [map {idx IDX | name NAME} MAP] [dev NAME] [pinmaps MAP_DIR]
bpftool prog attach PROG ATTACH_TYPE [MAP]
bpftool prog detach PROG ATTACH_TYPE [MAP]
bpftool prog tracelog
bpftool prog run PROG data_in FILE [data_out FILE [data_size_out L]] [ctx_in FILE [ctx_out FILE [ctx_size_out M]]] [repeat N]
bpftool prog profile PROG [duration DURATION] METRICs
bpftool prog help
MAP := { id MAP_ID | pinned FILE }
PROG := { id PROG_ID | pinned FILE | tag PROG_TAG | name PROG_NAME }
TYPE := {

socket | kprobe | kretprobe | classifier | action |
tracepoint | raw_tracepoint | xdp | perf_event | cgroup/skb |
cgroup/sock | cgroup/dev | lwt_in | lwt_out | lwt_xmit |
lwt_seg6local | sockops | sk_skb | sk_msg | lirc_mode2 |
cgroup/bind4 | cgroup/bind6 | cgroup/post_bind4 | cgroup/post_bind6 |
cgroup/connect4 | cgroup/connect6 | cgroup/getpeername4 | cgroup/getpeername6 |
cgroup/getsockname4 | cgroup/getsockname6 | cgroup/sendmsg4 | cgroup/sendmsg6 |
cgroup/recvmsg4 | cgroup/recvmsg6 | cgroup/sysctl |
cgroup/getsockopt | cgroup/setsockopt | cgroup/sock_release |
struct_ops | fentry | fexit | freplace | sk_lookup

}
ATTACH_TYPE := {

sk_msg_verdict | sk_skb_verdict | sk_skb_stream_verdict |
sk_skb_stream_parser | flow_dissector

}
METRICs := {

cycles | instructions | l1d_loads | llc_misses |
itlb_misses | dtlb_misses

}
```

## DESCRIPTION
bpftool prog { show | list } [PROG]

Show information about loaded programs. If PROG is specified show information only about given programs, otherwise list all programs currently loaded on the system. In case of tag or name, PROG may match several programs which will all be shown. Output will start with program ID followed by program type and zero or more named attributes (depending on kernel version).

Since Linux 5.1 the kernel can collect statistics on BPF programs (such as the total time spent running the program, and the number of times it was run). If available, bpftool shows such statistics. However, the kernel does not collect them by defaults, as it slightly impacts performance on each program run. Activation or deactivation of the feature is performed via the kernel.bpf_stats_enabled sysctl knob.

Since Linux 5.8 bpftool is able to discover information about processes that hold open file descriptors (FDs) against BPF programs. On such kernels bpftool will automatically emit this information as well.

bpftool prog dump xlated PROG [{ file FILE | opcodes | visual | linum }]

Dump eBPF instructions of the programs from the kernel. By default, eBPF will be disassembled and printed to standard output in human-readable format. In this case, opcodes controls if raw opcodes should be printed as well. In case of tag or name, PROG may match several programs which will all be dumped. However, if file or visual is specified, PROG must match a single program.

If file is specified, the binary image will instead be written to FILE.

If visual is specified, control flow graph (CFG) will be built instead, and eBPF instructions will be presented with CFG in DOT format, on standard output.

If the programs have line_info available, the source line will be displayed by default. If linum is specified, the filename, line number and line column will also be displayed on top of the source line.

bpftool prog dump jited PROG [{ file FILE | opcodes | linum }]

Dump jited image (host machine code) of the program. If FILE is specified image will be written to a file, otherwise it will be disassembled and printed to stdout. PROG must match a single program when file is specified.

opcodes controls if raw opcodes will be printed.

If the prog has line_info available, the source line will be displayed by default. If linum is specified, the filename, line number and line column will also be displayed on top of the source line.

bpftool prog pin PROG FILE

Pin program PROG as FILE. Note: FILE must be located in bpffs mount. It must not contain a dot character ('.'), which is reserved for future extensions of bpffs.

bpftool prog { load | loadall } OBJ PATH [type TYPE] [map {idx IDX | name NAME} MAP] [dev NAME] [pinmaps MAP_DIR]

Load bpf program(s) from binary OBJ and pin as PATH. bpftool prog load pins only the first program from the OBJ as PATH. bpftool prog loadall pins all programs from the OBJ under PATH directory. type is optional, if not specified program type will be inferred from section names. By default bpftool will create new maps as declared in the ELF object being loaded. map parameter allows for the reuse of existing maps. It can be specified multiple times, each time for a different map. IDX refers to index of the map to be replaced in the ELF file counting from 0, while NAME allows to replace a map by name. MAP specifies the map to use, referring to it by id or through a pinned file. If dev NAME is specified program will be loaded onto given networking device (offload). Optional pinmaps argument can be provided to pin all maps under MAP_DIR directory. Note: PATH must be located in bpffs mount. It must not contain a dot character ('.'), which is reserved for future extensions of bpffs.

bpftool prog attach PROG ATTACH_TYPE [MAP]

Attach bpf program PROG (with type specified by ATTACH_TYPE). Most ATTACH_TYPEs require a MAP parameter, with the exception of flow_dissector which is attached to current networking name space.

bpftool prog detach PROG ATTACH_TYPE [MAP]

Detach bpf program PROG (with type specified by ATTACH_TYPE). Most ATTACH_TYPEs require a MAP parameter, with the exception of flow_dissector which is detached from the current networking name space.

bpftool prog tracelog

Dump the trace pipe of the system to the console (stdout). Hit <Ctrl+C> to stop printing. BPF programs can write to this trace pipe at runtime with the bpf_trace_printk() helper. This should be used only for debugging purposes. For streaming data from BPF programs to user space, one can use perf events (see also bpftool-map(8)).

bpftool prog run PROG data_in FILE [data_out FILE [data_size_out L]] [ctx_in FILE [ctx_out FILE [ctx_size_out M]]] [repeat N]

Run BPF program PROG in the kernel testing infrastructure for BPF, meaning that the program works on the data and context provided by the user, and not on actual packets or monitored functions etc. Return value and duration for the test run are printed out to the console. Input data is read from the FILE passed with data_in. If this FILE is "-", input data is read from standard input. Input context, if any, is read from FILE passed with ctx_in. Again, "-" can be used to read from standard input, but only if standard input is not already in use for input data. If a FILE is passed with data_out, output data is written to that file. Similarly, output context is written to the FILE passed with ctx_out. For both output flows, "-" can be used to print to the standard output (as plain text, or JSON if relevant option was passed). If output keywords are omitted, output data and context are discarded. Keywords data_size_out and ctx_size_out are used to pass the size (in bytes) for the output buffers to the kernel, although the default of 32 kB should be more than enough for most cases.

Keyword repeat is used to indicate the number of consecutive runs to perform. Note that output data and context printed to files correspond to the last of those runs. The duration printed out at the end of the runs is an average over all runs performed by the command.

Not all program types support test run. Among those which do, not all of them can take the ctx_in/ctx_out arguments. bpftool does not perform checks on program types.

bpftool prog profile PROG [duration DURATION] METRICs

Profile METRICs for bpf program PROG for DURATION seconds or until user hits <Ctrl+C>. DURATION is optional. If DURATION is not specified, the profiling will run up to UINT_MAX seconds.

bpftool prog help

Print short help message.

## OPTIONS
-h, --help

Print short help message (similar to bpftool help).

-V, --version

Print bpftool's version number (similar to bpftool version), the number of the libbpf version in use, and optional features that were included when bpftool was compiled. Optional features include linking against libbfd to provide the disassembler for JIT-ted programs (bpftool prog dump jited) and usage of BPF skeletons (some features like bpftool prog profile or showing pids associated to BPF objects may rely on it).

-j, --json

Generate JSON output. For commands that cannot produce JSON, this option has no effect.

-p, --pretty

Generate human-readable JSON output. Implies -j.

-d, --debug

Print all logs available, even debug-level information. This includes logs from libbpf as well as from the verifier, when attempting to load programs.

-l, --legacy

Use legacy libbpf mode which has more relaxed BPF program requirements. By default, bpftool has more strict requirements about section names, changes pinning logic and doesn't support some of the older non-BTF map declarations. See https://github.com/libbpf/libbpf/wiki/Libbpf:-the-road-to-v1.0 for details.

-f, --bpffs

When showing BPF programs, show file names of pinned programs.

-m, --mapcompat

Allow loading maps with unknown map definitions.

-n, --nomount

Do not automatically attempt to mount any virtual file system (such as tracefs or BPF virtual file system) when necessary.

-L, --use-loader

Load program as a "loader" program. This is useful to debug the generation of such programs. When this option is in use, bpftool attempts to load the programs from the object file into the kernel, but does not pin them (therefore, the PATH must not be provided). When combined with the -d|--debug option, additional debug messages are generated, and the execution of the loader program will use the bpf_trace_printk() helper to log each step of loading BTF, creating the maps, and loading the programs (see bpftool prog tracelog as a way to dump those messages).

## EXAMPLES
\# bpftool prog show

```
10: xdp  name some_prog  tag 005a3d2123620c8b  gpl run_time_ns 81632 run_cnt 10

        loaded_at 2017-09-29T20:11:00+0000  uid 0

        xlated 528B  jited 370B  memlock 4096B  map_ids 10

        pids systemd(1)
```

\# bpftool --json --pretty prog show

```
[{

        "id": 10,

        "type": "xdp",

        "tag": "005a3d2123620c8b",

        "gpl_compatible": true,

        "run_time_ns": 81632,

        "run_cnt": 10,

        "loaded_at": 1506715860,

        "uid": 0,

        "bytes_xlated": 528,

        "jited": true,

        "bytes_jited": 370,

        "bytes_memlock": 4096,

        "map_ids": [10

        ],

        "pids": [{

                "pid": 1,

                "comm": "systemd"

            }

        ]

    }
]
```

```
# bpftool prog dump xlated id 10 file /tmp/t
$ ls -l /tmp/t
```

```
-rw------- 1 root root 560 Jul 22 01:42 /tmp/t
```

\# bpftool prog dump jited tag 005a3d2123620c8b

```
0:   push   %rbp
1:   mov    %rsp,%rbp
2:   sub    $0x228,%rsp
3:   sub    $0x28,%rbp
4:   mov    %rbx,0x0(%rbp)
```

```
# mount -t bpf none /sys/fs/bpf/
# bpftool prog pin id 10 /sys/fs/bpf/prog
# bpftool prog load ./my_prog.o /sys/fs/bpf/prog2
# ls -l /sys/fs/bpf/
```

```
-rw------- 1 root root 0 Jul 22 01:43 prog
-rw------- 1 root root 0 Jul 22 01:44 prog2
```

\# bpftool prog dump jited pinned /sys/fs/bpf/prog opcodes

```
0:   push   %rbp

     55
1:   mov    %rsp,%rbp

     48 89 e5
4:   sub    $0x228,%rsp

     48 81 ec 28 02 00 00
b:   sub    $0x28,%rbp

     48 83 ed 28
f:   mov    %rbx,0x0(%rbp)

     48 89 5d 00
```

```
# bpftool prog load xdp1_kern.o /sys/fs/bpf/xdp1 type xdp map name rxcnt id 7
# bpftool prog show pinned /sys/fs/bpf/xdp1
```

```
9: xdp  name xdp_prog1  tag 539ec6ce11b52f98  gpl

        loaded_at 2018-06-25T16:17:31-0700  uid 0

        xlated 488B  jited 336B  memlock 4096B  map_ids 7
```

\# rm /sys/fs/bpf/xdp1

```
# bpftool prog profile id 337 duration 10 cycles instructions llc_misses
```

```
   51397 run_cnt
40176203 cycles                                                 (83.05%)
42518139 instructions    #   1.06 insns per cycle               (83.39%)

     123 llc_misses      #   2.89 LLC misses per million insns  (83.15%)
```

```
Output below is for the trace logs.
Run in separate terminals:
# bpftool prog tracelog
# bpftool prog load -L -d file.o
```

```
bpftool-620059  [004] d... 2634685.517903: bpf_trace_printk: btf_load size 665 r=5
bpftool-620059  [004] d... 2634685.517912: bpf_trace_printk: map_create sample_map idx 0 type 2 value_size 4 value_btf_id 0 r=6
bpftool-620059  [004] d... 2634685.517997: bpf_trace_printk: prog_load sample insn_cnt 13 r=7
bpftool-620059  [004] d... 2634685.517999: bpf_trace_printk: close(5) = 0
```

## SEE ALSO
bpf(2), bpf-helpers(7), bpftool(8), bpftool-btf(8), bpftool-cgroup(8), bpftool-feature(8), bpftool-gen(8), bpftool-iter(8), bpftool-link(8), bpftool-map(8), bpftool-net(8), bpftool-perf(8), bpftool-struct_ops(8)


title: bpftool-struct_ops(8)

text:
# bpftool-struct_ops(8)

## NAME
bpftool-struct_ops - tool to register/unregister/introspect BPF struct_ops

## SYNOPSIS
bpftool [OPTIONS] struct_ops COMMAND OPTIONS := { { -j | --json } [{ -p | --pretty }] | { -d | --debug } | { -l | --legacy } }

COMMANDS := { show | list | dump | register | unregister | help }

## STRUCT_OPS COMMANDS

```
bpftool struct_ops { show | list } [STRUCT_OPS_MAP]
bpftool struct_ops dump [STRUCT_OPS_MAP]
bpftool struct_ops register OBJ
bpftool struct_ops unregister STRUCT_OPS_MAP
bpftool struct_ops help
STRUCT_OPS_MAP := { id STRUCT_OPS_MAP_ID | name STRUCT_OPS_MAP_NAME }
OBJ := /a/file/of/bpf_struct_ops.o
```

## DESCRIPTION
bpftool struct_ops { show | list } [STRUCT_OPS_MAP]

Show brief information about the struct_ops in the system. If STRUCT_OPS_MAP is specified, it shows information only for the given struct_ops. Otherwise, it lists all struct_ops currently existing in the system. Output will start with struct_ops map ID, followed by its map name and its struct_ops's kernel type.

bpftool struct_ops dump [STRUCT_OPS_MAP]

Dump details information about the struct_ops in the system. If STRUCT_OPS_MAP is specified, it dumps information only for the given struct_ops. Otherwise, it dumps all struct_ops currently existing in the system.

bpftool struct_ops register OBJ

Register bpf struct_ops from OBJ. All struct_ops under the ELF section ".struct_ops" will be registered to its kernel subsystem.

bpftool struct_ops unregister STRUCT_OPS_MAP

Unregister the STRUCT_OPS_MAP from the kernel subsystem.

bpftool struct_ops help

Print short help message.

## OPTIONS
-h, --help

Print short help message (similar to bpftool help).

-V, --version

Print bpftool's version number (similar to bpftool version), the number of the libbpf version in use, and optional features that were included when bpftool was compiled. Optional features include linking against libbfd to provide the disassembler for JIT-ted programs (bpftool prog dump jited) and usage of BPF skeletons (some features like bpftool prog profile or showing pids associated to BPF objects may rely on it).

-j, --json

Generate JSON output. For commands that cannot produce JSON, this option has no effect.

-p, --pretty

Generate human-readable JSON output. Implies -j.

-d, --debug

Print all logs available, even debug-level information. This includes logs from libbpf as well as from the verifier, when attempting to load programs.

-l, --legacy

Use legacy libbpf mode which has more relaxed BPF program requirements. By default, bpftool has more strict requirements about section names, changes pinning logic and doesn't support some of the older non-BTF map declarations. See https://github.com/libbpf/libbpf/wiki/Libbpf:-the-road-to-v1.0 for details.

## EXAMPLES
\# bpftool struct_ops show

```
100: dctcp           tcp_congestion_ops
105: cubic           tcp_congestion_ops
```

\# bpftool struct_ops unregister id 105

```
Unregistered tcp_congestion_ops cubic id 105
```

\# bpftool struct_ops register bpf_cubic.o

```
Registered tcp_congestion_ops cubic id 110
```

## SEE ALSO
bpf(2), bpf-helpers(7), bpftool(8), bpftool-btf(8), bpftool-cgroup(8), bpftool-feature(8), bpftool-gen(8), bpftool-iter(8), bpftool-link(8), bpftool-map(8), bpftool-net(8), bpftool-perf(8), bpftool-prog(8)


bpftrace

package: bpftrace

title: bpftrace(8)

summary: the eBPF tracing language & frontend

text:
# bpftrace(8)

## NAME
bpftrace - the eBPF tracing language & frontend

## SYNOPSIS
bpftrace [OPTIONS] FILE

bpftrace [OPTIONS] -e 'program code'

## DESCRIPTION
bpftrace is a high-level tracing language for Linux enhanced Berkeley Packet Filter (eBPF) available in recent Linux kernels (4.x).

bpftrace uses:

- LLVM as a backend to compile scripts to BPF-bytecode

- BCC for interacting with the Linux BPF system

As well as the existing Linux tracing capabilities:

kernel

userland

static

tracepoints

USDT* probes

dynamic

kprobes

uprobes

*USDT = user-level statically defined tracing

The bpftrace language is inspired by awk and C, and predecessor tracers such as DTrace and SystemTap.

See EXAMPLES and ONELINERS if you are impatient.

See PROBE TYPES and BUILTINS (variables/functions) for the bpftrace language elements.

## OPTIONS
-B MODE

The buffering mode to be used for output ('full', 'none').

-c CMD

Helper to run CMD. Equivalent to manually running CMD and then giving passing the PID to -p. This is useful to ensure you've traced at least the duration CMD's execution.

-d

Debug info dry run.

-dd

Verbose Debug info dry run.

-e 'PROGRAM'

Execute PROGRAM.

-f FORMAT

The format to be used for output ('text'. 'json').

-h | --help

Show help message.

-I DIR

Add the directory to the INCLUDE search path.

--include FILE

Add a #include file before preprocessing.

--info

Print information about kernel BPF support.

-k

Emit a warning when a bpf helper returns an error (except read functions).

-kk

Check all bpf helper functions.

-l [searchterm]

List probes.

-o FILE

Redirect bpftrace output to file.

-p PID

Enable USDT probes on PID. Will terminate bpftrace on PID termination. Note this is not a global PID filter on probes.

--unsafe

Enable unsafe builtin functions. By default, bpftrace runs in safe mode. Safe mode ensure programs cannot modify system state. Unsafe builtin functions are marked as such in BUILTINS (functions).

--usdt-file-activation

Activate usdt semaphores based on file path..

-v

Verbose messages.

-V | --version

Print version of bpftrace.

## EXAMPLES
bpftrace -l '*sleep*'

List probes containing "sleep".

bpftrace -e 'kprobe:do_nanosleep { printf("PID %d sleeping\n", pid); }'

Trace processes calling sleep.

bpftrace -c 'sleep 5' -e 'kprobe:do_nanosleep { printf("PID %d sleeping\n", pid); }'

run "sleep 5" in a new process and then trace processes calling sleep.

bpftrace -e 'tracepoint:raw_syscalls:sys_enter { @[comm]=count(); }'

Count syscalls by process name.

## ONELINERS
For brevity, just the the actual BPF code is shown below.

Usage: bpftrace -e 'bpf-code'

New processes with arguments:

tracepoint:syscalls:sys_enter_execve { join(args->argv); }

Files opened by process:

tracepoint:syscalls:sys_enter_open { printf("%s %s\n", comm, str(args->filename)); }

Syscall count by program:

tracepoint:raw_syscalls:sys_enter { @[comm] = count(); }

Syscall count by syscall:

tracepoint:syscalls:sys_enter_* { @[probe] = count(); }

Syscall count by process:

tracepoint:raw_syscalls:sys_enter { @[pid, comm] = count(); }

Read bytes by process:

tracepoint:syscalls:sys_exit_read /args->ret/ { @[comm] = sum(args->ret); }

Read size distribution by process:

tracepoint:syscalls:sys_exit_read { @[comm] = hist(args->ret); }

Disk size by process:

tracepoint:block:block_rq_issue { printf("%d %s %d\n", pid, comm, args->bytes); }

Pages paged in by process:

software:major-faults:1 { @[comm] = count(); }

Page faults by process:

software:faults:1 { @[comm] = count(); }

Profile user-level stacks at 99 Hertz, for PID 189:

profile:hz:99 /pid == 189/ { @[ustack] = count(); }

## PROBE TYPES

### KPROBES
Attach a bpftrace script to a kernel function, to be executed when that function is called:

kprobe:vfs_read { ... }

### UPROBES
Attach script to a userland function:

uprobe:/bin/bash:readline { ... }

### TRACEPOINTS
Attach script to a statically defined tracepoint in the kernel:

tracepoint:sched:sched_switch { ... }

Tracepoints are guaranteed to be stable between kernel versions, unlike kprobes.

### SOFTWARE
Attach script to kernel software events, executing once every provided count or use a default:

software:faults:100 software:faults:

### HARDWARE
Attach script to hardware events (PMCs), executing once every provided count or use a default:

hardware:cache-references:1000000 hardware:cache-references:

### PROFILE
Run the script on all CPUs at specified time intervals:

profile:hz:99 { ... }

profile:s:1 { ... }

profile:ms:20 { ... }

profile:us:1500 { ... }

### INTERVAL
Run the script once per interval, for printing interval output:

interval:s:1 { ... }

interval:ms:20 { ... }

### MULTIPLE ATTACHMENT POINTS
A single probe can be attached to multiple events:

kprobe:vfs_read,kprobe:vfs_write { ... }

### WILDCARDS
Some probe types allow wildcards to be used when attaching a probe:

kprobe:vfs_* { ... }

### PREDICATES
Define conditions for which a probe should be executed:

kprobe:sys_open / uid == 0 / { ... }

## BUILTINS
The following variables and functions are available for use in bpftrace scripts:

### VARIABLES
pid

Process ID (kernel tgid)

tid

Thread ID (kernel pid)

cgroup

Cgroup ID of the current process

uid

User ID

gid

Group ID

nsecs

Nanosecond timestamp

elapsed

Nanoseconds since bpftrace initialization

cpu

Processor ID

comm

Process name

kstack

Kernel stack trace

ustack

User stack trace

arg0, arg1, ... etc.

Arguments to the function being traced

retval

Return value from function being traced

func

Name of the function currently being traced

probe

Full name of the probe

curtask

Current task_struct as a u64.

rand

Random number of type u32.

### FUNCTIONS
hist(int n)

Produce a log2 histogram of values of n

lhist(int n, int min, int max, int step)

Produce a linear histogram of values of n

count()

Count the number of times this function is called

sum(int n)

Sum this value

min(int n)

Record the minimum value seen

max(int n)

Record the maximum value seen

avg(int n)

Average this value

stats(int n)

Return the count, average, and total for this value

delete(@x)

Delete the map element passed in as an argument

str(char *s)

Returns the string pointed to by s

printf(char *fmt, ...)

Print formatted to stdout

print(@x[, int top [, int div]])

Print a map, with optional top entry count and divisor

clear(@x)

Delete all key/values from a map

ksym(void *p)

Resolve kernel address

usym(void *p)

Resolve user space address

kaddr(char *name)

Resolve kernel symbol name

uaddr(char *name)

Resolve user space symbol name

reg(char *name)

Returns the value stored in the named register

join(char *arr[])

Prints the string array

time(char *fmt)

Print the current time

cat(char *filename)

Print file content

ntop([int af, ]int|char[4|16] addr)

Convert IP address data to text

system(char *fmt) (unsafe)

Execute shell command

exit()

Quit bpftrace

kstack([StackMode mode, ][int level])

Kernel stack trace

ustack([StackMode mode, ][int level])

User stack trace

## FURTHER READING
The official documentation can be found here:

https://github.com/iovisor/bpftrace/blob/master/docs

## HISTORY
The first official talk by Alastair on bpftrace happened at the Tracing Summit in Edinburgh, Oct 25th 2018.

## AUTHOR
Created by Alastair Robertson.

Manpage by Stephan Schuberth.

## SEE ALSO
man -k bcc, after having installed the bpfcc-tools package under Ubuntu.

## CONTRIBUTING
Prior to contributing new tools, read the official checklist at:

https://github.com/iovisor/bpftrace/blob/master/CONTRIBUTING-TOOLS.md


bpftrace-bashreadline

title: bpftrace-bashreadline(8)

text:
# bpftrace-bashreadline(8)

## NAME
bashreadline.bt - Print bash commands system wide. Uses bpftrace/eBPF.

## SYNOPSIS
bashreadline.bt

## DESCRIPTION
bashreadline traces the return of the readline() function using uretprobes, to show the bash commands that were entered interactively, system wide. The entered command may fail: this is just showing what was entered.

This program is also a basic example of bpftrace and uretprobes.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace bash commands system wide:

\# bashreadline.bt

## FIELDS
TIME

A timestamp on the output, in "HH:MM:SS" format.

PID

The process ID for bash.

COMMAND

Entered command.

## OVERHEAD
As the rate of interactive bash commands is expected to be very low (<<100/s), the overhead of this program is expected to be negligible.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool may provide more options and customizations.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
opensnoop(8)


bpftrace-biolatency

title: bpftrace-biolatency(8)

text:
# bpftrace-biolatency(8)

## NAME
biolatency.bt - Block I/O latency as a histogram. Uses bpftrace/eBPF.

## SYNOPSIS
biolatency.bt

## DESCRIPTION
This tool summarizes time (latency) spent in block device I/O (disk I/O) as a power-of-2 histogram. This allows the distribution to be studied, including modes and outliers. There are often two modes, one for device cache hits and one for cache misses, which can be shown by this tool. Latency outliers will also be shown.

This tool currently works by dynamic tracing of the blk_account*() kernel functions, which will need updating to match any changes to these functions in future kernels versions.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace block device I/O (disk I/O), and print a latency histogram on Ctrl-C:

\# biolatency.bt

## FIELDS
1st, 2nd

This is a range of latency, in microseconds (shown in "[...)" set notation).

3rd

A column showing the count of operations in this range.

4th

This is an ASCII histogram representing the count column.

## OVERHEAD
Since block device I/O usually has a relatively low frequency (< 10,000/s), the overhead for this tool is expected to be negligible. For high IOPS storage systems, test and quantify before use.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool may provide more options and customizations.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
biosnoop(1)


bpftrace-biosnoop

title: bpftrace-biosnoop(8)

text:
# bpftrace-biosnoop(8)

## NAME
biosnoop.bt - Block I/O tracing tool, showing per I/O latency. Uses bpftrace/eBPF.

## SYNOPSIS
biosnoop.bt

## DESCRIPTION
This is a basic block I/O (disk I/O) tracing tool, showing each I/O event along with the issuing process ID, and the I/O latency. This can be used to investigate disk I/O performance issues.

This tool currently works by dynamic tracing of the blk_account*() kernel functions, which will need updating to match any changes to these functions in future kernels versions.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace block I/O events, printing per-line summaries:

\# biosnoop.bt

## FIELDS
TIME

Time of the I/O completion, in milliseconds since program start.

COMM

Issuing process name. This often identifies the issuing application process, but I/O may be initiated from kernel threads only.

PID

Issuing process ID. This often identifies the issuing application process, but I/O may be initiated from kernel threads only.

ARGS

Process name and arguments (16 word maximum).

## OVERHEAD
Since block device I/O usually has a relatively low frequency (< 10,000/s), the overhead for this tool is expected to be negligible. For high IOPS storage systems, test and quantify before use.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool provides more fields.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
opensnoop(8)


bpftrace-biostacks

title: bpftrace-biostacks(8)

text:
# bpftrace-biostacks(8)

## NAME
biostacks - Show disk I/O latency with initialization stacks. Uses bpftrace/eBPF.

## SYNOPSIS
biostacks

## DESCRIPTION
This tool shows disk I/O latency histograms for each block I/O initialization path. This can help reveal the reason for different latencies, as some may be created by log flushing, others by application reads, etc.

This works by tracing the blk_account_io_start() and the blk_start_request() or blk_mq_start_request() functions using dynamic instrumentation. Linux 5.0 removed the classic I/O scheduler, so the blk_start_request() probe can be removed from the tool (just delete it). This tool may need other maintenance to keep working if these functions change in later kernels.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace disk I/O latency with initialization stacks:

\# biostacks.bt

## FIELDS
0th

An initialization kernel stack trace (shown in "@[...]") is printed before each I/O histogram.

1st, 2nd

This is a range of I/O latency, in microseconds (shown in "[...)" set notation).

3rd

A column showing the count of I/O in this range.

4th

This is an ASCII histogram representing the count column.

## OVERHEAD
The rate of biostacks should be low (bounded by device IOPS), such that the overhead of this tool is expected to be negligible.

## SOURCE
This tool originated from the book "BPF Performance Tools", published by Addison Wesley (2019):

http://www.brendangregg.com/bpf-performance-tools-book.html

See the book for more documentation on this tool.

This version is in the bpftrace repository:

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
biosnoop(8)


bpftrace-bitesize

title: bpftrace-bitesize(8)

text:
# bpftrace-bitesize(8)

## NAME
bitesize.bt - Show disk I/O size as a histogram. Uses bpftrace/eBPF.

## SYNOPSIS
bitesize.bt

## DESCRIPTION
This can be used to characterize the distribution of block device (disk) I/O sizes. To study block device I/O in more detail, see biosnoop(8).

This uses the tracepoint:block:block_rq_issue tracepoint, and is a simple example of bpftrace.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace block I/O and summarize as a histogram by process:

\# bitesize.bt

## FIELDS
0th

A process name (shown in "@[...]") is printed before each I/O histogram.

1st, 2nd

This is a range of I/O sizes, in Kbytes (shown in "[...)" set notation).

3rd

A column showing the count of I/O in this range.

4th

This is an ASCII histogram representing the count column.

## OVERHEAD
Since block device I/O usually has a relatively low frequency (< 10,000/s), the overhead for this tool is expected to be low or negligible. For high IOPS storage systems, test and quantify before use.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
biosnoop.bt(8)


bpftrace-capable

title: bpftrace-capable(8)

text:
# bpftrace-capable(8)

## NAME
capable.bt - Trace security capability checks (cap_capable()).

## SYNOPSIS
capable.bt

## DESCRIPTION
This traces security capability checks in the kernel, and prints details for each call. This can be useful for general debugging, and also security enforcement: determining a white list of capabilities an application needs.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF, bpftrace.

## EXAMPLES
Trace all capability checks system-wide:

\# capable.bt

## FIELDS
TIME(s)

Time of capability check: HH:MM:SS.

UID

User ID.

PID

Process ID.

COMM

Process name. CAP Capability number. NAME Capability name. See capabilities(7) for descriptions.

AUDIT

Whether this was an audit event.

## OVERHEAD
This adds low-overhead instrumentation to capability checks, which are expected to be low frequency, however, that depends on the application. Test in a lab environment before use.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool provides options to customize the output.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
capabilities(7)


bpftrace-cpuwalk

title: bpftrace-cpuwalk(8)

text:
# bpftrace-cpuwalk(8)

## NAME
cpuwalk.bt - Sample which CPUs are executing processes.. Uses bpftrace/eBPF.

## SYNOPSIS
cpuwalk.bt

## DESCRIPTION
This tool samples CPUs at 99 Hertz, then prints a histogram showing which CPUs were active. 99 Hertz is used to avoid lockstep sampling that would skew results. This tool can help identify if your application's workload is evenly using the CPUs, or if there is an imbalance problem.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Sample CPUs and print a summary on Ctrl-C:

\# cpuwalk.bt

## FIELDS
1st, 2nd

The CPU is shown in the first field, after the "[". Disregard the second field.

3rd

A column showing the number of samples for this CPU.

4th

This is an ASCII histogram representing the count column.

## OVERHEAD
This should be negligible.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
mpstat(1)


bpftrace-dcsnoop

title: bpftrace-dcsnoop(8)

text:
# bpftrace-dcsnoop(8)

## NAME
dcsnoop.bt - Trace directory entry cache (dcache) lookups. Uses bpftrace/eBPF.

## SYNOPSIS
dcsnoop.bt

## DESCRIPTION
By default, this traces every dcache lookup, and shows the process performing the lookup and the filename requested.

The output of this tool can be verbose, and is intended for further investigations of dcache performance beyond dcstat(8), which prints per-second summaries.

This uses kernel dynamic tracing of the d_lookup() function, and will need and will need updating to match any changes to this function.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bcc.

## EXAMPLES
Trace all dcache lookups:

\# dcsnoop.bt

## FIELDS
TIME(ms)

Time of lookup, in milliseconds.

PID

Process ID.

COMM

Process name.

T

Type: R == reference, M == miss. A miss will print two lines, one for the reference, and one for the miss.

FILE

The file name component that was being looked up. This contains trailing pathname components (after '/'), which will be the subject of subsequent lookups.

## OVERHEAD
File name lookups can be frequent (depending on the workload), and this tool prints a line for each failed lookup, and with -a, each reference as well. The output may be verbose, and the incurred overhead, while optimized to some extent, may still be from noticeable to significant. This is only really intended for deeper investigations beyond dcstat(8), when absolutely necessary. Measure and quantify the overhead in a test environment before use.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool may provide more options and customizations.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
dcstat(8)


bpftrace-execsnoop

title: bpftrace-execsnoop(8)

text:
# bpftrace-execsnoop(8)

## NAME
execsnoop.bt - Trace new processes via exec() syscalls. Uses bpftrace/eBPF.

## SYNOPSIS
execsnoop.bt

## DESCRIPTION
This traces when processes call exec() (execve()). It is handy for identifying new processes created via the usual fork()->exec() sequence. Note that the return value is not currently traced, so the exec() may have failed.

This tool is useful for debugging shell scripts, including application startup. It is also useful for identifying a type of performance issue: a flood of short-lived processes, that end quickly and aren't readily visible in top(1).

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace all new processes calling execve():

\# execsnoop.bt

## FIELDS
TIME

Time of the exec() call, in milliseconds since program start.

PID

Process ID

ARGS

Process name and arguments (16 word maximum).

## OVERHEAD
This traces the execve() tracepoint and prints output for each event. As the rate of this is generally expected to be low (< 100/s), the overhead is also expected to be negligible. If you have an application that is spawning a high rate of new processes for a reason (large build process), this could cause a small amount of overhead: test and understand overhead before use.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool provides more fields and options to customize the output.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
opensnoop(8)


bpftrace-gethostlatency

title: bpftrace-gethostlatency(8)

text:
# bpftrace-gethostlatency(8)

## NAME
gethostlatency.bt - Show latency for getaddrinfo/gethostbyname[2] calls. Uses bpftrace/eBPF.

## SYNOPSIS
gethostlatency.bt

## DESCRIPTION
This traces and prints when getaddrinfo(), gethostbyname(), and gethostbyname2() are called, system wide, and shows the responsible PID and command name, latency of the call (duration) in milliseconds, and the host string.

This tool can be useful for identifying DNS latency, by identifying which remote host name lookups were slow, and by how much.

This tool currently uses dynamic tracing of user-level functions and registers, and may need modifications to match your software and processor architecture.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bcc.

## EXAMPLES
Trace host lookups (getaddrinfo/gethostbyname[2]) system wide:

\# gethostlatency.bt

## FIELDS
TIME

Time of the command (HH:MM:SS).

PID

Process ID of the client performing the call.

COMM

Process (command) name of the client performing the call.

LATms

Latency of the call, in milliseconds.

HOST

Host name string: the target of the lookup.

## OVERHEAD
The rate of lookups should be relatively low, so the overhead is not expected to be a problem.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool provides command line options.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
tcpdump(8)


bpftrace-killsnoop

title: bpftrace-killsnoop(8)

text:
# bpftrace-killsnoop(8)

## NAME
killsnoop.bt - Trace signals issued by the kill() syscall. Uses bpftrace/eBPF.

## SYNOPSIS
killsnoop.bt

## DESCRIPTION
killsnoop traces the kill() syscall, to show signals sent via this method. This may be useful to troubleshoot failing applications, where an unknown mechanism is sending signals.

This works by tracing the kill() syscall tracepoints.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace all kill() syscalls:

\# killsnoop.bt

## FIELDS
TIME

Time of the kill call.

PID

Source process ID

COMM

Source process name

SIG

Signal number. See signal(7).

TPID

Target process ID

RES

Result. 0 == success, a negative value (of the error code) for failure.

## OVERHEAD
This traces the kernel kill function and prints output for each event. As the rate of this is generally expected to be low (< 100/s), the overhead is also expected to be negligible. If you have an application that is calling a very high rate of kill()s for some reason, then test and understand overhead before use.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool may provide more options and customizations.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
opensnoop(8)


bpftrace-loads

title: bpftrace-loads(8)

text:
# bpftrace-loads(8)

## NAME
loads.bt - Prints load averages. Uses bpftrace/eBPF.

## SYNOPSIS
loads.bt

## DESCRIPTION
These are the same load averages printed by "uptime", but to three decimal places instead of two (not that it really matters). This is really a demonstration of fetching and processing a kernel structure from bpftrace.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Print system load averages every second:

\# loads.bt

## FIELDS
HH:MM:SS

Each output line includes time of printing in "HH:MM:SS" format.

load averages:

These are exponentially-damped moving sum averages of the system loads. Load is a measurement of demand on system resources, which include CPUs and other resources that are accessed with the kernel in an uninterruptible state (TASK_UNINTERRUPTIBLE), which includes types of disk I/O and lock accesses. Linux load averages originally reflected CPU demand only, as it does in other OSes, but this was changed in Linux 0.99.14. This demand measurement reflects not just the utilized resource, but also the queued demand (a saturation measurement). Finally, the three numbers are called the "one-", "five-", and "fifteen-minute" load averages, however these times are constants used in the exponentially-damping equation, and the load averages reflect load beyond these times. Were you expecting an accurate description of load averages in the man page of a bpftrace tool?

## OVERHEAD
Other than bpftrace startup time, negligible.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

## REFERENCE
For more on load averages, see:

http://www.brendangregg.com/blog/2017-08-08/linux-load-averages.html

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
uptime(1)


bpftrace-mdflush

title: bpftrace-mdflush(8)

text:
# bpftrace-mdflush(8)

## NAME
mdflush.bt - Trace md flush events. Uses bpftrace/eBPF.

## SYNOPSIS
mdflush.bt

## DESCRIPTION
This tool traces flush events by md, the Linux multiple device driver (software RAID). The timestamp and md device for the flush are printed. Knowing when these flushes happen can be useful for correlation with unexplained spikes in disk latency.

This works by tracing the kernel md_flush_request() function using kernel dynamic tracing, and will need updating to match any changes to this function.

Note that the flushes themselves are likely to originate from higher in the I/O stack, such as from the file systems.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace md flush events:

\# mdflush.bt

## FIELDS
TIME

Time of the flush event (HH:MM:SS).

PID

The process ID that was on-CPU when the event was issued. This may identify the cause of the flush (eg, the "sync" command), but will often identify a kernel worker thread that was managing I/O.

COMM

The command name for the PID.

DEVICE

The md device name.

## OVERHEAD
Expected to be negligible.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool may provide more options and customizations.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
biosnoop(8)


bpftrace-naptime

title: bpftrace-naptime(8)

text:
# bpftrace-naptime(8)

## NAME
naptime.bt - Trace voluntary sleep calls. Uses bpftrace/eBPF.

## SYNOPSIS
naptime.bt

## DESCRIPTION
This tool traces application sleeps, and can be used for debugging high latency that may be caused by deliberate sleeps placed in application routines, especially administration scripts.

This tool works by tracing the nanosleep(2) syscall using the syscall tracepoints.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace application sleeps via nanosleep(2):

\# naptime.bt

## FIELDS
TIME

A timestamp in HH:MM:SS format.

PPID

Parent process ID.

PCOMM

Parent process name.

PID

The sleeping process ID.

COMM

The sleeping process name.

SECONDS

The requested duration of the sleep.

## OVERHEAD
nanosleep(2) calls are expected to be low frequency (<< 100/s), so the overhead of this tool is expected to be negligible.

## SOURCE
This tool originated from the book "BPF Performance Tools", published by Addison Wesley (2019):

http://www.brendangregg.com/bpf-performance-tools-book.html

See the book for more documentation on this tool.

This version is in the bpftrace repository:

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
capable(8)


bpftrace-oomkill

title: bpftrace-oomkill(8)

summary: Trace OOM killer. Uses bpftrace/eBPF.

text:
# bpftrace-oomkill(8)

## NAME
oomkill.bt - Trace OOM killer. Uses bpftrace/eBPF.

## SYNOPSIS
oomkill.bt

## DESCRIPTION
This traces the kernel out-of-memory killer, and prints basic details, including the system load averages at the time of the OOM kill. This can provide more context on the system state at the time: was it getting busier or steady, based on the load averages? This tool may also be useful to customize for investigations; for example, by adding other task_struct details at the time of OOM, or by adding other commands to run at the shell.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace OOM kill events:

\# oomkill.bt

## FIELDS
Triggered by ...

The process ID and process name of the task that was running when another task was OOM killed.

OOM kill of ...

The process ID and name of the target process that was OOM killed.

loadavg

Contents of /proc/loadavg. The first three numbers are 1, 5, and 15 minute load averages (where the average is an exponentially damped moving sum, and those numbers are constants in the equation); then there is the number of running tasks, a slash, and the total number of tasks; and then the last number is the last PID to be created.

## OVERHEAD
Negligible.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool may provide more options and customizations.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
dmesg(1)


bpftrace-opensnoop

title: bpftrace-opensnoop(8)

text:
# bpftrace-opensnoop(8)

## NAME
opensnoop.bt - Trace open() syscalls. Uses bpftrace/eBPF.

## SYNOPSIS
opensnoop.bt

## DESCRIPTION
opensnoop traces the open() syscall, showing which processes are attempting to open which files. This can be useful for determining the location of config and log files, or for troubleshooting applications that are failing, specially on startup.

This works by tracing the open() syscall tracepoint.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bcc.

## EXAMPLES
Trace all open() syscalls:

\# opensnoop.bt

## FIELDS
PID Process ID

TID

Thread ID

COMM

Process name

FD

File descriptor (if success), or -1 (if failed)

ERR

Error number (see the system's errno.h)

PATH

Open path

## OVERHEAD
This traces the open tracepoint and prints output for each event. As the rate of this is generally expected to be low (< 1000/s), the overhead is also expected to be negligible. If you have an application that is calling a high rate of open()s, then test and understand overhead before use.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool may provide more options and customizations.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
statsnoop(8), execsnoop(8)


bpftrace-pidpersec

title: bpftrace-pidpersec(8)

text:
# bpftrace-pidpersec(8)

## NAME
pidpersec.bt - Count new processes (via fork()). Uses bpftrace/eBPF.

## SYNOPSIS
pidpersec.bt

## DESCRIPTION
pidpersec shows how many new processes were created each second. There can be performance issues caused by many short-lived processes, which may not be visible in sampling tools like top(1). pidpersec provides one way to investigate this behavior.

This works by tracing the tracepoint:sched:sched_process_fork tracepoint.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Count new processes, printing per-second summaries until Ctrl-C is hit:

\# pidpersec.bt

## FIELDS
1st

Count of processes (after "@")

## OVERHEAD
This traces kernel forks, and maintains an in-kernel count which is read asynchronously from user-space. As the rate of this is generally expected to be low (<< 1000/s), the overhead is also expected to be negligible.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool may provide more options and customizations.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
top(1)


bpftrace-runqlat

title: bpftrace-runqlat(8)

text:
# bpftrace-runqlat(8)

## NAME
runqlat.bt - CPU scheduler run queue latency as a histogram. Uses bpftrace/eBPF.

## SYNOPSIS
runqlat.bt

## DESCRIPTION
This traces time spent waiting in the CPU scheduler for a turn on-CPU. This metric is often called run queue latency, or scheduler latency. This tool shows this latency as a power-of-2 histogram in nanoseconds, allowing multimodal distributions to be studied, as well as latency outliers.

This tool uses the sched tracepoints.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace CPU run queue latency system wide, printing a histogram on Ctrl-C:

\# runqlat.bt

## FIELDS
1st, 2nd

This is a range of latency, in microseconds (shown in "[...)" set notation).

3rd

A column showing the count of scheduler events in this range.

4th

This is an ASCII histogram representing the count column.

## OVERHEAD
This traces scheduler functions, which can become very frequent. While eBPF has very low overhead, and this tool uses in-kernel maps for efficiency, the frequency of scheduler events for some workloads may be high enough that the overhead of this tool becomes significant. Measure in a lab environment to quantify the overhead before use.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool may provide more options and customizations.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
runqlen(8), mpstat(1), pidstat(1), uptime(1)


bpftrace-runqlen

title: bpftrace-runqlen(8)

text:
# bpftrace-runqlen(8)

## NAME
runqlen.bt - CPU scheduler run queue length as a histogram. Uses bpftrace/eBPF.

## SYNOPSIS
runqlen.bt

## DESCRIPTION
This program summarizes scheduler queue length as a histogram, and can also show run queue occupancy. It works by sampling the run queue length on all CPUs at 99 Hertz.

This tool can be used to identify imbalances, eg, when processes are bound to CPUs causing queueing, or interrupt mappings causing the same.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace CPU run queue length system wide, printing a histogram on Ctrl-C:

\# runqlen.bt

## FIELDS
1st, 2nd

The run queue length is shown in the first field (after "[").

3rd

A column showing the count of samples in for that length.

4th

This is an ASCII histogram representing the count column.

## OVERHEAD
This samples scheduler structs at 99 Hertz across all CPUs. Relatively, this is a low rate of events, and the overhead of this tool is expected to be near zero.

## SOURCE
This is from bpftrace.

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

This is a bpftrace version of the bcc tool of the same name. The bcc tool may provide more options and customizations.

https://github.com/iovisor/bcc

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
runqlat(8), mpstat(1), pidstat(1), uptime(1)


bpftrace-setuids

title: bpftrace-setuids(8)

text:
# bpftrace-setuids(8)

## NAME
setuids.bt - Trace setuid family of syscalls. Uses bpftrace/eBPF.

## SYNOPSIS
setuids.bt

## DESCRIPTION
This tool traces privilege escalation via setuid syscalls, and can be used for debugging, whitelist creation, and intrusion detection.

It works by tracing the setuid(2), setfsuid(2), and retresuid(2) syscalls using the syscall tracepoints.

Since this uses BPF, only the root user can use this tool.

## REQUIREMENTS
CONFIG_BPF and bpftrace.

## EXAMPLES
Trace setuid syscalls:

\# setuids.bt

## FIELDS
PID

The calling process ID.

COMM

The calling process (thread) name.

UID

The UID of the caller.

SYSCALL

The syscall name.

ARGS

The arguments to the syscall

(RET)

The return value for the syscall: 0 == success, other numbers indicate an error code.

## OVERHEAD
setuid calls are expected to be low frequency (<< 100/s), so the overhead of this tool is expected to be negligible.

## SOURCE
This tool originated from the book "BPF Performance Tools", published by Addison Wesley (2019):

http://www.brendangregg.com/bpf-performance-tools-book.html

See the book for more documentation on this tool.

This version is in the bpftrace repository:

https://github.com/iovisor/bpftrace

Also look in the bpftrace distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

## OS
Linux

## STABILITY
Unstable - in development.

## AUTHOR
Brendan Gregg

## SEE ALSO
capable(8)