platform/upstream/bcc.git
2 years agolibbpf-tools: update execsnoop for libbpf 1.0
Andrii Nakryiko [Fri, 17 Dec 2021 22:22:09 +0000 (14:22 -0800)]
libbpf-tools: update execsnoop for libbpf 1.0

Switch to libbpf 1.0 mode and adapt libbpf API usage accordingly.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
2 years agolibbpf-tools: update cpudist for libbpf 1.0
Andrii Nakryiko [Fri, 17 Dec 2021 22:21:44 +0000 (14:21 -0800)]
libbpf-tools: update cpudist for libbpf 1.0

Switch to libbpf 1.0 mode and adapt libbpf API usage accordingly.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
2 years agolibbpf-tools: update drsnoop for libbpf 1.0
Andrii Nakryiko [Fri, 17 Dec 2021 22:21:25 +0000 (14:21 -0800)]
libbpf-tools: update drsnoop for libbpf 1.0

Switch to libbpf 1.0 mode and adapt libbpf API usage accordingly.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
2 years agolibbpf-tools: fix cpufreq.bpf.c and update cpufreq for libbpf 1.0
Andrii Nakryiko [Fri, 17 Dec 2021 22:20:32 +0000 (14:20 -0800)]
libbpf-tools: fix cpufreq.bpf.c and update cpufreq for libbpf 1.0

Switch to libbpf 1.0 mode and adapt libbpf API usage accordingly.

Also fix cachestat.bpf.c by adding a BPF assembly trick to ensure that
BPF verifier sees proper value bounds for cpu ID.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
2 years agolibbpf-tools: update cachestat for libbpf 1.0
Andrii Nakryiko [Fri, 17 Dec 2021 22:20:08 +0000 (14:20 -0800)]
libbpf-tools: update cachestat for libbpf 1.0

Switch to libbpf 1.0 mode and adapt libbpf API usage accordingly.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
2 years agolibbpf-tools: update bitesize for libbpf 1.0
Andrii Nakryiko [Fri, 17 Dec 2021 22:19:43 +0000 (14:19 -0800)]
libbpf-tools: update bitesize for libbpf 1.0

Switch to libbpf 1.0 mode and adapt libbpf API usage accordingly.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
2 years agolibbpf-tools: update biostacks for libbpf 1.0
Andrii Nakryiko [Fri, 17 Dec 2021 22:19:26 +0000 (14:19 -0800)]
libbpf-tools: update biostacks for libbpf 1.0

Switch to libbpf 1.0 mode and adapt libbpf API usage accordingly.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
2 years agolibbpf-tools: update biosnoop for libbpf 1.0
Andrii Nakryiko [Fri, 17 Dec 2021 22:19:03 +0000 (14:19 -0800)]
libbpf-tools: update biosnoop for libbpf 1.0

Switch to libbpf 1.0 mode and adapt libbpf API usage accordingly.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
2 years agolibbpf-tools: update biopattern for libbpf 1.0
Andrii Nakryiko [Fri, 17 Dec 2021 22:18:48 +0000 (14:18 -0800)]
libbpf-tools: update biopattern for libbpf 1.0

Switch to libbpf 1.0 mode and adapt libbpf API usage accordingly.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
2 years agolibbpf-tools: update biolatency for libbpf 1.0
Andrii Nakryiko [Fri, 17 Dec 2021 22:17:46 +0000 (14:17 -0800)]
libbpf-tools: update biolatency for libbpf 1.0

Switch to libbpf 1.0 mode and adapt libbpf API usage accordingly.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
2 years agolibbpf-tools: update bindsnoop for libbpf 1.0
Andrii Nakryiko [Fri, 17 Dec 2021 22:16:32 +0000 (14:16 -0800)]
libbpf-tools: update bindsnoop for libbpf 1.0

Switch to libbpf 1.0 mode and adapt libbpf API usage accordingly.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
2 years agolibbpf-tools: update bashreadline for libbpf 1.0
Andrii Nakryiko [Mon, 20 Dec 2021 21:17:19 +0000 (13:17 -0800)]
libbpf-tools: update bashreadline for libbpf 1.0

Switch to libbpf 1.0 mode and adapt libbpf API usage
accordingly.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
2 years agolibbpf-tools: update bpftool
Andrii Nakryiko [Fri, 17 Dec 2021 22:15:30 +0000 (14:15 -0800)]
libbpf-tools: update bpftool

We need up-to-date bpftool to support skeletons with multiple BPF programs per
SEC().

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
2 years agosync with latest libbpf repo
Yonghong Song [Mon, 20 Dec 2021 01:41:48 +0000 (17:41 -0800)]
sync with latest libbpf repo

sync upto the following commit:
  96268bf0c2b7 sync: latest libbpf changes from kernel

Signed-off-by: Yonghong Song <yhs@fb.com>
2 years agotools: tcptop: Get command name from BPF code. (#3760)
eiffel-fl [Sat, 18 Dec 2021 18:21:13 +0000 (19:21 +0100)]
tools: tcptop: Get command name from BPF code. (#3760)

Before this commit, command name was taken from PID using /proc/PID/comm.
But this method was not reliable as it does not work all the time.
So, this commit takes command name from BPF code using bpf_get_current_comm()
helper like it is done for biotop.

Signed-off-by: Francis Laniel <flaniel@linux.microsoft.com>
2 years agoReplace !StatusTuple::code() with StatusTuple::ok() in src/cc/api/BPFTable.h (#3751)
yzhao [Sat, 18 Dec 2021 18:17:46 +0000 (10:17 -0800)]
Replace !StatusTuple::code() with StatusTuple::ok() in src/cc/api/BPFTable.h (#3751)

Replace !StatusTuple::code() with StatusTuple::ok() in src/cc/api/BPFTable.h

2 years agoImplement bashreadline with libbpf.
Kui-Feng Lee [Wed, 15 Dec 2021 23:57:17 +0000 (15:57 -0800)]
Implement bashreadline with libbpf.

Bashreadline will print user inputs, returning from readline, of every
instance of bash shell.  Readline is in bash itself, linked
statically, for some devices, while others may link to libreadline.so.
This implementation finds the symbol in bash if possible. Or, it tries
to find libreadline.so using ldd if the symbol is not in bash.

Signed-off-by: Kui-Feng Lee <kuifeng@fb.com>
2 years agotools: Remove unused struct id_t definition in tcpstates
Hengqi Chen [Sat, 18 Dec 2021 14:01:40 +0000 (22:01 +0800)]
tools: Remove unused struct id_t definition in tcpstates

The tool tcpstates contains a struct id_t definition but not
referenced, remove it.

Signed-off-by: Hengqi Chen <chenhengqi@outlook.com>
2 years agodocs: Fix BPF_HISTGRAM typo in reference guide
Tommi Rantala [Sat, 18 Dec 2021 16:08:07 +0000 (18:08 +0200)]
docs: Fix BPF_HISTGRAM typo in reference guide

Fix typo in reference guide, should be BPF_HISTOGRAM.

2 years agoRemove P4 language support.
Dave Marchevsky [Fri, 17 Dec 2021 08:59:26 +0000 (03:59 -0500)]
Remove P4 language support.

Remove support for compiling P4 programs (see #3682 for explanation).

Signed-off-by: Dave Marchevsky <davemarchevsky@fb.com>
2 years agoRemove B language support
Dave Marchevsky [Fri, 17 Dec 2021 07:54:49 +0000 (02:54 -0500)]
Remove B language support

Remove support for compiling B programs (see #3682 for explanation).

There may be some vestigial logic in other files that needs to be
cleanded up for simplicity - bpf_module.cc most likely - but that can be
addressed in followup commits.

Signed-off-by: Dave Marchevsky <davemarchevsky@fb.com>
2 years agoUpdate INSTALL.md (#3758)
bighunter513 [Thu, 16 Dec 2021 17:55:20 +0000 (01:55 +0800)]
Update INSTALL.md (#3758)

add INSTALL from source for CentOS 8.5 scripts

2 years agotest_tools_smoke.py: Helpful fail msg for timeout cmd's ret code
Dave Marchevsky [Wed, 15 Dec 2021 17:28:11 +0000 (12:28 -0500)]
test_tools_smoke.py: Helpful fail msg for timeout cmd's ret code

The test_tools_smoke script uses bash's 'timeout' command to run bcc
tools for a limited duration, sending a HUP after 5s and a KILL 5s after
that. Currently, when a tool exits in an unexpected way (e.g. we
expected a HUP to be required, but the tool required a KILL), the test
failure message isn't very descriptive.

This adds a more human-readable explanation of what's going on.

2 years agoMerge pull request #3748 from chenhengqi/fix-bcc-bio-tools
Dave Marchevsky [Wed, 15 Dec 2021 17:45:15 +0000 (12:45 -0500)]
Merge pull request #3748 from chenhengqi/fix-bcc-bio-tools

tools: Fix BCC bio tools with recent kernel change

2 years agoReplace StatusTuple::code() != 0 with !StatusTuple.ok() in examples/
Yaxiong Zhao [Mon, 13 Dec 2021 20:24:04 +0000 (12:24 -0800)]
Replace StatusTuple::code() != 0 with !StatusTuple.ok() in examples/

2 years agoReplace StatusTuple::code() with StatusTuple::ok() in tests/
Yaxiong Zhao [Mon, 13 Dec 2021 18:57:25 +0000 (10:57 -0800)]
Replace StatusTuple::code() with StatusTuple::ok() in tests/

2 years agoSupport tracing of processes under cgroup path
Chethan Suresh [Fri, 10 Dec 2021 04:47:59 +0000 (10:17 +0530)]
Support tracing of processes under cgroup path

- Using bpf_current_task_under_cgroup() we can check whether the probe
  is being run in the context of a given subset of the cgroup2 hierarchy.
- Support cgroup path '-c' args to get the cgroup2 path
  and filter based on the cgroup2 path fd using bpf_current_task_under_cgroup()

Signed-off-by: Chethan Suresh <Chethan.Suresh@sony.com>
2 years agoFix format
Yaxiong Zhao [Sat, 11 Dec 2021 05:36:46 +0000 (21:36 -0800)]
Fix format

2 years ago!StatusTuple::ok() replaced StatusTuple::code() != 0
Yaxiong Zhao [Wed, 8 Dec 2021 20:04:59 +0000 (12:04 -0800)]
!StatusTuple::ok() replaced StatusTuple::code() != 0

This is done with the shell command:
```
sed -i 's|\([A-Z_a-z]\+\).code() != 0|!\1.ok()|' $(grep '\.code() != 0' src -rl)
```

2 years agouse probe_limt from env
congwu [Sat, 4 Dec 2021 06:38:15 +0000 (14:38 +0800)]
use probe_limt from env

2 years agotools: Fix BCC bio tools with recent kernel change
Hengqi Chen [Sat, 11 Dec 2021 09:36:17 +0000 (17:36 +0800)]
tools: Fix BCC bio tools with recent kernel change

Several BCC bio tools are broken due to kernel change ([0]).
blk_account_io_{start, done} were renamed to __blk_account_io_{start, done},
and the symbols gone from /proc/kallsyms. Fix them by checking symbol existence.

  [0]: https://github.com/torvalds/linux/commit/be6bfe36db1795babe9d92178a47b2e02193cb0f

Signed-off-by: Hengqi Chen <chenhengqi@outlook.com>
2 years agoAdd --uid option to filter by user ID (#3743)
evilpan [Fri, 10 Dec 2021 16:58:51 +0000 (00:58 +0800)]
Add --uid option to filter by user ID (#3743)

* Add --uid option to filter by user ID
* update examples and man page of the trace tool

2 years agohardirqs: fix issue if irq is triggered while idle task (tid=0)
Ism Hong [Wed, 8 Dec 2021 02:17:20 +0000 (10:17 +0800)]
hardirqs: fix issue if irq is triggered while idle task (tid=0)

Currently, hardirqs use tid as key to store information while tracepoint
irq_handler_entry. It works fine if irq is triggered while normal task
running, but there is a chance causing overwrite issue while irq is
triggered while idle task (a.k.a swapper/x, tid=0) running on multi-core
system.

Let's say there are two irq event trigger simultaneously on both CPU
core, irq A @ core #0, irq B @ core #1, and system load is pretty light,
so BPF program will get tid=0 since current task is swapper/x for both cpu
core. In this case, the information of first irq event stored in map could
be overwritten by incoming second irq event.

Use tid and cpu_id together to make sure the key is unique for each
event in this corner case.

Please check more detail at merge request #2804, #3733.

2 years agohardirqs: fix duplicated count for shared IRQ
Ism Hong [Mon, 6 Dec 2021 03:28:34 +0000 (11:28 +0800)]
hardirqs: fix duplicated count for shared IRQ

Currently, hardirqs will count interrupt event simply while tracepoint
irq:irq_handler_entry triggered, it's fine for system without shared IRQ
event, but it will cause wrong interrupt count result for system with
shared IRQ, because kernel will interate all irq handlers belong to this
IRQ descriptor.

Take an example for system with shared IRQ below.

root@localhost:/# cat /proc/interrupts
           CPU0
 13:     385248     GICv3  39 Level     DDOMAIN ISR, gdma
 ...
 23:      61532     GICv3  38 Level     VGIP ISR, OnlineMeasure ISR

DDOMAIN IRQ and gdma shared the IRQ 13, VGIP ISR and OnlineMeasure
shared the IRQ 23, and use 'hardirqs -C' to measure the count of these
interrupt event.

root@localhost:/# hardirqs -C 10 1
Tracing hard irq events... Hit Ctrl-C to end.

HARDIRQ                    TOTAL_count
OnlineMeasure ISR                  300
VGIP ISR                           300
gdma                              2103
DDOMAIN ISR                       2103
eth0                              6677

hardirqs reported the same interrupt count for shared IRQ
'OnlineMeasure ISR/VGIP ISR' and 'gdma/DDOMAIN ISR'.

We should check the ret field of tracepoint irq:irq_hanlder_exit is
IRQ_HANDLED or IRQ_WAKE_THREAD to make sure the current event is belong
to this interrupt handler. For simplifying, just check `args->ret !=
IRQ_NONE`.

In the meantimes, the same changes should be applied to interrupt time
measurement.

The fixed hardirqs will show below output.

(bcc)root@localhost:/# ./hardirqs -C 10 1
Tracing hard irq events... Hit Ctrl-C to end.

HARDIRQ                    TOTAL_count
OnlineMeasure ISR                    1
VGIP ISR                           294
gdma                              1168
DDOMAIN ISR                       1476
eth0                              5210

2 years agofix llvm compilation failure
Yonghong Song [Mon, 6 Dec 2021 17:05:08 +0000 (09:05 -0800)]
fix llvm compilation failure

Fix issue #3734

llvm upstream commit
  89eb85ac6eab [IRBuilder] Remove deprecated methods
deprecated some functions which are used by bcc.
Let us follow the above commit to use the underlying
implementation instead.

Note that I didn't create a common header for the newer
functions since b language support will be removed in
the near future.

Signed-off-by: Yonghong Song <yhs@fb.com>
2 years agosync with latest libbpf repo
Yonghong Song [Mon, 6 Dec 2021 15:48:52 +0000 (07:48 -0800)]
sync with latest libbpf repo

Sync with latest libbpf repo (libbpf 0.6.0 + one more commit below).
  93e89b34740c ci: upgrade s390x runner to v2.285.0

Signed-off-by: Yonghong Song <yhs@fb.com>
2 years agoMerge pull request #3728 from chendotjs/tcpconnlat-lport
Dave Marchevsky [Wed, 1 Dec 2021 18:08:17 +0000 (13:08 -0500)]
Merge pull request #3728 from chendotjs/tcpconnlat-lport

libbpf-tools: add option to include 'LPORT' in tcpconnlat

2 years agoUpdate for Ubuntu 21.x build dependencies (#3727)
DavadDi [Tue, 30 Nov 2021 06:22:14 +0000 (14:22 +0800)]
Update for Ubuntu 21.x build dependencies (#3727)

Add Ubuntu 21.x build dependencies

2 years agolibbpf-tools: add option to include 'LPORT' in tcpconnlat
chendotjs [Tue, 30 Nov 2021 03:56:03 +0000 (03:56 +0000)]
libbpf-tools: add option to include 'LPORT' in tcpconnlat

Signed-off-by: chendotjs <chendotjs@gmail.com>
2 years agoMerge pull request #3726 from chendotjs/tcprtt-ordering
Dave Marchevsky [Tue, 30 Nov 2021 02:14:14 +0000 (21:14 -0500)]
Merge pull request #3726 from chendotjs/tcprtt-ordering

libbpf-tools: fix local/remote address byte ordering in tcprtt

2 years agolibbpf-tools: fix local/remote address byte ordering in tcprtt
chendotjs [Mon, 29 Nov 2021 08:51:53 +0000 (08:51 +0000)]
libbpf-tools: fix local/remote address byte ordering in tcprtt

Since inet_aton() converts IPv4 numbers-and-dots notation to binary in network byte order, there is
no need to do htonl() again.

Signed-off-by: chendotjs <chendotjs@gmail.com>
2 years agobcc: remove trailing semicolon of macro
Jacky_Yin [Wed, 24 Nov 2021 07:56:23 +0000 (15:56 +0800)]
bcc: remove trailing semicolon of macro

The trailing semicolon of a do-while style macro will cause
a if-else condition without braces failed to compile.
Meanwhile, also align with other do-while style macros.

2 years agotools: improve sslsniff (send buffer & filtering)
Slava Bacherikov [Sun, 21 Nov 2021 13:31:49 +0000 (15:31 +0200)]
tools: improve sslsniff (send buffer & filtering)

This makes few improvements:
    * This can send much larger data payload and also adds
      --max-buffer-size CLI option which allow changing this param.
    * Fixes dealing with non ASCII protocols, previously struct was
      defined as array of chars which made python ctypes treat it as
      NULL terminated string and it prevents from displaying any data
      past the null byte (which is very common for http2).
    * Adds more filtering and displaying options (--print-uid,
      --print-tid, --uid <uid>)

This also deals correctly with rare cases when bpf_probe_read_user fails
(so buffer should be empty and should not be displayed).

2 years agotools/hardirqs: Using TP_DATA_LOC_READ_STR to read string field
Hengqi Chen [Mon, 22 Nov 2021 13:54:51 +0000 (21:54 +0800)]
tools/hardirqs: Using TP_DATA_LOC_READ_STR to read string field

Fixes #3720.

Signed-off-by: Hengqi Chen <chenhengqi@outlook.com>
2 years agobcc: Use bpf_probe_read_str to read tracepoint data_loc field
Hengqi Chen [Mon, 22 Nov 2021 13:49:03 +0000 (21:49 +0800)]
bcc: Use bpf_probe_read_str to read tracepoint data_loc field

The data_loc field (defined as __string in kernel source) should
be treated as string NOT a fixed-size array, add a new macro
TP_DATA_LOC_READ_STR which use bpf_probe_read_str to reflect this.

Signed-off-by: Hengqi Chen <chenhengqi@outlook.com>
2 years agosimplify AL2 Linux package install command
quiver [Sun, 21 Nov 2021 08:57:33 +0000 (09:57 +0100)]
simplify AL2 Linux package install command

By just running `$ sudo amazon-linux-extras install BCC`, dependencies are install.

```
$ sudo amazon-linux-extras install BCC
...

==================================================================================================================================================================
 Package                                       Arch                      Version                                 Repository                                  Size
==================================================================================================================================================================
Installing:
 bcc                                           x86_64                    0.18.0-1.amzn2.0.3                      amzn2-core                                  28 M
Installing for dependencies:
 bcc-tools                                     x86_64                    0.18.0-1.amzn2.0.3                      amzn2-core                                 557 k
 clang-libs                                    x86_64                    11.1.0-1.amzn2.0.2                      amzn2-core                                  22 M
 clang-resource-filesystem                     x86_64                    11.1.0-1.amzn2.0.2                      amzn2-core                                  17 k
 cpp10                                         x86_64                    10.3.1-1.amzn2.0.1                      amzn2-core                                 9.5 M
 elfutils-libelf-devel                         x86_64                    0.176-2.amzn2                           amzn2-core                                  40 k
 gcc10                                         x86_64                    10.3.1-1.amzn2.0.1                      amzn2-core                                  38 M
 gcc10-binutils                                x86_64                    2.35-21.amzn2.0.1                       amzn2-core                                 2.9 M
 gcc10-binutils-gold                           x86_64                    2.35-21.amzn2.0.1                       amzn2-core                                 795 k
 glibc-devel                                   x86_64                    2.26-56.amzn2                           amzn2-core                                 994 k
 glibc-headers                                 x86_64                    2.26-56.amzn2                           amzn2-core                                 514 k
 isl                                           x86_64                    0.16.1-6.amzn2                          amzn2-core                                 833 k
 kernel-devel                                  x86_64                    5.10.75-79.358.amzn2                    amzn2extra-kernel-5.10                      16 M
 kernel-headers                                x86_64                    5.10.75-79.358.amzn2                    amzn2extra-kernel-5.10                     1.3 M
 libbpf                                        x86_64                    0.3.0-2.amzn2.0.3                       amzn2-core                                 102 k
 libmpc                                        x86_64                    1.0.1-3.amzn2.0.2                       amzn2-core                                  52 k
 libzstd                                       x86_64                    1.3.3-1.amzn2.0.1                       amzn2-core                                 203 k
 llvm-libs                                     x86_64                    11.1.0-1.amzn2.0.2                      amzn2-core                                  22 M
 mpfr                                          x86_64                    3.1.1-4.amzn2.0.2                       amzn2-core                                 208 k
 python3-bcc                                   noarch                    0.18.0-1.amzn2.0.3                      amzn2-core                                  86 k
 python3-netaddr                               noarch                    0.7.18-3.amzn2.0.2                      amzn2-core                                 1.3 M
 zlib-devel                                    x86_64                    1.2.7-18.amzn2                          amzn2-core                                  50 k
...
```

2 years agoCreate examples/tracing/undump.py examples text file (#3714)
rtoax [Fri, 19 Nov 2021 17:09:47 +0000 (01:09 +0800)]
Create examples/tracing/undump.py examples text file (#3714)

Create examples/tracing/undump.py examples text file and update permission (+x) for undump.py.

2 years agoEnable CMP0074 to allow `${pkg}_ROOT`, especially for LLVM_ROOT (#3713)
FUJI Goro [Fri, 19 Nov 2021 04:40:01 +0000 (13:40 +0900)]
Enable CMP0074 to allow `${pkg}_ROOT`, especially for LLVM_ROOT (#3713)

set CMP0074 to allow the use of `LLVM_ROOT` env var

2 years agoMark test_call1.py mayFail
Yonghong Song [Thu, 18 Nov 2021 00:57:20 +0000 (16:57 -0800)]
Mark test_call1.py mayFail

The test send a udp packet to test tailcalls.
The test may fail due to udp packet loss.
Let us mark the test as mayFail.

Signed-off-by: Yonghong Song <yhs@fb.com>
2 years agoadd batch methods into libbpf.h
hsqStephenZhang [Thu, 18 Nov 2021 00:19:38 +0000 (00:19 +0000)]
add batch methods into libbpf.h

2 years agoSync with latest libbpf repo
Yonghong Song [Wed, 17 Nov 2021 23:57:02 +0000 (15:57 -0800)]
Sync with latest libbpf repo

Sync with latest libbpf repo upto commit:
   94a49850c5ee Makefile: enforce gnu89 standard

Signed-off-by: Yonghong Song <yhs@fb.com>
2 years agoMerge pull request #3707 from iovisor/davemarchevsky-patch-4
Dave Marchevsky [Wed, 17 Nov 2021 04:48:34 +0000 (23:48 -0500)]
Merge pull request #3707 from iovisor/davemarchevsky-patch-4

add ubuntu-20.04 to bcc-test.yml

2 years agoGH Actions: run bcc-test and publish workflows on push to master branch too
Dave Marchevsky [Wed, 17 Nov 2021 03:01:59 +0000 (22:01 -0500)]
GH Actions: run bcc-test and publish workflows on push to master branch too

2 years agopython tests: mayFail py_smoke_tests' ttysnoop test on gh actions for now
Dave Marchevsky [Wed, 17 Nov 2021 02:34:57 +0000 (21:34 -0500)]
python tests: mayFail py_smoke_tests' ttysnoop test on gh actions for now

2 years agotests: Don't run py test_rlimit test on newer kernels
Dave Marchevsky [Wed, 17 Nov 2021 00:50:32 +0000 (19:50 -0500)]
tests: Don't run py test_rlimit test on newer kernels

Since commit d5299b67dd59 ("bpf: Memcg-based memory accounting for bpf
maps"), memory locked by bpf maps is no longer counted against rlimit.

Ubuntu 20.04's 5.11 kernel has this commit, so we should skip this test
there. When we add future distros to github actions it may be necessary
to modify the version check here.

2 years agomark 'test sk_storage map' mayfail
Dave Marchevsky [Tue, 16 Nov 2021 04:40:39 +0000 (23:40 -0500)]
mark 'test sk_storage map' mayfail

it wasn't running on ubuntu-18.04 test runner b/c of the kernel version check and is failing now as I try to add ubuntu-20.04 test runner

Will investigate separately from GH actions changes

2 years agoadd ubuntu-20.04 to bcc-test.yml
Dave Marchevsky [Tue, 16 Nov 2021 01:31:14 +0000 (20:31 -0500)]
add ubuntu-20.04 to bcc-test.yml

resending

2 years agoMerge pull request #3708 from davemarchevsky/davemarchevsky/gh-actions-1
Dave Marchevsky [Wed, 17 Nov 2021 02:32:35 +0000 (21:32 -0500)]
Merge pull request #3708 from davemarchevsky/davemarchevsky/gh-actions-1

gh actions: run test and publish actions on pull_request, not push

2 years agogh actions: run test and publish actions on pull_request, not push
Dave Marchevsky [Wed, 17 Nov 2021 02:07:28 +0000 (21:07 -0500)]
gh actions: run test and publish actions on pull_request, not push

2 years agoupdate debian changelog for release v0.23.0 v0.23.0
Yonghong Song [Mon, 15 Nov 2021 18:02:24 +0000 (10:02 -0800)]
update debian changelog for release v0.23.0

  * Support for kernel up to 5.15
  * bcc tools: update for kvmexit.py, tcpv4connect.py, cachetop.py, cachestat.py, etc.
  * libbpf tools: update for update for mountsnoop, ksnoop, gethostlatency, etc.
  * fix renaming of task_struct->state
  * get pid namespace properly for a number of tools
  * initial work for more libbpf utilization (less section names)
  * doc update, bug fixes and other tools improvement

Signed-off-by: Yonghong Song <yhs@fb.com>
2 years agodocs: correct typos in BPF.XDP in reference guide
Liz Rice [Mon, 15 Nov 2021 16:43:51 +0000 (16:43 +0000)]
docs: correct typos in BPF.XDP in reference guide

2 years agoAdd comment to sorting function
Eddie Elizondo [Mon, 15 Nov 2021 15:34:43 +0000 (10:34 -0500)]
Add comment to sorting function

2 years agoGuarantee strict weak order in Probe::finalize_locations
Eddie Elizondo [Mon, 15 Nov 2021 06:12:07 +0000 (01:12 -0500)]
Guarantee strict weak order in Probe::finalize_locations

2 years agoFix garbled java class name problem of uobjnew.py
denghui.ddh [Tue, 9 Nov 2021 13:11:40 +0000 (21:11 +0800)]
Fix garbled java class name problem of uobjnew.py

After this fix, the output may look like this:
NAME/TYPE                      # ALLOCS      # BYTES
[B                                    1         1016
[D                                    1         8016

2 years agotools: fix cachetop.py with 5.15 kernel
Yonghong Song [Thu, 11 Nov 2021 01:21:11 +0000 (17:21 -0800)]
tools: fix cachetop.py with 5.15 kernel

The tool cachetop.py doesn't work with 5.15 kernel due to
kprobe function renaming. Adapt to the new function.
Commit 61087b961716 ("tools: fix cachestat.py with 5.15 kernel")
fixed a similar issue for cachestat.py.

Signed-off-by: Yonghong Song <yhs@fb.com>
2 years agotools: fix cachestat.py with 5.15 kernel
Yonghong Song [Mon, 8 Nov 2021 19:12:37 +0000 (11:12 -0800)]
tools: fix cachestat.py with 5.15 kernel

Fix issue #3687.
The tool cachestat.py doesn't work with 5.15 kernel due to
kprobe function renaming. Adapt to the new function.
Also added a comment that static functions might
get inlined and the result may not be accurate if this happens.
More work can be done in the future to make the tool
more robust.

Signed-off-by: Yonghong Song <yhs@fb.com>
2 years agoMerge pull request #3673 from chenhengqi/fix-libbpf-tools-memleak
Dave Marchevsky [Thu, 4 Nov 2021 20:14:40 +0000 (16:14 -0400)]
Merge pull request #3673 from chenhengqi/fix-libbpf-tools-memleak

libbpf-tools: Fix memory leaks in ksnoop/gethostlatency

2 years agobcc/tools: Fix renaming of the state field of task_struct
Hengqi Chen [Sun, 31 Oct 2021 15:20:10 +0000 (23:20 +0800)]
bcc/tools: Fix renaming of the state field of task_struct

Kernel commit 2f064a59a1 ("sched: Change task_struct::state") changes
the name of task_struct::state to task_struct::__state, which breaks
several bcc tools. Fix this issue by checking field existence in vmlinux
BTF. Since this change was intruduce in kernel v5.14, we should have
BTF support. Closes #3658 .

Signed-off-by: Hengqi Chen <chenhengqi@outlook.com>
2 years agobcc: Add kernel_struct_has_field function to BPF object
Hengqi Chen [Sun, 31 Oct 2021 13:29:45 +0000 (21:29 +0800)]
bcc: Add kernel_struct_has_field function to BPF object

Add a new function kernel_struct_has_field, which allows user to
check that whether a kernel struct has a specific field. This
enable us to deal with some kernel changes like in 2f064a59a1 ([0])
of the linux kernel.

  [0]: https://github.com/torvalds/linux/commit/2f064a59a1

Signed-off-by: Hengqi Chen <chenhengqi@outlook.com>
2 years agosync latest libbpf repo
Yonghong Song [Mon, 1 Nov 2021 05:32:17 +0000 (22:32 -0700)]
sync latest libbpf repo

sync up to the following libbpf commit:
  eaea2bce024f sync: remove redundant test on $BPF_BRANCH

Signed-off-by: Yonghong Song <yhs@fb.com>
2 years agoFix build on RISC-V
r-value [Thu, 28 Oct 2021 04:58:19 +0000 (12:58 +0800)]
Fix build on RISC-V

ref #3536

2 years agotools/runqlat.py:get the pid namespace by following task_active_pid_ns()
Li Chengyuan [Fri, 22 Oct 2021 10:23:02 +0000 (03:23 -0700)]
tools/runqlat.py:get the pid namespace by following task_active_pid_ns()

Simliar fix as commit bced75aae53c22524fd335b04a005ce60384b8a8

Signed-off-by: Li Chengyuan chengyuanli@hotmail.com
2 years agolibbpf-tools: Fix memory leaks in ksnoop/gethostlatency
Hengqi Chen [Sat, 23 Oct 2021 15:04:27 +0000 (23:04 +0800)]
libbpf-tools: Fix memory leaks in ksnoop/gethostlatency

There are memory leaks when attach a BPF program to multiple
targets in these tools. This is because we misuse the
bpf_program__attach_kprobe function, the returned struct bpf_link
object is not freed after use. Closes #3664.

Signed-off-by: Hengqi Chen <chenhengqi@outlook.com>
2 years agoget the pid namespace by following task_active_pid_ns()
Li Chengyuan [Fri, 1 Oct 2021 06:36:24 +0000 (23:36 -0700)]
get the pid namespace by following task_active_pid_ns()

When unsharing of pid namespace, though nsproxy->pid_ns_for_children is
new, the process is still in the orignal pid namespace, only the forked
children processes will be in the new pid namespace.
So it's not correct to get process's pid namespace by nsproxy->pid_ns_for_children,
should get pid namespace by task_active_pid_ns() way, i.e.
pid->numbers[pid->level].ns

Signed-off-by: Li Chengyuan chengyuanli@hotmail.com
2 years agoreduce counter.enabled checking in test_perf_event.cc
Yonghong Song [Wed, 20 Oct 2021 06:17:40 +0000 (23:17 -0700)]
reduce counter.enabled checking in test_perf_event.cc

The test "attach perf event" is often flaky with the failure:

  3: /home/fedora/jenkins/workspace/bcc-pr/label/fc28/tests/cc/test_perf_event.cc:139: FAILED:
  3:   REQUIRE( counter.enabled >= 800000000 )
  3: with expansion:
  3:   774406106 (0x2e287fda)
  3:   >=
  3:   800000000 (0x2faf0800)

Previous workaround with 800000000 nano-second doesn't work 100%.
Let us change to 200000000 nano-second.

Signed-off-by: Yonghong Song <yhs@fb.com>
2 years agoksnoop: Fix info command output
Hengqi Chen [Fri, 15 Oct 2021 11:44:14 +0000 (19:44 +0800)]
ksnoop: Fix info command output

The info command is used to print kernel function signature.
This commit makes the output conform to the kernel code style.
Before this fix:
```
$ sudo ./ksnoop info sk_alloc
struct sock  *  sk_alloc(struct net  * net, int family, gfp_t priority, struct proto  * prot, int kern);
$ sudo ./ksnoop info dma_buf_end_cpu_access
$ sudo ./ksnoop info array_map_alloc_check
int array_map_alloc_check(unionbpf_attr *attr);
```
After this fix:
```
$ sudo ./ksnoop info sk_alloc
struct sock *sk_alloc(struct net *net, int family, gfp_t priority, struct proto *prot, int kern);
$ sudo ./ksnoop info dma_buf_end_cpu_access
int dma_buf_end_cpu_access(struct dma_buf *dmabuf, enum dma_data_direction direction);
$ sudo ./ksnoop info array_map_alloc_check
int array_map_alloc_check(union bpf_attr *attr);
```

Signed-off-by: Hengqi Chen <chenhengqi@outlook.com>
2 years agoksnoop: fix verification failures on 5.15 kernel
Alan Maguire [Mon, 18 Oct 2021 13:20:40 +0000 (14:20 +0100)]
ksnoop: fix verification failures on 5.15 kernel

hengqi.chen@gmail.com reported:

I have two VMs:

One has the kernel built against the following commit:

0693b27644f04852e46f7f034e3143992b658869 (bpf-next)

The ksnoop tool (from BCC repo) works well on this VM.

Another has the kernel built against the following commit:

5319255b8df9271474bc9027cabf82253934f28d (bpf-next)

On this VM, the ksnoop tool failed with the following message:

[snip]

; last_ip = func_stack->ips[last_stack_depth];

141: (67) r6 <<= 3

142: (0f) r3 += r6

; ip = func_stack->ips[stack_depth];

143: (79) r2 = *(u64 *)(r4 +0)

 frame1: R0=map_value(id=0,off=0,ks=8,vs=144,imm=0) R1_w=invP(id=4,smin_value=-1,smax_value=14) R2_w=invP(id=0,umax_value=2040,var_off=(0x0; 0x7f8)) R3_w=map_value(id=0,off=8,ks=8,vs=144,umax_value=120,var_off=(0x0; 0x78)) R4_w=map_value(id=0,off=8,ks=8,vs=144,umax_value=2040,var_off=(0x0; 0x7f8)) R6_w=invP(id=0,umax_value=120,var_off=(0x0; 0x78)) R7=map_value(id=0,off=0,ks=8,vs=144,imm=0) R9=ctx(id=0,off=0,imm=0) R10=fp0 fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm fp-48=mmmmmmmm fp-56=mmmmmmmm fp-64=mmmmmmmm fp-72=mmmmmmmm fp-80=mmmmmmmm fp-88=mmmmmmmm fp-96=mmmmmmmm fp-104=mmmmmmmm fp-112=mmmmmmmm fp-120=mmmmmmmm fp-128=mmmmmmmm fp-136=mmmmmmmm fp-144=mmmmmmmm fp-152=mmmmmmmm fp-160=mmmmmmmm fp-168=00000000

invalid access to map value, value_size=144 off=2048 size=8

R4 max value is outside of the allowed memory range

processed 65 insns (limit 1000000) max_states_per_insn 0 total_states 3 peak_states 3 mark_read 2

libbpf: -- END LOG --

libbpf: failed to load program 'kprobe_return'

libbpf: failed to load object 'ksnoop_bpf'

libbpf: failed to load BPF skeleton 'ksnoop_bpf': -4007

Error: Could not load ksnoop BPF: Unknown error 4007

The above invalid map access appears to stem from the fact the
"stack_depth" variable (used to retrieve the instruction pointer
from the recorded call stack) is decremented.  The off=2048
value is a clue; this suggests an index resulting from an underflow
of the __u8 index value.  Adding a bitmask to the decrement operation
solves the problem.  It appears that the guards on stack_depth size
around the array dereference were optimized out.

Reported-by: Hengqi Chen <hengqi.chen@gmail.com>
Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
2 years agoexport/helpers: only put helpers in special section for B lang
Dave Marchevsky [Mon, 18 Oct 2021 00:25:48 +0000 (20:25 -0400)]
export/helpers: only put helpers in special section for B lang

B's code generation needs these functions to exist in the object as it
emits some calls to these functions at IR stage, where 'always_inline'
directive results in no symbol for the function being emitted otherwise
as all uses are inlined.

For C, stop putting these helpers in a "helpers" section in object file.
For B, add a `B_WORKAROUND` ifdef check so the "helpers" section is
populated as expected.

There is almost certainly a more elegant way to fix this but would
require digging deep in the b frontend and potentially breaking other
things. Since B frontend hasn't been touched in many years and still
works, let's take the safer but uglier route.

2 years agoMerge pull request #3665 from chenhengqi/dev/kfunc-no-args
Dave Marchevsky [Mon, 18 Oct 2021 00:49:29 +0000 (20:49 -0400)]
Merge pull request #3665 from chenhengqi/dev/kfunc-no-args

bcc: Allow KFUNC_PROBE to instrument function without arguments

2 years agotools: Remove unused variable stub
Hengqi Chen [Sun, 17 Oct 2021 15:55:17 +0000 (23:55 +0800)]
tools: Remove unused variable stub

Signed-off-by: Hengqi Chen <chenhengqi@outlook.com>
2 years agobcc: Allow KFUNC_PROBE to instrument function without arguments
Hengqi Chen [Sun, 17 Oct 2021 15:50:09 +0000 (23:50 +0800)]
bcc: Allow KFUNC_PROBE to instrument function without arguments

Update KFUNC_PROBE and its family to allow instrument kernel
function without specifying arguments. Sometimes, we don't need
to bookkeep arguments at function entry, just store a timestamp.
This fix would allow this use case.

Signed-off-by: Hengqi Chen <chenhengqi@outlook.com>
2 years agoMerge pull request #3662 from chenhengqi/enable-warn-unused-vars
Dave Marchevsky [Sat, 16 Oct 2021 01:01:01 +0000 (21:01 -0400)]
Merge pull request #3662 from chenhengqi/enable-warn-unused-vars

libbpf-tools: Enable compilation warnings for BPF programs

2 years agolibbpf-tools: Fix renaming of the state field of task_struct
Hengqi Chen [Thu, 14 Oct 2021 13:40:45 +0000 (21:40 +0800)]
libbpf-tools: Fix renaming of the state field of task_struct

Kernel commit 2f064a59a1 ("sched: Change task_struct::state") changes
the name of task_struct::state to task_struct::__state, which breaks
several libbpf tools. Fix them by utilizing the libbpf CO-RE support.

Signed-off-by: Hengqi Chen <chenhengqi@outlook.com>
2 years agolibbpf-tools: Enable compilation warnings for BPF programs
Hengqi Chen [Fri, 15 Oct 2021 05:50:43 +0000 (13:50 +0800)]
libbpf-tools: Enable compilation warnings for BPF programs

Enable -Wall option when compile BPF programs and fix all
compilation warnings.

Signed-off-by: Hengqi Chen <chenhengqi@outlook.com>
2 years agoMerge pull request #3661 from iovisor/yhs_dev
Dave Marchevsky [Fri, 15 Oct 2021 02:08:37 +0000 (22:08 -0400)]
Merge pull request #3661 from iovisor/yhs_dev

fix a llvm14 compilation error

2 years agofix a llvm14 compilation error
Yonghong Song [Thu, 14 Oct 2021 23:00:27 +0000 (16:00 -0700)]
fix a llvm14 compilation error

Upstream commit https://reviews.llvm.org/D111454
moved header file llvm/Support/TargetRegistry.h to
llvm/MC/TargetRegistry.h. Let us adjust accordingly
to avoid compilation error.

Signed-off-by: Yonghong Song <yhs@fb.com>
2 years agotools/capable: Set data to zero before setting fields.
Francis Laniel [Mon, 11 Oct 2021 15:35:12 +0000 (17:35 +0200)]
tools/capable: Set data to zero before setting fields.

This commit ensures data contains all 0 before setting its fields.
So, even if some fields are not set, there should be no problem with unaligned
access.

Signed-off-by: Francis Laniel <flaniel@microsoft.com>
2 years agoMerge pull request #3652 from kinvolk/francis/upstream/c-mountsnoop-fix-example-strings
Dave Marchevsky [Tue, 12 Oct 2021 17:20:25 +0000 (13:20 -0400)]
Merge pull request #3652 from kinvolk/francis/upstream/c-mountsnoop-fix-example-strings

libbpf-tools/mountsnoop: Fix example strings.

2 years agolibbpf-tools/mountsnoop: Fix example strings.
Francis Laniel [Mon, 11 Oct 2021 16:34:55 +0000 (18:34 +0200)]
libbpf-tools/mountsnoop: Fix example strings.

Signed-off-by: Francis Laniel <flaniel@microsoft.com>
2 years agoMerge pull request #3642 from shunghsiyu/fix_linking_when_disable_usdt
Dave Marchevsky [Thu, 30 Sep 2021 22:56:00 +0000 (18:56 -0400)]
Merge pull request #3642 from shunghsiyu/fix_linking_when_disable_usdt

Do not export USDT function when ENABLE_USDT is OFF

2 years agoDo not export USDT function when ENABLE_USDT is OFF
Shung-Hsi Yu [Thu, 30 Sep 2021 06:11:46 +0000 (14:11 +0800)]
Do not export USDT function when ENABLE_USDT is OFF

When compiling with CMAKE_USE_LIBBPF_PACKAGE=yes and ENABLE_USDT=OFF, linking
of test_static will fail due to undefined references to
`bcc_usdt_new_frompath', `bcc_usdt_close' and `bcc_usdt_new_frompid'. The
reference comes from link_all.cc which references those functions irrespective
of ENABLE_USDT.

As a fix, introduce EXPORT_USDT and wrap references to USDT functions inside
link_all.cc within #ifdef.

2 years agoMerge pull request #3615 from Rtoax/patch-3
Dave Marchevsky [Wed, 29 Sep 2021 14:02:59 +0000 (10:02 -0400)]
Merge pull request #3615 from Rtoax/patch-3

Capture UNIX domain socket packet

2 years agoUpdate tcpv4connect.py
Zdravko Bozakov [Tue, 28 Sep 2021 20:45:17 +0000 (22:45 +0200)]
Update tcpv4connect.py

fix byte string comparison so we can run with python3

2 years agotools/kvmexit: Display header after KeyboardInterrupt
Hengqi Chen [Thu, 23 Sep 2021 08:58:44 +0000 (16:58 +0800)]
tools/kvmexit: Display header after KeyboardInterrupt

When Ctrl+C is hit, `^C` messes up the output header. Fix that
by adding a blank line before printing. Also remove unused import
and signal handler.

Signed-off-by: Hengqi Chen <chenhengqi@outlook.com>
2 years agoMerge pull request #3630 from kinvolk/mauricio/handle-eint-libbpftools
Dave Marchevsky [Sat, 25 Sep 2021 01:05:06 +0000 (21:05 -0400)]
Merge pull request #3630 from kinvolk/mauricio/handle-eint-libbpftools

libbpf-tools: fix EINTR related issues

2 years agoMerge pull request #3636 from iovisor/davemarchevsky_bye_1604
Dave Marchevsky [Fri, 24 Sep 2021 18:41:44 +0000 (14:41 -0400)]
Merge pull request #3636 from iovisor/davemarchevsky_bye_1604

remove ubuntu 16.04 from bcc-test github action

2 years agoremove ubuntu 16.04 from bcc-test github action
Dave Marchevsky [Fri, 24 Sep 2021 18:09:42 +0000 (14:09 -0400)]
remove ubuntu 16.04 from bcc-test github action

As per [github](https://github.com/actions/virtual-environments/issues/3287), this is no longer supported. CI fails with errors like

```
This request was automatically failed because there were no enabled runners online to process the request for more than 1 days.
```

I will add 20.04 in a followup PR to match `publish.yml`. Want to keep it separate in case adding 20.04 causes issues, removing 16.04 should be much less likely to.

2 years agolibbpf-tools: fix EINTR related issues
Mauricio Vásquez [Fri, 17 Sep 2021 14:40:41 +0000 (09:40 -0500)]
libbpf-tools: fix EINTR related issues

1. Most of the tools that use perf_buffer__poll() were not handling the
case when it was interrupted by a signal, they were just ending.
We noticed this issue by running the tools inside a container, after
some seconds they will finish:

```
$ time /execsnoop
...
runc             210198 939      0 /usr/sbin/runc --version
docker-init      210205 939      0 /usr/bin/docker-init --version
Error polling perf buffer: -4

real 0m48.913s
user 0m0.020s
sys 0m0.033s
```

This commit fixes that by checking if errno is EINTR after calling
perf_buffer__poll().

2. Many tools were returning non zero when ended by SIG_INT.

```
$ sudo ./execsnoop
PCOMM            PID    PPID   RET ARGS
runc             203967 939      0 /usr/sbin/runc --version
docker-init      203973 939      0 /usr/bin/docker-init --version
calico           203974 724      0 /opt/cni/bin/calico
portmap          203985 724      0 /opt/cni/bin/portmap
bandwidth        203990 724      0 /opt/cni/bin/bandwidth
^C
$ echo $?
130
```

3. Some tools were missing the SIG_INT handler

Signed-off-by: Mauricio Vásquez <mauricio@kinvolk.io>
2 years agoMerge pull request #3633 from iovisor/davemarchevsky_remove_ksnoop_include
Dave Marchevsky [Thu, 23 Sep 2021 07:58:09 +0000 (03:58 -0400)]
Merge pull request #3633 from iovisor/davemarchevsky_remove_ksnoop_include

ksnoop: remove duplicate include

2 years agoksnoop: remove duplicate include
Dave Marchevsky [Thu, 23 Sep 2021 02:54:48 +0000 (22:54 -0400)]
ksnoop: remove duplicate include

`ksnoop` is the only libbpf tool which is including both `<linux/bpf.h>` and `<bpf/bpf.h>` - the rest of the tools just include the latter

build fails for me because of redefinition errors as a result. Let's use `<bpf/bpf.h>` like the rest of the tools