1331722Seadler================== 2239268SgonzoBPF Selftest Notes 3239268Sgonzo================== 4239268SgonzoGeneral instructions on running selftests can be found in 5239268Sgonzo`Documentation/bpf/bpf_devel_QA.rst`__. 6239268Sgonzo 7239268Sgonzo__ /Documentation/bpf/bpf_devel_QA.rst#q-how-to-run-bpf-selftests 8239268Sgonzo 9239268Sgonzo============= 10239268SgonzoBPF CI System 11239268Sgonzo============= 12239268Sgonzo 13239268SgonzoBPF employs a continuous integration (CI) system to check patch submission in an 14239268Sgonzoautomated fashion. The system runs selftests for each patch in a series. Results 15239268Sgonzoare propagated to patchwork, where failures are highlighted similar to 16239268Sgonzoviolations of other checks (such as additional warnings being emitted or a 17239268Sgonzo``scripts/checkpatch.pl`` reported deficiency): 18239268Sgonzo 19239268Sgonzo https://patchwork.kernel.org/project/netdevbpf/list/?delegate=121173 20239268Sgonzo 21239268SgonzoThe CI system executes tests on multiple architectures. It uses a kernel 22239268Sgonzoconfiguration derived from both the generic and architecture specific config 23239268Sgonzofile fragments below ``tools/testing/selftests/bpf/`` (e.g., ``config`` and 24239268Sgonzo``config.x86_64``). 25239268Sgonzo 26239268SgonzoDenylisting Tests 27239268Sgonzo================= 28239268Sgonzo 29239268SgonzoIt is possible for some architectures to not have support for all BPF features. 30239268SgonzoIn such a case tests in CI may fail. An example of such a shortcoming is BPF 31239268Sgonzotrampoline support on IBM's s390x architecture. For cases like this, an in-tree 32239268Sgonzodeny list file, located at ``tools/testing/selftests/bpf/DENYLIST.<arch>``, can 33239268Sgonzobe used to prevent the test from running on such an architecture. 34239268Sgonzo 35239268SgonzoIn addition to that, the generic ``tools/testing/selftests/bpf/DENYLIST`` is 36239268Sgonzohonored on every architecture running tests. 37239268Sgonzo 38239268SgonzoThese files are organized in three columns. The first column lists the test in 39239268Sgonzoquestion. This can be the name of a test suite or of an individual test. The 40239268Sgonzoremaining two columns provide additional meta data that helps identify and 41239268Sgonzoclassify the entry: column two is a copy and paste of the error being reported 42239268Sgonzowhen running the test in the setting in question. The third column, if 43239268Sgonzoavailable, summarizes the underlying problem. A value of ``trampoline``, for 44239268Sgonzoexample, indicates that lack of trampoline support is causing the test to fail. 45239268SgonzoThis last entry helps identify tests that can be re-enabled once such support is 46239268Sgonzoadded. 47239268Sgonzo 48239268Sgonzo========================= 49239268SgonzoRunning Selftests in a VM 50251712Sandrew========================= 51251712Sandrew 52251712SandrewIt's now possible to run the selftests using ``tools/testing/selftests/bpf/vmtest.sh``. 53239268SgonzoThe script tries to ensure that the tests are run with the same environment as they 54239268Sgonzowould be run post-submit in the CI used by the Maintainers, with the exception 55239268Sgonzothat deny lists are not automatically honored. 56239268Sgonzo 57239268SgonzoThis script uses the in-tree kernel configuration and downloads a VM userspace 58239268Sgonzoimage from the system used by the CI. It builds the kernel (without overwriting 59239268Sgonzoyour existing Kconfig), recompiles the bpf selftests, runs them (by default 60239268Sgonzo``tools/testing/selftests/bpf/test_progs``) and saves the resulting output (by 61239268Sgonzodefault in ``~/.bpf_selftests``). 62239268Sgonzo 63239268SgonzoScript dependencies: 64239268Sgonzo- clang (preferably built from sources, https://github.com/llvm/llvm-project); 65239268Sgonzo- pahole (preferably built from sources, https://git.kernel.org/pub/scm/devel/pahole/pahole.git/); 66239268Sgonzo- qemu; 67239268Sgonzo- docutils (for ``rst2man``); 68239268Sgonzo- libcap-devel. 69239268Sgonzo 70239268SgonzoFor more information about using the script, run: 71239268Sgonzo 72239268Sgonzo.. code-block:: console 73239268Sgonzo 74239268Sgonzo $ tools/testing/selftests/bpf/vmtest.sh -h 75239268Sgonzo 76239268SgonzoIn case of linker errors when running selftests, try using static linking: 77239268Sgonzo 78239268Sgonzo.. code-block:: console 79239268Sgonzo 80239268Sgonzo $ LDLIBS=-static PKG_CONFIG='pkg-config --static' vmtest.sh 81239268Sgonzo 82239268Sgonzo.. note:: Some distros may not support static linking. 83239268Sgonzo 84239268Sgonzo.. note:: The script uses pahole and clang based on host environment setting. 85239268Sgonzo If you want to change pahole and llvm, you can change `PATH` environment 86239268Sgonzo variable in the beginning of script. 87239268Sgonzo 88239268Sgonzo.. note:: The script currently only supports x86_64 and s390x architectures. 89239268Sgonzo 90239268SgonzoAdditional information about selftest failures are 91239268Sgonzodocumented here. 92239268Sgonzo 93239268Sgonzoprofiler[23] test failures with clang/llvm <12.0.0 94239268Sgonzo================================================== 95290243Sgonzo 96263910SandrewWith clang/llvm <12.0.0, the profiler[23] test may fail. 97263910SandrewThe symptom looks like 98263910Sandrew 99263910Sandrew.. code-block:: c 100263910Sandrew 101239268Sgonzo // r9 is a pointer to map_value 102239268Sgonzo // r7 is a scalar 103239268Sgonzo 17: bf 96 00 00 00 00 00 00 r6 = r9 104239268Sgonzo 18: 0f 76 00 00 00 00 00 00 r6 += r7 105239268Sgonzo math between map_value pointer and register with unbounded min value is not allowed 106239268Sgonzo 107239268Sgonzo // the instructions below will not be seen in the verifier log 108239268Sgonzo 19: a5 07 01 00 01 01 00 00 if r7 < 257 goto +1 109239268Sgonzo 20: bf 96 00 00 00 00 00 00 r6 = r9 110239268Sgonzo // r6 is used here 111239268Sgonzo 112239268SgonzoThe verifier will reject such code with above error. 113239268SgonzoAt insn 18 the r7 is indeed unbounded. The later insn 19 checks the bounds and 114239268Sgonzothe insn 20 undoes map_value addition. It is currently impossible for the 115239268Sgonzoverifier to understand such speculative pointer arithmetic. 116239268SgonzoHence `this patch`__ addresses it on the compiler side. It was committed on llvm 12. 117239268Sgonzo 118239268Sgonzo__ https://github.com/llvm/llvm-project/commit/ddf1864ace484035e3cde5e83b3a31ac81e059c6 119239268Sgonzo 120239268SgonzoThe corresponding C code 121239268Sgonzo 122325810Sjhb.. code-block:: c 123325810Sjhb 124325810Sjhb for (int i = 0; i < MAX_CGROUPS_PATH_DEPTH; i++) { 125325810Sjhb filepart_length = bpf_probe_read_str(payload, ...); 126325810Sjhb if (filepart_length <= MAX_PATH) { 127325810Sjhb barrier_var(filepart_length); // workaround 128239268Sgonzo payload += filepart_length; 129239268Sgonzo } 130239268Sgonzo } 131239268Sgonzo 132239268Sgonzobpf_iter test failures with clang/llvm 10.0.0 133239268Sgonzo============================================= 134239268Sgonzo 135239268SgonzoWith clang/llvm 10.0.0, the following two bpf_iter tests failed: 136239268Sgonzo * ``bpf_iter/ipv6_route`` 137239268Sgonzo * ``bpf_iter/netlink`` 138239268Sgonzo 139239268SgonzoThe symptom for ``bpf_iter/ipv6_route`` looks like 140239268Sgonzo 141262941Sian.. code-block:: c 142317005Smmel 143317005Smmel 2: (79) r8 = *(u64 *)(r1 +8) 144317005Smmel ... 145317005Smmel 14: (bf) r2 = r8 146317005Smmel 15: (0f) r2 += r1 147317005Smmel ; BPF_SEQ_PRINTF(seq, "%pi6 %02x ", &rt->fib6_dst.addr, rt->fib6_dst.plen); 148317005Smmel 16: (7b) *(u64 *)(r8 +64) = r2 149317005Smmel only read is supported 150317005Smmel 151325831SjhbThe symptom for ``bpf_iter/netlink`` looks like 152325831Sjhb 153247341Scognet.. code-block:: c 154262941Sian 155262948Sian ; struct netlink_sock *nlk = ctx->sk; 156317005Smmel 2: (79) r7 = *(u64 *)(r1 +8) 157317005Smmel ... 158239268Sgonzo 15: (bf) r2 = r7 159239268Sgonzo 16: (0f) r2 += r1 160 ; BPF_SEQ_PRINTF(seq, "%pK %-3d ", s, s->sk_protocol); 161 17: (7b) *(u64 *)(r7 +0) = r2 162 only read is supported 163 164This is due to a llvm BPF backend bug. `The fix`__ 165has been pushed to llvm 10.x release branch and will be 166available in 10.0.1. The patch is available in llvm 11.0.0 trunk. 167 168__ https://github.com/llvm/llvm-project/commit/3cb7e7bf959dcd3b8080986c62e10a75c7af43f0 169 170bpf_verif_scale/loop6.bpf.o test failure with Clang 12 171====================================================== 172 173With Clang 12, the following bpf_verif_scale test failed: 174 * ``bpf_verif_scale/loop6.bpf.o`` 175 176The verifier output looks like 177 178.. code-block:: c 179 180 R1 type=ctx expected=fp 181 The sequence of 8193 jumps is too complex. 182 183The reason is compiler generating the following code 184 185.. code-block:: c 186 187 ; for (i = 0; (i < VIRTIO_MAX_SGS) && (i < num); i++) { 188 14: 16 05 40 00 00 00 00 00 if w5 == 0 goto +64 <LBB0_6> 189 15: bc 51 00 00 00 00 00 00 w1 = w5 190 16: 04 01 00 00 ff ff ff ff w1 += -1 191 17: 67 05 00 00 20 00 00 00 r5 <<= 32 192 18: 77 05 00 00 20 00 00 00 r5 >>= 32 193 19: a6 01 01 00 05 00 00 00 if w1 < 5 goto +1 <LBB0_4> 194 20: b7 05 00 00 06 00 00 00 r5 = 6 195 00000000000000a8 <LBB0_4>: 196 21: b7 02 00 00 00 00 00 00 r2 = 0 197 22: b7 01 00 00 00 00 00 00 r1 = 0 198 ; for (i = 0; (i < VIRTIO_MAX_SGS) && (i < num); i++) { 199 23: 7b 1a e0 ff 00 00 00 00 *(u64 *)(r10 - 32) = r1 200 24: 7b 5a c0 ff 00 00 00 00 *(u64 *)(r10 - 64) = r5 201 202Note that insn #15 has w1 = w5 and w1 is refined later but 203r5(w5) is eventually saved on stack at insn #24 for later use. 204This cause later verifier failure. The bug has been `fixed`__ in 205Clang 13. 206 207__ https://github.com/llvm/llvm-project/commit/1959ead525b8830cc8a345f45e1c3ef9902d3229 208 209BPF CO-RE-based tests and Clang version 210======================================= 211 212A set of selftests use BPF target-specific built-ins, which might require 213bleeding-edge Clang versions (Clang 12 nightly at this time). 214 215Few sub-tests of core_reloc test suit (part of test_progs test runner) require 216the following built-ins, listed with corresponding Clang diffs introducing 217them to Clang/LLVM. These sub-tests are going to be skipped if Clang is too 218old to support them, they shouldn't cause build failures or runtime test 219failures: 220 221- __builtin_btf_type_id() [0_, 1_, 2_]; 222- __builtin_preserve_type_info(), __builtin_preserve_enum_value() [3_, 4_]. 223 224.. _0: https://github.com/llvm/llvm-project/commit/6b01b465388b204d543da3cf49efd6080db094a9 225.. _1: https://github.com/llvm/llvm-project/commit/072cde03aaa13a2c57acf62d79876bf79aa1919f 226.. _2: https://github.com/llvm/llvm-project/commit/00602ee7ef0bf6c68d690a2bd729c12b95c95c99 227.. _3: https://github.com/llvm/llvm-project/commit/6d218b4adb093ff2e9764febbbc89f429412006c 228.. _4: https://github.com/llvm/llvm-project/commit/6d6750696400e7ce988d66a1a00e1d0cb32815f8 229 230Floating-point tests and Clang version 231====================================== 232 233Certain selftests, e.g. core_reloc, require support for the floating-point 234types, which was introduced in `Clang 13`__. The older Clang versions will 235either crash when compiling these tests, or generate an incorrect BTF. 236 237__ https://github.com/llvm/llvm-project/commit/a7137b238a07d9399d3ae96c0b461571bd5aa8b2 238 239Kernel function call test and Clang version 240=========================================== 241 242Some selftests (e.g. kfunc_call and bpf_tcp_ca) require a LLVM support 243to generate extern function in BTF. It was introduced in `Clang 13`__. 244 245Without it, the error from compiling bpf selftests looks like: 246 247.. code-block:: console 248 249 libbpf: failed to find BTF for extern 'tcp_slow_start' [25] section: -2 250 251__ https://github.com/llvm/llvm-project/commit/886f9ff53155075bd5f1e994f17b85d1e1b7470c 252 253btf_tag test and Clang version 254============================== 255 256The btf_tag selftest requires LLVM support to recognize the btf_decl_tag and 257btf_type_tag attributes. They are introduced in `Clang 14` [0_, 1_]. 258The subtests ``btf_type_tag_user_{mod1, mod2, vmlinux}`` also requires 259pahole version ``1.23``. 260 261Without them, the btf_tag selftest will be skipped and you will observe: 262 263.. code-block:: console 264 265 #<test_num> btf_tag:SKIP 266 267.. _0: https://github.com/llvm/llvm-project/commit/a162b67c98066218d0d00aa13b99afb95d9bb5e6 268.. _1: https://github.com/llvm/llvm-project/commit/3466e00716e12e32fdb100e3fcfca5c2b3e8d784 269 270Clang dependencies for static linking tests 271=========================================== 272 273linked_vars, linked_maps, and linked_funcs tests depend on `Clang fix`__ to 274generate valid BTF information for weak variables. Please make sure you use 275Clang that contains the fix. 276 277__ https://github.com/llvm/llvm-project/commit/968292cb93198442138128d850fd54dc7edc0035 278 279Clang relocation changes 280======================== 281 282Clang 13 patch `clang reloc patch`_ made some changes on relocations such 283that existing relocation types are broken into more types and 284each new type corresponds to only one way to resolve relocation. 285See `kernel llvm reloc`_ for more explanation and some examples. 286Using clang 13 to compile old libbpf which has static linker support, 287there will be a compilation failure:: 288 289 libbpf: ELF relo #0 in section #6 has unexpected type 2 in .../bpf_tcp_nogpl.bpf.o 290 291Here, ``type 2`` refers to new relocation type ``R_BPF_64_ABS64``. 292To fix this issue, user newer libbpf. 293 294.. Links 295.. _clang reloc patch: https://github.com/llvm/llvm-project/commit/6a2ea84600ba4bd3b2733bd8f08f5115eb32164b 296.. _kernel llvm reloc: /Documentation/bpf/llvm_reloc.rst 297 298Clang dependencies for the u32 spill test (xdpwall) 299=================================================== 300The xdpwall selftest requires a change in `Clang 14`__. 301 302Without it, the xdpwall selftest will fail and the error message 303from running test_progs will look like: 304 305.. code-block:: console 306 307 test_xdpwall:FAIL:Does LLVM have https://github.com/llvm/llvm-project/commit/ea72b0319d7b0f0c2fcf41d121afa5d031b319d5? unexpected error: -4007 308 309__ https://github.com/llvm/llvm-project/commit/ea72b0319d7b0f0c2fcf41d121afa5d031b319d5 310