1.. SPDX-License-Identifier: GPL-2.0-only
2.. Copyright (C) 2022 Red Hat, Inc.
3
4================================================
5BPF_MAP_TYPE_ARRAY and BPF_MAP_TYPE_PERCPU_ARRAY
6================================================
7
8.. note::
9   - ``BPF_MAP_TYPE_ARRAY`` was introduced in kernel version 3.19
10   - ``BPF_MAP_TYPE_PERCPU_ARRAY`` was introduced in version 4.6
11
12``BPF_MAP_TYPE_ARRAY`` and ``BPF_MAP_TYPE_PERCPU_ARRAY`` provide generic array
13storage. The key type is an unsigned 32-bit integer (4 bytes) and the map is
14of constant size. The size of the array is defined in ``max_entries`` at
15creation time. All array elements are pre-allocated and zero initialized when
16created. ``BPF_MAP_TYPE_PERCPU_ARRAY`` uses a different memory region for each
17CPU whereas ``BPF_MAP_TYPE_ARRAY`` uses the same memory region. The value
18stored can be of any size, however, all array elements are aligned to 8
19bytes.
20
21Since kernel 5.5, memory mapping may be enabled for ``BPF_MAP_TYPE_ARRAY`` by
22setting the flag ``BPF_F_MMAPABLE``. The map definition is page-aligned and
23starts on the first page. Sufficient page-sized and page-aligned blocks of
24memory are allocated to store all array values, starting on the second page,
25which in some cases will result in over-allocation of memory. The benefit of
26using this is increased performance and ease of use since userspace programs
27would not be required to use helper functions to access and mutate data.
28
29Usage
30=====
31
32Kernel BPF
33----------
34
35bpf_map_lookup_elem()
36~~~~~~~~~~~~~~~~~~~~~
37
38.. code-block:: c
39
40   void *bpf_map_lookup_elem(struct bpf_map *map, const void *key)
41
42Array elements can be retrieved using the ``bpf_map_lookup_elem()`` helper.
43This helper returns a pointer into the array element, so to avoid data races
44with userspace reading the value, the user must use primitives like
45``__sync_fetch_and_add()`` when updating the value in-place.
46
47bpf_map_update_elem()
48~~~~~~~~~~~~~~~~~~~~~
49
50.. code-block:: c
51
52   long bpf_map_update_elem(struct bpf_map *map, const void *key, const void *value, u64 flags)
53
54Array elements can be updated using the ``bpf_map_update_elem()`` helper.
55
56``bpf_map_update_elem()`` returns 0 on success, or negative error in case of
57failure.
58
59Since the array is of constant size, ``bpf_map_delete_elem()`` is not supported.
60To clear an array element, you may use ``bpf_map_update_elem()`` to insert a
61zero value to that index.
62
63Per CPU Array
64-------------
65
66Values stored in ``BPF_MAP_TYPE_ARRAY`` can be accessed by multiple programs
67across different CPUs. To restrict storage to a single CPU, you may use a
68``BPF_MAP_TYPE_PERCPU_ARRAY``.
69
70When using a ``BPF_MAP_TYPE_PERCPU_ARRAY`` the ``bpf_map_update_elem()`` and
71``bpf_map_lookup_elem()`` helpers automatically access the slot for the current
72CPU.
73
74bpf_map_lookup_percpu_elem()
75~~~~~~~~~~~~~~~~~~~~~~~~~~~~
76
77.. code-block:: c
78
79   void *bpf_map_lookup_percpu_elem(struct bpf_map *map, const void *key, u32 cpu)
80
81The ``bpf_map_lookup_percpu_elem()`` helper can be used to lookup the array
82value for a specific CPU. Returns value on success , or ``NULL`` if no entry was
83found or ``cpu`` is invalid.
84
85Concurrency
86-----------
87
88Since kernel version 5.1, the BPF infrastructure provides ``struct bpf_spin_lock``
89to synchronize access.
90
91Userspace
92---------
93
94Access from userspace uses libbpf APIs with the same names as above, with
95the map identified by its ``fd``.
96
97Examples
98========
99
100Please see the ``tools/testing/selftests/bpf`` directory for functional
101examples. The code samples below demonstrate API usage.
102
103Kernel BPF
104----------
105
106This snippet shows how to declare an array in a BPF program.
107
108.. code-block:: c
109
110    struct {
111            __uint(type, BPF_MAP_TYPE_ARRAY);
112            __type(key, u32);
113            __type(value, long);
114            __uint(max_entries, 256);
115    } my_map SEC(".maps");
116
117
118This example BPF program shows how to access an array element.
119
120.. code-block:: c
121
122    int bpf_prog(struct __sk_buff *skb)
123    {
124            struct iphdr ip;
125            int index;
126            long *value;
127
128            if (bpf_skb_load_bytes(skb, ETH_HLEN, &ip, sizeof(ip)) < 0)
129                    return 0;
130
131            index = ip.protocol;
132            value = bpf_map_lookup_elem(&my_map, &index);
133            if (value)
134                    __sync_fetch_and_add(value, skb->len);
135
136            return 0;
137    }
138
139Userspace
140---------
141
142BPF_MAP_TYPE_ARRAY
143~~~~~~~~~~~~~~~~~~
144
145This snippet shows how to create an array, using ``bpf_map_create_opts`` to
146set flags.
147
148.. code-block:: c
149
150    #include <bpf/libbpf.h>
151    #include <bpf/bpf.h>
152
153    int create_array()
154    {
155            int fd;
156            LIBBPF_OPTS(bpf_map_create_opts, opts, .map_flags = BPF_F_MMAPABLE);
157
158            fd = bpf_map_create(BPF_MAP_TYPE_ARRAY,
159                                "example_array",       /* name */
160                                sizeof(__u32),         /* key size */
161                                sizeof(long),          /* value size */
162                                256,                   /* max entries */
163                                &opts);                /* create opts */
164            return fd;
165    }
166
167This snippet shows how to initialize the elements of an array.
168
169.. code-block:: c
170
171    int initialize_array(int fd)
172    {
173            __u32 i;
174            long value;
175            int ret;
176
177            for (i = 0; i < 256; i++) {
178                    value = i;
179                    ret = bpf_map_update_elem(fd, &i, &value, BPF_ANY);
180                    if (ret < 0)
181                            return ret;
182            }
183
184            return ret;
185    }
186
187This snippet shows how to retrieve an element value from an array.
188
189.. code-block:: c
190
191    int lookup(int fd)
192    {
193            __u32 index = 42;
194            long value;
195            int ret;
196
197            ret = bpf_map_lookup_elem(fd, &index, &value);
198            if (ret < 0)
199                    return ret;
200
201            /* use value here */
202            assert(value == 42);
203
204            return ret;
205    }
206
207BPF_MAP_TYPE_PERCPU_ARRAY
208~~~~~~~~~~~~~~~~~~~~~~~~~
209
210This snippet shows how to initialize the elements of a per CPU array.
211
212.. code-block:: c
213
214    int initialize_array(int fd)
215    {
216            int ncpus = libbpf_num_possible_cpus();
217            long values[ncpus];
218            __u32 i, j;
219            int ret;
220
221            for (i = 0; i < 256 ; i++) {
222                    for (j = 0; j < ncpus; j++)
223                            values[j] = i;
224                    ret = bpf_map_update_elem(fd, &i, &values, BPF_ANY);
225                    if (ret < 0)
226                            return ret;
227            }
228
229            return ret;
230    }
231
232This snippet shows how to access the per CPU elements of an array value.
233
234.. code-block:: c
235
236    int lookup(int fd)
237    {
238            int ncpus = libbpf_num_possible_cpus();
239            __u32 index = 42, j;
240            long values[ncpus];
241            int ret;
242
243            ret = bpf_map_lookup_elem(fd, &index, &values);
244            if (ret < 0)
245                    return ret;
246
247            for (j = 0; j < ncpus; j++) {
248                    /* Use per CPU value here */
249                    assert(values[j] == 42);
250            }
251
252            return ret;
253    }
254
255Semantics
256=========
257
258As shown in the example above, when accessing a ``BPF_MAP_TYPE_PERCPU_ARRAY``
259in userspace, each value is an array with ``ncpus`` elements.
260
261When calling ``bpf_map_update_elem()`` the flag ``BPF_NOEXIST`` can not be used
262for these maps.
263