ipvlan: hold lower dev to avoid possible use-after-free

Recently syzkaller discovered the issue of disappearing lower
device (NETDEV_UNREGISTER) while the virtual device (like
macvlan) is still having it as a lower device. So it's just
a matter of time similar discovery will be made for IPvlan
device setup. So fixing it preemptively. Also while at it,
add a refcount tracker.

Fixes: 2ad7bf363841 ("ipvlan: Initial check-in of the IPVLAN driver.")
Signed-off-by: Mahesh Bandewar <maheshb@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

ipvlan: adopt u64_stats_t

As explained in commit 316580b69d0a ("u64_stats: provide u64_stats_t type")
we should use u64_stats_t and related accessors to avoid load/store tearing.

Add READ_ONCE() when reading rx_errs & tx_drps.

Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152

Based on 1 normalized pattern(s):

this program is free software you can redistribute it and or modify
it under the terms of the gnu general public license as published by
the free software foundation either version 2 of the license or at
your option any later version

extracted by the scancode license scanner the SPDX license identifier

GPL-2.0-or-later

has been chosen to replace the boilerplate/reference in 3029 file(s).

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Allison Randal <allison@lohutok.net>
Cc: linux-spdx@vger.kernel.org
Link: https://lkml.kernel.org/r/20190527070032.746973796@linutronix.de
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

ipvlan: decouple l3s mode dependencies from other modes

Right now ipvlan has a hard dependency on CONFIG_NETFILTER and
otherwise it cannot be built. However, the only ipvlan operation
mode that actually depends on netfilter is l3s, everything else
is independent of it. Break this hard dependency such that users
are able to use ipvlan l3 mode on systems where netfilter is not
compiled in.

Therefore, this adds a hidden CONFIG_IPVLAN_L3S bool which is
defaulting to y when CONFIG_NETFILTER is set in order to retain
existing behavior for l3s. All l3s related code is refactored
into ipvlan_l3s.c that is compiled in when enabled.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Cc: Mahesh Bandewar <maheshb@google.com>
Cc: Florian Westphal <fw@strlen.de>
Cc: Martynas Pumputis <m@lambda.lt>
Acked-by: Florian Westphal <fw@strlen.de>
Signed-off-by: David S. Miller <davem@davemloft.net>

ipvlan: properly annotate rx_handler access

The rx_handler field is rcu-protected, but I forgot to use the
proper accessor while refactoring netif_is_ipvlan_port(). Such
function only check the rx_handler value, so it is safe, but we need
to properly read rx_handler via rcu_access_pointer() to avoid sparse
warnings.

Fixes: 1ec54cb44e67 ("net: unpollute priv_flags space")
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

net: unpollute priv_flags space

the ipvlan device driver defines and uses 2 bits inside the priv_flags
net_device field. Such bits and the related helper are used only
inside the ipvlan device driver, and the core networking does not
need to be aware of them.

This change moves netif_is_ipvlan* helper in the ipvlan driver and
re-implement them looking for ipvlan specific symbols instead of
using priv_flags.

Overall this frees two bits inside priv_flags - and move the following
ones to avoid gaps - without any intended functional change.

Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

ipvlan: use per device spinlock to protect addrs list updates

This changeset moves ipvlan address under RCU protection, using
a per ipvlan device spinlock to protect list mutation and RCU
read access to protect list traversal.

Also explicitly use RCU read lock to traverse the per port
ipvlans list, so that we can now perform a full address lookup
without asserting the RTNL lock.

Overall this allows the ipvlan driver to check fully for duplicate
addresses - before this commit ipv6 addresses assigned by autoconf
via prefix delegation where accepted without any check - and avoid
the following rntl assertion failure still in the same code path:

RTNL: assertion failed at drivers/net/ipvlan/ipvlan_core.c (124)
WARNING: CPU: 15 PID: 0 at drivers/net/ipvlan/ipvlan_core.c:124 ipvlan_addr_busy+0x97/0xa0 [ipvlan]
Modules linked in: ipvlan(E) ixgbe
CPU: 15 PID: 0 Comm: swapper/15 Tainted: G E 4.16.0-rc2.ipvlan+ #1782
Hardware name: Dell Inc. PowerEdge R730/072T6D, BIOS 2.1.7 06/16/2016
RIP: 0010:ipvlan_addr_busy+0x97/0xa0 [ipvlan]
RSP: 0018:ffff881ff9e03768 EFLAGS: 00010286
RAX: 0000000000000000 RBX: ffff881fdf2a9000 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 00000000000000f6 RDI: 0000000000000300
RBP: ffff881fdf2a8000 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000001 R11: ffff881ff9e034c0 R12: ffff881fe07bcc00
R13: 0000000000000001 R14: ffffffffa02002b0 R15: 0000000000000001
FS: 0000000000000000(0000) GS:ffff881ff9e00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc5c1a4f248 CR3: 000000207e012005 CR4: 00000000001606e0
Call Trace:
<IRQ>
ipvlan_addr6_event+0x6c/0xd0 [ipvlan]
notifier_call_chain+0x49/0x90
atomic_notifier_call_chain+0x6a/0x100
ipv6_add_addr+0x5f9/0x720
addrconf_prefix_rcv_add_addr+0x244/0x3c0
addrconf_prefix_rcv+0x2f3/0x790
ndisc_router_discovery+0x633/0xb70
ndisc_rcv+0x155/0x180
icmpv6_rcv+0x4ac/0x5f0
ip6_input_finish+0x138/0x6a0
ip6_input+0x41/0x1f0
ipv6_rcv+0x4db/0x8d0
__netif_receive_skb_core+0x3d5/0xe40
netif_receive_skb_internal+0x89/0x370
napi_gro_receive+0x14f/0x1e0
ixgbe_clean_rx_irq+0x4ce/0x1020 [ixgbe]
ixgbe_poll+0x31a/0x7a0 [ixgbe]
net_rx_action+0x296/0x4f0
__do_softirq+0xcf/0x4f5
irq_exit+0xf5/0x110
do_IRQ+0x62/0x110
common_interrupt+0x91/0x91
</IRQ>

v1 -> v2: drop unneeded in_softirq check in ipvlan_addr6_validator_event()

Fixes: e9997c2938b2 ("ipvlan: fix check for IP addresses in control path")
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

ipvlan: implement VEPA mode

This is very similar to the Macvlan VEPA mode, however, there is some
difference. IPvlan uses the mac-address of the lower device, so the VEPA
mode has implications of ICMP-redirects for packets destined for its
immediate neighbors sharing same master since the packets will have same
source and dest mac. The external switch/router will send redirect msg.

Having said that, this will be useful tool in terms of debugging
since IPvlan will not switch packets within its slaves and rely completely
on the external entity as intended in 802.1Qbg.

Signed-off-by: Mahesh Bandewar <maheshb@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

ipvlan: introduce 'private' attribute for all existing modes.

IPvlan has always operated in bridge mode. However there are scenarios
where each slave should be able to talk through the master device but
not necessarily across each other. Think of an environment where each
of a namespace is a private and independant customer. In this scenario
the machine which is hosting these namespaces neither want to tell who
their neighbor is nor the individual namespaces care to talk to neighbor
on short-circuited network path.

This patch implements the mode that is very similar to the 'private' mode
in macvlan where individual slaves can send and receive traffic through
the master device, just that they can not talk among slave devices.

Signed-off-by: Mahesh Bandewar <maheshb@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

net: add netlink_ext_ack argument to rtnl_link_ops.newlink

Add support for extended error reporting.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
Acked-by: David Ahern <dsahern@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

ipvlan: use pernet operations and restrict l3s hooks to master netns

commit 4fbae7d83c98c30efc ("ipvlan: Introduce l3s mode") added
registration of netfilter hooks via nf_register_hooks().

This API provides the illusion of 'global' netfilter hooks by placing the
hooks in all current and future network namespaces.

In case of ipvlan the hook appears to be only needed in the namespace
that contains the ipvlan master device (i.e., usually init_net), so
placing them in all namespaces is not needed.

This switches ipvlan driver to pernet operations, and then only registers
hooks in namespaces where a ipvlan master device is set to l3s mode.

Extra care has to be taken when the master device is moved to another
namespace, as we might have to 'move' the netfilter hooks too.

This is done by storing the namespace the ipvlan port was created in.
On REGISTER event, do (un)register operations in the old/new namespaces.

This will also allow removal of the nf_register_hooks() in a future patch.

Cc: Mahesh Bandewar <maheshb@google.com>
Signed-off-by: Florian Westphal <fw@strlen.de>
Signed-off-by: David S. Miller <davem@davemloft.net>

ipvtap: IP-VLAN based tap driver

This patch adds a tap character device driver that is based on the
IP-VLAN network interface, called ipvtap. An ipvtap device can be created
in the same way as an ipvlan device, using 'type ipvtap', and then accessed
using the tap user space interface.

Signed-off-by: Sainath Grandhi <sainath.grandhi@intel.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

ipvlan: improvise dev_id generation logic in IPvlan

The patch 009146d117b ("ipvlan: assign unique dev-id for each slave
device.") used ida_simple_get() to generate dev_ids assigned to the
slave devices. However (Eric has pointed out that) there is a shortcoming
with that approach as it always uses the first available ID. This
becomes a problem when a slave gets deleted and a new slave gets added.
The ID gets reassigned causing the new slave to get the same link-local
address. This side-effect is undesirable.

This patch adds a per-port variable that keeps track of the IDs
assigned and used as the stat-base for the IDR api. This base will be
wrapped around when it reaches the MAX (0xFFFE) value possibly on a
busy system where slaves are added and deleted routinely.

Fixes: 009146d117b ("ipvlan: assign unique dev-id for each slave device.")
Signed-off-by: Mahesh Bandewar <maheshb@google.com>
CC: Eric Dumazet <edumazet@google.com>
CC: David Miller <davem@davemloft.org>
Signed-off-by: David S. Miller <davem@davemloft.net>

ipvlan: assign unique dev-id for each slave device.

IPvlan setup uses one mac-address (of master). The IPv6 link-local
addresses are derived using the mac-address on the link. Lack of
dev-ids makes these link-local addresses same for all slaves including
that of master device. dev-ids are necessary to add differentiation
when L2 address is shared.

Signed-off-by: Mahesh Bandewar <maheshb@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

ipvlan: fix multicast processing

In an IPvlan setup when master is set in loopback mode e.g.

ethtool -K eth0 set loopback on

where eth0 is master device for IPvlan setup.

The failure is caused by the faulty logic that determines if the
packet is from TX-path vs. RX-path by just looking at the mac-
addresses on the packet while processing multicast packets.

In the loopback-mode where this crash was happening, the packets
that are sent out are reflected by the NIC and are processed on
the RX path, but mac-address check tricks into thinking this
packet is from TX path and falsely uses dev_forward_skb() to pass
packets to the slave (virtual) devices.

This patch records the path while queueing packets and eliminates
logic of looking at mac-addresses for the same decision.

------------[ cut here ]------------
kernel BUG at include/linux/skbuff.h:1737!
Call Trace:
[<ffffffff921fbbc2>] dev_forward_skb+0x92/0xd0
[<ffffffffc031ac65>] ipvlan_process_multicast+0x395/0x4c0 [ipvlan]
[<ffffffffc031a9a7>] ? ipvlan_process_multicast+0xd7/0x4c0 [ipvlan]
[<ffffffff91cdfea7>] ? process_one_work+0x147/0x660
[<ffffffff91cdff09>] process_one_work+0x1a9/0x660
[<ffffffff91cdfea7>] ? process_one_work+0x147/0x660
[<ffffffff91ce086d>] worker_thread+0x11d/0x360
[<ffffffff91ce0750>] ? rescuer_thread+0x350/0x350
[<ffffffff91ce960b>] kthread+0xdb/0xe0
[<ffffffff91c05c70>] ? _raw_spin_unlock_irq+0x30/0x50
[<ffffffff91ce9530>] ? flush_kthread_worker+0xc0/0xc0
[<ffffffff92348b7a>] ret_from_fork+0x9a/0xd0
[<ffffffff91ce9530>] ? flush_kthread_worker+0xc0/0xc0

Fixes: ba35f8588f47 ("ipvlan: Defer multicast / broadcast processing to a work-queue")
Signed-off-by: Mahesh Bandewar <maheshb@google.com>
CC: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

driver: ipvlan: Free ipvl_port directly with kfree instead of kfree_rcu

There are two functions which would free the ipvl_port now. The first
is ipvlan_port_create. It frees the ipvl_port in the error handler,
so it could kfree it directly. The second is ipvlan_port_destroy. It
invokes netdev_rx_handler_unregister which enforces one grace period
by synchronize_net firstly, so it also could kfree the ipvl_port
directly and safely.

So it is unnecessary to use kfree_rcu to free ipvl_port.

Signed-off-by: Gao Feng <fgao@ikuai8.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

driver: ipvlan: Remove useless member mtu_adj of struct ipvl_dev

The mtu_adj is initialized to zero when alloc mem, there is no any
assignment to mtu_adj. It is only used in ipvlan_adjust_mtu as one
right value.
So it is useless member of struct ipvl_dev, then remove it.

Signed-off-by: Gao Feng <fgao@ikuai8.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

ipvlan: Introduce l3s mode

In a typical IPvlan L3 setup where master is in default-ns and
each slave is into different (slave) ns. In this setup egress
packet processing for traffic originating from slave-ns will
hit all NF_HOOKs in slave-ns as well as default-ns. However same
is not true for ingress processing. All these NF_HOOKs are
hit only in the slave-ns skipping them in the default-ns.
IPvlan in L3 mode is restrictive and if admins want to deploy
iptables rules in default-ns, this asymmetric data path makes it
impossible to do so.

This patch makes use of the l3_rcv() (added as part of l3mdev
enhancements) to perform input route lookup on RX packets without
changing the skb->dev and then uses nf_hook at NF_INET_LOCAL_IN
to change the skb->dev just before handing over skb to L4.

Signed-off-by: Mahesh Bandewar <maheshb@google.com>
CC: David Ahern <dsa@cumulusnetworks.com>
Reviewed-by: David Ahern <dsa@cumulusnetworks.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

ipvlan: misc changes

1. scope correction for few functions that are used in single file.
2. Adjust variables that are used in fast-path to fit into single cacheline
3. Update rcv_frame() to skip shared check for frames coming over wire

Signed-off-by: Mahesh Bandewar <maheshb@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

ipvlan: mode is u16

The mode argument was erronusly defined as u32 but it has always
been u16. Also use ipvlan_set_mode() helper to set the mode instead
of assigning directly. This should avoid future erronus assignments /
updates.

Signed-off-by: Mahesh Bandewar <maheshb@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

ipvlan: use rcu_deference_bh() in ipvlan_queue_xmit()

In tx path rcu_read_lock_bh() is held, so we need rcu_deference_bh().
This fixes the following warning:

===============================
[ INFO: suspicious RCU usage. ]
4.1.0-rc1+ #1007 Not tainted
-------------------------------
drivers/net/ipvlan/ipvlan.h:106 suspicious rcu_dereference_check() usage!

other info that might help us debug this:

rcu_scheduler_active = 1, debug_locks = 0
1 lock held by dhclient/1076:
#0: (rcu_read_lock_bh){......}, at: [<ffffffff817e8d84>] rcu_lock_acquire+0x0/0x26

stack backtrace:
CPU: 2 PID: 1076 Comm: dhclient Not tainted 4.1.0-rc1+ #1007
Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
0000000000000001 ffff8800d381bac8 ffffffff81a4154f 000000003c1a3c19
ffff8800d4d0a690 ffff8800d381baf8 ffffffff810b849f ffff880117d41148
ffff880117d40000 ffff880117d40068 0000000000000156 ffff8800d381bb18
Call Trace:
[<ffffffff81a4154f>] dump_stack+0x4c/0x65
[<ffffffff810b849f>] lockdep_rcu_suspicious+0x107/0x110
[<ffffffff8165a522>] ipvlan_port_get_rcu+0x47/0x4e
[<ffffffff8165ad14>] ipvlan_queue_xmit+0x35/0x450
[<ffffffff817ea45d>] ? rcu_read_unlock+0x3e/0x5f
[<ffffffff810a20bf>] ? local_clock+0x19/0x22
[<ffffffff810b4781>] ? __lock_is_held+0x39/0x52
[<ffffffff8165b64c>] ipvlan_start_xmit+0x1b/0x44
[<ffffffff817edf7f>] dev_hard_start_xmit+0x2ae/0x467
[<ffffffff817ee642>] __dev_queue_xmit+0x50a/0x60c
[<ffffffff817ee7a7>] dev_queue_xmit_sk+0x13/0x15
[<ffffffff81997596>] dev_queue_xmit+0x10/0x12
[<ffffffff8199b41c>] packet_sendmsg+0xb6b/0xbdf
[<ffffffff810b5ea7>] ? mark_lock+0x2e/0x226
[<ffffffff810a1fcc>] ? sched_clock_cpu+0x9e/0xb7
[<ffffffff817d56f9>] sock_sendmsg_nosec+0x12/0x1d
[<ffffffff817d7257>] sock_sendmsg+0x29/0x2e
[<ffffffff817d72cc>] sock_write_iter+0x70/0x91
[<ffffffff81199563>] __vfs_write+0x7e/0xa7
[<ffffffff811996bc>] vfs_write+0x92/0xe8
[<ffffffff811997d7>] SyS_write+0x47/0x7e
[<ffffffff81a4d517>] system_call_fastpath+0x12/0x6f

Fixes: 2ad7bf363841 ("ipvlan: Initial check-in of the IPVLAN driver.")
Cc: Mahesh Bandewar <maheshb@google.com>
Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
Acked-by: Mahesh Bandewar <maheshb@google.com>
Acked-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
Signed-off-by: David S. Miller <davem@davemloft.net>

ipvlan: unhash addresses without synchronize_rcu

All structures used in traffic forwarding are rcu-protected:
ipvl_addr, ipvl_dev and ipvl_port. Thus we can unhash addresses
without synchronization. We'll anyway hash it back into the same
bucket: in worst case lockless lookup will scan hash once again.

Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
Signed-off-by: David S. Miller <davem@davemloft.net>

ipvlan: remove counters of ipv4 and ipv6 addresses

They are unused after commit f631c44bbe15 ("ipvlan: Always set broadcast bit in
multicast filter").

Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
Signed-off-by: David S. Miller <davem@davemloft.net>

ipvlan: Defer multicast / broadcast processing to a work-queue

Processing multicast / broadcast in fast path is performance draining
and having more links means more cloning and bringing performance
down further.

Broadcast; in particular, need to be given to all the virtual links.
Earlier tricks of enabling broadcast bit for IPv4 only interfaces are not
really working since it fails autoconf. Which means enabling broadcast
for all the links if protocol specific hacks do not have to be added into
the driver.

This patch defers all (incoming as well as outgoing) multicast traffic to
a work-queue leaving only the unicast traffic in the fast-path. Now if we
need to apply any additional tricks to further reduce the impact of this
(multicast / broadcast) type of traffic, it can be implemented while
processing this work without affecting the fast-path.

Signed-off-by: Mahesh Bandewar <maheshb@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

ipvlan: fix check for IP addresses in control path

When an ipvlan interface is down, its addresses are not on the hash list.
Fix checks for existence of addresses not to depend on the hash list, walk
through all interface addresses instead.

Signed-off-by: Jiri Benc <jbenc@redhat.com>
Acked-by: Mahesh Bandewar <maheshb@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

ipvlan: add a missing __percpu pcpu_stats

Cosmetic patch to add __percpu qualifier to pcpu_stats

Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

ipvlan: move the device check function into netdevice.h

Move the port check [ipvlan_dev_master()] and device check
[ipvlan_dev_slave()] functions to netdevice.h and rename them
netif_is_ipvlan_port() and netif_is_ipvlan() resp. to be
consistent with macvlan api naming.

Signed-off-by: Mahesh Bandewar <maheshb@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

ipvlan: ipvlan depends on INET and IPV6

This driver uses ip_out_local() and ip6_route_output() which are
defined only if CONFIG_INET and CONFIG_IPV6 are enabled respectively.

Reported-by: Jim Davis <jim.epost@gmail.com>
Signed-off-by: Mahesh Bandewar <maheshb@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

ipvlan: Initial check-in of the IPVLAN driver.

This driver is very similar to the macvlan driver except that it
uses L3 on the frame to determine the logical interface while
functioning as packet dispatcher. It inherits L2 of the master
device hence the packets on wire will have the same L2 for all
the packets originating from all virtual devices off of the same
master device.

This driver was developed keeping the namespace use-case in
mind. Hence most of the examples given here take that as the
base setup where main-device belongs to the default-ns and
virtual devices are assigned to the additional namespaces.

The device operates in two different modes and the difference
in these two modes in primarily in the TX side.

(a) L2 mode : In this mode, the device behaves as a L2 device.
TX processing upto L2 happens on the stack of the virtual device
associated with (namespace). Packets are switched after that
into the main device (default-ns) and queued for xmit.

RX processing is simple and all multicast, broadcast (if
applicable), and unicast belonging to the address(es) are
delivered to the virtual devices.

(b) L3 mode : In this mode, the device behaves like a L3 device.
TX processing upto L3 happens on the stack of the virtual device
associated with (namespace). Packets are switched to the
main-device (default-ns) for the L2 processing. Hence the routing
table of the default-ns will be used in this mode.

RX processins is somewhat similar to the L2 mode except that in
this mode only Unicast packets are delivered to the virtual device
while main-dev will handle all other packets.

The devices can be added using the "ip" command from the iproute2
package -

ip link add link <master> <virtual> type ipvlan mode [ l2 | l3 ]

Signed-off-by: Mahesh Bandewar <maheshb@google.com>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Maciej Żenczykowski <maze@google.com>
Cc: Laurent Chavey <chavey@google.com>
Cc: Tim Hockin <thockin@google.com>
Cc: Brandon Philips <brandon.philips@coreos.com>
Cc: Pavel Emelianov <xemul@parallels.com>
Signed-off-by: David S. Miller <davem@davemloft.net>