sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

Mechanically convert age(4), alc(4), and ale(4) to IfAPI

Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D37787

ale: Remove unused devclass argument to DRIVER_MODULE.

Remove unused miibus_devclass and miibus_fdt_devclass.

ale: clean up empty lines in .c and .h files

Remove compatibility code for changing sysctl definitions for pre FreeBSD 9
versions. These are just noise these days.

Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)

r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly marked).
Use it in preparation for a general review of all nodes.

This is non-functional change that adds annotations to SYSCTL_NODE and
SYSCTL_PROC nodes using one of the soon-to-be-required flags.

Mark all obvious cases as MPSAFE. All entries that haven't been marked
as MPSAFE before are by default marked as NEEDGIANT

Approved by: kib (mentor, blanket)
Commented by: kib, gallatin, melifaro
Differential Revision: https://reviews.freebsd.org/D23718

Use NET_TASK_INIT() and NET_GROUPTASK_INIT() for drivers that process
incoming packets in taskqueue context.

Reviewed by: hselasky
Differential Revision: https://reviews.freebsd.org/D23518

Convert to if_foreach_llmaddr() KPI.

Reapply, with minor tweaks, r338025, from the original commit:

Remove unused and easy to misuse PNP macro parameter

Inspired by r338025, just remove the element size parameter to the
MODULE_PNP_INFO macro entirely. The 'table' parameter is now required to
have correct pointer (or array) type. Since all invocations of the macro
already had this property and the emitted PNP data continues to include the
element size, there is no functional change.

Mostly done with the coccinelle 'spatch' tool:

$ cat modpnpsize0.cocci
@normaltables@
identifier b,c;
expression a,d,e;
declarer MODULE_PNP_INFO;
@@
MODULE_PNP_INFO(a,b,c,d,
-sizeof(d[0]),
e);

@singletons@
identifier b,c,d;
expression a;
declarer MODULE_PNP_INFO;
@@
MODULE_PNP_INFO(a,b,c,&d,
-sizeof(d),
1);

$ rg -l MODULE_PNP_INFO -- sys | \
xargs spatch --in-place --sp-file modpnpsize0.cocci

(Note that coccinelle invokes diff(1) via a PATH search and expects diff to
tolerate the -B flag, which BSD diff does not. So I had to link gdiff into
PATH as diff to use spatch.)

Tinderbox'd (-DMAKE_JUST_KERNELS).
Approved by: re (glen)

Back out r338035 until Warner is finished churning GSoC PNP patches

I was not aware Warner was making or planning to make forward progress in
this area and have since been informed of that.

It's easy to apply/reapply when churn dies down.

Remove unused and easy to misuse PNP macro parameter

Inspired by r338025, just remove the element size parameter to the
MODULE_PNP_INFO macro entirely. The 'table' parameter is now required to
have correct pointer (or array) type. Since all invocations of the macro
already had this property and the emitted PNP data continues to include the
element size, there is no functional change.

Mostly done with the coccinelle 'spatch' tool:

$ cat modpnpsize0.cocci
@normaltables@
identifier b,c;
expression a,d,e;
declarer MODULE_PNP_INFO;
@@
MODULE_PNP_INFO(a,b,c,d,
-sizeof(d[0]),
e);

@singletons@
identifier b,c,d;
expression a;
declarer MODULE_PNP_INFO;
@@
MODULE_PNP_INFO(a,b,c,&d,
-sizeof(d),
1);

$ rg -l MODULE_PNP_INFO -- sys | \
xargs spatch --in-place --sp-file modpnpsize0.cocci

(Note that coccinelle invokes diff(1) via a PATH search and expects diff to
tolerate the -B flag, which BSD diff does not. So I had to link gdiff into
PATH as diff to use spatch.)

Tinderbox'd (-DMAKE_JUST_KERNELS).

Add PNP info to PCI attachment of ale driver

Reviewed by: imp, chuck
Submitted by: Lakhan Shiva Kamireddy <lakhanshiva@gmail.com>
Sponsored by: Google, Inc. (GSoC 2018)

ifnet: Replace if_addr_lock rwlock with epoch + mutex

Run on LLNW canaries and tested by pho@

gallatin:
Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5
based ConnectX 4-LX NIC, I see an almost 12% improvement in received
packet rate, and a larger improvement in bytes delivered all the way
to userspace.

When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1,
I see, using nstat -I mce0 1 before the patch:

InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree
4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32
4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32
4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32
4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32
4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32
4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32
4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32

After the patch

InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree
5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51
5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51
5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51
5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51
5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52
5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52

Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch

Reviewed by: gallatin
Sponsored by: Limelight Networks
Differential Revision: https://reviews.freebsd.org/D15366

sys/dev: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

sys/dev: use our nitems() macro when it is avaliable through param.h.

No functional change, only trivial cases are done in this sweep,
Drivers that can get further enhancements will be done independently.

Discussed in: freebsd-current

Fix variable assignment.

Found by: PVS-Studio

Fix a long standing bug in MAC statistics register access. One
additional register was erroneously added in the MAC register set
such that 7 TX statistics counters were wrong.

Mechanically convert to if_inc_counter().

Use define from if_var.h to access a field inside struct if_data,
that resides in struct ifnet.

Sponsored by: Nginx, Inc.

Fix various NIC drivers to properly cleanup static DMA resources.
In particular, don't check the value of the bus_dma map against NULL
to determine if either bus_dmamem_alloc() or bus_dmamap_load() succeeded.
Instead, assume that bus_dmamap_load() succeeeded (and thus that
bus_dmamap_unload() should be called) if the bus address for a resource
is non-zero, and assume that bus_dmamem_alloc() succeeded (and thus
that bus_dmamem_free() should be called) if the virtual address for a
resource is not NULL.

In many cases these bugs could result in leaks when a driver was detached.

Reviewed by: yongari
MFC after: 2 weeks

m_defrag(9) does not touch original mbuf chain when it can't
allocate new mbuf. Free original mbuf chain when driver is not
able to send the packet.

The r48589 promised to remove implicit inclusion of if_var.h soon. Prepare
to this event, adding if_var.h to files that do need it. Also, include
all includes that now are included due to implicit pollution via if_var.h

Sponsored by: Netflix
Sponsored by: Nginx, Inc.

Make sure to stop both TX and RX MACs in ale_stop_mac(). Previously
it used to stop TX MAC only such that MAC reconfiguration after
getting a link didn't work as expected.

PR: kern/173652

Mechanically substitute flags from historic mbuf allocator with
malloc(9) flags in sys/dev.

Remove duplicate const specifiers in many drivers (I hope I got all of
them, please let me know if not). Most of these are of the form:

static const struct bzzt_type {
[...list of members...]
} const bzzt_devs[] = {
[...list of initializers...]
};

The second const is unnecessary, as arrays cannot be modified anyway,
and if the elements are const, the whole thing is const automatically
(e.g. it is placed in .rodata).

I have verified this does not change the binary output of a full kernel
build (except for build timestamps embedded in the object files).

Reviewed by: yongari, marius
MFC after: 1 week

Fix typo. Check against number of allocated MSI-X vectors.
There is no ale(4) controller that supports MSI-X so this is not
real issue.

PR: kern/171825

- Const'ify the device lookup-table.
- Use DEVMETHOD_END.
- Use NULL instead of 0 for pointers.
- Enable support for flow control.
Tested by: yongari

MFC after: 1 week

Do not report current link status if driver is not running.
This change also workarounds dhclient's link state handling bug by
not giving current link status.

Unlike other controllers, ale(4)'s PHY hibernation perfectly works
such that driver does not see a valid link if the controller is not
brought up. If dhclient(8) runs on ale(4) it will blindly waits
until link UP and then gives up after 10 seconds. Because
dhclient(8) still thinks interface got a valid link when IFM_AVALID
is not set for selected media, this change makes dhclient initiate
DHCP without waiting for link UP.

Remove task queue based link state change handler. Driver no longer
needs to defer link state handling.
While I'm here, mark IFF_DRV_RUNNING before changing media. If
link is established without any delay, that link state change
handling could be lost.

Close a race where SIOCGIFMEDIA ioctl get inconsistent link status.
Because driver is accessing a common MII structure in
mii_pollstat(), updating user supplied structure should be done
before dropping a driver lock.

Reported by: Karim (fodillemlinkarimi <> gmail dot com)

- Remove attempts to implement setting of BMCR_LOOP/MIIF_NOLOOP
(reporting IFM_LOOP based on BMCR_LOOP is left in place though as
it might provide useful for debugging). For most mii(4) drivers it
was unclear whether the PHYs driven by them actually support
loopback or not. Moreover, typically loopback mode also needs to
be activated on the MAC, which none of the Ethernet drivers using
mii(4) implements. Given that loopback media has no real use (and
obviously hardly had a chance to actually work) besides for driver
development (which just loopback mode should be sufficient for
though, i.e one doesn't necessary need support for loopback media)
support for it is just dropped as both NetBSD and OpenBSD already
did quite some time ago.
- Let mii_phy_add_media() also announce the support of IFM_NONE.
- Restructure the PHY entry points to use a structure of entry points
instead of discrete function pointers, and extend this to include
a "reset" entry point. Make sure any PHY-specific reset routine is
always used, and provide one for lxtphy(4) which disables MII
interrupts (as is done for a few other PHYs we have drivers for).
This includes changing NIC drivers which previously just called the
generic mii_phy_reset() to now actually call the PHY-specific reset
routine, which might be crucial in some cases. While at it, the
redundant checks in these NIC drivers for mii->mii_instance not being
zero before calling the reset routines were removed because as soon
as one PHY driver attaches mii->mii_instance is incremented and we
hardly can end up in their media change callbacks etc if no PHY driver
has attached as mii_attach() would have failed in that case and not
attach a miibus(4) instance.
Consequently, NIC drivers now no longer should call mii_phy_reset()
directly, so it was removed from EXPORT_SYMS.
- Add a mii_phy_dev_attach() as a companion helper to mii_phy_dev_probe().
The purpose of that function is to perform the common steps to attach
a PHY driver instance and to hook it up to the miibus(4) instance and to
optionally also handle the probing, addition and initialization of the
supported media. So all a PHY driver without any special requirements
has to do in its bus attach method is to call mii_phy_dev_attach()
along with PHY-specific MIIF_* flags, a pointer to its PHY functions
and the add_media set to one. All PHY drivers were updated to take
advantage of mii_phy_dev_attach() as appropriate. Along with these
changes the capability mask was added to the mii_softc structure so
PHY drivers taking advantage of mii_phy_dev_attach() but still
handling media on their own do not need to fiddle with the MII attach
arguments anyway.
- Keep track of the PHY offset in the mii_softc structure. This is done
for compatibility with NetBSD/OpenBSD.
- Keep track of the PHY's OUI, model and revision in the mii_softc
structure. Several PHY drivers require this information also after
attaching and previously had to wrap their own softc around mii_softc.
NetBSD/OpenBSD also keep track of the model and revision on their
mii_softc structure. All PHY drivers were updated to take advantage
as appropriate.
- Convert the mebers of the MII data structure to unsigned where
appropriate. This is partly inspired by NetBSD/OpenBSD.
- According to IEEE 802.3-2002 the bits actually have to be reversed
when mapping an OUI to the MII ID registers. All PHY drivers and
miidevs where changed as necessary. Actually this now again allows to
largely share miidevs with NetBSD, which fixed this problem already
9 years ago. Consequently miidevs was synced as far as possible.
- Add MIIF_NOMANPAUSE and mii_phy_flowstatus() calls to drivers that
weren't explicitly converted to support flow control before. It's
unclear whether flow control actually works with these but typically
it should and their net behavior should be more correct with these
changes in place than without if the MAC driver sets MIIF_DOPAUSE.

Obtained from: NetBSD (partially)
Reviewed by: yongari (earlier version), silence on arch@ and net@

Do a sweep of the tree replacing calls to pci_find_extcap() with calls to
pci_find_cap() instead.

Fix some bugs in my last set of changes to ale(4):
- Remove extra unlock from end of ale_start_locked().
- Expand scope of locking in interrupt handler.
- Move ether_ifdetach() earlier and retire now-unneeded DETACH flag.

Tested by: Aryeh Friedman
Reviewed by: yongari (earlier version)

sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly.

Commit the rest of the devices.

Add a 'locked' variant of the foo_start() routine and call it directly
from interrupt handlers and watchdog routines instead of queueing a task
to call foo_start().

Reviewed by: yongari
MFC after: 1 month

Remove unecessary and clearly wrong usage of atomic(9).

Reported by: avg

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Convert the PHY drivers to honor the mii_flags passed down and convert
the NIC drivers as well as the PHY drivers to take advantage of the
mii_attach() introduced in r213878 to get rid of certain hacks. For
the most part these were:
- Artificially limiting miibus_{read,write}reg methods to certain PHY
addresses; we now let mii_attach() only probe the PHY at the desired
address(es) instead.
- PHY drivers setting MIIF_* flags based on the NIC driver they hang
off from, partly even based on grabbing and using the softc of the
parent; we now pass these flags down from the NIC to the PHY drivers
via mii_attach(). This got us rid of all such hacks except those of
brgphy() in combination with bce(4) and bge(4), which is way beyond
what can be expressed with simple flags.

While at it, I took the opportunity to change the NIC drivers to pass
up the error returned by mii_attach() (previously by mii_phy_probe())
and unify the error message used in this case where and as appropriate
as mii_attach() actually can fail for a number of reasons, not just
because of no PHY(s) being present at the expected address(es).

Reviewed by: jhb, yongari

Make sure to not use stale ip/tcp header pointers. The ip/tcp
header parser uses m_pullup(9) to get access to mbuf chain.
m_pullup(9) can allocate new mbuf chain and free old one if the
space left in the mbuf chain is not enough to hold requested
contiguous bytes. Previously drivers can use stale ip/tcp header
pointer if m_pullup(9) returned new mbuf chain.

Reported by: Andrew Boyer (aboyer <> averesystems dot com)
MFC after: 10 days

MFC r207251:
It seems ale(4) controllers do not like to see TCP payload in the
first descriptor in TSO case. Otherwise controller can generate bad
frames during TSO. To address it, make sure to pull up ethernet +
IP + TCP header with options in first buffer. Also ensure the
buffer length of the first descriptor for TSO covers entire ethernet
+ IP + TCP with options and setup additional Tx descriptor if the
first buffer includes TCP payload.

Tested by: Amar Takhar <verm <> darkbeer dot org >

It seems ale(4) controllers do not like to see TCP payload in the
first descriptor in TSO case. Otherwise controller can generate bad
frames during TSO. To address it, make sure to pull up ethernet +
IP + TCP header with options in first buffer. Also ensure the
buffer length of the first descriptor for TSO covers entire ethernet
+ IP + TCP with options and setup additional Tx descriptor if the
first buffer includes TCP payload.

Tested by: Amar Takhar <verm <> darkbeer dot org >
MFC after: 1 week

MFC r206876:
With r206844, CSUM_TCP is also set for CSUM_TSO case. Modify
drivers to take into account for the change. Basically CSUM_TSO
should be checked before checking CSUM_TCP.

With r206844, CSUM_TCP is also set for CSUM_TSO case. Modify
drivers to take into account for the change. Basically CSUM_TSO
should be checked before checking CSUM_TCP.

MFC r204378:
Add TSO support on VLANs. While I'm here remove unnecessary check
of VLAN hardware checksum offloading. vlan(4) already takes care of
this.

Add TSO support on VLANs. While I'm here remove unnecessary check
of VLAN hardware checksum offloading. vlan(4) already takes care of
this.

MFC 197627.
Fix multicast handling. All Atheros controllers use big-endian form
in computing multicast hash.

PR: kern/139137

Fix multicast handling. All Atheros controllers use big-endian form
in computing multicast hash.

PR: kern/139137

Disable Rx checksum offload until I find more clue why it breaks
under certain environments. However give users chance to override
it when he/she surely knows his/her hardware works with Rx checksum
offload.

Reported by: Ulrich Spoerlein ( uqs <> spoerlein dot net )
MFC after: 1 week
Approved by: re (kensmith)

Use if_maddr_rlock()/if_maddr_runlock() rather than IF_ADDR_LOCK()/
IF_ADDR_UNLOCK() across network device drivers when accessing the
per-interface multicast address list, if_multiaddrs. This will
allow us to change the locking strategy without affecting our driver
programming interface or binary interface.

For two wireless drivers, remove unnecessary locking, since they
don't actually access the multicast address list.

Approved by: re (kib)
MFC after: 6 weeks

Now pci(4) handles PCIM_CMD_INTxDIS so there is no need to poke
this bit in driver.

AR8113 also need to set DMA read burst value. This should fix
occasional DMA read error seen on AR8113.

Submitted by: Jie Yang < Jie.Yang <> Atheros com >

Add some PHY magic to enable PHY hibernation and 1000baseT/10baseT
power adjustment. This change is required to guarantee correct
operation on certain switches.

Submitted by: Jie Yang < Jie.Yang <> Atheros com >

Add ale(4), a driver for Atheros AR8121/AR8113/AR8114 PCIe ethernet
controller. The controller is also known as L1E(AR8121) and
L2E(AR8113/AR8114). Unlike its predecessor Attansic L1,
AR8121/AR8113/AR8114 uses completely different Rx logic such that
it requires separate driver. Datasheet for AR81xx is not available
to open source driver writers but it shares large part of Tx and
PHY logic of L1. I still don't understand some part of register
meaning and some MAC statistics counters but the driver seems to
have no critical issues for performance and stability.

The AR81xx requires copy operation to pass received frames to upper
stack such that ale(4) consumes a lot of CPU cycles than that of
other controller. A couple of silicon bugs also adds more CPU
cycles to address the known hardware bug. However, if you have fast
CPU you can still saturate the link.
Currently ale(4) supports the following hardware features.
- MSI.
- TCP Segmentation offload.
- Hardware VLAN tag insertion/stripping with checksum offload.
- Tx TCP/UDP checksum offload and Rx IP/TCP/UDP checksum offload.
- Tx/Rx interrupt moderation.
- Hardware statistics counters.
- Jumbo frame.
- WOL.

AR81xx PCIe ethernet controllers are mainly found on ASUS EeePC or
P5Q series of ASUS motherboards. Special thanks to Jeremy Chadwick
who sent the hardware to me. Without his donation writing a driver
for AR81xx would never have been possible. Big thanks to all people
who reported feedback or tested patches.

HW donated by: koitsu
Tested by: bsam, Joao Barros <joao.barros <> gmail DOT com >
Jan Henrik Sylvester <me <> janh DOT de >
Ivan Brawley < ivan <> brawley DOT id DOT au >,
CURRENT ML