OFED: Unbreak SDP support in ibcore

This regression was introduced in the r326169 Linux v4.9 Infiniband upgrade.
Restore the functionality.

Reviewed by: hselasky
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D21298

SDP: SYSCTL_DECL SDP-wide sysctl node in header

This allows use of the shared _net_inet_sdp in more than one compilation
unit. (Nothing in-tree uses this today, but some of Isilon's out-of-tree
SDP enhancements add sysctls below the node.)

Sponsored by: Dell EMC Isilon

Adapt the existing SDP ULP code to the new ibcore APIs.

Requested by: Sobczak, Bartosz <bartosz.sobczak@intel.com>
Sponsored by: Mellanox Technologies

sdp: Use malloc(9) instead of the Linux compat layer.

SDP transmit and receive rings are always created in a sleepable context,
so we can use M_WAITOK and remove error checks.

Sponsored by: EMC / Isilon Storage Division

sdp: Use an mbufq for received control packets.

This is simpler than the hand-rolled queue, and fixes a use-after-free.

Sponsored by: EMC / Isilon Storage Division

Prepare for activation of LinuxKPI module parameters as read-only
tunable SYSCTL's. Linux module parameters are associated with the
module they belong to. FreeBSD does not share this concept of a parent
module. Instead add macros which define the prefix to use for the
module parameters in the LinuxKPI consumers.

While at it convert all "bool" LinuxKPI module parameters to "byte"
type, because we don't have a "bool" type of SYSCTL in FreeBSD.

Sponsored by: Mellanox Technologies
MFC after: 1 week

ofed/drivers: minor spelling fixes.

No functional change.

Reviewed by: hselasky

Forward declare struct kiocb, which is only used for an unsued function
argument but not actually defined anywhere.

This fixes the compile complaining about
"declaration of 'struct kiocb' will not be visible outside of this function".

MFC after: 2 weeks
X-MFC with: whatever changed caused the breakage ;-)

The r48589 promised to remove implicit inclusion of if_var.h soon. Prepare
to this event, adding if_var.h to files that do need it. Also, include
all includes that now are included due to implicit pollution via if_var.h

Sponsored by: Netflix
Sponsored by: Nginx, Inc.

- Merge in OFED 1.5.3 from projects/ofed/head