MFC r282970: Close some potential races around socket start/close.

There are some reports about panics on ic->ic_socket NULL derefence.
This kind of races is the only way I can imagine it to happen.

Merge r263233 from HEAD to stable/10:

Update kernel inclusions of capability.h to use capsicum.h instead; some
further refinement is required as some device drivers intended to be
portable over FreeBSD versions rely on __FreeBSD_version to decide whether
to include capability.h.

Sponsored by: Google, Inc.

MFC r274853:
For both iSCSI initiator and target increase socket buffer sizes before
establishing connection.

This is a workaround for Chelsio TOE driver, that does not update socket
buffer size in hardware after connection established, and unless that is
done beforehand, kernel code will stuck, attempting to send/receive full
PDU at once.

MFC r272765: Remove one second wait for threads exit from icl_conn_close().

Switch it from polling with pause() to using cv_wait()/cv_signal().

MFC r274033 (by trasz):
s/icl_pdu_new/icl_pdu_new_empty/g; no functional changes.

MFC r274036 (by trasz):
s/icl_pdu_new_bhs/icl_pdu_new/; no functional changes, just a little
nicer code.

MFC r272812: Make iSCSI connection close somewhat less aggressive.

It allows to push out some final data from the send queue to the socket
before its close. In particular, it increases chances for logout response
to be delivered to the initiator.

MFC r270282:

Use proper include paths in kernel iSCSI code.

Sponsored by: The FreeBSD Foundation

MFC r270279:

Make the iSCSI stack use __FBSDID() properly.

Sponsored by: The FreeBSD Foundation

MFC r269197:

Fix potential double free that could happen after connection error.

MFC r264355 by mav@:

Remove unused val argument value from SYSCTL_INT() calls.

MFC r264545:

Fix typo.

Sponsored by: The FreeBSD Foundation

MFC r264348 by mav@:

Improve use of socket buffer upcalls.

Use soreadable()/sowriteable() in socket upcalls to avoid extra wakeups
until we have enough data to read or space to write.

Increase partial receive len from 1K to 128K to not wake up on every
received packet.

This significantly reduces locks congestion and CPU usage and improves
throughput for large I/Os on NICs without TSO and LRO.

MFC r264552 by mav@:

Close the race in older code, that caused connection stuck after r264348.

Sponsored by: iXsystems, Inc.

MFC r263740:

Use a less unusual syntax in debug printfs.

Sponsored by: The FreeBSD Foundation

MFC r264163:

Remove hack to pass STAILQ to a function and do it properly instead.

Sponsored by: The FreeBSD Foundation

MFC r264122:

Rework the iSCSI PDU transmit code to avoid lock contention and coalesce
PDUs before sending.

Sponsored by: The FreeBSD Foundation

MFC r264110:

All the iSCSI sysctls are also tunables; advertise that.

Sponsored by: The FreeBSD Foundation

MFC r264109:

We don't need TAILQ for iSCSI PDUs; STAILQ is enough.

Sponsored by: The FreeBSD Foundation

MFC r264026:

Enable a KASSERT.

Sponsored by: The FreeBSD Foundation

MFC r264025:

Get rid of the "autoscaling", instead just set socket buffer sizes
in the usual way. The only thing the old code did was making things
less predictable.

MFC r264058:

Fix build, broken by r264025.

Sponsored by: The FreeBSD Foundation

MFC r264023:

Instead of "icltx" and "iclrx", use thread names with prefix from upper
layer, so that one can see which side of the stack the threads are for.

Sponsored by: The FreeBSD Foundation

MFC r264022:

Get rid of ICL lock; use upper-layer (initiator or target) lock instead.
This avoids extra locking in icl_pdu_queue(); the upper layer needs to call
it while holding its own lock anyway, to avoid sending PDUs out of order.

Sponsored by: The FreeBSD Foundation

MFC r263743:

Move the ic_outstanding_count under #ifdef DIAGNOSTIC.

Sponsored by: The FreeBSD Foundation

MFC r260389:

Fix a rare "truncated checksums" problem, which manifested like this:

WARNING: icl_pdu_check_data_digest: data digest check failed; got 0xf23b,
should be 0xdb7f23b

Sponsored by: The FreeBSD Foundation

MFC r260083:

Fix extremely slow operation with data digests enabled. This was caused
by receive code waiting for data digest even when the data segment was
empty. It didn't actually read it, but it waited until those four bytes
become available in the socket buffer, i.e. until any other PDU (e.g. NOP)
came in.

Sponsored by: The FreeBSD Foundation

MFC r257061:

Don't spin with mutex hold when there is not enough room in the send socket
buffer. While here, make the code flow somewhat nicer.

Thanks to mav@ for tracking it down.

Approved by: re (glebius)

Merge r263233 from HEAD to stable/10:

Update kernel inclusions of capability.h to use capsicum.h instead; some
further refinement is required as some device drivers intended to be
portable over FreeBSD versions rely on __FreeBSD_version to decide whether
to include capability.h.

Sponsored by: Google, Inc.

MFC r274853:
For both iSCSI initiator and target increase socket buffer sizes before
establishing connection.

This is a workaround for Chelsio TOE driver, that does not update socket
buffer size in hardware after connection established, and unless that is
done beforehand, kernel code will stuck, attempting to send/receive full
PDU at once.

MFC r272765: Remove one second wait for threads exit from icl_conn_close().

Switch it from polling with pause() to using cv_wait()/cv_signal().

MFC r274033 (by trasz):
s/icl_pdu_new/icl_pdu_new_empty/g; no functional changes.

MFC r274036 (by trasz):
s/icl_pdu_new_bhs/icl_pdu_new/; no functional changes, just a little
nicer code.

MFC r272812: Make iSCSI connection close somewhat less aggressive.

It allows to push out some final data from the send queue to the socket
before its close. In particular, it increases chances for logout response
to be delivered to the initiator.

MFC r270282:

Use proper include paths in kernel iSCSI code.

Sponsored by: The FreeBSD Foundation

MFC r270279:

Make the iSCSI stack use __FBSDID() properly.

Sponsored by: The FreeBSD Foundation

MFC r269197:

Fix potential double free that could happen after connection error.

MFC r264355 by mav@:

Remove unused val argument value from SYSCTL_INT() calls.

MFC r264545:

Fix typo.

Sponsored by: The FreeBSD Foundation

MFC r264348 by mav@:

Improve use of socket buffer upcalls.

Use soreadable()/sowriteable() in socket upcalls to avoid extra wakeups
until we have enough data to read or space to write.

Increase partial receive len from 1K to 128K to not wake up on every
received packet.

This significantly reduces locks congestion and CPU usage and improves
throughput for large I/Os on NICs without TSO and LRO.

MFC r264552 by mav@:

Close the race in older code, that caused connection stuck after r264348.

Sponsored by: iXsystems, Inc.

MFC r263740:

Use a less unusual syntax in debug printfs.

Sponsored by: The FreeBSD Foundation

MFC r264163:

Remove hack to pass STAILQ to a function and do it properly instead.

Sponsored by: The FreeBSD Foundation

MFC r264122:

Rework the iSCSI PDU transmit code to avoid lock contention and coalesce
PDUs before sending.

Sponsored by: The FreeBSD Foundation

MFC r264110:

All the iSCSI sysctls are also tunables; advertise that.

Sponsored by: The FreeBSD Foundation

MFC r264109:

We don't need TAILQ for iSCSI PDUs; STAILQ is enough.

Sponsored by: The FreeBSD Foundation

MFC r264026:

Enable a KASSERT.

Sponsored by: The FreeBSD Foundation

MFC r264025:

Get rid of the "autoscaling", instead just set socket buffer sizes
in the usual way. The only thing the old code did was making things
less predictable.

MFC r264058:

Fix build, broken by r264025.

Sponsored by: The FreeBSD Foundation

MFC r264023:

Instead of "icltx" and "iclrx", use thread names with prefix from upper
layer, so that one can see which side of the stack the threads are for.

Sponsored by: The FreeBSD Foundation

MFC r264022:

Get rid of ICL lock; use upper-layer (initiator or target) lock instead.
This avoids extra locking in icl_pdu_queue(); the upper layer needs to call
it while holding its own lock anyway, to avoid sending PDUs out of order.

Sponsored by: The FreeBSD Foundation

MFC r263743:

Move the ic_outstanding_count under #ifdef DIAGNOSTIC.

Sponsored by: The FreeBSD Foundation

MFC r260389:

Fix a rare "truncated checksums" problem, which manifested like this:

WARNING: icl_pdu_check_data_digest: data digest check failed; got 0xf23b,
should be 0xdb7f23b

Sponsored by: The FreeBSD Foundation

MFC r260083:

Fix extremely slow operation with data digests enabled. This was caused
by receive code waiting for data digest even when the data segment was
empty. It didn't actually read it, but it waited until those four bytes
become available in the socket buffer, i.e. until any other PDU (e.g. NOP)
came in.

Sponsored by: The FreeBSD Foundation

MFC r257061:

Don't spin with mutex hold when there is not enough room in the send socket
buffer. While here, make the code flow somewhat nicer.

Thanks to mav@ for tracking it down.

Approved by: re (glebius)