Don't fetch data beyond end of inquiry buffer, which, here, is not
NUL-terminated.

Reduce target buffer to needed size (product name + NUL terminator).

Set disk product name as disk type. The information can be queried through
drvctl(4).

branches: 1.38.16;
Refactored sd and cd to use common disk subroutines.

branches: 1.37.2;
would you like some freshly ground _KERNEL_OPT with that?
yes? excellent choice, sir/madam.

Convert sys/dev to use <sys/rndsource.h>.

branches: 1.35.2;
Don't nest structure definitions.

branches: 1.34.6;
Entropy-pool implementation move and cleanup.

1) Move core entropy-pool code and source/sink/sample management code
to sys/kern from sys/dev.

2) Remove use of NRND as test for presence of entropy-pool code throughout
source tree.

3) Remove use of RND_ENABLED in device drivers as microoptimization to
avoid expensive operations on disabled entropy sources; make the
rnd_add calls do this directly so all callers benefit.

4) Fix bug in recent rnd_add_data()/rnd_add_uint32() changes that might
have lead to slight entropy overestimation for some sources.

5) Add new source types for environmental sensors, power sensors, VM
system events, and skew between clocks, with a sample implementation
for each.

ok releng to go in before the branch due to the difficulty of later
pullup (widespread #ifdef removal and moved files). Tested with release
builds on amd64 and evbarm and live testing on amd64.

branches: 1.33.2;
First step of random number subsystem rework described in
<20111022023242.BA26F14A158@mail.netbsd.org>. This change includes
the following:

An initial cleanup and minor reorganization of the entropy pool
code in sys/dev/rnd.c and sys/dev/rndpool.c. Several bugs are
fixed. Some effort is made to accumulate entropy more quickly at
boot time.

A generic interface, "rndsink", is added, for stream generators to
request that they be re-keyed with good quality entropy from the pool
as soon as it is available.

The arc4random()/arc4randbytes() implementation in libkern is
adjusted to use the rndsink interface for rekeying, which helps
address the problem of low-quality keys at boot time.

An implementation of the FIPS 140-2 statistical tests for random
number generator quality is provided (libkern/rngtest.c). This
is based on Greg Rose's implementation from Qualcomm.

A new random stream generator, nist_ctr_drbg, is provided. It is
based on an implementation of the NIST SP800-90 CTR_DRBG by
Henric Jungheim. This generator users AES in a modified counter
mode to generate a backtracking-resistant random stream.

An abstraction layer, "cprng", is provided for in-kernel consumers
of randomness. The arc4random/arc4randbytes API is deprecated for
in-kernel use. It is replaced by "cprng_strong". The current
cprng_fast implementation wraps the existing arc4random
implementation. The current cprng_strong implementation wraps the
new CTR_DRBG implementation. Both interfaces are rekeyed from
the entropy pool automatically at intervals justifiable from best
current cryptographic practice.

In some quick tests, cprng_fast() is about the same speed as
the old arc4randbytes(), and cprng_strong() is about 20% faster
than rnd_extract_data(). Performance is expected to improve.

The AES code in src/crypto/rijndael is no longer an optional
kernel component, as it is required by cprng_strong, which is
not an optional kernel component.

The entropy pool output is subjected to the rngtest tests at
startup time; if it fails, the system will reboot. There is
approximately a 3/10000 chance of a false positive from these
tests. Entropy pool _input_ from hardware random numbers is
subjected to the rngtest tests at attach time, as well as the
FIPS continuous-output test, to detect bad or stuck hardware
RNGs; if any are detected, they are detached, but the system
continues to run.

A problem with rndctl(8) is fixed -- datastructures with
pointers in arrays are no longer passed to userspace (this
was not a security problem, but rather a major issue for
compat32). A new kernel will require a new rndctl.

The sysctl kern.arandom() and kern.urandom() nodes are hooked
up to the new generators, but the /dev/*random pseudodevices
are not, yet.

Manual pages for the new kernel interfaces are forthcoming.

branches: 1.32.12;
Use a PMF shutdown hook instead of shutdownhook_establish(). This ought
to stop the crash during shutdown that yamt@ reports in kern/41182.

branches: 1.31.2; 1.31.8;
split device/softc for sd (tested with a USB stick)

branches: 1.30.2; 1.30.4; 1.30.6;
Remove clause 3 and 4 from TNF licenses

branches: 1.29.28; 1.29.30; 1.29.32;
Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements

branches: 1.28.30; 1.28.32;
merge ktrace-lwp.

- change the way to specify a bufq strategy. (by string rather than by number)
- rather than embedding bufq_state in driver softc,
have a pointer to the former.
- move bufq related functions from kern/subr_disk.c to kern/subr_bufq.c.
- rename method to strategy for consistency.
- move some definitions which don't need to be exposed to the rest of kernel
from sys/bufq.h to sys/bufq_impl.h.
(is it better to move it to kern/ or somewhere?)
- fix some obvious breakage in dev/qbus/ts.c. (not tested)

branches: 1.26.6;
Backing out changes to clean up scsipi. I was pointed out there were
problems i hadn't seen. To prevent lossage i'd decided to back off all
changes and let them be reviewed on tech-kern.

Part of the cleanup of sys/scsipi's use of types; rename all u_int* to
uint* and change the u_long's to uint32_t's where possible. Note that the
iocl definitions/hooks have to be ulong (or u_long) or they'll bomb out.

branches: 1.24.4;
Don't need the lock in the softc anymore.

Do not manipulate xs->bp in "generic" code -- do it only in the psw_done
routine. As part of this, pass down our pre-parsed error code -- though this
interface will probably change later to accomodate better error handling.

Improve handling of memory shortage, to fix problems like:
sd3(mpt0:0:1:0): unable to allocate scsipi_xfer
sd3: not queued, error 12
Havard Eidnes's analysis of this problem is that the scsipi_xfer pool is
competing for resources with other pools, including the the inode and vnode
pools which can grow quite large.

*_scsipi_cmd(): don't biodone the buffer if scsipi_make_xs() fails, let the
caller deal with the problem
start function of block devices drivers: dequeue the buffer after the
scsipi_command() call. If scsipi_command() fails with ENOMEM don't dequeue
the buffer, and schedule a callout to call the start function after
some delay.
scsipi_init(): prime the scsipi_xfer_pool with one page. This ensure that
there is always some scsipi_xfer to play with. If scsipi_command() fails
because of pool_get(), we're sure there will be resources available later,
when the pending commands have completed.

Reviewed by Jason Thorpe and Havard Eidnes.
Todo: remove the "unable to allocate scsipi_xfer" and "not queued, error %d"
printfs, but I choose to keep them for now, to help make sure the code does
what it should.

branches: 1.21.2;
parameter with two es. From Peter Postma.

Merge the geometry and cache handling code for all direct access and optical
devices, as it's general to all SCSI MMC devices. In the process, remove
PQUIRK_NO_FLEX_PAGE.

branches: 1.19.2;
Check RAW_PART against the media size instead of the disklabel.
Add the media size in 512-byte sectors to the softc, to avoid
some 64 bit computations. Bump the capacity stored in softcs
for disks to 64 bits.

Add multiple inclusion protection.

Implement a new device buffer queue interface.
One basic struct, a function to setup a queue with a specific strategy and
three macros to put buf's into the queue, get and remove the next buf or
get the next buf without removal.

The BUFQ_XXX interface will be removed in the future.
The B_ORDERED flag is not longer supported.

Approved by: Jason R. Thorpe <thorpej@wasabisystems.com>

branches: 1.16.8;
Add ioctls to get (DIOCGCACHE) and set (DIOCSCACHE) cache enables
on disks in a generic way. Implement these ioctls for SCSI disks.

This is not fully fleshed-out yet, but it allows people to experiment
with disk caches more easily.

branches: 1.15.2;
fix typo in previous revision so that the scsi code compiles again.

Make SDRETRIES an option you can dig out of opt_scsi.h.

Add SD_IO_TIMEOUT (an option in opt_scsi.h) also, which defaults to the
normal 60 * 1000 timeout for normal read/write commamds. This allows you
to shorten or lengthen the timeout as needed.

Pull up the thorpej_scsipi branch to main branch.
This is a completely rewritten scsipi_xfer execution engine, and the
associated changes to HBA drivers. Overview of changes & features:
- All xfers are queued in the mid-layer, rather than doing so in an
ad-hoc fashion in individual adapter drivers.
- Adapter/channel resource management in the mid-layer, avoids even trying
to start running an xfer if the adapter/channel doesn't have the resources.
- Better communication between the mid-layer and the adapters.
- Asynchronous event notification mechanism from adapter to mid-layer and
peripherals.
- Better peripheral queue management: freeze/thaw, sorted requeueing during
recovery, etc.
- Clean separation of peripherals, adapters, and adapter channels (no more
scsipi_link).
- Kernel thread for each scsipi_channel makes error recovery much easier
(no more dealing with interrupt context when recovering from an error).
- Mid-layer support for tagged queueing: commands can have the tag type
set explicitly, tag IDs are allocated in the mid-layer (thus eliminating
the need to use buggy tag ID allocation schemes in many adapter drivers).
- support for QUEUE FULL and CHECK CONDITION status in mid-layer; the command
will be requeued, or a REQUEST SENSE will be sent as appropriate.

Just before the merge syssrc has been tagged with thorpej_scsipi_beforemerge

branches: 1.12.6;
sd_interpret_sense() can be called from interrupt context; call scsipi_start()
with XS_CTL_ASYNC if the failed command was called with XS_CTL_ASYNC.
Add a SDF_RESTART flag to keep state, cleared in sddone().
A mounted disk can now spin down, it will propely spin up at the next access.

include "rnd.h" explicitely since the size of sd_softc depends on it
(sd_scsi.c and sd_atapi.c missed it, leading to random memory corruption)

Update for sys/buf.h/disksort_*() changes.

branches: 1.9.2;
Implement detaching SCSI and ATAPI disks.

Make `sc_ops->sdo_flush' synchronous. The flags `SDF_FLUSHING' and
`SDF_DIRTY' were never reset because `sddone' doesn't get called from
synchronous scsi commands.

branches: 1.7.6;
Assign my copyrights to TNF.

Keep track if the disk has had done i/o, making it dirty. On last close,
or at shutdown, flush the cache if the disk is still dirty and it support
cache flushing. Fixes PRs #5588 (Jim Bernard) and #5589 (Justin Gibbs).

Improve generation of default disklabels:
-store printable product ID in cd's and sd's softc, use it as "typename"
-for this, add a "destination buffer length" argument to scsipi_strvis()
-return ATAPI device type for ATAPI devices

Add a shutdown hook for SCSI-2 and higher disks that issues a SYNCHRONIZE CACHE
operation with address 0 length 0, which, according to the SCSI-2 spec, should
be interpreted as "synchronize all remaining blocks beginning at address 0".

Backout previous change, and rather, remove all protection but the one
actually currently required. Suggested by Charles M. Hannum.

Fix or add protection for mutiple inclusion.

add support for an ATAPI attachment for 'sd'.
fix 'cd' driver's NCD_SCSI bogosity (was using testing wrong macro!)
clean up in various ways:
* make common atapi_mode_{sense,select}() functions.
* put ATAPI data structures in more sensible headers, split up by
device type.
* include headers a bit more carefully.
* pass flags to attachment-specific cd functions, and use them.
* get rid of SCSI bits in scsipi_base.h's scsipi_make_xs(), move
them into the correct place in scsi_base.c.
* fix minor typo in struct name in scsipiconf.h (which was apparently
never used except in a #define later in the same file).
* use __attribute__ to force 4-byte alignment for xs command store,
so that architectures trying to bus_space_write_multi_N() (where
N > 1) that data to a controller won't lose.
* clean up a few comments in typos, and make a few #defines easier to
understand/maintain.
* rename cd_link.h to cdvar.h (via repository copy). This is exactly
what a 'var' file is supposed to be.

Set disk product name as disk type. The information can be queried through
drvctl(4).

Refactored sd and cd to use common disk subroutines.

branches: 1.37.2;
would you like some freshly ground _KERNEL_OPT with that?
yes? excellent choice, sir/madam.

Convert sys/dev to use <sys/rndsource.h>.

branches: 1.35.2;
Don't nest structure definitions.

branches: 1.34.6;
Entropy-pool implementation move and cleanup.

1) Move core entropy-pool code and source/sink/sample management code
to sys/kern from sys/dev.

2) Remove use of NRND as test for presence of entropy-pool code throughout
source tree.

3) Remove use of RND_ENABLED in device drivers as microoptimization to
avoid expensive operations on disabled entropy sources; make the
rnd_add calls do this directly so all callers benefit.

4) Fix bug in recent rnd_add_data()/rnd_add_uint32() changes that might
have lead to slight entropy overestimation for some sources.

5) Add new source types for environmental sensors, power sensors, VM
system events, and skew between clocks, with a sample implementation
for each.

ok releng to go in before the branch due to the difficulty of later
pullup (widespread #ifdef removal and moved files). Tested with release
builds on amd64 and evbarm and live testing on amd64.

branches: 1.33.2;
First step of random number subsystem rework described in
<20111022023242.BA26F14A158@mail.netbsd.org>. This change includes
the following:

An initial cleanup and minor reorganization of the entropy pool
code in sys/dev/rnd.c and sys/dev/rndpool.c. Several bugs are
fixed. Some effort is made to accumulate entropy more quickly at
boot time.

A generic interface, "rndsink", is added, for stream generators to
request that they be re-keyed with good quality entropy from the pool
as soon as it is available.

The arc4random()/arc4randbytes() implementation in libkern is
adjusted to use the rndsink interface for rekeying, which helps
address the problem of low-quality keys at boot time.

An implementation of the FIPS 140-2 statistical tests for random
number generator quality is provided (libkern/rngtest.c). This
is based on Greg Rose's implementation from Qualcomm.

A new random stream generator, nist_ctr_drbg, is provided. It is
based on an implementation of the NIST SP800-90 CTR_DRBG by
Henric Jungheim. This generator users AES in a modified counter
mode to generate a backtracking-resistant random stream.

An abstraction layer, "cprng", is provided for in-kernel consumers
of randomness. The arc4random/arc4randbytes API is deprecated for
in-kernel use. It is replaced by "cprng_strong". The current
cprng_fast implementation wraps the existing arc4random
implementation. The current cprng_strong implementation wraps the
new CTR_DRBG implementation. Both interfaces are rekeyed from
the entropy pool automatically at intervals justifiable from best
current cryptographic practice.

In some quick tests, cprng_fast() is about the same speed as
the old arc4randbytes(), and cprng_strong() is about 20% faster
than rnd_extract_data(). Performance is expected to improve.

The AES code in src/crypto/rijndael is no longer an optional
kernel component, as it is required by cprng_strong, which is
not an optional kernel component.

The entropy pool output is subjected to the rngtest tests at
startup time; if it fails, the system will reboot. There is
approximately a 3/10000 chance of a false positive from these
tests. Entropy pool _input_ from hardware random numbers is
subjected to the rngtest tests at attach time, as well as the
FIPS continuous-output test, to detect bad or stuck hardware
RNGs; if any are detected, they are detached, but the system
continues to run.

A problem with rndctl(8) is fixed -- datastructures with
pointers in arrays are no longer passed to userspace (this
was not a security problem, but rather a major issue for
compat32). A new kernel will require a new rndctl.

The sysctl kern.arandom() and kern.urandom() nodes are hooked
up to the new generators, but the /dev/*random pseudodevices
are not, yet.

Manual pages for the new kernel interfaces are forthcoming.

branches: 1.32.12;
Use a PMF shutdown hook instead of shutdownhook_establish(). This ought
to stop the crash during shutdown that yamt@ reports in kern/41182.

branches: 1.31.2; 1.31.8;
split device/softc for sd (tested with a USB stick)

branches: 1.30.2; 1.30.4; 1.30.6;
Remove clause 3 and 4 from TNF licenses

branches: 1.29.28; 1.29.30; 1.29.32;
Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements

branches: 1.28.30; 1.28.32;
merge ktrace-lwp.

- change the way to specify a bufq strategy. (by string rather than by number)
- rather than embedding bufq_state in driver softc,
have a pointer to the former.
- move bufq related functions from kern/subr_disk.c to kern/subr_bufq.c.
- rename method to strategy for consistency.
- move some definitions which don't need to be exposed to the rest of kernel
from sys/bufq.h to sys/bufq_impl.h.
(is it better to move it to kern/ or somewhere?)
- fix some obvious breakage in dev/qbus/ts.c. (not tested)

branches: 1.26.6;
Backing out changes to clean up scsipi. I was pointed out there were
problems i hadn't seen. To prevent lossage i'd decided to back off all
changes and let them be reviewed on tech-kern.

Part of the cleanup of sys/scsipi's use of types; rename all u_int* to
uint* and change the u_long's to uint32_t's where possible. Note that the
iocl definitions/hooks have to be ulong (or u_long) or they'll bomb out.

branches: 1.24.4;
Don't need the lock in the softc anymore.

Do not manipulate xs->bp in "generic" code -- do it only in the psw_done
routine. As part of this, pass down our pre-parsed error code -- though this
interface will probably change later to accomodate better error handling.

Improve handling of memory shortage, to fix problems like:
sd3(mpt0:0:1:0): unable to allocate scsipi_xfer
sd3: not queued, error 12
Havard Eidnes's analysis of this problem is that the scsipi_xfer pool is
competing for resources with other pools, including the the inode and vnode
pools which can grow quite large.

*_scsipi_cmd(): don't biodone the buffer if scsipi_make_xs() fails, let the
caller deal with the problem
start function of block devices drivers: dequeue the buffer after the
scsipi_command() call. If scsipi_command() fails with ENOMEM don't dequeue
the buffer, and schedule a callout to call the start function after
some delay.
scsipi_init(): prime the scsipi_xfer_pool with one page. This ensure that
there is always some scsipi_xfer to play with. If scsipi_command() fails
because of pool_get(), we're sure there will be resources available later,
when the pending commands have completed.

Reviewed by Jason Thorpe and Havard Eidnes.
Todo: remove the "unable to allocate scsipi_xfer" and "not queued, error %d"
printfs, but I choose to keep them for now, to help make sure the code does
what it should.

branches: 1.21.2;
parameter with two es. From Peter Postma.

Merge the geometry and cache handling code for all direct access and optical
devices, as it's general to all SCSI MMC devices. In the process, remove
PQUIRK_NO_FLEX_PAGE.

branches: 1.19.2;
Check RAW_PART against the media size instead of the disklabel.
Add the media size in 512-byte sectors to the softc, to avoid
some 64 bit computations. Bump the capacity stored in softcs
for disks to 64 bits.

Add multiple inclusion protection.

Implement a new device buffer queue interface.
One basic struct, a function to setup a queue with a specific strategy and
three macros to put buf's into the queue, get and remove the next buf or
get the next buf without removal.

The BUFQ_XXX interface will be removed in the future.
The B_ORDERED flag is not longer supported.

Approved by: Jason R. Thorpe <thorpej@wasabisystems.com>

branches: 1.16.8;
Add ioctls to get (DIOCGCACHE) and set (DIOCSCACHE) cache enables
on disks in a generic way. Implement these ioctls for SCSI disks.

This is not fully fleshed-out yet, but it allows people to experiment
with disk caches more easily.

branches: 1.15.2;
fix typo in previous revision so that the scsi code compiles again.

Make SDRETRIES an option you can dig out of opt_scsi.h.

Add SD_IO_TIMEOUT (an option in opt_scsi.h) also, which defaults to the
normal 60 * 1000 timeout for normal read/write commamds. This allows you
to shorten or lengthen the timeout as needed.

Pull up the thorpej_scsipi branch to main branch.
This is a completely rewritten scsipi_xfer execution engine, and the
associated changes to HBA drivers. Overview of changes & features:
- All xfers are queued in the mid-layer, rather than doing so in an
ad-hoc fashion in individual adapter drivers.
- Adapter/channel resource management in the mid-layer, avoids even trying
to start running an xfer if the adapter/channel doesn't have the resources.
- Better communication between the mid-layer and the adapters.
- Asynchronous event notification mechanism from adapter to mid-layer and
peripherals.
- Better peripheral queue management: freeze/thaw, sorted requeueing during
recovery, etc.
- Clean separation of peripherals, adapters, and adapter channels (no more
scsipi_link).
- Kernel thread for each scsipi_channel makes error recovery much easier
(no more dealing with interrupt context when recovering from an error).
- Mid-layer support for tagged queueing: commands can have the tag type
set explicitly, tag IDs are allocated in the mid-layer (thus eliminating
the need to use buggy tag ID allocation schemes in many adapter drivers).
- support for QUEUE FULL and CHECK CONDITION status in mid-layer; the command
will be requeued, or a REQUEST SENSE will be sent as appropriate.

Just before the merge syssrc has been tagged with thorpej_scsipi_beforemerge

branches: 1.12.6;
sd_interpret_sense() can be called from interrupt context; call scsipi_start()
with XS_CTL_ASYNC if the failed command was called with XS_CTL_ASYNC.
Add a SDF_RESTART flag to keep state, cleared in sddone().
A mounted disk can now spin down, it will propely spin up at the next access.

include "rnd.h" explicitely since the size of sd_softc depends on it
(sd_scsi.c and sd_atapi.c missed it, leading to random memory corruption)

Update for sys/buf.h/disksort_*() changes.

branches: 1.9.2;
Implement detaching SCSI and ATAPI disks.

Make `sc_ops->sdo_flush' synchronous. The flags `SDF_FLUSHING' and
`SDF_DIRTY' were never reset because `sddone' doesn't get called from
synchronous scsi commands.

branches: 1.7.6;
Assign my copyrights to TNF.

Keep track if the disk has had done i/o, making it dirty. On last close,
or at shutdown, flush the cache if the disk is still dirty and it support
cache flushing. Fixes PRs #5588 (Jim Bernard) and #5589 (Justin Gibbs).

Improve generation of default disklabels:
-store printable product ID in cd's and sd's softc, use it as "typename"
-for this, add a "destination buffer length" argument to scsipi_strvis()
-return ATAPI device type for ATAPI devices

Add a shutdown hook for SCSI-2 and higher disks that issues a SYNCHRONIZE CACHE
operation with address 0 length 0, which, according to the SCSI-2 spec, should
be interpreted as "synchronize all remaining blocks beginning at address 0".

Backout previous change, and rather, remove all protection but the one
actually currently required. Suggested by Charles M. Hannum.

Fix or add protection for mutiple inclusion.

add support for an ATAPI attachment for 'sd'.
fix 'cd' driver's NCD_SCSI bogosity (was using testing wrong macro!)
clean up in various ways:
* make common atapi_mode_{sense,select}() functions.
* put ATAPI data structures in more sensible headers, split up by
device type.
* include headers a bit more carefully.
* pass flags to attachment-specific cd functions, and use them.
* get rid of SCSI bits in scsipi_base.h's scsipi_make_xs(), move
them into the correct place in scsi_base.c.
* fix minor typo in struct name in scsipiconf.h (which was apparently
never used except in a #define later in the same file).
* use __attribute__ to force 4-byte alignment for xs command store,
so that architectures trying to bus_space_write_multi_N() (where
N > 1) that data to a controller won't lose.
* clean up a few comments in typos, and make a few #defines easier to
understand/maintain.
* rename cd_link.h to cdvar.h (via repository copy). This is exactly
what a 'var' file is supposed to be.

Refactored sd and cd to use common disk subroutines.

would you like some freshly ground _KERNEL_OPT with that?
yes? excellent choice, sir/madam.

Convert sys/dev to use <sys/rndsource.h>.

branches: 1.35.2;
Don't nest structure definitions.

Entropy-pool implementation move and cleanup.

1) Move core entropy-pool code and source/sink/sample management code
to sys/kern from sys/dev.

2) Remove use of NRND as test for presence of entropy-pool code throughout
source tree.

3) Remove use of RND_ENABLED in device drivers as microoptimization to
avoid expensive operations on disabled entropy sources; make the
rnd_add calls do this directly so all callers benefit.

4) Fix bug in recent rnd_add_data()/rnd_add_uint32() changes that might
have lead to slight entropy overestimation for some sources.

5) Add new source types for environmental sensors, power sensors, VM
system events, and skew between clocks, with a sample implementation
for each.

ok releng to go in before the branch due to the difficulty of later
pullup (widespread #ifdef removal and moved files). Tested with release
builds on amd64 and evbarm and live testing on amd64.

branches: 1.33.2;
First step of random number subsystem rework described in
<20111022023242.BA26F14A158@mail.netbsd.org>. This change includes
the following:

An initial cleanup and minor reorganization of the entropy pool
code in sys/dev/rnd.c and sys/dev/rndpool.c. Several bugs are
fixed. Some effort is made to accumulate entropy more quickly at
boot time.

A generic interface, "rndsink", is added, for stream generators to
request that they be re-keyed with good quality entropy from the pool
as soon as it is available.

The arc4random()/arc4randbytes() implementation in libkern is
adjusted to use the rndsink interface for rekeying, which helps
address the problem of low-quality keys at boot time.

An implementation of the FIPS 140-2 statistical tests for random
number generator quality is provided (libkern/rngtest.c). This
is based on Greg Rose's implementation from Qualcomm.

A new random stream generator, nist_ctr_drbg, is provided. It is
based on an implementation of the NIST SP800-90 CTR_DRBG by
Henric Jungheim. This generator users AES in a modified counter
mode to generate a backtracking-resistant random stream.

An abstraction layer, "cprng", is provided for in-kernel consumers
of randomness. The arc4random/arc4randbytes API is deprecated for
in-kernel use. It is replaced by "cprng_strong". The current
cprng_fast implementation wraps the existing arc4random
implementation. The current cprng_strong implementation wraps the
new CTR_DRBG implementation. Both interfaces are rekeyed from
the entropy pool automatically at intervals justifiable from best
current cryptographic practice.

In some quick tests, cprng_fast() is about the same speed as
the old arc4randbytes(), and cprng_strong() is about 20% faster
than rnd_extract_data(). Performance is expected to improve.

The AES code in src/crypto/rijndael is no longer an optional
kernel component, as it is required by cprng_strong, which is
not an optional kernel component.

The entropy pool output is subjected to the rngtest tests at
startup time; if it fails, the system will reboot. There is
approximately a 3/10000 chance of a false positive from these
tests. Entropy pool _input_ from hardware random numbers is
subjected to the rngtest tests at attach time, as well as the
FIPS continuous-output test, to detect bad or stuck hardware
RNGs; if any are detected, they are detached, but the system
continues to run.

A problem with rndctl(8) is fixed -- datastructures with
pointers in arrays are no longer passed to userspace (this
was not a security problem, but rather a major issue for
compat32). A new kernel will require a new rndctl.

The sysctl kern.arandom() and kern.urandom() nodes are hooked
up to the new generators, but the /dev/*random pseudodevices
are not, yet.

Manual pages for the new kernel interfaces are forthcoming.

branches: 1.32.12;
Use a PMF shutdown hook instead of shutdownhook_establish(). This ought
to stop the crash during shutdown that yamt@ reports in kern/41182.

branches: 1.31.2; 1.31.8;
split device/softc for sd (tested with a USB stick)

branches: 1.30.2; 1.30.4; 1.30.6;
Remove clause 3 and 4 from TNF licenses

branches: 1.29.28; 1.29.30; 1.29.32;
Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements

branches: 1.28.30; 1.28.32;
merge ktrace-lwp.

- change the way to specify a bufq strategy. (by string rather than by number)
- rather than embedding bufq_state in driver softc,
have a pointer to the former.
- move bufq related functions from kern/subr_disk.c to kern/subr_bufq.c.
- rename method to strategy for consistency.
- move some definitions which don't need to be exposed to the rest of kernel
from sys/bufq.h to sys/bufq_impl.h.
(is it better to move it to kern/ or somewhere?)
- fix some obvious breakage in dev/qbus/ts.c. (not tested)

branches: 1.26.6;
Backing out changes to clean up scsipi. I was pointed out there were
problems i hadn't seen. To prevent lossage i'd decided to back off all
changes and let them be reviewed on tech-kern.

Part of the cleanup of sys/scsipi's use of types; rename all u_int* to
uint* and change the u_long's to uint32_t's where possible. Note that the
iocl definitions/hooks have to be ulong (or u_long) or they'll bomb out.

branches: 1.24.4;
Don't need the lock in the softc anymore.

Do not manipulate xs->bp in "generic" code -- do it only in the psw_done
routine. As part of this, pass down our pre-parsed error code -- though this
interface will probably change later to accomodate better error handling.

Improve handling of memory shortage, to fix problems like:
sd3(mpt0:0:1:0): unable to allocate scsipi_xfer
sd3: not queued, error 12
Havard Eidnes's analysis of this problem is that the scsipi_xfer pool is
competing for resources with other pools, including the the inode and vnode
pools which can grow quite large.

*_scsipi_cmd(): don't biodone the buffer if scsipi_make_xs() fails, let the
caller deal with the problem
start function of block devices drivers: dequeue the buffer after the
scsipi_command() call. If scsipi_command() fails with ENOMEM don't dequeue
the buffer, and schedule a callout to call the start function after
some delay.
scsipi_init(): prime the scsipi_xfer_pool with one page. This ensure that
there is always some scsipi_xfer to play with. If scsipi_command() fails
because of pool_get(), we're sure there will be resources available later,
when the pending commands have completed.

Reviewed by Jason Thorpe and Havard Eidnes.
Todo: remove the "unable to allocate scsipi_xfer" and "not queued, error %d"
printfs, but I choose to keep them for now, to help make sure the code does
what it should.

branches: 1.21.2;
parameter with two es. From Peter Postma.

Merge the geometry and cache handling code for all direct access and optical
devices, as it's general to all SCSI MMC devices. In the process, remove
PQUIRK_NO_FLEX_PAGE.

branches: 1.19.2;
Check RAW_PART against the media size instead of the disklabel.
Add the media size in 512-byte sectors to the softc, to avoid
some 64 bit computations. Bump the capacity stored in softcs
for disks to 64 bits.

Add multiple inclusion protection.

Implement a new device buffer queue interface.
One basic struct, a function to setup a queue with a specific strategy and
three macros to put buf's into the queue, get and remove the next buf or
get the next buf without removal.

The BUFQ_XXX interface will be removed in the future.
The B_ORDERED flag is not longer supported.

Approved by: Jason R. Thorpe <thorpej@wasabisystems.com>

branches: 1.16.8;
Add ioctls to get (DIOCGCACHE) and set (DIOCSCACHE) cache enables
on disks in a generic way. Implement these ioctls for SCSI disks.

This is not fully fleshed-out yet, but it allows people to experiment
with disk caches more easily.

branches: 1.15.2;
fix typo in previous revision so that the scsi code compiles again.

Make SDRETRIES an option you can dig out of opt_scsi.h.

Add SD_IO_TIMEOUT (an option in opt_scsi.h) also, which defaults to the
normal 60 * 1000 timeout for normal read/write commamds. This allows you
to shorten or lengthen the timeout as needed.

Pull up the thorpej_scsipi branch to main branch.
This is a completely rewritten scsipi_xfer execution engine, and the
associated changes to HBA drivers. Overview of changes & features:
- All xfers are queued in the mid-layer, rather than doing so in an
ad-hoc fashion in individual adapter drivers.
- Adapter/channel resource management in the mid-layer, avoids even trying
to start running an xfer if the adapter/channel doesn't have the resources.
- Better communication between the mid-layer and the adapters.
- Asynchronous event notification mechanism from adapter to mid-layer and
peripherals.
- Better peripheral queue management: freeze/thaw, sorted requeueing during
recovery, etc.
- Clean separation of peripherals, adapters, and adapter channels (no more
scsipi_link).
- Kernel thread for each scsipi_channel makes error recovery much easier
(no more dealing with interrupt context when recovering from an error).
- Mid-layer support for tagged queueing: commands can have the tag type
set explicitly, tag IDs are allocated in the mid-layer (thus eliminating
the need to use buggy tag ID allocation schemes in many adapter drivers).
- support for QUEUE FULL and CHECK CONDITION status in mid-layer; the command
will be requeued, or a REQUEST SENSE will be sent as appropriate.

Just before the merge syssrc has been tagged with thorpej_scsipi_beforemerge

branches: 1.12.6;
sd_interpret_sense() can be called from interrupt context; call scsipi_start()
with XS_CTL_ASYNC if the failed command was called with XS_CTL_ASYNC.
Add a SDF_RESTART flag to keep state, cleared in sddone().
A mounted disk can now spin down, it will propely spin up at the next access.

include "rnd.h" explicitely since the size of sd_softc depends on it
(sd_scsi.c and sd_atapi.c missed it, leading to random memory corruption)

Update for sys/buf.h/disksort_*() changes.

branches: 1.9.2;
Implement detaching SCSI and ATAPI disks.

Make `sc_ops->sdo_flush' synchronous. The flags `SDF_FLUSHING' and
`SDF_DIRTY' were never reset because `sddone' doesn't get called from
synchronous scsi commands.

branches: 1.7.6;
Assign my copyrights to TNF.

Keep track if the disk has had done i/o, making it dirty. On last close,
or at shutdown, flush the cache if the disk is still dirty and it support
cache flushing. Fixes PRs #5588 (Jim Bernard) and #5589 (Justin Gibbs).

Improve generation of default disklabels:
-store printable product ID in cd's and sd's softc, use it as "typename"
-for this, add a "destination buffer length" argument to scsipi_strvis()
-return ATAPI device type for ATAPI devices

Add a shutdown hook for SCSI-2 and higher disks that issues a SYNCHRONIZE CACHE
operation with address 0 length 0, which, according to the SCSI-2 spec, should
be interpreted as "synchronize all remaining blocks beginning at address 0".

Backout previous change, and rather, remove all protection but the one
actually currently required. Suggested by Charles M. Hannum.

Fix or add protection for mutiple inclusion.

add support for an ATAPI attachment for 'sd'.
fix 'cd' driver's NCD_SCSI bogosity (was using testing wrong macro!)
clean up in various ways:
* make common atapi_mode_{sense,select}() functions.
* put ATAPI data structures in more sensible headers, split up by
device type.
* include headers a bit more carefully.
* pass flags to attachment-specific cd functions, and use them.
* get rid of SCSI bits in scsipi_base.h's scsipi_make_xs(), move
them into the correct place in scsi_base.c.
* fix minor typo in struct name in scsipiconf.h (which was apparently
never used except in a #define later in the same file).
* use __attribute__ to force 4-byte alignment for xs command store,
so that architectures trying to bus_space_write_multi_N() (where
N > 1) that data to a controller won't lose.
* clean up a few comments in typos, and make a few #defines easier to
understand/maintain.
* rename cd_link.h to cdvar.h (via repository copy). This is exactly
what a 'var' file is supposed to be.