Implement hot removal of spares and components. From manu@.

Implement a long desired feature of automatically incorporating
a used spare into the array after a reconstruct.

Given the configuration:
Components:
/dev/wd0e: failed
/dev/wd1e: optimal
/dev/wd2e: optimal
Spares:
/dev/wd3e: spare

Running 'raidctl -F /dev/wd0e raid0' will now result in the
following configuration after a successful rebuild:
Components:
/dev/wd3e: optimal
/dev/wd1e: optimal
/dev/wd2e: optimal
No spares.

Thanks to manu@ for the development of the initial set of changes
which allowed the changes to automatically incorporate a used spare
to come to fruition. Thanks also to manu@ for useful discussions
about and additional testing of these changes.

rf_CreateDiskQueueData() no longer uses waitflag, and will always succeed.
Cleanup the error path for the (no longer needed) PR_NOWAIT cases.

Various disk queue "peek" routines were only ever used in the simulator
version of RAIDFrame. Remove them from here.

Extensive mechanical changes to the pools used in RAIDframe.

Alloclist remains not per-RAID, so initialize that pool
separately/differently than the rest.

The remainder of pools in RF_Pools_s are now per-RAID pools. Mostly
mechanical changes to functions to allocate/destroy per-RAID pools.
Needed to make raidPtr available in certain cases to be able to find
the per-RAID pools.

Extend rf_pool_init() to now populate a per-RAID wchan value that is
unique to each pool for a given RAID device.

TODO: Complete the analysis of the minimum number of items that are
required for each pool to allow IO to progress (i.e. so that a request
for pool resources can always be satisfied), and dynamically scale
minimum pool sizes based on RAID configuration.

branches: 1.26.6;
pass down b_flags B_PHYS|B_RAW|B_MEDIA_FLAGS from bio subsystem
to component I/O

fixes the xbd(4) KASSERT() triggered by raidframe, noted in PR kern/55397
by Frank Kardel

fix the function pointer and callback mess:
- callback functions return 0 and their result is not checked; make them void.
- there are two types of callbacks and they used to overload their parameters
and the callback structure; separate them into "function" and "value"
callbacks.
- make the wait function signature consistent.

branches: 1.24.56;
convert the diskqueue mutex into a kmutex

branches: 1.23.4; 1.23.6;
Rework/simplify the disk queuing code. A bunch of this was still
holdovers from the simulator and would never be seen/used in-kernel.

branches: 1.22.40; 1.22.50; 1.22.52; 1.22.56; 1.22.60;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

branches: 1.21.26;
merge ktrace-lwp.

branches: 1.20.4;
nuke trailing whitespace

The 'next' argument to rf_CreateDiskQueueData is always NULL. Since
there is no particular reason to pass an extra NULL argument, turf it,
and initialize p->next to NULL within the function.

Add a 'waitflag' argument to rf_CreateDiskQueueData() and use it to
determine if we are willing to wait for memory to come from the
diskqueuedata (dqd) and bufpool pools. Cleanup the mess related to
code calling rf_CreateDiskQueueData() with different expectations
(and/or blatent disregard) of what might happen if there were
insufficient pool resources.

CreateDiskQueueData() doesn't need to use void * for raidPtr.
RF_Raid_t is well known in RF, and there's no reason not to use it here.

branches: 1.16.8; 1.16.10;
Cleanup one more leftover 'row' variable. Cleanup function prototypes
in rf_diskqueue.h

occured -> occurred. From Peter Postma.

Nuke a bunch of unused variables:
- node_queue_cond
- quiescent_cond
- eq_cond
- desc->cond
- desc->head
- diskqueue->numWaiting

Nuke rf_print_unable_to_init_cond().
Nuke rf_TerminateDiskQueues prototype from rf_diskqueue.h.

[Having received a definite lack of strenuous objection, a small amount
of strenuous agreement, and some general agreement, this commit is
going ahead because it's now starting to block some other changes I
wish to make.]

Remove most of the support for the concept of "rows" from RAIDframe.
While the "row" interface has been exported to the world, RAIDframe
internals have really only supported a single row, even though they
have feigned support of multiple rows.

Nothing changes in configuration land -- config files still need to
specify a single row, etc. All auto-config structures remain fully
forward/backwards compatible.

The only visible difference to the average user should be a
reduction in the size of a GENERIC kernel (i386) by 4.5K. For those
of us trolling through RAIDframe kernel code, a lot of the driver
configuration code has become a LOT easier to read.

branches: 1.12.2;
constify some

As part of the effort to get RAIDframe playing nicely with LOCKDEBUG,
rework how completed requests are handled. In particular, instead of
doing all sorts of work and locking in interrupt context, completed
requests are now queued. A new kernel thread (rf_RaidIOThread) now
handles calling rf_DiskIOComplete() and (req->CompleteFunc)() for each
completed request. There is still work to be done to make RAIDframe
LOCKDEBUG friendly, but this change is a huge step forward.

Reviewed by (and many thanks to): thorpej

Nuke unlockingOp -- it was only used for the userland version of the
code.

Remove a couple of unneeded structure members.

'buf2' is unused. Nuke it.

- AuxFunc is no longer used.
- cleanup function prototypes in rf_diskqueue.h

branches: 1.6.10;
Step 2 of the disentanglement. We now look to <dev/raidframe/*> for
the stuff that used to live in rf_types.h, rf_raidframe.h, rf_layout.h,
rf_netbsd.h, rf_raid.h, rf_decluster,h, and a few other places.
Believe it or not, when this is all done, things will be cleaner.

No functional changes to RAIDframe.

branches: 1.5.6; 1.5.8; 1.5.10;
Get recent changes into the tree:
- make component_label variables more consistent (==> clabel)
- re-work incorrect component configuration code
- re-work disk configuration code
- cleanup initial configuration of raidPtr info
- add auto-detection of components and RAID sets (Disabled, for now)
- allow / on RAID sets (Disabled, for now)
- rename "config_disk_queue" to "rf_ConfigureDiskQueue" and properly prototype
in rf_diskqueue.h
- protect some headers with #if _KERNEL (XXX this needs to be fixed properly)
and cleanup header formatting.
- expand the component labels (yes, they should be backward/forward compatible)
- other bits and pieces (some function names are still bogus, and will get
changed soon)

Make raidPtr a 'RF_Raid_t *' like it's supposed to be.

branches: 1.3.8;
Phase 2 of the RAIDframe cleanup. The source is now closer to KNF
and is much easier to read. No functionality changes.

RAIDframe cleanup, phase 1. Nuke simulator support, user-land driver,
out-dated comments, and other unneeded stuff. This helps prepare
for cleaning up the rest of the code, and adding new functionality.

No functional changes to the kernel code in this commit.

RAIDframe, version 1.1, from the Parallel Data Laboratory at
Carnegie Mellon University. Full RAID implementation, including
levels 0, 1, 4, 5, 6, parity logging, and a few other goodies.
Ported to NetBSD by Greg Oster.

rf_CreateDiskQueueData() no longer uses waitflag, and will always succeed.
Cleanup the error path for the (no longer needed) PR_NOWAIT cases.

Various disk queue "peek" routines were only ever used in the simulator
version of RAIDFrame. Remove them from here.

Extensive mechanical changes to the pools used in RAIDframe.

Alloclist remains not per-RAID, so initialize that pool
separately/differently than the rest.

The remainder of pools in RF_Pools_s are now per-RAID pools. Mostly
mechanical changes to functions to allocate/destroy per-RAID pools.
Needed to make raidPtr available in certain cases to be able to find
the per-RAID pools.

Extend rf_pool_init() to now populate a per-RAID wchan value that is
unique to each pool for a given RAID device.

TODO: Complete the analysis of the minimum number of items that are
required for each pool to allow IO to progress (i.e. so that a request
for pool resources can always be satisfied), and dynamically scale
minimum pool sizes based on RAID configuration.

pass down b_flags B_PHYS|B_RAW|B_MEDIA_FLAGS from bio subsystem
to component I/O

fixes the xbd(4) KASSERT() triggered by raidframe, noted in PR kern/55397
by Frank Kardel

fix the function pointer and callback mess:
- callback functions return 0 and their result is not checked; make them void.
- there are two types of callbacks and they used to overload their parameters
and the callback structure; separate them into "function" and "value"
callbacks.
- make the wait function signature consistent.

branches: 1.24.56;
convert the diskqueue mutex into a kmutex

branches: 1.23.4; 1.23.6;
Rework/simplify the disk queuing code. A bunch of this was still
holdovers from the simulator and would never be seen/used in-kernel.

branches: 1.22.40; 1.22.50; 1.22.52; 1.22.56; 1.22.60;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

branches: 1.21.26;
merge ktrace-lwp.

branches: 1.20.4;
nuke trailing whitespace

The 'next' argument to rf_CreateDiskQueueData is always NULL. Since
there is no particular reason to pass an extra NULL argument, turf it,
and initialize p->next to NULL within the function.

Add a 'waitflag' argument to rf_CreateDiskQueueData() and use it to
determine if we are willing to wait for memory to come from the
diskqueuedata (dqd) and bufpool pools. Cleanup the mess related to
code calling rf_CreateDiskQueueData() with different expectations
(and/or blatent disregard) of what might happen if there were
insufficient pool resources.

CreateDiskQueueData() doesn't need to use void * for raidPtr.
RF_Raid_t is well known in RF, and there's no reason not to use it here.

branches: 1.16.8; 1.16.10;
Cleanup one more leftover 'row' variable. Cleanup function prototypes
in rf_diskqueue.h

occured -> occurred. From Peter Postma.

Nuke a bunch of unused variables:
- node_queue_cond
- quiescent_cond
- eq_cond
- desc->cond
- desc->head
- diskqueue->numWaiting

Nuke rf_print_unable_to_init_cond().
Nuke rf_TerminateDiskQueues prototype from rf_diskqueue.h.

[Having received a definite lack of strenuous objection, a small amount
of strenuous agreement, and some general agreement, this commit is
going ahead because it's now starting to block some other changes I
wish to make.]

Remove most of the support for the concept of "rows" from RAIDframe.
While the "row" interface has been exported to the world, RAIDframe
internals have really only supported a single row, even though they
have feigned support of multiple rows.

Nothing changes in configuration land -- config files still need to
specify a single row, etc. All auto-config structures remain fully
forward/backwards compatible.

The only visible difference to the average user should be a
reduction in the size of a GENERIC kernel (i386) by 4.5K. For those
of us trolling through RAIDframe kernel code, a lot of the driver
configuration code has become a LOT easier to read.

branches: 1.12.2;
constify some

As part of the effort to get RAIDframe playing nicely with LOCKDEBUG,
rework how completed requests are handled. In particular, instead of
doing all sorts of work and locking in interrupt context, completed
requests are now queued. A new kernel thread (rf_RaidIOThread) now
handles calling rf_DiskIOComplete() and (req->CompleteFunc)() for each
completed request. There is still work to be done to make RAIDframe
LOCKDEBUG friendly, but this change is a huge step forward.

Reviewed by (and many thanks to): thorpej

Nuke unlockingOp -- it was only used for the userland version of the
code.

Remove a couple of unneeded structure members.

'buf2' is unused. Nuke it.

- AuxFunc is no longer used.
- cleanup function prototypes in rf_diskqueue.h

branches: 1.6.10;
Step 2 of the disentanglement. We now look to <dev/raidframe/*> for
the stuff that used to live in rf_types.h, rf_raidframe.h, rf_layout.h,
rf_netbsd.h, rf_raid.h, rf_decluster,h, and a few other places.
Believe it or not, when this is all done, things will be cleaner.

No functional changes to RAIDframe.

branches: 1.5.6; 1.5.8; 1.5.10;
Get recent changes into the tree:
- make component_label variables more consistent (==> clabel)
- re-work incorrect component configuration code
- re-work disk configuration code
- cleanup initial configuration of raidPtr info
- add auto-detection of components and RAID sets (Disabled, for now)
- allow / on RAID sets (Disabled, for now)
- rename "config_disk_queue" to "rf_ConfigureDiskQueue" and properly prototype
in rf_diskqueue.h
- protect some headers with #if _KERNEL (XXX this needs to be fixed properly)
and cleanup header formatting.
- expand the component labels (yes, they should be backward/forward compatible)
- other bits and pieces (some function names are still bogus, and will get
changed soon)

Make raidPtr a 'RF_Raid_t *' like it's supposed to be.

branches: 1.3.8;
Phase 2 of the RAIDframe cleanup. The source is now closer to KNF
and is much easier to read. No functionality changes.

RAIDframe cleanup, phase 1. Nuke simulator support, user-land driver,
out-dated comments, and other unneeded stuff. This helps prepare
for cleaning up the rest of the code, and adding new functionality.

No functional changes to the kernel code in this commit.

RAIDframe, version 1.1, from the Parallel Data Laboratory at
Carnegie Mellon University. Full RAID implementation, including
levels 0, 1, 4, 5, 6, parity logging, and a few other goodies.
Ported to NetBSD by Greg Oster.

Various disk queue "peek" routines were only ever used in the simulator
version of RAIDFrame. Remove them from here.

Extensive mechanical changes to the pools used in RAIDframe.

Alloclist remains not per-RAID, so initialize that pool
separately/differently than the rest.

The remainder of pools in RF_Pools_s are now per-RAID pools. Mostly
mechanical changes to functions to allocate/destroy per-RAID pools.
Needed to make raidPtr available in certain cases to be able to find
the per-RAID pools.

Extend rf_pool_init() to now populate a per-RAID wchan value that is
unique to each pool for a given RAID device.

TODO: Complete the analysis of the minimum number of items that are
required for each pool to allow IO to progress (i.e. so that a request
for pool resources can always be satisfied), and dynamically scale
minimum pool sizes based on RAID configuration.

pass down b_flags B_PHYS|B_RAW|B_MEDIA_FLAGS from bio subsystem
to component I/O

fixes the xbd(4) KASSERT() triggered by raidframe, noted in PR kern/55397
by Frank Kardel

fix the function pointer and callback mess:
- callback functions return 0 and their result is not checked; make them void.
- there are two types of callbacks and they used to overload their parameters
and the callback structure; separate them into "function" and "value"
callbacks.
- make the wait function signature consistent.

branches: 1.24.56;
convert the diskqueue mutex into a kmutex

branches: 1.23.4; 1.23.6;
Rework/simplify the disk queuing code. A bunch of this was still
holdovers from the simulator and would never be seen/used in-kernel.

branches: 1.22.40; 1.22.50; 1.22.52; 1.22.56; 1.22.60;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

branches: 1.21.26;
merge ktrace-lwp.

branches: 1.20.4;
nuke trailing whitespace

The 'next' argument to rf_CreateDiskQueueData is always NULL. Since
there is no particular reason to pass an extra NULL argument, turf it,
and initialize p->next to NULL within the function.

Add a 'waitflag' argument to rf_CreateDiskQueueData() and use it to
determine if we are willing to wait for memory to come from the
diskqueuedata (dqd) and bufpool pools. Cleanup the mess related to
code calling rf_CreateDiskQueueData() with different expectations
(and/or blatent disregard) of what might happen if there were
insufficient pool resources.

CreateDiskQueueData() doesn't need to use void * for raidPtr.
RF_Raid_t is well known in RF, and there's no reason not to use it here.

branches: 1.16.8; 1.16.10;
Cleanup one more leftover 'row' variable. Cleanup function prototypes
in rf_diskqueue.h

occured -> occurred. From Peter Postma.

Nuke a bunch of unused variables:
- node_queue_cond
- quiescent_cond
- eq_cond
- desc->cond
- desc->head
- diskqueue->numWaiting

Nuke rf_print_unable_to_init_cond().
Nuke rf_TerminateDiskQueues prototype from rf_diskqueue.h.

[Having received a definite lack of strenuous objection, a small amount
of strenuous agreement, and some general agreement, this commit is
going ahead because it's now starting to block some other changes I
wish to make.]

Remove most of the support for the concept of "rows" from RAIDframe.
While the "row" interface has been exported to the world, RAIDframe
internals have really only supported a single row, even though they
have feigned support of multiple rows.

Nothing changes in configuration land -- config files still need to
specify a single row, etc. All auto-config structures remain fully
forward/backwards compatible.

The only visible difference to the average user should be a
reduction in the size of a GENERIC kernel (i386) by 4.5K. For those
of us trolling through RAIDframe kernel code, a lot of the driver
configuration code has become a LOT easier to read.

branches: 1.12.2;
constify some

As part of the effort to get RAIDframe playing nicely with LOCKDEBUG,
rework how completed requests are handled. In particular, instead of
doing all sorts of work and locking in interrupt context, completed
requests are now queued. A new kernel thread (rf_RaidIOThread) now
handles calling rf_DiskIOComplete() and (req->CompleteFunc)() for each
completed request. There is still work to be done to make RAIDframe
LOCKDEBUG friendly, but this change is a huge step forward.

Reviewed by (and many thanks to): thorpej

Nuke unlockingOp -- it was only used for the userland version of the
code.

Remove a couple of unneeded structure members.

'buf2' is unused. Nuke it.

- AuxFunc is no longer used.
- cleanup function prototypes in rf_diskqueue.h

branches: 1.6.10;
Step 2 of the disentanglement. We now look to <dev/raidframe/*> for
the stuff that used to live in rf_types.h, rf_raidframe.h, rf_layout.h,
rf_netbsd.h, rf_raid.h, rf_decluster,h, and a few other places.
Believe it or not, when this is all done, things will be cleaner.

No functional changes to RAIDframe.

branches: 1.5.6; 1.5.8; 1.5.10;
Get recent changes into the tree:
- make component_label variables more consistent (==> clabel)
- re-work incorrect component configuration code
- re-work disk configuration code
- cleanup initial configuration of raidPtr info
- add auto-detection of components and RAID sets (Disabled, for now)
- allow / on RAID sets (Disabled, for now)
- rename "config_disk_queue" to "rf_ConfigureDiskQueue" and properly prototype
in rf_diskqueue.h
- protect some headers with #if _KERNEL (XXX this needs to be fixed properly)
and cleanup header formatting.
- expand the component labels (yes, they should be backward/forward compatible)
- other bits and pieces (some function names are still bogus, and will get
changed soon)

Make raidPtr a 'RF_Raid_t *' like it's supposed to be.

branches: 1.3.8;
Phase 2 of the RAIDframe cleanup. The source is now closer to KNF
and is much easier to read. No functionality changes.

RAIDframe cleanup, phase 1. Nuke simulator support, user-land driver,
out-dated comments, and other unneeded stuff. This helps prepare
for cleaning up the rest of the code, and adding new functionality.

No functional changes to the kernel code in this commit.

RAIDframe, version 1.1, from the Parallel Data Laboratory at
Carnegie Mellon University. Full RAID implementation, including
levels 0, 1, 4, 5, 6, parity logging, and a few other goodies.
Ported to NetBSD by Greg Oster.

Extensive mechanical changes to the pools used in RAIDframe.

Alloclist remains not per-RAID, so initialize that pool
separately/differently than the rest.

The remainder of pools in RF_Pools_s are now per-RAID pools. Mostly
mechanical changes to functions to allocate/destroy per-RAID pools.
Needed to make raidPtr available in certain cases to be able to find
the per-RAID pools.

Extend rf_pool_init() to now populate a per-RAID wchan value that is
unique to each pool for a given RAID device.

TODO: Complete the analysis of the minimum number of items that are
required for each pool to allow IO to progress (i.e. so that a request
for pool resources can always be satisfied), and dynamically scale
minimum pool sizes based on RAID configuration.

pass down b_flags B_PHYS|B_RAW|B_MEDIA_FLAGS from bio subsystem
to component I/O

fixes the xbd(4) KASSERT() triggered by raidframe, noted in PR kern/55397
by Frank Kardel

fix the function pointer and callback mess:
- callback functions return 0 and their result is not checked; make them void.
- there are two types of callbacks and they used to overload their parameters
and the callback structure; separate them into "function" and "value"
callbacks.
- make the wait function signature consistent.

branches: 1.24.56;
convert the diskqueue mutex into a kmutex

branches: 1.23.4; 1.23.6;
Rework/simplify the disk queuing code. A bunch of this was still
holdovers from the simulator and would never be seen/used in-kernel.

branches: 1.22.40; 1.22.50; 1.22.52; 1.22.56; 1.22.60;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

branches: 1.21.26;
merge ktrace-lwp.

branches: 1.20.4;
nuke trailing whitespace

The 'next' argument to rf_CreateDiskQueueData is always NULL. Since
there is no particular reason to pass an extra NULL argument, turf it,
and initialize p->next to NULL within the function.

Add a 'waitflag' argument to rf_CreateDiskQueueData() and use it to
determine if we are willing to wait for memory to come from the
diskqueuedata (dqd) and bufpool pools. Cleanup the mess related to
code calling rf_CreateDiskQueueData() with different expectations
(and/or blatent disregard) of what might happen if there were
insufficient pool resources.

CreateDiskQueueData() doesn't need to use void * for raidPtr.
RF_Raid_t is well known in RF, and there's no reason not to use it here.

branches: 1.16.8; 1.16.10;
Cleanup one more leftover 'row' variable. Cleanup function prototypes
in rf_diskqueue.h

occured -> occurred. From Peter Postma.

Nuke a bunch of unused variables:
- node_queue_cond
- quiescent_cond
- eq_cond
- desc->cond
- desc->head
- diskqueue->numWaiting

Nuke rf_print_unable_to_init_cond().
Nuke rf_TerminateDiskQueues prototype from rf_diskqueue.h.

[Having received a definite lack of strenuous objection, a small amount
of strenuous agreement, and some general agreement, this commit is
going ahead because it's now starting to block some other changes I
wish to make.]

Remove most of the support for the concept of "rows" from RAIDframe.
While the "row" interface has been exported to the world, RAIDframe
internals have really only supported a single row, even though they
have feigned support of multiple rows.

Nothing changes in configuration land -- config files still need to
specify a single row, etc. All auto-config structures remain fully
forward/backwards compatible.

The only visible difference to the average user should be a
reduction in the size of a GENERIC kernel (i386) by 4.5K. For those
of us trolling through RAIDframe kernel code, a lot of the driver
configuration code has become a LOT easier to read.

branches: 1.12.2;
constify some

As part of the effort to get RAIDframe playing nicely with LOCKDEBUG,
rework how completed requests are handled. In particular, instead of
doing all sorts of work and locking in interrupt context, completed
requests are now queued. A new kernel thread (rf_RaidIOThread) now
handles calling rf_DiskIOComplete() and (req->CompleteFunc)() for each
completed request. There is still work to be done to make RAIDframe
LOCKDEBUG friendly, but this change is a huge step forward.

Reviewed by (and many thanks to): thorpej

Nuke unlockingOp -- it was only used for the userland version of the
code.

Remove a couple of unneeded structure members.

'buf2' is unused. Nuke it.

- AuxFunc is no longer used.
- cleanup function prototypes in rf_diskqueue.h

branches: 1.6.10;
Step 2 of the disentanglement. We now look to <dev/raidframe/*> for
the stuff that used to live in rf_types.h, rf_raidframe.h, rf_layout.h,
rf_netbsd.h, rf_raid.h, rf_decluster,h, and a few other places.
Believe it or not, when this is all done, things will be cleaner.

No functional changes to RAIDframe.

branches: 1.5.6; 1.5.8; 1.5.10;
Get recent changes into the tree:
- make component_label variables more consistent (==> clabel)
- re-work incorrect component configuration code
- re-work disk configuration code
- cleanup initial configuration of raidPtr info
- add auto-detection of components and RAID sets (Disabled, for now)
- allow / on RAID sets (Disabled, for now)
- rename "config_disk_queue" to "rf_ConfigureDiskQueue" and properly prototype
in rf_diskqueue.h
- protect some headers with #if _KERNEL (XXX this needs to be fixed properly)
and cleanup header formatting.
- expand the component labels (yes, they should be backward/forward compatible)
- other bits and pieces (some function names are still bogus, and will get
changed soon)

Make raidPtr a 'RF_Raid_t *' like it's supposed to be.

branches: 1.3.8;
Phase 2 of the RAIDframe cleanup. The source is now closer to KNF
and is much easier to read. No functionality changes.

RAIDframe cleanup, phase 1. Nuke simulator support, user-land driver,
out-dated comments, and other unneeded stuff. This helps prepare
for cleaning up the rest of the code, and adding new functionality.

No functional changes to the kernel code in this commit.

RAIDframe, version 1.1, from the Parallel Data Laboratory at
Carnegie Mellon University. Full RAID implementation, including
levels 0, 1, 4, 5, 6, parity logging, and a few other goodies.
Ported to NetBSD by Greg Oster.

pass down b_flags B_PHYS|B_RAW|B_MEDIA_FLAGS from bio subsystem
to component I/O

fixes the xbd(4) KASSERT() triggered by raidframe, noted in PR kern/55397
by Frank Kardel

fix the function pointer and callback mess:
- callback functions return 0 and their result is not checked; make them void.
- there are two types of callbacks and they used to overload their parameters
and the callback structure; separate them into "function" and "value"
callbacks.
- make the wait function signature consistent.

branches: 1.24.56;
convert the diskqueue mutex into a kmutex

branches: 1.23.4; 1.23.6;
Rework/simplify the disk queuing code. A bunch of this was still
holdovers from the simulator and would never be seen/used in-kernel.

branches: 1.22.40; 1.22.50; 1.22.52; 1.22.56; 1.22.60;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

branches: 1.21.26;
merge ktrace-lwp.

branches: 1.20.4;
nuke trailing whitespace

The 'next' argument to rf_CreateDiskQueueData is always NULL. Since
there is no particular reason to pass an extra NULL argument, turf it,
and initialize p->next to NULL within the function.

Add a 'waitflag' argument to rf_CreateDiskQueueData() and use it to
determine if we are willing to wait for memory to come from the
diskqueuedata (dqd) and bufpool pools. Cleanup the mess related to
code calling rf_CreateDiskQueueData() with different expectations
(and/or blatent disregard) of what might happen if there were
insufficient pool resources.

CreateDiskQueueData() doesn't need to use void * for raidPtr.
RF_Raid_t is well known in RF, and there's no reason not to use it here.

branches: 1.16.8; 1.16.10;
Cleanup one more leftover 'row' variable. Cleanup function prototypes
in rf_diskqueue.h

occured -> occurred. From Peter Postma.

Nuke a bunch of unused variables:
- node_queue_cond
- quiescent_cond
- eq_cond
- desc->cond
- desc->head
- diskqueue->numWaiting

Nuke rf_print_unable_to_init_cond().
Nuke rf_TerminateDiskQueues prototype from rf_diskqueue.h.

[Having received a definite lack of strenuous objection, a small amount
of strenuous agreement, and some general agreement, this commit is
going ahead because it's now starting to block some other changes I
wish to make.]

Remove most of the support for the concept of "rows" from RAIDframe.
While the "row" interface has been exported to the world, RAIDframe
internals have really only supported a single row, even though they
have feigned support of multiple rows.

Nothing changes in configuration land -- config files still need to
specify a single row, etc. All auto-config structures remain fully
forward/backwards compatible.

The only visible difference to the average user should be a
reduction in the size of a GENERIC kernel (i386) by 4.5K. For those
of us trolling through RAIDframe kernel code, a lot of the driver
configuration code has become a LOT easier to read.

branches: 1.12.2;
constify some

As part of the effort to get RAIDframe playing nicely with LOCKDEBUG,
rework how completed requests are handled. In particular, instead of
doing all sorts of work and locking in interrupt context, completed
requests are now queued. A new kernel thread (rf_RaidIOThread) now
handles calling rf_DiskIOComplete() and (req->CompleteFunc)() for each
completed request. There is still work to be done to make RAIDframe
LOCKDEBUG friendly, but this change is a huge step forward.

Reviewed by (and many thanks to): thorpej

Nuke unlockingOp -- it was only used for the userland version of the
code.

Remove a couple of unneeded structure members.

'buf2' is unused. Nuke it.

- AuxFunc is no longer used.
- cleanup function prototypes in rf_diskqueue.h

branches: 1.6.10;
Step 2 of the disentanglement. We now look to <dev/raidframe/*> for
the stuff that used to live in rf_types.h, rf_raidframe.h, rf_layout.h,
rf_netbsd.h, rf_raid.h, rf_decluster,h, and a few other places.
Believe it or not, when this is all done, things will be cleaner.

No functional changes to RAIDframe.

branches: 1.5.6; 1.5.8; 1.5.10;
Get recent changes into the tree:
- make component_label variables more consistent (==> clabel)
- re-work incorrect component configuration code
- re-work disk configuration code
- cleanup initial configuration of raidPtr info
- add auto-detection of components and RAID sets (Disabled, for now)
- allow / on RAID sets (Disabled, for now)
- rename "config_disk_queue" to "rf_ConfigureDiskQueue" and properly prototype
in rf_diskqueue.h
- protect some headers with #if _KERNEL (XXX this needs to be fixed properly)
and cleanup header formatting.
- expand the component labels (yes, they should be backward/forward compatible)
- other bits and pieces (some function names are still bogus, and will get
changed soon)

Make raidPtr a 'RF_Raid_t *' like it's supposed to be.

branches: 1.3.8;
Phase 2 of the RAIDframe cleanup. The source is now closer to KNF
and is much easier to read. No functionality changes.

RAIDframe cleanup, phase 1. Nuke simulator support, user-land driver,
out-dated comments, and other unneeded stuff. This helps prepare
for cleaning up the rest of the code, and adding new functionality.

No functional changes to the kernel code in this commit.

RAIDframe, version 1.1, from the Parallel Data Laboratory at
Carnegie Mellon University. Full RAID implementation, including
levels 0, 1, 4, 5, 6, parity logging, and a few other goodies.
Ported to NetBSD by Greg Oster.

fix the function pointer and callback mess:
- callback functions return 0 and their result is not checked; make them void.
- there are two types of callbacks and they used to overload their parameters
and the callback structure; separate them into "function" and "value"
callbacks.
- make the wait function signature consistent.

convert the diskqueue mutex into a kmutex

branches: 1.23.4; 1.23.6;
Rework/simplify the disk queuing code. A bunch of this was still
holdovers from the simulator and would never be seen/used in-kernel.

branches: 1.22.40; 1.22.50; 1.22.52; 1.22.56; 1.22.60;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

branches: 1.21.26;
merge ktrace-lwp.

branches: 1.20.4;
nuke trailing whitespace

The 'next' argument to rf_CreateDiskQueueData is always NULL. Since
there is no particular reason to pass an extra NULL argument, turf it,
and initialize p->next to NULL within the function.

Add a 'waitflag' argument to rf_CreateDiskQueueData() and use it to
determine if we are willing to wait for memory to come from the
diskqueuedata (dqd) and bufpool pools. Cleanup the mess related to
code calling rf_CreateDiskQueueData() with different expectations
(and/or blatent disregard) of what might happen if there were
insufficient pool resources.

CreateDiskQueueData() doesn't need to use void * for raidPtr.
RF_Raid_t is well known in RF, and there's no reason not to use it here.

branches: 1.16.8; 1.16.10;
Cleanup one more leftover 'row' variable. Cleanup function prototypes
in rf_diskqueue.h

occured -> occurred. From Peter Postma.

Nuke a bunch of unused variables:
- node_queue_cond
- quiescent_cond
- eq_cond
- desc->cond
- desc->head
- diskqueue->numWaiting

Nuke rf_print_unable_to_init_cond().
Nuke rf_TerminateDiskQueues prototype from rf_diskqueue.h.

[Having received a definite lack of strenuous objection, a small amount
of strenuous agreement, and some general agreement, this commit is
going ahead because it's now starting to block some other changes I
wish to make.]

Remove most of the support for the concept of "rows" from RAIDframe.
While the "row" interface has been exported to the world, RAIDframe
internals have really only supported a single row, even though they
have feigned support of multiple rows.

Nothing changes in configuration land -- config files still need to
specify a single row, etc. All auto-config structures remain fully
forward/backwards compatible.

The only visible difference to the average user should be a
reduction in the size of a GENERIC kernel (i386) by 4.5K. For those
of us trolling through RAIDframe kernel code, a lot of the driver
configuration code has become a LOT easier to read.

branches: 1.12.2;
constify some

As part of the effort to get RAIDframe playing nicely with LOCKDEBUG,
rework how completed requests are handled. In particular, instead of
doing all sorts of work and locking in interrupt context, completed
requests are now queued. A new kernel thread (rf_RaidIOThread) now
handles calling rf_DiskIOComplete() and (req->CompleteFunc)() for each
completed request. There is still work to be done to make RAIDframe
LOCKDEBUG friendly, but this change is a huge step forward.

Reviewed by (and many thanks to): thorpej

Nuke unlockingOp -- it was only used for the userland version of the
code.

Remove a couple of unneeded structure members.

'buf2' is unused. Nuke it.

- AuxFunc is no longer used.
- cleanup function prototypes in rf_diskqueue.h

branches: 1.6.10;
Step 2 of the disentanglement. We now look to <dev/raidframe/*> for
the stuff that used to live in rf_types.h, rf_raidframe.h, rf_layout.h,
rf_netbsd.h, rf_raid.h, rf_decluster,h, and a few other places.
Believe it or not, when this is all done, things will be cleaner.

No functional changes to RAIDframe.

branches: 1.5.6; 1.5.8; 1.5.10;
Get recent changes into the tree:
- make component_label variables more consistent (==> clabel)
- re-work incorrect component configuration code
- re-work disk configuration code
- cleanup initial configuration of raidPtr info
- add auto-detection of components and RAID sets (Disabled, for now)
- allow / on RAID sets (Disabled, for now)
- rename "config_disk_queue" to "rf_ConfigureDiskQueue" and properly prototype
in rf_diskqueue.h
- protect some headers with #if _KERNEL (XXX this needs to be fixed properly)
and cleanup header formatting.
- expand the component labels (yes, they should be backward/forward compatible)
- other bits and pieces (some function names are still bogus, and will get
changed soon)

Make raidPtr a 'RF_Raid_t *' like it's supposed to be.

branches: 1.3.8;
Phase 2 of the RAIDframe cleanup. The source is now closer to KNF
and is much easier to read. No functionality changes.

RAIDframe cleanup, phase 1. Nuke simulator support, user-land driver,
out-dated comments, and other unneeded stuff. This helps prepare
for cleaning up the rest of the code, and adding new functionality.

No functional changes to the kernel code in this commit.

RAIDframe, version 1.1, from the Parallel Data Laboratory at
Carnegie Mellon University. Full RAID implementation, including
levels 0, 1, 4, 5, 6, parity logging, and a few other goodies.
Ported to NetBSD by Greg Oster.