Implement hot removal of spares and components. From manu@.

Implement a long desired feature of automatically incorporating
a used spare into the array after a reconstruct.

Given the configuration:
Components:
/dev/wd0e: failed
/dev/wd1e: optimal
/dev/wd2e: optimal
Spares:
/dev/wd3e: spare

Running 'raidctl -F /dev/wd0e raid0' will now result in the
following configuration after a successful rebuild:
Components:
/dev/wd3e: optimal
/dev/wd1e: optimal
/dev/wd2e: optimal
No spares.

Thanks to manu@ for the development of the initial set of changes
which allowed the changes to automatically incorporate a used spare
to come to fruition. Thanks also to manu@ for useful discussions
about and additional testing of these changes.

call buf_destroy() after buf_init(). hopefully fixes the lockdebug
problem shown here:

http://mail-index.netbsd.org/tech-kern/2021/12/10/msg027851.html

but seems unlikely to fix the original problem.

tested on i386/anita. ok oster@

XXX: pullup-9 (netbsd-8 uses old APIs.)

rf_CreateDiskQueueData() no longer uses waitflag, and will always succeed.
Cleanup the error path for the (no longer needed) PR_NOWAIT cases.

Various disk queue "peek" routines were only ever used in the simulator
version of RAIDFrame. Remove them from here.

Extensive mechanical changes to the pools used in RAIDframe.

Alloclist remains not per-RAID, so initialize that pool
separately/differently than the rest.

The remainder of pools in RF_Pools_s are now per-RAID pools. Mostly
mechanical changes to functions to allocate/destroy per-RAID pools.
Needed to make raidPtr available in certain cases to be able to find
the per-RAID pools.

Extend rf_pool_init() to now populate a per-RAID wchan value that is
unique to each pool for a given RAID device.

TODO: Complete the analysis of the minimum number of items that are
required for each pool to allow IO to progress (i.e. so that a request
for pool resources can always be satisfied), and dynamically scale
minimum pool sizes based on RAID configuration.

getiobuf() can return NULL if there are no IO buffers available.
RAIDframe can't deal with that, so create a dedicated pool of buffers
to use for IO. PR_WAITOK is fine here, as we pre-allocate more than
we need to guarantee IO can make progress. Tuning of pool still to
come.

branches: 1.58.6;
remove unnnecessary splbio() in rf_FreeDiskQueueData()

pass down b_flags B_PHYS|B_RAW|B_MEDIA_FLAGS from bio subsystem
to component I/O

fixes the xbd(4) KASSERT() triggered by raidframe, noted in PR kern/55397
by Frank Kardel

fix the function pointer and callback mess:
- callback functions return 0 and their result is not checked; make them void.
- there are two types of callbacks and they used to overload their parameters
and the callback structure; separate them into "function" and "value"
callbacks.
- make the wait function signature consistent.

branches: 1.55.4;
Introduce PR_ZERO to avoid open-coding memset()s everywhere. OK riastradh@.

- Change the allocation macros to be more like function calls
- Change sizeof(type) -> sizeof(*variable)
- Use macros for the long buffer length allocations
- Remove "bit polishing" memsets() -- do them only once
- Remove unnecessary casts

Thanks to oster@ for finding bugs and testing.

branches: 1.53.56;
convert the diskqueue mutex into a kmutex

branches: 1.52.4; 1.52.6;
Rework/simplify the disk queuing code. A bunch of this was still
holdovers from the simulator and would never be seen/used in-kernel.

branches: 1.51.4; 1.51.6; 1.51.10; 1.51.14;
Mark a buffer `busy` in getnewbuf() when it came from the pool_cache since
its not on a free list.

Also change buf_init() to not automatically mark buffers `busy' since this
only makes sense for bufcache buffers.

Mark all buf_init'd buffers 'busy' on the places where they ought to be
flagged as such to not confuse the buffer cache.

Fixes PR 38923.

branches: 1.50.6; 1.50.10; 1.50.12; 1.50.14;
Merge vmlocking2 to head.

branches: 1.49.2; 1.49.16; 1.49.22; 1.49.24; 1.49.28;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

branches: 1.48.4;
__unused removal on arguments; approved by core.

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

Protect calls to pool_put/pool_get that may occur in interrupt context
with spl used to protect other allocations and frees, or datastructure
element insertion and removal, in adjacent code.

It is almost unquestionably the case that some of the spl()/splx() calls
added here are superfluous, but it really seems wrong to see:

s=splfoo();
/* frob data structure */
splx(s);
pool_put(x);

and if we think we need to protect the first operation, then it is hard
to see why we should not think we need to protect the next. "Better
safe than sorry".

It is also almost unquestionably the case that I missed some pool
gets/puts from interrupt context with my strategy for finding these
calls; use of PR_NOWAIT is a strong hint that a pool may be used from
interrupt context but many callers in the kernel pass a "can wait/can't
wait" flag down such that my searches might not have found them. One
notable area that needs to be looked at is pf.

See also:

http://mail-index.netbsd.org/tech-kern/2006/07/19/0003.html
http://mail-index.netbsd.org/tech-kern/2006/07/19/0009.html

branches: 1.45.18; 1.45.20;
Cleanup the initialization of buffers a bit, and remove some old code
that works around a bug that no longer exists. From yamt. Thanks!
(a bit more cleanup to follow)

initialize necessary members of struct buf. PR/32462 from Reinoud Zandijk.

- add simple functions to allocate/free a buffer for i/o.
- make bufpool static.

branches: 1.42.2;
merge ktrace-lwp.

branches: 1.41.2;
- avoid variable shadowing
- add a lot of const
- remove parameters from functin declarations

branches: 1.40.2;
nuke trailing whitespace

The 'next' argument to rf_CreateDiskQueueData is always NULL. Since
there is no particular reason to pass an extra NULL argument, turf it,
and initialize p->next to NULL within the function.

Add a 'waitflag' argument to rf_CreateDiskQueueData() and use it to
determine if we are willing to wait for memory to come from the
diskqueuedata (dqd) and bufpool pools. Cleanup the mess related to
code calling rf_CreateDiskQueueData() with different expectations
(and/or blatent disregard) of what might happen if there were
insufficient pool resources.

CreateDiskQueueData() doesn't need to use void * for raidPtr.
RF_Raid_t is well known in RF, and there's no reason not to use it here.

branches: 1.36.4; 1.36.6;
Only touch bufpool whilst in splbio(). (That should be the case
already, but this makes it explicit and safer in the case where
that changes for some reason.)

branches: 1.35.4;
bufpool must be accessed at splbio().

Doesn't hurt much to zero this before we start mucking with it.

- Introduce rf_pools which contains all of the various global pools used
by RAIDframe. Convert all other RAIDframe global pools to use pools
defined within this new structure.
- Introduce rf_pool_init(), used for initializing a single pool in
RAIDframe. Teach each of the configuration routines to use
rf_pool_init().
- Cleanup a few pool-related comments.
- Cleanup revent initialization and #defines.
- Add a missing pool_destroy() for the reconbuffer pool.

(Saves another 1K off of an i386 GENERIC kernel, and makes
stuff a lot more readable)

- Introduce rf_pools which contains all of the various global pools used
by RAIDframe. Convert all other RAIDframe global pools to use pools
defined within this new structure.
- Introduce rf_pool_init(), used for initializing a single pool in
RAIDframe. Teach each of the configuration routines to use
rf_pool_init().
- Cleanup a few pool-related comments.
- Cleanup revent initialization and #defines.
- Add a missing pool_destroy() for the reconbuffer pool.

(Saves another 1K off of an i386 GENERIC kernel, and makes
stuff a lot more readable)

- remove the RF_*_INC's, as necessary. They are not needed any more.
- introduce RF_MIN_*'s, as necessary. These will indicate the
low-water mark for pools as well as the pool_prime() value.
- add pool_setlowat() for the critical pools.
- pool_prime() and pool_setlowat() the raidframe_cbufpool.
- re-order some pool_prime()'s and pool_sethiwat()'s for clarity.

Adjust _rf_ShutdownCreate() so that it is willing to wait for more
memory. Since we only now ever "return(0)", just return (void)
instead.

Cleanup all uses of rf_ShutdownCreate() to not worry about
it ever failing. Shaves another 600 bytes off of an i386 GENERIC kernel.

Nuke a bunch of unused variables:
- node_queue_cond
- quiescent_cond
- eq_cond
- desc->cond
- desc->head
- diskqueue->numWaiting

Nuke rf_print_unable_to_init_cond().
Nuke rf_TerminateDiskQueues prototype from rf_diskqueue.h.

init_dqd and clean_dqd don't do enough to be their own functions,
and just serve to cause confusion. Back-merge their contents to their
(only) calling spots and nuke them.

Some days you wonder if some of the function declaration consistency
was just an accident in the first place. Cleanup function decls and
a few comments. [ok.. so I wasn't going to fix this many.. but once
you're on a roll....]

_rf_create_managed_cond() is now left doing nothing. Convert callers.
Mash DO_RAID_COND in rf_driver.c out of existance.

- Nuke (already #if 0'ed) _rf_create_managed_lkmgr_mutex() while we're
busy here.

simplify DO_INIT in rf_engine.c

_rf_create_managed_mutex() is doing just a simple:

rf_mutex_init(m)

now. The rest of the fluff is no longer needed.
It also cannot fail, so error checking on rf_create_managed_mutex()
is just wasting space.

Nuke the #define's associated with rf_create_managed_mutex().
Convert rf_create_managed_mutex(listp,m) to just rf_mutex_init(m).
Remove wasteful "error checking" and simplify all instances where this
is called. (another 0.3K saved in the binary, but the real savings
is in code readability!)

- first kick at a major reworking of RAIDframe's memory allocation code:
- all freelists converted to pools
- initialization of structure members in certain cases where
code was relying on specific allocation and usage properties
to keep structures in a "known state" (that doesn't work with
pools!).
- make most pool_get() be "PR_WAITOK" until they can be analyzed
further, and/or have proper error handling added.
- all RF_Mallocs zero the space returned, so there is no difference
between RF_Calloc and RF_Malloc. In fact, all the RF_Calloc()'s
do is tend to do is get things horribly confused.
Make RF_Malloc() the "general memory allocator", with
RF_MallocAndAdd() the "general memory allocator with
allocation list".
- some of these RF_Malloc's et al. are destined to disappear.
- remove rf_rdp_freelist entirely (it's not used anywhere!)
- remove: #include "rf_freelist.h"
- to the files that were relying on the above, add: #include "rf_general.h"
- add: #include "rf_debugMem.h" to rf_shutdown.h to make it happy
about the loss of: #include "rf_freelist.h".

This shrinks an i386 GENERIC kernel by approx 5K. RAIDframe now
weighs in at about 162K on i386.

[Having received a definite lack of strenuous objection, a small amount
of strenuous agreement, and some general agreement, this commit is
going ahead because it's now starting to block some other changes I
wish to make.]

Remove most of the support for the concept of "rows" from RAIDframe.
While the "row" interface has been exported to the world, RAIDframe
internals have really only supported a single row, even though they
have feigned support of multiple rows.

Nothing changes in configuration land -- config files still need to
specify a single row, etc. All auto-config structures remain fully
forward/backwards compatible.

The only visible difference to the average user should be a
reduction in the size of a GENERIC kernel (i386) by 4.5K. For those
of us trolling through RAIDframe kernel code, a lot of the driver
configuration code has become a LOT easier to read.

branches: 1.22.2;
constify some

Use RF_DEBUG_DISKQUEUE to eliminate more debuggin printfs that the
vast majority of the world will never see much less care about.

Nuke unlockingOp -- it was only used for the userland version of the
code.

'buf2' is unused. Nuke it.

Everyone and their dog was using RF_ERRORMSG3 to print out the same
sort of error message, over and over again, in different files.
Rather than having the same text repeated in multiple .o files,
create a couple of little functions to do the printing, and save a
bundle of space. Also improves readability of code.

- AuxFunc is no longer used.
- cleanup function prototypes in rf_diskqueue.h

It appears that nothing is using rf_CreateDiskQueueDataFull. Make it even
harder to do so by removing it.

branches: 1.15.8;
add RCSIDs

Step 2 of the disentanglement. We now look to <dev/raidframe/*> for
the stuff that used to live in rf_types.h, rf_raidframe.h, rf_layout.h,
rf_netbsd.h, rf_raid.h, rf_decluster,h, and a few other places.
Believe it or not, when this is all done, things will be cleaner.

No functional changes to RAIDframe.

branches: 1.13.6; 1.13.8; 1.13.10;
Reorganize some comments.

Garbage collect the (already disabled) "random" queue type.

Garbage collect some unused stuff. Include rf_kintf.h for a
function prototype.

- record the pointer to the DiskQueueSW so we can use that when
we hot-add a spare.
- use the above info when adding a spare.

Get recent changes into the tree:
- make component_label variables more consistent (==> clabel)
- re-work incorrect component configuration code
- re-work disk configuration code
- cleanup initial configuration of raidPtr info
- add auto-detection of components and RAID sets (Disabled, for now)
- allow / on RAID sets (Disabled, for now)
- rename "config_disk_queue" to "rf_ConfigureDiskQueue" and properly prototype
in rf_diskqueue.h
- protect some headers with #if _KERNEL (XXX this needs to be fixed properly)
and cleanup header formatting.
- expand the component labels (yes, they should be backward/forward compatible)
- other bits and pieces (some function names are still bogus, and will get
changed soon)

Nuke: #include "rf_threadid.h" and call to rf_get_threadid().

branches: 1.7.2;
At config time, initialize a small amount of space for the disk queues
corresponding to the hot spares. We'll need that space when a spare is
actually added. Huge Thanks to Martin Laubach for helping track this down.

branches: 1.6.2; 1.6.4; 1.6.6;
Phase 2 of the RAIDframe cleanup. The source is now closer to KNF
and is much easier to read. No functionality changes.

RAIDframe cleanup, phase 1. Nuke simulator support, user-land driver,
out-dated comments, and other unneeded stuff. This helps prepare
for cleaning up the rest of the code, and adding new functionality.

No functional changes to the kernel code in this commit.

Use M_RAIDFRAME.

Duh... check that malloc() hasn't returned a NULL *before* we zero the buffer,
not after. Pointed out by Michael Graff.

Fix (potential) referencing of null pointer in (verbose) debugging mode.

RAIDframe, version 1.1, from the Parallel Data Laboratory at
Carnegie Mellon University. Full RAID implementation, including
levels 0, 1, 4, 5, 6, parity logging, and a few other goodies.
Ported to NetBSD by Greg Oster.

call buf_destroy() after buf_init(). hopefully fixes the lockdebug
problem shown here:

http://mail-index.netbsd.org/tech-kern/2021/12/10/msg027851.html

but seems unlikely to fix the original problem.

tested on i386/anita. ok oster@

XXX: pullup-9 (netbsd-8 uses old APIs.)

rf_CreateDiskQueueData() no longer uses waitflag, and will always succeed.
Cleanup the error path for the (no longer needed) PR_NOWAIT cases.

Various disk queue "peek" routines were only ever used in the simulator
version of RAIDFrame. Remove them from here.

Extensive mechanical changes to the pools used in RAIDframe.

Alloclist remains not per-RAID, so initialize that pool
separately/differently than the rest.

The remainder of pools in RF_Pools_s are now per-RAID pools. Mostly
mechanical changes to functions to allocate/destroy per-RAID pools.
Needed to make raidPtr available in certain cases to be able to find
the per-RAID pools.

Extend rf_pool_init() to now populate a per-RAID wchan value that is
unique to each pool for a given RAID device.

TODO: Complete the analysis of the minimum number of items that are
required for each pool to allow IO to progress (i.e. so that a request
for pool resources can always be satisfied), and dynamically scale
minimum pool sizes based on RAID configuration.

getiobuf() can return NULL if there are no IO buffers available.
RAIDframe can't deal with that, so create a dedicated pool of buffers
to use for IO. PR_WAITOK is fine here, as we pre-allocate more than
we need to guarantee IO can make progress. Tuning of pool still to
come.

branches: 1.58.6;
remove unnnecessary splbio() in rf_FreeDiskQueueData()

pass down b_flags B_PHYS|B_RAW|B_MEDIA_FLAGS from bio subsystem
to component I/O

fixes the xbd(4) KASSERT() triggered by raidframe, noted in PR kern/55397
by Frank Kardel

fix the function pointer and callback mess:
- callback functions return 0 and their result is not checked; make them void.
- there are two types of callbacks and they used to overload their parameters
and the callback structure; separate them into "function" and "value"
callbacks.
- make the wait function signature consistent.

branches: 1.55.4;
Introduce PR_ZERO to avoid open-coding memset()s everywhere. OK riastradh@.

- Change the allocation macros to be more like function calls
- Change sizeof(type) -> sizeof(*variable)
- Use macros for the long buffer length allocations
- Remove "bit polishing" memsets() -- do them only once
- Remove unnecessary casts

Thanks to oster@ for finding bugs and testing.

branches: 1.53.56;
convert the diskqueue mutex into a kmutex

branches: 1.52.4; 1.52.6;
Rework/simplify the disk queuing code. A bunch of this was still
holdovers from the simulator and would never be seen/used in-kernel.

branches: 1.51.4; 1.51.6; 1.51.10; 1.51.14;
Mark a buffer `busy` in getnewbuf() when it came from the pool_cache since
its not on a free list.

Also change buf_init() to not automatically mark buffers `busy' since this
only makes sense for bufcache buffers.

Mark all buf_init'd buffers 'busy' on the places where they ought to be
flagged as such to not confuse the buffer cache.

Fixes PR 38923.

branches: 1.50.6; 1.50.10; 1.50.12; 1.50.14;
Merge vmlocking2 to head.

branches: 1.49.2; 1.49.16; 1.49.22; 1.49.24; 1.49.28;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

branches: 1.48.4;
__unused removal on arguments; approved by core.

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

Protect calls to pool_put/pool_get that may occur in interrupt context
with spl used to protect other allocations and frees, or datastructure
element insertion and removal, in adjacent code.

It is almost unquestionably the case that some of the spl()/splx() calls
added here are superfluous, but it really seems wrong to see:

s=splfoo();
/* frob data structure */
splx(s);
pool_put(x);

and if we think we need to protect the first operation, then it is hard
to see why we should not think we need to protect the next. "Better
safe than sorry".

It is also almost unquestionably the case that I missed some pool
gets/puts from interrupt context with my strategy for finding these
calls; use of PR_NOWAIT is a strong hint that a pool may be used from
interrupt context but many callers in the kernel pass a "can wait/can't
wait" flag down such that my searches might not have found them. One
notable area that needs to be looked at is pf.

See also:

http://mail-index.netbsd.org/tech-kern/2006/07/19/0003.html
http://mail-index.netbsd.org/tech-kern/2006/07/19/0009.html

branches: 1.45.18; 1.45.20;
Cleanup the initialization of buffers a bit, and remove some old code
that works around a bug that no longer exists. From yamt. Thanks!
(a bit more cleanup to follow)

initialize necessary members of struct buf. PR/32462 from Reinoud Zandijk.

- add simple functions to allocate/free a buffer for i/o.
- make bufpool static.

branches: 1.42.2;
merge ktrace-lwp.

branches: 1.41.2;
- avoid variable shadowing
- add a lot of const
- remove parameters from functin declarations

branches: 1.40.2;
nuke trailing whitespace

The 'next' argument to rf_CreateDiskQueueData is always NULL. Since
there is no particular reason to pass an extra NULL argument, turf it,
and initialize p->next to NULL within the function.

Add a 'waitflag' argument to rf_CreateDiskQueueData() and use it to
determine if we are willing to wait for memory to come from the
diskqueuedata (dqd) and bufpool pools. Cleanup the mess related to
code calling rf_CreateDiskQueueData() with different expectations
(and/or blatent disregard) of what might happen if there were
insufficient pool resources.

CreateDiskQueueData() doesn't need to use void * for raidPtr.
RF_Raid_t is well known in RF, and there's no reason not to use it here.

branches: 1.36.4; 1.36.6;
Only touch bufpool whilst in splbio(). (That should be the case
already, but this makes it explicit and safer in the case where
that changes for some reason.)

branches: 1.35.4;
bufpool must be accessed at splbio().

Doesn't hurt much to zero this before we start mucking with it.

- Introduce rf_pools which contains all of the various global pools used
by RAIDframe. Convert all other RAIDframe global pools to use pools
defined within this new structure.
- Introduce rf_pool_init(), used for initializing a single pool in
RAIDframe. Teach each of the configuration routines to use
rf_pool_init().
- Cleanup a few pool-related comments.
- Cleanup revent initialization and #defines.
- Add a missing pool_destroy() for the reconbuffer pool.

(Saves another 1K off of an i386 GENERIC kernel, and makes
stuff a lot more readable)

- Introduce rf_pools which contains all of the various global pools used
by RAIDframe. Convert all other RAIDframe global pools to use pools
defined within this new structure.
- Introduce rf_pool_init(), used for initializing a single pool in
RAIDframe. Teach each of the configuration routines to use
rf_pool_init().
- Cleanup a few pool-related comments.
- Cleanup revent initialization and #defines.
- Add a missing pool_destroy() for the reconbuffer pool.

(Saves another 1K off of an i386 GENERIC kernel, and makes
stuff a lot more readable)

- remove the RF_*_INC's, as necessary. They are not needed any more.
- introduce RF_MIN_*'s, as necessary. These will indicate the
low-water mark for pools as well as the pool_prime() value.
- add pool_setlowat() for the critical pools.
- pool_prime() and pool_setlowat() the raidframe_cbufpool.
- re-order some pool_prime()'s and pool_sethiwat()'s for clarity.

Adjust _rf_ShutdownCreate() so that it is willing to wait for more
memory. Since we only now ever "return(0)", just return (void)
instead.

Cleanup all uses of rf_ShutdownCreate() to not worry about
it ever failing. Shaves another 600 bytes off of an i386 GENERIC kernel.

Nuke a bunch of unused variables:
- node_queue_cond
- quiescent_cond
- eq_cond
- desc->cond
- desc->head
- diskqueue->numWaiting

Nuke rf_print_unable_to_init_cond().
Nuke rf_TerminateDiskQueues prototype from rf_diskqueue.h.

init_dqd and clean_dqd don't do enough to be their own functions,
and just serve to cause confusion. Back-merge their contents to their
(only) calling spots and nuke them.

Some days you wonder if some of the function declaration consistency
was just an accident in the first place. Cleanup function decls and
a few comments. [ok.. so I wasn't going to fix this many.. but once
you're on a roll....]

_rf_create_managed_cond() is now left doing nothing. Convert callers.
Mash DO_RAID_COND in rf_driver.c out of existance.

- Nuke (already #if 0'ed) _rf_create_managed_lkmgr_mutex() while we're
busy here.

simplify DO_INIT in rf_engine.c

_rf_create_managed_mutex() is doing just a simple:

rf_mutex_init(m)

now. The rest of the fluff is no longer needed.
It also cannot fail, so error checking on rf_create_managed_mutex()
is just wasting space.

Nuke the #define's associated with rf_create_managed_mutex().
Convert rf_create_managed_mutex(listp,m) to just rf_mutex_init(m).
Remove wasteful "error checking" and simplify all instances where this
is called. (another 0.3K saved in the binary, but the real savings
is in code readability!)

- first kick at a major reworking of RAIDframe's memory allocation code:
- all freelists converted to pools
- initialization of structure members in certain cases where
code was relying on specific allocation and usage properties
to keep structures in a "known state" (that doesn't work with
pools!).
- make most pool_get() be "PR_WAITOK" until they can be analyzed
further, and/or have proper error handling added.
- all RF_Mallocs zero the space returned, so there is no difference
between RF_Calloc and RF_Malloc. In fact, all the RF_Calloc()'s
do is tend to do is get things horribly confused.
Make RF_Malloc() the "general memory allocator", with
RF_MallocAndAdd() the "general memory allocator with
allocation list".
- some of these RF_Malloc's et al. are destined to disappear.
- remove rf_rdp_freelist entirely (it's not used anywhere!)
- remove: #include "rf_freelist.h"
- to the files that were relying on the above, add: #include "rf_general.h"
- add: #include "rf_debugMem.h" to rf_shutdown.h to make it happy
about the loss of: #include "rf_freelist.h".

This shrinks an i386 GENERIC kernel by approx 5K. RAIDframe now
weighs in at about 162K on i386.

[Having received a definite lack of strenuous objection, a small amount
of strenuous agreement, and some general agreement, this commit is
going ahead because it's now starting to block some other changes I
wish to make.]

Remove most of the support for the concept of "rows" from RAIDframe.
While the "row" interface has been exported to the world, RAIDframe
internals have really only supported a single row, even though they
have feigned support of multiple rows.

Nothing changes in configuration land -- config files still need to
specify a single row, etc. All auto-config structures remain fully
forward/backwards compatible.

The only visible difference to the average user should be a
reduction in the size of a GENERIC kernel (i386) by 4.5K. For those
of us trolling through RAIDframe kernel code, a lot of the driver
configuration code has become a LOT easier to read.

branches: 1.22.2;
constify some

Use RF_DEBUG_DISKQUEUE to eliminate more debuggin printfs that the
vast majority of the world will never see much less care about.

Nuke unlockingOp -- it was only used for the userland version of the
code.

'buf2' is unused. Nuke it.

Everyone and their dog was using RF_ERRORMSG3 to print out the same
sort of error message, over and over again, in different files.
Rather than having the same text repeated in multiple .o files,
create a couple of little functions to do the printing, and save a
bundle of space. Also improves readability of code.

- AuxFunc is no longer used.
- cleanup function prototypes in rf_diskqueue.h

It appears that nothing is using rf_CreateDiskQueueDataFull. Make it even
harder to do so by removing it.

branches: 1.15.8;
add RCSIDs

Step 2 of the disentanglement. We now look to <dev/raidframe/*> for
the stuff that used to live in rf_types.h, rf_raidframe.h, rf_layout.h,
rf_netbsd.h, rf_raid.h, rf_decluster,h, and a few other places.
Believe it or not, when this is all done, things will be cleaner.

No functional changes to RAIDframe.

branches: 1.13.6; 1.13.8; 1.13.10;
Reorganize some comments.

Garbage collect the (already disabled) "random" queue type.

Garbage collect some unused stuff. Include rf_kintf.h for a
function prototype.

- record the pointer to the DiskQueueSW so we can use that when
we hot-add a spare.
- use the above info when adding a spare.

Get recent changes into the tree:
- make component_label variables more consistent (==> clabel)
- re-work incorrect component configuration code
- re-work disk configuration code
- cleanup initial configuration of raidPtr info
- add auto-detection of components and RAID sets (Disabled, for now)
- allow / on RAID sets (Disabled, for now)
- rename "config_disk_queue" to "rf_ConfigureDiskQueue" and properly prototype
in rf_diskqueue.h
- protect some headers with #if _KERNEL (XXX this needs to be fixed properly)
and cleanup header formatting.
- expand the component labels (yes, they should be backward/forward compatible)
- other bits and pieces (some function names are still bogus, and will get
changed soon)

Nuke: #include "rf_threadid.h" and call to rf_get_threadid().

branches: 1.7.2;
At config time, initialize a small amount of space for the disk queues
corresponding to the hot spares. We'll need that space when a spare is
actually added. Huge Thanks to Martin Laubach for helping track this down.

branches: 1.6.2; 1.6.4; 1.6.6;
Phase 2 of the RAIDframe cleanup. The source is now closer to KNF
and is much easier to read. No functionality changes.

RAIDframe cleanup, phase 1. Nuke simulator support, user-land driver,
out-dated comments, and other unneeded stuff. This helps prepare
for cleaning up the rest of the code, and adding new functionality.

No functional changes to the kernel code in this commit.

Use M_RAIDFRAME.

Duh... check that malloc() hasn't returned a NULL *before* we zero the buffer,
not after. Pointed out by Michael Graff.

Fix (potential) referencing of null pointer in (verbose) debugging mode.

RAIDframe, version 1.1, from the Parallel Data Laboratory at
Carnegie Mellon University. Full RAID implementation, including
levels 0, 1, 4, 5, 6, parity logging, and a few other goodies.
Ported to NetBSD by Greg Oster.

rf_CreateDiskQueueData() no longer uses waitflag, and will always succeed.
Cleanup the error path for the (no longer needed) PR_NOWAIT cases.

Various disk queue "peek" routines were only ever used in the simulator
version of RAIDFrame. Remove them from here.

Extensive mechanical changes to the pools used in RAIDframe.

Alloclist remains not per-RAID, so initialize that pool
separately/differently than the rest.

The remainder of pools in RF_Pools_s are now per-RAID pools. Mostly
mechanical changes to functions to allocate/destroy per-RAID pools.
Needed to make raidPtr available in certain cases to be able to find
the per-RAID pools.

Extend rf_pool_init() to now populate a per-RAID wchan value that is
unique to each pool for a given RAID device.

TODO: Complete the analysis of the minimum number of items that are
required for each pool to allow IO to progress (i.e. so that a request
for pool resources can always be satisfied), and dynamically scale
minimum pool sizes based on RAID configuration.

getiobuf() can return NULL if there are no IO buffers available.
RAIDframe can't deal with that, so create a dedicated pool of buffers
to use for IO. PR_WAITOK is fine here, as we pre-allocate more than
we need to guarantee IO can make progress. Tuning of pool still to
come.

remove unnnecessary splbio() in rf_FreeDiskQueueData()

pass down b_flags B_PHYS|B_RAW|B_MEDIA_FLAGS from bio subsystem
to component I/O

fixes the xbd(4) KASSERT() triggered by raidframe, noted in PR kern/55397
by Frank Kardel

fix the function pointer and callback mess:
- callback functions return 0 and their result is not checked; make them void.
- there are two types of callbacks and they used to overload their parameters
and the callback structure; separate them into "function" and "value"
callbacks.
- make the wait function signature consistent.

Introduce PR_ZERO to avoid open-coding memset()s everywhere. OK riastradh@.

- Change the allocation macros to be more like function calls
- Change sizeof(type) -> sizeof(*variable)
- Use macros for the long buffer length allocations
- Remove "bit polishing" memsets() -- do them only once
- Remove unnecessary casts

Thanks to oster@ for finding bugs and testing.

branches: 1.53.56;
convert the diskqueue mutex into a kmutex

branches: 1.52.4; 1.52.6;
Rework/simplify the disk queuing code. A bunch of this was still
holdovers from the simulator and would never be seen/used in-kernel.

branches: 1.51.4; 1.51.6; 1.51.10; 1.51.14;
Mark a buffer `busy` in getnewbuf() when it came from the pool_cache since
its not on a free list.

Also change buf_init() to not automatically mark buffers `busy' since this
only makes sense for bufcache buffers.

Mark all buf_init'd buffers 'busy' on the places where they ought to be
flagged as such to not confuse the buffer cache.

Fixes PR 38923.

branches: 1.50.6; 1.50.10; 1.50.12; 1.50.14;
Merge vmlocking2 to head.

branches: 1.49.2; 1.49.16; 1.49.22; 1.49.24; 1.49.28;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

branches: 1.48.4;
__unused removal on arguments; approved by core.

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

Protect calls to pool_put/pool_get that may occur in interrupt context
with spl used to protect other allocations and frees, or datastructure
element insertion and removal, in adjacent code.

It is almost unquestionably the case that some of the spl()/splx() calls
added here are superfluous, but it really seems wrong to see:

s=splfoo();
/* frob data structure */
splx(s);
pool_put(x);

and if we think we need to protect the first operation, then it is hard
to see why we should not think we need to protect the next. "Better
safe than sorry".

It is also almost unquestionably the case that I missed some pool
gets/puts from interrupt context with my strategy for finding these
calls; use of PR_NOWAIT is a strong hint that a pool may be used from
interrupt context but many callers in the kernel pass a "can wait/can't
wait" flag down such that my searches might not have found them. One
notable area that needs to be looked at is pf.

See also:

http://mail-index.netbsd.org/tech-kern/2006/07/19/0003.html
http://mail-index.netbsd.org/tech-kern/2006/07/19/0009.html

branches: 1.45.18; 1.45.20;
Cleanup the initialization of buffers a bit, and remove some old code
that works around a bug that no longer exists. From yamt. Thanks!
(a bit more cleanup to follow)

initialize necessary members of struct buf. PR/32462 from Reinoud Zandijk.

- add simple functions to allocate/free a buffer for i/o.
- make bufpool static.

branches: 1.42.2;
merge ktrace-lwp.

branches: 1.41.2;
- avoid variable shadowing
- add a lot of const
- remove parameters from functin declarations

branches: 1.40.2;
nuke trailing whitespace

The 'next' argument to rf_CreateDiskQueueData is always NULL. Since
there is no particular reason to pass an extra NULL argument, turf it,
and initialize p->next to NULL within the function.

Add a 'waitflag' argument to rf_CreateDiskQueueData() and use it to
determine if we are willing to wait for memory to come from the
diskqueuedata (dqd) and bufpool pools. Cleanup the mess related to
code calling rf_CreateDiskQueueData() with different expectations
(and/or blatent disregard) of what might happen if there were
insufficient pool resources.

CreateDiskQueueData() doesn't need to use void * for raidPtr.
RF_Raid_t is well known in RF, and there's no reason not to use it here.

branches: 1.36.4; 1.36.6;
Only touch bufpool whilst in splbio(). (That should be the case
already, but this makes it explicit and safer in the case where
that changes for some reason.)

branches: 1.35.4;
bufpool must be accessed at splbio().

Doesn't hurt much to zero this before we start mucking with it.

- Introduce rf_pools which contains all of the various global pools used
by RAIDframe. Convert all other RAIDframe global pools to use pools
defined within this new structure.
- Introduce rf_pool_init(), used for initializing a single pool in
RAIDframe. Teach each of the configuration routines to use
rf_pool_init().
- Cleanup a few pool-related comments.
- Cleanup revent initialization and #defines.
- Add a missing pool_destroy() for the reconbuffer pool.

(Saves another 1K off of an i386 GENERIC kernel, and makes
stuff a lot more readable)

- Introduce rf_pools which contains all of the various global pools used
by RAIDframe. Convert all other RAIDframe global pools to use pools
defined within this new structure.
- Introduce rf_pool_init(), used for initializing a single pool in
RAIDframe. Teach each of the configuration routines to use
rf_pool_init().
- Cleanup a few pool-related comments.
- Cleanup revent initialization and #defines.
- Add a missing pool_destroy() for the reconbuffer pool.

(Saves another 1K off of an i386 GENERIC kernel, and makes
stuff a lot more readable)

- remove the RF_*_INC's, as necessary. They are not needed any more.
- introduce RF_MIN_*'s, as necessary. These will indicate the
low-water mark for pools as well as the pool_prime() value.
- add pool_setlowat() for the critical pools.
- pool_prime() and pool_setlowat() the raidframe_cbufpool.
- re-order some pool_prime()'s and pool_sethiwat()'s for clarity.

Adjust _rf_ShutdownCreate() so that it is willing to wait for more
memory. Since we only now ever "return(0)", just return (void)
instead.

Cleanup all uses of rf_ShutdownCreate() to not worry about
it ever failing. Shaves another 600 bytes off of an i386 GENERIC kernel.

Nuke a bunch of unused variables:
- node_queue_cond
- quiescent_cond
- eq_cond
- desc->cond
- desc->head
- diskqueue->numWaiting

Nuke rf_print_unable_to_init_cond().
Nuke rf_TerminateDiskQueues prototype from rf_diskqueue.h.

init_dqd and clean_dqd don't do enough to be their own functions,
and just serve to cause confusion. Back-merge their contents to their
(only) calling spots and nuke them.

Some days you wonder if some of the function declaration consistency
was just an accident in the first place. Cleanup function decls and
a few comments. [ok.. so I wasn't going to fix this many.. but once
you're on a roll....]

_rf_create_managed_cond() is now left doing nothing. Convert callers.
Mash DO_RAID_COND in rf_driver.c out of existance.

- Nuke (already #if 0'ed) _rf_create_managed_lkmgr_mutex() while we're
busy here.

simplify DO_INIT in rf_engine.c

_rf_create_managed_mutex() is doing just a simple:

rf_mutex_init(m)

now. The rest of the fluff is no longer needed.
It also cannot fail, so error checking on rf_create_managed_mutex()
is just wasting space.

Nuke the #define's associated with rf_create_managed_mutex().
Convert rf_create_managed_mutex(listp,m) to just rf_mutex_init(m).
Remove wasteful "error checking" and simplify all instances where this
is called. (another 0.3K saved in the binary, but the real savings
is in code readability!)

- first kick at a major reworking of RAIDframe's memory allocation code:
- all freelists converted to pools
- initialization of structure members in certain cases where
code was relying on specific allocation and usage properties
to keep structures in a "known state" (that doesn't work with
pools!).
- make most pool_get() be "PR_WAITOK" until they can be analyzed
further, and/or have proper error handling added.
- all RF_Mallocs zero the space returned, so there is no difference
between RF_Calloc and RF_Malloc. In fact, all the RF_Calloc()'s
do is tend to do is get things horribly confused.
Make RF_Malloc() the "general memory allocator", with
RF_MallocAndAdd() the "general memory allocator with
allocation list".
- some of these RF_Malloc's et al. are destined to disappear.
- remove rf_rdp_freelist entirely (it's not used anywhere!)
- remove: #include "rf_freelist.h"
- to the files that were relying on the above, add: #include "rf_general.h"
- add: #include "rf_debugMem.h" to rf_shutdown.h to make it happy
about the loss of: #include "rf_freelist.h".

This shrinks an i386 GENERIC kernel by approx 5K. RAIDframe now
weighs in at about 162K on i386.

[Having received a definite lack of strenuous objection, a small amount
of strenuous agreement, and some general agreement, this commit is
going ahead because it's now starting to block some other changes I
wish to make.]

Remove most of the support for the concept of "rows" from RAIDframe.
While the "row" interface has been exported to the world, RAIDframe
internals have really only supported a single row, even though they
have feigned support of multiple rows.

Nothing changes in configuration land -- config files still need to
specify a single row, etc. All auto-config structures remain fully
forward/backwards compatible.

The only visible difference to the average user should be a
reduction in the size of a GENERIC kernel (i386) by 4.5K. For those
of us trolling through RAIDframe kernel code, a lot of the driver
configuration code has become a LOT easier to read.

branches: 1.22.2;
constify some

Use RF_DEBUG_DISKQUEUE to eliminate more debuggin printfs that the
vast majority of the world will never see much less care about.

Nuke unlockingOp -- it was only used for the userland version of the
code.

'buf2' is unused. Nuke it.

Everyone and their dog was using RF_ERRORMSG3 to print out the same
sort of error message, over and over again, in different files.
Rather than having the same text repeated in multiple .o files,
create a couple of little functions to do the printing, and save a
bundle of space. Also improves readability of code.

- AuxFunc is no longer used.
- cleanup function prototypes in rf_diskqueue.h

It appears that nothing is using rf_CreateDiskQueueDataFull. Make it even
harder to do so by removing it.

branches: 1.15.8;
add RCSIDs

Step 2 of the disentanglement. We now look to <dev/raidframe/*> for
the stuff that used to live in rf_types.h, rf_raidframe.h, rf_layout.h,
rf_netbsd.h, rf_raid.h, rf_decluster,h, and a few other places.
Believe it or not, when this is all done, things will be cleaner.

No functional changes to RAIDframe.

branches: 1.13.6; 1.13.8; 1.13.10;
Reorganize some comments.

Garbage collect the (already disabled) "random" queue type.

Garbage collect some unused stuff. Include rf_kintf.h for a
function prototype.

- record the pointer to the DiskQueueSW so we can use that when
we hot-add a spare.
- use the above info when adding a spare.

Get recent changes into the tree:
- make component_label variables more consistent (==> clabel)
- re-work incorrect component configuration code
- re-work disk configuration code
- cleanup initial configuration of raidPtr info
- add auto-detection of components and RAID sets (Disabled, for now)
- allow / on RAID sets (Disabled, for now)
- rename "config_disk_queue" to "rf_ConfigureDiskQueue" and properly prototype
in rf_diskqueue.h
- protect some headers with #if _KERNEL (XXX this needs to be fixed properly)
and cleanup header formatting.
- expand the component labels (yes, they should be backward/forward compatible)
- other bits and pieces (some function names are still bogus, and will get
changed soon)

Nuke: #include "rf_threadid.h" and call to rf_get_threadid().

branches: 1.7.2;
At config time, initialize a small amount of space for the disk queues
corresponding to the hot spares. We'll need that space when a spare is
actually added. Huge Thanks to Martin Laubach for helping track this down.

branches: 1.6.2; 1.6.4; 1.6.6;
Phase 2 of the RAIDframe cleanup. The source is now closer to KNF
and is much easier to read. No functionality changes.

RAIDframe cleanup, phase 1. Nuke simulator support, user-land driver,
out-dated comments, and other unneeded stuff. This helps prepare
for cleaning up the rest of the code, and adding new functionality.

No functional changes to the kernel code in this commit.

Use M_RAIDFRAME.

Duh... check that malloc() hasn't returned a NULL *before* we zero the buffer,
not after. Pointed out by Michael Graff.

Fix (potential) referencing of null pointer in (verbose) debugging mode.

RAIDframe, version 1.1, from the Parallel Data Laboratory at
Carnegie Mellon University. Full RAID implementation, including
levels 0, 1, 4, 5, 6, parity logging, and a few other goodies.
Ported to NetBSD by Greg Oster.

Various disk queue "peek" routines were only ever used in the simulator
version of RAIDFrame. Remove them from here.

Extensive mechanical changes to the pools used in RAIDframe.

Alloclist remains not per-RAID, so initialize that pool
separately/differently than the rest.

The remainder of pools in RF_Pools_s are now per-RAID pools. Mostly
mechanical changes to functions to allocate/destroy per-RAID pools.
Needed to make raidPtr available in certain cases to be able to find
the per-RAID pools.

Extend rf_pool_init() to now populate a per-RAID wchan value that is
unique to each pool for a given RAID device.

TODO: Complete the analysis of the minimum number of items that are
required for each pool to allow IO to progress (i.e. so that a request
for pool resources can always be satisfied), and dynamically scale
minimum pool sizes based on RAID configuration.

getiobuf() can return NULL if there are no IO buffers available.
RAIDframe can't deal with that, so create a dedicated pool of buffers
to use for IO. PR_WAITOK is fine here, as we pre-allocate more than
we need to guarantee IO can make progress. Tuning of pool still to
come.

remove unnnecessary splbio() in rf_FreeDiskQueueData()

pass down b_flags B_PHYS|B_RAW|B_MEDIA_FLAGS from bio subsystem
to component I/O

fixes the xbd(4) KASSERT() triggered by raidframe, noted in PR kern/55397
by Frank Kardel

fix the function pointer and callback mess:
- callback functions return 0 and their result is not checked; make them void.
- there are two types of callbacks and they used to overload their parameters
and the callback structure; separate them into "function" and "value"
callbacks.
- make the wait function signature consistent.

Introduce PR_ZERO to avoid open-coding memset()s everywhere. OK riastradh@.

- Change the allocation macros to be more like function calls
- Change sizeof(type) -> sizeof(*variable)
- Use macros for the long buffer length allocations
- Remove "bit polishing" memsets() -- do them only once
- Remove unnecessary casts

Thanks to oster@ for finding bugs and testing.

branches: 1.53.56;
convert the diskqueue mutex into a kmutex

branches: 1.52.4; 1.52.6;
Rework/simplify the disk queuing code. A bunch of this was still
holdovers from the simulator and would never be seen/used in-kernel.

branches: 1.51.4; 1.51.6; 1.51.10; 1.51.14;
Mark a buffer `busy` in getnewbuf() when it came from the pool_cache since
its not on a free list.

Also change buf_init() to not automatically mark buffers `busy' since this
only makes sense for bufcache buffers.

Mark all buf_init'd buffers 'busy' on the places where they ought to be
flagged as such to not confuse the buffer cache.

Fixes PR 38923.

branches: 1.50.6; 1.50.10; 1.50.12; 1.50.14;
Merge vmlocking2 to head.

branches: 1.49.2; 1.49.16; 1.49.22; 1.49.24; 1.49.28;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

branches: 1.48.4;
__unused removal on arguments; approved by core.

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

Protect calls to pool_put/pool_get that may occur in interrupt context
with spl used to protect other allocations and frees, or datastructure
element insertion and removal, in adjacent code.

It is almost unquestionably the case that some of the spl()/splx() calls
added here are superfluous, but it really seems wrong to see:

s=splfoo();
/* frob data structure */
splx(s);
pool_put(x);

and if we think we need to protect the first operation, then it is hard
to see why we should not think we need to protect the next. "Better
safe than sorry".

It is also almost unquestionably the case that I missed some pool
gets/puts from interrupt context with my strategy for finding these
calls; use of PR_NOWAIT is a strong hint that a pool may be used from
interrupt context but many callers in the kernel pass a "can wait/can't
wait" flag down such that my searches might not have found them. One
notable area that needs to be looked at is pf.

See also:

http://mail-index.netbsd.org/tech-kern/2006/07/19/0003.html
http://mail-index.netbsd.org/tech-kern/2006/07/19/0009.html

branches: 1.45.18; 1.45.20;
Cleanup the initialization of buffers a bit, and remove some old code
that works around a bug that no longer exists. From yamt. Thanks!
(a bit more cleanup to follow)

initialize necessary members of struct buf. PR/32462 from Reinoud Zandijk.

- add simple functions to allocate/free a buffer for i/o.
- make bufpool static.

branches: 1.42.2;
merge ktrace-lwp.

branches: 1.41.2;
- avoid variable shadowing
- add a lot of const
- remove parameters from functin declarations

branches: 1.40.2;
nuke trailing whitespace

The 'next' argument to rf_CreateDiskQueueData is always NULL. Since
there is no particular reason to pass an extra NULL argument, turf it,
and initialize p->next to NULL within the function.

Add a 'waitflag' argument to rf_CreateDiskQueueData() and use it to
determine if we are willing to wait for memory to come from the
diskqueuedata (dqd) and bufpool pools. Cleanup the mess related to
code calling rf_CreateDiskQueueData() with different expectations
(and/or blatent disregard) of what might happen if there were
insufficient pool resources.

CreateDiskQueueData() doesn't need to use void * for raidPtr.
RF_Raid_t is well known in RF, and there's no reason not to use it here.

branches: 1.36.4; 1.36.6;
Only touch bufpool whilst in splbio(). (That should be the case
already, but this makes it explicit and safer in the case where
that changes for some reason.)

branches: 1.35.4;
bufpool must be accessed at splbio().

Doesn't hurt much to zero this before we start mucking with it.

- Introduce rf_pools which contains all of the various global pools used
by RAIDframe. Convert all other RAIDframe global pools to use pools
defined within this new structure.
- Introduce rf_pool_init(), used for initializing a single pool in
RAIDframe. Teach each of the configuration routines to use
rf_pool_init().
- Cleanup a few pool-related comments.
- Cleanup revent initialization and #defines.
- Add a missing pool_destroy() for the reconbuffer pool.

(Saves another 1K off of an i386 GENERIC kernel, and makes
stuff a lot more readable)

- Introduce rf_pools which contains all of the various global pools used
by RAIDframe. Convert all other RAIDframe global pools to use pools
defined within this new structure.
- Introduce rf_pool_init(), used for initializing a single pool in
RAIDframe. Teach each of the configuration routines to use
rf_pool_init().
- Cleanup a few pool-related comments.
- Cleanup revent initialization and #defines.
- Add a missing pool_destroy() for the reconbuffer pool.

(Saves another 1K off of an i386 GENERIC kernel, and makes
stuff a lot more readable)

- remove the RF_*_INC's, as necessary. They are not needed any more.
- introduce RF_MIN_*'s, as necessary. These will indicate the
low-water mark for pools as well as the pool_prime() value.
- add pool_setlowat() for the critical pools.
- pool_prime() and pool_setlowat() the raidframe_cbufpool.
- re-order some pool_prime()'s and pool_sethiwat()'s for clarity.

Adjust _rf_ShutdownCreate() so that it is willing to wait for more
memory. Since we only now ever "return(0)", just return (void)
instead.

Cleanup all uses of rf_ShutdownCreate() to not worry about
it ever failing. Shaves another 600 bytes off of an i386 GENERIC kernel.

Nuke a bunch of unused variables:
- node_queue_cond
- quiescent_cond
- eq_cond
- desc->cond
- desc->head
- diskqueue->numWaiting

Nuke rf_print_unable_to_init_cond().
Nuke rf_TerminateDiskQueues prototype from rf_diskqueue.h.

init_dqd and clean_dqd don't do enough to be their own functions,
and just serve to cause confusion. Back-merge their contents to their
(only) calling spots and nuke them.

Some days you wonder if some of the function declaration consistency
was just an accident in the first place. Cleanup function decls and
a few comments. [ok.. so I wasn't going to fix this many.. but once
you're on a roll....]

_rf_create_managed_cond() is now left doing nothing. Convert callers.
Mash DO_RAID_COND in rf_driver.c out of existance.

- Nuke (already #if 0'ed) _rf_create_managed_lkmgr_mutex() while we're
busy here.

simplify DO_INIT in rf_engine.c

_rf_create_managed_mutex() is doing just a simple:

rf_mutex_init(m)

now. The rest of the fluff is no longer needed.
It also cannot fail, so error checking on rf_create_managed_mutex()
is just wasting space.

Nuke the #define's associated with rf_create_managed_mutex().
Convert rf_create_managed_mutex(listp,m) to just rf_mutex_init(m).
Remove wasteful "error checking" and simplify all instances where this
is called. (another 0.3K saved in the binary, but the real savings
is in code readability!)

- first kick at a major reworking of RAIDframe's memory allocation code:
- all freelists converted to pools
- initialization of structure members in certain cases where
code was relying on specific allocation and usage properties
to keep structures in a "known state" (that doesn't work with
pools!).
- make most pool_get() be "PR_WAITOK" until they can be analyzed
further, and/or have proper error handling added.
- all RF_Mallocs zero the space returned, so there is no difference
between RF_Calloc and RF_Malloc. In fact, all the RF_Calloc()'s
do is tend to do is get things horribly confused.
Make RF_Malloc() the "general memory allocator", with
RF_MallocAndAdd() the "general memory allocator with
allocation list".
- some of these RF_Malloc's et al. are destined to disappear.
- remove rf_rdp_freelist entirely (it's not used anywhere!)
- remove: #include "rf_freelist.h"
- to the files that were relying on the above, add: #include "rf_general.h"
- add: #include "rf_debugMem.h" to rf_shutdown.h to make it happy
about the loss of: #include "rf_freelist.h".

This shrinks an i386 GENERIC kernel by approx 5K. RAIDframe now
weighs in at about 162K on i386.

[Having received a definite lack of strenuous objection, a small amount
of strenuous agreement, and some general agreement, this commit is
going ahead because it's now starting to block some other changes I
wish to make.]

Remove most of the support for the concept of "rows" from RAIDframe.
While the "row" interface has been exported to the world, RAIDframe
internals have really only supported a single row, even though they
have feigned support of multiple rows.

Nothing changes in configuration land -- config files still need to
specify a single row, etc. All auto-config structures remain fully
forward/backwards compatible.

The only visible difference to the average user should be a
reduction in the size of a GENERIC kernel (i386) by 4.5K. For those
of us trolling through RAIDframe kernel code, a lot of the driver
configuration code has become a LOT easier to read.

branches: 1.22.2;
constify some

Use RF_DEBUG_DISKQUEUE to eliminate more debuggin printfs that the
vast majority of the world will never see much less care about.

Nuke unlockingOp -- it was only used for the userland version of the
code.

'buf2' is unused. Nuke it.

Everyone and their dog was using RF_ERRORMSG3 to print out the same
sort of error message, over and over again, in different files.
Rather than having the same text repeated in multiple .o files,
create a couple of little functions to do the printing, and save a
bundle of space. Also improves readability of code.

- AuxFunc is no longer used.
- cleanup function prototypes in rf_diskqueue.h

It appears that nothing is using rf_CreateDiskQueueDataFull. Make it even
harder to do so by removing it.

branches: 1.15.8;
add RCSIDs

Step 2 of the disentanglement. We now look to <dev/raidframe/*> for
the stuff that used to live in rf_types.h, rf_raidframe.h, rf_layout.h,
rf_netbsd.h, rf_raid.h, rf_decluster,h, and a few other places.
Believe it or not, when this is all done, things will be cleaner.

No functional changes to RAIDframe.

branches: 1.13.6; 1.13.8; 1.13.10;
Reorganize some comments.

Garbage collect the (already disabled) "random" queue type.

Garbage collect some unused stuff. Include rf_kintf.h for a
function prototype.

- record the pointer to the DiskQueueSW so we can use that when
we hot-add a spare.
- use the above info when adding a spare.

Get recent changes into the tree:
- make component_label variables more consistent (==> clabel)
- re-work incorrect component configuration code
- re-work disk configuration code
- cleanup initial configuration of raidPtr info
- add auto-detection of components and RAID sets (Disabled, for now)
- allow / on RAID sets (Disabled, for now)
- rename "config_disk_queue" to "rf_ConfigureDiskQueue" and properly prototype
in rf_diskqueue.h
- protect some headers with #if _KERNEL (XXX this needs to be fixed properly)
and cleanup header formatting.
- expand the component labels (yes, they should be backward/forward compatible)
- other bits and pieces (some function names are still bogus, and will get
changed soon)

Nuke: #include "rf_threadid.h" and call to rf_get_threadid().

branches: 1.7.2;
At config time, initialize a small amount of space for the disk queues
corresponding to the hot spares. We'll need that space when a spare is
actually added. Huge Thanks to Martin Laubach for helping track this down.

branches: 1.6.2; 1.6.4; 1.6.6;
Phase 2 of the RAIDframe cleanup. The source is now closer to KNF
and is much easier to read. No functionality changes.

RAIDframe cleanup, phase 1. Nuke simulator support, user-land driver,
out-dated comments, and other unneeded stuff. This helps prepare
for cleaning up the rest of the code, and adding new functionality.

No functional changes to the kernel code in this commit.

Use M_RAIDFRAME.

Duh... check that malloc() hasn't returned a NULL *before* we zero the buffer,
not after. Pointed out by Michael Graff.

Fix (potential) referencing of null pointer in (verbose) debugging mode.

RAIDframe, version 1.1, from the Parallel Data Laboratory at
Carnegie Mellon University. Full RAID implementation, including
levels 0, 1, 4, 5, 6, parity logging, and a few other goodies.
Ported to NetBSD by Greg Oster.

Extensive mechanical changes to the pools used in RAIDframe.

Alloclist remains not per-RAID, so initialize that pool
separately/differently than the rest.

The remainder of pools in RF_Pools_s are now per-RAID pools. Mostly
mechanical changes to functions to allocate/destroy per-RAID pools.
Needed to make raidPtr available in certain cases to be able to find
the per-RAID pools.

Extend rf_pool_init() to now populate a per-RAID wchan value that is
unique to each pool for a given RAID device.

TODO: Complete the analysis of the minimum number of items that are
required for each pool to allow IO to progress (i.e. so that a request
for pool resources can always be satisfied), and dynamically scale
minimum pool sizes based on RAID configuration.

getiobuf() can return NULL if there are no IO buffers available.
RAIDframe can't deal with that, so create a dedicated pool of buffers
to use for IO. PR_WAITOK is fine here, as we pre-allocate more than
we need to guarantee IO can make progress. Tuning of pool still to
come.

remove unnnecessary splbio() in rf_FreeDiskQueueData()

pass down b_flags B_PHYS|B_RAW|B_MEDIA_FLAGS from bio subsystem
to component I/O

fixes the xbd(4) KASSERT() triggered by raidframe, noted in PR kern/55397
by Frank Kardel

fix the function pointer and callback mess:
- callback functions return 0 and their result is not checked; make them void.
- there are two types of callbacks and they used to overload their parameters
and the callback structure; separate them into "function" and "value"
callbacks.
- make the wait function signature consistent.

Introduce PR_ZERO to avoid open-coding memset()s everywhere. OK riastradh@.

- Change the allocation macros to be more like function calls
- Change sizeof(type) -> sizeof(*variable)
- Use macros for the long buffer length allocations
- Remove "bit polishing" memsets() -- do them only once
- Remove unnecessary casts

Thanks to oster@ for finding bugs and testing.

branches: 1.53.56;
convert the diskqueue mutex into a kmutex

branches: 1.52.4; 1.52.6;
Rework/simplify the disk queuing code. A bunch of this was still
holdovers from the simulator and would never be seen/used in-kernel.

branches: 1.51.4; 1.51.6; 1.51.10; 1.51.14;
Mark a buffer `busy` in getnewbuf() when it came from the pool_cache since
its not on a free list.

Also change buf_init() to not automatically mark buffers `busy' since this
only makes sense for bufcache buffers.

Mark all buf_init'd buffers 'busy' on the places where they ought to be
flagged as such to not confuse the buffer cache.

Fixes PR 38923.

branches: 1.50.6; 1.50.10; 1.50.12; 1.50.14;
Merge vmlocking2 to head.

branches: 1.49.2; 1.49.16; 1.49.22; 1.49.24; 1.49.28;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

branches: 1.48.4;
__unused removal on arguments; approved by core.

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

Protect calls to pool_put/pool_get that may occur in interrupt context
with spl used to protect other allocations and frees, or datastructure
element insertion and removal, in adjacent code.

It is almost unquestionably the case that some of the spl()/splx() calls
added here are superfluous, but it really seems wrong to see:

s=splfoo();
/* frob data structure */
splx(s);
pool_put(x);

and if we think we need to protect the first operation, then it is hard
to see why we should not think we need to protect the next. "Better
safe than sorry".

It is also almost unquestionably the case that I missed some pool
gets/puts from interrupt context with my strategy for finding these
calls; use of PR_NOWAIT is a strong hint that a pool may be used from
interrupt context but many callers in the kernel pass a "can wait/can't
wait" flag down such that my searches might not have found them. One
notable area that needs to be looked at is pf.

See also:

http://mail-index.netbsd.org/tech-kern/2006/07/19/0003.html
http://mail-index.netbsd.org/tech-kern/2006/07/19/0009.html

branches: 1.45.18; 1.45.20;
Cleanup the initialization of buffers a bit, and remove some old code
that works around a bug that no longer exists. From yamt. Thanks!
(a bit more cleanup to follow)

initialize necessary members of struct buf. PR/32462 from Reinoud Zandijk.

- add simple functions to allocate/free a buffer for i/o.
- make bufpool static.

branches: 1.42.2;
merge ktrace-lwp.

branches: 1.41.2;
- avoid variable shadowing
- add a lot of const
- remove parameters from functin declarations

branches: 1.40.2;
nuke trailing whitespace

The 'next' argument to rf_CreateDiskQueueData is always NULL. Since
there is no particular reason to pass an extra NULL argument, turf it,
and initialize p->next to NULL within the function.

Add a 'waitflag' argument to rf_CreateDiskQueueData() and use it to
determine if we are willing to wait for memory to come from the
diskqueuedata (dqd) and bufpool pools. Cleanup the mess related to
code calling rf_CreateDiskQueueData() with different expectations
(and/or blatent disregard) of what might happen if there were
insufficient pool resources.

CreateDiskQueueData() doesn't need to use void * for raidPtr.
RF_Raid_t is well known in RF, and there's no reason not to use it here.

branches: 1.36.4; 1.36.6;
Only touch bufpool whilst in splbio(). (That should be the case
already, but this makes it explicit and safer in the case where
that changes for some reason.)

branches: 1.35.4;
bufpool must be accessed at splbio().

Doesn't hurt much to zero this before we start mucking with it.

- Introduce rf_pools which contains all of the various global pools used
by RAIDframe. Convert all other RAIDframe global pools to use pools
defined within this new structure.
- Introduce rf_pool_init(), used for initializing a single pool in
RAIDframe. Teach each of the configuration routines to use
rf_pool_init().
- Cleanup a few pool-related comments.
- Cleanup revent initialization and #defines.
- Add a missing pool_destroy() for the reconbuffer pool.

(Saves another 1K off of an i386 GENERIC kernel, and makes
stuff a lot more readable)

- Introduce rf_pools which contains all of the various global pools used
by RAIDframe. Convert all other RAIDframe global pools to use pools
defined within this new structure.
- Introduce rf_pool_init(), used for initializing a single pool in
RAIDframe. Teach each of the configuration routines to use
rf_pool_init().
- Cleanup a few pool-related comments.
- Cleanup revent initialization and #defines.
- Add a missing pool_destroy() for the reconbuffer pool.

(Saves another 1K off of an i386 GENERIC kernel, and makes
stuff a lot more readable)

- remove the RF_*_INC's, as necessary. They are not needed any more.
- introduce RF_MIN_*'s, as necessary. These will indicate the
low-water mark for pools as well as the pool_prime() value.
- add pool_setlowat() for the critical pools.
- pool_prime() and pool_setlowat() the raidframe_cbufpool.
- re-order some pool_prime()'s and pool_sethiwat()'s for clarity.

Adjust _rf_ShutdownCreate() so that it is willing to wait for more
memory. Since we only now ever "return(0)", just return (void)
instead.

Cleanup all uses of rf_ShutdownCreate() to not worry about
it ever failing. Shaves another 600 bytes off of an i386 GENERIC kernel.

Nuke a bunch of unused variables:
- node_queue_cond
- quiescent_cond
- eq_cond
- desc->cond
- desc->head
- diskqueue->numWaiting

Nuke rf_print_unable_to_init_cond().
Nuke rf_TerminateDiskQueues prototype from rf_diskqueue.h.

init_dqd and clean_dqd don't do enough to be their own functions,
and just serve to cause confusion. Back-merge their contents to their
(only) calling spots and nuke them.

Some days you wonder if some of the function declaration consistency
was just an accident in the first place. Cleanup function decls and
a few comments. [ok.. so I wasn't going to fix this many.. but once
you're on a roll....]

_rf_create_managed_cond() is now left doing nothing. Convert callers.
Mash DO_RAID_COND in rf_driver.c out of existance.

- Nuke (already #if 0'ed) _rf_create_managed_lkmgr_mutex() while we're
busy here.

simplify DO_INIT in rf_engine.c

_rf_create_managed_mutex() is doing just a simple:

rf_mutex_init(m)

now. The rest of the fluff is no longer needed.
It also cannot fail, so error checking on rf_create_managed_mutex()
is just wasting space.

Nuke the #define's associated with rf_create_managed_mutex().
Convert rf_create_managed_mutex(listp,m) to just rf_mutex_init(m).
Remove wasteful "error checking" and simplify all instances where this
is called. (another 0.3K saved in the binary, but the real savings
is in code readability!)

- first kick at a major reworking of RAIDframe's memory allocation code:
- all freelists converted to pools
- initialization of structure members in certain cases where
code was relying on specific allocation and usage properties
to keep structures in a "known state" (that doesn't work with
pools!).
- make most pool_get() be "PR_WAITOK" until they can be analyzed
further, and/or have proper error handling added.
- all RF_Mallocs zero the space returned, so there is no difference
between RF_Calloc and RF_Malloc. In fact, all the RF_Calloc()'s
do is tend to do is get things horribly confused.
Make RF_Malloc() the "general memory allocator", with
RF_MallocAndAdd() the "general memory allocator with
allocation list".
- some of these RF_Malloc's et al. are destined to disappear.
- remove rf_rdp_freelist entirely (it's not used anywhere!)
- remove: #include "rf_freelist.h"
- to the files that were relying on the above, add: #include "rf_general.h"
- add: #include "rf_debugMem.h" to rf_shutdown.h to make it happy
about the loss of: #include "rf_freelist.h".

This shrinks an i386 GENERIC kernel by approx 5K. RAIDframe now
weighs in at about 162K on i386.

[Having received a definite lack of strenuous objection, a small amount
of strenuous agreement, and some general agreement, this commit is
going ahead because it's now starting to block some other changes I
wish to make.]

Remove most of the support for the concept of "rows" from RAIDframe.
While the "row" interface has been exported to the world, RAIDframe
internals have really only supported a single row, even though they
have feigned support of multiple rows.

Nothing changes in configuration land -- config files still need to
specify a single row, etc. All auto-config structures remain fully
forward/backwards compatible.

The only visible difference to the average user should be a
reduction in the size of a GENERIC kernel (i386) by 4.5K. For those
of us trolling through RAIDframe kernel code, a lot of the driver
configuration code has become a LOT easier to read.

branches: 1.22.2;
constify some

Use RF_DEBUG_DISKQUEUE to eliminate more debuggin printfs that the
vast majority of the world will never see much less care about.

Nuke unlockingOp -- it was only used for the userland version of the
code.

'buf2' is unused. Nuke it.

Everyone and their dog was using RF_ERRORMSG3 to print out the same
sort of error message, over and over again, in different files.
Rather than having the same text repeated in multiple .o files,
create a couple of little functions to do the printing, and save a
bundle of space. Also improves readability of code.

- AuxFunc is no longer used.
- cleanup function prototypes in rf_diskqueue.h

It appears that nothing is using rf_CreateDiskQueueDataFull. Make it even
harder to do so by removing it.

branches: 1.15.8;
add RCSIDs

Step 2 of the disentanglement. We now look to <dev/raidframe/*> for
the stuff that used to live in rf_types.h, rf_raidframe.h, rf_layout.h,
rf_netbsd.h, rf_raid.h, rf_decluster,h, and a few other places.
Believe it or not, when this is all done, things will be cleaner.

No functional changes to RAIDframe.

branches: 1.13.6; 1.13.8; 1.13.10;
Reorganize some comments.

Garbage collect the (already disabled) "random" queue type.

Garbage collect some unused stuff. Include rf_kintf.h for a
function prototype.

- record the pointer to the DiskQueueSW so we can use that when
we hot-add a spare.
- use the above info when adding a spare.

Get recent changes into the tree:
- make component_label variables more consistent (==> clabel)
- re-work incorrect component configuration code
- re-work disk configuration code
- cleanup initial configuration of raidPtr info
- add auto-detection of components and RAID sets (Disabled, for now)
- allow / on RAID sets (Disabled, for now)
- rename "config_disk_queue" to "rf_ConfigureDiskQueue" and properly prototype
in rf_diskqueue.h
- protect some headers with #if _KERNEL (XXX this needs to be fixed properly)
and cleanup header formatting.
- expand the component labels (yes, they should be backward/forward compatible)
- other bits and pieces (some function names are still bogus, and will get
changed soon)

Nuke: #include "rf_threadid.h" and call to rf_get_threadid().

branches: 1.7.2;
At config time, initialize a small amount of space for the disk queues
corresponding to the hot spares. We'll need that space when a spare is
actually added. Huge Thanks to Martin Laubach for helping track this down.

branches: 1.6.2; 1.6.4; 1.6.6;
Phase 2 of the RAIDframe cleanup. The source is now closer to KNF
and is much easier to read. No functionality changes.

RAIDframe cleanup, phase 1. Nuke simulator support, user-land driver,
out-dated comments, and other unneeded stuff. This helps prepare
for cleaning up the rest of the code, and adding new functionality.

No functional changes to the kernel code in this commit.

Use M_RAIDFRAME.

Duh... check that malloc() hasn't returned a NULL *before* we zero the buffer,
not after. Pointed out by Michael Graff.

Fix (potential) referencing of null pointer in (verbose) debugging mode.

RAIDframe, version 1.1, from the Parallel Data Laboratory at
Carnegie Mellon University. Full RAID implementation, including
levels 0, 1, 4, 5, 6, parity logging, and a few other goodies.
Ported to NetBSD by Greg Oster.

remove unnnecessary splbio() in rf_FreeDiskQueueData()

pass down b_flags B_PHYS|B_RAW|B_MEDIA_FLAGS from bio subsystem
to component I/O

fixes the xbd(4) KASSERT() triggered by raidframe, noted in PR kern/55397
by Frank Kardel

fix the function pointer and callback mess:
- callback functions return 0 and their result is not checked; make them void.
- there are two types of callbacks and they used to overload their parameters
and the callback structure; separate them into "function" and "value"
callbacks.
- make the wait function signature consistent.

Introduce PR_ZERO to avoid open-coding memset()s everywhere. OK riastradh@.

- Change the allocation macros to be more like function calls
- Change sizeof(type) -> sizeof(*variable)
- Use macros for the long buffer length allocations
- Remove "bit polishing" memsets() -- do them only once
- Remove unnecessary casts

Thanks to oster@ for finding bugs and testing.

branches: 1.53.56;
convert the diskqueue mutex into a kmutex

branches: 1.52.4; 1.52.6;
Rework/simplify the disk queuing code. A bunch of this was still
holdovers from the simulator and would never be seen/used in-kernel.

branches: 1.51.4; 1.51.6; 1.51.10; 1.51.14;
Mark a buffer `busy` in getnewbuf() when it came from the pool_cache since
its not on a free list.

Also change buf_init() to not automatically mark buffers `busy' since this
only makes sense for bufcache buffers.

Mark all buf_init'd buffers 'busy' on the places where they ought to be
flagged as such to not confuse the buffer cache.

Fixes PR 38923.

branches: 1.50.6; 1.50.10; 1.50.12; 1.50.14;
Merge vmlocking2 to head.

branches: 1.49.2; 1.49.16; 1.49.22; 1.49.24; 1.49.28;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

branches: 1.48.4;
__unused removal on arguments; approved by core.

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

Protect calls to pool_put/pool_get that may occur in interrupt context
with spl used to protect other allocations and frees, or datastructure
element insertion and removal, in adjacent code.

It is almost unquestionably the case that some of the spl()/splx() calls
added here are superfluous, but it really seems wrong to see:

s=splfoo();
/* frob data structure */
splx(s);
pool_put(x);

and if we think we need to protect the first operation, then it is hard
to see why we should not think we need to protect the next. "Better
safe than sorry".

It is also almost unquestionably the case that I missed some pool
gets/puts from interrupt context with my strategy for finding these
calls; use of PR_NOWAIT is a strong hint that a pool may be used from
interrupt context but many callers in the kernel pass a "can wait/can't
wait" flag down such that my searches might not have found them. One
notable area that needs to be looked at is pf.

See also:

http://mail-index.netbsd.org/tech-kern/2006/07/19/0003.html
http://mail-index.netbsd.org/tech-kern/2006/07/19/0009.html

branches: 1.45.18; 1.45.20;
Cleanup the initialization of buffers a bit, and remove some old code
that works around a bug that no longer exists. From yamt. Thanks!
(a bit more cleanup to follow)

initialize necessary members of struct buf. PR/32462 from Reinoud Zandijk.

- add simple functions to allocate/free a buffer for i/o.
- make bufpool static.

branches: 1.42.2;
merge ktrace-lwp.

branches: 1.41.2;
- avoid variable shadowing
- add a lot of const
- remove parameters from functin declarations

branches: 1.40.2;
nuke trailing whitespace

The 'next' argument to rf_CreateDiskQueueData is always NULL. Since
there is no particular reason to pass an extra NULL argument, turf it,
and initialize p->next to NULL within the function.

Add a 'waitflag' argument to rf_CreateDiskQueueData() and use it to
determine if we are willing to wait for memory to come from the
diskqueuedata (dqd) and bufpool pools. Cleanup the mess related to
code calling rf_CreateDiskQueueData() with different expectations
(and/or blatent disregard) of what might happen if there were
insufficient pool resources.

CreateDiskQueueData() doesn't need to use void * for raidPtr.
RF_Raid_t is well known in RF, and there's no reason not to use it here.

branches: 1.36.4; 1.36.6;
Only touch bufpool whilst in splbio(). (That should be the case
already, but this makes it explicit and safer in the case where
that changes for some reason.)

branches: 1.35.4;
bufpool must be accessed at splbio().

Doesn't hurt much to zero this before we start mucking with it.

- Introduce rf_pools which contains all of the various global pools used
by RAIDframe. Convert all other RAIDframe global pools to use pools
defined within this new structure.
- Introduce rf_pool_init(), used for initializing a single pool in
RAIDframe. Teach each of the configuration routines to use
rf_pool_init().
- Cleanup a few pool-related comments.
- Cleanup revent initialization and #defines.
- Add a missing pool_destroy() for the reconbuffer pool.

(Saves another 1K off of an i386 GENERIC kernel, and makes
stuff a lot more readable)

- Introduce rf_pools which contains all of the various global pools used
by RAIDframe. Convert all other RAIDframe global pools to use pools
defined within this new structure.
- Introduce rf_pool_init(), used for initializing a single pool in
RAIDframe. Teach each of the configuration routines to use
rf_pool_init().
- Cleanup a few pool-related comments.
- Cleanup revent initialization and #defines.
- Add a missing pool_destroy() for the reconbuffer pool.

(Saves another 1K off of an i386 GENERIC kernel, and makes
stuff a lot more readable)

- remove the RF_*_INC's, as necessary. They are not needed any more.
- introduce RF_MIN_*'s, as necessary. These will indicate the
low-water mark for pools as well as the pool_prime() value.
- add pool_setlowat() for the critical pools.
- pool_prime() and pool_setlowat() the raidframe_cbufpool.
- re-order some pool_prime()'s and pool_sethiwat()'s for clarity.

Adjust _rf_ShutdownCreate() so that it is willing to wait for more
memory. Since we only now ever "return(0)", just return (void)
instead.

Cleanup all uses of rf_ShutdownCreate() to not worry about
it ever failing. Shaves another 600 bytes off of an i386 GENERIC kernel.

Nuke a bunch of unused variables:
- node_queue_cond
- quiescent_cond
- eq_cond
- desc->cond
- desc->head
- diskqueue->numWaiting

Nuke rf_print_unable_to_init_cond().
Nuke rf_TerminateDiskQueues prototype from rf_diskqueue.h.

init_dqd and clean_dqd don't do enough to be their own functions,
and just serve to cause confusion. Back-merge their contents to their
(only) calling spots and nuke them.

Some days you wonder if some of the function declaration consistency
was just an accident in the first place. Cleanup function decls and
a few comments. [ok.. so I wasn't going to fix this many.. but once
you're on a roll....]

_rf_create_managed_cond() is now left doing nothing. Convert callers.
Mash DO_RAID_COND in rf_driver.c out of existance.

- Nuke (already #if 0'ed) _rf_create_managed_lkmgr_mutex() while we're
busy here.

simplify DO_INIT in rf_engine.c

_rf_create_managed_mutex() is doing just a simple:

rf_mutex_init(m)

now. The rest of the fluff is no longer needed.
It also cannot fail, so error checking on rf_create_managed_mutex()
is just wasting space.

Nuke the #define's associated with rf_create_managed_mutex().
Convert rf_create_managed_mutex(listp,m) to just rf_mutex_init(m).
Remove wasteful "error checking" and simplify all instances where this
is called. (another 0.3K saved in the binary, but the real savings
is in code readability!)

- first kick at a major reworking of RAIDframe's memory allocation code:
- all freelists converted to pools
- initialization of structure members in certain cases where
code was relying on specific allocation and usage properties
to keep structures in a "known state" (that doesn't work with
pools!).
- make most pool_get() be "PR_WAITOK" until they can be analyzed
further, and/or have proper error handling added.
- all RF_Mallocs zero the space returned, so there is no difference
between RF_Calloc and RF_Malloc. In fact, all the RF_Calloc()'s
do is tend to do is get things horribly confused.
Make RF_Malloc() the "general memory allocator", with
RF_MallocAndAdd() the "general memory allocator with
allocation list".
- some of these RF_Malloc's et al. are destined to disappear.
- remove rf_rdp_freelist entirely (it's not used anywhere!)
- remove: #include "rf_freelist.h"
- to the files that were relying on the above, add: #include "rf_general.h"
- add: #include "rf_debugMem.h" to rf_shutdown.h to make it happy
about the loss of: #include "rf_freelist.h".

This shrinks an i386 GENERIC kernel by approx 5K. RAIDframe now
weighs in at about 162K on i386.

[Having received a definite lack of strenuous objection, a small amount
of strenuous agreement, and some general agreement, this commit is
going ahead because it's now starting to block some other changes I
wish to make.]

Remove most of the support for the concept of "rows" from RAIDframe.
While the "row" interface has been exported to the world, RAIDframe
internals have really only supported a single row, even though they
have feigned support of multiple rows.

Nothing changes in configuration land -- config files still need to
specify a single row, etc. All auto-config structures remain fully
forward/backwards compatible.

The only visible difference to the average user should be a
reduction in the size of a GENERIC kernel (i386) by 4.5K. For those
of us trolling through RAIDframe kernel code, a lot of the driver
configuration code has become a LOT easier to read.

branches: 1.22.2;
constify some

Use RF_DEBUG_DISKQUEUE to eliminate more debuggin printfs that the
vast majority of the world will never see much less care about.

Nuke unlockingOp -- it was only used for the userland version of the
code.

'buf2' is unused. Nuke it.

Everyone and their dog was using RF_ERRORMSG3 to print out the same
sort of error message, over and over again, in different files.
Rather than having the same text repeated in multiple .o files,
create a couple of little functions to do the printing, and save a
bundle of space. Also improves readability of code.

- AuxFunc is no longer used.
- cleanup function prototypes in rf_diskqueue.h

It appears that nothing is using rf_CreateDiskQueueDataFull. Make it even
harder to do so by removing it.

branches: 1.15.8;
add RCSIDs

Step 2 of the disentanglement. We now look to <dev/raidframe/*> for
the stuff that used to live in rf_types.h, rf_raidframe.h, rf_layout.h,
rf_netbsd.h, rf_raid.h, rf_decluster,h, and a few other places.
Believe it or not, when this is all done, things will be cleaner.

No functional changes to RAIDframe.

branches: 1.13.6; 1.13.8; 1.13.10;
Reorganize some comments.

Garbage collect the (already disabled) "random" queue type.

Garbage collect some unused stuff. Include rf_kintf.h for a
function prototype.

- record the pointer to the DiskQueueSW so we can use that when
we hot-add a spare.
- use the above info when adding a spare.

Get recent changes into the tree:
- make component_label variables more consistent (==> clabel)
- re-work incorrect component configuration code
- re-work disk configuration code
- cleanup initial configuration of raidPtr info
- add auto-detection of components and RAID sets (Disabled, for now)
- allow / on RAID sets (Disabled, for now)
- rename "config_disk_queue" to "rf_ConfigureDiskQueue" and properly prototype
in rf_diskqueue.h
- protect some headers with #if _KERNEL (XXX this needs to be fixed properly)
and cleanup header formatting.
- expand the component labels (yes, they should be backward/forward compatible)
- other bits and pieces (some function names are still bogus, and will get
changed soon)

Nuke: #include "rf_threadid.h" and call to rf_get_threadid().

branches: 1.7.2;
At config time, initialize a small amount of space for the disk queues
corresponding to the hot spares. We'll need that space when a spare is
actually added. Huge Thanks to Martin Laubach for helping track this down.

branches: 1.6.2; 1.6.4; 1.6.6;
Phase 2 of the RAIDframe cleanup. The source is now closer to KNF
and is much easier to read. No functionality changes.

RAIDframe cleanup, phase 1. Nuke simulator support, user-land driver,
out-dated comments, and other unneeded stuff. This helps prepare
for cleaning up the rest of the code, and adding new functionality.

No functional changes to the kernel code in this commit.

Use M_RAIDFRAME.

Duh... check that malloc() hasn't returned a NULL *before* we zero the buffer,
not after. Pointed out by Michael Graff.

Fix (potential) referencing of null pointer in (verbose) debugging mode.

RAIDframe, version 1.1, from the Parallel Data Laboratory at
Carnegie Mellon University. Full RAID implementation, including
levels 0, 1, 4, 5, 6, parity logging, and a few other goodies.
Ported to NetBSD by Greg Oster.

fix the function pointer and callback mess:
- callback functions return 0 and their result is not checked; make them void.
- there are two types of callbacks and they used to overload their parameters
and the callback structure; separate them into "function" and "value"
callbacks.
- make the wait function signature consistent.

Introduce PR_ZERO to avoid open-coding memset()s everywhere. OK riastradh@.

- Change the allocation macros to be more like function calls
- Change sizeof(type) -> sizeof(*variable)
- Use macros for the long buffer length allocations
- Remove "bit polishing" memsets() -- do them only once
- Remove unnecessary casts

Thanks to oster@ for finding bugs and testing.

branches: 1.53.56;
convert the diskqueue mutex into a kmutex

branches: 1.52.4; 1.52.6;
Rework/simplify the disk queuing code. A bunch of this was still
holdovers from the simulator and would never be seen/used in-kernel.

branches: 1.51.4; 1.51.6; 1.51.10; 1.51.14;
Mark a buffer `busy` in getnewbuf() when it came from the pool_cache since
its not on a free list.

Also change buf_init() to not automatically mark buffers `busy' since this
only makes sense for bufcache buffers.

Mark all buf_init'd buffers 'busy' on the places where they ought to be
flagged as such to not confuse the buffer cache.

Fixes PR 38923.

branches: 1.50.6; 1.50.10; 1.50.12; 1.50.14;
Merge vmlocking2 to head.

branches: 1.49.2; 1.49.16; 1.49.22; 1.49.24; 1.49.28;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

branches: 1.48.4;
__unused removal on arguments; approved by core.

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

Protect calls to pool_put/pool_get that may occur in interrupt context
with spl used to protect other allocations and frees, or datastructure
element insertion and removal, in adjacent code.

It is almost unquestionably the case that some of the spl()/splx() calls
added here are superfluous, but it really seems wrong to see:

s=splfoo();
/* frob data structure */
splx(s);
pool_put(x);

and if we think we need to protect the first operation, then it is hard
to see why we should not think we need to protect the next. "Better
safe than sorry".

It is also almost unquestionably the case that I missed some pool
gets/puts from interrupt context with my strategy for finding these
calls; use of PR_NOWAIT is a strong hint that a pool may be used from
interrupt context but many callers in the kernel pass a "can wait/can't
wait" flag down such that my searches might not have found them. One
notable area that needs to be looked at is pf.

See also:

http://mail-index.netbsd.org/tech-kern/2006/07/19/0003.html
http://mail-index.netbsd.org/tech-kern/2006/07/19/0009.html

branches: 1.45.18; 1.45.20;
Cleanup the initialization of buffers a bit, and remove some old code
that works around a bug that no longer exists. From yamt. Thanks!
(a bit more cleanup to follow)

initialize necessary members of struct buf. PR/32462 from Reinoud Zandijk.