fix word endings toin -> tion in comments.

s/occured/occurred/ in comments, log messages and man pages.

Extensive mechanical changes to the pools used in RAIDframe.

Alloclist remains not per-RAID, so initialize that pool
separately/differently than the rest.

The remainder of pools in RF_Pools_s are now per-RAID pools. Mostly
mechanical changes to functions to allocate/destroy per-RAID pools.
Needed to make raidPtr available in certain cases to be able to find
the per-RAID pools.

Extend rf_pool_init() to now populate a per-RAID wchan value that is
unique to each pool for a given RAID device.

TODO: Complete the analysis of the minimum number of items that are
required for each pool to allow IO to progress (i.e. so that a request
for pool resources can always be satisfied), and dynamically scale
minimum pool sizes based on RAID configuration.

branches: 1.29.12;
fix the function pointer and callback mess:
- callback functions return 0 and their result is not checked; make them void.
- there are two types of callbacks and they used to overload their parameters
and the callback structure; separate them into "function" and "value"
callbacks.
- make the wait function signature consistent.

branches: 1.28.56;
convert rb_mutex to a kmutex/cv.

re-apply the reverted previous, after fixing the missed wakeup:

convert eq_mutex to a kmutex/cv.
convert ltsleep() used for plain timeout into kpause().

rever the previous for now; it hangs reconstruction.

convert eq_mutex to a kmutex.
convert ltsleep() used for plain timeout into kpause().

branches: 1.24.20; 1.24.26;
Re-work some of the guts of the reconstruction code.

Reconmap used to have one pointer for every reconstruction unit. This
does not scale well in the land of 1TB disks, where some 100MB+ of
"status pointers" are required for typical configurations. Convert
the reconstruction code to use a "sliding status window" which will
scale nicely regardless of the number of stripes/reconstruction units
in the RAID set. Convert the main reconstruction loop to rebuild the
array in chunks rather than in one big lump.

As part of these changes, introduce a function to kick any waiters on
the head separation callback list, and use that in the main
reconstruction event queue to wake up the waiters if things have
stalled. (I believe this may fix a race condition that could occur at
at least at the very end of a disk during reconstruction under heavy
IO load.)

Thanks to Brian Buhrow for all his help, support, and patience in
testing these changes.

branches: 1.23.36; 1.23.38; 1.23.40; 1.23.42;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

branches: 1.22.24; 1.22.26; 1.22.30;
merge ktrace-lwp.

branches: 1.21.2;
- initialize numRUsTotal before we indicate that we are doing a reconstruct.

- make numRUsComplete and numRUsTotal 64-bit quantities like
everything else that records this information.

branches: 1.20.2;
nuke trailing whitespace

Vastly improve the error handling in the case of a read/write error
that occurs during a reconstruction. We go from zero error handling
and likely panicing if something goes amiss, to gracefully bailing and
leaving the system in the best, usable state possible.

- introduce rf_DrainReconEventQueue() to allow easy cleaning of the
reconstruction event queue

- change how we cleanup the floating recon buffers in
rf_FreeReconControl(). Detect the end of the list rather
than traversing according to a count.

- keep track of the number of pending reconstruction writes. In the
event of a read error, use this to wait long enough for the pending
writes to (hopefully) drain.

- more cleanup is still needed on this code, but I didn't want to
start mixing major functional changes with minor cleanups.

XXX: There is a known issue with pool items left outstanding due to
the IO failure, and this can show up in the form of a panic at the
tail end of a shutdown. This problem is much less severe than before
these changes, and the hope/plan is that this problem will go away
once this code gets overhauled again.

branches: 1.18.4; 1.18.6;
continueFunc and continueArg arn't used. Turf. Simplify calls to
rf_GetNextReconEvent().

Re-work the locking mechanisms for reconstruct and PSS structures
such that we don't actually hold a simplelock while we are doing
a pool_get(), but that we still effectively protecting critical code.

This should fix all of the outstanding LOCKDEBUG warnings related to
rebuilding RAID sets.

Minor cleanups. No functional changes.

- fix up initialization of rf_recond_pool
- introduce rf_reconbuffer_pool and teach rf_MakeReconBuffer() to use it

Introduce RF_REVENT_READ_FAILED, RF_REVENT_WRITE_FAILED and RF_REVENT_FORCEREAD_FAILED.
This removes 3 more RF_PANIC()'s (but we'll currently still panic if any of these cases occur).
fix up a few printf's.
XXX: still needs more cleanup and testing (and be taught to not panic).

Nuke a few more 'row'-related bits that were missed in the row-removal.

Nuke a bunch of unused variables:
- node_queue_cond
- quiescent_cond
- eq_cond
- desc->cond
- desc->head
- diskqueue->numWaiting

Nuke rf_print_unable_to_init_cond().
Nuke rf_TerminateDiskQueues prototype from rf_diskqueue.h.

Nuke a forgotten 'row' from struct RF_PerDiskReconCtrl_s.

[Having received a definite lack of strenuous objection, a small amount
of strenuous agreement, and some general agreement, this commit is
going ahead because it's now starting to block some other changes I
wish to make.]

Remove most of the support for the concept of "rows" from RAIDframe.
While the "row" interface has been exported to the world, RAIDframe
internals have really only supported a single row, even though they
have feigned support of multiple rows.

Nothing changes in configuration land -- config files still need to
specify a single row, etc. All auto-config structures remain fully
forward/backwards compatible.

The only visible difference to the average user should be a
reduction in the size of a GENERIC kernel (i386) by 4.5K. For those
of us trolling through RAIDframe kernel code, a lot of the driver
configuration code has become a LOT easier to read.

branches: 1.9.6;
Bye-bye to the completely unused reconCtrlPtr->priorityList.

rf_RegisterReconDoneProc() isn't needed.

This is the last of the 'easy' ones that Krister made me aware of.
Total savings on i386 GENERIC kernel: 13151 bytes
RAIDframe in GENERIC is now at: 179033
Thanks again Krister!

'char *arrived' was such a nice variable. Too bad all we ever did
with it was malloc/free space for it. *poof*

Step 2 of the disentanglement. We now look to <dev/raidframe/*> for
the stuff that used to live in rf_types.h, rf_raidframe.h, rf_layout.h,
rf_netbsd.h, rf_raid.h, rf_decluster,h, and a few other places.
Believe it or not, when this is all done, things will be cleaner.

No functional changes to RAIDframe.

branches: 1.5.4; 1.5.6; 1.5.8;
- Add a mechanism for obtaining finer-grained 'progress' information
regarding reconstructs, copybacks, etc.

- RAID 0 doesn't do copybacks, but don't make raidctl sweat about it.

branches: 1.4.8; 1.4.16;
Update for recent changes including component label support, clean
bits, rebuilding components in-place, adding hot spares, shutdownhooks, etc.

Phase 2 of the RAIDframe cleanup. The source is now closer to KNF
and is much easier to read. No functionality changes.

RAIDframe cleanup, phase 1. Nuke simulator support, user-land driver,
out-dated comments, and other unneeded stuff. This helps prepare
for cleaning up the rest of the code, and adding new functionality.

No functional changes to the kernel code in this commit.

RAIDframe, version 1.1, from the Parallel Data Laboratory at
Carnegie Mellon University. Full RAID implementation, including
levels 0, 1, 4, 5, 6, parity logging, and a few other goodies.
Ported to NetBSD by Greg Oster.