Implement hot removal of spares and components. From manu@.

Implement a long desired feature of automatically incorporating
a used spare into the array after a reconstruct.

Given the configuration:
Components:
/dev/wd0e: failed
/dev/wd1e: optimal
/dev/wd2e: optimal
Spares:
/dev/wd3e: spare

Running 'raidctl -F /dev/wd0e raid0' will now result in the
following configuration after a successful rebuild:
Components:
/dev/wd3e: optimal
/dev/wd1e: optimal
/dev/wd2e: optimal
No spares.

Thanks to manu@ for the development of the initial set of changes
which allowed the changes to automatically incorporate a used spare
to come to fruition. Thanks also to manu@ for useful discussions
about and additional testing of these changes.

Revision 1.104 actually fixed the issues that were preventing
us from freeing the ReconControl structures. So free them
and thus also prevent a panic on shutdown due to items not
being correctly returned to the pool.

Thanks to manu@ for report of the panic, and for initial testing
of the changes.

XXX pullup-9
XXX pullup-10

branches: 1.127.10;
rf_CreateDiskQueueData() no longer uses waitflag, and will always succeed.
Cleanup the error path for the (no longer needed) PR_NOWAIT cases.

Extensive mechanical changes to the pools used in RAIDframe.

Alloclist remains not per-RAID, so initialize that pool
separately/differently than the rest.

The remainder of pools in RF_Pools_s are now per-RAID pools. Mostly
mechanical changes to functions to allocate/destroy per-RAID pools.
Needed to make raidPtr available in certain cases to be able to find
the per-RAID pools.

Extend rf_pool_init() to now populate a per-RAID wchan value that is
unique to each pool for a given RAID device.

TODO: Complete the analysis of the minimum number of items that are
required for each pool to allow IO to progress (i.e. so that a request
for pool resources can always be satisfied), and dynamically scale
minimum pool sizes based on RAID configuration.

branches: 1.125.4;
Fix a long long-standing off-by-one error in computing lastPSID.

SUsPerPU is only really supported for a value of 1, and since the
first PSID is 0, the last will be numStripe-1. Also update the
setting of pending_writes to reflect the change to lastPSID.

Needs pullups to -8 and -9.

branches: 1.124.8;
Switch to vn_bdev_open* functions.

fix the function pointer and callback mess:
- callback functions return 0 and their result is not checked; make them void.
- there are two types of callbacks and they used to overload their parameters
and the callback structure; separate them into "function" and "value"
callbacks.
- make the wait function signature consistent.

branches: 1.122.4;
- Change the allocation macros to be more like function calls
- Change sizeof(type) -> sizeof(*variable)
- Use macros for the long buffer length allocations
- Remove "bit polishing" memsets() -- do them only once
- Remove unnecessary casts

Thanks to oster@ for finding bugs and testing.

branches: 1.121.12; 1.121.20;


Fix a long-standing bug related to rebooting while a
reconstruct-to-spare is underway but not yet complete.

The issue was that a component was being marked as a used_spare when
the rebuild started, not when the rebuild was actually finished.
Marking it as a used_spare meant that the component label on the spare
was being updated such that after a reboot the component would be
considered up-to-date, regardless of whether the rebuild actually
completed!

This fix includes:
1) Add an additional state "rf_ds_rebuilding_spare" which is used
to denote that a spare is currently being rebuilt from the live
components.
2) Update the comments on the disk states, which were out-of-sync
with reality.
3) When rebuilding to a spare component, that spare now enters the
state rf_ds_rebuilding_spare instead of the state rf_ds_used_spare.
4) When the rebuild is actually complete then the spare component
enters the rf_ds_used_spare state. rf_ds_used_spare is now used
exclusively for the case where the rebuilding to the spare has
completed successfully.

XXX: Someday we need to teach raidctl(8) about this new state, and
take out the backwards compatibility code in rf_netbsdkintf.c (see
RAIDFRAME_GET_INFO in raidioctl()). For today, this fix needs to be
generic enough that it can get backported without major grief.

XXX: Needs pullup to netbsd-5*, netbsd-6*, and netbsd-7

Fixes PR#49244.

branches: 1.120.2;
Change dk_lookup() to return an anonymous vnode not associated with
any file system. Change all consumers of dk_lookup() to get the
device from "v_rdev" instead of VOP_GETATTR() as specfs does not
support VOP_GETATTR(). Devices obtained with dk_lookup() will no
longer disappear on forced unmounts.

Fix for PR kern/48849 (root mirror raid fails on shutdown)

Welcome to 6.99.44

branches: 1.119.10;
fix parens in a message

branches: 1.118.2;
Add logic to the main reconstruction loop to handle RAID5 with rotated
spares. While here, observe that we were actually doing one more
stripe than we thought we were, and correct that too (it didn't matter
for non-RAID5_RS, but it definitely does for RAID5_RS). Add some
bounds-checking at the beginning to handle the case where the number
of stripes in the set is smaller than the sliding reconstruction window.

XXX: this problem likely needs to be fixed for PARITY_DECLUSTERING too.

branches: 1.117.2; 1.117.6; 1.117.8;
Change the vnode locking protocol of VOP_GETATTR() to request at least
a shared lock. Make all calls outside of file systems respect it.

The calls from file systems need review.

No objections from tech-kern.

Address part of PR kern/44972. From YAMAMOTO Takashi. Thanks!

rf_ReconstructInPlace: don't leave a vnode open on errors.
fixes a part of PR/44972.

Suggested to oster@ and approved via private e-mail as a help to
people who are getting reconstruction failures.

convert the main raidPtr mutex to a kmutex, and add a couple of cv's to
cover the old sleep/wakeup points for adding_hot_spare and waitForReconCond.
convert all remaining simple_lock's to kmutexes (they're not used or compiled
right now... even with all options enabled) and remove the support for them.

this leaves just a pair of tsleep()/wakeup() calls using old scheduling APIs.

convert rb_mutex to a kmutex/cv.

Define accessors for number of blocks and partition size in the
component label and use them where appropriate. Disscussed on tech-kern.

branches: 1.110.2; 1.110.4;
Introduce struct pathbuf. This is an abstraction to hold a pathname
and the metadata required to interpret it. Callers of namei must now
create a pathbuf and pass it to NDINIT (instead of a string and a
uio_seg), then destroy the pathbuf after the namei session is
complete.

Update all namei call sites accordingly. Add a pathbuf(9) man page and
update namei(9).

The pathbuf interface also now appears in a couple of related
additional places that were passing string/uio_seg pairs that were
later fed into NDINIT. Update other call sites accordingly.

add support for >2TB raid devices.

- add two new members to the component label:
u_int numBlocksHi
u_int partitionSizeHi
and store the top 32 bits of the real number of blocks and
partition size. modify rf_print_component_label(),
rf_does_it_fit(), rf_AutoConfigureDisks() and
rf_ReconstructFailedDiskBasic().

- call disk_blocksize() after disk_attach() [ from mlelstv ]

- shift the block number relative to DEV_BSHIFT in raidstart()
and InitBP() so that accesses work for non 512-byte devices.
[ from mlelstv ]

- update rf_getdisksize() to use the new getdisksize() [ from
mlelstv. this part needs a separate change for netbsd-5. ]


reviewed by: oster, christos and darrenr

branches: 1.108.2; 1.108.4;
Finally commit the RAIDframe parity map Summer Of Code project.

Drastically reduces the amount of time spent rewriting parity after an
unclean shutdown by keeping better track of which regions might have had
outstanding writes. Enabled by default; can be disabled on a per-set
basis, or tuned, with the new raidctl(8) commands.

Discussed on tech-kern@ to a general air of approval; exhortations to
commit from mrg@, christos@, and others.

Thanks to Google for their sponsorship, oster@ for mentoring the
project, assorted developers for trying very hard to break it, and
probably more I'm forgetting.

If we see a RF_RECON_WRITE_ERROR event we know a write has finished and
we need to account for that. Failure to do so means we can end up
waiting forever for writes we think are outstanding, but which have
already completed.

Addresses the RAIDframe part of PR#40569. Thanks to Matthias Scheler
for reporting the issue and verifying the fix.

branches: 1.106.2;
When unconfiguring an array where a reconstruct is in progress, abort
the reconstruct and wait for IOs to drain before pulling the plug.

Should fix the panic reported by der Mouse on tech-kern.

branches: 1.105.2; 1.105.4;
Nuke unneeded printf(). Spotted by pooka@.

branches: 1.104.4;
Re-work some of the guts of the reconstruction code.

Reconmap used to have one pointer for every reconstruction unit. This
does not scale well in the land of 1TB disks, where some 100MB+ of
"status pointers" are required for typical configurations. Convert
the reconstruction code to use a "sliding status window" which will
scale nicely regardless of the number of stripes/reconstruction units
in the RAID set. Convert the main reconstruction loop to rebuild the
array in chunks rather than in one big lump.

As part of these changes, introduce a function to kick any waiters on
the head separation callback list, and use that in the main
reconstruction event queue to wake up the waiters if things have
stalled. (I believe this may fix a race condition that could occur at
at least at the very end of a disk during reconstruction under heavy
IO load.)

Thanks to Brian Buhrow for all his help, support, and patience in
testing these changes.

branches: 1.103.2; 1.103.4; 1.103.6;
A forced recon read should not default to indicating that the reads
for that disk have stopped, since this will bump us out of the normal
reconstruction loop prematurely.

Fixes the (mostly cosmetic) bug where the reconstruction
status values stop updating, and from raidctl it appears that
reconstruction has totally stalled (which it actually hasn't -- the
reconstruction does complete properly, but not in the normal way).

Print out the status value if a reconstruction read fails.
Don't print out write promotions during reconstruct unless
we are debugging reconstructs.

branches: 1.101.6;
In a land before time, when kernel processes roamed the system, we
needed to keep track of the kernel process that opened a device in
order to close it with the right credentials. Flash forward to today
where curlwp is now quite sufficient.

Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start. In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern

branches: 1.99.6;
Fix wording in a comment and correct a debug line. From Olivier Cherrier
(via private mail). Thanks!

branches: 1.98.4; 1.98.6; 1.98.8;
Fix fallout from recent kthread changes.

branches: 1.97.2;
Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements

Change dk_lookup() to accept an additional argument of the type enum uio_seg
that tells whether the given path is in user space or kernel space, so it
can tell NDINIT().

While the raidframe calls were ok, both ccd(4) and cgd(4) were passing
pointers to user space data, which leads to strange error on i386, as
reported by Jukka Salmi on current-users.

The issue has been there since last august, I'm actually a bit surprised
that no one in the meantime has used ccd(4) or cgd(4) on an arch where it
would have simply faulted.

branches: 1.95.2; 1.95.8; 1.95.10; 1.95.16;
__unused removal on arguments; approved by core.

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

branches: 1.93.2; 1.93.4;
- use dk_lookup instead of our home-spun version.
- allow raid to be configured in a wedge
- allow wedges to be configured in a raid
- add autoconfiguration of wedges in a raid

- Use the LWP cached credentials where sane.
- Minor cosmetic changes.

integrate kauth.

branches: 1.90.4; 1.90.6; 1.90.8; 1.90.10; 1.90.12;
merge ktrace-lwp.

If rf_SubmitReconBuffer indicates the submission was blocked (for
whatever reason), return 0 instead of the default
RF_RECON_READ_STOPPED. Returning RF_RECON_READ_STOPPED would result
in rf_ContinueReconstructFailedDisk() thinking that the given
component was "done" and breaking out of the main reconstruction loop
far too early. Reconstruction still worked correctly as long as there
were no errors, but RAIDframe wouldn't be in a position to properly
handle read/write errors during reconstruction.

This fixes the "raidctl's progress bar spins at 0% until
reconstruction finishes" problem.

branches: 1.88.2;
- initialize numRUsTotal before we indicate that we are doing a reconstruct.

- make numRUsComplete and numRUsTotal 64-bit quantities like
everything else that records this information.

branches: 1.87.2;
nuke trailing whitespace

The 'next' argument to rf_CreateDiskQueueData is always NULL. Since
there is no particular reason to pass an extra NULL argument, turf it,
and initialize p->next to NULL within the function.

Add a 'waitflag' argument to rf_CreateDiskQueueData() and use it to
determine if we are willing to wait for memory to come from the
diskqueuedata (dqd) and bufpool pools. Cleanup the mess related to
code calling rf_CreateDiskQueueData() with different expectations
(and/or blatent disregard) of what might happen if there were
insufficient pool resources.

It's not a bad idea to update the component labels whether or not the
reconstruction was successful.

rf_GetNextReconEvent() *will* return a valid event, so no need for
the assert. (we'd have panic'ed in there long before this assert
if that wasn't the case).

Minor whitespace changes.

Vastly improve the error handling in the case of a read/write error
that occurs during a reconstruction. We go from zero error handling
and likely panicing if something goes amiss, to gracefully bailing and
leaving the system in the best, usable state possible.

- introduce rf_DrainReconEventQueue() to allow easy cleaning of the
reconstruction event queue

- change how we cleanup the floating recon buffers in
rf_FreeReconControl(). Detect the end of the list rather
than traversing according to a count.

- keep track of the number of pending reconstruction writes. In the
event of a read error, use this to wait long enough for the pending
writes to (hopefully) drain.

- more cleanup is still needed on this code, but I didn't want to
start mixing major functional changes with minor cleanups.

XXX: There is a known issue with pool items left outstanding due to
the IO failure, and this can show up in the form of a panic at the
tail end of a shutdown. This problem is much less severe than before
these changes, and the hope/plan is that this problem will go away
once this code gets overhauled again.

branches: 1.81.2;
Torch some #define's missed in last commit.

Reconstruction Descriptors are only allocated once per reconstruction,
and don't need their own pool or freelist or anything fancier than a
malloc/free.

ForceReconReadDoneProc() needs a return after doing the first
rf_CauseReconEvent().

branches: 1.78.2;
The switch() in rf_ContinueReconstructFailedDisk() is never actually
used in non-simulation code, and thus is just wasting space (and
making the code more confusing to read!). Turf the switch, left-shift
the indentation of code, and nuke 'state' field of struct RF_RaidReconDesc_s.

No real functional changes.

continueFunc and continueArg arn't used. Turf. Simplify calls to
rf_GetNextReconEvent().

branches: 1.76.4;
Re-work the locking mechanisms for reconstruct and PSS structures
such that we don't actually hold a simplelock while we are doing
a pool_get(), but that we still effectively protecting critical code.

This should fix all of the outstanding LOCKDEBUG warnings related to
rebuilding RAID sets.

- don't use rf_PrintUserStats() for recon statistics.
rf_PrintUserStats() was mean for the simulator, and doesn't provide
any real info in kernel-space, especially for reconstructs.
Reconstructing actually renders the stats even more useless, since it
resets them all to zero before the reconstruct starts!

- since rf_PrintUserStats() is no longer used, nuke it along with the
routines that feed it. Nothing was using this code, and if we ever
need it again, we know where to find it.

- Introduce rf_pools which contains all of the various global pools used
by RAIDframe. Convert all other RAIDframe global pools to use pools
defined within this new structure.
- Introduce rf_pool_init(), used for initializing a single pool in
RAIDframe. Teach each of the configuration routines to use
rf_pool_init().
- Cleanup a few pool-related comments.
- Cleanup revent initialization and #defines.
- Add a missing pool_destroy() for the reconbuffer pool.

(Saves another 1K off of an i386 GENERIC kernel, and makes
stuff a lot more readable)

- fix up initialization of rf_recond_pool
- introduce rf_reconbuffer_pool and teach rf_MakeReconBuffer() to use it

Use RF_INCLUDE_PARITY_DECLUSTERING_DS to #if-out more unneeded bits.
(We can't do RF_DISTRIBUTE_SPARE bits without the parity declustering stuff.)

Nuke some unnecessary casts. No functional changes.

Introduce RF_REVENT_READ_FAILED, RF_REVENT_WRITE_FAILED and RF_REVENT_FORCEREAD_FAILED.
This removes 3 more RF_PANIC()'s (but we'll currently still panic if any of these cases occur).
fix up a few printf's.
XXX: still needs more cleanup and testing (and be taught to not panic).

Cleanup function prototypes.

- cleanup memory allocation in rf_AllocPSStatus()
- change function signature of rf_LookupRUStatus(). The last argument
is now a pointer to a new PSS, in case one is needed. Rather than
having rf_LookupRUStatus() allocate a new PSS, we pre-allocate one
beforehand, where necessary, just in case.
- change callers of rf_lookupRUStatus() to deal with the new way of
calling rf_lookupRUStatus().

[no improvement or worsening of parity rebuild/initialization performance.]

Use RF_ACC_TRACE to #if out more chunks of code related only
to access tracing. (not turned on yet)

Adjust _rf_ShutdownCreate() so that it is willing to wait for more
memory. Since we only now ever "return(0)", just return (void)
instead.

Cleanup all uses of rf_ShutdownCreate() to not worry about
it ever failing. Shaves another 600 bytes off of an i386 GENERIC kernel.

raidPtr->reconControl->percentCompleted only gets used in one
debugging printf, and in rf_netbsdkintf.c. We can do the calculations
inside of RF_DEBUG_RECON for the one debugging printf, and only
perform the percentCompleted calculation "on demand" in the
rf_netbsdkintf.c case. Shaves a few more bytes off an i386 GENERIC
kernel, and ever-so-slightly decreases the amount of work performed
during a reconstruct.

Add in a bunch of RF_SIGNAL_COND()'s that were missing. Tidy up
a few lines.

Left-shift another else{} chunk. No functional changes.

left-shift the "else" part of the if(!lp_SubmitReconBuffer) condition.
Cleanup. No real functional changes, just more readable.

Negate a condition, and flip if/else parts. Preparation for left-shifting
the (now) else part. No real functional change.

Some days you wonder if some of the function declaration consistency
was just an accident in the first place. Cleanup function decls and
a few comments. [ok.. so I wasn't going to fix this many.. but once
you're on a roll....]

Let's see... raidPtr->recon_done_procs is never set to anything
(other than NULL when raidPtr is initialized). That means
SignalReconDone() never does anything useful. Bye-bye!

Say good-bye to recon_done_procs and recon_done_procs_mutex (and its
initializer) as well.

- first kick at a major reworking of RAIDframe's memory allocation code:
- all freelists converted to pools
- initialization of structure members in certain cases where
code was relying on specific allocation and usage properties
to keep structures in a "known state" (that doesn't work with
pools!).
- make most pool_get() be "PR_WAITOK" until they can be analyzed
further, and/or have proper error handling added.
- all RF_Mallocs zero the space returned, so there is no difference
between RF_Calloc and RF_Malloc. In fact, all the RF_Calloc()'s
do is tend to do is get things horribly confused.
Make RF_Malloc() the "general memory allocator", with
RF_MallocAndAdd() the "general memory allocator with
allocation list".
- some of these RF_Malloc's et al. are destined to disappear.
- remove rf_rdp_freelist entirely (it's not used anywhere!)
- remove: #include "rf_freelist.h"
- to the files that were relying on the above, add: #include "rf_general.h"
- add: #include "rf_debugMem.h" to rf_shutdown.h to make it happy
about the loss of: #include "rf_freelist.h".

This shrinks an i386 GENERIC kernel by approx 5K. RAIDframe now
weighs in at about 162K on i386.

[Having received a definite lack of strenuous objection, a small amount
of strenuous agreement, and some general agreement, this commit is
going ahead because it's now starting to block some other changes I
wish to make.]

Remove most of the support for the concept of "rows" from RAIDframe.
While the "row" interface has been exported to the world, RAIDframe
internals have really only supported a single row, even though they
have feigned support of multiple rows.

Nothing changes in configuration land -- config files still need to
specify a single row, etc. All auto-config structures remain fully
forward/backwards compatible.

The only visible difference to the average user should be a
reduction in the size of a GENERIC kernel (i386) by 4.5K. For those
of us trolling through RAIDframe kernel code, a lot of the driver
configuration code has become a LOT easier to read.

branches: 1.56.2;
Back out the lwp/ktrace changes. They contained a lot of colateral damage,
and need to be examined and discussed more.

Pass lwp pointers throughtout the kernel, as required, so that the lwpid can
be inserted into ktrace records. The general change has been to replace
"struct proc *" with "struct lwp *" in various function prototypes, pass
the lwp through and use l_proc to get the process pointer when needed.

Bump the kernel rev up to 1.6V

Remove an assigned-to but unused variable.

Use 'void *' instead of 'caddr_t' in prototypes of VOP_IOCTL, VOP_FCNTL
and VOP_ADVLOCK, delete casts from callers (and some to copyin/out).

constify some

For reconstructs, move checks for failed components to before the
kernel threads are created.

Cleanup more printfs.

After a rebuild-in-place, a reconstruct, or a copyback, we should
really be updating the component labels.

Improve and/or re-arrange a number of locks. While much of the locking is
still a mess, and there are a number of unresolved issues here, this
gets us closer to being happier in LOCKDEBUG land.

Add a missing RF_LOCK_MUTEX().

Introduce a temp variable, and allocate the ReconCtrl structure before
we protect raidPtr. One less thing for LOCKDEBUG to complain about.

Nuke "baddisk". Thanks to Simon B.

rf_RegisterReconDoneProc() isn't needed.

This is the last of the 'easy' ones that Krister made me aware of.
Total savings on i386 GENERIC kernel: 13151 bytes
RAIDframe in GENERIC is now at: 179033
Thanks again Krister!

Introduce and use RF_DEBUG_PSS, and save a few more bytes.

One signal will do, thanks.

Cast the RF_DEBUG_RECON net a little wider.

Rename RF_DEBUG_RECONBUFFER to RF_DEBUG_RECON in order to facilitate
disabling other stuff without having to introduce another #define.

Cleanup some comments.

rf_CheckFloatingRbufCount() is only really useful when debugging the
reconstruct buffer stuff. #if it out in the general case.

Cleanup some printf's, and disable some (debugging) output.

Everyone and their dog was using RF_ERRORMSG3 to print out the same
sort of error message, over and over again, in different files.
Rather than having the same text repeated in multiple .o files,
create a couple of little functions to do the printing, and save a
bundle of space. Also improves readability of code.

Disallow 'reconstruct-in-place' on a component that has failed
and has already been reconstructed to a hot spare.

Nuke a redundant wakeup().

branches: 1.33.8;
Move a bunch of debugging stuff to be only used if DEBUG is turned on.

don't need <sys/types.h> when including <sys/param.h>

add RCSIDs

Step 2 of the disentanglement. We now look to <dev/raidframe/*> for
the stuff that used to live in rf_types.h, rf_raidframe.h, rf_layout.h,
rf_netbsd.h, rf_raid.h, rf_decluster,h, and a few other places.
Believe it or not, when this is all done, things will be cleaner.

No functional changes to RAIDframe.

branches: 1.29.2;
bzero -> memset

branches: 1.28.2;
It's silly to need a parity rebuild after a reconstruction has completed.
If we've just reconstructed a disk, then the parity is known to
be correct. (XXX doesn't hold for RAID 6!)

branches: 1.27.2;
Ensure we update the 'partitionSize' field of the component labels
when doing a reconstruct or a copyback. If we don't, junk might be
there, and that could cause the component to be not correctly
autoconfigured on reboot. Thanks to Simon Burge for helping track this down.

branches: 1.26.2;
Merge rf_update_component_labels() and rf_final_update_component_labels().

Oops.. reconstruction percentages were being reported incorrectly.
Thanks to Manuel Bouyer for noting this.

Umm.. Complete is not equal to 'left to do'. Fix the math.

- Add a mechanism for obtaining finer-grained 'progress' information
regarding reconstructs, copybacks, etc.

- RAID 0 doesn't do copybacks, but don't make raidctl sweat about it.

branches: 1.22.2;
Fix doubled 'the's in comments.

Create a new rf_close_component() to handle vnode operations for closing
components. Teach rf_UnconfigureVnodes() how to use it, and tell
the copyback and reconstruction code about it too.

When we close autoconfigured components, we need to note that they
are no longer in 'autoconfigured' status.

Fix a (slightly) bogus status message.

Make sure we close auto-configured components appropriately when
attempting a rebuild-in-place.

Be more aggressive about updating component labels in the event
of a real component failure (or a simulated failure):
- add 'numNewFailures' to keep track of the number of disk failures
since mod_counter was last updated for each component label.
- make sure we call rf_update_component_labels() upon any component failure,
real or simulated.

Do a better job of (re)initializing the component labels after
a reconstruct or a copyback.

Get recent changes into the tree:
- make component_label variables more consistent (==> clabel)
- re-work incorrect component configuration code
- re-work disk configuration code
- cleanup initial configuration of raidPtr info
- add auto-detection of components and RAID sets (Disabled, for now)
- allow / on RAID sets (Disabled, for now)
- rename "config_disk_queue" to "rf_ConfigureDiskQueue" and properly prototype
in rf_diskqueue.h
- protect some headers with #if _KERNEL (XXX this needs to be fixed properly)
and cleanup header formatting.
- expand the component labels (yes, they should be backward/forward compatible)
- other bits and pieces (some function names are still bogus, and will get
changed soon)

Nuke dependencies on rf_cpuutils.h.

Nuke unused debugging stuff. Clean up a whole bunch of comments.

- move a bunch of function prototypes to rf_kintf.h
- general cleanup of a number of prototypes that were scattered around.

Nuke #if 0'ed code.

- nuke calls to rf_get_threadid() and associated #include
- change a bunch of debugging printfs from
"[%d] ...", tid (where tid is the "thread id")
to
"raid%d: ...", raidPtr->raidid
- other minor rototillage

- update RF_CREATE_THREAD to handle a 'process name' argument.
- fire up a new thread for parity re-writes, copybacks, and reconstructs.
The ioctl's which trigger these actions now return immediately.
- add progress accounting for the above actions.
- minor rototillage of rf_netbsdkintf.c to deal with all of the above.

branches: 1.8.2;
Remove a 'struct proc *'-passing abomination that's been bugging me
for quite some time.

rf_sys.h does not need to be #included in any of these files, and, actually,
is no longer needed at all.

Clean up reconstruction accounting a bit. While it worked before, it was
slightly broken in the case where the RAID set did not support reconstruction.

branches: 1.5.2;
Update for recent changes including component label support, clean
bits, rebuilding components in-place, adding hot spares, shutdownhooks, etc.

Phase 2 of the RAIDframe cleanup. The source is now closer to KNF
and is much easier to read. No functionality changes.

Nuke more bits of RAIDframe "demo" code. We're not "demoing" here,
we're doing the Real Thing!

RAIDframe cleanup, phase 1. Nuke simulator support, user-land driver,
out-dated comments, and other unneeded stuff. This helps prepare
for cleaning up the rest of the code, and adding new functionality.

No functional changes to the kernel code in this commit.

RAIDframe, version 1.1, from the Parallel Data Laboratory at
Carnegie Mellon University. Full RAID implementation, including
levels 0, 1, 4, 5, 6, parity logging, and a few other goodies.
Ported to NetBSD by Greg Oster.

Revision 1.104 actually fixed the issues that were preventing
us from freeing the ReconControl structures. So free them
and thus also prevent a panic on shutdown due to items not
being correctly returned to the pool.

Thanks to manu@ for report of the panic, and for initial testing
of the changes.

XXX pullup-9
XXX pullup-10

branches: 1.127.10;
rf_CreateDiskQueueData() no longer uses waitflag, and will always succeed.
Cleanup the error path for the (no longer needed) PR_NOWAIT cases.

Extensive mechanical changes to the pools used in RAIDframe.

Alloclist remains not per-RAID, so initialize that pool
separately/differently than the rest.

The remainder of pools in RF_Pools_s are now per-RAID pools. Mostly
mechanical changes to functions to allocate/destroy per-RAID pools.
Needed to make raidPtr available in certain cases to be able to find
the per-RAID pools.

Extend rf_pool_init() to now populate a per-RAID wchan value that is
unique to each pool for a given RAID device.

TODO: Complete the analysis of the minimum number of items that are
required for each pool to allow IO to progress (i.e. so that a request
for pool resources can always be satisfied), and dynamically scale
minimum pool sizes based on RAID configuration.

branches: 1.125.4;
Fix a long long-standing off-by-one error in computing lastPSID.

SUsPerPU is only really supported for a value of 1, and since the
first PSID is 0, the last will be numStripe-1. Also update the
setting of pending_writes to reflect the change to lastPSID.

Needs pullups to -8 and -9.

branches: 1.124.8;
Switch to vn_bdev_open* functions.

fix the function pointer and callback mess:
- callback functions return 0 and their result is not checked; make them void.
- there are two types of callbacks and they used to overload their parameters
and the callback structure; separate them into "function" and "value"
callbacks.
- make the wait function signature consistent.

branches: 1.122.4;
- Change the allocation macros to be more like function calls
- Change sizeof(type) -> sizeof(*variable)
- Use macros for the long buffer length allocations
- Remove "bit polishing" memsets() -- do them only once
- Remove unnecessary casts

Thanks to oster@ for finding bugs and testing.

branches: 1.121.12; 1.121.20;


Fix a long-standing bug related to rebooting while a
reconstruct-to-spare is underway but not yet complete.

The issue was that a component was being marked as a used_spare when
the rebuild started, not when the rebuild was actually finished.
Marking it as a used_spare meant that the component label on the spare
was being updated such that after a reboot the component would be
considered up-to-date, regardless of whether the rebuild actually
completed!

This fix includes:
1) Add an additional state "rf_ds_rebuilding_spare" which is used
to denote that a spare is currently being rebuilt from the live
components.
2) Update the comments on the disk states, which were out-of-sync
with reality.
3) When rebuilding to a spare component, that spare now enters the
state rf_ds_rebuilding_spare instead of the state rf_ds_used_spare.
4) When the rebuild is actually complete then the spare component
enters the rf_ds_used_spare state. rf_ds_used_spare is now used
exclusively for the case where the rebuilding to the spare has
completed successfully.

XXX: Someday we need to teach raidctl(8) about this new state, and
take out the backwards compatibility code in rf_netbsdkintf.c (see
RAIDFRAME_GET_INFO in raidioctl()). For today, this fix needs to be
generic enough that it can get backported without major grief.

XXX: Needs pullup to netbsd-5*, netbsd-6*, and netbsd-7

Fixes PR#49244.

branches: 1.120.2;
Change dk_lookup() to return an anonymous vnode not associated with
any file system. Change all consumers of dk_lookup() to get the
device from "v_rdev" instead of VOP_GETATTR() as specfs does not
support VOP_GETATTR(). Devices obtained with dk_lookup() will no
longer disappear on forced unmounts.

Fix for PR kern/48849 (root mirror raid fails on shutdown)

Welcome to 6.99.44

branches: 1.119.10;
fix parens in a message

branches: 1.118.2;
Add logic to the main reconstruction loop to handle RAID5 with rotated
spares. While here, observe that we were actually doing one more
stripe than we thought we were, and correct that too (it didn't matter
for non-RAID5_RS, but it definitely does for RAID5_RS). Add some
bounds-checking at the beginning to handle the case where the number
of stripes in the set is smaller than the sliding reconstruction window.

XXX: this problem likely needs to be fixed for PARITY_DECLUSTERING too.

branches: 1.117.2; 1.117.6; 1.117.8;
Change the vnode locking protocol of VOP_GETATTR() to request at least
a shared lock. Make all calls outside of file systems respect it.

The calls from file systems need review.

No objections from tech-kern.

Address part of PR kern/44972. From YAMAMOTO Takashi. Thanks!

rf_ReconstructInPlace: don't leave a vnode open on errors.
fixes a part of PR/44972.

Suggested to oster@ and approved via private e-mail as a help to
people who are getting reconstruction failures.

convert the main raidPtr mutex to a kmutex, and add a couple of cv's to
cover the old sleep/wakeup points for adding_hot_spare and waitForReconCond.
convert all remaining simple_lock's to kmutexes (they're not used or compiled
right now... even with all options enabled) and remove the support for them.

this leaves just a pair of tsleep()/wakeup() calls using old scheduling APIs.

convert rb_mutex to a kmutex/cv.

Define accessors for number of blocks and partition size in the
component label and use them where appropriate. Disscussed on tech-kern.

branches: 1.110.2; 1.110.4;
Introduce struct pathbuf. This is an abstraction to hold a pathname
and the metadata required to interpret it. Callers of namei must now
create a pathbuf and pass it to NDINIT (instead of a string and a
uio_seg), then destroy the pathbuf after the namei session is
complete.

Update all namei call sites accordingly. Add a pathbuf(9) man page and
update namei(9).

The pathbuf interface also now appears in a couple of related
additional places that were passing string/uio_seg pairs that were
later fed into NDINIT. Update other call sites accordingly.

add support for >2TB raid devices.

- add two new members to the component label:
u_int numBlocksHi
u_int partitionSizeHi
and store the top 32 bits of the real number of blocks and
partition size. modify rf_print_component_label(),
rf_does_it_fit(), rf_AutoConfigureDisks() and
rf_ReconstructFailedDiskBasic().

- call disk_blocksize() after disk_attach() [ from mlelstv ]

- shift the block number relative to DEV_BSHIFT in raidstart()
and InitBP() so that accesses work for non 512-byte devices.
[ from mlelstv ]

- update rf_getdisksize() to use the new getdisksize() [ from
mlelstv. this part needs a separate change for netbsd-5. ]


reviewed by: oster, christos and darrenr

branches: 1.108.2; 1.108.4;
Finally commit the RAIDframe parity map Summer Of Code project.

Drastically reduces the amount of time spent rewriting parity after an
unclean shutdown by keeping better track of which regions might have had
outstanding writes. Enabled by default; can be disabled on a per-set
basis, or tuned, with the new raidctl(8) commands.

Discussed on tech-kern@ to a general air of approval; exhortations to
commit from mrg@, christos@, and others.

Thanks to Google for their sponsorship, oster@ for mentoring the
project, assorted developers for trying very hard to break it, and
probably more I'm forgetting.

If we see a RF_RECON_WRITE_ERROR event we know a write has finished and
we need to account for that. Failure to do so means we can end up
waiting forever for writes we think are outstanding, but which have
already completed.

Addresses the RAIDframe part of PR#40569. Thanks to Matthias Scheler
for reporting the issue and verifying the fix.

branches: 1.106.2;
When unconfiguring an array where a reconstruct is in progress, abort
the reconstruct and wait for IOs to drain before pulling the plug.

Should fix the panic reported by der Mouse on tech-kern.

branches: 1.105.2; 1.105.4;
Nuke unneeded printf(). Spotted by pooka@.

branches: 1.104.4;
Re-work some of the guts of the reconstruction code.

Reconmap used to have one pointer for every reconstruction unit. This
does not scale well in the land of 1TB disks, where some 100MB+ of
"status pointers" are required for typical configurations. Convert
the reconstruction code to use a "sliding status window" which will
scale nicely regardless of the number of stripes/reconstruction units
in the RAID set. Convert the main reconstruction loop to rebuild the
array in chunks rather than in one big lump.

As part of these changes, introduce a function to kick any waiters on
the head separation callback list, and use that in the main
reconstruction event queue to wake up the waiters if things have
stalled. (I believe this may fix a race condition that could occur at
at least at the very end of a disk during reconstruction under heavy
IO load.)

Thanks to Brian Buhrow for all his help, support, and patience in
testing these changes.

branches: 1.103.2; 1.103.4; 1.103.6;
A forced recon read should not default to indicating that the reads
for that disk have stopped, since this will bump us out of the normal
reconstruction loop prematurely.

Fixes the (mostly cosmetic) bug where the reconstruction
status values stop updating, and from raidctl it appears that
reconstruction has totally stalled (which it actually hasn't -- the
reconstruction does complete properly, but not in the normal way).

Print out the status value if a reconstruction read fails.
Don't print out write promotions during reconstruct unless
we are debugging reconstructs.

branches: 1.101.6;
In a land before time, when kernel processes roamed the system, we
needed to keep track of the kernel process that opened a device in
order to close it with the right credentials. Flash forward to today
where curlwp is now quite sufficient.

Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start. In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern

branches: 1.99.6;
Fix wording in a comment and correct a debug line. From Olivier Cherrier
(via private mail). Thanks!

branches: 1.98.4; 1.98.6; 1.98.8;
Fix fallout from recent kthread changes.

branches: 1.97.2;
Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements

Change dk_lookup() to accept an additional argument of the type enum uio_seg
that tells whether the given path is in user space or kernel space, so it
can tell NDINIT().

While the raidframe calls were ok, both ccd(4) and cgd(4) were passing
pointers to user space data, which leads to strange error on i386, as
reported by Jukka Salmi on current-users.

The issue has been there since last august, I'm actually a bit surprised
that no one in the meantime has used ccd(4) or cgd(4) on an arch where it
would have simply faulted.

branches: 1.95.2; 1.95.8; 1.95.10; 1.95.16;
__unused removal on arguments; approved by core.

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

branches: 1.93.2; 1.93.4;
- use dk_lookup instead of our home-spun version.
- allow raid to be configured in a wedge
- allow wedges to be configured in a raid
- add autoconfiguration of wedges in a raid

- Use the LWP cached credentials where sane.
- Minor cosmetic changes.

integrate kauth.

branches: 1.90.4; 1.90.6; 1.90.8; 1.90.10; 1.90.12;
merge ktrace-lwp.

If rf_SubmitReconBuffer indicates the submission was blocked (for
whatever reason), return 0 instead of the default
RF_RECON_READ_STOPPED. Returning RF_RECON_READ_STOPPED would result
in rf_ContinueReconstructFailedDisk() thinking that the given
component was "done" and breaking out of the main reconstruction loop
far too early. Reconstruction still worked correctly as long as there
were no errors, but RAIDframe wouldn't be in a position to properly
handle read/write errors during reconstruction.

This fixes the "raidctl's progress bar spins at 0% until
reconstruction finishes" problem.

branches: 1.88.2;
- initialize numRUsTotal before we indicate that we are doing a reconstruct.

- make numRUsComplete and numRUsTotal 64-bit quantities like
everything else that records this information.

branches: 1.87.2;
nuke trailing whitespace

The 'next' argument to rf_CreateDiskQueueData is always NULL. Since
there is no particular reason to pass an extra NULL argument, turf it,
and initialize p->next to NULL within the function.

Add a 'waitflag' argument to rf_CreateDiskQueueData() and use it to
determine if we are willing to wait for memory to come from the
diskqueuedata (dqd) and bufpool pools. Cleanup the mess related to
code calling rf_CreateDiskQueueData() with different expectations
(and/or blatent disregard) of what might happen if there were
insufficient pool resources.

It's not a bad idea to update the component labels whether or not the
reconstruction was successful.

rf_GetNextReconEvent() *will* return a valid event, so no need for
the assert. (we'd have panic'ed in there long before this assert
if that wasn't the case).

Minor whitespace changes.

Vastly improve the error handling in the case of a read/write error
that occurs during a reconstruction. We go from zero error handling
and likely panicing if something goes amiss, to gracefully bailing and
leaving the system in the best, usable state possible.

- introduce rf_DrainReconEventQueue() to allow easy cleaning of the
reconstruction event queue

- change how we cleanup the floating recon buffers in
rf_FreeReconControl(). Detect the end of the list rather
than traversing according to a count.

- keep track of the number of pending reconstruction writes. In the
event of a read error, use this to wait long enough for the pending
writes to (hopefully) drain.

- more cleanup is still needed on this code, but I didn't want to
start mixing major functional changes with minor cleanups.

XXX: There is a known issue with pool items left outstanding due to
the IO failure, and this can show up in the form of a panic at the
tail end of a shutdown. This problem is much less severe than before
these changes, and the hope/plan is that this problem will go away
once this code gets overhauled again.

branches: 1.81.2;
Torch some #define's missed in last commit.

Reconstruction Descriptors are only allocated once per reconstruction,
and don't need their own pool or freelist or anything fancier than a
malloc/free.

ForceReconReadDoneProc() needs a return after doing the first
rf_CauseReconEvent().

branches: 1.78.2;
The switch() in rf_ContinueReconstructFailedDisk() is never actually
used in non-simulation code, and thus is just wasting space (and
making the code more confusing to read!). Turf the switch, left-shift
the indentation of code, and nuke 'state' field of struct RF_RaidReconDesc_s.

No real functional changes.

continueFunc and continueArg arn't used. Turf. Simplify calls to
rf_GetNextReconEvent().

branches: 1.76.4;
Re-work the locking mechanisms for reconstruct and PSS structures
such that we don't actually hold a simplelock while we are doing
a pool_get(), but that we still effectively protecting critical code.

This should fix all of the outstanding LOCKDEBUG warnings related to
rebuilding RAID sets.

- don't use rf_PrintUserStats() for recon statistics.
rf_PrintUserStats() was mean for the simulator, and doesn't provide
any real info in kernel-space, especially for reconstructs.
Reconstructing actually renders the stats even more useless, since it
resets them all to zero before the reconstruct starts!

- since rf_PrintUserStats() is no longer used, nuke it along with the
routines that feed it. Nothing was using this code, and if we ever
need it again, we know where to find it.

- Introduce rf_pools which contains all of the various global pools used
by RAIDframe. Convert all other RAIDframe global pools to use pools
defined within this new structure.
- Introduce rf_pool_init(), used for initializing a single pool in
RAIDframe. Teach each of the configuration routines to use
rf_pool_init().
- Cleanup a few pool-related comments.
- Cleanup revent initialization and #defines.
- Add a missing pool_destroy() for the reconbuffer pool.

(Saves another 1K off of an i386 GENERIC kernel, and makes
stuff a lot more readable)

- fix up initialization of rf_recond_pool
- introduce rf_reconbuffer_pool and teach rf_MakeReconBuffer() to use it

Use RF_INCLUDE_PARITY_DECLUSTERING_DS to #if-out more unneeded bits.
(We can't do RF_DISTRIBUTE_SPARE bits without the parity declustering stuff.)

Nuke some unnecessary casts. No functional changes.

Introduce RF_REVENT_READ_FAILED, RF_REVENT_WRITE_FAILED and RF_REVENT_FORCEREAD_FAILED.
This removes 3 more RF_PANIC()'s (but we'll currently still panic if any of these cases occur).
fix up a few printf's.
XXX: still needs more cleanup and testing (and be taught to not panic).

Cleanup function prototypes.

- cleanup memory allocation in rf_AllocPSStatus()
- change function signature of rf_LookupRUStatus(). The last argument
is now a pointer to a new PSS, in case one is needed. Rather than
having rf_LookupRUStatus() allocate a new PSS, we pre-allocate one
beforehand, where necessary, just in case.
- change callers of rf_lookupRUStatus() to deal with the new way of
calling rf_lookupRUStatus().

[no improvement or worsening of parity rebuild/initialization performance.]

Use RF_ACC_TRACE to #if out more chunks of code related only
to access tracing. (not turned on yet)

Adjust _rf_ShutdownCreate() so that it is willing to wait for more
memory. Since we only now ever "return(0)", just return (void)
instead.

Cleanup all uses of rf_ShutdownCreate() to not worry about
it ever failing. Shaves another 600 bytes off of an i386 GENERIC kernel.

raidPtr->reconControl->percentCompleted only gets used in one
debugging printf, and in rf_netbsdkintf.c. We can do the calculations
inside of RF_DEBUG_RECON for the one debugging printf, and only
perform the percentCompleted calculation "on demand" in the
rf_netbsdkintf.c case. Shaves a few more bytes off an i386 GENERIC
kernel, and ever-so-slightly decreases the amount of work performed
during a reconstruct.

Add in a bunch of RF_SIGNAL_COND()'s that were missing. Tidy up
a few lines.

Left-shift another else{} chunk. No functional changes.

left-shift the "else" part of the if(!lp_SubmitReconBuffer) condition.
Cleanup. No real functional changes, just more readable.

Negate a condition, and flip if/else parts. Preparation for left-shifting
the (now) else part. No real functional change.

Some days you wonder if some of the function declaration consistency
was just an accident in the first place. Cleanup function decls and
a few comments. [ok.. so I wasn't going to fix this many.. but once
you're on a roll....]

Let's see... raidPtr->recon_done_procs is never set to anything
(other than NULL when raidPtr is initialized). That means
SignalReconDone() never does anything useful. Bye-bye!

Say good-bye to recon_done_procs and recon_done_procs_mutex (and its
initializer) as well.

- first kick at a major reworking of RAIDframe's memory allocation code:
- all freelists converted to pools
- initialization of structure members in certain cases where
code was relying on specific allocation and usage properties
to keep structures in a "known state" (that doesn't work with
pools!).
- make most pool_get() be "PR_WAITOK" until they can be analyzed
further, and/or have proper error handling added.
- all RF_Mallocs zero the space returned, so there is no difference
between RF_Calloc and RF_Malloc. In fact, all the RF_Calloc()'s
do is tend to do is get things horribly confused.
Make RF_Malloc() the "general memory allocator", with
RF_MallocAndAdd() the "general memory allocator with
allocation list".
- some of these RF_Malloc's et al. are destined to disappear.
- remove rf_rdp_freelist entirely (it's not used anywhere!)
- remove: #include "rf_freelist.h"
- to the files that were relying on the above, add: #include "rf_general.h"
- add: #include "rf_debugMem.h" to rf_shutdown.h to make it happy
about the loss of: #include "rf_freelist.h".

This shrinks an i386 GENERIC kernel by approx 5K. RAIDframe now
weighs in at about 162K on i386.

[Having received a definite lack of strenuous objection, a small amount
of strenuous agreement, and some general agreement, this commit is
going ahead because it's now starting to block some other changes I
wish to make.]

Remove most of the support for the concept of "rows" from RAIDframe.
While the "row" interface has been exported to the world, RAIDframe
internals have really only supported a single row, even though they
have feigned support of multiple rows.

Nothing changes in configuration land -- config files still need to
specify a single row, etc. All auto-config structures remain fully
forward/backwards compatible.

The only visible difference to the average user should be a
reduction in the size of a GENERIC kernel (i386) by 4.5K. For those
of us trolling through RAIDframe kernel code, a lot of the driver
configuration code has become a LOT easier to read.

branches: 1.56.2;
Back out the lwp/ktrace changes. They contained a lot of colateral damage,
and need to be examined and discussed more.

Pass lwp pointers throughtout the kernel, as required, so that the lwpid can
be inserted into ktrace records. The general change has been to replace
"struct proc *" with "struct lwp *" in various function prototypes, pass
the lwp through and use l_proc to get the process pointer when needed.

Bump the kernel rev up to 1.6V

Remove an assigned-to but unused variable.

Use 'void *' instead of 'caddr_t' in prototypes of VOP_IOCTL, VOP_FCNTL
and VOP_ADVLOCK, delete casts from callers (and some to copyin/out).

constify some

For reconstructs, move checks for failed components to before the
kernel threads are created.

Cleanup more printfs.

After a rebuild-in-place, a reconstruct, or a copyback, we should
really be updating the component labels.

Improve and/or re-arrange a number of locks. While much of the locking is
still a mess, and there are a number of unresolved issues here, this
gets us closer to being happier in LOCKDEBUG land.

Add a missing RF_LOCK_MUTEX().

Introduce a temp variable, and allocate the ReconCtrl structure before
we protect raidPtr. One less thing for LOCKDEBUG to complain about.

Nuke "baddisk". Thanks to Simon B.

rf_RegisterReconDoneProc() isn't needed.

This is the last of the 'easy' ones that Krister made me aware of.
Total savings on i386 GENERIC kernel: 13151 bytes
RAIDframe in GENERIC is now at: 179033
Thanks again Krister!

Introduce and use RF_DEBUG_PSS, and save a few more bytes.

One signal will do, thanks.

Cast the RF_DEBUG_RECON net a little wider.

Rename RF_DEBUG_RECONBUFFER to RF_DEBUG_RECON in order to facilitate
disabling other stuff without having to introduce another #define.

Cleanup some comments.

rf_CheckFloatingRbufCount() is only really useful when debugging the
reconstruct buffer stuff. #if it out in the general case.

Cleanup some printf's, and disable some (debugging) output.

Everyone and their dog was using RF_ERRORMSG3 to print out the same
sort of error message, over and over again, in different files.
Rather than having the same text repeated in multiple .o files,
create a couple of little functions to do the printing, and save a
bundle of space. Also improves readability of code.

Disallow 'reconstruct-in-place' on a component that has failed
and has already been reconstructed to a hot spare.

Nuke a redundant wakeup().

branches: 1.33.8;
Move a bunch of debugging stuff to be only used if DEBUG is turned on.

don't need <sys/types.h> when including <sys/param.h>

add RCSIDs

Step 2 of the disentanglement. We now look to <dev/raidframe/*> for
the stuff that used to live in rf_types.h, rf_raidframe.h, rf_layout.h,
rf_netbsd.h, rf_raid.h, rf_decluster,h, and a few other places.
Believe it or not, when this is all done, things will be cleaner.

No functional changes to RAIDframe.

branches: 1.29.2;
bzero -> memset

branches: 1.28.2;
It's silly to need a parity rebuild after a reconstruction has completed.
If we've just reconstructed a disk, then the parity is known to
be correct. (XXX doesn't hold for RAID 6!)

branches: 1.27.2;
Ensure we update the 'partitionSize' field of the component labels
when doing a reconstruct or a copyback. If we don't, junk might be
there, and that could cause the component to be not correctly
autoconfigured on reboot. Thanks to Simon Burge for helping track this down.

branches: 1.26.2;
Merge rf_update_component_labels() and rf_final_update_component_labels().

Oops.. reconstruction percentages were being reported incorrectly.
Thanks to Manuel Bouyer for noting this.

Umm.. Complete is not equal to 'left to do'. Fix the math.

- Add a mechanism for obtaining finer-grained 'progress' information
regarding reconstructs, copybacks, etc.

- RAID 0 doesn't do copybacks, but don't make raidctl sweat about it.

branches: 1.22.2;
Fix doubled 'the's in comments.

Create a new rf_close_component() to handle vnode operations for closing
components. Teach rf_UnconfigureVnodes() how to use it, and tell
the copyback and reconstruction code about it too.

When we close autoconfigured components, we need to note that they
are no longer in 'autoconfigured' status.

Fix a (slightly) bogus status message.

Make sure we close auto-configured components appropriately when
attempting a rebuild-in-place.

Be more aggressive about updating component labels in the event
of a real component failure (or a simulated failure):
- add 'numNewFailures' to keep track of the number of disk failures
since mod_counter was last updated for each component label.
- make sure we call rf_update_component_labels() upon any component failure,
real or simulated.

Do a better job of (re)initializing the component labels after
a reconstruct or a copyback.

Get recent changes into the tree:
- make component_label variables more consistent (==> clabel)
- re-work incorrect component configuration code
- re-work disk configuration code
- cleanup initial configuration of raidPtr info
- add auto-detection of components and RAID sets (Disabled, for now)
- allow / on RAID sets (Disabled, for now)
- rename "config_disk_queue" to "rf_ConfigureDiskQueue" and properly prototype
in rf_diskqueue.h
- protect some headers with #if _KERNEL (XXX this needs to be fixed properly)
and cleanup header formatting.
- expand the component labels (yes, they should be backward/forward compatible)
- other bits and pieces (some function names are still bogus, and will get
changed soon)

Nuke dependencies on rf_cpuutils.h.

Nuke unused debugging stuff. Clean up a whole bunch of comments.

- move a bunch of function prototypes to rf_kintf.h
- general cleanup of a number of prototypes that were scattered around.

Nuke #if 0'ed code.

- nuke calls to rf_get_threadid() and associated #include
- change a bunch of debugging printfs from
"[%d] ...", tid (where tid is the "thread id")
to
"raid%d: ...", raidPtr->raidid
- other minor rototillage

- update RF_CREATE_THREAD to handle a 'process name' argument.
- fire up a new thread for parity re-writes, copybacks, and reconstructs.
The ioctl's which trigger these actions now return immediately.
- add progress accounting for the above actions.
- minor rototillage of rf_netbsdkintf.c to deal with all of the above.

branches: 1.8.2;
Remove a 'struct proc *'-passing abomination that's been bugging me
for quite some time.

rf_sys.h does not need to be #included in any of these files, and, actually,
is no longer needed at all.

Clean up reconstruction accounting a bit. While it worked before, it was
slightly broken in the case where the RAID set did not support reconstruction.

branches: 1.5.2;
Update for recent changes including component label support, clean
bits, rebuilding components in-place, adding hot spares, shutdownhooks, etc.

Phase 2 of the RAIDframe cleanup. The source is now closer to KNF
and is much easier to read. No functionality changes.

Nuke more bits of RAIDframe "demo" code. We're not "demoing" here,
we're doing the Real Thing!

RAIDframe cleanup, phase 1. Nuke simulator support, user-land driver,
out-dated comments, and other unneeded stuff. This helps prepare
for cleaning up the rest of the code, and adding new functionality.

No functional changes to the kernel code in this commit.

RAIDframe, version 1.1, from the Parallel Data Laboratory at
Carnegie Mellon University. Full RAID implementation, including
levels 0, 1, 4, 5, 6, parity logging, and a few other goodies.
Ported to NetBSD by Greg Oster.

rf_CreateDiskQueueData() no longer uses waitflag, and will always succeed.
Cleanup the error path for the (no longer needed) PR_NOWAIT cases.

Extensive mechanical changes to the pools used in RAIDframe.

Alloclist remains not per-RAID, so initialize that pool
separately/differently than the rest.

The remainder of pools in RF_Pools_s are now per-RAID pools. Mostly
mechanical changes to functions to allocate/destroy per-RAID pools.
Needed to make raidPtr available in certain cases to be able to find
the per-RAID pools.

Extend rf_pool_init() to now populate a per-RAID wchan value that is
unique to each pool for a given RAID device.

TODO: Complete the analysis of the minimum number of items that are
required for each pool to allow IO to progress (i.e. so that a request
for pool resources can always be satisfied), and dynamically scale
minimum pool sizes based on RAID configuration.

Fix a long long-standing off-by-one error in computing lastPSID.

SUsPerPU is only really supported for a value of 1, and since the
first PSID is 0, the last will be numStripe-1. Also update the
setting of pending_writes to reflect the change to lastPSID.

Needs pullups to -8 and -9.

branches: 1.124.8;
Switch to vn_bdev_open* functions.

fix the function pointer and callback mess:
- callback functions return 0 and their result is not checked; make them void.
- there are two types of callbacks and they used to overload their parameters
and the callback structure; separate them into "function" and "value"
callbacks.
- make the wait function signature consistent.

branches: 1.122.4;
- Change the allocation macros to be more like function calls
- Change sizeof(type) -> sizeof(*variable)
- Use macros for the long buffer length allocations
- Remove "bit polishing" memsets() -- do them only once
- Remove unnecessary casts

Thanks to oster@ for finding bugs and testing.

branches: 1.121.12; 1.121.20;


Fix a long-standing bug related to rebooting while a
reconstruct-to-spare is underway but not yet complete.

The issue was that a component was being marked as a used_spare when
the rebuild started, not when the rebuild was actually finished.
Marking it as a used_spare meant that the component label on the spare
was being updated such that after a reboot the component would be
considered up-to-date, regardless of whether the rebuild actually
completed!

This fix includes:
1) Add an additional state "rf_ds_rebuilding_spare" which is used
to denote that a spare is currently being rebuilt from the live
components.
2) Update the comments on the disk states, which were out-of-sync
with reality.
3) When rebuilding to a spare component, that spare now enters the
state rf_ds_rebuilding_spare instead of the state rf_ds_used_spare.
4) When the rebuild is actually complete then the spare component
enters the rf_ds_used_spare state. rf_ds_used_spare is now used
exclusively for the case where the rebuilding to the spare has
completed successfully.

XXX: Someday we need to teach raidctl(8) about this new state, and
take out the backwards compatibility code in rf_netbsdkintf.c (see
RAIDFRAME_GET_INFO in raidioctl()). For today, this fix needs to be
generic enough that it can get backported without major grief.

XXX: Needs pullup to netbsd-5*, netbsd-6*, and netbsd-7

Fixes PR#49244.

branches: 1.120.2;
Change dk_lookup() to return an anonymous vnode not associated with
any file system. Change all consumers of dk_lookup() to get the
device from "v_rdev" instead of VOP_GETATTR() as specfs does not
support VOP_GETATTR(). Devices obtained with dk_lookup() will no
longer disappear on forced unmounts.

Fix for PR kern/48849 (root mirror raid fails on shutdown)

Welcome to 6.99.44

branches: 1.119.10;
fix parens in a message

branches: 1.118.2;
Add logic to the main reconstruction loop to handle RAID5 with rotated
spares. While here, observe that we were actually doing one more
stripe than we thought we were, and correct that too (it didn't matter
for non-RAID5_RS, but it definitely does for RAID5_RS). Add some
bounds-checking at the beginning to handle the case where the number
of stripes in the set is smaller than the sliding reconstruction window.

XXX: this problem likely needs to be fixed for PARITY_DECLUSTERING too.

branches: 1.117.2; 1.117.6; 1.117.8;
Change the vnode locking protocol of VOP_GETATTR() to request at least
a shared lock. Make all calls outside of file systems respect it.

The calls from file systems need review.

No objections from tech-kern.

Address part of PR kern/44972. From YAMAMOTO Takashi. Thanks!

rf_ReconstructInPlace: don't leave a vnode open on errors.
fixes a part of PR/44972.

Suggested to oster@ and approved via private e-mail as a help to
people who are getting reconstruction failures.

convert the main raidPtr mutex to a kmutex, and add a couple of cv's to
cover the old sleep/wakeup points for adding_hot_spare and waitForReconCond.
convert all remaining simple_lock's to kmutexes (they're not used or compiled
right now... even with all options enabled) and remove the support for them.

this leaves just a pair of tsleep()/wakeup() calls using old scheduling APIs.

convert rb_mutex to a kmutex/cv.

Define accessors for number of blocks and partition size in the
component label and use them where appropriate. Disscussed on tech-kern.

branches: 1.110.2; 1.110.4;
Introduce struct pathbuf. This is an abstraction to hold a pathname
and the metadata required to interpret it. Callers of namei must now
create a pathbuf and pass it to NDINIT (instead of a string and a
uio_seg), then destroy the pathbuf after the namei session is
complete.

Update all namei call sites accordingly. Add a pathbuf(9) man page and
update namei(9).

The pathbuf interface also now appears in a couple of related
additional places that were passing string/uio_seg pairs that were
later fed into NDINIT. Update other call sites accordingly.

add support for >2TB raid devices.

- add two new members to the component label:
u_int numBlocksHi
u_int partitionSizeHi
and store the top 32 bits of the real number of blocks and
partition size. modify rf_print_component_label(),
rf_does_it_fit(), rf_AutoConfigureDisks() and
rf_ReconstructFailedDiskBasic().

- call disk_blocksize() after disk_attach() [ from mlelstv ]

- shift the block number relative to DEV_BSHIFT in raidstart()
and InitBP() so that accesses work for non 512-byte devices.
[ from mlelstv ]

- update rf_getdisksize() to use the new getdisksize() [ from
mlelstv. this part needs a separate change for netbsd-5. ]


reviewed by: oster, christos and darrenr

branches: 1.108.2; 1.108.4;
Finally commit the RAIDframe parity map Summer Of Code project.

Drastically reduces the amount of time spent rewriting parity after an
unclean shutdown by keeping better track of which regions might have had
outstanding writes. Enabled by default; can be disabled on a per-set
basis, or tuned, with the new raidctl(8) commands.

Discussed on tech-kern@ to a general air of approval; exhortations to
commit from mrg@, christos@, and others.

Thanks to Google for their sponsorship, oster@ for mentoring the
project, assorted developers for trying very hard to break it, and
probably more I'm forgetting.

If we see a RF_RECON_WRITE_ERROR event we know a write has finished and
we need to account for that. Failure to do so means we can end up
waiting forever for writes we think are outstanding, but which have
already completed.

Addresses the RAIDframe part of PR#40569. Thanks to Matthias Scheler
for reporting the issue and verifying the fix.

branches: 1.106.2;
When unconfiguring an array where a reconstruct is in progress, abort
the reconstruct and wait for IOs to drain before pulling the plug.

Should fix the panic reported by der Mouse on tech-kern.

branches: 1.105.2; 1.105.4;
Nuke unneeded printf(). Spotted by pooka@.

branches: 1.104.4;
Re-work some of the guts of the reconstruction code.

Reconmap used to have one pointer for every reconstruction unit. This
does not scale well in the land of 1TB disks, where some 100MB+ of
"status pointers" are required for typical configurations. Convert
the reconstruction code to use a "sliding status window" which will
scale nicely regardless of the number of stripes/reconstruction units
in the RAID set. Convert the main reconstruction loop to rebuild the
array in chunks rather than in one big lump.

As part of these changes, introduce a function to kick any waiters on
the head separation callback list, and use that in the main
reconstruction event queue to wake up the waiters if things have
stalled. (I believe this may fix a race condition that could occur at
at least at the very end of a disk during reconstruction under heavy
IO load.)

Thanks to Brian Buhrow for all his help, support, and patience in
testing these changes.

branches: 1.103.2; 1.103.4; 1.103.6;
A forced recon read should not default to indicating that the reads
for that disk have stopped, since this will bump us out of the normal
reconstruction loop prematurely.

Fixes the (mostly cosmetic) bug where the reconstruction
status values stop updating, and from raidctl it appears that
reconstruction has totally stalled (which it actually hasn't -- the
reconstruction does complete properly, but not in the normal way).

Print out the status value if a reconstruction read fails.
Don't print out write promotions during reconstruct unless
we are debugging reconstructs.

branches: 1.101.6;
In a land before time, when kernel processes roamed the system, we
needed to keep track of the kernel process that opened a device in
order to close it with the right credentials. Flash forward to today
where curlwp is now quite sufficient.

Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start. In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern

branches: 1.99.6;
Fix wording in a comment and correct a debug line. From Olivier Cherrier
(via private mail). Thanks!

branches: 1.98.4; 1.98.6; 1.98.8;
Fix fallout from recent kthread changes.

branches: 1.97.2;
Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements

Change dk_lookup() to accept an additional argument of the type enum uio_seg
that tells whether the given path is in user space or kernel space, so it
can tell NDINIT().

While the raidframe calls were ok, both ccd(4) and cgd(4) were passing
pointers to user space data, which leads to strange error on i386, as
reported by Jukka Salmi on current-users.

The issue has been there since last august, I'm actually a bit surprised
that no one in the meantime has used ccd(4) or cgd(4) on an arch where it
would have simply faulted.

branches: 1.95.2; 1.95.8; 1.95.10; 1.95.16;
__unused removal on arguments; approved by core.

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

branches: 1.93.2; 1.93.4;
- use dk_lookup instead of our home-spun version.
- allow raid to be configured in a wedge
- allow wedges to be configured in a raid
- add autoconfiguration of wedges in a raid

- Use the LWP cached credentials where sane.
- Minor cosmetic changes.

integrate kauth.

branches: 1.90.4; 1.90.6; 1.90.8; 1.90.10; 1.90.12;
merge ktrace-lwp.

If rf_SubmitReconBuffer indicates the submission was blocked (for
whatever reason), return 0 instead of the default
RF_RECON_READ_STOPPED. Returning RF_RECON_READ_STOPPED would result
in rf_ContinueReconstructFailedDisk() thinking that the given
component was "done" and breaking out of the main reconstruction loop
far too early. Reconstruction still worked correctly as long as there
were no errors, but RAIDframe wouldn't be in a position to properly
handle read/write errors during reconstruction.

This fixes the "raidctl's progress bar spins at 0% until
reconstruction finishes" problem.

branches: 1.88.2;
- initialize numRUsTotal before we indicate that we are doing a reconstruct.

- make numRUsComplete and numRUsTotal 64-bit quantities like
everything else that records this information.

branches: 1.87.2;
nuke trailing whitespace

The 'next' argument to rf_CreateDiskQueueData is always NULL. Since
there is no particular reason to pass an extra NULL argument, turf it,
and initialize p->next to NULL within the function.

Add a 'waitflag' argument to rf_CreateDiskQueueData() and use it to
determine if we are willing to wait for memory to come from the
diskqueuedata (dqd) and bufpool pools. Cleanup the mess related to
code calling rf_CreateDiskQueueData() with different expectations
(and/or blatent disregard) of what might happen if there were
insufficient pool resources.

It's not a bad idea to update the component labels whether or not the
reconstruction was successful.

rf_GetNextReconEvent() *will* return a valid event, so no need for
the assert. (we'd have panic'ed in there long before this assert
if that wasn't the case).

Minor whitespace changes.

Vastly improve the error handling in the case of a read/write error
that occurs during a reconstruction. We go from zero error handling
and likely panicing if something goes amiss, to gracefully bailing and
leaving the system in the best, usable state possible.

- introduce rf_DrainReconEventQueue() to allow easy cleaning of the
reconstruction event queue

- change how we cleanup the floating recon buffers in
rf_FreeReconControl(). Detect the end of the list rather
than traversing according to a count.

- keep track of the number of pending reconstruction writes. In the
event of a read error, use this to wait long enough for the pending
writes to (hopefully) drain.

- more cleanup is still needed on this code, but I didn't want to
start mixing major functional changes with minor cleanups.

XXX: There is a known issue with pool items left outstanding due to
the IO failure, and this can show up in the form of a panic at the
tail end of a shutdown. This problem is much less severe than before
these changes, and the hope/plan is that this problem will go away
once this code gets overhauled again.

branches: 1.81.2;
Torch some #define's missed in last commit.

Reconstruction Descriptors are only allocated once per reconstruction,
and don't need their own pool or freelist or anything fancier than a
malloc/free.

ForceReconReadDoneProc() needs a return after doing the first
rf_CauseReconEvent().

branches: 1.78.2;
The switch() in rf_ContinueReconstructFailedDisk() is never actually
used in non-simulation code, and thus is just wasting space (and
making the code more confusing to read!). Turf the switch, left-shift
the indentation of code, and nuke 'state' field of struct RF_RaidReconDesc_s.

No real functional changes.

continueFunc and continueArg arn't used. Turf. Simplify calls to
rf_GetNextReconEvent().

branches: 1.76.4;
Re-work the locking mechanisms for reconstruct and PSS structures
such that we don't actually hold a simplelock while we are doing
a pool_get(), but that we still effectively protecting critical code.

This should fix all of the outstanding LOCKDEBUG warnings related to
rebuilding RAID sets.

- don't use rf_PrintUserStats() for recon statistics.
rf_PrintUserStats() was mean for the simulator, and doesn't provide
any real info in kernel-space, especially for reconstructs.
Reconstructing actually renders the stats even more useless, since it
resets them all to zero before the reconstruct starts!

- since rf_PrintUserStats() is no longer used, nuke it along with the
routines that feed it. Nothing was using this code, and if we ever
need it again, we know where to find it.

- Introduce rf_pools which contains all of the various global pools used
by RAIDframe. Convert all other RAIDframe global pools to use pools
defined within this new structure.
- Introduce rf_pool_init(), used for initializing a single pool in
RAIDframe. Teach each of the configuration routines to use
rf_pool_init().
- Cleanup a few pool-related comments.
- Cleanup revent initialization and #defines.
- Add a missing pool_destroy() for the reconbuffer pool.

(Saves another 1K off of an i386 GENERIC kernel, and makes
stuff a lot more readable)

- fix up initialization of rf_recond_pool
- introduce rf_reconbuffer_pool and teach rf_MakeReconBuffer() to use it

Use RF_INCLUDE_PARITY_DECLUSTERING_DS to #if-out more unneeded bits.
(We can't do RF_DISTRIBUTE_SPARE bits without the parity declustering stuff.)

Nuke some unnecessary casts. No functional changes.

Introduce RF_REVENT_READ_FAILED, RF_REVENT_WRITE_FAILED and RF_REVENT_FORCEREAD_FAILED.
This removes 3 more RF_PANIC()'s (but we'll currently still panic if any of these cases occur).
fix up a few printf's.
XXX: still needs more cleanup and testing (and be taught to not panic).

Cleanup function prototypes.

- cleanup memory allocation in rf_AllocPSStatus()
- change function signature of rf_LookupRUStatus(). The last argument
is now a pointer to a new PSS, in case one is needed. Rather than
having rf_LookupRUStatus() allocate a new PSS, we pre-allocate one
beforehand, where necessary, just in case.
- change callers of rf_lookupRUStatus() to deal with the new way of
calling rf_lookupRUStatus().

[no improvement or worsening of parity rebuild/initialization performance.]

Use RF_ACC_TRACE to #if out more chunks of code related only
to access tracing. (not turned on yet)

Adjust _rf_ShutdownCreate() so that it is willing to wait for more
memory. Since we only now ever "return(0)", just return (void)
instead.

Cleanup all uses of rf_ShutdownCreate() to not worry about
it ever failing. Shaves another 600 bytes off of an i386 GENERIC kernel.

raidPtr->reconControl->percentCompleted only gets used in one
debugging printf, and in rf_netbsdkintf.c. We can do the calculations
inside of RF_DEBUG_RECON for the one debugging printf, and only
perform the percentCompleted calculation "on demand" in the
rf_netbsdkintf.c case. Shaves a few more bytes off an i386 GENERIC
kernel, and ever-so-slightly decreases the amount of work performed
during a reconstruct.

Add in a bunch of RF_SIGNAL_COND()'s that were missing. Tidy up
a few lines.

Left-shift another else{} chunk. No functional changes.

left-shift the "else" part of the if(!lp_SubmitReconBuffer) condition.
Cleanup. No real functional changes, just more readable.

Negate a condition, and flip if/else parts. Preparation for left-shifting
the (now) else part. No real functional change.

Some days you wonder if some of the function declaration consistency
was just an accident in the first place. Cleanup function decls and
a few comments. [ok.. so I wasn't going to fix this many.. but once
you're on a roll....]

Let's see... raidPtr->recon_done_procs is never set to anything
(other than NULL when raidPtr is initialized). That means
SignalReconDone() never does anything useful. Bye-bye!

Say good-bye to recon_done_procs and recon_done_procs_mutex (and its
initializer) as well.

- first kick at a major reworking of RAIDframe's memory allocation code:
- all freelists converted to pools
- initialization of structure members in certain cases where
code was relying on specific allocation and usage properties
to keep structures in a "known state" (that doesn't work with
pools!).
- make most pool_get() be "PR_WAITOK" until they can be analyzed
further, and/or have proper error handling added.
- all RF_Mallocs zero the space returned, so there is no difference
between RF_Calloc and RF_Malloc. In fact, all the RF_Calloc()'s
do is tend to do is get things horribly confused.
Make RF_Malloc() the "general memory allocator", with
RF_MallocAndAdd() the "general memory allocator with
allocation list".
- some of these RF_Malloc's et al. are destined to disappear.
- remove rf_rdp_freelist entirely (it's not used anywhere!)
- remove: #include "rf_freelist.h"
- to the files that were relying on the above, add: #include "rf_general.h"
- add: #include "rf_debugMem.h" to rf_shutdown.h to make it happy
about the loss of: #include "rf_freelist.h".

This shrinks an i386 GENERIC kernel by approx 5K. RAIDframe now
weighs in at about 162K on i386.

[Having received a definite lack of strenuous objection, a small amount
of strenuous agreement, and some general agreement, this commit is
going ahead because it's now starting to block some other changes I
wish to make.]

Remove most of the support for the concept of "rows" from RAIDframe.
While the "row" interface has been exported to the world, RAIDframe
internals have really only supported a single row, even though they
have feigned support of multiple rows.

Nothing changes in configuration land -- config files still need to
specify a single row, etc. All auto-config structures remain fully
forward/backwards compatible.

The only visible difference to the average user should be a
reduction in the size of a GENERIC kernel (i386) by 4.5K. For those
of us trolling through RAIDframe kernel code, a lot of the driver
configuration code has become a LOT easier to read.

branches: 1.56.2;
Back out the lwp/ktrace changes. They contained a lot of colateral damage,
and need to be examined and discussed more.

Pass lwp pointers throughtout the kernel, as required, so that the lwpid can
be inserted into ktrace records. The general change has been to replace
"struct proc *" with "struct lwp *" in various function prototypes, pass
the lwp through and use l_proc to get the process pointer when needed.

Bump the kernel rev up to 1.6V

Remove an assigned-to but unused variable.

Use 'void *' instead of 'caddr_t' in prototypes of VOP_IOCTL, VOP_FCNTL
and VOP_ADVLOCK, delete casts from callers (and some to copyin/out).

constify some

For reconstructs, move checks for failed components to before the
kernel threads are created.

Cleanup more printfs.

After a rebuild-in-place, a reconstruct, or a copyback, we should
really be updating the component labels.

Improve and/or re-arrange a number of locks. While much of the locking is
still a mess, and there are a number of unresolved issues here, this
gets us closer to being happier in LOCKDEBUG land.

Add a missing RF_LOCK_MUTEX().

Introduce a temp variable, and allocate the ReconCtrl structure before
we protect raidPtr. One less thing for LOCKDEBUG to complain about.

Nuke "baddisk". Thanks to Simon B.

rf_RegisterReconDoneProc() isn't needed.

This is the last of the 'easy' ones that Krister made me aware of.
Total savings on i386 GENERIC kernel: 13151 bytes
RAIDframe in GENERIC is now at: 179033
Thanks again Krister!

Introduce and use RF_DEBUG_PSS, and save a few more bytes.

One signal will do, thanks.

Cast the RF_DEBUG_RECON net a little wider.

Rename RF_DEBUG_RECONBUFFER to RF_DEBUG_RECON in order to facilitate
disabling other stuff without having to introduce another #define.

Cleanup some comments.

rf_CheckFloatingRbufCount() is only really useful when debugging the
reconstruct buffer stuff. #if it out in the general case.

Cleanup some printf's, and disable some (debugging) output.

Everyone and their dog was using RF_ERRORMSG3 to print out the same
sort of error message, over and over again, in different files.
Rather than having the same text repeated in multiple .o files,
create a couple of little functions to do the printing, and save a
bundle of space. Also improves readability of code.

Disallow 'reconstruct-in-place' on a component that has failed
and has already been reconstructed to a hot spare.

Nuke a redundant wakeup().

branches: 1.33.8;
Move a bunch of debugging stuff to be only used if DEBUG is turned on.

don't need <sys/types.h> when including <sys/param.h>

add RCSIDs

Step 2 of the disentanglement. We now look to <dev/raidframe/*> for
the stuff that used to live in rf_types.h, rf_raidframe.h, rf_layout.h,
rf_netbsd.h, rf_raid.h, rf_decluster,h, and a few other places.
Believe it or not, when this is all done, things will be cleaner.

No functional changes to RAIDframe.

branches: 1.29.2;
bzero -> memset

branches: 1.28.2;
It's silly to need a parity rebuild after a reconstruction has completed.
If we've just reconstructed a disk, then the parity is known to
be correct. (XXX doesn't hold for RAID 6!)

branches: 1.27.2;
Ensure we update the 'partitionSize' field of the component labels
when doing a reconstruct or a copyback. If we don't, junk might be
there, and that could cause the component to be not correctly
autoconfigured on reboot. Thanks to Simon Burge for helping track this down.

branches: 1.26.2;
Merge rf_update_component_labels() and rf_final_update_component_labels().

Oops.. reconstruction percentages were being reported incorrectly.
Thanks to Manuel Bouyer for noting this.

Umm.. Complete is not equal to 'left to do'. Fix the math.

- Add a mechanism for obtaining finer-grained 'progress' information
regarding reconstructs, copybacks, etc.

- RAID 0 doesn't do copybacks, but don't make raidctl sweat about it.

branches: 1.22.2;
Fix doubled 'the's in comments.

Create a new rf_close_component() to handle vnode operations for closing
components. Teach rf_UnconfigureVnodes() how to use it, and tell
the copyback and reconstruction code about it too.

When we close autoconfigured components, we need to note that they
are no longer in 'autoconfigured' status.

Fix a (slightly) bogus status message.

Make sure we close auto-configured components appropriately when
attempting a rebuild-in-place.

Be more aggressive about updating component labels in the event
of a real component failure (or a simulated failure):
- add 'numNewFailures' to keep track of the number of disk failures
since mod_counter was last updated for each component label.
- make sure we call rf_update_component_labels() upon any component failure,
real or simulated.

Do a better job of (re)initializing the component labels after
a reconstruct or a copyback.

Get recent changes into the tree:
- make component_label variables more consistent (==> clabel)
- re-work incorrect component configuration code
- re-work disk configuration code
- cleanup initial configuration of raidPtr info
- add auto-detection of components and RAID sets (Disabled, for now)
- allow / on RAID sets (Disabled, for now)
- rename "config_disk_queue" to "rf_ConfigureDiskQueue" and properly prototype
in rf_diskqueue.h
- protect some headers with #if _KERNEL (XXX this needs to be fixed properly)
and cleanup header formatting.
- expand the component labels (yes, they should be backward/forward compatible)
- other bits and pieces (some function names are still bogus, and will get
changed soon)

Nuke dependencies on rf_cpuutils.h.

Nuke unused debugging stuff. Clean up a whole bunch of comments.

- move a bunch of function prototypes to rf_kintf.h
- general cleanup of a number of prototypes that were scattered around.

Nuke #if 0'ed code.

- nuke calls to rf_get_threadid() and associated #include
- change a bunch of debugging printfs from
"[%d] ...", tid (where tid is the "thread id")
to
"raid%d: ...", raidPtr->raidid
- other minor rototillage

- update RF_CREATE_THREAD to handle a 'process name' argument.
- fire up a new thread for parity re-writes, copybacks, and reconstructs.
The ioctl's which trigger these actions now return immediately.
- add progress accounting for the above actions.
- minor rototillage of rf_netbsdkintf.c to deal with all of the above.

branches: 1.8.2;
Remove a 'struct proc *'-passing abomination that's been bugging me
for quite some time.

rf_sys.h does not need to be #included in any of these files, and, actually,
is no longer needed at all.

Clean up reconstruction accounting a bit. While it worked before, it was
slightly broken in the case where the RAID set did not support reconstruction.

branches: 1.5.2;
Update for recent changes including component label support, clean
bits, rebuilding components in-place, adding hot spares, shutdownhooks, etc.

Phase 2 of the RAIDframe cleanup. The source is now closer to KNF
and is much easier to read. No functionality changes.

Nuke more bits of RAIDframe "demo" code. We're not "demoing" here,
we're doing the Real Thing!

RAIDframe cleanup, phase 1. Nuke simulator support, user-land driver,
out-dated comments, and other unneeded stuff. This helps prepare
for cleaning up the rest of the code, and adding new functionality.

No functional changes to the kernel code in this commit.

RAIDframe, version 1.1, from the Parallel Data Laboratory at
Carnegie Mellon University. Full RAID implementation, including
levels 0, 1, 4, 5, 6, parity logging, and a few other goodies.
Ported to NetBSD by Greg Oster.

Extensive mechanical changes to the pools used in RAIDframe.

Alloclist remains not per-RAID, so initialize that pool
separately/differently than the rest.

The remainder of pools in RF_Pools_s are now per-RAID pools. Mostly
mechanical changes to functions to allocate/destroy per-RAID pools.
Needed to make raidPtr available in certain cases to be able to find
the per-RAID pools.

Extend rf_pool_init() to now populate a per-RAID wchan value that is
unique to each pool for a given RAID device.

TODO: Complete the analysis of the minimum number of items that are
required for each pool to allow IO to progress (i.e. so that a request
for pool resources can always be satisfied), and dynamically scale
minimum pool sizes based on RAID configuration.

Fix a long long-standing off-by-one error in computing lastPSID.

SUsPerPU is only really supported for a value of 1, and since the
first PSID is 0, the last will be numStripe-1. Also update the
setting of pending_writes to reflect the change to lastPSID.

Needs pullups to -8 and -9.

branches: 1.124.8;
Switch to vn_bdev_open* functions.