Extensive mechanical changes to the pools used in RAIDframe.

Alloclist remains not per-RAID, so initialize that pool
separately/differently than the rest.

The remainder of pools in RF_Pools_s are now per-RAID pools. Mostly
mechanical changes to functions to allocate/destroy per-RAID pools.
Needed to make raidPtr available in certain cases to be able to find
the per-RAID pools.

Extend rf_pool_init() to now populate a per-RAID wchan value that is
unique to each pool for a given RAID device.

TODO: Complete the analysis of the minimum number of items that are
required for each pool to allow IO to progress (i.e. so that a request
for pool resources can always be satisfied), and dynamically scale
minimum pool sizes based on RAID configuration.

Fix typo (s/supress/suppress/).

- Change the allocation macros to be more like function calls
- Change sizeof(type) -> sizeof(*variable)
- Use macros for the long buffer length allocations
- Remove "bit polishing" memsets() -- do them only once
- Remove unnecessary casts

Thanks to oster@ for finding bugs and testing.

branches: 1.32.56;
convert stripelocks to kmutex.

- convert rf_printf_mutex to a kmutex
- convert rf_rad_lock and the per-raid "cv" to per-raid kmutex/and real cv
- use rf_mutex_init() in places, and move it with the similar definitions

branches: 1.30.4; 1.30.6;
ansify function definitions

branches: 1.29.52; 1.29.62; 1.29.68;
__unused removal on arguments; approved by core.

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

branches: 1.27.20; 1.27.22;
merge ktrace-lwp.

nuke trailing whitespace

branches: 1.25.8; 1.25.10;
- Introduce rf_pools which contains all of the various global pools used
by RAIDframe. Convert all other RAIDframe global pools to use pools
defined within this new structure.
- Introduce rf_pool_init(), used for initializing a single pool in
RAIDframe. Teach each of the configuration routines to use
rf_pool_init().
- Cleanup a few pool-related comments.
- Cleanup revent initialization and #defines.
- Add a missing pool_destroy() for the reconbuffer pool.

(Saves another 1K off of an i386 GENERIC kernel, and makes
stuff a lot more readable)

- remove the RF_*_INC's, as necessary. They are not needed any more.
- introduce RF_MIN_*'s, as necessary. These will indicate the
low-water mark for pools as well as the pool_prime() value.
- add pool_setlowat() for the critical pools.
- pool_prime() and pool_setlowat() the raidframe_cbufpool.
- re-order some pool_prime()'s and pool_sethiwat()'s for clarity.

Adjust _rf_ShutdownCreate() so that it is willing to wait for more
memory. Since we only now ever "return(0)", just return (void)
instead.

Cleanup all uses of rf_ShutdownCreate() to not worry about
it ever failing. Shaves another 600 bytes off of an i386 GENERIC kernel.

Always ask for a new RF_StripeLockDesc_t "just in case", and then
give it back if we don't need it. If we don't allocate it before
we take our lock, LOCKDEBUG (rightfully) complains that we're trying
to grab something from the pool with PR_WAITOK. This code (and the
PR_WAITOK in particular) really needs to be revisited at some point.

Some days you wonder if some of the function declaration consistency
was just an accident in the first place. Cleanup function decls and
a few comments. [ok.. so I wasn't going to fix this many.. but once
you're on a roll....]

Clean up code/comments. No functional changes. Just a lot easier to read now.

Since rf_mutex_destroy() is effectively a NOP now, nuke it.

Simplify defn's of rf_mutex_init() and rf_mutex_destroy(). Since
neither of these ever fail, no need to have a return value. That
makes all the "error detection" on these functions completely
unneeded. But since we're here, if we don't have a return value, then
why not make these macros? My.. look how things keep shrinking, with
no loss in functionality!

- first kick at a major reworking of RAIDframe's memory allocation code:
- all freelists converted to pools
- initialization of structure members in certain cases where
code was relying on specific allocation and usage properties
to keep structures in a "known state" (that doesn't work with
pools!).
- make most pool_get() be "PR_WAITOK" until they can be analyzed
further, and/or have proper error handling added.
- all RF_Mallocs zero the space returned, so there is no difference
between RF_Calloc and RF_Malloc. In fact, all the RF_Calloc()'s
do is tend to do is get things horribly confused.
Make RF_Malloc() the "general memory allocator", with
RF_MallocAndAdd() the "general memory allocator with
allocation list".
- some of these RF_Malloc's et al. are destined to disappear.
- remove rf_rdp_freelist entirely (it's not used anywhere!)
- remove: #include "rf_freelist.h"
- to the files that were relying on the above, add: #include "rf_general.h"
- add: #include "rf_debugMem.h" to rf_shutdown.h to make it happy
about the loss of: #include "rf_freelist.h".

This shrinks an i386 GENERIC kernel by approx 5K. RAIDframe now
weighs in at about 162K on i386.

branches: 1.16.2;
rf_MakeLockTable() and rf_ShutdownStripeLocks() are used only in
rf_stripelocks.c - make them static functions.

lockTable is not needed for AddToWaitersQueue(). Thanks to Simon B.
(cleanup a couple comments while I'm here.)

Everyone and their dog was using RF_ERRORMSG3 to print out the same
sort of error message, over and over again, in different files.
Rather than having the same text repeated in multiple .o files,
create a couple of little functions to do the printing, and save a
bundle of space. Also improves readability of code.

Use the newly added RF_DEBUG_STRIPELOCK to effectively remove a bunch
of stuff that most of the world will never want to use.

Bah.. Not sure how those ";"'s got missed :-(

Only include some debugging printf's if DEBUG is defined.

add RCSIDs

Nuke some #if 0 stuff.

Step 2 of the disentanglement. We now look to <dev/raidframe/*> for
the stuff that used to live in rf_types.h, rf_raidframe.h, rf_layout.h,
rf_netbsd.h, rf_raid.h, rf_decluster,h, and a few other places.
Believe it or not, when this is all done, things will be cleaner.

No functional changes to RAIDframe.

branches: 1.7.2;
Disable all of the RF_ASSERT()s by default, enabling them if the
RAID_DIAGNOSTIC option is specified in the kernel configuration
file.

branches: 1.6.2; 1.6.4;
Comment out 2 cases of unitialized variable use (only used when
debugging is switched on, though).

- more cleanup of rf_get_threadid() and debugging printfs

Nuke unused #define.

branches: 1.3.8;
Phase 2 of the RAIDframe cleanup. The source is now closer to KNF
and is much easier to read. No functionality changes.

RAIDframe cleanup, phase 1. Nuke simulator support, user-land driver,
out-dated comments, and other unneeded stuff. This helps prepare
for cleaning up the rest of the code, and adding new functionality.

No functional changes to the kernel code in this commit.

RAIDframe, version 1.1, from the Parallel Data Laboratory at
Carnegie Mellon University. Full RAID implementation, including
levels 0, 1, 4, 5, 6, parity logging, and a few other goodies.
Ported to NetBSD by Greg Oster.

Fix typo (s/supress/suppress/).

- Change the allocation macros to be more like function calls
- Change sizeof(type) -> sizeof(*variable)
- Use macros for the long buffer length allocations
- Remove "bit polishing" memsets() -- do them only once
- Remove unnecessary casts

Thanks to oster@ for finding bugs and testing.

branches: 1.32.56;
convert stripelocks to kmutex.

- convert rf_printf_mutex to a kmutex
- convert rf_rad_lock and the per-raid "cv" to per-raid kmutex/and real cv
- use rf_mutex_init() in places, and move it with the similar definitions

branches: 1.30.4; 1.30.6;
ansify function definitions

branches: 1.29.52; 1.29.62; 1.29.68;
__unused removal on arguments; approved by core.

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

branches: 1.27.20; 1.27.22;
merge ktrace-lwp.

nuke trailing whitespace

branches: 1.25.8; 1.25.10;
- Introduce rf_pools which contains all of the various global pools used
by RAIDframe. Convert all other RAIDframe global pools to use pools
defined within this new structure.
- Introduce rf_pool_init(), used for initializing a single pool in
RAIDframe. Teach each of the configuration routines to use
rf_pool_init().
- Cleanup a few pool-related comments.
- Cleanup revent initialization and #defines.
- Add a missing pool_destroy() for the reconbuffer pool.

(Saves another 1K off of an i386 GENERIC kernel, and makes
stuff a lot more readable)

- remove the RF_*_INC's, as necessary. They are not needed any more.
- introduce RF_MIN_*'s, as necessary. These will indicate the
low-water mark for pools as well as the pool_prime() value.
- add pool_setlowat() for the critical pools.
- pool_prime() and pool_setlowat() the raidframe_cbufpool.
- re-order some pool_prime()'s and pool_sethiwat()'s for clarity.

Adjust _rf_ShutdownCreate() so that it is willing to wait for more
memory. Since we only now ever "return(0)", just return (void)
instead.

Cleanup all uses of rf_ShutdownCreate() to not worry about
it ever failing. Shaves another 600 bytes off of an i386 GENERIC kernel.

Always ask for a new RF_StripeLockDesc_t "just in case", and then
give it back if we don't need it. If we don't allocate it before
we take our lock, LOCKDEBUG (rightfully) complains that we're trying
to grab something from the pool with PR_WAITOK. This code (and the
PR_WAITOK in particular) really needs to be revisited at some point.

Some days you wonder if some of the function declaration consistency
was just an accident in the first place. Cleanup function decls and
a few comments. [ok.. so I wasn't going to fix this many.. but once
you're on a roll....]

Clean up code/comments. No functional changes. Just a lot easier to read now.

Since rf_mutex_destroy() is effectively a NOP now, nuke it.

Simplify defn's of rf_mutex_init() and rf_mutex_destroy(). Since
neither of these ever fail, no need to have a return value. That
makes all the "error detection" on these functions completely
unneeded. But since we're here, if we don't have a return value, then
why not make these macros? My.. look how things keep shrinking, with
no loss in functionality!

- first kick at a major reworking of RAIDframe's memory allocation code:
- all freelists converted to pools
- initialization of structure members in certain cases where
code was relying on specific allocation and usage properties
to keep structures in a "known state" (that doesn't work with
pools!).
- make most pool_get() be "PR_WAITOK" until they can be analyzed
further, and/or have proper error handling added.
- all RF_Mallocs zero the space returned, so there is no difference
between RF_Calloc and RF_Malloc. In fact, all the RF_Calloc()'s
do is tend to do is get things horribly confused.
Make RF_Malloc() the "general memory allocator", with
RF_MallocAndAdd() the "general memory allocator with
allocation list".
- some of these RF_Malloc's et al. are destined to disappear.
- remove rf_rdp_freelist entirely (it's not used anywhere!)
- remove: #include "rf_freelist.h"
- to the files that were relying on the above, add: #include "rf_general.h"
- add: #include "rf_debugMem.h" to rf_shutdown.h to make it happy
about the loss of: #include "rf_freelist.h".

This shrinks an i386 GENERIC kernel by approx 5K. RAIDframe now
weighs in at about 162K on i386.

branches: 1.16.2;
rf_MakeLockTable() and rf_ShutdownStripeLocks() are used only in
rf_stripelocks.c - make them static functions.

lockTable is not needed for AddToWaitersQueue(). Thanks to Simon B.
(cleanup a couple comments while I'm here.)

Everyone and their dog was using RF_ERRORMSG3 to print out the same
sort of error message, over and over again, in different files.
Rather than having the same text repeated in multiple .o files,
create a couple of little functions to do the printing, and save a
bundle of space. Also improves readability of code.

Use the newly added RF_DEBUG_STRIPELOCK to effectively remove a bunch
of stuff that most of the world will never want to use.

Bah.. Not sure how those ";"'s got missed :-(

Only include some debugging printf's if DEBUG is defined.

add RCSIDs

Nuke some #if 0 stuff.

Step 2 of the disentanglement. We now look to <dev/raidframe/*> for
the stuff that used to live in rf_types.h, rf_raidframe.h, rf_layout.h,
rf_netbsd.h, rf_raid.h, rf_decluster,h, and a few other places.
Believe it or not, when this is all done, things will be cleaner.

No functional changes to RAIDframe.

branches: 1.7.2;
Disable all of the RF_ASSERT()s by default, enabling them if the
RAID_DIAGNOSTIC option is specified in the kernel configuration
file.

branches: 1.6.2; 1.6.4;
Comment out 2 cases of unitialized variable use (only used when
debugging is switched on, though).

- more cleanup of rf_get_threadid() and debugging printfs

Nuke unused #define.

branches: 1.3.8;
Phase 2 of the RAIDframe cleanup. The source is now closer to KNF
and is much easier to read. No functionality changes.

RAIDframe cleanup, phase 1. Nuke simulator support, user-land driver,
out-dated comments, and other unneeded stuff. This helps prepare
for cleaning up the rest of the code, and adding new functionality.

No functional changes to the kernel code in this commit.

RAIDframe, version 1.1, from the Parallel Data Laboratory at
Carnegie Mellon University. Full RAID implementation, including
levels 0, 1, 4, 5, 6, parity logging, and a few other goodies.
Ported to NetBSD by Greg Oster.

- Change the allocation macros to be more like function calls
- Change sizeof(type) -> sizeof(*variable)
- Use macros for the long buffer length allocations
- Remove "bit polishing" memsets() -- do them only once
- Remove unnecessary casts

Thanks to oster@ for finding bugs and testing.

convert stripelocks to kmutex.

- convert rf_printf_mutex to a kmutex
- convert rf_rad_lock and the per-raid "cv" to per-raid kmutex/and real cv
- use rf_mutex_init() in places, and move it with the similar definitions

branches: 1.30.4; 1.30.6;
ansify function definitions

branches: 1.29.52; 1.29.62; 1.29.68;
__unused removal on arguments; approved by core.

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

branches: 1.27.20; 1.27.22;
merge ktrace-lwp.

nuke trailing whitespace

branches: 1.25.8; 1.25.10;
- Introduce rf_pools which contains all of the various global pools used
by RAIDframe. Convert all other RAIDframe global pools to use pools
defined within this new structure.
- Introduce rf_pool_init(), used for initializing a single pool in
RAIDframe. Teach each of the configuration routines to use
rf_pool_init().
- Cleanup a few pool-related comments.
- Cleanup revent initialization and #defines.
- Add a missing pool_destroy() for the reconbuffer pool.

(Saves another 1K off of an i386 GENERIC kernel, and makes
stuff a lot more readable)

- remove the RF_*_INC's, as necessary. They are not needed any more.
- introduce RF_MIN_*'s, as necessary. These will indicate the
low-water mark for pools as well as the pool_prime() value.
- add pool_setlowat() for the critical pools.
- pool_prime() and pool_setlowat() the raidframe_cbufpool.
- re-order some pool_prime()'s and pool_sethiwat()'s for clarity.

Adjust _rf_ShutdownCreate() so that it is willing to wait for more
memory. Since we only now ever "return(0)", just return (void)
instead.

Cleanup all uses of rf_ShutdownCreate() to not worry about
it ever failing. Shaves another 600 bytes off of an i386 GENERIC kernel.

Always ask for a new RF_StripeLockDesc_t "just in case", and then
give it back if we don't need it. If we don't allocate it before
we take our lock, LOCKDEBUG (rightfully) complains that we're trying
to grab something from the pool with PR_WAITOK. This code (and the
PR_WAITOK in particular) really needs to be revisited at some point.

Some days you wonder if some of the function declaration consistency
was just an accident in the first place. Cleanup function decls and
a few comments. [ok.. so I wasn't going to fix this many.. but once
you're on a roll....]

Clean up code/comments. No functional changes. Just a lot easier to read now.

Since rf_mutex_destroy() is effectively a NOP now, nuke it.

Simplify defn's of rf_mutex_init() and rf_mutex_destroy(). Since
neither of these ever fail, no need to have a return value. That
makes all the "error detection" on these functions completely
unneeded. But since we're here, if we don't have a return value, then
why not make these macros? My.. look how things keep shrinking, with
no loss in functionality!

- first kick at a major reworking of RAIDframe's memory allocation code:
- all freelists converted to pools
- initialization of structure members in certain cases where
code was relying on specific allocation and usage properties
to keep structures in a "known state" (that doesn't work with
pools!).
- make most pool_get() be "PR_WAITOK" until they can be analyzed
further, and/or have proper error handling added.
- all RF_Mallocs zero the space returned, so there is no difference
between RF_Calloc and RF_Malloc. In fact, all the RF_Calloc()'s
do is tend to do is get things horribly confused.
Make RF_Malloc() the "general memory allocator", with
RF_MallocAndAdd() the "general memory allocator with
allocation list".
- some of these RF_Malloc's et al. are destined to disappear.
- remove rf_rdp_freelist entirely (it's not used anywhere!)
- remove: #include "rf_freelist.h"
- to the files that were relying on the above, add: #include "rf_general.h"
- add: #include "rf_debugMem.h" to rf_shutdown.h to make it happy
about the loss of: #include "rf_freelist.h".

This shrinks an i386 GENERIC kernel by approx 5K. RAIDframe now
weighs in at about 162K on i386.

branches: 1.16.2;
rf_MakeLockTable() and rf_ShutdownStripeLocks() are used only in
rf_stripelocks.c - make them static functions.

lockTable is not needed for AddToWaitersQueue(). Thanks to Simon B.
(cleanup a couple comments while I'm here.)

Everyone and their dog was using RF_ERRORMSG3 to print out the same
sort of error message, over and over again, in different files.
Rather than having the same text repeated in multiple .o files,
create a couple of little functions to do the printing, and save a
bundle of space. Also improves readability of code.

Use the newly added RF_DEBUG_STRIPELOCK to effectively remove a bunch
of stuff that most of the world will never want to use.

Bah.. Not sure how those ";"'s got missed :-(

Only include some debugging printf's if DEBUG is defined.

add RCSIDs

Nuke some #if 0 stuff.

Step 2 of the disentanglement. We now look to <dev/raidframe/*> for
the stuff that used to live in rf_types.h, rf_raidframe.h, rf_layout.h,
rf_netbsd.h, rf_raid.h, rf_decluster,h, and a few other places.
Believe it or not, when this is all done, things will be cleaner.

No functional changes to RAIDframe.

branches: 1.7.2;
Disable all of the RF_ASSERT()s by default, enabling them if the
RAID_DIAGNOSTIC option is specified in the kernel configuration
file.

branches: 1.6.2; 1.6.4;
Comment out 2 cases of unitialized variable use (only used when
debugging is switched on, though).

- more cleanup of rf_get_threadid() and debugging printfs

Nuke unused #define.

branches: 1.3.8;
Phase 2 of the RAIDframe cleanup. The source is now closer to KNF
and is much easier to read. No functionality changes.

RAIDframe cleanup, phase 1. Nuke simulator support, user-land driver,
out-dated comments, and other unneeded stuff. This helps prepare
for cleaning up the rest of the code, and adding new functionality.

No functional changes to the kernel code in this commit.

RAIDframe, version 1.1, from the Parallel Data Laboratory at
Carnegie Mellon University. Full RAID implementation, including
levels 0, 1, 4, 5, 6, parity logging, and a few other goodies.
Ported to NetBSD by Greg Oster.