lib: remove CONSTCOND comment

Since 2021-01-31, lint doesn't need it anymore for the common pattern of
'do ... while (0)'.

Allow calling puffs_mount(3) before puffs_daemon(3)

puffs_daemon(3) creates a pipe before forking, and the parent process
waits for the child to either complete puffs_mount(3) or fail. If a
user calls puffs_daemon(3) after puffs_mount(3), the function
deadlocks. While this error-reporting functionality is really a nice
thing to have, deadlocking is not great. If the filesystem has already
been mounted, puffs_mount(3) should just daemonize the process and
return.

This became an issue because fuse_daemonize(3) in FUSE API had no such
requirement and some FUSE filesystems in the wild suffered deadlocks.

Use size_t in place of voff_t in puffs_setrootinfo(3)

Chuck Silvers pointed out that voff_t was also supposed to be
kernel-only. The correct type to use in userland would be off_t, but
since changing vsize_t to either voff_t or off_t is an ABI change on
32-bit platforms, we use size_t knowing that it is technically
incorrect.

Avoid using register_t or vsize_t in <puffs.h>

<puffs.h> is a user-space header, and should not use any of
kernel-only types. It's not reasonable to require user-land
filesystems to #define _KERNTYPES.

puffs(3): Replace realloc(x * y) with reallocarr

use a switch.

branches: 1.123.2; 1.123.4;
Typos.

fix error messages (use __func__, quotes)
also differentiate between canonical and non-relative.

Adjust to prototype.

fix error messages

Do not warn about relative path because of trailing slash

libpuffs calls realpath() to obtain an absolute path to use for mounting.
If the obtained path is different from the one given by the caller, a
warning is issued. This included the situation where the path passed by
the caller just have trailing slashes, a situation where we just want them
to be striped without a warning.

libpuffs support for fallocate and fdiscard operations

branches: 1.117.18;
fix crashes caused by using the results of getcontext()
after the caller returns.

branches: 1.116.4;
Call advlock method if supplied

Remove PUFFS_WITH_THREADS ifdef garbage, just use libc weak symbols.

unwhine lint

stat mount target path only if we're actually goint to use it.
This avoids errors from the comfd case when the mount target does
not exist in our local namespace.

set time_t compat where necessary

When using the top secret PUFFS_COMFD model, write out size of the
puffs kernel args before the contents. This allows parties which
have no clue about the contents to proxy the protocol easily.

Reenable pathconf now that the retval argument is register_t.

Actually, disable pathconf for a while longer: the retval needs a
little finetuning (not that everyone is jumping right to using it,
but fixing will have to wait until tomorrow, and who knows what
i'll forget during the night and how long it will eventually take).

signal pathconf in the vector of supported ops

* support extended attributes
* bump major due to structure growth
* add some spare space
* remove ABI sillyness

remove three years out-of-date comment

Add convenience routine puffs_unmountonsignal(), which does exactly that.

Use typecast instead of ULL for a less whiny lint on LP64.

Rewrite puffs_exit() in terms of the PUFFSOP_UNMOUNT kernel request.
Now server exit is a lot quite cleaner than previously.

Set high bit on server->kernel requests to better differentiate
between the direction of the request.

Add an error handler which syslogs but does not abort() and make it
the default. Most servers just shovel bits from their backend to
the kernel without checking the contents and dumping core in case
the backend gives garbage is a bit harsh.

Make it possible to supply parameters to puffs_init() later (but
before puffs_mount()).

Dispatch puffs_node_abortop().

Note: We use the storage of puffs_cache_write from puffs_ops for
this purpose. It's not issued by the kernel and hence currently
unused, and this saves us from the trouble of bumping the lib major
version.

fix some sign comparison warnings

Actually we need 'long int' to avoif a stupid 'truncating long to int'
warning.

Change type of 'n' to 'int' so that it can be printed with %d.
The domain of the value is [-1..sizeof (int)] so a type of 'int' is fine.

Close write-end of daemonpipe in reader to get EOF if the writer
goes awol.

Add puffs_cancel(), which allows to back down after puffs_init()
and before puffs_mount().
(this should actually be integrated with puffs_exit(), but that
routine is currently so non-sensible that I didn't bother now)

add:
puffs_setspecific(pu, private)
Can be used to set the specific data after the call to
puffs_init().

branches: 1.92.4; 1.92.8;
Replace void * by puffs_cookie_t where appropriate. No functional change.

clear cached puffs_cc's in puffs_exit()

* make sure we come out of mainloop in the main context. otherwise in
certain cases the server would jump to hyperspace when unmounted
* sprinkle some debug prints

branches: 1.89.6;
Implement lazy contexts for file systems using puffs_mainloop().
Previously each request was executed on its own callcontext and
switched to every time the request was being processed. Now requests
borrow the mainloop context and switch only if/when they yield.
This takes two context switches away from a file system request
bringing down the typical amounts 2->0 (e.g. dtfs) and 4->2 (e.g.
psshfs).

The interfaces for manually executing requests changed a bit:
puffs_dispatch_create() and puffs_dispatch_exec() must now be used.
They are not tested, as nothing in-tree wants them and I doubt
anyone else is really interested in them either.

Also do some misc code cleanup related to execution contexts. The
"work-in-progress checkpoint" committed over a year ago was starting
to look slightly weed-infested.

Cache a number of execution contexts instead of re-initializing
them every time. Speeds up pure in-memory file systems such as
sysctlfs or dtfs quite a bit. For actual I/O-workhorses the result
is of course less tasty.

Use PROT_NONE page to protect bottom of stack from overrun

Introduce PUFFS_STACKSIZE_MIN to avoid overloading 0 and getting
a warning from the library when really wanting to say "just the
minimum, please".

liberate gas-huffin crack-puffin' assert

fix pasto

Also pass fromname in the "distributed" case.

Provide some transparent "support" for "distributed" file systems.
If the env variable PUFFS_COMFD is set, the descriptor value
contained in it is used for communication instead of opening
/dev/puffs and doing mount(2).

This feature is obviously very undocumented and should not be used
without adult supervision.

Add a public interface for setting a cookie mapper.

Send a response message for flush operations from the kernel instead
of abusing the return value of write(2).

Part 3/n of extensive changes to message handling:

Get rid of the original puffs_req(3) framework and use puffs_framebuf(3)
instead for file system requests. It has the advantage of being
suitable for transporting a distributed message passing protocol
and therefore us being able to run the file system server on any
host.

Ok, puffs is not quite here yet: libpuffs needs to grow request
routing support and the message contents need to be munged into a
host independent format. Saying which format would be telling,
but it might begin with an X, end in an L and have the 13th character
in the middle. Keep an eye out for the sequels: Parts 3+m/n.

* allocate pcc as a slice from the stack instead of allocating it
separately
* provide puffs_cc_getcc()

This is in preparation for the removal of you-should-guess-what as
an argument to routines here and there and everywhere.

print more info when receiving an abort

puffs_mount():
* initialize values better
* always go through "fail" to see if puffs_daemon() needs to be
woken up
* open device r/w instead of r/o

Introduce puffs_daemon(), which works like daemon(3) except that it
also synchronizes with puffs_mount() and does not return (exit) in the
parent process until the file system has been mounted. This makes
it possible to reliably run e.g. mount_foo jippi /kai ; cd /kai/ee

Open the kernel descriptor as part of mount(), not init(). Then
it doesn't matter if someone fork()s or does other tricks between
init() and mount() (and besides, now it's where it logically should be).

Pull the daemonizing code out of the library mainloop into the file
servers. Calling daemon() (i.e. fork()ing) inside a library can
cause nice surprises for e.g. threaded programs. As discussed with
Greg Oster & others.

Sprinkle some hackish locks here and there, just enough to allow
libp2k & rump to run multithreaded.

Add possibility to set pre- and post callbacks which will be called
for all operations before or after executing the specific callback
(the api is not be final yet, though).

Add some very initial support for a threading worker model as an
alternative to the (vastly superior ;) continuation model. This
is very preliminary stuff and not compiled by default (which it
even won't do without some other patches I cannot commit yet).

The raison d'commit of the patch is a snippet which ensures proper
in-order dispatching of all operations, including those which don't
require a response. Previously many of them would be dispatched
simultaneosly, e.g. fsync and reclaim on the same node, which
obviously isn't all that nice for correct operation.

mmap stacks, makes it possible to align them

Make pu_haskq a flag instead of its own variable.

Add puffs_cc_schedule() which marks a pcc runnable and will pass
execution to it when in the main loop the next time.

Part 1/n of some pretty extensive changes to how the kernel module
interacts with the userspace file server:

* since the kernel-user communication is not purely request-response
anymore (hasn't been since 2006), try to rename some "request" to
"message". more similar mangling will take place in the future.

* completely rework how messages are allocated. previously most of
them were borrowed from the stack (originally *all* of them),
but now always allocate dynamically. this makes the structure
of the code much cleaner. also makes it possible to fix a
locking order violation. it enables plenty of future enhancements.

* start generalizing the transport interface to be independent of puffs

* move transport interface to read/write instead of ioctl. the
old one had legacy design problems, and besides, ioctl's suck.
implement a very generic version for now; this will be
worked on later hopefully some day reaching "highly optimized".

* implement libpuffs support behind existing library request
interfaces. this will change eventually (I hate those interfaces)

Pass reason string to pu_errnotify. Print it in the default
version before aborting.

silence lint. also noticed by xtraeme

Add error notifications, which are used to deliver errors from the
kernel to the file server for silly things the file server did,
e.g. attempting to create a file with size VSIZENOTSET. The file
server can handle these as it chooses, but the default action is
for it to throw its hands in the air and sing "goodbye, cruel world,
it's over, walk on by".

branches: 1.62.2;
run realpath() on the mountpath

branches: 1.61.4;
Add puffs_framev_enqueue_waitevent, which waits for read/write/error
events from an i/o descriptor. It yields until an event happens
and continues execution when the event loop notices the event has
happened.

Don't float puffs_kargs needlessly around after mount, saves
a couple k of memory (and there was much rejoycing!)

Use ioctl(PUFFSREQSIZEOP) instead of mount(MNT_GETARGS) to fetch
the maximum request size after mount. Calling mount(MNT_GETARGS)
from the file server is currently not kosher, as it vrele()s the
root vnode, potentially causing an inactive, which the file server
cannot handle while it itself is inactive in the kernel (deadlock).

* add mntfromname parameter to puffs_init()
* set it in file servers

Add new 'data_len' parameter to mount system call.
Add an additional call with MNT_GETARGS to retrieve the modified
information instead of relying on the kernel code abusing the mount
interface during mount.

Make puffs_framebuf better suited from environments with multiple sources:

* in addition add/remove, allow enable/disable, which can be used
to control events for descriptors without having to remove all the
data associated with them
* add directsend/receive, which can be used to pass the same buffer
from the caller to read/writeframe and back again
* add flags to enqueue functions and allow urgent buffers to be
processed as the next PDU

* reorder calls to get results processed even if our I/O descriptors
go back to zero
* don't give up on the read-side of an fd if there is still unreceived data

Get rid of the "int *refs" parameter to inactive: the same can be
accomplished now with puffs_setbacks.

print a specific warning if /dev/puffs cannot be opened in init

Move puffs to a two clause license where it already isn't so. And
as agc pointed out, even files with the third clause were already
effectively two clause because of a slight bug in the language...

remember to put the results of the final operations (including unmount)
to the kernel in mainloop

* stuff results back to kernel before calling kevent - the file server
loop function might generate some results. and this is still "after"
event handling (except for the first call, but I'm not too keen on
optimizing for that)
* don't be such a baby about EINTR from kevent(). if we get it, suck
it up and continue instead of quitting

Reflect kernel changes for supplying root node information in mount
instead of puffs_start(). Get completely rid of puffs_start(), as
everything it used to do is now handled by the mount routine.
Introduce an optional pre-mount call puffs_setrootinfo() for setting
non-default root node information. As the old puffs_mount() is
now virtually useless, say byebye to it and rename the old
puffs_domount() to puffs_mount(), but add a root cookie parameter
to compensate for the late puffs_start().

switch the arbitrary default errno value for network i/o from
ECONNABORTED to ECONNRESET, which is at least a little bit better

memset freshly allocated puffs_usermount to 0

Step 3 and currently final step of a more general purpose event loop:
support removal and addition of i/o file descriptors on the fly.

* detect closed file descriptors
* automatically free waiters of a dead file descriptor
* give the file server the possibility to specify a callback which
notifies of a dead file descriptor
* move loop function to be a property of the mainloop instead of
framebuf (doesn't change effective behaviour)
* add the possibility to configure a timespec parameter which
attempts to call the loop function periodically
* move the event loop functions from the puffs_framebuf namespace
to puffs_framev to differential between pure memory management
functions

move puffs request dispatching code into it's own module.
no functional change

Step 2 of general-purpose eventloop: get rid of puffs_framebuf_eventloop().
puffs_mainloop() now handles that functionality as well.

Add some kludges to be able to execute requests on the regular program
stack instead of the continuation stack. This is for lib/36011, where
pthread gets confused since we aren't running on the regular stack.
I'm not really sure which direction to go to with this quite yet, so
make the hack hard to enable on purpose. The whole request dispatch
code needs cleaning anyway.

Remove the puffs_usermount argument from puffs_req_handle(). It can
be found from the other arguments and this way puffs_req_handle() is
more symmetric with the other puffs_req() routines.

Return 0 from puffs_mainloop() if no error occurs.
pointed out by Phil Nelson

Add puffs_setback(), which can be used to set setbacks for nodes before
the call returns to the kernel. Currently it can be called only from
open(), mmap(), remove() and rmdir(). (But that might change)

assert that the file server doesn't try to return more cookies than
it was supposed to

alloc to set flags when setting fhsize

some routines for setting mount parameters

Instead of keeping on adding parameters to puffs_mount(), make it
only take the bare essentials, which currently means removing
"maxreqlen" from the argument list (all current callers I'm aware
of set it as 0 anyway). Introduce puffs_init(), which provides a
context for setting various parameters and puffs_domount(), which
can be used to mount the file system. Keep puffs_mount() as a
shortcut for the above two for simple file systems.

Bump development ABI version to 13. After all, it's Friday the 13th.
Watch out! Bad things can happen on Friday the 13th. --No carrier--

Hide struct puffs_usermount from the rest of the world and provide
accessors for interesting data in it. Namely, you can now get
pu->pu_privdata with puffs_getspecific(), pu->pu_pn_root with
puffs_set/getroot() and pu->pu_maxreqlen with puffs_getmaxreqlen().

* support nodetofh and fhtonode callbacks
* use new signature of node_readdir

initial support for cacheops

Until someone can sort non-priviledged mounts and kauth, force
MNT_NOSUID | MNT_NODEV for mounts with geteuid() != 0. I'm tired
of typing them on the command line every time I test a file system.

dump return values for operations

Introduce puffs_path_walkcmp(), which is to be called from nodewalk
and compares the path of the node against the given pathobject.
Also make comparison method take a flag to indicate if it should
check if the second path is a true prefix of the first.

plus some namespace cleanup

* get rid of the holy kernel-given ISDOTDOT, and instead define a
macro which does strcmp against ".." and (the untranslated)
componentname
* make PUFFS_FLAG_BUILDPATH build paths also if dotdot is the case,
and adapt the regular path objects to this
* make nullfs lookup readable because we can now get rid of dotdot
processing there

add interfaces for suspending the file system

Be consistent with the rest of the framework about the argument
order of puffs_docc and puffs_dopreq

be consistent in naming request handlers

revoke revoke

Checkpoint some more work in progress: for the benefit those file
system backends which operate purely based on paths, push out more
path management into the library and make path management more
abstract: enable a file system to define a bunch of path management
callbacks, which are used by the framework. Management of normal
/this/is/a/path type paths is provided by the library.

For directory renames we need to rename all the child nodes for
the given directory if the file system wants paths (PUFFS_FLAG_BUILDPATH).
Do this by walking the nodelist and adjusting the path prefix of
each matching node.

* don't chdir in possible daemon() call for the benefit of file systems
which specify a relative path as the root
* if (buildpath), build link source full pathname for the duration of
the operation

* get rid of the mount callback; it's no great surprise to the
server that it needs to mount the file system backend if it wants
to call mount
* provide some options for getmntopts(), assume that callers will parse
command line (or fstab) args
* reorganize the puffs_cc interface just a bit, preparing for a bigger
revamp later

* inform kernel of our version
* homegrown lib ABI check (to avoid bumping major a ludicrillion times
before reaching stability)

checkpoint some experimental work-in-progress, namely:

Add support for having multiple outstanding operations. This is done
by exposing enough interfaces so that it is convenient to have the
main event loop in the implementation itself and by providing a
continuation framework for convinient blocking and rescheduling.

works fine, but will undergo further cleanup & development

CID4360: don't leak `buf' upon return
ok pooka@

outline & export code for interfacing with kernel GET/PUT requests

g/c out-of-date comment

support mmap

cleanup generated stuff from comments

rototill the operation callbacks: unify the ops under struct puffs_ops
and namespace them

adapt to kernel change:
Allow multiple requests to be transferred in each GET/PUTOP

PUFFSFLAG -> PUFFS_FLAG to be consistent with the kernel

build & send implemented operations list to kernel

summon daemon(3) in mainloop unless the nodaemon flag is given

* call statvfs() at mount() time, pass that info along
* start is gone

shuffle flags a bit

* catch up with kernel changes
* better error handling when mounting

* support FAF (Fire-And-Forget) class operations
* adapt to new lookup

pass rdev for specfs support

handle PUFFS_VN_INACTIVE

pass lint; fix install.

add libpuffs - the userspace programming interface for puffs

This work was initially started and completed for Google SoC 2005
and tweaked to work a bit better in the past few weeks. While
being far from complete, it is functional enough to be able and
stable to host a fairly general-purpose in-memory file system in
userspace. Even so, puffs should be considered experimental and
no binary compatibility for interfaces or crash-freedom or zero
security implications should be relied upon just yet.

The GSoC project was mentored by William Studenmund and the final
review for the code was done by Christos.

Allow calling puffs_mount(3) before puffs_daemon(3)

puffs_daemon(3) creates a pipe before forking, and the parent process
waits for the child to either complete puffs_mount(3) or fail. If a
user calls puffs_daemon(3) after puffs_mount(3), the function
deadlocks. While this error-reporting functionality is really a nice
thing to have, deadlocking is not great. If the filesystem has already
been mounted, puffs_mount(3) should just daemonize the process and
return.

This became an issue because fuse_daemonize(3) in FUSE API had no such
requirement and some FUSE filesystems in the wild suffered deadlocks.

Use size_t in place of voff_t in puffs_setrootinfo(3)

Chuck Silvers pointed out that voff_t was also supposed to be
kernel-only. The correct type to use in userland would be off_t, but
since changing vsize_t to either voff_t or off_t is an ABI change on
32-bit platforms, we use size_t knowing that it is technically
incorrect.

Avoid using register_t or vsize_t in <puffs.h>

<puffs.h> is a user-space header, and should not use any of
kernel-only types. It's not reasonable to require user-land
filesystems to #define _KERNTYPES.

puffs(3): Replace realloc(x * y) with reallocarr

use a switch.

branches: 1.123.2; 1.123.4;
Typos.

fix error messages (use __func__, quotes)
also differentiate between canonical and non-relative.

Adjust to prototype.

fix error messages

Do not warn about relative path because of trailing slash

libpuffs calls realpath() to obtain an absolute path to use for mounting.
If the obtained path is different from the one given by the caller, a
warning is issued. This included the situation where the path passed by
the caller just have trailing slashes, a situation where we just want them
to be striped without a warning.

libpuffs support for fallocate and fdiscard operations

branches: 1.117.18;
fix crashes caused by using the results of getcontext()
after the caller returns.

branches: 1.116.4;
Call advlock method if supplied

Remove PUFFS_WITH_THREADS ifdef garbage, just use libc weak symbols.

unwhine lint

stat mount target path only if we're actually goint to use it.
This avoids errors from the comfd case when the mount target does
not exist in our local namespace.

set time_t compat where necessary

When using the top secret PUFFS_COMFD model, write out size of the
puffs kernel args before the contents. This allows parties which
have no clue about the contents to proxy the protocol easily.

Reenable pathconf now that the retval argument is register_t.

Actually, disable pathconf for a while longer: the retval needs a
little finetuning (not that everyone is jumping right to using it,
but fixing will have to wait until tomorrow, and who knows what
i'll forget during the night and how long it will eventually take).

signal pathconf in the vector of supported ops

* support extended attributes
* bump major due to structure growth
* add some spare space
* remove ABI sillyness

remove three years out-of-date comment

Add convenience routine puffs_unmountonsignal(), which does exactly that.

Use typecast instead of ULL for a less whiny lint on LP64.

Rewrite puffs_exit() in terms of the PUFFSOP_UNMOUNT kernel request.
Now server exit is a lot quite cleaner than previously.

Set high bit on server->kernel requests to better differentiate
between the direction of the request.

Add an error handler which syslogs but does not abort() and make it
the default. Most servers just shovel bits from their backend to
the kernel without checking the contents and dumping core in case
the backend gives garbage is a bit harsh.

Make it possible to supply parameters to puffs_init() later (but
before puffs_mount()).

Dispatch puffs_node_abortop().

Note: We use the storage of puffs_cache_write from puffs_ops for
this purpose. It's not issued by the kernel and hence currently
unused, and this saves us from the trouble of bumping the lib major
version.

fix some sign comparison warnings

Actually we need 'long int' to avoif a stupid 'truncating long to int'
warning.

Change type of 'n' to 'int' so that it can be printed with %d.
The domain of the value is [-1..sizeof (int)] so a type of 'int' is fine.

Close write-end of daemonpipe in reader to get EOF if the writer
goes awol.

Add puffs_cancel(), which allows to back down after puffs_init()
and before puffs_mount().
(this should actually be integrated with puffs_exit(), but that
routine is currently so non-sensible that I didn't bother now)

add:
puffs_setspecific(pu, private)
Can be used to set the specific data after the call to
puffs_init().

branches: 1.92.4; 1.92.8;
Replace void * by puffs_cookie_t where appropriate. No functional change.

clear cached puffs_cc's in puffs_exit()

* make sure we come out of mainloop in the main context. otherwise in
certain cases the server would jump to hyperspace when unmounted
* sprinkle some debug prints

branches: 1.89.6;
Implement lazy contexts for file systems using puffs_mainloop().
Previously each request was executed on its own callcontext and
switched to every time the request was being processed. Now requests
borrow the mainloop context and switch only if/when they yield.
This takes two context switches away from a file system request
bringing down the typical amounts 2->0 (e.g. dtfs) and 4->2 (e.g.
psshfs).

The interfaces for manually executing requests changed a bit:
puffs_dispatch_create() and puffs_dispatch_exec() must now be used.
They are not tested, as nothing in-tree wants them and I doubt
anyone else is really interested in them either.

Also do some misc code cleanup related to execution contexts. The
"work-in-progress checkpoint" committed over a year ago was starting
to look slightly weed-infested.

Cache a number of execution contexts instead of re-initializing
them every time. Speeds up pure in-memory file systems such as
sysctlfs or dtfs quite a bit. For actual I/O-workhorses the result
is of course less tasty.

Use PROT_NONE page to protect bottom of stack from overrun

Introduce PUFFS_STACKSIZE_MIN to avoid overloading 0 and getting
a warning from the library when really wanting to say "just the
minimum, please".

liberate gas-huffin crack-puffin' assert

fix pasto

Also pass fromname in the "distributed" case.

Provide some transparent "support" for "distributed" file systems.
If the env variable PUFFS_COMFD is set, the descriptor value
contained in it is used for communication instead of opening
/dev/puffs and doing mount(2).

This feature is obviously very undocumented and should not be used
without adult supervision.

Add a public interface for setting a cookie mapper.

Send a response message for flush operations from the kernel instead
of abusing the return value of write(2).

Part 3/n of extensive changes to message handling:

Get rid of the original puffs_req(3) framework and use puffs_framebuf(3)
instead for file system requests. It has the advantage of being
suitable for transporting a distributed message passing protocol
and therefore us being able to run the file system server on any
host.

Ok, puffs is not quite here yet: libpuffs needs to grow request
routing support and the message contents need to be munged into a
host independent format. Saying which format would be telling,
but it might begin with an X, end in an L and have the 13th character
in the middle. Keep an eye out for the sequels: Parts 3+m/n.

* allocate pcc as a slice from the stack instead of allocating it
separately
* provide puffs_cc_getcc()

This is in preparation for the removal of you-should-guess-what as
an argument to routines here and there and everywhere.

print more info when receiving an abort

puffs_mount():
* initialize values better
* always go through "fail" to see if puffs_daemon() needs to be
woken up
* open device r/w instead of r/o

Introduce puffs_daemon(), which works like daemon(3) except that it
also synchronizes with puffs_mount() and does not return (exit) in the
parent process until the file system has been mounted. This makes
it possible to reliably run e.g. mount_foo jippi /kai ; cd /kai/ee

Open the kernel descriptor as part of mount(), not init(). Then
it doesn't matter if someone fork()s or does other tricks between
init() and mount() (and besides, now it's where it logically should be).

Pull the daemonizing code out of the library mainloop into the file
servers. Calling daemon() (i.e. fork()ing) inside a library can
cause nice surprises for e.g. threaded programs. As discussed with
Greg Oster & others.

Sprinkle some hackish locks here and there, just enough to allow
libp2k & rump to run multithreaded.

Add possibility to set pre- and post callbacks which will be called
for all operations before or after executing the specific callback
(the api is not be final yet, though).

Add some very initial support for a threading worker model as an
alternative to the (vastly superior ;) continuation model. This
is very preliminary stuff and not compiled by default (which it
even won't do without some other patches I cannot commit yet).

The raison d'commit of the patch is a snippet which ensures proper
in-order dispatching of all operations, including those which don't
require a response. Previously many of them would be dispatched
simultaneosly, e.g. fsync and reclaim on the same node, which
obviously isn't all that nice for correct operation.

mmap stacks, makes it possible to align them

Make pu_haskq a flag instead of its own variable.

Add puffs_cc_schedule() which marks a pcc runnable and will pass
execution to it when in the main loop the next time.

Part 1/n of some pretty extensive changes to how the kernel module
interacts with the userspace file server:

* since the kernel-user communication is not purely request-response
anymore (hasn't been since 2006), try to rename some "request" to
"message". more similar mangling will take place in the future.

* completely rework how messages are allocated. previously most of
them were borrowed from the stack (originally *all* of them),
but now always allocate dynamically. this makes the structure
of the code much cleaner. also makes it possible to fix a
locking order violation. it enables plenty of future enhancements.

* start generalizing the transport interface to be independent of puffs

* move transport interface to read/write instead of ioctl. the
old one had legacy design problems, and besides, ioctl's suck.
implement a very generic version for now; this will be
worked on later hopefully some day reaching "highly optimized".

* implement libpuffs support behind existing library request
interfaces. this will change eventually (I hate those interfaces)

Pass reason string to pu_errnotify. Print it in the default
version before aborting.

silence lint. also noticed by xtraeme

Add error notifications, which are used to deliver errors from the
kernel to the file server for silly things the file server did,
e.g. attempting to create a file with size VSIZENOTSET. The file
server can handle these as it chooses, but the default action is
for it to throw its hands in the air and sing "goodbye, cruel world,
it's over, walk on by".

branches: 1.62.2;
run realpath() on the mountpath

branches: 1.61.4;
Add puffs_framev_enqueue_waitevent, which waits for read/write/error
events from an i/o descriptor. It yields until an event happens
and continues execution when the event loop notices the event has
happened.

Don't float puffs_kargs needlessly around after mount, saves
a couple k of memory (and there was much rejoycing!)

Use ioctl(PUFFSREQSIZEOP) instead of mount(MNT_GETARGS) to fetch
the maximum request size after mount. Calling mount(MNT_GETARGS)
from the file server is currently not kosher, as it vrele()s the
root vnode, potentially causing an inactive, which the file server
cannot handle while it itself is inactive in the kernel (deadlock).

* add mntfromname parameter to puffs_init()
* set it in file servers

Add new 'data_len' parameter to mount system call.
Add an additional call with MNT_GETARGS to retrieve the modified
information instead of relying on the kernel code abusing the mount
interface during mount.

Make puffs_framebuf better suited from environments with multiple sources:

* in addition add/remove, allow enable/disable, which can be used
to control events for descriptors without having to remove all the
data associated with them
* add directsend/receive, which can be used to pass the same buffer
from the caller to read/writeframe and back again
* add flags to enqueue functions and allow urgent buffers to be
processed as the next PDU

* reorder calls to get results processed even if our I/O descriptors
go back to zero
* don't give up on the read-side of an fd if there is still unreceived data

Get rid of the "int *refs" parameter to inactive: the same can be
accomplished now with puffs_setbacks.

print a specific warning if /dev/puffs cannot be opened in init

Move puffs to a two clause license where it already isn't so. And
as agc pointed out, even files with the third clause were already
effectively two clause because of a slight bug in the language...

remember to put the results of the final operations (including unmount)
to the kernel in mainloop

* stuff results back to kernel before calling kevent - the file server
loop function might generate some results. and this is still "after"
event handling (except for the first call, but I'm not too keen on
optimizing for that)
* don't be such a baby about EINTR from kevent(). if we get it, suck
it up and continue instead of quitting

Reflect kernel changes for supplying root node information in mount
instead of puffs_start(). Get completely rid of puffs_start(), as
everything it used to do is now handled by the mount routine.
Introduce an optional pre-mount call puffs_setrootinfo() for setting
non-default root node information. As the old puffs_mount() is
now virtually useless, say byebye to it and rename the old
puffs_domount() to puffs_mount(), but add a root cookie parameter
to compensate for the late puffs_start().

switch the arbitrary default errno value for network i/o from
ECONNABORTED to ECONNRESET, which is at least a little bit better

memset freshly allocated puffs_usermount to 0

Step 3 and currently final step of a more general purpose event loop:
support removal and addition of i/o file descriptors on the fly.

* detect closed file descriptors
* automatically free waiters of a dead file descriptor
* give the file server the possibility to specify a callback which
notifies of a dead file descriptor
* move loop function to be a property of the mainloop instead of
framebuf (doesn't change effective behaviour)
* add the possibility to configure a timespec parameter which
attempts to call the loop function periodically
* move the event loop functions from the puffs_framebuf namespace
to puffs_framev to differential between pure memory management
functions

move puffs request dispatching code into it's own module.
no functional change

Step 2 of general-purpose eventloop: get rid of puffs_framebuf_eventloop().
puffs_mainloop() now handles that functionality as well.

Add some kludges to be able to execute requests on the regular program
stack instead of the continuation stack. This is for lib/36011, where
pthread gets confused since we aren't running on the regular stack.
I'm not really sure which direction to go to with this quite yet, so
make the hack hard to enable on purpose. The whole request dispatch
code needs cleaning anyway.

Remove the puffs_usermount argument from puffs_req_handle(). It can
be found from the other arguments and this way puffs_req_handle() is
more symmetric with the other puffs_req() routines.

Return 0 from puffs_mainloop() if no error occurs.
pointed out by Phil Nelson

Add puffs_setback(), which can be used to set setbacks for nodes before
the call returns to the kernel. Currently it can be called only from
open(), mmap(), remove() and rmdir(). (But that might change)

assert that the file server doesn't try to return more cookies than
it was supposed to

alloc to set flags when setting fhsize

some routines for setting mount parameters

Instead of keeping on adding parameters to puffs_mount(), make it
only take the bare essentials, which currently means removing
"maxreqlen" from the argument list (all current callers I'm aware
of set it as 0 anyway). Introduce puffs_init(), which provides a
context for setting various parameters and puffs_domount(), which
can be used to mount the file system. Keep puffs_mount() as a
shortcut for the above two for simple file systems.

Bump development ABI version to 13. After all, it's Friday the 13th.
Watch out! Bad things can happen on Friday the 13th. --No carrier--

Hide struct puffs_usermount from the rest of the world and provide
accessors for interesting data in it. Namely, you can now get
pu->pu_privdata with puffs_getspecific(), pu->pu_pn_root with
puffs_set/getroot() and pu->pu_maxreqlen with puffs_getmaxreqlen().

* support nodetofh and fhtonode callbacks
* use new signature of node_readdir

initial support for cacheops

Until someone can sort non-priviledged mounts and kauth, force
MNT_NOSUID | MNT_NODEV for mounts with geteuid() != 0. I'm tired
of typing them on the command line every time I test a file system.

dump return values for operations

Introduce puffs_path_walkcmp(), which is to be called from nodewalk
and compares the path of the node against the given pathobject.
Also make comparison method take a flag to indicate if it should
check if the second path is a true prefix of the first.

plus some namespace cleanup

* get rid of the holy kernel-given ISDOTDOT, and instead define a
macro which does strcmp against ".." and (the untranslated)
componentname
* make PUFFS_FLAG_BUILDPATH build paths also if dotdot is the case,
and adapt the regular path objects to this
* make nullfs lookup readable because we can now get rid of dotdot
processing there

add interfaces for suspending the file system

Be consistent with the rest of the framework about the argument
order of puffs_docc and puffs_dopreq

be consistent in naming request handlers

revoke revoke

Checkpoint some more work in progress: for the benefit those file
system backends which operate purely based on paths, push out more
path management into the library and make path management more
abstract: enable a file system to define a bunch of path management
callbacks, which are used by the framework. Management of normal
/this/is/a/path type paths is provided by the library.

For directory renames we need to rename all the child nodes for
the given directory if the file system wants paths (PUFFS_FLAG_BUILDPATH).
Do this by walking the nodelist and adjusting the path prefix of
each matching node.

* don't chdir in possible daemon() call for the benefit of file systems
which specify a relative path as the root
* if (buildpath), build link source full pathname for the duration of
the operation

* get rid of the mount callback; it's no great surprise to the
server that it needs to mount the file system backend if it wants
to call mount
* provide some options for getmntopts(), assume that callers will parse
command line (or fstab) args
* reorganize the puffs_cc interface just a bit, preparing for a bigger
revamp later

* inform kernel of our version
* homegrown lib ABI check (to avoid bumping major a ludicrillion times
before reaching stability)

checkpoint some experimental work-in-progress, namely:

Add support for having multiple outstanding operations. This is done
by exposing enough interfaces so that it is convenient to have the
main event loop in the implementation itself and by providing a
continuation framework for convinient blocking and rescheduling.

works fine, but will undergo further cleanup & development

CID4360: don't leak `buf' upon return
ok pooka@

outline & export code for interfacing with kernel GET/PUT requests

g/c out-of-date comment

support mmap

cleanup generated stuff from comments

rototill the operation callbacks: unify the ops under struct puffs_ops
and namespace them

adapt to kernel change:
Allow multiple requests to be transferred in each GET/PUTOP

PUFFSFLAG -> PUFFS_FLAG to be consistent with the kernel

build & send implemented operations list to kernel

summon daemon(3) in mainloop unless the nodaemon flag is given

* call statvfs() at mount() time, pass that info along
* start is gone

shuffle flags a bit

* catch up with kernel changes
* better error handling when mounting

* support FAF (Fire-And-Forget) class operations
* adapt to new lookup

pass rdev for specfs support

handle PUFFS_VN_INACTIVE

pass lint; fix install.

add libpuffs - the userspace programming interface for puffs

This work was initially started and completed for Google SoC 2005
and tweaked to work a bit better in the past few weeks. While
being far from complete, it is functional enough to be able and
stable to host a fairly general-purpose in-memory file system in
userspace. Even so, puffs should be considered experimental and
no binary compatibility for interfaces or crash-freedom or zero
security implications should be relied upon just yet.

The GSoC project was mentored by William Studenmund and the final
review for the code was done by Christos.

Use size_t in place of voff_t in puffs_setrootinfo(3)

Chuck Silvers pointed out that voff_t was also supposed to be
kernel-only. The correct type to use in userland would be off_t, but
since changing vsize_t to either voff_t or off_t is an ABI change on
32-bit platforms, we use size_t knowing that it is technically
incorrect.

Avoid using register_t or vsize_t in <puffs.h>

<puffs.h> is a user-space header, and should not use any of
kernel-only types. It's not reasonable to require user-land
filesystems to #define _KERNTYPES.

puffs(3): Replace realloc(x * y) with reallocarr

use a switch.

branches: 1.123.2; 1.123.4;
Typos.

fix error messages (use __func__, quotes)
also differentiate between canonical and non-relative.

Adjust to prototype.

fix error messages

Do not warn about relative path because of trailing slash

libpuffs calls realpath() to obtain an absolute path to use for mounting.
If the obtained path is different from the one given by the caller, a
warning is issued. This included the situation where the path passed by
the caller just have trailing slashes, a situation where we just want them
to be striped without a warning.

libpuffs support for fallocate and fdiscard operations

branches: 1.117.18;
fix crashes caused by using the results of getcontext()
after the caller returns.

branches: 1.116.4;
Call advlock method if supplied

Remove PUFFS_WITH_THREADS ifdef garbage, just use libc weak symbols.

unwhine lint

stat mount target path only if we're actually goint to use it.
This avoids errors from the comfd case when the mount target does
not exist in our local namespace.

set time_t compat where necessary

When using the top secret PUFFS_COMFD model, write out size of the
puffs kernel args before the contents. This allows parties which
have no clue about the contents to proxy the protocol easily.

Reenable pathconf now that the retval argument is register_t.

Actually, disable pathconf for a while longer: the retval needs a
little finetuning (not that everyone is jumping right to using it,
but fixing will have to wait until tomorrow, and who knows what
i'll forget during the night and how long it will eventually take).

signal pathconf in the vector of supported ops

* support extended attributes
* bump major due to structure growth
* add some spare space
* remove ABI sillyness

remove three years out-of-date comment

Add convenience routine puffs_unmountonsignal(), which does exactly that.

Use typecast instead of ULL for a less whiny lint on LP64.

Rewrite puffs_exit() in terms of the PUFFSOP_UNMOUNT kernel request.
Now server exit is a lot quite cleaner than previously.

Set high bit on server->kernel requests to better differentiate
between the direction of the request.

Add an error handler which syslogs but does not abort() and make it
the default. Most servers just shovel bits from their backend to
the kernel without checking the contents and dumping core in case
the backend gives garbage is a bit harsh.

Make it possible to supply parameters to puffs_init() later (but
before puffs_mount()).

Dispatch puffs_node_abortop().

Note: We use the storage of puffs_cache_write from puffs_ops for
this purpose. It's not issued by the kernel and hence currently
unused, and this saves us from the trouble of bumping the lib major
version.

fix some sign comparison warnings

Actually we need 'long int' to avoif a stupid 'truncating long to int'
warning.

Change type of 'n' to 'int' so that it can be printed with %d.
The domain of the value is [-1..sizeof (int)] so a type of 'int' is fine.

Close write-end of daemonpipe in reader to get EOF if the writer
goes awol.

Add puffs_cancel(), which allows to back down after puffs_init()
and before puffs_mount().
(this should actually be integrated with puffs_exit(), but that
routine is currently so non-sensible that I didn't bother now)

add:
puffs_setspecific(pu, private)
Can be used to set the specific data after the call to
puffs_init().

branches: 1.92.4; 1.92.8;
Replace void * by puffs_cookie_t where appropriate. No functional change.

clear cached puffs_cc's in puffs_exit()

* make sure we come out of mainloop in the main context. otherwise in
certain cases the server would jump to hyperspace when unmounted
* sprinkle some debug prints

branches: 1.89.6;
Implement lazy contexts for file systems using puffs_mainloop().
Previously each request was executed on its own callcontext and
switched to every time the request was being processed. Now requests
borrow the mainloop context and switch only if/when they yield.
This takes two context switches away from a file system request
bringing down the typical amounts 2->0 (e.g. dtfs) and 4->2 (e.g.
psshfs).

The interfaces for manually executing requests changed a bit:
puffs_dispatch_create() and puffs_dispatch_exec() must now be used.
They are not tested, as nothing in-tree wants them and I doubt
anyone else is really interested in them either.

Also do some misc code cleanup related to execution contexts. The
"work-in-progress checkpoint" committed over a year ago was starting
to look slightly weed-infested.

Cache a number of execution contexts instead of re-initializing
them every time. Speeds up pure in-memory file systems such as
sysctlfs or dtfs quite a bit. For actual I/O-workhorses the result
is of course less tasty.

Use PROT_NONE page to protect bottom of stack from overrun

Introduce PUFFS_STACKSIZE_MIN to avoid overloading 0 and getting
a warning from the library when really wanting to say "just the
minimum, please".

liberate gas-huffin crack-puffin' assert

fix pasto

Also pass fromname in the "distributed" case.

Provide some transparent "support" for "distributed" file systems.
If the env variable PUFFS_COMFD is set, the descriptor value
contained in it is used for communication instead of opening
/dev/puffs and doing mount(2).

This feature is obviously very undocumented and should not be used
without adult supervision.

Add a public interface for setting a cookie mapper.

Send a response message for flush operations from the kernel instead
of abusing the return value of write(2).

Part 3/n of extensive changes to message handling:

Get rid of the original puffs_req(3) framework and use puffs_framebuf(3)
instead for file system requests. It has the advantage of being
suitable for transporting a distributed message passing protocol
and therefore us being able to run the file system server on any
host.

Ok, puffs is not quite here yet: libpuffs needs to grow request
routing support and the message contents need to be munged into a
host independent format. Saying which format would be telling,
but it might begin with an X, end in an L and have the 13th character
in the middle. Keep an eye out for the sequels: Parts 3+m/n.

* allocate pcc as a slice from the stack instead of allocating it
separately
* provide puffs_cc_getcc()

This is in preparation for the removal of you-should-guess-what as
an argument to routines here and there and everywhere.

print more info when receiving an abort

puffs_mount():
* initialize values better
* always go through "fail" to see if puffs_daemon() needs to be
woken up
* open device r/w instead of r/o

Introduce puffs_daemon(), which works like daemon(3) except that it
also synchronizes with puffs_mount() and does not return (exit) in the
parent process until the file system has been mounted. This makes
it possible to reliably run e.g. mount_foo jippi /kai ; cd /kai/ee

Open the kernel descriptor as part of mount(), not init(). Then
it doesn't matter if someone fork()s or does other tricks between
init() and mount() (and besides, now it's where it logically should be).

Pull the daemonizing code out of the library mainloop into the file
servers. Calling daemon() (i.e. fork()ing) inside a library can
cause nice surprises for e.g. threaded programs. As discussed with
Greg Oster & others.

Sprinkle some hackish locks here and there, just enough to allow
libp2k & rump to run multithreaded.

Add possibility to set pre- and post callbacks which will be called
for all operations before or after executing the specific callback
(the api is not be final yet, though).

Add some very initial support for a threading worker model as an
alternative to the (vastly superior ;) continuation model. This
is very preliminary stuff and not compiled by default (which it
even won't do without some other patches I cannot commit yet).

The raison d'commit of the patch is a snippet which ensures proper
in-order dispatching of all operations, including those which don't
require a response. Previously many of them would be dispatched
simultaneosly, e.g. fsync and reclaim on the same node, which
obviously isn't all that nice for correct operation.

mmap stacks, makes it possible to align them

Make pu_haskq a flag instead of its own variable.

Add puffs_cc_schedule() which marks a pcc runnable and will pass
execution to it when in the main loop the next time.

Part 1/n of some pretty extensive changes to how the kernel module
interacts with the userspace file server:

* since the kernel-user communication is not purely request-response
anymore (hasn't been since 2006), try to rename some "request" to
"message". more similar mangling will take place in the future.

* completely rework how messages are allocated. previously most of
them were borrowed from the stack (originally *all* of them),
but now always allocate dynamically. this makes the structure
of the code much cleaner. also makes it possible to fix a
locking order violation. it enables plenty of future enhancements.

* start generalizing the transport interface to be independent of puffs

* move transport interface to read/write instead of ioctl. the
old one had legacy design problems, and besides, ioctl's suck.
implement a very generic version for now; this will be
worked on later hopefully some day reaching "highly optimized".

* implement libpuffs support behind existing library request
interfaces. this will change eventually (I hate those interfaces)

Pass reason string to pu_errnotify. Print it in the default
version before aborting.

silence lint. also noticed by xtraeme

Add error notifications, which are used to deliver errors from the
kernel to the file server for silly things the file server did,
e.g. attempting to create a file with size VSIZENOTSET. The file
server can handle these as it chooses, but the default action is
for it to throw its hands in the air and sing "goodbye, cruel world,
it's over, walk on by".

branches: 1.62.2;
run realpath() on the mountpath

branches: 1.61.4;
Add puffs_framev_enqueue_waitevent, which waits for read/write/error
events from an i/o descriptor. It yields until an event happens
and continues execution when the event loop notices the event has
happened.

Don't float puffs_kargs needlessly around after mount, saves
a couple k of memory (and there was much rejoycing!)

Use ioctl(PUFFSREQSIZEOP) instead of mount(MNT_GETARGS) to fetch
the maximum request size after mount. Calling mount(MNT_GETARGS)
from the file server is currently not kosher, as it vrele()s the
root vnode, potentially causing an inactive, which the file server
cannot handle while it itself is inactive in the kernel (deadlock).

* add mntfromname parameter to puffs_init()
* set it in file servers

Add new 'data_len' parameter to mount system call.
Add an additional call with MNT_GETARGS to retrieve the modified
information instead of relying on the kernel code abusing the mount
interface during mount.

Make puffs_framebuf better suited from environments with multiple sources:

* in addition add/remove, allow enable/disable, which can be used
to control events for descriptors without having to remove all the
data associated with them
* add directsend/receive, which can be used to pass the same buffer
from the caller to read/writeframe and back again
* add flags to enqueue functions and allow urgent buffers to be
processed as the next PDU

* reorder calls to get results processed even if our I/O descriptors
go back to zero
* don't give up on the read-side of an fd if there is still unreceived data

Get rid of the "int *refs" parameter to inactive: the same can be
accomplished now with puffs_setbacks.

print a specific warning if /dev/puffs cannot be opened in init

Move puffs to a two clause license where it already isn't so. And
as agc pointed out, even files with the third clause were already
effectively two clause because of a slight bug in the language...

remember to put the results of the final operations (including unmount)
to the kernel in mainloop

* stuff results back to kernel before calling kevent - the file server
loop function might generate some results. and this is still "after"
event handling (except for the first call, but I'm not too keen on
optimizing for that)
* don't be such a baby about EINTR from kevent(). if we get it, suck
it up and continue instead of quitting

Reflect kernel changes for supplying root node information in mount
instead of puffs_start(). Get completely rid of puffs_start(), as
everything it used to do is now handled by the mount routine.
Introduce an optional pre-mount call puffs_setrootinfo() for setting
non-default root node information. As the old puffs_mount() is
now virtually useless, say byebye to it and rename the old
puffs_domount() to puffs_mount(), but add a root cookie parameter
to compensate for the late puffs_start().

switch the arbitrary default errno value for network i/o from
ECONNABORTED to ECONNRESET, which is at least a little bit better

memset freshly allocated puffs_usermount to 0

Step 3 and currently final step of a more general purpose event loop:
support removal and addition of i/o file descriptors on the fly.

* detect closed file descriptors
* automatically free waiters of a dead file descriptor
* give the file server the possibility to specify a callback which
notifies of a dead file descriptor
* move loop function to be a property of the mainloop instead of
framebuf (doesn't change effective behaviour)
* add the possibility to configure a timespec parameter which
attempts to call the loop function periodically
* move the event loop functions from the puffs_framebuf namespace
to puffs_framev to differential between pure memory management
functions

move puffs request dispatching code into it's own module.
no functional change

Step 2 of general-purpose eventloop: get rid of puffs_framebuf_eventloop().
puffs_mainloop() now handles that functionality as well.

Add some kludges to be able to execute requests on the regular program
stack instead of the continuation stack. This is for lib/36011, where
pthread gets confused since we aren't running on the regular stack.
I'm not really sure which direction to go to with this quite yet, so
make the hack hard to enable on purpose. The whole request dispatch
code needs cleaning anyway.

Remove the puffs_usermount argument from puffs_req_handle(). It can
be found from the other arguments and this way puffs_req_handle() is
more symmetric with the other puffs_req() routines.

Return 0 from puffs_mainloop() if no error occurs.
pointed out by Phil Nelson

Add puffs_setback(), which can be used to set setbacks for nodes before
the call returns to the kernel. Currently it can be called only from
open(), mmap(), remove() and rmdir(). (But that might change)

assert that the file server doesn't try to return more cookies than
it was supposed to

alloc to set flags when setting fhsize

some routines for setting mount parameters

Instead of keeping on adding parameters to puffs_mount(), make it
only take the bare essentials, which currently means removing
"maxreqlen" from the argument list (all current callers I'm aware
of set it as 0 anyway). Introduce puffs_init(), which provides a
context for setting various parameters and puffs_domount(), which
can be used to mount the file system. Keep puffs_mount() as a
shortcut for the above two for simple file systems.

Bump development ABI version to 13. After all, it's Friday the 13th.
Watch out! Bad things can happen on Friday the 13th. --No carrier--

Hide struct puffs_usermount from the rest of the world and provide
accessors for interesting data in it. Namely, you can now get
pu->pu_privdata with puffs_getspecific(), pu->pu_pn_root with
puffs_set/getroot() and pu->pu_maxreqlen with puffs_getmaxreqlen().

* support nodetofh and fhtonode callbacks
* use new signature of node_readdir

initial support for cacheops

Until someone can sort non-priviledged mounts and kauth, force
MNT_NOSUID | MNT_NODEV for mounts with geteuid() != 0. I'm tired
of typing them on the command line every time I test a file system.

dump return values for operations

Introduce puffs_path_walkcmp(), which is to be called from nodewalk
and compares the path of the node against the given pathobject.
Also make comparison method take a flag to indicate if it should
check if the second path is a true prefix of the first.

plus some namespace cleanup

* get rid of the holy kernel-given ISDOTDOT, and instead define a
macro which does strcmp against ".." and (the untranslated)
componentname
* make PUFFS_FLAG_BUILDPATH build paths also if dotdot is the case,
and adapt the regular path objects to this
* make nullfs lookup readable because we can now get rid of dotdot
processing there

add interfaces for suspending the file system

Be consistent with the rest of the framework about the argument
order of puffs_docc and puffs_dopreq

be consistent in naming request handlers

revoke revoke

Checkpoint some more work in progress: for the benefit those file
system backends which operate purely based on paths, push out more
path management into the library and make path management more
abstract: enable a file system to define a bunch of path management
callbacks, which are used by the framework. Management of normal
/this/is/a/path type paths is provided by the library.

For directory renames we need to rename all the child nodes for
the given directory if the file system wants paths (PUFFS_FLAG_BUILDPATH).
Do this by walking the nodelist and adjusting the path prefix of
each matching node.

* don't chdir in possible daemon() call for the benefit of file systems
which specify a relative path as the root
* if (buildpath), build link source full pathname for the duration of
the operation

* get rid of the mount callback; it's no great surprise to the
server that it needs to mount the file system backend if it wants
to call mount
* provide some options for getmntopts(), assume that callers will parse
command line (or fstab) args
* reorganize the puffs_cc interface just a bit, preparing for a bigger
revamp later

* inform kernel of our version
* homegrown lib ABI check (to avoid bumping major a ludicrillion times
before reaching stability)

checkpoint some experimental work-in-progress, namely:

Add support for having multiple outstanding operations. This is done
by exposing enough interfaces so that it is convenient to have the
main event loop in the implementation itself and by providing a
continuation framework for convinient blocking and rescheduling.

works fine, but will undergo further cleanup & development

CID4360: don't leak `buf' upon return
ok pooka@

outline & export code for interfacing with kernel GET/PUT requests

g/c out-of-date comment

support mmap

cleanup generated stuff from comments

rototill the operation callbacks: unify the ops under struct puffs_ops
and namespace them

adapt to kernel change:
Allow multiple requests to be transferred in each GET/PUTOP

PUFFSFLAG -> PUFFS_FLAG to be consistent with the kernel

build & send implemented operations list to kernel

summon daemon(3) in mainloop unless the nodaemon flag is given

* call statvfs() at mount() time, pass that info along
* start is gone

shuffle flags a bit

* catch up with kernel changes
* better error handling when mounting

* support FAF (Fire-And-Forget) class operations
* adapt to new lookup

pass rdev for specfs support

handle PUFFS_VN_INACTIVE

pass lint; fix install.

add libpuffs - the userspace programming interface for puffs

This work was initially started and completed for Google SoC 2005
and tweaked to work a bit better in the past few weeks. While
being far from complete, it is functional enough to be able and
stable to host a fairly general-purpose in-memory file system in
userspace. Even so, puffs should be considered experimental and
no binary compatibility for interfaces or crash-freedom or zero
security implications should be relied upon just yet.

The GSoC project was mentored by William Studenmund and the final
review for the code was done by Christos.

puffs(3): Replace realloc(x * y) with reallocarr

use a switch.

branches: 1.123.2; 1.123.4;
Typos.

fix error messages (use __func__, quotes)
also differentiate between canonical and non-relative.

Adjust to prototype.

fix error messages

Do not warn about relative path because of trailing slash

libpuffs calls realpath() to obtain an absolute path to use for mounting.
If the obtained path is different from the one given by the caller, a
warning is issued. This included the situation where the path passed by
the caller just have trailing slashes, a situation where we just want them
to be striped without a warning.

libpuffs support for fallocate and fdiscard operations

branches: 1.117.18;
fix crashes caused by using the results of getcontext()
after the caller returns.

branches: 1.116.4;
Call advlock method if supplied

Remove PUFFS_WITH_THREADS ifdef garbage, just use libc weak symbols.

unwhine lint

stat mount target path only if we're actually goint to use it.
This avoids errors from the comfd case when the mount target does
not exist in our local namespace.

set time_t compat where necessary

When using the top secret PUFFS_COMFD model, write out size of the
puffs kernel args before the contents. This allows parties which
have no clue about the contents to proxy the protocol easily.

Reenable pathconf now that the retval argument is register_t.

Actually, disable pathconf for a while longer: the retval needs a
little finetuning (not that everyone is jumping right to using it,
but fixing will have to wait until tomorrow, and who knows what
i'll forget during the night and how long it will eventually take).

signal pathconf in the vector of supported ops

* support extended attributes
* bump major due to structure growth
* add some spare space
* remove ABI sillyness

remove three years out-of-date comment

Add convenience routine puffs_unmountonsignal(), which does exactly that.

Use typecast instead of ULL for a less whiny lint on LP64.

Rewrite puffs_exit() in terms of the PUFFSOP_UNMOUNT kernel request.
Now server exit is a lot quite cleaner than previously.

Set high bit on server->kernel requests to better differentiate
between the direction of the request.

Add an error handler which syslogs but does not abort() and make it
the default. Most servers just shovel bits from their backend to
the kernel without checking the contents and dumping core in case
the backend gives garbage is a bit harsh.

Make it possible to supply parameters to puffs_init() later (but
before puffs_mount()).

Dispatch puffs_node_abortop().

Note: We use the storage of puffs_cache_write from puffs_ops for
this purpose. It's not issued by the kernel and hence currently
unused, and this saves us from the trouble of bumping the lib major
version.

fix some sign comparison warnings

Actually we need 'long int' to avoif a stupid 'truncating long to int'
warning.

Change type of 'n' to 'int' so that it can be printed with %d.
The domain of the value is [-1..sizeof (int)] so a type of 'int' is fine.

Close write-end of daemonpipe in reader to get EOF if the writer
goes awol.

Add puffs_cancel(), which allows to back down after puffs_init()
and before puffs_mount().
(this should actually be integrated with puffs_exit(), but that
routine is currently so non-sensible that I didn't bother now)

add:
puffs_setspecific(pu, private)
Can be used to set the specific data after the call to
puffs_init().

branches: 1.92.4; 1.92.8;
Replace void * by puffs_cookie_t where appropriate. No functional change.

clear cached puffs_cc's in puffs_exit()

* make sure we come out of mainloop in the main context. otherwise in
certain cases the server would jump to hyperspace when unmounted
* sprinkle some debug prints

branches: 1.89.6;
Implement lazy contexts for file systems using puffs_mainloop().
Previously each request was executed on its own callcontext and
switched to every time the request was being processed. Now requests
borrow the mainloop context and switch only if/when they yield.
This takes two context switches away from a file system request
bringing down the typical amounts 2->0 (e.g. dtfs) and 4->2 (e.g.
psshfs).

The interfaces for manually executing requests changed a bit:
puffs_dispatch_create() and puffs_dispatch_exec() must now be used.
They are not tested, as nothing in-tree wants them and I doubt
anyone else is really interested in them either.

Also do some misc code cleanup related to execution contexts. The
"work-in-progress checkpoint" committed over a year ago was starting
to look slightly weed-infested.

Cache a number of execution contexts instead of re-initializing
them every time. Speeds up pure in-memory file systems such as
sysctlfs or dtfs quite a bit. For actual I/O-workhorses the result
is of course less tasty.

Use PROT_NONE page to protect bottom of stack from overrun

Introduce PUFFS_STACKSIZE_MIN to avoid overloading 0 and getting
a warning from the library when really wanting to say "just the
minimum, please".

liberate gas-huffin crack-puffin' assert

fix pasto

Also pass fromname in the "distributed" case.

Provide some transparent "support" for "distributed" file systems.
If the env variable PUFFS_COMFD is set, the descriptor value
contained in it is used for communication instead of opening
/dev/puffs and doing mount(2).

This feature is obviously very undocumented and should not be used
without adult supervision.

Add a public interface for setting a cookie mapper.

Send a response message for flush operations from the kernel instead
of abusing the return value of write(2).

Part 3/n of extensive changes to message handling:

Get rid of the original puffs_req(3) framework and use puffs_framebuf(3)
instead for file system requests. It has the advantage of being
suitable for transporting a distributed message passing protocol
and therefore us being able to run the file system server on any
host.

Ok, puffs is not quite here yet: libpuffs needs to grow request
routing support and the message contents need to be munged into a
host independent format. Saying which format would be telling,
but it might begin with an X, end in an L and have the 13th character
in the middle. Keep an eye out for the sequels: Parts 3+m/n.

* allocate pcc as a slice from the stack instead of allocating it
separately
* provide puffs_cc_getcc()

This is in preparation for the removal of you-should-guess-what as
an argument to routines here and there and everywhere.

print more info when receiving an abort

puffs_mount():
* initialize values better
* always go through "fail" to see if puffs_daemon() needs to be
woken up
* open device r/w instead of r/o

Introduce puffs_daemon(), which works like daemon(3) except that it
also synchronizes with puffs_mount() and does not return (exit) in the
parent process until the file system has been mounted. This makes
it possible to reliably run e.g. mount_foo jippi /kai ; cd /kai/ee

Open the kernel descriptor as part of mount(), not init(). Then
it doesn't matter if someone fork()s or does other tricks between
init() and mount() (and besides, now it's where it logically should be).

Pull the daemonizing code out of the library mainloop into the file
servers. Calling daemon() (i.e. fork()ing) inside a library can
cause nice surprises for e.g. threaded programs. As discussed with
Greg Oster & others.

Sprinkle some hackish locks here and there, just enough to allow
libp2k & rump to run multithreaded.

Add possibility to set pre- and post callbacks which will be called
for all operations before or after executing the specific callback
(the api is not be final yet, though).

Add some very initial support for a threading worker model as an
alternative to the (vastly superior ;) continuation model. This
is very preliminary stuff and not compiled by default (which it
even won't do without some other patches I cannot commit yet).

The raison d'commit of the patch is a snippet which ensures proper
in-order dispatching of all operations, including those which don't
require a response. Previously many of them would be dispatched
simultaneosly, e.g. fsync and reclaim on the same node, which
obviously isn't all that nice for correct operation.

mmap stacks, makes it possible to align them

Make pu_haskq a flag instead of its own variable.

Add puffs_cc_schedule() which marks a pcc runnable and will pass
execution to it when in the main loop the next time.

Part 1/n of some pretty extensive changes to how the kernel module
interacts with the userspace file server:

* since the kernel-user communication is not purely request-response
anymore (hasn't been since 2006), try to rename some "request" to
"message". more similar mangling will take place in the future.

* completely rework how messages are allocated. previously most of
them were borrowed from the stack (originally *all* of them),
but now always allocate dynamically. this makes the structure
of the code much cleaner. also makes it possible to fix a
locking order violation. it enables plenty of future enhancements.

* start generalizing the transport interface to be independent of puffs

* move transport interface to read/write instead of ioctl. the
old one had legacy design problems, and besides, ioctl's suck.
implement a very generic version for now; this will be
worked on later hopefully some day reaching "highly optimized".

* implement libpuffs support behind existing library request
interfaces. this will change eventually (I hate those interfaces)

Pass reason string to pu_errnotify. Print it in the default
version before aborting.