kernel: Add event queue implementation to wait for objects efficiently.

Based on hamishm's original patch from 2015, but heavily modified,
refactored, and reworked.

From the original commit message:

> When an object is deleted, a B_EVENT_INVALID event is delivered,
> and the object is unregistered from the queue.
>
> The special event flag B_EVENT_ONE_SHOT can be passed in when adding
> an object so that the object is automatically unregistered when an
> event is delivered.

Modifications to the original change include:

* Removed the public interface (syscalls remain private for the moment)

* Event list queueing/dequeueing almost entirely rewritten, including:
- Clear events field when dequeueing.

- Have B_EVENT_QUEUED actually indicate whether the event has been
appended to the linked list (or not), based around lock state.
The previous logic was prone to races and double-insertions.

- "Modify" is now just "Deselect + Select" performed at once;
previously it could cause use-after-frees.

- Unlock for deselect only once at the end of dequeue.

- Handle INVALID events still in the queue upon destruction,
fixing memory leaks.

* Deduplified code with wait_for_objects.

* Use of C++ virtual dispatch instead of C-style enum + function calls,
and BReferenceable plus destructors for teardown.

* Removed select/modify/delete flags. Select/Modify are now the same
operation on the syscall interface, and "Delete" is done when 0
is passed for "events". Additionally, the events selected can be fetched
by passing -1 for "events".

* Implemented level-triggered mode.

* Use of BStackOrHeapArray and other convenience routines in syscalls.

Change-Id: I1d2f094fd981c95215a59adbc087523c7bbbe40b
Reviewed-on: https://review.haiku-os.org/c/haiku/+/6745
Tested-by: Commit checker robot <no-reply+buildbot@haiku-os.org>
Reviewed-by: Jérôme Duval <jerome.duval@gmail.com>

vfs: Bind flock locks to file descriptors

* File locks created by flock should only apply for the file descriptor
that was used to lock the file. Another fd on the same file should then
be denied access (calling flock should fail).
* fcntl based locks, however, are in a separate namespace and are global
to a team.
* This issue was found when running webkitpy test suite, and should close
ticket #13795.
* Don't use session or team as comparison in release_advisory_lock(), as
that information might not be available anymore (e.g. when called from
Team::~Team()). This fixes #14121.

Change-Id: I9efb96cfcefe7e72b0060220c635a665e7e643cc
Co-authored-by: Axel Dörfler <axeld@pinc-software.de>

Added dup_foreign_fd() to duplicate a FD from another team.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29468 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added get_open_fd() function which gets the descriptor and also
increments its open count.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25317 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added close_fd_index() which is pretty much like close(), but gets the
IO context in which to close the FD.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24937 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added fd_ops::set_flags() hook. It's only implemented by socket FDs
ATM.
* Changed common_fcntl() to also work for non-vnode-based FDs. Only the
operations that require a vnode do fail now. F_SETFL uses the new
fd_ops::set_flags() if available, falling back to the vnode respective
operation.
* Changed some direct accesses to file_descriptor::u::vnode. It was safe
in those cases, but might not be anymore if we ever change fd_vnode().

ftp can at least connect to servers again, now, but apparently we still
have a problem with syscall restarts -- downloads fail after one second,
when the timer goes off.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24917 a95241bf-73f2-0310-859d-f6bbb57e9c96

Preparation for socket file descriptors.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24905 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Implemented support for chroot:
- Added a "root" vnode to the io_context. It is used for resolving
paths and converting nodes to paths instead of sRoot. Some more
passing around of io_context structures was necessary.
- Introduced a new lock sIOContextRootLock to protect
io_context::root. The current uses of io_context::io_mutex
(put_vnode(), remove_vnode() while holding it) looked too suspicious
to use that mutex in vnode_path_to_vnode().
- Added _kern_change_root() syscall and chroot() libroot function.
- Added chroot coreutils program to the image. Funnily it seems to be
much easier to set up a little jail than under Linux (just copy
bash and libroot.so into respective subdirs; mount another pipefs
if you want pipe support).
With Haiku allowing direct access to directories via inode IDs
jailing is obviously not very secure at the moment.
- Added /var/empty to the image. It will be the chroot target for ssh.
* Changed vfs.cpp:get_cwd() so that the io_context::io_mutex is no
longer held when calling dir_vnode_to_path().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24673 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Renamed fs/vfs_select.cpp to wait_for_objects.cpp and got rid of
vfs_select.h, respectively moved most of it into the new kernel
private header wait_for_objects.h.
* Added new experimental API functions wait_for_objects[_etc](). They
work pretty much like poll(), but also for semaphores, ports, and
threads.
* Removed the "ref" parameter from notify_select_events() and the
select_sync_pool functions as well as from fd_ops::fd_[de]select(). It
is no longer needed. The FS interface select() hook still has it,
though -- the VFS will always pass 0.
* de]select_fd() take a select_info* instead of a select_sync* + ref
pair, now. Added respective functions for semaphores, ports, and
threads.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22416 a95241bf-73f2-0310-859d-f6bbb57e9c96

* fd.c -> fd.cpp
* Reworked the select support:
- The io_context additionally stores a table of lists of select_infos,
which enables it to deselect events of a pending select() when
closing a FD. This prevents a race condition potentially causing a
write to stale memory.
- The opaque selectsync* passed to FSs is now actually a select_info*.
This was necessary, since the FDs deselect() hook (unlike the
select() hook) doesn't take a "ref" argument and deselecting a
single info (e.g. caused by a premature close()) was not possible.
The select() hook's "ref" argument has become superfluous.
- It should now be relatively easy to implement a poll_on_steroids()
that can also wait for objects other than FDs (e.g. semaphores,
ports, threads etc.).
* Set/reset the signal mask in common_select(). This makes pselect()
work as required.
* Reorganized vfs_resize_fd_table().



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22391 a95241bf-73f2-0310-859d-f6bbb57e9c96

fixed the way accept() works in regards to the cookie pointer. It is no longer visible to userspace, we pass the fd instead. Also renamed kernel's shutdown() to system_shutdown as it collides with Posix's shutdown().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21223 a95241bf-73f2-0310-859d-f6bbb57e9c96

Implemented force unmounting - by passing the "-f" option to "unmount" you
can now safely unmount volumes that are still in use by some applications.
Minor fixes to the FD disconnection implementation:
* put_fd() checked the condition for being able to disconnect a descriptor
incorrectly (causing the FD to never be disconnected).
* remove_fd() would hand out disconnected descriptors (but should have
returned NULL for them).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15952 a95241bf-73f2-0310-859d-f6bbb57e9c96

Implemented disconnecting file descriptors: this will be used by the
force unmounting code.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15949 a95241bf-73f2-0310-859d-f6bbb57e9c96

* vfs_init() is now called later in the startup sequence - since no core kernel
service depends on it, it doesn't make any sense to call it that early in the
game.
* The VFS now has a low memory handler for vnodes as well. If there is enough
memory left, it won't free any vnodes anymore.
* Potential crashing bug fix: some functions did not check if the FD passed
in belonged to the right type; they just assumed it had a valid vnode, but
it could have had a mount structure associated as well.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15566 a95241bf-73f2-0310-859d-f6bbb57e9c96

Our close-on-exec flag handling was broken: it's supposed to be specified per slot, not
per file descriptor (a descriptor can be shared among several slots). There is now a
second table in the io_context structure that contains that information in a bitmap.
There are now two new (private) functions to control the close-on-exec flag, fd_close_on_exec(),
and fd_set_close_on_exec().
F_DUPFD, dup(), and dup2() are supposed to clear the close-on-exec flag on the duplicated
slot - this fixes bug #57 (no output after a redirect of a shell builtin).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14313 a95241bf-73f2-0310-859d-f6bbb57e9c96

Minor header cleanup: moved some headers to better matching directories,
removed unused headers. Adapted sources to still compile with the new
header locations.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@11913 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added dup_foreign_fd() to duplicate a FD from another team.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29468 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added get_open_fd() function which gets the descriptor and also
increments its open count.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25317 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added close_fd_index() which is pretty much like close(), but gets the
IO context in which to close the FD.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24937 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added fd_ops::set_flags() hook. It's only implemented by socket FDs
ATM.
* Changed common_fcntl() to also work for non-vnode-based FDs. Only the
operations that require a vnode do fail now. F_SETFL uses the new
fd_ops::set_flags() if available, falling back to the vnode respective
operation.
* Changed some direct accesses to file_descriptor::u::vnode. It was safe
in those cases, but might not be anymore if we ever change fd_vnode().

ftp can at least connect to servers again, now, but apparently we still
have a problem with syscall restarts -- downloads fail after one second,
when the timer goes off.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24917 a95241bf-73f2-0310-859d-f6bbb57e9c96

Preparation for socket file descriptors.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24905 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Implemented support for chroot:
- Added a "root" vnode to the io_context. It is used for resolving
paths and converting nodes to paths instead of sRoot. Some more
passing around of io_context structures was necessary.
- Introduced a new lock sIOContextRootLock to protect
io_context::root. The current uses of io_context::io_mutex
(put_vnode(), remove_vnode() while holding it) looked too suspicious
to use that mutex in vnode_path_to_vnode().
- Added _kern_change_root() syscall and chroot() libroot function.
- Added chroot coreutils program to the image. Funnily it seems to be
much easier to set up a little jail than under Linux (just copy
bash and libroot.so into respective subdirs; mount another pipefs
if you want pipe support).
With Haiku allowing direct access to directories via inode IDs
jailing is obviously not very secure at the moment.
- Added /var/empty to the image. It will be the chroot target for ssh.
* Changed vfs.cpp:get_cwd() so that the io_context::io_mutex is no
longer held when calling dir_vnode_to_path().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24673 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Renamed fs/vfs_select.cpp to wait_for_objects.cpp and got rid of
vfs_select.h, respectively moved most of it into the new kernel
private header wait_for_objects.h.
* Added new experimental API functions wait_for_objects[_etc](). They
work pretty much like poll(), but also for semaphores, ports, and
threads.
* Removed the "ref" parameter from notify_select_events() and the
select_sync_pool functions as well as from fd_ops::fd_[de]select(). It
is no longer needed. The FS interface select() hook still has it,
though -- the VFS will always pass 0.
* de]select_fd() take a select_info* instead of a select_sync* + ref
pair, now. Added respective functions for semaphores, ports, and
threads.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22416 a95241bf-73f2-0310-859d-f6bbb57e9c96

* fd.c -> fd.cpp
* Reworked the select support:
- The io_context additionally stores a table of lists of select_infos,
which enables it to deselect events of a pending select() when
closing a FD. This prevents a race condition potentially causing a
write to stale memory.
- The opaque selectsync* passed to FSs is now actually a select_info*.
This was necessary, since the FDs deselect() hook (unlike the
select() hook) doesn't take a "ref" argument and deselecting a
single info (e.g. caused by a premature close()) was not possible.
The select() hook's "ref" argument has become superfluous.
- It should now be relatively easy to implement a poll_on_steroids()
that can also wait for objects other than FDs (e.g. semaphores,
ports, threads etc.).
* Set/reset the signal mask in common_select(). This makes pselect()
work as required.
* Reorganized vfs_resize_fd_table().



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22391 a95241bf-73f2-0310-859d-f6bbb57e9c96

fixed the way accept() works in regards to the cookie pointer. It is no longer visible to userspace, we pass the fd instead. Also renamed kernel's shutdown() to system_shutdown as it collides with Posix's shutdown().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21223 a95241bf-73f2-0310-859d-f6bbb57e9c96

Implemented force unmounting - by passing the "-f" option to "unmount" you
can now safely unmount volumes that are still in use by some applications.
Minor fixes to the FD disconnection implementation:
* put_fd() checked the condition for being able to disconnect a descriptor
incorrectly (causing the FD to never be disconnected).
* remove_fd() would hand out disconnected descriptors (but should have
returned NULL for them).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15952 a95241bf-73f2-0310-859d-f6bbb57e9c96

Implemented disconnecting file descriptors: this will be used by the
force unmounting code.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15949 a95241bf-73f2-0310-859d-f6bbb57e9c96

* vfs_init() is now called later in the startup sequence - since no core kernel
service depends on it, it doesn't make any sense to call it that early in the
game.
* The VFS now has a low memory handler for vnodes as well. If there is enough
memory left, it won't free any vnodes anymore.
* Potential crashing bug fix: some functions did not check if the FD passed
in belonged to the right type; they just assumed it had a valid vnode, but
it could have had a mount structure associated as well.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15566 a95241bf-73f2-0310-859d-f6bbb57e9c96

Our close-on-exec flag handling was broken: it's supposed to be specified per slot, not
per file descriptor (a descriptor can be shared among several slots). There is now a
second table in the io_context structure that contains that information in a bitmap.
There are now two new (private) functions to control the close-on-exec flag, fd_close_on_exec(),
and fd_set_close_on_exec().
F_DUPFD, dup(), and dup2() are supposed to clear the close-on-exec flag on the duplicated
slot - this fixes bug #57 (no output after a redirect of a shell builtin).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14313 a95241bf-73f2-0310-859d-f6bbb57e9c96

Minor header cleanup: moved some headers to better matching directories,
removed unused headers. Adapted sources to still compile with the new
header locations.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@11913 a95241bf-73f2-0310-859d-f6bbb57e9c96