kernel/team: Fix race condition in team loading wait.

The condition variable that the load_image'ing thread is waiting on
is also owned by that same thread, so as soon as it wakes up, it
will soon return, thus destroying it. Under high load or other unlucky
scheduling conditions, it seems this could occur before the other thread
had even returned from the condition variable's NotifyAll.

Since team->loading_info is protected by the team lock, simply
acquire the team lock once more after being awoken and returning,
to synchronize and prevent this race.

Should fix #18352.

kernel: Break thread-related AutoLockers into a separate header.

Including thread.h brings a massive array of things with it from
the kernel thread arch headers, team and thread definitions,
hash tables, linked lists, Referenceable, etc. that the vast majority
of AutoLock.h consumers neither want nor need.

So, put these in a separate header, and adjust all consumers of these
lockers to include the new file.

This change exposes the fact that a lot of files were inadvertently
making use of headers included indirectly through thread.h. Those
will be fixed in the next commit.

kernel: Fix race condition when waiting for load of new team.

There was no synchronization of the check of the done flag and the
waiting thread suspending to wait for it. It was therefore possible that
the new team both set the flag and triggered the wakeup of the waiting
thread in that time window, causing it to miss both the set flag and the
thread resumption.

Use a condition variable instead.

Fixes #13081.

Change-Id: I93c45db8dd773fe42b45c4b67153bcd39e200d3b
Reviewed-on: https://review.haiku-os.org/803
Reviewed-by: waddlesplash <waddlesplash@gmail.com>

Revert the rest of the COMPAT_MODE changes (back to hrev52003.)

This reverts commit 458e758f3792ef11ca26d6ff7e24600c88326e83.
This reverts commit ce5eb94a82b1b377ef5909e65411e031f54ceb15.
This reverts commit aac8d4c317ca11a9a6e194e2c668e8183ec23dd6.
This reverts commit c70cba914aa79c01bbc2da38085936f589899c8c.
This reverts commit 2ffbe7aaca8668c5a68ac7488459bace7a0700f2
This reverts commit c6e120e2d2f909d95f95839fa99fccf811fdb3c5.

kernel/x86_64: compatibility syscalls for image.cpp.

* define compat_image_info, compat_extended_image_info
to be used for respective 32-bit types of syscalls in compatibility mode.
* handle 32-bit types in _user_register_image, _user_get_image_info,
_user_get_next_image_info, other syscalls are compatible as is.

Change-Id: Ibbd33e6796208dfa70d869e36bf745bc3e18d330

kernel: image: _get_next_image_info shouldn't use a user buffer directly

_user_get_next_image_info() now copies the cookie on the stack, calls
_get_next_image_info() and copy back the cookie to the user buffer.

runtime loader: Register extended image info with kernel

Add structure extended_image_info which extends image_info by the
fields symbol_table, symbol_hash, string_table.

kernel: +image_iterate_through_team_images()

Like image_iterate_through_images(), but iterates though the images of
the given team only.

kernel: Use anonymous namespaces to avoid type collisions.

The anonymous namespace makes type definitions local to the translation
unit (like static does for objects). For pretty much any type not shared
across multiple files this is what one wants to happen (and might
erroneously expect to happen automatically).

This commit solves some actual collisions that were present:

* The VFS and the rootfs both used an incompatible VnodeHash struct for
their BOpenHashTable.
* XSI semaphores and message queues both used queued_thread, Ipc and
IpcHashTableDefinition.

For release builds these did not cause problems as the types were fully
inlined. Debug builds would crash at boot however because parts of a
BOpenHashTable<VnodeHash> from the rootfs meant to operate on struct
rootfs_vnode would be applied to one from the VFS expecting struct
vnode.

As such collisions are violations of the one definition rule, the code
is at fault and unfortunatley the compiler isn't required to diagnose
such problems across translation units (which isn't actually trivial).
This can lead to subtle and hard to debug problems and it's therefore
best to avoid leaking types into the global namespace whenever possible.

kernel: Properly synchronize suspending new thread

After load_image() the child thread is suspended and the parent is
expected to resume it later. However, it is possible that the parent
attempts to resume its child after it has been notified that the image
had been loaded but before the child managed to suspend itself. In such
case the child would suspends itself after that wake up attempt and,
consequently will not be ever resumed.

To mitigate that problem flag Thread::going_to_suspend has been added
which helps synchronizing thread suspension and continuation in a similar
way that "traditional" thread blocking is performed. This means that
the child should behave in a following manner: set its going_to_suspend flag,
notify the parent (i.e. any thread that may want to resume it), acquire
its scheduler_lock and suspend itself if the going_to_suspend flag is set.
The parent should follow pattern: clear going_to_suspend flag of the thread
that is about to be resumed, acquire that thread scheduler_lock and enqueue
it in a run queue if it is suspended.

Thanks Oliver for reporting the bug and identifying what causes it.

kernel: Remove Thread::next_state

kernel: Remove gSchedulerLock

* Thread::scheduler_lock protects thread state, priority, etc.
* sThreadCreationLock protects thread creation and removal and list of
threads in team.
* Team::signal_lock and Team::time_lock protect list of threads in team
as well.
* Scheduler uses its own internal locking.

Rework DefaultNotificationService registration.

- Instead of implicitly registering and unregistering a service
instance on construction/destruction, DefaultNotificationService
now exports explicit Register()/Unregister() calls, which subclasses
are expected to call when they're ready.

- Adjust all implementing subclasses. Resolves an issue with deadlocks
when booting a DEBUG=1 build.

Fixed formatting of output in some debugger commands.

Currently all debugger commands assume 32-bit pointers when formatting their
output. This means that on x86_64 the output is incorrectly formatted. Fixed
this by adding a B_PRINTF_POINTER_WIDTH definition (16 on 64-bit, 8 on
32-bit), and using this to correctly format the output. Not all commands have
been fixed yet, but all VM, slab, VFS, team, thread and image commands should
be correct.

First round of 64-bit safety fixes in the kernel.

* Most of this is incorrect printf format strings. Changed all strings
causing errors to use the B_PRI* format string definitions, which
means the strings should be correct across all platforms.
* Some other fixes for errors, casts required, etc.

Merged signals-merge branch into trunk with the following changes:
* Reorganized the kernel locking related to threads and teams.
* We now discriminate correctly between process and thread signals. Signal
handlers have been moved to teams. Fixes #5679.
* Implemented real-time signal support, including signal queuing, SA_SIGINFO
support, sigqueue(), sigwaitinfo(), sigtimedwait(), waitid(), and the addition
of the real-time signal range. Closes #1935 and #2695.
* Gave SIGBUS a separate signal number. Fixes #6704.
* Implemented <time.h> clock and timer support, and fixed/completed alarm() and
[set]itimer(). Closes #5682.
* Implemented support for thread cancellation. Closes #5686.
* Moved send_signal() from <signal.h> to <OS.h>. Fixes #7554.
* Lots over smaller more or less related changes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@42116 a95241bf-73f2-0310-859d-f6bbb57e9c96

Merged the signals branch into trunk, with these changes:
* The team and thread kernel structures have been renamed to Team and Thread
respectively and moved into the new BKernel namespace.
* Several (kernel add-on) sources have been converted from C to C++ since
private kernel headers are included that are no longer C compatible.

Changes after merging:
* Fixed gcc 2 build (warnings mainly in the scary firewire bus manager).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40196 a95241bf-73f2-0310-859d-f6bbb57e9c96

headers/private/kernel/util/OpenHashTable.h, Hugo's version, is a bit nicer than
Tracker's OpenHashTable.h which it should eventually replace. We've renamed the
class to BOpenHashTable and changed the interface slightly so that HashTableLink
became superfluous.
Adapted all the code that used it. Since the OpenHashTables no longer clash,
this should fix the GCC4 build.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31791 a95241bf-73f2-0310-859d-f6bbb57e9c96

Extended image_info by fields for the Haiku version and ABI. The runtime loader
and the kernel read those values from the shared object (if available). In the
runtime loader this should eventually replace the gcc version guessing method
currently used (at least for shared objects built for Haiku). The optional
packages need to be rebuilt first, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30729 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Cleared up a misconception in the notification mechanism. We already had
methods that used an "event mask" field. There was no need to introduce
a "flags" field for the same purpose.
* Renamed protected DefaultNotificationService methods (removed "_" prefix).
* Adjusted the code providing a notification service accordingly.
* Changed the event message several notification services generated by renaming
the "opcode" field to "event".
* Implemented the TEAM_ADDED event and also added a TEAM_EXEC event.
* Added notifications for threads and images.
* Added visitor-like iteration functions for teams, threads, and images.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30126 a95241bf-73f2-0310-859d-f6bbb57e9c96

Insert the application image at the head of the list. Work-around for
code (like found in Firefox) that relies on get_next_image_info() to
return it first.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27732 a95241bf-73f2-0310-859d-f6bbb57e9c96

* image.c -> image.cpp
* Added global hash table for images.
* Improved a bit of code by using the new image hash table. E.g.
_get_image_info() can return infos for images of any team, now.
* Fixed remove_images() comment: The function must not be invoked with
the team lock being held.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27606 a95241bf-73f2-0310-859d-f6bbb57e9c96

kernel: Properly synchronize suspending new thread

After load_image() the child thread is suspended and the parent is
expected to resume it later. However, it is possible that the parent
attempts to resume its child after it has been notified that the image
had been loaded but before the child managed to suspend itself. In such
case the child would suspends itself after that wake up attempt and,
consequently will not be ever resumed.

To mitigate that problem flag Thread::going_to_suspend has been added
which helps synchronizing thread suspension and continuation in a similar
way that "traditional" thread blocking is performed. This means that
the child should behave in a following manner: set its going_to_suspend flag,
notify the parent (i.e. any thread that may want to resume it), acquire
its scheduler_lock and suspend itself if the going_to_suspend flag is set.
The parent should follow pattern: clear going_to_suspend flag of the thread
that is about to be resumed, acquire that thread scheduler_lock and enqueue
it in a run queue if it is suspended.

Thanks Oliver for reporting the bug and identifying what causes it.

kernel: Remove Thread::next_state

kernel: Remove gSchedulerLock

* Thread::scheduler_lock protects thread state, priority, etc.
* sThreadCreationLock protects thread creation and removal and list of
threads in team.
* Team::signal_lock and Team::time_lock protect list of threads in team
as well.
* Scheduler uses its own internal locking.

Rework DefaultNotificationService registration.

- Instead of implicitly registering and unregistering a service
instance on construction/destruction, DefaultNotificationService
now exports explicit Register()/Unregister() calls, which subclasses
are expected to call when they're ready.

- Adjust all implementing subclasses. Resolves an issue with deadlocks
when booting a DEBUG=1 build.

Fixed formatting of output in some debugger commands.

Currently all debugger commands assume 32-bit pointers when formatting their
output. This means that on x86_64 the output is incorrectly formatted. Fixed
this by adding a B_PRINTF_POINTER_WIDTH definition (16 on 64-bit, 8 on
32-bit), and using this to correctly format the output. Not all commands have
been fixed yet, but all VM, slab, VFS, team, thread and image commands should
be correct.

First round of 64-bit safety fixes in the kernel.

* Most of this is incorrect printf format strings. Changed all strings
causing errors to use the B_PRI* format string definitions, which
means the strings should be correct across all platforms.
* Some other fixes for errors, casts required, etc.

Merged signals-merge branch into trunk with the following changes:
* Reorganized the kernel locking related to threads and teams.
* We now discriminate correctly between process and thread signals. Signal
handlers have been moved to teams. Fixes #5679.
* Implemented real-time signal support, including signal queuing, SA_SIGINFO
support, sigqueue(), sigwaitinfo(), sigtimedwait(), waitid(), and the addition
of the real-time signal range. Closes #1935 and #2695.
* Gave SIGBUS a separate signal number. Fixes #6704.
* Implemented <time.h> clock and timer support, and fixed/completed alarm() and
[set]itimer(). Closes #5682.
* Implemented support for thread cancellation. Closes #5686.
* Moved send_signal() from <signal.h> to <OS.h>. Fixes #7554.
* Lots over smaller more or less related changes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@42116 a95241bf-73f2-0310-859d-f6bbb57e9c96

Merged the signals branch into trunk, with these changes:
* The team and thread kernel structures have been renamed to Team and Thread
respectively and moved into the new BKernel namespace.
* Several (kernel add-on) sources have been converted from C to C++ since
private kernel headers are included that are no longer C compatible.

Changes after merging:
* Fixed gcc 2 build (warnings mainly in the scary firewire bus manager).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40196 a95241bf-73f2-0310-859d-f6bbb57e9c96

headers/private/kernel/util/OpenHashTable.h, Hugo's version, is a bit nicer than
Tracker's OpenHashTable.h which it should eventually replace. We've renamed the
class to BOpenHashTable and changed the interface slightly so that HashTableLink
became superfluous.
Adapted all the code that used it. Since the OpenHashTables no longer clash,
this should fix the GCC4 build.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31791 a95241bf-73f2-0310-859d-f6bbb57e9c96

Extended image_info by fields for the Haiku version and ABI. The runtime loader
and the kernel read those values from the shared object (if available). In the
runtime loader this should eventually replace the gcc version guessing method
currently used (at least for shared objects built for Haiku). The optional
packages need to be rebuilt first, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30729 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Cleared up a misconception in the notification mechanism. We already had
methods that used an "event mask" field. There was no need to introduce
a "flags" field for the same purpose.
* Renamed protected DefaultNotificationService methods (removed "_" prefix).
* Adjusted the code providing a notification service accordingly.
* Changed the event message several notification services generated by renaming
the "opcode" field to "event".
* Implemented the TEAM_ADDED event and also added a TEAM_EXEC event.
* Added notifications for threads and images.
* Added visitor-like iteration functions for teams, threads, and images.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30126 a95241bf-73f2-0310-859d-f6bbb57e9c96

Insert the application image at the head of the list. Work-around for
code (like found in Firefox) that relies on get_next_image_info() to
return it first.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27732 a95241bf-73f2-0310-859d-f6bbb57e9c96

* image.c -> image.cpp
* Added global hash table for images.
* Improved a bit of code by using the new image hash table. E.g.
_get_image_info() can return infos for images of any team, now.
* Fixed remove_images() comment: The function must not be invoked with
the team lock being held.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27606 a95241bf-73f2-0310-859d-f6bbb57e9c96