libroot: set __main_thread_id in libroot initialize_before instead of entry point

DT_INIT, DT_INIT_ARRAY, initialize_before, initialize_after are called before
executable entry point so __main_thread_id may be uninitalized there. It cause
getpid() to return zero if called in one of image initalizers before main().

Fixes #11797.

Change-Id: Ia745db9ffeaeb6de4dd46b868b1575d0780b54d9
Reviewed-on: https://review.haiku-os.org/c/haiku/+/5254
Reviewed-by: Adrien Destugues <pulkomandy@pulkomandy.tk>
Tested-by: Commit checker robot <no-reply+buildbot@haiku-os.org>

Implement stack protection support

this adds kernel & libroot stack protector hooks. it uses /dev/random in userspace.
A configure option --enable-stack-protector is added to activate -fstack-protector
on selected system components (ATM apps, kits, servers).

Change-Id: If3a2920ba9aa0a85eaff4ba6778947f8c76ade31
Reviewed-on: https://review.haiku-os.org/c/haiku/+/3895
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>

libroot: make API version available.

Change-Id: I370d5c8e9b4f076e7f115ca6c87bc7f943d36c17
Reviewed-on: https://review.haiku-os.org/c/haiku/+/1850
Reviewed-by: Rene Gollent <rene@gollent.com>

libroot: Revert changes to hide BeOS-only functions.

I wasn't thinking correctly when I made the previous change --
init_term_dyn is compiled into all applications, and so these
functions need to remain in order to not break them.

libroot: Move some BeOS-only API calls behind _BEOS_R5_COMPATIBLE_.

This is defined by our build system on GCC2 only, so they will now
be excluded under all other architectures.

headers/kernel: Include declarations of initialization functions in image.h.

No "real" functional change, but this causes GCC7 to throw errors when
these functions are declared without the image_id argument, which
in some files they were (as this commit repairs.)

This change is largely inconsequential on x86, but on callee-cleanup-args
targets, leaving out the argument would probably cause stack corruption.

libroot_debug: Merge guarded heap into libroot_debug.

The individual debug heap implementations are now exposed via a
structure of function pointers and a common frontend dispatches the
malloc and malloc_debug APIs through them.

The MALLOC_DEBUG environment variable can be used to select the guarded
heap by adding 'g', otherwise the debug heap is used as the default.
Consequently the separate libroot_guarded is not needed anymore and has
been removed.

To allow the use of environment variables this early, init_env_post_heap
has been added and the heap dependent atfork() moved there. This allowed
to fold the code of init_heap_post_env into init_heap so the former has
been removed.

malloc_debug: Implement allocation dump on exit in guarded heap.

When enabled (using heap_debug_dump_allocations_on_exit(true) or
MALLOC_DEBUG=e) this causes a dump of all remaining allocations when
libroot_debug is unloaded. It uses terminate_after to be called as
late as possible.

When combined with alloc stack traces this makes for a nice if a bit
crude leak checker. Note that a lot of allocations usually remain
even at that stage due to statically, lazyly and globally allocated
stuff from the various system libraries where it isn't necessarily
worth the overhead to free them when the program terminates anyway.

libroot: Add adaptive mutex implementation

libroot: explicitly check ABI version

As Axel suggested use simple, explicit checks for legacy ABI version
instead of obscure "compatibility mode".

libroot: make all areas executable for old binaries

* If at least one image is either B_HAIKU_ABI_GCC_2_ANCIENT or
B_HAIKU_ABI_GCC_2_BEOS almost all areas are marked as executable.
* B_EXECUTE_AREA and B_STACK_AREA are made public. The former is enforced since
the introduction of DEP and apps need it to correctly set area protection.
The latter is currently needed only to recognize stack areas and fix their
protection in compatibility mode, but may also be useful if an app wants
to use sigaltstack from POSIX API.

commpage: randomize position of commpage

This patch introduces randomization of commpage position. From now on commpage
table contains offsets from begining to of the commpage to the particular
commpage entry. Similary addresses of symbols in ELF memory image "commpage"
are just offsets from the begining of the commpage.

This patch also updates KDL so that commpage entries are recognized and shown
correctly in stack trace. An update of Debugger is yet to be done.

Merged signals-merge branch into trunk with the following changes:
* Reorganized the kernel locking related to threads and teams.
* We now discriminate correctly between process and thread signals. Signal
handlers have been moved to teams. Fixes #5679.
* Implemented real-time signal support, including signal queuing, SA_SIGINFO
support, sigqueue(), sigwaitinfo(), sigtimedwait(), waitid(), and the addition
of the real-time signal range. Closes #1935 and #2695.
* Gave SIGBUS a separate signal number. Fixes #6704.
* Implemented <time.h> clock and timer support, and fixed/completed alarm() and
[set]itimer(). Closes #5682.
* Implemented support for thread cancellation. Closes #5686.
* Moved send_signal() from <signal.h> to <OS.h>. Fixes #7554.
* Lots over smaller more or less related changes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@42116 a95241bf-73f2-0310-859d-f6bbb57e9c96

now correctly init the thread id in the pthread struct for main thread and threads spawned with spawn_thread(). This definitely helps for #7235.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40665 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Implemented a way to specify additional debug options via MALLOC_DEBUG when
using libroot_debug.so, instead of having to hardcode them in the application
via calling private heap functions.
* The following options are implemented: 'p' turns on paranoid validation,
'w' triggers periodic wall checking every 500ms ('W' does the same, but every
100ms), 'g' to use guard pages (beware, this will dramatically increase
memory usage), and 'r' which forbids reusing of memory, freed memory is never
actually freed.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40514 a95241bf-73f2-0310-859d-f6bbb57e9c96

Closing #7056:
* inherit umask of calling process to images loaded via exec...()

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40071 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Private libroot locking primitives:
- Reimplemented mutex to use the _kern_mutex*() syscalls.
- The initializer functions cannot fail anymore -- changed their return type
to void.
- Changed the initializer function semantics to not copy the name by default
anymore (as in the kernel). Also added *_etc() versions of them that take an
additional flags.
- Added static initializer macros.
- Made the mutex (and thus recursive_lock) lock functions non-interruptable.
- Got rid of the "lazy" version. They are no longer needed, since the
initialization of the standard types can be done statically and cannot fail.
* Adjusted libroot, runtime loader, and other code using the private libroot
locking primitives to the new semantics.
* pthreads mutexes and condition variables:
- Reimplemented using the _kern_mutex*() syscalls.
- Consistently use POSIX error codes.
- Fixed some not quite POSIX compliant behavior.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36323 a95241bf-73f2-0310-859d-f6bbb57e9c96

* This fixes the remaining warnings in exit.c


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34294 a95241bf-73f2-0310-859d-f6bbb57e9c96

Moved the rld_export structure from <user_runtime.h> to
<runtime_loader.h>, since it isn't a kernel <-> userland interface.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28456 a95241bf-73f2-0310-859d-f6bbb57e9c96

Works much better when actually allocating the TLS entry...
Now the backend works, Login can list users.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23509 a95241bf-73f2-0310-859d-f6bbb57e9c96

bonefish+mmlr:
As (our) gcc unfortunately uses atexit() to clean up lazily initialized static
variables inside functions we have to ensure that we do the right thing with
unloadable shared objects. In case a shared object was unloaded that installed
an atexit() hook the application would crash on exit. We now implement a
callback into libroot that is used to call all the atexit() hooks of a
component that is to be unloaded. Most prominently this fixes the media_server
crash at shutdown.

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23486 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added a mechanism to retrieve a BMessage with eventual error descriptions
for _kern_load_image().
* Added KMessage to the runtime_loader (a bit hacky, though) - it will use
it to deliver the above mentioned functionality.
* load_dependencies() did return the wrong status code in case a library
was missing; now it returns B_MISSING_LIBRARY.
* load_dependencies() will now try to load all dependencies when a report
message is requested; therefore, all missing libraries are listed.
* Renamed uspace_program_args to user_space_program_args.
* The kernel filled in various members of the user_space_program_args structure
unsafely, ie. was not using user_memcpy().
* Renamed some local variables in team.c to better fit our style guide (ie.
uargs to userArgs).
* Changed Tracker to use the new _kern_load_image() variant on Haiku to retrieve
and report all missing libraries. This fixes bug #1324.
* Adapted kernel_cpp.cpp to the runtime loader as well; the latter will now
compile with _LOADER_MODE defined.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21715 a95241bf-73f2-0310-859d-f6bbb57e9c96

Made our glue code compatible to BeOS again. IOW executables compiled for Haiku will
now run under BeOS as well (as long as they don't use any functions that are not
available under R5).
The solution is a bit messy, but we have to live with it :-)
The runtime loader now patches the __gRuntimeLoader symbol in libroot.so to point
to its exported structure instead of passing it to the init functions as an
argument.
(Hax0red by axeld and bonefish on stippi's assimilated machine -- resistence is futile)


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15848 a95241bf-73f2-0310-859d-f6bbb57e9c96

Renamed src/kernel to src/system.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@12359 a95241bf-73f2-0310-859d-f6bbb57e9c96

libroot_debug: Merge guarded heap into libroot_debug.

The individual debug heap implementations are now exposed via a
structure of function pointers and a common frontend dispatches the
malloc and malloc_debug APIs through them.

The MALLOC_DEBUG environment variable can be used to select the guarded
heap by adding 'g', otherwise the debug heap is used as the default.
Consequently the separate libroot_guarded is not needed anymore and has
been removed.

To allow the use of environment variables this early, init_env_post_heap
has been added and the heap dependent atfork() moved there. This allowed
to fold the code of init_heap_post_env into init_heap so the former has
been removed.

malloc_debug: Implement allocation dump on exit in guarded heap.

When enabled (using heap_debug_dump_allocations_on_exit(true) or
MALLOC_DEBUG=e) this causes a dump of all remaining allocations when
libroot_debug is unloaded. It uses terminate_after to be called as
late as possible.

When combined with alloc stack traces this makes for a nice if a bit
crude leak checker. Note that a lot of allocations usually remain
even at that stage due to statically, lazyly and globally allocated
stuff from the various system libraries where it isn't necessarily
worth the overhead to free them when the program terminates anyway.

libroot: Add adaptive mutex implementation

libroot: explicitly check ABI version

As Axel suggested use simple, explicit checks for legacy ABI version
instead of obscure "compatibility mode".

libroot: make all areas executable for old binaries

* If at least one image is either B_HAIKU_ABI_GCC_2_ANCIENT or
B_HAIKU_ABI_GCC_2_BEOS almost all areas are marked as executable.
* B_EXECUTE_AREA and B_STACK_AREA are made public. The former is enforced since
the introduction of DEP and apps need it to correctly set area protection.
The latter is currently needed only to recognize stack areas and fix their
protection in compatibility mode, but may also be useful if an app wants
to use sigaltstack from POSIX API.

commpage: randomize position of commpage

This patch introduces randomization of commpage position. From now on commpage
table contains offsets from begining to of the commpage to the particular
commpage entry. Similary addresses of symbols in ELF memory image "commpage"
are just offsets from the begining of the commpage.

This patch also updates KDL so that commpage entries are recognized and shown
correctly in stack trace. An update of Debugger is yet to be done.

Merged signals-merge branch into trunk with the following changes:
* Reorganized the kernel locking related to threads and teams.
* We now discriminate correctly between process and thread signals. Signal
handlers have been moved to teams. Fixes #5679.
* Implemented real-time signal support, including signal queuing, SA_SIGINFO
support, sigqueue(), sigwaitinfo(), sigtimedwait(), waitid(), and the addition
of the real-time signal range. Closes #1935 and #2695.
* Gave SIGBUS a separate signal number. Fixes #6704.
* Implemented <time.h> clock and timer support, and fixed/completed alarm() and
[set]itimer(). Closes #5682.
* Implemented support for thread cancellation. Closes #5686.
* Moved send_signal() from <signal.h> to <OS.h>. Fixes #7554.
* Lots over smaller more or less related changes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@42116 a95241bf-73f2-0310-859d-f6bbb57e9c96

now correctly init the thread id in the pthread struct for main thread and threads spawned with spawn_thread(). This definitely helps for #7235.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40665 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Implemented a way to specify additional debug options via MALLOC_DEBUG when
using libroot_debug.so, instead of having to hardcode them in the application
via calling private heap functions.
* The following options are implemented: 'p' turns on paranoid validation,
'w' triggers periodic wall checking every 500ms ('W' does the same, but every
100ms), 'g' to use guard pages (beware, this will dramatically increase
memory usage), and 'r' which forbids reusing of memory, freed memory is never
actually freed.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40514 a95241bf-73f2-0310-859d-f6bbb57e9c96

Closing #7056:
* inherit umask of calling process to images loaded via exec...()

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40071 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Private libroot locking primitives:
- Reimplemented mutex to use the _kern_mutex*() syscalls.
- The initializer functions cannot fail anymore -- changed their return type
to void.
- Changed the initializer function semantics to not copy the name by default
anymore (as in the kernel). Also added *_etc() versions of them that take an
additional flags.
- Added static initializer macros.
- Made the mutex (and thus recursive_lock) lock functions non-interruptable.
- Got rid of the "lazy" version. They are no longer needed, since the
initialization of the standard types can be done statically and cannot fail.
* Adjusted libroot, runtime loader, and other code using the private libroot
locking primitives to the new semantics.
* pthreads mutexes and condition variables:
- Reimplemented using the _kern_mutex*() syscalls.
- Consistently use POSIX error codes.
- Fixed some not quite POSIX compliant behavior.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36323 a95241bf-73f2-0310-859d-f6bbb57e9c96

* This fixes the remaining warnings in exit.c


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34294 a95241bf-73f2-0310-859d-f6bbb57e9c96

Moved the rld_export structure from <user_runtime.h> to
<runtime_loader.h>, since it isn't a kernel <-> userland interface.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28456 a95241bf-73f2-0310-859d-f6bbb57e9c96

Works much better when actually allocating the TLS entry...
Now the backend works, Login can list users.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23509 a95241bf-73f2-0310-859d-f6bbb57e9c96

bonefish+mmlr:
As (our) gcc unfortunately uses atexit() to clean up lazily initialized static
variables inside functions we have to ensure that we do the right thing with
unloadable shared objects. In case a shared object was unloaded that installed
an atexit() hook the application would crash on exit. We now implement a
callback into libroot that is used to call all the atexit() hooks of a
component that is to be unloaded. Most prominently this fixes the media_server
crash at shutdown.

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23486 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added a mechanism to retrieve a BMessage with eventual error descriptions
for _kern_load_image().
* Added KMessage to the runtime_loader (a bit hacky, though) - it will use
it to deliver the above mentioned functionality.
* load_dependencies() did return the wrong status code in case a library
was missing; now it returns B_MISSING_LIBRARY.
* load_dependencies() will now try to load all dependencies when a report
message is requested; therefore, all missing libraries are listed.
* Renamed uspace_program_args to user_space_program_args.
* The kernel filled in various members of the user_space_program_args structure
unsafely, ie. was not using user_memcpy().
* Renamed some local variables in team.c to better fit our style guide (ie.
uargs to userArgs).
* Changed Tracker to use the new _kern_load_image() variant on Haiku to retrieve
and report all missing libraries. This fixes bug #1324.
* Adapted kernel_cpp.cpp to the runtime loader as well; the latter will now
compile with _LOADER_MODE defined.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21715 a95241bf-73f2-0310-859d-f6bbb57e9c96

Made our glue code compatible to BeOS again. IOW executables compiled for Haiku will
now run under BeOS as well (as long as they don't use any functions that are not
available under R5).
The solution is a bit messy, but we have to live with it :-)
The runtime loader now patches the __gRuntimeLoader symbol in libroot.so to point
to its exported structure instead of passing it to the init functions as an
argument.
(Hax0red by axeld and bonefish on stippi's assimilated machine -- resistence is futile)


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15848 a95241bf-73f2-0310-859d-f6bbb57e9c96

Renamed src/kernel to src/system.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@12359 a95241bf-73f2-0310-859d-f6bbb57e9c96