Malloc_debug: Fix Wformat-security

Change-Id: I2f891fff74a003aac7774ee91f2c28d5e65a18f7
Reviewed-on: https://review.haiku-os.org/c/haiku/+/4486
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>

libroot_debug: Replace two more uses of printf in guarded heap.

Use the internal print_stdout() instead as done when printing the
stack traces.

libroot_debug: Track freeing thread in guarded heap separatley.

Previously the thread member was overwritten with the freeing thread
when a page was freed, leading to confusion when hitting unallocated
pages due to the debugger message still stating "allocated by thread".

Track the freeing thread separately as it might be interesting to know
both, which thread initially allocated and which thread eventually freed
an allocation.

libroot_debug: Support alignments > B_PAGE_SIZE in guarded heap.

These are always allocated using an area. The allocation size is
increased as to guarantee the availability of a suitable address. The
pages between the allocation info and the actual, aligned start address
and the pages past the allocation end are then protected.

This commit also fixes corruption of the allocation info for large
allocations that used areas. The alignment wasn't taken into account
when calculating the amount of space needed. The alignment could then
lead to rounding down the allocation start such that it would overlap
with the allocation info.

libroot_debug.so: Fix missing alignment in guarded realloc.

libroot_debug: Revert to a legacy default alignment of 8.

This reverts the legacy default alignment (in absence of max_align_t)
to 8, as it was before.

libroot_debug: Merge guarded heap into libroot_debug.

The individual debug heap implementations are now exposed via a
structure of function pointers and a common frontend dispatches the
malloc and malloc_debug APIs through them.

The MALLOC_DEBUG environment variable can be used to select the guarded
heap by adding 'g', otherwise the debug heap is used as the default.
Consequently the separate libroot_guarded is not needed anymore and has
been removed.

To allow the use of environment variables this early, init_env_post_heap
has been added and the heap dependent atfork() moved there. This allowed
to fold the code of init_heap_post_env into init_heap so the former has
been removed.

guarded_heap: Use __arch_get_stack_trace() instead of syscall.

The stack base and end addresses are stored in TLS slots that are
prepared when enabling stack traces and filled in lazily on use for
each thread. This avoids the need of calling get_thread_info to get
these values.

Also simplifies the code somewhat due to proper frame skipping support.

guarded_heap: Replace symbol lookup syscall with runtime_loader.

Use the private runtime_loader API to do the symbol lookup instead of
using the syscall.

malloc_debug: Remove condition that is always true.

malloc_debug: Implement allocation dump on exit in guarded heap.

When enabled (using heap_debug_dump_allocations_on_exit(true) or
MALLOC_DEBUG=e) this causes a dump of all remaining allocations when
libroot_debug is unloaded. It uses terminate_after to be called as
late as possible.

When combined with alloc stack traces this makes for a nice if a bit
crude leak checker. Note that a lot of allocations usually remain
even at that stage due to statically, lazyly and globally allocated
stuff from the various system libraries where it isn't necessarily
worth the overhead to free them when the program terminates anyway.

malloc_debug: Also print stack traces when dumping guarded heap.

malloc_debug: Impl. heap_debug_dump_allocations in guarded heap.

malloc_debug: Implement alloc/free stack traces in guarded heap.

When configured to do so (using heap_debug_set_stack_trace_depth(depth)
or MALLOC_DEBUG=s<depth>) the guarded heap now captures stack traces on
alloc and free.

A crash due to hitting a guard page or an already freed page now dumps
these stack traces. In the case of use-after-free one can therefore see
both where the allocation was done and where it was freed.

Note that there is a hardcoded maximum stack trace depth of 50 and that
the alloc stack trace takes away space from the free stack trace which
uses up the rest of that maximum.

malloc_debug: Set default alignment to max_align_t if available.

For it to be available we build malloc_debug in C++11 mode when not
using GCC2. Note that max_align_t is not in the std namespace in GCC4
versions prior to GCC 4.9. The extra "using namespace std" is there to
be forward compatible once we update.

malloc_debug: Add default alignment option.

This allows for something similar as was implemented in 217f090 but
makes it optional and configurable.

The MALLOC_DEBUG environment variable now can take "a<size>" to set
the default alignment to the specified size. Note that not all
alignments may be supported depending on the heap implementation.

Revert "malloc_debug: align allocations".

This reverts commit 217f090f9e247d1d4c5644e626642c430fafe4e5.

At least for the guarded heap this completely defeats the purpose. If
software requires a certain alignment it should request it using
memalign explicitly instead of assuming it.

libroot_debug: Fix build of guarded_heap after atomic changes.

malloc_debug: align allocations

* Align all allocations of more than 8 bytes to 8-byte.
* Avoids hitting ASSERTs in WebKit when built in debug mode (it assumes
at least 8 byte alignment)

Fix a few function signatures in the guarded heap.

* Not including malloc.h caused the memalign() signature to not be a C
signature, therefore leading to linking errors. Fix the missing
include and explicitly add extern "C" as well.
* Some remaining asterisk style cleanup.

Update the guarded heap areas after fork.

We don't actually use them for anything yet though.

Add a userland version of the guarded heap to libroot_debug.

The guarded heap uses mprotect() to protect freed/unallocated pages so
that any access to such a page results in a segfault. It also installs
a segfault handler that in such an event prints some info about the
accessed page and then calls the debugger with a meaningful message.

It implements the same interface as the debug heap so it can simply be
swapped out by changing the Jamfile. As it doesn't support most of the
extra debug features (wall checking is obviously superfluous, but it
also doesn't help in leak checking) and as it is hugely space
inefficient I left it disabled for now.

libroot_debug: Replace two more uses of printf in guarded heap.

Use the internal print_stdout() instead as done when printing the
stack traces.

libroot_debug: Track freeing thread in guarded heap separatley.

Previously the thread member was overwritten with the freeing thread
when a page was freed, leading to confusion when hitting unallocated
pages due to the debugger message still stating "allocated by thread".

Track the freeing thread separately as it might be interesting to know
both, which thread initially allocated and which thread eventually freed
an allocation.

libroot_debug: Support alignments > B_PAGE_SIZE in guarded heap.

These are always allocated using an area. The allocation size is
increased as to guarantee the availability of a suitable address. The
pages between the allocation info and the actual, aligned start address
and the pages past the allocation end are then protected.

This commit also fixes corruption of the allocation info for large
allocations that used areas. The alignment wasn't taken into account
when calculating the amount of space needed. The alignment could then
lead to rounding down the allocation start such that it would overlap
with the allocation info.

libroot_debug.so: Fix missing alignment in guarded realloc.

libroot_debug: Revert to a legacy default alignment of 8.

This reverts the legacy default alignment (in absence of max_align_t)
to 8, as it was before.

libroot_debug: Merge guarded heap into libroot_debug.

The individual debug heap implementations are now exposed via a
structure of function pointers and a common frontend dispatches the
malloc and malloc_debug APIs through them.

The MALLOC_DEBUG environment variable can be used to select the guarded
heap by adding 'g', otherwise the debug heap is used as the default.
Consequently the separate libroot_guarded is not needed anymore and has
been removed.

To allow the use of environment variables this early, init_env_post_heap
has been added and the heap dependent atfork() moved there. This allowed
to fold the code of init_heap_post_env into init_heap so the former has
been removed.

guarded_heap: Use __arch_get_stack_trace() instead of syscall.

The stack base and end addresses are stored in TLS slots that are
prepared when enabling stack traces and filled in lazily on use for
each thread. This avoids the need of calling get_thread_info to get
these values.

Also simplifies the code somewhat due to proper frame skipping support.

guarded_heap: Replace symbol lookup syscall with runtime_loader.

Use the private runtime_loader API to do the symbol lookup instead of
using the syscall.

malloc_debug: Remove condition that is always true.

malloc_debug: Implement allocation dump on exit in guarded heap.

When enabled (using heap_debug_dump_allocations_on_exit(true) or
MALLOC_DEBUG=e) this causes a dump of all remaining allocations when
libroot_debug is unloaded. It uses terminate_after to be called as
late as possible.

When combined with alloc stack traces this makes for a nice if a bit
crude leak checker. Note that a lot of allocations usually remain
even at that stage due to statically, lazyly and globally allocated
stuff from the various system libraries where it isn't necessarily
worth the overhead to free them when the program terminates anyway.

malloc_debug: Also print stack traces when dumping guarded heap.

malloc_debug: Impl. heap_debug_dump_allocations in guarded heap.

malloc_debug: Implement alloc/free stack traces in guarded heap.

When configured to do so (using heap_debug_set_stack_trace_depth(depth)
or MALLOC_DEBUG=s<depth>) the guarded heap now captures stack traces on
alloc and free.

A crash due to hitting a guard page or an already freed page now dumps
these stack traces. In the case of use-after-free one can therefore see
both where the allocation was done and where it was freed.

Note that there is a hardcoded maximum stack trace depth of 50 and that
the alloc stack trace takes away space from the free stack trace which
uses up the rest of that maximum.

malloc_debug: Set default alignment to max_align_t if available.

For it to be available we build malloc_debug in C++11 mode when not
using GCC2. Note that max_align_t is not in the std namespace in GCC4
versions prior to GCC 4.9. The extra "using namespace std" is there to
be forward compatible once we update.

malloc_debug: Add default alignment option.

This allows for something similar as was implemented in 217f090 but
makes it optional and configurable.

The MALLOC_DEBUG environment variable now can take "a<size>" to set
the default alignment to the specified size. Note that not all
alignments may be supported depending on the heap implementation.

Revert "malloc_debug: align allocations".

This reverts commit 217f090f9e247d1d4c5644e626642c430fafe4e5.

At least for the guarded heap this completely defeats the purpose. If
software requires a certain alignment it should request it using
memalign explicitly instead of assuming it.

libroot_debug: Fix build of guarded_heap after atomic changes.

malloc_debug: align allocations

* Align all allocations of more than 8 bytes to 8-byte.
* Avoids hitting ASSERTs in WebKit when built in debug mode (it assumes
at least 8 byte alignment)

Fix a few function signatures in the guarded heap.

* Not including malloc.h caused the memalign() signature to not be a C
signature, therefore leading to linking errors. Fix the missing
include and explicitly add extern "C" as well.
* Some remaining asterisk style cleanup.

Update the guarded heap areas after fork.

We don't actually use them for anything yet though.

Add a userland version of the guarded heap to libroot_debug.

The guarded heap uses mprotect() to protect freed/unallocated pages so
that any access to such a page results in a segfault. It also installs
a segfault handler that in such an event prints some info about the
accessed page and then calls the debugger with a meaningful message.

It implements the same interface as the debug heap so it can simply be
swapped out by changing the Jamfile. As it doesn't support most of the
extra debug features (wall checking is obviously superfluous, but it
also doesn't help in leak checking) and as it is hugely space
inefficient I left it disabled for now.