libroot: Fix open flags in shm_open.

We need O_CLOEXEC here, not FD_CLOEXEC (which is equal to 0x1, which
in openflags is O_WRONLY and thus leads to the conflicts caught by
the previous commit.)

Fixes haikuports/haikuports#7524.
(cherry picked from commit 59e09f7bebcb8fa9d4ed82f501e6516c3d67d65b)

Change-Id: I5751822ac48ae7a865755494da241e4bfd34c45f
Reviewed-on: https://review.haiku-os.org/c/haiku/+/5853
Reviewed-by: waddlesplash <waddlesplash@gmail.com>

implement mlock(), munlock()

Change-Id: I2f04b8986d2ed32bb4d30d238d668e21a1505778
Reviewed-on: https://review.haiku-os.org/c/haiku/+/1991
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>

libroot/kernel: Implement MADV_FREE madvise() extension.

It allows an application to signal that it no longer needs the data in
the given address range and the underlying pages can be discarded and
reused elsewhere. This is finer grained than working with full areas
or mappings at a time and enables unmapping sections of partially used
mappings without giving up its address space.

Compared with punching holes into a mapping by "mapping over" with
PROT_NONE and MAP_NORESERVE, this has the obvious advantage of not
producing a lot of unused extra areas and saves the corresponding
resources. It is also a lot "lighter" of an operation than cutting
existing areas.

This introduces madvise() alongside the existing posix_madvise() to
allow for OS specific extensions. The constants for both functions are
aliased, the POSIX_MADV_* being a subset of the MADV_* ones without the
non-POSIX extensions. Internally posix_madvise() simply calls madvise().

MADV_FREE is commonly supported in other OSes with various subtle
semantic differences as to when pages are actually freed/cleared and how
or whether the pages are counted against the memory use of a process.
In the variant implemented here, pages are always immediately discarded
and memory counting is not altered. This behaviour should be considered
an implementation detail and may be altered later. The actual unmap and
discard could for example be delayed until pages are needed elsewhere to
reduce overhead in case of repeated discarding and remapping.

Note that MADV_FREE doesn't really align with the rest of the madvise()
API as it works like a command (i.e. discard these pages) and does not
add an attribute to the pages in the given range (i.e. mark these pages
for quick access from now on). As such, an MADV_FREE does not need to be
undone by setting a different advice later on, unlike how the other
flags work. This discrepancy may be the reason why it is not part of
POSIX.

Change-Id: Icc093379125a43e465dc4409d8f5ae0f64e107e0
Reviewed-on: https://review.haiku-os.org/c/haiku/+/2844
Reviewed-by: waddlesplash <waddlesplash@gmail.com>

mmap: Use MAP_NORESERVE to request overcommit, not PROT_NONE.

This reverts hrev54120 and instead adds the commonly supported
MAP_NORESERVE flag to request overcommit.

Using PROT_NONE for overcommit is problematic as the protection of
individual pages can still be changed via mprotect to make them
accessible, but that won't change the commitment. An application
using such a pattern may then unexpectedly run into out of memory
conditions on random writes into the address space.

With MAP_NORESERVE the overcommit can explicitly be requested by
applications that want to reserve address space without producing
memory pressure.

Change-Id: Id213d2245c5e23103e8e0857f7902e0cd8a2c65d
Reviewed-on: https://review.haiku-os.org/c/haiku/+/2611
Reviewed-by: waddlesplash <waddlesplash@gmail.com>
Reviewed-by: Jérôme Duval <jerome.duval@gmail.com>

mmap: using PROT_NONE usually means the memory isn't actually used.

A program can mmap virtual memory which will only trigger signal handlers.
This is for instance needed for ASAN.

Change-Id: I4a42b4860b5acab17465683e9cf73c486bea7d40
Reviewed-on: https://review.haiku-os.org/c/haiku/+/2554
Reviewed-by: waddlesplash <waddlesplash@gmail.com>

libroot: Name mmap areas by the name of the image that allocated them.

This adds an average overhead of 10-15 us, which seems acceptable
(without the previous patch it adds 200-250 us, which is less so),
and it makes "listarea" output much more enlightening for applications
like app_server or WebKit which use a lot of them.

Change-Id: I48a4148e6e9fb0d1a8bbf67294deeafad9372f44
Reviewed-on: https://review.haiku-os.org/c/1462
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>

kernel/OS.h: Include sys/types.h instead of pthread.h.

This avoids polluting the namespace significantly. Also adjust
all files which depended on this behavior to include pthread.h
directly.

POSIX: shm_open: set the FD_CLOEXEC file descriptor flag...

on the new file descriptor.
Fixes #13774.

mmap(): Fix !MAP_FIXED with non-NULL address case

In that case the caller ideally wants to obtain an allocation at the
specified address, which was thwarted by using
B_RANDOMIZED_BASE_ADDRESS. Use B_BASE_ADDRESS instead.

This improves the experience with the gcc 4 pre-compiled headers
implementation (which expects to be able to map the PCH file at the same
address where it was located originally when it had been created), but
doesn't fix it completely. As long as ASLR is active, it is always
possible that something else (mapped shared objects, heap, stack) is in
the way.

libroot: randomize position of areas created by mmap()

When mmap() is invoked without specifying address hint B_RANDOMIZED_ANY_ADDRESS
is used.
Otherwise, unless MAP_FIXED flag is set (which requires mmap() to return an area
positioned exactly at given address), B_RANDOMIZED_BASE_ADDRESS is used.

mmap should try to honour the address hint

Introduce __set_errno() throughout libroot.

* add errno_private.h, which defines the __set_errno() macro with
and without tracing
* instead of setting errno manually, all libroot's code now invokes
__set_errno(), which makes it much easier to trace changes to errno
* redirect glibc's use of __set_errno() to our own version

Merged signals-merge branch into trunk with the following changes:
* Reorganized the kernel locking related to threads and teams.
* We now discriminate correctly between process and thread signals. Signal
handlers have been moved to teams. Fixes #5679.
* Implemented real-time signal support, including signal queuing, SA_SIGINFO
support, sigqueue(), sigwaitinfo(), sigtimedwait(), waitid(), and the addition
of the real-time signal range. Closes #1935 and #2695.
* Gave SIGBUS a separate signal number. Fixes #6704.
* Implemented <time.h> clock and timer support, and fixed/completed alarm() and
[set]itimer(). Closes #5682.
* Implemented support for thread cancellation. Closes #5686.
* Moved send_signal() from <signal.h> to <OS.h>. Fixes #7554.
* Lots over smaller more or less related changes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@42116 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Moved the "var" directory to /boot/common/.
* Moved the "tmp" directory out of /var, and to /boot/common/cache/.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35104 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added an "unmapAddressRange" argument to the file mapping syscall. This is
the first part of making the runtime loader behave itself; it should already
make Clockwerk run okay with any number of translators (even if not all of
them will work yet).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31113 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Implemented mprotect(). A vm_area does now have an optional array
specifying the protection of each page (4 bits per page).
* Added no-op implementation of posix_madvise().
* Replaced a few "addr_t size" parameters by "size_t size".


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26871 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added vm_page_write_modified_page_range(), which is similar to
vm_page_write_modified_pages(), save that it only writes pages in the
given range.
* Added vm_page_schedule_write_page_range() which schedules all modified
pages in the given cache's range for writing by the page writer.
* Added _kern_sync_memory() syscall and the msync() POSIX function.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25620 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Introduced new header directory headers/private/system which is supposed
to contain headers shared by kernel and userland (mainly libroot).
* Moved quite a few private kernel headers to the new location. Split
several kernel headers into a shared part and one that is still kernel
private. Adjusted all affected Jamfiles and source in the standard x86
build accordingly. The build for other architectures and for test code
may be broken.
* Quite a bit of userland code still includes private kernel headers.
Mostly those are <util/*> headers. The ones that aren't strictly
kernel-only should be moved to some other place (maybe
headers/private/shared/util).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25486 a95241bf-73f2-0310-859d-f6bbb57e9c96

Implement shm_open() and shm_unlink(). The shared memory objects are
simply created as files in /boot/var/shared_memory/. The Bootscript
clears the directory.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25374 a95241bf-73f2-0310-859d-f6bbb57e9c96

mmap() was a little strict: The length parameter need not be page
aligned. It shall be rounded up by the implementation.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24967 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added <sys/mman.h> header. It declares only mmap() and munmap() yet
and defines the macros needed by them.
* Renamed syscall sys_vm_map_file() to _kern_map_file() and changed the
path to an FD parameter. Changed vm_map_file() accordingly and
adjusted the kernel ELF loader and the runtime loader.
* Added syscall _kern_unmap_memory().
* Added bool unmapAddressRange parameter to vm_create_anonymous_area()
and map_backing_store(). If true and the address specification is
B_EXACT_ADDRESS, all areas in the specified address range will be
deleted (unless an area is covered only partially).
* Introduced B_SHARED_AREA flag, which is set on areas that have been
created by {vm,_user}_map_file() with REGION_NO_PRIVATE_MAP. When
fork()ing those areas won't be copied CoW, but rather be cloned. This
is needed for mmap() MAP_SHARED.
* {vm,_user}_map_file() also accept an FD argument < 0, in which case an
anonymous area is created.
* Implemented mmap() and munmap(). Currently there's the restriction
that we can't partially unmap areas. Otherwise the functions should be
rather compliant. We also support the non-POSIX extension
MAP_ANONYMOUS.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24964 a95241bf-73f2-0310-859d-f6bbb57e9c96

mmap(): Fix !MAP_FIXED with non-NULL address case

In that case the caller ideally wants to obtain an allocation at the
specified address, which was thwarted by using
B_RANDOMIZED_BASE_ADDRESS. Use B_BASE_ADDRESS instead.

This improves the experience with the gcc 4 pre-compiled headers
implementation (which expects to be able to map the PCH file at the same
address where it was located originally when it had been created), but
doesn't fix it completely. As long as ASLR is active, it is always
possible that something else (mapped shared objects, heap, stack) is in
the way.

libroot: randomize position of areas created by mmap()

When mmap() is invoked without specifying address hint B_RANDOMIZED_ANY_ADDRESS
is used.
Otherwise, unless MAP_FIXED flag is set (which requires mmap() to return an area
positioned exactly at given address), B_RANDOMIZED_BASE_ADDRESS is used.

mmap should try to honour the address hint

Introduce __set_errno() throughout libroot.

* add errno_private.h, which defines the __set_errno() macro with
and without tracing
* instead of setting errno manually, all libroot's code now invokes
__set_errno(), which makes it much easier to trace changes to errno
* redirect glibc's use of __set_errno() to our own version

Merged signals-merge branch into trunk with the following changes:
* Reorganized the kernel locking related to threads and teams.
* We now discriminate correctly between process and thread signals. Signal
handlers have been moved to teams. Fixes #5679.
* Implemented real-time signal support, including signal queuing, SA_SIGINFO
support, sigqueue(), sigwaitinfo(), sigtimedwait(), waitid(), and the addition
of the real-time signal range. Closes #1935 and #2695.
* Gave SIGBUS a separate signal number. Fixes #6704.
* Implemented <time.h> clock and timer support, and fixed/completed alarm() and
[set]itimer(). Closes #5682.
* Implemented support for thread cancellation. Closes #5686.
* Moved send_signal() from <signal.h> to <OS.h>. Fixes #7554.
* Lots over smaller more or less related changes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@42116 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Moved the "var" directory to /boot/common/.
* Moved the "tmp" directory out of /var, and to /boot/common/cache/.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35104 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added an "unmapAddressRange" argument to the file mapping syscall. This is
the first part of making the runtime loader behave itself; it should already
make Clockwerk run okay with any number of translators (even if not all of
them will work yet).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31113 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Implemented mprotect(). A vm_area does now have an optional array
specifying the protection of each page (4 bits per page).
* Added no-op implementation of posix_madvise().
* Replaced a few "addr_t size" parameters by "size_t size".


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26871 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added vm_page_write_modified_page_range(), which is similar to
vm_page_write_modified_pages(), save that it only writes pages in the
given range.
* Added vm_page_schedule_write_page_range() which schedules all modified
pages in the given cache's range for writing by the page writer.
* Added _kern_sync_memory() syscall and the msync() POSIX function.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25620 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Introduced new header directory headers/private/system which is supposed
to contain headers shared by kernel and userland (mainly libroot).
* Moved quite a few private kernel headers to the new location. Split
several kernel headers into a shared part and one that is still kernel
private. Adjusted all affected Jamfiles and source in the standard x86
build accordingly. The build for other architectures and for test code
may be broken.
* Quite a bit of userland code still includes private kernel headers.
Mostly those are <util/*> headers. The ones that aren't strictly
kernel-only should be moved to some other place (maybe
headers/private/shared/util).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25486 a95241bf-73f2-0310-859d-f6bbb57e9c96

Implement shm_open() and shm_unlink(). The shared memory objects are
simply created as files in /boot/var/shared_memory/. The Bootscript
clears the directory.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25374 a95241bf-73f2-0310-859d-f6bbb57e9c96

mmap() was a little strict: The length parameter need not be page
aligned. It shall be rounded up by the implementation.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24967 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added <sys/mman.h> header. It declares only mmap() and munmap() yet
and defines the macros needed by them.
* Renamed syscall sys_vm_map_file() to _kern_map_file() and changed the
path to an FD parameter. Changed vm_map_file() accordingly and
adjusted the kernel ELF loader and the runtime loader.
* Added syscall _kern_unmap_memory().
* Added bool unmapAddressRange parameter to vm_create_anonymous_area()
and map_backing_store(). If true and the address specification is
B_EXACT_ADDRESS, all areas in the specified address range will be
deleted (unless an area is covered only partially).
* Introduced B_SHARED_AREA flag, which is set on areas that have been
created by {vm,_user}_map_file() with REGION_NO_PRIVATE_MAP. When
fork()ing those areas won't be copied CoW, but rather be cloned. This
is needed for mmap() MAP_SHARED.
* {vm,_user}_map_file() also accept an FD argument < 0, in which case an
anonymous area is created.
* Implemented mmap() and munmap(). Currently there's the restriction
that we can't partially unmap areas. Otherwise the functions should be
rather compliant. We also support the non-POSIX extension
MAP_ANONYMOUS.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24964 a95241bf-73f2-0310-859d-f6bbb57e9c96