runtime_loader: dlopen() should respect RPATH of the loading module

Change-Id: Ic58edb53114dfff30cc7188957cd32508fa8bd48
Reviewed-on: https://review.haiku-os.org/c/haiku/+/3798
Reviewed-by: Jérôme Duval <jerome.duval@gmail.com>

libroot: make API version available.

Change-Id: I370d5c8e9b4f076e7f115ca6c87bc7f943d36c17
Reviewed-on: https://review.haiku-os.org/c/haiku/+/1850
Reviewed-by: Rene Gollent <rene@gollent.com>

runtime_loader: increase max length for image names.

* was previously 32, which would truncate longer shared object names.
* fixed #13436.

runtime_loader: add support for ELF init/term routine arrays.

* binutils 2.27 defaults to DT_INIT_ARRAY instead of DT_INIT.

system: Build using public elf.h header

Reduce duplication of code by

* Removing from elf_common.h definitions available in os/kernel/elf.h
* Deleting elf32.h and elf64.h
* Renaming elf_common.h to elf_private.h
* Updating source to build using public and private ELF header files
together

Signed-off-by: Jessica Hamilton <jessica.l.hamilton@gmail.com>

runtime_loader: Add imageName and exactMatch to symbol lookup.

Extend the get_nearest_symbol_at_address() private runtime_loader
export to include imageName and exactMatch arguments.

The imageName holds the SONAME of the image, if available, so cannot
neccessarily be extracted from the image path.

Whether or not there was an exact match, i.e. the symbol with its size
contains the address, is now returned in exactMatch.

runtime_loader: add support for ELF based TLS

This patch introduces support of ELF based TLS handling with lazy allocation
and initalization of TLS block for each DSO and thread. The implementation
generally follows the official ABI except that generation counter in dtv
is in fact a pointer to Generation object that contains both generation
counter and size of the dtv. That simplified the implementation a bit, but
could be changed later. The ABI requirements regariding in memory position
of TLS block is not honoured what results in static TLS model being
unsupported. However, that should not be a problem as long as
"executables" in Haiku are in fact shared objects and optimizations which
require specific TLS block in memory layout are not possible anyway.

runtime loader: Export get_executable_architecture() function

Given a path of an ELF file, it tries to determine its architecture.

libroot: make all areas executable for old binaries

* If at least one image is either B_HAIKU_ABI_GCC_2_ANCIENT or
B_HAIKU_ABI_GCC_2_BEOS almost all areas are marked as executable.
* B_EXECUTE_AREA and B_STACK_AREA are made public. The former is enforced since
the introduction of DEP and apps need it to correctly set area protection.
The latter is currently needed only to recognize stack areas and fix their
protection in compatibility mode, but may also be useful if an app wants
to use sigaltstack from POSIX API.

commpage: randomize position of commpage

This patch introduces randomization of commpage position. From now on commpage
table contains offsets from begining to of the commpage to the particular
commpage entry. Similary addresses of symbols in ELF memory image "commpage"
are just offsets from the begining of the commpage.

This patch also updates KDL so that commpage entries are recognized and shown
correctly in stack trace. An update of Debugger is yet to be done.

Changed runtime_loader to use elf_* typedefs over Elf32_*.

This means that it will be using ELF64 types on x86_64 rather than
ELF32. The next step for supporting x86_64 is to implement relocations.

Support ELF64 in the kernel.

This has been done by adding typedefs in elf_common.h to the correct ELF
structures for the architecture, and changing all Elf32_* uses to those
types. I don't know whether image loading works as I cannot test it yet,
there may be some 64-bit safety issues around. However, symbol lookup for
the kernel is working correctly.

Added x86_64 ELF64 relocation functions for the bootloader. All that's left to do now is handle the 64-bit load address properly.

Fix dladdr behaviour

* If dladdr can't find an exact match, it returns the nearest symbol
less than the given address.
* If no suitable symbol can be found, but the address is within a
loaded library, dladdr returns the library name and base address.

Signed-off-by: Ingo Weinhold <ingo_weinhold@gmx.de>

* Implemented dladdr() in the runtime loader. This is like a gazillion times
faster than before.
* This also solves a TODO in dladdr(), although I did not use
get_library_symbol() as I didn't quite see how that could fit as the comment
suggested; there is now a new function get_symbol_at_address() for this.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@42598 a95241bf-73f2-0310-859d-f6bbb57e9c96

Removed unused elf_version_info::hidden field.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30817 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Also define the build system variables TARGET_STATIC_{LIBSTDC++,LIBSUPC++}.
* runtime loader:
- Fixed gcc 4 warnings.
- Enabled -Werror.
- Renamed all remaining *.c source files to *.cpp.
- Implemented GNU style ELF symbol versioning support.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30806 a95241bf-73f2-0310-859d-f6bbb57e9c96

Removed the gcc_version from the runtime loader's image_t. Instead we always
determine (or guess) Haiku version and ABI and use those for compatibility
decisions.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30747 a95241bf-73f2-0310-859d-f6bbb57e9c96

Extended image_info by fields for the Haiku version and ABI. The runtime loader
and the kernel read those values from the shared object (if available). In the
runtime loader this should eventually replace the gcc version guessing method
currently used (at least for shared objects built for Haiku). The optional
packages need to be rebuilt first, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30729 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added private get_image_symbol_etc() that can recursively search for a symbol
(similar to how dlsym() works).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29356 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Reworked undefined symbol resolution in the runtime loader. Got rid of
the per-root-image breadth-first sorted image array. Instead we have a
per-image hook function to resolve the symbols. The default function
uses the sLoadedImages list directly, which is breadth-first sorted
anyway. There's also a BeOS function for old-style symbol resolution
and one for add-ons, which lacks a proper implementation yet (just
uses old-style ATM).
* Made the dl*() functions POSIX compliant:
- dlopen() does no longer use load_add_on(), but loads the object as a
library. It also properly supports a NULL name, now -- the previous
"_APP_" work-around did only work, if this soname was set on the
program (unlikely for programs using this API).
- Implemented RTLD_{GLOBAL,LOCAL}.
- dlsym() looks up symbols properly now, i.e. not just in the given
image, but breadth-first for an actual image or in load order for
the global scope. It also supports the not-quite POSIX RTLD_DEFAULT
and RTLD_NEXT extensions. Our RTLD_NEXT finds more symbols than in
Linux (also in later dlopen()ed libraries), but that should be fine.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28568 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added an "add-on" interface for the runtime loader. Pre-loaded images
can export a structure containing callback hooks invoked by the
runtime loader when certain image events occur (image loaded,
relocated, initialized, etc.).
* Also added a mechanism to patch image symbols. For an image callback
functions can be installed that patch symbols exported or imported by
the image.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28475 a95241bf-73f2-0310-859d-f6bbb57e9c96

Moved the rld_export structure from <user_runtime.h> to
<runtime_loader.h>, since it isn't a kernel <-> userland interface.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28456 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Changed the way symbols are resolved. For each root image (program,
add-on) we create a breadth-first sorted image list and use that to
search for symbols.
* Added support for preloading libraries. The environment variable
LD_PRELOAD can contain a whitespace-separated list of shared objects
that will be loaded before the program. This allows to replace
symbols without changing the executable or libraries.
* Resolved TODO in load_program() regarding the order of remapping the
images and initializing the dependencies (problem fixed in r28453).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28455 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Threw away the broken stat() vs. _stat() mechanism to allow for more fields
in struct stat.
* Instead, I followed Marcus' great idea and added a compatibility check in
the runtime loader: now, R5 binaries (also shared libraries) are detected,
and they get special versions for stat(), fstat(), and lstat() that return
the smaller stat struct.
* However, I've disabled (in src/system/libroot/posix/sys/stat.c) using the
larger stat field for now, as this breaks some of our optional packages.
So until we rebuild them all, this shouldn't be enabled.
* This should now also be used for BeOS compatibility in libnetwork.so.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27961 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Decide whether to use BeOS style symbol resolution at run time
depending on the gcc version of the executable.
* Adjusted non-BeOS-style symbol resolution so that add-ons and
dynamically loaded libraries find symbols in the executable.

This change re-enables support for undefined symbols.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24537 a95241bf-73f2-0310-859d-f6bbb57e9c96

The runtime loader now reads a part of the .comment section of each
shared object to find out the GCC version it has been compiled with.
This is not an exact science, since we've got the version string for
every single object file that has been linked in, but my heuristic seems
to be good enough.

Having the gcc version at hand will allow for two features: Enabling
work-arounds for old executables (like the type info problem in BeOS
apps), and automatically selecting the right set of libraries in a mixed
gcc 2/gcc 4 environment.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24534 a95241bf-73f2-0310-859d-f6bbb57e9c96

Now supports symbolic links correctly, and no longer loads the same shared
library twice.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@17929 a95241bf-73f2-0310-859d-f6bbb57e9c96

The runtime loader now creates a debug area through which other teams can
access a team's internal loader data.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@13697 a95241bf-73f2-0310-859d-f6bbb57e9c96

runtime_loader: Add imageName and exactMatch to symbol lookup.

Extend the get_nearest_symbol_at_address() private runtime_loader
export to include imageName and exactMatch arguments.

The imageName holds the SONAME of the image, if available, so cannot
neccessarily be extracted from the image path.

Whether or not there was an exact match, i.e. the symbol with its size
contains the address, is now returned in exactMatch.

runtime_loader: add support for ELF based TLS

This patch introduces support of ELF based TLS handling with lazy allocation
and initalization of TLS block for each DSO and thread. The implementation
generally follows the official ABI except that generation counter in dtv
is in fact a pointer to Generation object that contains both generation
counter and size of the dtv. That simplified the implementation a bit, but
could be changed later. The ABI requirements regariding in memory position
of TLS block is not honoured what results in static TLS model being
unsupported. However, that should not be a problem as long as
"executables" in Haiku are in fact shared objects and optimizations which
require specific TLS block in memory layout are not possible anyway.

runtime loader: Export get_executable_architecture() function

Given a path of an ELF file, it tries to determine its architecture.

libroot: make all areas executable for old binaries

* If at least one image is either B_HAIKU_ABI_GCC_2_ANCIENT or
B_HAIKU_ABI_GCC_2_BEOS almost all areas are marked as executable.
* B_EXECUTE_AREA and B_STACK_AREA are made public. The former is enforced since
the introduction of DEP and apps need it to correctly set area protection.
The latter is currently needed only to recognize stack areas and fix their
protection in compatibility mode, but may also be useful if an app wants
to use sigaltstack from POSIX API.

commpage: randomize position of commpage

This patch introduces randomization of commpage position. From now on commpage
table contains offsets from begining to of the commpage to the particular
commpage entry. Similary addresses of symbols in ELF memory image "commpage"
are just offsets from the begining of the commpage.

This patch also updates KDL so that commpage entries are recognized and shown
correctly in stack trace. An update of Debugger is yet to be done.

Changed runtime_loader to use elf_* typedefs over Elf32_*.

This means that it will be using ELF64 types on x86_64 rather than
ELF32. The next step for supporting x86_64 is to implement relocations.

Support ELF64 in the kernel.

This has been done by adding typedefs in elf_common.h to the correct ELF
structures for the architecture, and changing all Elf32_* uses to those
types. I don't know whether image loading works as I cannot test it yet,
there may be some 64-bit safety issues around. However, symbol lookup for
the kernel is working correctly.

Added x86_64 ELF64 relocation functions for the bootloader. All that's left to do now is handle the 64-bit load address properly.

Fix dladdr behaviour

* If dladdr can't find an exact match, it returns the nearest symbol
less than the given address.
* If no suitable symbol can be found, but the address is within a
loaded library, dladdr returns the library name and base address.

Signed-off-by: Ingo Weinhold <ingo_weinhold@gmx.de>

* Implemented dladdr() in the runtime loader. This is like a gazillion times
faster than before.
* This also solves a TODO in dladdr(), although I did not use
get_library_symbol() as I didn't quite see how that could fit as the comment
suggested; there is now a new function get_symbol_at_address() for this.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@42598 a95241bf-73f2-0310-859d-f6bbb57e9c96

Removed unused elf_version_info::hidden field.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30817 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Also define the build system variables TARGET_STATIC_{LIBSTDC++,LIBSUPC++}.
* runtime loader:
- Fixed gcc 4 warnings.
- Enabled -Werror.
- Renamed all remaining *.c source files to *.cpp.
- Implemented GNU style ELF symbol versioning support.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30806 a95241bf-73f2-0310-859d-f6bbb57e9c96

Removed the gcc_version from the runtime loader's image_t. Instead we always
determine (or guess) Haiku version and ABI and use those for compatibility
decisions.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30747 a95241bf-73f2-0310-859d-f6bbb57e9c96

Extended image_info by fields for the Haiku version and ABI. The runtime loader
and the kernel read those values from the shared object (if available). In the
runtime loader this should eventually replace the gcc version guessing method
currently used (at least for shared objects built for Haiku). The optional
packages need to be rebuilt first, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30729 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added private get_image_symbol_etc() that can recursively search for a symbol
(similar to how dlsym() works).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29356 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Reworked undefined symbol resolution in the runtime loader. Got rid of
the per-root-image breadth-first sorted image array. Instead we have a
per-image hook function to resolve the symbols. The default function
uses the sLoadedImages list directly, which is breadth-first sorted
anyway. There's also a BeOS function for old-style symbol resolution
and one for add-ons, which lacks a proper implementation yet (just
uses old-style ATM).
* Made the dl*() functions POSIX compliant:
- dlopen() does no longer use load_add_on(), but loads the object as a
library. It also properly supports a NULL name, now -- the previous
"_APP_" work-around did only work, if this soname was set on the
program (unlikely for programs using this API).
- Implemented RTLD_{GLOBAL,LOCAL}.
- dlsym() looks up symbols properly now, i.e. not just in the given
image, but breadth-first for an actual image or in load order for
the global scope. It also supports the not-quite POSIX RTLD_DEFAULT
and RTLD_NEXT extensions. Our RTLD_NEXT finds more symbols than in
Linux (also in later dlopen()ed libraries), but that should be fine.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28568 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added an "add-on" interface for the runtime loader. Pre-loaded images
can export a structure containing callback hooks invoked by the
runtime loader when certain image events occur (image loaded,
relocated, initialized, etc.).
* Also added a mechanism to patch image symbols. For an image callback
functions can be installed that patch symbols exported or imported by
the image.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28475 a95241bf-73f2-0310-859d-f6bbb57e9c96

Moved the rld_export structure from <user_runtime.h> to
<runtime_loader.h>, since it isn't a kernel <-> userland interface.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28456 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Changed the way symbols are resolved. For each root image (program,
add-on) we create a breadth-first sorted image list and use that to
search for symbols.
* Added support for preloading libraries. The environment variable
LD_PRELOAD can contain a whitespace-separated list of shared objects
that will be loaded before the program. This allows to replace
symbols without changing the executable or libraries.
* Resolved TODO in load_program() regarding the order of remapping the
images and initializing the dependencies (problem fixed in r28453).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28455 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Threw away the broken stat() vs. _stat() mechanism to allow for more fields
in struct stat.
* Instead, I followed Marcus' great idea and added a compatibility check in
the runtime loader: now, R5 binaries (also shared libraries) are detected,
and they get special versions for stat(), fstat(), and lstat() that return
the smaller stat struct.
* However, I've disabled (in src/system/libroot/posix/sys/stat.c) using the
larger stat field for now, as this breaks some of our optional packages.
So until we rebuild them all, this shouldn't be enabled.
* This should now also be used for BeOS compatibility in libnetwork.so.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27961 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Decide whether to use BeOS style symbol resolution at run time
depending on the gcc version of the executable.
* Adjusted non-BeOS-style symbol resolution so that add-ons and
dynamically loaded libraries find symbols in the executable.

This change re-enables support for undefined symbols.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24537 a95241bf-73f2-0310-859d-f6bbb57e9c96

The runtime loader now reads a part of the .comment section of each
shared object to find out the GCC version it has been compiled with.
This is not an exact science, since we've got the version string for
every single object file that has been linked in, but my heuristic seems
to be good enough.

Having the gcc version at hand will allow for two features: Enabling
work-arounds for old executables (like the type info problem in BeOS
apps), and automatically selecting the right set of libraries in a mixed
gcc 2/gcc 4 environment.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24534 a95241bf-73f2-0310-859d-f6bbb57e9c96

Now supports symbolic links correctly, and no longer loads the same shared
library twice.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@17929 a95241bf-73f2-0310-859d-f6bbb57e9c96

The runtime loader now creates a debug area through which other teams can
access a team's internal loader data.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@13697 a95241bf-73f2-0310-859d-f6bbb57e9c96