rtld-elf: Use a proper struct type for tlsdesc entries

This clarifies the code and makes it less error-prone. It also makes it
easier to extend downstream in CheriBSD (where pointer and integer
members no longer have the same representation and an additional member
is present).

Reviewed by: jhb, kib
Differential Revision: https://reviews.freebsd.org/D45143

rtld: Add support for arm64 variant pcs

The aarch64 ELF spec has support for a variant of the normal procedure
call standard that doesn't follow the normal register convention, e.g.
using more registers as arguments, or different register state is
preserved.

Add support to rtld to handle this. As we don't know which registers
need to be preserved disable lazy binding for these functions.

Reviewed by: kib
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D44869

Support BTI in rtld

Read the elf note to decide when to set the guard page on arm64.

Reviewed by: kib
Sponsored by: Arm Ltd
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39452

libexec: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

rtld: introduce STATIC_TLS_EXTRA

The new STATIC_TLS_EXTRA variable provides a means for applications
to increases the size of the extra static TLS space allocated by
rtld beyond the default of '128'. This extra static TLS space is used
for objects loaded with dlopen.

The value specified in the variable must be no less than the default
value and no greater than the maximum allowed value for size_t type.

If an invalid value is specified, rtld will ignore it and just use
the default value.

The rtld(1) man page is updated to document this new option.

Obtained from: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D42025

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

rtld: rename tls_done to tls_static

The meaning of the flag is that static TLS allocation was done.

Taken from NetBSD Joerg Sonnenberger change for src/libexec/ld.elf_so/tls.c
rev. 1.18.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

TLS: Use <machine/tls.h> for libc and rtld.

- Include <machine/tls.h> in MD rtld_machdep.h headers.

- Remove local definitions of TLS_* constants from rtld_machdep.h
headers and libc using the values from <machine/tls.h> instead.

- Use _tcb_set() instead of inlined versions in MD
allocate_initial_tls() routines in rtld. The one exception is amd64
whose _tcb_set() invokes the amd64_set_fsbase ifunc. rtld cannot
use ifuncs, so amd64 inlines the logic to optionally write to fsbase
directly.

- Use _tcb_set() instead of _set_tp() in libc.

- Use '&_tcb_get()->tcb_dtv' instead of _get_tp() in both rtld and libc.
This permits removing _get_tp.c from rtld.

- Use TLS_TCB_SIZE and TLS_TCB_ALIGN with allocate_tls() in MD
allocate_initial_tls() routines in rtld.

Reviewed by: kib, jrtc27 (earlier version)
Differential Revision: https://reviews.freebsd.org/D33353

rtld-elf: Use _get_tp in __tls_get_addr for aarch64 and riscv64.

Reviewed by: kib
Sponsored by: The University of Cambridge, Google Inc.
Differential Revision: https://reviews.freebsd.org/D33047

rtld/arm64: Remove checks for undefined symbols when processing TPREL64

lld emits several GOT relocations referencing the null sumbol in libc.so
when compiled with -ftls-model=initial-exec. This symbol is specified
to be undefined.

We generally do not handle dynamic TLS relocations against weak,
undefined symbols, so avoid printing a warning here. This makes it
possible to compile libc.so using the initial-exec TLS model on arm64.

Reviewed by: jrtc27, kib
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31069

rtld-elf(1): remove obsolete pre_init() hook

It's no longer used since 600ee699ed2805894f5972c6ac2c3d17dca7f6ce
and r358358 respectively.

Handle non-plt IRELATIVE relocations, at least for x86.

lld 10.0 seems to generate this relocation for rdtsc_mb() ifunc in our libc.

Reported, reviewed, and tested by: dim (amd64, previous version)
Discussed with: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D23652

Improve R_AARCH64_TLSDESC relocation.
The original code did not support dynamically loaded libraries and used
suboptimal access to TLS variables.
New implementation removes lazy resolving of TLS relocation - due to flaw
in TLSDESC design is impossible to switch resolver function at runtime
without expensive locking.

Due to this, 3 specialized resolvers are implemented:
- load time resolver for TLS relocation from libraries loaded with main
executable (thus with known TLS offset).
- resolver for undefined thread weak symbols.
- slower lazy resolver for dynamically loaded libraries with fast path for
already resolved symbols.

PR: 228892, 232149, 233204, 232311
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D18417

Implement R_AARCH64_TLS_DTPMOD64 and A_AARCH64_TLS_DTPREL64 relocations.
Although these are slightly obsolete in favor of R_AARCH64_TLSDESC,
gcc -mtls-dialect=trad still use them.

Please note that definition of TLS_DTPMOD64 and TLS_DTPREL64 are incorrectly
exchanged in GNU binutils. TLS_DTPREL64 should be encoded to 1028 (as is
defined in ARM ELF ABI) but binutils encode it to 1029. And vice versa,
TLS_DTPMOD64 should be encoded to 1029 but binutils encode it to 1028.

While I'm in, add also R_AARCH64_NONE. It can be produced as result of linker
relaxation.

MFC after: 1 week

Tidy up arm64 reloc_jmpslots() implementation.
- don't relocate jump slots multiple times (if LD_BIND_NOW is defined).
- process only R_AARCH64_JUMP_SLOT here, other relocation types are handled
by reloc_plt().

MFC after: 1 week

Implement arm64 version of __tls_get_addr().

MFC after: 1 week

Fix style(9).
Not a functional change.

MFC after: 1 week

rtld-elf: compile with WANRS=4 warnings other than -Wcast-align

Reviewed By: kib
Approved By: brooks (mentor)
Differential Revision: https://reviews.freebsd.org/D17153

Add STT_GNU_IFUNC and R_AARCH64_IRELATIVE support on arm64.

This is based on the amd64 implementation. Support for both PLT and
non-PLT (e.g. a global variable initilised with a pointer to an ifunc)
cases are supported.

We don't pass anything to the resolver as it is expected they will read
the ID registers directly, with the number of registers with CPU info
likely to increase in the future.

Reviewed by: kib
Approved by: re (gjb)
Differential Revision: https://reviews.freebsd.org/D17341

o Let rtld(1) set up psABI user trap handlers prior to executing the
objects' init functions instead of doing the setup via a constructor
in libc as the init functions may already depend on these handlers
to be in place. This gets us rid of:
- the undefined order in which libc constructors as __guard_setup()
and jemalloc_constructor() are executed WRT __sparc_utrap_setup(),
- the requirement to link libc last so __sparc_utrap_setup() gets
called prior to constructors in other libraries (see r122883).
For static binaries, crt1.o still sets up the user trap handlers.
o Move misplaced prototypes for MD functions in to the MD prototype
section of rtld.h.
o Sprinkle nitems().

Implement LD_BIND_NOT knob for rtld.

From the manpage:
When set to a nonempty string, prevents modifications of the PLT slots
when doing bindings. As result, each call of the PLT-resolved
function is resolved. In combination with debug output, this provides
complete account of all bind actions at runtime.

Same feature exists on Linux and Solaris.

Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks

Pull the R_AARCH64_TLSDESC code out into a common function and use them in
both the plt and non-plt case.

This fixes an issue where libraries built with LLD can fail with
"Unhandled relocation 1031"

PR: 214971
Obtained from: 1 week
Sponsored by: DARPA, AFRL

Adjust r308689 to make rtld compilable with either in-tree or
(hopefully) stock gcc 4.2.1 on i386 and other arches.

In particular:
- Do not use %ebx in the asm constraints on i386, since rtld is
compiled with -fPIC and gcc cannot handle GOT-base register reload
(clang and newer gcc can).
- Avoid direct use of [static N] construct in the function
declaration/definion. In-tree gcc was patched to support this, but
stock 4.2.1 cannot handle the feature.

Requested by: bde
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Pass CPUID[1] %edx (cpu_feature), %ecx (cpu_feature2) and
CPUID[7].%ebx (cpu_stdext_feature), %ecx (cpu_stdext_feature2) to the
ifunc resolvers on x86.

It is much more clean to use CPUID instruction in usermode to retrieve
this information than to pass AT_HWCAP aux vector from kernel, on
x86. Still, the change does allow for use of AT_HWCAP on arches where it is
needed, by passing aux array to ifunc_init() initializer which should
prepare arguments for ifunc resolvers.

Current signature for resolvers on x86 is
func_t iresolve(uint32_t cpu_feature, uint32_t cpu_feature2,
uint32_t cpu_stdext_feature, uint32_t cpu_stdext_feature2);
where arguments have identical meaning as the kernel variables of the
same name. The ABIs allow to use resolvers with the void or shortened
list of arguments.

Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D8448

Do not call callbacks for dl_iterate_phdr(3) with the rtld bind and
phdr locks locked. This allows to call rtld services from the
callback, which is only reasonable for dlopen(path, RTLD_NOLOAD) to
test existence of the library in the image, and for dlsym(). The
later might still be not quite safe, due to the lazy resolution of
filters.

To allow dropping the locks around iteration in dl_iterate_phdr(3), we
insert markers to track current position between relocks. The global
objects list is converted to tailq and all iterators skip markers,
globallist_next() and globallist_curr() helpers are added.

Reported and tested by: davide
Reviewed by: kan
Sponsored by: The FreeBSD Foundation
MFC after: 3 weeks

Fix how we place each objects thread local data. The code used was based
on the Variant II code, however arm64 uses Variant I. The former placed the
thread pointer after the data, pointing at the thread control block, while
the latter places these before said data.

Because of this we need to use the size of the previous entry to calculate
where to place the current entry. We also need to reserve 16 bytes at the
start for the thread control block.

This also fixes the value of TLS_TCB_SIZE to be correct. This is the size
of two unsigned longs, i.e. 2 * 8 bytes.

While here remove the bogus adjustment of the pointer in the
R_AARCH64_TLS_TPREL64 case. It should be the offset of the data relative
to the thread pointer, including the thread control block.

Sponsored by: ABT Systems Ltd

Add on the addend when in the R_AARCH64_ABS64 and R_AARCH64_GLOB_DAT cases.
This fixes at least sshd, and some of the boehm-gc tests.

Sponsored by: ABT Systems Ltd

Use the correct value to get the offset of the objects tls data.

Sponsored by: The FreeBSD Foundation

Add support for thread local storage on arm64 to the runtime linker. The
ABI specifies that, for R_AARCH64_TLSDESC relocations, we use the symbol
value, addend, and object tls offset to calculate the offset from the tls
base. We then cache this value for future reference.

Differential Revision: https://reviews.freebsd.org/D2183
Reviewed by: kib
Sponsored by: The FreeBSD Foundation

Correct copyright typo

Add the arm64 code to the runtime linker. It's not able to be built as we
still need libc_pic for a few things, but this is expected to be ready
soon.

Differential Revision: https://reviews.freebsd.org/D2136
Reviewed by: kib
Sponsored by: The FreeBSD Foundation