MFC r346216:
ld-elf.so: make LD_DEBUG always functional.

MFC r314833
Convert absolute links to relative links.
Style.Makefile(9) has been ignored to produce minimal diffs.

MFC r314837
The relative symlink fix causes downstream issues for
EMC DELL Isilon so revert the relative symlink fix
pending a better solution.

Reported by: ngie

MFC r315091
Revert r314833 until the problem with INSTALL_RSYMLINKS can be found
as it appears to break arm release builds.

PR: 217705
Reported by: cyclaero@gmail.com

Approved by: grehan (mentor)

MFC r308688:
Assert that there is no unresolved symbols during rtld linking.

MFC r308687:
Update hint to utilize user variable.

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Fix race for incrementally rebuilding VERSION_MAP.

The dependency is needed in PROG_FULL since only the build of PROG_FULL
is using the LDFLAGS and depending on VERSION_MAP. This was not a problem
with MK_DEBUG_FILES==no since it only builds PROG.

This should probably be using bsd.lib.mk instead [1]

Reported by: swills, gjb
Reviewed by: emaste
Noted by: rgrimes [1]
Sponsored by: EMC / Isilon Storage Division
Approved by: re (kib)

Merge the projects/release-pkg branch to head.

This allows packaging the base system with pkg(8), including
but not limited to providing the ability to provide upstream
binary update possibilities for non-tier-1 architectures.

This merge is a requirement of the 11.0-RELEASE, and as such,
thank you to everyone that has tested the project branch.

Documentation in build(7) etc. is still somewhat sparse, but
updates to those parts will follow.

Sponsored by: The FreeBSD Foundation

WITHOUT_TOOLCHAIN: Fix build of rtld.

MK_TOOLCHAIN==no disables building and installing of pic archives.
c_pic.a is still needed for rtld though so force it to build in lib/libc
and link directly to the objdir version of it for rtld.

Somehow this has been broken since r148725.

Sponsored by: EMC / Isilon Storage Division

Follow-on to r291666: use -ffreestanding instead of -fno-builtin.

Requested by: kib

Fix build on GCC 5.2 where, at least on PPC64, the compiler would "optimize"
the malloc() + memset() in the local implementation of calloc() into a call
to calloc(), helpfully turning it into an infinite loop. Clean up some
unneeded flags on PPC64 while here.

MFC after: 1 month

Fix build after r291620.

"don't know how to make /Versions.def. Stop"

This was trying to define a target in bsd.symver.mk based on LIBCDIR which was
not yet defined. Switching the order of inclusion of bsd.prog.mk and
bsd.symver.mk fixes it and seems fine.

Pointyhat to: bdrewery
Sponsored by: EMC / Isilon Storage Division

Don't override LIB*DIR variables from src.libnames.mk.

In some cases switch to the LIB*SRCDIR value.

These recently were defined in r291327 and r291619.

Sponsored by: EMC / Isilon Storage Division

Use LIBEXECDIR for /usr/libexec.

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

META_MODE: No need to fix the link in this case.

The exists(${DESTDIR}...) check runs with DESTDIR being blank. When the
target runs it does have DESTDIR=${STAGE_OBJTOP} via bsd.sys.mk. This
results in the first execution warning that the symlink is missing. The
second run does run fine. However, this chflags is not needed at all
for META_MODE/STAGING since we never had this path being a schg file
while using META_MODE.

Sponsored by: EMC / Isilon Storage Division

Change compiler setting to make default visibility of the symbols for
rtld on x86 to be hidden. This is a micro-optimization, which allows
intrinsic references inside rtld to be handled without indirection
through PLT. The visibility of rtld symbols for other objects in the
symbol namespace is controlled by a version script.

Reviewed by: kan, jilles
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks

Convert to LIBADD
Reduce overlinking

Remove MK_ARM_EABI, the armeb issues have been fixed. The code to support
the oabi is still in the tree, but it is expected this will be removed
as developers work on surrounding code.

With this commit the ARM EABI is the only supported supported ABI by
FreeBSD on ARMa 32-bit processors.

X-MFC after: never
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D876

Document the whole settings needed to build a debug version of rtld.

Sponsored by: The FreeBSD Foundation
MFC after: 3 days

Revert r267233 for now. PIE support needs to be reworked.

1. 50+% of NO_PIE use is fixed by adding -fPIC to INTERNALLIB and other
build-only utility libraries.
2. Another 40% is fixed by generating _pic.a variants of various libraries.
3. Some of the NO_PIE use is a bit absurd as it is disabling PIE (and ASLR)
where it never would work anyhow, such as csu or loader. This suggests
there may be better ways of adding support to the tree. Many of these
cases can be fixed such that -fPIE will work but there is really no
reason to have it in those cases.
4. Some of the uses are working around hacks done to some Makefiles that are
really building libraries but have been using bsd.prog.mk because the code
is cleaner. Had they been using bsd.lib.mk then NO_PIE would not have
been needed.

We likely do want to enable PIE by default (opt-out) for non-tree consumers
(such as ports). For in-tree though we probably want to only enable PIE
(opt-in) for common attack targets such as remote service daemons and setuid
utilities. This is also a great performance compromise since ASLR is expected
to reduce performance. As such it does not make sense to enable it in all
utilities such as ls(1) that have little benefit to having it enabled.

Reported by: kib

Test RTLD's new LD_LIBRARY_PATH_FDS variable.

Test LD_LIBRARY_PATH_FDS by linking a binary that requires a shared
library that isn't in any of the usual search paths. Ensure this fails
when we don't supply LD_LIBRARY_PATH_FDS or we pass invalid information
in it. Ensure it works when we pass the correct directory in various
places in the variable.

Approved by: rwatson (mentor)
MFC after: 3 weeks
Sponsored by: DARPA/AFRL

In preparation for ASLR [1] support add WITH_PIE to support building with -fPIE.

This is currently an opt-in build flag. Once ASLR support is ready and stable
it should changed to opt-out and be enabled by default along with ASLR.

Each application Makefile uses opt-out to ensure that ASLR will be enabled by
default in new directories when the system is compiled with PIE/ASLR. [2]

Mark known build failures as NO_PIE for now.

The only known runtime failure was rtld.

[1] http://www.bsdcan.org/2014/schedule/events/452.en.html
Submitted by: Shawn Webb <lattera@gmail.com>
Discussed between: des@ and Shawn Webb [2]

Use src.opts.mk in preference to bsd.own.mk except where we need stuff
from the latter.

Replace LIBGCC by LIBCOMPILER_RT.

We now use libcompiler_rt on all platforms now. Instead of referring
directly to -lgcc and LIBGCC, use -lcompiler_rt and LIBCOMPILER_RT.

When building for ARM EABI link against libgcc for the __aeabi_* functions.

Use absolute path for /usr/libexec/ld-elf.so.1 symlink.

Requested by: kan, kib

Use -h flags for chflags, so we won't remove 'schg' flag from system's
/libexec/ld-elf.so.1.

MFC after: 2 weeks

Do not change entire BINDIR, it might be needed later, just change
symlink target.

MFC after: 2 weeks

Change /usr/libexec/ld-elf.so.1 to point at ../../libexec/ld-elf.so.1
instead of /libexec/ld-elf.so.1. Below in the Makefile we execute
'chflags noschg ${DESTDIR}/usr/libexec/ld-elf.so.1', which follows
symlink and removes 'schg' flag from system's /libexec/ld-elf.so.1
instead of the one in DESTDIR. It is also more friendly to use
replative paths in symlink in case of jail/chroot environments.

Obtained from: WHEEL Systems
MFC after: 2 weeks

Remove libssp_nonshared from the rtld linking set. The only use for the
library was definition for the weak alias of __stack_chk_fail.

No objections from: kan
MFC after: 2 weeks

Centralize the calculation of the top source directory. This
simplifies the build of rtld with partial checkout, allowing to
override only one place to reference other tree.

Submitted by: bde
MFC after: 2 weeks

Add support for preinit, init and fini arrays. Some ABIs, in
particular on ARM, do require working init arrays.

Traditional FreeBSD crt1 calls _init and _fini of the binary, instead
of allowing runtime linker to arrange the calls. This was probably
done to have the same crt code serve both statically and dynamically
linked binaries. Since ABI mandates that first is called preinit
array functions, then init, and then init array functions, the init
have to be called from rtld now.

To provide binary compatibility to old FreeBSD crt1, which calls _init
itself, rtld only calls intializers and finalizers for main binary if
binary has a note indicating that new crt was used for linking. Add
parsing of ELF notes to rtld, and cache p_osrel value since we parsed
it anyway.

The patch is inspired by init_array support for DragonflyBSD, written
by John Marino.

Reviewed by: kan
Tested by: andrew (arm, previous version), flo (sparc64, previous version)
MFC after: 3 weeks

Rtld links with the specially built pic static libc library to get some
C runtime services, like printf(). Unfortunately, the multithread-safeness
measures in the libc do not work in rtld environment.

Rip the kernel printf() implementation and use it in the rtld instead of
libc version. This printf does not require any shared global data and thus
is mt-safe. Systematically use rtld_printf() and related functions, remove
the calls to err(3).

Note that stdio is still pulled from libc due to libmap implementaion using
fopen(). This is safe but unoptimal, and can be changed later.

Reported and tested by: pgj
Diagnosed and reviewed by: kan (previous version)
Approved by: re (bz)

Compile RTLD with global dot symbols on 64-bit PowerPC, as a crutch for
GDB's ability to locate r_debug_state (which is actually the only function
that need be compiled this way).

Add a hook to pass debug flags to the build of rtld when doing make in
the rtld directory.

Reviewed by: kan

Use -fPIC to build libexec/rtld-elf on sparc64, so it will also be able
to link with newer binutils, without overflowing the GOT.

Obtained from: projects/binutils-2.17

Unbreak ia64.

With r169630 I disabled symbol versioning because it broke rtld. With
r211706 rtld got broken for ia64 & powerpc64. It was fixed for powerpc64
with r212497. In between, r211749 removed the exports table because the
version script handled the exports. But wait, symbol versioning was
disabled on ia64.

With exports controlled by the version script and symbol versioning
disabled, all symbols are exported and too many symbols bind to the
definition in rtld. Let's just say that waird things happen.

So, enable symbol versioning on ia64 and apply a work-around for the
SIGSEGV that triggered r169630 to begin with: when rtld relocates
itself, it comes across r_debug_state and for some reason can't find the
definition. This causes a failure, relocation aborts and null pointers
galore. The work-around is to ignore the missing definition when rtld
is relocating itself and keep going.

Maybe with the next binutils this will all go away. Maybe not, in
which case I still need to figure out why r_debug_state cannot be found.

BTW: r_debug_state is in the symbol map -- I don't think any other rtld
symbols that rtld references are in the symbol map...

Make RTLD work on powerpc64 again. If there is a sub-directory named
MACHINE_ARCH, use that specific one, otherwise use MACHINE_CPUARCH.

Reviewed by: imp

MFtbemd:

Prefer MACHNE_CPUARCH to MACHINE_ARCH in most contexts where you want
to test of all the CPUs of a given family conform.

RTLD support for powerpc64. A few small modifications to the Makefile
and symbol map are required to support various consequences of the dot
symbol scheme:

- Symbols beginning with a dot are reserved, so start private symbols with
an underscore.
- In order to set RTLD breakpoints, gdb must be able to locate the text
entry point, not the data section function descriptor, so add
.r_debug_state to the symbol map on powerpc64.

Obtained from: projects/ppc64

Make WARNS=6 the default for libexec/.

Just like bin/ and sbin/, I think setting WARNS to the highest value
possible will make it more attractive for people to fix warnings.

- The WARNS variable is set in the Makefile in the directory of the
application itself, making it more likely that it will be removed out
of curiosity to see what happens.
- New applications will most likely build with WARNS=6 out of the box,
because the author would more likely fix the warnings during
development than lower WARNS.

Unfortunately almost all apps in libexec require a lowered value of
WARNS.

Second attempt at eliminating .text relocations in shared libraries
compiled with stack protector.

Use libssp_nonshared library to pull __stack_chk_fail_local symbol into
each library that needs it instead of pulling it from libc. GCC
generates local calls to this function which result in absolute
relocations put into position-independent code segment, making dynamic
loader do extra work every time given shared library is being relocated
and making affected text pages non-shareable.

Reviewed by: kib
Approved by: re (kib)

Back out previous revision until better tested fix is ready.

Approved by: re (impliciti, by approving previos check-in)

Eliminate .text relocations in shared libraries compiled with stack protector.

Use libssp_nonshared library to pull __stack_chk_fail_local symbol into
each library that needs it instead of pulling it from libc. GCC generates
local calls to this function which result in absolute relocations put into
position-independent code segment, making dynamic loader do extra work everys
time given shared library is being relocated and making affected text pages
non-shareable.

Reviewed by: kib
Approved by: re (kensmith)

Fix build when WITH_SSP is set explicitly.

Submitted by: Jeremie Le Hen

Enable GCC stack protection (aka Propolice) for userland:
- It is opt-out for now so as to give it maximum testing, but it may be
turned opt-in for stable branches depending on the consensus. You
can turn it off with WITHOUT_SSP.
- WITHOUT_SSP was previously used to disable the build of GNU libssp.
It is harmless to steal the knob as SSP symbols have been provided
by libc for a long time, GNU libssp should not have been much used.
- SSP is disabled in a few corners such as system bootstrap programs
(sys/boot), process bootstrap code (rtld, csu) and SSP symbols themselves.
- It should be safe to use -fstack-protector-all to build world, however
libc will be automatically downgraded to -fstack-protector because it
breaks rtld otherwise.
- This option is unavailable on ia64.

Enable GCC stack protection (aka Propolice) for kernel:
- It is opt-out for now so as to give it maximum testing.
- Do not compile your kernel with -fstack-protector-all, it won't work.

Submitted by: Jeremie Le Hen <jeremie@le-hen.org>

Don't enable symbol versioning on ia64 for now. It causes
symbol lookup failures that later result in null-pointer
dereferences. This needs looking into, but since we're
close to release it's possible that it's not resolved before
that time.

Enable symbol versioning by default. Use WITHOUT_SYMVER to disable it.
Warning, after symbol versioning is enabled, going back is not easy
(use WITHOUT_SYMVER at your own risk).

Change the default thread library to libthr.

There most likely still needs to be a version bump for at least the
thread libraries. If necessary, this will happen later.

Retire rtld-specific Versions.def. Symbols exported by rtld are supposed
to override weak symbols exported by libc, so by definition these two
are using the same symbol version names.

Reflect the reality by referring to libc's Versions.def directly.

Bring rtld exports in line with corresponding symbols exported from
libc.

Disable SYMVER_DEFAULT n rtld until its implications are understood
better.

Prepare rtld for symbol versioning. Disable it by default for now.

So do it like we do in usr.bin/tip/tip/Makefile. ;)

Revert previous commit. As ru explains:

In the old world (as the surrounding comment in makefile says), there
was the /usr/libexec/ld-elf.so.1 binary which is now a symlink to
/libexec/ld-elf.so.1. To symlink, we need to make sure that the
_target_ (and the target is /usr/libexec/ld-elf.so.1) doesn't have
"schg" flag set. A real solution is to protect the chflags call only if
target exists, like we do in usr.bin/tip/tip/Makefile.

Requested by: ru

Avoid an (ignored) error by invoking chflags on the link target, not the
symlink.

PR: kern/73016
Submitted by: John E. Hein <jhein@timing.com>
MFC after: 1 week

Introduce the PRECIOUSPROG knob in bsd.prog.mk, similar
to PRECIOUSLIB from bsd.lib.mk. The side effect of this
is making installing the world under jail(8) possible by
using another knob, NOFSCHG.

Reviewed by: oliver

For variables that are only checked with defined(), don't provide
any fake value.

This comment should have been removed in the previous commit.

Spotted out by: marcus, simon

Woohoo !
the latest binutils import mades this gross hack useless, so just remove it.

Work around a problem somewhere with binutils (?) on arm, hopefully without
breaking any other arch this time.

Fix breakage caused by alphabetically sorting SRCS: rtld_start.S must come first!
The previous version made all shared binaries dump core.

Import arm bits for rtld-elf.

Obtained from: NetBSD

Add initial support for compiling a special 32 bit version of
ld-elf.so.1 on 64 bit systems. Most of this involves using alternate
paths, environment variables and diagnostic messages.

The build glue is seperate.

Retire the WITH_LIBMAP compile knob; libmap is now a standard feature.

Forgot one instance of ld-elf.so.1. Convert to ${PROG}

Pointed out by: obrien

Don't forget to honor DESTDIR. Also switch over to using PROG instead of
the binary name directly.

Don't forget to chflags noschg the existing binary so we can symlink
over it safely.

Pointed out by: yosimoto@waishi.jp

As long threatened, stage 2 of making a dynamically-linked root a reality.
Install rtld into /libexec.

Set CSTD to gnu99. We can only use on of the gnu?9 C languages.
We can't use c89 due to use of 'inline', and c99 produces bad code.

Allow threading libraries to register their own locking
implementation in case default one provided by rtld is
not suitable.

Consolidate various identical MD lock implementation into
a single file using appropriate machine/atomic.h.

Approved by: re (scottl)

Dynamic object dependency mapping: libmap.

This is an optional feature, disabled by default.

This will be useful to people testing the various POSIX threading
libraries under -CURRENT but can easily serve other needs.

Advertize rtld(1) as ld.so(1) in manual pages world

Dillon's recent commits to the dynamic linker without running them
by me first have given me a good excuse to drop my MAINTAINERship.

MFC after: 1 week

Use new backup feature of install(1).

- Backout botched attempt to intoduce MANSECT feature.
- MAN[1-9] -> MAN.

We shouldn't use cp to save the old ld-elf.so.1. Use the sanctioned tool
${INSTALL} with -C -p instead.

When installing the dynamic linker, save the previous version in
"ld-elf.so.1.old". The dynamic linker is a critical component of
the system, and it is difficult to recover if it is damaged and
there isn't a working backup available. For instance, parts of
the toolchain such as the assembler are dynamically linked, making
it impossible to build a new dynamic linker if the installed one
doesn't work.

Fixed missing DPADDs.

Fixed some style bugs (some usual ones for LDADD, and misformatting of
$FreeBSD$).

Add a manual page for the ELF dynamic linker. I initially created
rtld.1 by means of a repository copy from "src/libexec/rtld-aout/rtld.1".
Then I edited it to make it (more) accurate for the ELF dynamic
linker.

Add a new function dllockinit() for registering thread locking
functions to be used by the dynamic linker. This can be called by
threads packages at start-up time. I will add the call to libc_r
soon.

Also add a default locking method that is used up until dllockinit()
is called. The default method works by blocking SIGVTALRM, SIGPROF,
and SIGALRM in critical sections. It is based on the observation
that most user-space threads packages implement thread preemption
with one of these signals (usually SIGVTALRM).

The dynamic linker has never been reentrant, but it became less
reentrant in revision 1.34 of "src/libexec/rtld-elf/rtld.c".
Starting with that revision, multiple threads each doing lazy
binding could interfere with each other. The usual symptom was
that a symbol was falsely reported as undefined at start-up time.
It was rare but not unseen. This commit fixes it.

$Id$ -> $FreeBSD$

Add a MAINTAINER line naming myself. We control the vertical. We
control the horizontal.

Eliminate all machine-dependent code from the main source body and
the Makefile, and move it down into the architecture-specific
subdirectories.

Eliminate an asm() statement for the i386.

Make the dynamic linker work if it is built as an executable instead
of as a shared library. See i386/Makefile.inc to find out how to
do it. Note, this change is not enabled and it might never be
enabled. But it might be useful in the future. Building the
dynamic linker as an executable should make it start up faster,
because it won't have any relocations. But in practice I suspect
the difference is negligible.

- Set the system immutable flag when installing ld.so to avoid people
accidentally clobbering it.

Submitted by: numberous people on -current

Add alpha support.

Submitted by: John Birrell <jb@cimlogic.com.au> (with extra hacks by me)
Obtained from: Probably NetBSD

Add "-C" to INSTALLFLAGS to install atomically. An elf->elf
installworld dies at this point otherwise, leaving the system
without a dynamic linker.

Add GDB support. The method and some of the code came from NetBSD's elf
runtime linker.

This commit was generated by cvs2svn to compensate for changes in r34192,
which included commits to RCS files with non-trunk default branches.

Import the ELF dynamic linker. This is the ElfKit version with
quite a few enhancements and bug fixes. There are still some known
deficiencies, but it should be adequate to get us started with ELF.

Submitted by: John Polstra <jdp@polstra.com>