[musl][ldso] Fix for undefined behavior on pointer arithmetic

Test: Manually

Change-Id: I8a94ba1c9e0fdb1fd97edcf3225aa87552d4f83f

[ulib][musl][ldso] Work around compiler-generated memset PLT calls

Early in the dynamic linker it's not safe to use exported symbols
in calls since those go via PLT. When the compiler generates a
call to memset, it will be via the PLT unless some symbol renaming
shenanigans are done to avoid it.

Test: CQ & manually tested arm64 LTO build
Change-Id: Ie9387bd457d7af8f53700d9016fed3a626f921c8

[musl][ulib][ldso] Support for DT_PREINIT_ARRAY

This is needed for some compiler runtimes such as XRay.

Test: CQ
Change-Id: I835f395fabd0a35ed81578d771452ec2a23448a3

[musl][ulib][ldso] Remove unused declarations

These declarations are no longer used and can be removed.

Test: CQ
Change-Id: Ibf41563749aa07528785fe5068c69ab3af585c35

[syscalls] Change vmar_... calls

- Create zx_vm_option_t type
- Change ZX_VM_FLAG_... to ZX_VM_... and make them new type.
- Change int32_t flags parameters to zx_vm_option_t options
- Make the options parameter be second.

- Modify vmar_..._old functions to keep other repos working.
Once they've all been updated these functions will go away.

ZX-2264
Test:CQ

Change-Id: I1faf4cc4e1e4dabf3d8ef680e398d83a545b0f09

[musl][ldso] Support DT_RELR

The new DT_RELR format makes the most common type of relocation
entry much more compact. LLD supports this with a new flag.

Test: CQ
Change-Id: I65a6e6aa2579b2e6e9640b434ad722a53cf04d54

[musl] Fix alignment issues for tls on AArch64

musl had been assuming that alignment of TLS segments would
always be <= 16. This change removes use of TPOFF_K and
ensures the the offset of every tls segment is calculated
with the gap in mind. This change ensures that dso.tls.offset
is always the offset from the thread pointer itself rather
than attempting to be from the start of the tls block. This
is consistent (though backwards) with how things work on
x86-64.

Fixes ZX-1646

Test: boot/test/core/elf-tls-test
runtests in garnet
fixed nacl libc-tests in garnet

* elf-tls-test only tests the simplist access model
* it isn't clear runtests tests tls at all
* the fixed nacl libc-tests confirms that all initial-exec tests work

Change-Id: Ida3ff39a25adc7a1b5c85af7e643a44a4f748766

[debuglog] Migrate to zx_debuglog_create

Test: cq

Change-Id: Ia7aa2efefd0c1ac0ce3460159a9405d6f0b1777e

[loader] Allow break on loader initialization.

The loader sets the process property ZX_PROP_PROCESS_DEBUG_ADDR, when it
initializes. This can be used for debuggers to get the list of loaded
libraries and to set breakpoints on library loads.

But since this is set programatically by the process code itself, the
debugger has no way to know when the query this property. If it does not
get queried before user code runs, symbolic breakpoints will not work.

This patch adds a constant that the debugger can set this property to
before execution starts. The loader checks for this and if set, issues a
manual debug breakpoint after the value is set.

A debugger can then catch this breakpoint, complete setup with the
correct debug address, and continue program execution.

TEST=manual

Change-Id: I63fc824531d1f7b09029ea01be5a93e1e9abddbb

[syscalls] Remove read_status from zx_channel_call

This parameter no longer has a purpose. Also, remove ZX_ERR_CALL_FAILED.
This status code no longer has a purpose.

Test: No behavior change
Change-Id: I71945ea9d1d6d591a20476106df1e598dfaf97b1

[syscalls] zx_channel_call should always consume handles

Previously, whether zx_channel_call consumed handles depended on whether
it succeeded or failed.

ZX-2204 #comment

Test: utest/core/channel

Change-Id: Ic7b1149a85a1a892144479ee2f543fc7fedc6114

[logging] Migrate zircon off zx_log_read and zx_log_write

zx_log_create is trickier. It's essentially a version of
zx_debuglog_create that allows not passing a root resource.

ZX-2184

Change-Id: Ida3d6bacce2c4601a4c6a00fd0cf13c902eb1d41

[processargs] Rename PA_SVC_LOADER to PA_LDSVC_LOADER

We ended up calling the loader service LDSVC in FIDL. This change aligns
the processargs name with the FIDL name.

Change-Id: I5cb2332f047676c2e49fa5b20719aa841185cf19

[ulib][musl][ldso] Split log lines at \n

TC-21

Change-Id: I2c35df11b487e9de464f7426dc1f614a30ce7fd8

[ulib][musl][ldso] Avoid : in synthesized names for VMO objects

The ':' character is reserved as a field separator in the symbolizer
markup spec, so it doesn't work well in module names.

TC-21

Change-Id: Ib425ce6b151b7233f1e4bc5ec41fd990ad83ab67

[musl][ldso][sanitizer] Emit a {{{reset}} before the first {{{module}}}

This is in the spec to tell the symbolizer that it's seeing a fresh
process even if the same PID appeared in the log stream before.

TC-21 #comment Zircon dynamic linker emits {{{reset}}}

Change-Id: If7e57fd2347477b519c3501af142d2c3f924b043

[ulib][musl][ldso] Fix infinite recursion in dlopen for circular dependencies

Circular DT_NEEDED dependencies between shared libraries work fine
in general, though they are impossible to construct with the normal
ways of building libraries. Make them work in the dlopen case too.

ZX-2090 #comment Don't crash on circular DT_NEEDED dependencies in dlopen.

Change-Id: Iffe312c7ba621ddd8fe389b84ee8f56fa5cd0a13

[ldso][ldmsg] Remove DebugPrint from ldsvc; always use zx_log_write for now

We were already using zx_log_write for __sanitizer_log_write and it's
the only thing at hand that does good PID/thread/timestamp tagging.
The symbolizer needs the contextual elements emitted by dynlink.c and
the sanitizer logging to go the same route so they get consistent PID
tagging. So just use zx_log_write for all of it.

This should be replaced ASAP with a non-zx_log_write method for most
processes. Then _dl_log_write will again use zx_log_write only if the
process received PA_FDIO_LOGGER, as before.

ZX-2072 #done

Change-Id: I56e0290d92cf2a7efcc58a065dd61dc001b178a5

[ulib][musl][ldso] Don't set txid for zx_channel_call

The kernel magically assigns txid values now.

Change-Id: Ie5413be2327a5ebbbf89f8cdc6ca877a97ddc500

[libc] Remove dependence on pthread_attr_t from thread allocation

Instead, just pass the requested guard and stack sizes directly.

Change-Id: I806c010273082da42d36259ef5e713f3fc0262f5

[musl] Make __allocate_thread return a thrd_t

Change-Id: I035db3cc291f0c07a8cfaa402806f0d4e4141d60

[ulib][musl][ldso] Emit sanitizer_markup.md format for loaded modules

ZX-1894 #done

Change-Id: Ifa8f936775db8281c5cc868704fd069769d4a7da

[ulib][musl][ldso] Reject loading a file with PT_GNU_STACK PF_X

If an ELF file is marked with the GNU mechanism to say it requires
an executable stack, then refuse to load it.

ZX-1625 #done

Change-Id: I760c5eab390a21b0d02ee8f33bce97d68dae96ee

[libc] Remove LD_PRELOAD

Change-Id: I557160987e47655ac36d1dba9ae2227f14403118

[syscall][vmo] removal of zx_vmo_{read,write}() actual parameter 10/n

Remove remaining usage of _old syscalls in Zircon, in preparation for
deleting them.

ZX-1802

Change-Id: If518f15ab2b411295d7ad1061f50525f99ef4952

[syscall][vmo] removal of zx_vmo_{read,write}() actual parameter 1/n

In preparation for changing the zx_vmo_read() and write() syscalls,
introduce aliases (zx_vmo_read_old() and write_old()) for the existing
behaviour, migrate zircon callers to the aliases, and mark the original
syscalls as deprecated.

Once the garnet and above layers have been migrated, the original
syscalls will be adjusted to remove the "actual" argument, and the
temporary _old() versions will be deprecated, and then removed.

ZX-1802

Change-Id: I2fab9a59caf4093fb585ada6015197017767aba2

[libc] Rename pthread_impl.h to threads_impl.h

Baby step towards separating pthreads out of libc

Change-Id: Ibb1ba6656c796218909f4a3451db9bb66b99fb80

[ulib][musl][ldso] Switch to "runtime mode" only after thread descriptor setup

It's not safe to throw errors to be caught by dlerror until the
thread descriptor has been set up.

ZX-1596 #done

Change-Id: Ib4dbc9271b5e7661941dc9486fec52c4ced9047d

[ulib][musl] Remove obsolete TODO comment

The VMO interface has changed in the opposite direction,
so this TODO comment no longer makes any sense.

ZX-698

Change-Id: I809c5960e625f71345c21a6038c5b44695e3985f

[ldmsg] Convert the loader service to FIDL wire format

Previously, the loader service used a hand-rolled protocol. Now the
protocol is specified in //zircon/system/public/zircon/loader.fidl.

Because this protocol is used by such low-level components (e.g.,
userboot and libc), we do not use generated code to implement the
protocol. Instead, we use a stand-alone library, ldmsg, which manually
encodes and decodes messages according to the wire format specified in
loader.fidl.

This CL also moves fidl/types.h into zircon/fidl.h because these type
definitions are now fundamental to the operation of the system. Putting
their definition in this location makes them easier to reference from
userboot and libc.

Change-Id: I9a7619b09b0ff543b8883ac53852ade9d5c033b6

Revert "[ldmsg] Convert the loader service to FIDL wire format"

This reverts commit 3ff2a3f316904d6acf709169fb12ee059231d5d9.

Additional review comments need to be addressed.

Original change's description:
> [ldmsg] Convert the loader service to FIDL wire format
>
> Previously, the loader service used a hand-rolled protocol. Now the
> protocol is specified in //zircon/system/public/zircon/loader.fidl.
>
> Because this protocol is used by such low-level components (e.g.,
> userboot and libc), we do not use generated code to implement the
> protocol. Instead, we use a stand-alone library, ldmsg, which manually
> encodes and decodes messages according to the wire format specified in
> loader.fidl.
>
> This CL also moves fidl/types.h into zircon/fidl.h because these type
> definitions are now fundamental to the operation of the system. Putting
> their definition in this location makes them easier to reference from
> userboot and libc.
>
> Change-Id: I806455813d3a397a2eff41badadf61c28812e328

TBR=kulakowski@google.com,swetland@google.com,abarth@google.com

Change-Id: I7ac90b53cdc152dc8cd892eb394ec7f641161ee9
No-Presubmit: true
No-Tree-Checks: true
No-Try: true

[ldmsg] Convert the loader service to FIDL wire format

Previously, the loader service used a hand-rolled protocol. Now the
protocol is specified in //zircon/system/public/zircon/loader.fidl.

Because this protocol is used by such low-level components (e.g.,
userboot and libc), we do not use generated code to implement the
protocol. Instead, we use a stand-alone library, ldmsg, which manually
encodes and decodes messages according to the wire format specified in
loader.fidl.

This CL also moves fidl/types.h into zircon/fidl.h because these type
definitions are now fundamental to the operation of the system. Putting
their definition in this location makes them easier to reference from
userboot and libc.

Change-Id: I806455813d3a397a2eff41badadf61c28812e328

[debug] Fix compiler nits so we can build with -Og

All changes were related to uninitialized variables accesses
that are optimized out in other builds.

Change-Id: Ide3d49fb7dcbf8211fa7a3624b7119e837c62019

[musl] Fix buffer-size calculation in dynamic loader.

format_build_id_log() was being passed the library's name including the
terminating null character, but build_id_log_size() expects, and was
being passed, the un-terminated name length.

Change-Id: I952620f9959f700d5eb4719f958134a8a1e41553

[musl][ldso] Fix dladdr when the address matches an object but no symbol

According to the Linux man page, this case should return success but
zero the dli_sname and dli_saddr fields.

Change-Id: I5e9be8e7cd11f79cd3f122f7cbf4eb9c58ffdb02

[musl][ldso] Add NO_ASAN markers on some more functions

These functions used to always get inlined, but now they don't.
So make sure they don't use instrumentation that isn't set up yet.

Change-Id: I70ecea9e845be0c7f6ba507e0eee7b08d2b39da3

[ldso] Remove an unused bit of MIPS plt relocation logic

Change-Id: If10fa39180856e9a847e21fc84b949e853436682

[ldso] Make some macros static functions

Change-Id: Ic444286e42af34218fdd35f72ce980024406a376

[ldso] Use the r_debug and link_map structs in ldso

Change-Id: I3c3cf094fc5823acad7b22aca1c3963c60155719

[ldso] Encapsulate dso next and prev pointers into accessors

Change-Id: I9527cb3a22f1140f9f76a112bbf5fe8091920c5c

[ldso] Encapsulate the first fields of struct dso into a struct

Change-Id: I30fd6eeb6d5a720a8ed619d24fd6556a7fff51d9

[ldso] Line up the names in ldso's struct dso with struct link_map

Change-Id: I959b72d10ed171a5ec5298dc837d989b6e53dd54

[ldso] Line up the names in ldso's struct debug with struct r_debug

Change-Id: I18647c8246f791de53346b5bb8bc77b53c682ea2

[ulib][musl][ldso] Synthesize a name for dlopen_vmo without SONAME

dlopen_vmo should allow a module without a DT_SONAME.
Synthesize a name for the module from the VMO handle.

Change-Id: I83c269ffc329df040ccaca07a894eaf6aee9e1bf

[ldso][loader-service] loader service cloning bugfixes

- dl_clone_loader_service() now spelled correctly
- dl_clone_loader_service() no longer uses the common call
path which depends on static buffers (cloning could interact
badly with dlopen() and friends)
- loader_svc_txid is now atomic
- loader_service_get_default() now attempts to clone the active
loader service (if it exists) before failing back to trying to
connect to the system service or creating an in-process loader
against /

Change-Id: If77e9bb07f276f4e00197820abec805c80e621f5

[zx] Rename bug references from MG-nnn to ZX-nnn

Change-Id: I07b0ee32bd7e8cc08e344896352c156b3b420349

[zx] Magenta -> Zircon

The Great Renaming is here!

Change-Id: I3229bdeb2a3d0e40fb4db6fec8ca7d971fbffb94

[musl][dl] provide dl_clone_loader_service()

If there is an active loader service and it supports obtaining
a new connection, this function will return a new connection.

Change-Id: I58fe4d2450dc656340ed352424ef095ad19227fe

[ulib][musl][ldso] Clean up R_*_RELATIVE reloc handling

Rewrite _dl_start to be more readable and not use a stack array.
It now depends on ld.so being linked with -z combreloc so it can
use optimal code relying on DT_RELACOUNT. Also clear out some
remaining MIPS reloc support and drop the IS_RELATIVE macro.

Change-Id: Ia8e97db9edcd74c6c55351ebbd0fe5f38f0ab317

[musl] Clean up -Wno-parentheses in musl.

Change-Id: I26fed03750841222a400b418cd92f64206f5f397

[musl][dynlink] Update TODO(dje) -> TODO(MG-519)

Change-Id: I30fee6a06b48f361e0636fee76f09851ed6f68af

[ulib][musl][ldso] Fix dlopen and dlsym with indirect DT_NEEDED dependencies

Symbol resolution needs to follow the whole dependency graph, not just
one level deep. This matters for dlopen to resolve references to
indirect dependencies at load time. It also matters for dlsym to find
the named symbol anywhere in the dependency graph of the given object,
not just in the object itself or a direct dependency.

MG-963 # done

Change-Id: I385b36ca53c5014dd3d46563a42930e9d9c87757

Revert "[ulib][musl][ldso] Fix dlopen and dlsym with indirect DT_NEEDED dependencies"

This reverts commit 7cacc9fb9b203055b9aae2f9d607deea77157036.

Reason for revert: <INSERT REASONING HERE>

Original change's description:
> [ulib][musl][ldso] Fix dlopen and dlsym with indirect DT_NEEDED dependencies
>
> Symbol resolution needs to follow the whole dependency graph, not just
> one level deep. This matters for dlopen to resolve references to
> indirect dependencies at load time. It also matters for dlsym to find
> the named symbol anywhere in the dependency graph of the given object,
> not just in the object itself or a direct dependency.
>
> MG-963 # done
>
> Change-Id: I9770e81937050248ed73d1185311246cc9b4ac60

TBR=kulakowski@google.com,mcgrathr@google.com

Change-Id: I7076bca50396c0906a5386247ed20ce513e93ff2
No-Presubmit: true
No-Tree-Checks: true
No-Try: true

[ulib][musl][ldso] Fix dlopen and dlsym with indirect DT_NEEDED dependencies

Symbol resolution needs to follow the whole dependency graph, not just
one level deep. This matters for dlopen to resolve references to
indirect dependencies at load time. It also matters for dlsym to find
the named symbol anywhere in the dependency graph of the given object,
not just in the object itself or a direct dependency.

MG-963 # done

Change-Id: I9770e81937050248ed73d1185311246cc9b4ac60

[ulib][musl][ldso] Log new libraries from dlopen under LD_DEBUG

Change-Id: I26565919fba9a33333e3772ccf16f14237a7ff81

[build] Support SanitizerCoverage userland build with USE_SANCOV=true

Some arcane shenanigans are required to resolve bootstrapping issues
with the coverage instrumentation in the dynamic linker itself.

Change-Id: I3a9d38c31879ba2792d3cf55e4f7116306a5acad

[ulib][ldso][asan] Use LOADER_SVC_OP_CONFIG

MG-71 # Dynamic linker support for loading sanitized shared libraries.

Change-Id: I815a9c238aba5c21a5572dd371aecddbed4c8027

[ulib][musl][ldso] Delay TLS layout until after loading

TLS layout for the main program must always precede TLS layout for
DSOs. Since we load some DSOs before loading the main program
(vDSO, which doesn't have TLS; and sanitizer runtime, which might),
delay the TLS layout to a separate pass after everything has been
loaded at startup.

When dlopen loads a new module, that TLS layout remains immediate.

Change-Id: I7d71d6b7133ed3267a39b39031b1d8de0b5b2259

[ulib][musl][ldso] Fix an instance of causing MX_ERR_BAD_HANDLE during startup

Avoid passing MX_HANDLE_INVALID to mx_handle_close().

This fixes an exception which can be reproduced with:
launch -sH /boot/bin/launch

MG-599

Change-Id: I8adf250f1563c9e9f6fd31efb9e1a28e301b9018

[ulib][musl][ldso] Use static buffer for loader-service messages

The large stack buffer has the danger of producing stack overflow.

Change-Id: I5930621e2f25aff94f3c9b894e7a5b2f324a36ca

[ulib][musl][sanitizers] Log pending DSOs in __sanitizer_log_write

Track which DSO build ID and address lines have been logged already.
Make sure each is logged exactly once in __sanitizer_log_write before
emitting the sanitizer logging text.

Change-Id: Id0eb6698993a902703819c25f943acf53b7bd01c

[ulib][musl][ldso] Pre-format log lines for each DSO

For now, these use crashlogger's "dso: ..." format, since
that's what our symbolizer scripts understand.

Use these to replace the existing formatting for LD_DEBUG.
Later, they will also be used for sanitizer logging.

Change-Id: I9c7bc60be012c62b2b1dd5eaba0450d9fd8fd83d

[ulib][musl][mxio] __sanitizer_get_configuration via loader service

This adds a <magenta/sanitizer.h> interface for getting a read-only
VMO for a debugging configuration file. The libc implementation of
this function works by simply sending the configuration name in an
RPC to the loader service. Currently this is treated on the server
side just like script interepreters, i.e. just read in from an
absolute file name.

Change-Id: I08aa5aa42ebc376a510ae9d5d7b69162a47ec037

[ulib][musl][mxio] __sanitizer_publish_data via loader service

This adds a <magenta/sanitizer.h> interface for publishing a VMO
to a "data sink". This will be used for exporting binary blobs
such as coverage data. The libc implementation of this function
works by simply sending the VMO and data sink name in an RPC to
the loader service. Later this will be implemented on the server
side to do something with the data.

Change-Id: I21a25fe66445a0ef5e26bcd3da9e73d1a0ba448d

[ulib][musl][ldso] Load libc's own DT_NEEDED dependencies

This also reorders the list so that libc appears after its dependencies for
purposes of symbol resolution.

Change-Id: Ibc335b2e7f27f0a86ef522918f9efb027177699c

[ulib][musl][ldso] Remove ldd_mode and associated special cases

The ldd_mode support doesn't actually work and we don't really need it.
Simplify code by removing it. This also removes the special-case kludge
for known library names. We don't need this for lookups of "libc.so" to
work, and we don't care to support lookups of "libpthread.so" et al since
we don't actually have any DSOs by those names and aren't trying to provide
loose compatibility with dlopen("libpthread.so", ...) and the like.

This exposed a bug in the logic for lookups of the "detached" libraries,
i.e. libc itself and the vDSO. That's fixed by the introduction of the
detached_head variable.

Change-Id: Iab51ebaa073dba24cf671987c6d7e88205eee9b2

[ldso] Use the namespaced _mx_syscall names in libc

Change-Id: Idd1ae79e1afe7c94d6c276fdb0ee699f761d0148

[musl][pthread][ldso] Improve default thread names and stack VMO names

When a thread has an explicit name, give its TCB/stack VMO a name
derived from that name but that also includes the thrd_t (aka
pthread_t) value if it fits. Also include the TLS size if that
fits, since it's the piece of that VMO's size calculation that
you can't see explicitly from the mapping sizes (the other two
mappings are exactly the stack size, while the TCB mapping's size
is rounded up from the TLS size plus internal state).

For threads with no explicit name, give both the VMO and the
thread a default name that includes the same information.

Change-Id: Id3b5986a56c3322c8f073a6fff31ca7ecc658255

[musl] Use new magenta errors

Change-Id: I72d62767392cb2ce84e1ed1b35a338ab3d716e4a

[ulib][musl][ldso] Name data/bss VMOs

These names are consistent with what ulib/elfload uses.

Change-Id: Idb6c94001da85651f1b8cde9cda98111de5ff53b

[musl][ldso] Remove a bit of dead code

This check used to be needed for a special FDPIC path that would
preempt the normal symbol lookup.

Change-Id: I5db82f105cb546ae85a07a990f4ed066f567994c

[ulib][musl][ldso][asan] Set up shadow memory as early as possible

In the ASan build, the dynamic linker is responsible for setting
up the shadow memory early in startup so that most of the libc
and dynamic linker code itself can be sanitized.

Change-Id: I4f4b5f1951f247deefa132ed211ad577c31f6e01

[ulib][musl][ldso][asan] Add no_sanitize attribute on early startup path

Annotate functions on the early startup path of the dynamic
linker to disable ASan code generation. This goes on the code
that has to run before the shadow memory is set up. Once vDSO
symbol references are resolved, the dynamic linker can set up the
shadow memory and then the rest of the dynamic linker's own code
can be sanitized and run safely without crashing or calling any
ASan runtime functions (unless it has a bug).

Change-Id: Icbe214b5e51381c0773f71cb05d1674e4b4da1d1

[musl] Remove internal signal handling code

Change-Id: I6e2c61766c41e2fcb21fda7a82aeb2ee52ad4afc

[ulib][musl][ldso] Use normal reference to __dls3

The only reason for the symbol lookup shenanigans about calling
__dls3 was to ensure that none of the relocation work in __dls2
can be reordered so that it might not be safely done before
entering __dls3. atomic_signal_fence ensures the same thing.

Change-Id: Ia544f6e5595bf21507dfb1ca85eb22c63435d5ca

[ulib][musl][ldso] Fix error path for mapping failure

Remove a shadowing 'status' variable that prevented the
proper error code from being returned when _mx_vmar_map fails.

Change-Id: I2726365d8d065686868f04f8e1adc28264c1bb28

[abi][runtime][launchpad] provide name table in process args

This will be used to supply strings for use with namespace
related handles.

Change-Id: Ie0f1a334557b4d1a34ea4d5d06f89e0ad399cba5

[processargs] tidy up naming and numbering

This cleans stuff up ahead of the addition of string tables for
the upcoming PA_NS_DIR and PA_NS_REQ types and the eventual
flattening of PA_MXIO_* to a single type, using subtypes for
protocol.

- Align names with current-day usage
- Organize numbering / ordering around subsystem usage
- Reduce "type" field to 8 bits
- Reserve space for "subtype" field
- Use PA_ namespace and shorter a lot of very long names

Change-Id: I5e59f412c3b9bd79d90f921a248a6d98d4b703bc

[musl][threads] Simplify thread creation locking

This locking mechanism only needs to exclude dlopen and the allocation
of per-thread thread_local storage. As such, let's move it into the
TCB allocation routine.

This patch also documents and renames the locking mechanism.

Change-Id: I090c5a9d4f3469a26648ddcf87ef6df17db4b5b9

[ulib][musl][ldso] Use mx_vmo_clone for writable segments

We now have copy-on-write support, so use it.

Change-Id: I830bb6ab37d53ba01d1751aee90851f2fcaaf71f

[ulib][musl] Fix up a few mx_* calls to be _mx_*

A few name space violations leaked in since the main cleanup.

Change-Id: I9a6b65b7e1108fb49b0fd500d33c99fa374b420d

[musl][pthread] Remove pthread cancellation machinery

Nothing currently in Fuchsia uses this, nor am I aware of any 2nd or
3rd party code that is likely to. I do not believe that, if someone
were to attempt to truly implement pthread_cancel etc. on Magenta,
that the removal of this code and comments would be a meaningful
impediment.

Change-Id: Ib8435a68d0b053ba3f1d0590640ef54c23e59840

[ulib][musl][ldso] Use __ehdr_start in place of _BASE

With the layouts we use, __ehdr_start has st_value of 0 so it
works fine for the purpose for which we defined _BASE. Also,
LLD is brain-damaged about absolute symbols but has a special
case for __ehdr_start (the maintainers refuse to fix it properly),
so this works around its nonsense.

Change-Id: Ia38e28743b9333c35f5a4bdb6fccbf0af64ddfe7

[types][transaction] Name the transaction ID type.

We've discussed potentially moving this to 8 bytes as part of the
FIDLv2.0 work.

Change-Id: I510064cfe231cda09163759d9b4247682d8300e9

[musl][ldso] Be more typesafe in iteration over _DYNAMIC

Change-Id: I3154e653cb3be3780e110f82cb14b4aae3cc403c

[ldso][crashlogger] Add MX_PROP_PROCESS_DEBUG_ADDR support

When aslr is turned on the address of the dynamic linker is no longer
fixed and thus tools like crashlogger and debuggers can no longer find
the address of ld.so's _dl_debug_addr. This patch adds
MX_PROP_PROCESS_DEBUG_ADDR as a way for tools to obtain this value.

MG-536 #done

Change-Id: I26abfbbf8719a0b226e86d87b506ca319be191a4

[musl][ldso] Avoid looking at an uninitialized mx_handle_t

For MG-553: "Sometimes programs fail to link when started at boot"

Change-Id: Ie096b3e393990045a1962344a61af5d05e9ff871

[build][userboot][ulib][runtime][musl] Support userland safe-stack build

Add build machinery to enable safe-stack for user-mode code by
default when building with Clang.

Annotate code on the early startup path so it won't use the
safe-stack ABI before the thread pointer and stacks are set up.

Disable safe-stack entirely for everything that goes into userboot.

Change-Id: Ifb1bc785bb0cc97164b937b3a32d5bfe26d67ef1

[musl] Include magenta/process.h for vmar_root_self

For MG-540: "Move process_self et al. out of syscalls.h"

Change-Id: I51559d7989397a1e2239cac32995549be151be2c

[musl][magenta] Move process handle header into magenta

Change-Id: I00eb8f5ca7fbaf9297f3b1e5b391be00ce5d11e2

[ulib][musl] Revamp stack allocation

Allocate a separate block for the thread descriptor and TLS area.
Allocate separate safe and unsafe stacks. Use the same allocation
code for new threads and for the initial thread.

The initial stack allocated by the process creator is now used only
for dynamic linker startup and to hold the processargs data. The
initial thread switches to its newly-allocated safe stack before it
starts running application code. The dynamic linker is now
responsible for noticing a PT_GNU_STACK program header and obeying
its requested size for the initial thread's stack.

Change-Id: I4b5952adadb592213fbcb2456140cf455e4f5102

[ulib][musl][ldso] Fix clobberation in load_deps

In the runtime (dlopen) case, when load_deps hits the special
ldso and vdso objects, it will try to use nonexistent buffer
space for their ->deps and clobber adjacent static variables.

Change-Id: Ic3c5c346f51d85ae0c526d8a585121f271028580

[ulib][musl][ldso] Use a trivial bespoke allocator in the dynamic linker

The dynamic linker's internal data structures are never freed and
are only allocated by a single thread (i.e. at startup or under a lock).
Use a trivial allocator rather than malloc.

Change-Id: Idf44aaf0ab02f6242098cef98de62ac38cc28abd

[ulib][musl][ldso] Preallocate deps array along with struct dso

At the cost of a second loop through the PT_DYNAMIC entries,
this avoids the need to use realloc.

Change-Id: I987a51028d0be22fc5b585c78dc5a1cc21fd1e46

[ulib][musl] Move thread pointer setup into __libc_start_main

The dynamic linker doesn't need the thread pointer to be set up
during its startup work. If the main program's entry point doesn't
call __libc_start_main (what the _start provided by crt1.o does),
then the program doesn't expect DSO constructors to run and so can't
expect to call any libc entry points and have things work. Hence,
it doesn't need the thread pointer to have been set up.

Also some miscellaneous cleanup of the startup path.

Change-Id: I61ab0ff4712920611c705ffd3e3dbfc7fd001dd8

[ulib][musl] Get rid of builtin_tls, early thread-pointer setup

The only need for an initialized thread pointer during dynamic
linker startup was code paths that set errno. Rework the
internal code to return mx_status_t values instead of setting
errno. Then eliminate the builtin_tls buffer altogether and
don't initialize the thread pointer until all libraries are
loaded and TLS size is known.

If the malloc/calloc calls made during startup fail, they will
try to set errno and crash from a null thread pointer. But that
is an unrecoverable failure in startup that can only happen from
extremely bogus ELF files or catastrophic system failure, so
diagnosing it with a message is not much better. In the long
run, these calls will be replaced with uses of an internal
allocator that does not use errno.

One of the malloc calls on the startup path was to dynamically
allocate space for large numbers of ELF program headers. Avoid
that by simply limiting support to a fixed maximum of 16 program
headers, which is more than enough for real-world ELF files.

Change-Id: Ice8098cc8ec14204a9e603646648858a045c18b3

[musl] Clang format

Change-Id: Ib80026c68fb9347fb070fcb1672d0d0581e98516

[musl][atomics] Remove bespoke atomics in favor of C11 versions

Change-Id: Iba2a97fbe0e696858d8f94ed5089c8dfa7685ce5

[ulib][musl] Make stack-guard and unsafe-sp thread-pointer offsets part of the ABI

This fixes offsets from the thread pointer to the stack-guard
(-fstack-protector canary value) and unsafe-sp (-fsanitize=safe-stack
unsafe stack pointer) words as part of the Fuchsia C/C++ ABI. The
offsets are published in the <magenta/tls.h> header and will be baked
into compilers' target support.

Change-Id: If7f6005554eb94a01accd3c66aaa3c5031fa2e4b

[musl] Remove the bespoke a_crash macro and just use __builtin_trap

Change-Id: I6ea6a565b9f9194557cb2062a6bb4860fad3f86b

[musl][ldso] Add support for dumping load maps to debuglog.

This is used by Intel PT (Processor Trace) support so that it
can map PC values in the trace back to the ELF file.

Tracing can be turned on from boot by passing ldso.trace on the
kernel command line. Tracing can be turned on afterwards by setting
env var LD_TRACE=1.

TODO: Replace use of debuglog with formal tracing mechanism
when it exists. MG-519

Change-Id: Icd7c4e73ff603fbec8a113a70556cebb9e4df09c

[musl] Remove effective euid/egid based notion of libc.secure

This is not a useful notion for us. We don't have these IDs, for one
thing. Any per-process granting or denial of privilege is likely to
happen at another layer.

Change-Id: Ic4c5252af49013157c1adaab334ec96fb0e8e755

[ulib][musl] Remove unused __init_tls function, make __init_tp static

The nonempty version was the weak definition that's never used.
We don't need any of that logic since we don't support static linking.

The __init_tp function is only called from dynlink.c, so move it
there and make it static. It's quite short and likely to be inlined.

Change-Id: I5d523b9b5b801fcec792a0e3c4f8dcab239b4309

[processargs][userboot][launchpad] Plumb through MX_HND_TYPE_VMAR_LOADED

Add a processargs handle type for the VMAR used to load the initial
program image (usually the dynamic linker). This is passed down by
launchpad (and userboot), and received by the dynamic linker. It
enables the dynamic linker to apply RELRO page protections to its
own image after relocation.

Change-Id: Ib91933a41c52617207d33a631ff0436baf704ef1

[ulib][musl][ldso] Enable RELRO support

Adjust page protections after relocation for RELRO segments.
This requires keeping track of the handle for the library's VMAR
until relocation is complete, since apply protections can only be
done via the VMAR handle.

Change-Id: I7e7d2f4e1f0e2a16dd0ebfd32cc7a01d6d1a7bd1

[ulib][musl][ldso] Don't conflate mx_handle_t and mx_status_t

Don't assume handle values are positive.
Don't assume error values are negative.

Change-Id: I11eb5f1b90193085c1ebbc4c97f1b071fc6d735c

[ulib][musl][launchpad] Use mx_channel_call for loader service RPCs

Change-Id: I6b8adc757632594423dde95b1b0b5bc877788ada

[musl][malloc][ldso] Remove the donate_heap mechanic

Change-Id: Ie87814a0c9552691fdfbf89c3f4ef6550dab1a8d

[musl] Reenable missing-field-initializers for clang

Change-Id: I375e3a15d156a5c8eb1de4af0e380133ec33595d

[musl] Reenable incompatible-pointer-types-discards-qualifiers for clang

Change-Id: Ib6ff47da98678e82695ad55dc5c8fc928bc7a70b

[ulib][runtime] Use thread-self to setup TLS thread pointer.

Use the thread-self handle passed through from userboot or launchpad when
setting up TLS.

Change-Id: I6c002de66abe74916d14c493d8af195451a8d8b3

[ulib][musl][ldso] Pass thread-self to stage 3 of the dynamic linker.

This allows us to propagate the thread-self handle that was passed to use via
either userboot or launchpad.

Change-Id: Iaeaa8448c96f8ec9344a198ddc581f0f04e3c236

[ulib][musl][dynlink] Use a VMAR in the ELF loader

Make the dynamic linker's ELF loader allocate a VMAR for the
whole address region occupied by an ELF load image.

Change-Id: I54c22aff7806e4b3f90d804e0d6b2dafcb024b1c

[signals] Remove legacy signal names

Change-Id: I6d4543358a16d16b9b9028ede06af40a632ba31b

[musl] clang-fmt musl

Change-Id: I372caaed6754eedb2679d25547eccee2c1d81d96

[vmar][loader] Make the dynamic loader use the new VMAR interface

Change-Id: Ib838c041bed782a99a1d1896d75a7d9d4ddf378a

Remove mx_size_t and mx_ssize_t types

mx_ssize_t is no longer used. mx_size_t is superfluous and
can be replaced by the standard C type size_t.

Change-Id: I2137af6f166099d2e8aaca4fab63ff101f2a43ac

[syscalls] update vmo_{create,read,write}()

Change-Id: I6b46a48813de4e74767174c2786d04cf414b017a

[headers] declare a slot for the process handle to live in

This change removes a dependency from the public/magenta/syscalls.h
onto third_party/ulib/musl/include/magenta/process.h where
mx_process_self() was previously declared. Instead, we declare
a slot in public/magenta/internal.h where the runtime is expected
to store the process handle and provide an inline accessor for it.

Change-Id: I04f080beca3ccef554b148aa0ff67a66ba189b33

[ldso][channel] Use channels not msgpipes in the dynamic linker

Change-Id: I1a61c6a630994d7a65379a5646a18fdf37791be8

[ulib][musl][ldso] Refuse to open ET_EXEC files

We don't really support fixed-address ELF files (ET_EXEC).
So just reject them outright.

Change-Id: I022ca9b2e39562038c8eba20b93bd414cc9a766c

[ulib][runtime][magenta] mx_strstatus -> mx_status_get_string

Rename it to follow the syscall naming style, and put it in the vDSO
rather than in libruntime, which is not exported outside the magenta
build tree.

Change-Id: I956e532f2c57913e3b654591592f7b7c1d489171

[ulib][musl] Simplify dlsym

The dlsym assembly wrappers calling __dlsym served just to pick
out the return address to the caller of dlsym. Defining dlsym
directly in C can do this itself with __builtin_return_address.
Eliminate the unnecessary assembly code.

Change-Id: Ife29c2bd26c5004f78c224791c5485d3d3feb656

[musl][format] Clang format musl

Change-Id: I9d0d2d586a6587dbedda055a3b3664296c7eb89a

[syscalls][process] drop the magic 0 handle for syscalls that take process handle

This is largely just replacing all cases with mx_process_self()

Change-Id: I5fbe00f814c84537400c6a2eec18e747386ec7db

[ulib][musl][ldso] Fix VMO handle leak when mapping writable segments

The new structure of the code better represents how things will
look when there is a call to create a COW proxy object for a VMO.

Change-Id: I979827ef50dd612dae210592d9e552f751be9d0a

[ulib][magenta][runtime][musl] Give system calls _mx_* names, use them in musl

The mx_* name space is not kosher for the standard C library to use
for its own purpose. A conforming standard C program can define its
own functions called mx_* and this must not break the C library.

Give all the system call entry points in the vDSO names starting
with _mx_, with the old mx_ names as aliases. Use only the _mx_
names in runtime and musl.

Change-Id: Id01b57b87c0bb91733e9492159401ff019943822

[magenta][syscalls] migrate all syscalls to the new names

If DEPCRECATE_COMPAT_SYSCALLS is defined, flag the compatibility
wrappers with the deprecated attribute.

Enable this in the Magenta build only for now.

Change-Id: I422b1294d9db2e56551eca65dede0e5d83f8b363

[ulib][musl] Add mx_process_self function, use received proc handle

Now that the process handle is actually delivered, use it.
Also add a user function to retrieve it.

Change-Id: I1b1efe44d082d83597080e3e6e4d35dd5fbd89d0

[musl] Clean up a bunch of extern declarations

There's no reason not to declare these in headers to have the types
checked, so do so.

Change-Id: Ida3af3e56fe1f92e607f460527b9b8a2e9c115ac

[ulib] [musl] [ldso] Use the vDSO as an automatic preloaded library

Change-Id: I2278599b1c3ef0c80c3c9afdedcd492f4b2874a1

[tls] Reintroduce ELF TLS into musl

Change-Id: I87a7458387aa65b9d1534e51ad5f3abed8587af6

[ulib] [musl] [ldso] Make internal struct match 'struct link_map' again

The upstream musl penchant for repeating layouts rather than using
the proper types (and not even leaving any comments to remind you!)
led me to forget the public 'struct link_map' protocol when I added
the soname field to the internal 'struct dso'. Move the field so
that the public 'struct link_map' layout is intact.

Change-Id: Ib3651c5b2852e12ea6900422d15d86bd9cc61c7f

[ulib] [runtime] [musl] NULL-terminate the argv array

C specifies that argv[argc] == NULL.

Change-Id: I5a8505d2a67b2e9d9ad760c475a055d854e352d4

[ulib] [ldso] Don't crash under LD_DEBUG if given no argv

Change-Id: Ic89dba68c5bac8b5a74cccbecbcc0ed96d729377

[ulib] [launchpad] [musl] [ldso] Pass argument strings to dynamic linker, use argv[0] in startup messages

Make launchpad pass the argument strings as well as the environment
strings in the initial bootstrap message read by the dynamic linker.
Make the dynamic linker use the arguments so that the argv[0] is
printed in error messages and the LD_DEBUG logging.

Change-Id: Id91761e044f45aaa67b340d92d22094a2f144bef

[ulib] [musl] Prepare code for using proc-self handle

This should cover all the uses of the process handle in ld.so/libc.
This is in preparation for launchpad passing the process handle and
the kernel requiring it in calls. But actually setting the value is
disabled for now, since there isn't one we can use yet.

Change-Id: I70fb363151726d0ac6537a4b0358edecadf3b5eb

[ulib] [musl] [mxio] Add dl_set_loader_service, generalize mxio_loader_service

This adds a Magenta-specific -ldl interface to set the handle
to the loader service that the dynamic linker will use. As
part of implementing the test for this, the mxio_loader_service
function is generalized.

Change-Id: I4c07704d47839f79fafd0ddc497c9e771c02cb35

[ulib] [musl] Add dlopen_vmo function

This adds a Magenta-specific extension to -ldl, declared in the
new header file <magenta/dlfcn.h>. The new function behaves like
dlopen, but takes a VM object handle instead of a file name.

Change-Id: Ia0748cf7b3e4c40ae0bf563f6d9f3182da04547b

[ulib] [musl] [ldso] Handle LD_DEBUG environment variable

This rejiggers dynamic linker startup so that the bootstrap message
buffer stays on the dynamic linker's initial stack and can be used
to hold environment strings, but then is popped before jumping to
the user entry point. This reenables the latent LD_PRELOAD support.
Also check the environment variable LD_DEBUG for a nonempty value;
if it's set, print all the load addresses at the end of startup.

Change-Id: Ida6dbcfe006b16a6e61b963a4395e7a990163185

[ulib] [runtime] New <runtime/processargs.h> API

This is a new API for handling the processargs bootstrap message
that is clean and reentrant. Eventually this will supplant the
old <runtime/process.h> API. So far only the dynamic linker's
startup code uses it.

Change-Id: Iaf1f6fb5e3a4c91ad9308802306febba9f435123

[ulib] [runtime] Add mxr_message_size helper

This prettifies the standard use of mx_message_read
for probing message size.

Change-Id: I2246accc7bec59cfd27a4bd06843029a5ed1de02

[ulib] [musl] [ldso] Log via MXIO_LOGGER if available, fall back to mx_debug_write

This makes the dynamic linker check its bootstrap message for a
MXIO_LOGGER handle, and use mx_log_write on that handle for its
debug/error logging if possible, falling back to using the loader
service debug print RPC (as before) if possible, or to mx_debug_write if
all else fails. Lack of a loader service and numerous other error
conditions now try to log a message and fail as gracefully as possible,
rather than intentionally crashing immediately.

Change-Id: If874e70f4e68a2426ae2f86fdac40d6bb750875f

[ulib] [musl] [ldso] Handle empty PT_LOAD segments

If a PT_LOAD segment is zero-length, or the file-mapped portion
of it is zero-length, mapping the segment should not fail.

Change-Id: I1eaf24ac0f0052c19d1090f31efdf60aea9c6b88

[musl] [ldso] Don't stub out __init_tls in the dynamic linker

The static __init_tls works right for dynamic too as things stand.
Without this, the pthread_t data structure is not properly initialized
and things like CURRENT_LOCALE don't work right.

This will all be different when we restore proper ELF TLS.

Change-Id: Ifedabcf635c7b3a8f6e6ec65526b7140254a0c25

[ulib] [musl] [ldso] Fix double-free via reclaim_gaps

Since we now use map_library for the main executable (&app),
it's already been passed to reclaim_gaps. Don't do it twice!

Change-Id: If8f21aa89de04c147bac106498a36b8297ab72fd

[musl] Fix dynamic linker handling of bootstrap message-pipe

Using mxr_process_parse_args to read the dynamic linker's initial
bootstrap message closes the handle so the main program can't read its
bootstrap message. Instead, open-code minimal parsing of the bootstrap
message.

Change-Id: I4f0138b4a85b82ee5094837289aebb72bc6d034d

[musl] Clean out more linux syscall goop

- Remove some more unused SYS_* #defines
- Remove some dead internal wrappers around syscalls
- Remove Linux's prctl
- Move __syscall(SYS_fcntl, ...) to fcntl(...)
- Move __syscall(SYS_exit, ...) to exit(...)

Change-Id: I017599ce5afb6bf78021be0c71824056e16c1f65

[musl] Copy writable segments to simulate copy-on-write

The kernel does not yet support copy-on-write mapping semantics.
The dynamic linker needs those semantics for writable segments.
Otherwise it modifies the VM object representing the file, which
is also used to back the read-only segments and in canonical ELF
layout the read-only and writable portions will overlap in the file,
so touching the writable data also touches some read-only pages.
To work around the problem, eagerly copy data into a fresh VM
object for a writable segment.

Change-Id: Ib3b762613b8411c61644c794397458f32faf808b

[musl] Revamp dynamic linker startup protocol

In the old scheme, the system would load the main ELF file and
then load the PT_INTERP file and pass information about where
things were loaded via auxv entries.

In the new scheme, the system will load only the PT_INTERP file
(the dynamic linker itself) and pass it a handle to a VM object
representing the main ELF file. The dynamic linker then maps in
the main program image itself. There is no need for auxv at all.

Change-Id: I0795e72c9dfeef9567e1ce318f566342328a6419

[musl] Move "heap donation" magic into malloc implementation

The dynamic linker's "reclaim" function is full of malloc implementation
details. Factor those out into a subroutine that lives in the malloc
implementation, so it's more clear what the magic is doing. This way,
if the malloc implementation is changed or replaced then the dynamic
linker code won't need arcane adjustments.

Change-Id: Ia574b10fc2c9a1c55b003948f9a6640f8c772263

[musl] Dynamic linking support

This enables the build of musl's libc.so, which is also the dynamic linker.

Change-Id: I26de0d8cc410667aa346c1138b327777f64bd969

[musl] Clean up some warnings in musl's make build

Change-Id: Ib305f1d161634156fad8f7303208c49be66c855c

[musl] clang format musl

Change-Id: I3bc6ab7655410514691cd07f08454f3afd41d9b3

[magenta] Initial commit