Merge llvm, clang, compiler-rt, libc++, libunwind, lld, lldb and openmp
9.0.0 final release r372316.

(NOTE: This does not yet consolidate all the llvm project sources under
contrib/llvm-project (e.g., MFC r355940), due to Subversion limitations.
It will be done in a follow-up MFC.)

MFC r348610 (by emaste):

build llvm-ar and llvm-nm with Clang (promote out of CLANG_EXTRAS)

To facilitate experimentation with LTO we require an ar that supports
LLVM IR, and to a lesser degree also an nm. As a first step always
install llvm-ar and llvm-nm.

Sponsored by: The FreeBSD Foundation

MFC r350453 (by asomers):

Add a CXXWARNFLAGS variable

Some warning flags are valid for C++ but not C. GCC 8 complains if you pass
such flags when building a C file. Using a separate variable for these
flags allows building both C and C++ files in the same directory (such as
the fusefs tests) under GCC.

Reviewed by: cem, emaste
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D21116

MFC r350467 (by luporl):

[PPC64] Backport fix for missing IRELATIVE relocations

This is a backport of LLVM commit 8331f61a51a7a0a1efbf5ed398e181593023d151,
llvm-svn: 353981:

ELF: Allow GOT relocs pointing to non-preemptable ifunc to resolve to an
IRELATIVE where possible.

This is needed in order to make ifuncs work correctly on PPC64.

It fixes an issue with lld, in which it would skip emitting necessary IRELATIVE
relocations. Without this change, indirect calls to ifuncs would result in a
segmentation fault, in static binaries or when defined in the main binary
(outside shared libraries).

This change also reverts the local
"Preserve relocations against ifuncs when -zifunc-noplt" commit and
replaces it by its upstream version, as part of the merge.

Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D21102

MFC r351662 (by emaste):

lldb: shorten thread names to make logs easier to follow

lldb prepends the thread name to log entries, and the existing thread
name for the FreeBSD ProcessMonitor thread was longer than the kernel's
supported thread name length, and so was truncated. This made logs hard
to read, as the truncated thread name ran into the log message. Shorten
"lldb.process.freebsd.operation" to just "freebsd.op" so that logs are
more readable.

(Upstreaming to lldb still to be done).

MFC r352095 (by emaste):

compiler-rt: use more __sanitizer_time_t on FreeBSD

A few structs were using long for time_t members.

Obtained from: LLVM r370755

MFC r352096 (by emaste):

compiler-rt: use 64-bit time_t for all FreeBSD archs except i386

Obtained from: LLVM r370756

MFC r352167 (by imp):

Remove dir empty since r276851

MFC r352168 (by imp):

Remove dirs empty since r280031

MFC r352169 (by imp):

Remove dir empty since r314564

MFC r352170 (by imp):

Remove dir empty since r327952

MFC r352171 (by imp):

Remove dirs (and their now-empty parents) empty since r344779

MFC r352792 (by emaste):

compiler-rt: correct RISC-V struct_kernel_stat64_sz

The value of struct_kernel_stat64_sz introduced by review D5021 for
RISC-V was incorrect.

Also add a __riscv_xlen == 64 conditional as the 32-bit ABI is not yet
finalized.

Submitted by: Luís Marques
Differential Revision: https://reviews.freebsd.org/D21684

MFC r353018 (by kevans):

clang: use -mxgot for 32-bit mips

Various bits in usr.bin/clang/* will fail to compile without -mxgot due to
truncated relocations. -mxgot entails a speed penalty, but I suspect we
don't care as much about compiler performance in 32-bit mips land.

Reviewed by: arichardson
Differential Revision: https://reviews.freebsd.org/D21698

MFC r353358:

Merge llvm, clang, compiler-rt, libc++, libunwind, lld, lldb and openmp
9.0.0 final release r372316.

Release notes for llvm, clang, lld and libc++ 9.0.0 are available here:

https://releases.llvm.org/9.0.0/docs/ReleaseNotes.html
https://releases.llvm.org/9.0.0/tools/clang/docs/ReleaseNotes.html
https://releases.llvm.org/9.0.0/tools/lld/docs/ReleaseNotes.html
https://releases.llvm.org/9.0.0/projects/libcxx/docs/ReleaseNotes.html

PR: 240629

MFC r353363:

Put in a band-aid fix for lldb 9 exiting with "Expected<T> must be
checked before access or destruction" when launching executables, while
we sort this out with upstream.

Reported by: jbeich
PR: 241137

MFC r353415:

Revert r353363 in preparation for applying upstream fix:

Put in a band-aid fix for lldb 9 exiting with "Expected<T> must be
checked before access or destruction" when launching executables, while
we sort this out with upstream.

PR: 241137

MFC r353416:

Pull in r374444 from upstream lldb trunk (by me):

Fix process launch failure on FreeBSD after r365761

Summary:
After rLLDB365761, and with `LLVM_ENABLE_ABI_BREAKING_CHECKS`
enabled, launching any process on FreeBSD crashes lldb with:

```
Expected<T> must be checked before access or destruction.
Expected<T> value was in success state. (Note: Expected<T> values in
success mode must still be checked prior to being destroyed).
```

This is because `m_operation_thread` and `m_monitor_thread` were
wrapped in `llvm::Expected<>`, but this requires the objects to be
correctly initialized before accessing them.

To fix the crashes, use `llvm::Optional<>` for the members (as
indicated by labath), and use local variables to store the return
values of `LaunchThread` and `StartMonitoringChildProcess`. Then,
only assign to the member variables after checking if the return
values indicated success.

Reviewers: devnexen, emaste, MaskRay, mgorny

Reviewed By: devnexen

Subscribers: jfb, labath, krytarowski, lldb-commits

Differential Revision: https://reviews.llvm.org/D68723

PR: 241137

MFC r353579 (by jhb):

Use __FreeBSD_version to determine if gets() has been removed.

GCC compilers set __FreeBSD__ statically to a build-time determined
targeted version (which in ports always matches the build host's
version). This means that when building any version (12 or 13, etc.)
of riscv or some other architecture via GCC on a 12.x host,
__FreeBSD__ will always be set to 12. As a result, __FreeBSD__ cannot
be used to reliably detect the target FreeBSD version being built.
Instead, __FreeBSD_version from either <sys/param.h> (in the kernel)
or <osreldate.h> (in userland) should be used.

This changes the gets() test in libc++ to use __FreeBSD_version from
<osreldate.h>.

Reported by: jenkins (riscv64 and amd64-gcc)
Reviewed by: dim, imp
Differential Revision: https://reviews.freebsd.org/D22034

MFC r353711 (by mhorne):

Fix build of LLVM RISC-V backend

Reviewed by: dim
MFC with: r353358
Differential Revision: https://reviews.freebsd.org/D21963

MFC r353738:

Pull in r372651 from upstream lld trunk (by Simon Atanasyan):

[mips] Support elf32btsmipn32_fbsd / elf32ltsmipn32_fbsd emulations

Patch by Kyle Evans.

Requested by: kevans

MFC r353739:

Pull in r374154 from upstream clang trunk (by Simon Atanasyan):

[mips] Set default float ABI to "soft" on FreeBSD

Initial patch by Kyle Evans.

Fix PR43596

Requested by: kevans

MFC r353936:

Bump clang's default target CPU for the i386 architecture (aka "x86") to
i686, as per the discussion on the freebsd-arch mailing list. Earlier
in r352030, I had already bumped it to i586, to work around missing
atomic 64 bit functions for the i386 architecture.

Relnotes: yes

MFC r354097:

Pull in r372186 from upstream llvm trunk (by Eli Friedman):

[ARM] VFPv2 only supports 16 D registers.

r361845 changed the way we handle "D16" vs. "D32" targets; there used
to be a negative "d16" which removed instructions from the
instruction set, and now there's a "d32" feature which adds
instructions to the instruction set. This is good, but there was an
oversight in the implementation: the behavior of VFPv2 was changed.
In particular, the "vfp2" feature was changed to imply "d32". This is
wrong: VFPv2 only supports 16 D registers.

In practice, this means if you specify -mfpu=vfpv2, the compiler will
generate illegal instructions.

This patch gets rid of "vfp2d16" and "vfp2d16sp", and fixes "vfp2"
and "vfp2sp" so they don't imply "d32".

Differential Revision: https://reviews.llvm.org/D67375

Pull in r372187 from upstream clang trunk (by Eli Friedman):

[ARM] Update clang for removal of vfp2d16 and vfp2d16sp

Matching fix for https://reviews.llvm.org/D67375 (r372186).

Differential Revision: https://reviews.llvm.org/D67467

This should fix clang generating invalid opcodes for floating point
operations on armv6.

Requested by: mmel

MFC r354146:

Pull in r373338 from upstream llvm trunk (by Simon Pilgrim):

Revert rL349624 : Let TableGen write output only if it changed,
instead of doing so in cmake, attempt 2

Differential Revision: https://reviews.llvm.org/D55842
-----------------
As discussed on PR43385 this is causing Visual Studio msbuilds to
perpetually rebuild all tablegen generated files

Pull in r373664 from upstream llvm trunk (by Nico Weber):

Reland r349624: Let TableGen write output only if it changed, instead
of doing so in cmake

Move the write-if-changed logic behind a flag and don't pass it with
the MSVC generator. msbuild doesn't have a restat optimization, so
not doing write-if-change there doesn't have a cost, and it should
fix whatever causes PR43385.

This should fix the scenario where an incremental build from before
r353358 (the clang 9.0.0 upgrade) to r353358 or later fails to update
the timestamp of the generated lib/clang/headers/arm_fp16.h header.

After such a build, installing world from read-only source and object
directories would attempt to generate the header again, leading to
"clang-tblgen: error opening arm_fp16.h.d:Read-only file system".

Reported by: avg, np
PR: 241402

MFC r354339:

Merge commit 97e362607 from llvm git (by Nemanja Ivanovic):

[PowerPC] Do not emit HW loop if the body contains calls to
lrint/lround

These two intrinsics are lowered to calls so should prevent the
formation of CTR loops. In a subsequent patch, we will handle all
currently known intrinsics and prevent the formation of HW loops if
any unknown intrinsics are encountered.

Differential revision: https://reviews.llvm.org/D68841

This should fix an "invalid CRT loop" assertion when building the
www/node port for powerpc64.

Requested by: Alfredo Dal'Ava Júnior <alfredo.junior@eldorado.org.br>

MFC r354347 (by cem):

Fix llvm-libunwind userspace build on ARM

GCC's libgcc exports a few ARM-specific symbols for ARM EABI, AEABI, or
EHABI or whatever it's called. Export the same ones from LLVM-libunwind's
libgcc_s, on ARM. As part of this, convert libgcc_s from a direct
Version.map to one constructed from component Symbol.map files. This allows
the ARM-specific Symbol.map to be included only on ARM.

Fix ARM-only oddities in struct name/aliases in LLVM-libunwind to match
non-ARM definitions and ARM-specific expectations in libcxxrt /
libcompiler_rt.

No functional change intended for non-ARM architectures.

This commit does not actually flip the switch for ARM defaults from libgcc
to llvm-libunwind, but makes it possible (to compile, anyway).

MFC r354418 (by cem):

clang: Enable unwind tables on !amd64

There doesn't seem to be much sense in defaulting "on" unwind tables on
amd64 and not on other arches. It causes surprising differences between
platforms, such as the PR below.

Prior to this change, FreeBSD inherited the default implementation of the
method from the Gnu.h Generic_Elf => Generic_GCC parent class, which
returned true only for amd64 targets. Override that and opt on always,
similar to, e.g., NetBSD.

PR: 241562
Reported by: lwhsu
Reviewed by: dim
Discussed with: emaste
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D22252

MFC r354429:

Merge commit 8e34dd941 from llvm git (by Sanjay Patel):

[x86] avoid crashing when splitting AVX stores with non-simple type
(PR43916)

The store splitting transform was assuming a simple type (MVT), but
that's not necessarily the case as shown in the test.

This should fix 'Assertion failed: (isSimple() && "Expected a
SimpleValueType!")' when building the security/openssl111 port targeting
a CPU that supports AVX, but not AVX2, such as sandybridge.

PR: 241747

MFC r354469:

Merge commit f596f4507 from llvm git (by Sam Elliott):

[RISCV] Add Custom Parser for Atomic Memory Operands

Summary:
GCC Accepts both (reg) and 0(reg) for atomic instruction memory
operands. These instructions do not allow for an offset in their
encoding, so in the latter case, the 0 is silently dropped.

Due to how we have structured the RISCVAsmParser, the easiest way to
add support for parsing this offset is to add a custom AsmOperand and
parser. This parser drops all the parens, and just keeps the
register.

This commit also adds a custom printer for these operands, which
matches the GCC canonical printer, printing both `(a0)` and `0(a0)`
as `(a0)`.

Reviewers: asb, lewis-revill

Reviewed By: asb

Subscribers: s.egerton, hiraditya, rbar, johnrusso, simoncook,
apazos, sabuasal, niosHD, kito-cheng, shiva0217, jrtc27, MaskRay,
zzheng, edward-jones, rogfer01, MartinMosbeck, brucehoult, the_o,
rkruppe, jfb, PkmX, jocewei, psnobl, benna, Jim, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D65205

llvm-svn: 367553

Merge commit f596f4507 from llvm git (by Sam Elliott):

[RISCV] Add FreeBSD targets

Reviewers: asb

Reviewed By: asb

Subscribers: simoncook, s.egerton, lenary, psnobl, benna, mhorne,
emaste, kito-cheng, shiva0217, rogfer01, rkruppe, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D57795

Patch by James Clarke (jrtc27)

llvm-svn: 367557

Merge commit f596f4507 from llvm git (by Hsiangkai Wang):

[DebugInfo] Generate fixups as emitting DWARF .debug_frame/.eh_frame.

It is necessary to generate fixups in .debug_frame or .eh_frame as
relaxation is enabled due to the address delta may be changed after
relaxation.

There is an opcode with 6-bits data in debug frame encoding. So, we
also need 6-bits fixup types.

Differential Revision: https://reviews.llvm.org/D58335

llvm-svn: 366524

Merge commit f596f4507 from llvm git (by Hsiangkai Wang):

[DebugInfo] Some fields do not need relocations even relax is enabled.

In debug frame information, some fields, e.g., Length in CIE/FDE and
Offset in FDE are attributes to describe the structure of CIE/FDE.
They are not related to the relaxed code. However, these attributes
are symbol differences. So, in current design, these attributes will
be filled as zero and LLVM generates relocations for them.

We only need to generate relocations for symbols in executable
sections. So, if the symbols are not located in executable sections,
we still evaluate their values under relaxation.

Differential Revision: https://reviews.llvm.org/D61584

llvm-svn: 366531

Merge commit f596f4507 from llvm git (by Alex Bradbury):

[RISCV] Don't force absolute FK_Data_X fixups to relocs

The current behavior of shouldForceRelocation forces relocations for
the majority of fixups when relaxation is enabled. This makes sense
for fixups which incorporate symbols but is unnecessary for simple
data fixups where the fixup target is already resolved to an absolute
value.

Differential Revision: https://reviews.llvm.org/D63404
Patch by Edward Jones.

llvm-svn: 369257

Merge commit f596f4507 from llvm git (by Alex Bradbury):

[RISCV] Implement getExprForFDESymbol to ensure RISCV_32_PCREL is
used for the FDE location

Follow binutils in using RISCV_32_PCREL for the FDE initial location.
As explained in the relevant binutils commit
<https://github.com/riscv/riscv-binutils-gdb/commit/a6cbf936e3dce68114d28cdf60d510a3f78a6d40>,
the ADD/SUB pair of relocations is problematic in the presence of
linker relaxation.

This patch has the same end goal as D64715 but includes test changes
and avoids adding a new global VariantKind to MCExpr.h (preferring
RISCVMCExpr VKs like the rest of the RISC-V backend).

Differential Revision: https://reviews.llvm.org/D66419

llvm-svn: 369375

This series of merges will permit riscv64 kernels and riscv64sf worlds
to build with clang instead of gcc (but still using the bfd linker).

Requested by: jhb
Obtained from: https://github.com/freebsd/freebsd/compare/master...bsdjhb:riscv_clang

MFC r354662 (by jhb):

Sync target triple generation with the version in Makefile.inc1.

Reviewed by: dim
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D22333

MFC r354692 (by emaste):

llvm: use AT_EXECPATH from ELF auxiliary vectors for getExecutablePath

/proc/curproc/file and the KERN_PROC_PATHNAME sysctl may not return the
desired path if there are multiple hardlinks to the file.

PR: 241932
Tested by: ler
Sponsored by: The FreeBSD Foundation

MFC r354707 (by emaste):

llvm: use elf_aux_info to get executable's path, if available

Obtained from: LLVM a0a38b81ea
MFC with: r354692
Sponsored by: The FreeBSD Foundation

MFC r354979:

Merge commit 7bed381ea from llvm git (by Simon Atanasyan):

[mips] Implement Octeon+ `saa` and `saad` instructions

`saa` and `saad` are 32-bit and 64-bit store atomic add instructions.

memory[base] = memory[base] + rt

These instructions are available for "Octeon+" CPU. The patch adds
support for both instructions to MIPS assembler and diassembler and
introduces new CPU type - "octeon+".

Next patches will implement `.set arch=octeon+` directive and
`AFL_EXT_OCTEONP` ISA extension flag support.

Differential Revision: https://reviews.llvm.org/D69849

This is one of the upstream changes needed for adding support for the
OCTEON+ CPU type, so that we can test Clang builds using the most
commonly available FreeBSD/mips64 reference platform, the Edge Router
Lite.

Requested by: kevans

MFC r354980:

Merge commit 3718102d4 from llvm git (by Simon Atanasyan):

[mips] Support `octeon+` CPU in the `.set arch=` directive

Differential Revision: https://reviews.llvm.org/D69850

This is one of the upstream changes needed for adding support for the
OCTEON+ CPU type, so that we can test Clang builds using the most
commonly available FreeBSD/mips64 reference platform, the Edge Router
Lite.

Requested by: kevans

MFC r354981:

Merge commit bf996f761 from llvm git (by Simon Atanasyan):

[mips] Write `AFL_EXT_OCTEONP` flag to the `.MIPS.abiflags` section

Differential Revision: https://reviews.llvm.org/D69851

This is one of the upstream changes needed for adding support for the
OCTEON+ CPU type, so that we can test Clang builds using the most
commonly available FreeBSD/mips64 reference platform, the Edge Router
Lite.

Requested by: kevans

MFC r354982:

Merge commit 3552d3e0f from llvm git (by Simon Atanasyan):

[mips] Add `octeon+` to the list of CPUs accepted by the driver

This is one of the upstream changes needed for adding support for the
OCTEON+ CPU type, so that we can test Clang builds using the most
commonly available FreeBSD/mips64 reference platform, the Edge Router
Lite.

Requested by: kevans

MFC r354983:

Merge commit e578d0fd2 from llvm git (by Simon Atanasyan):

[mips] Fix `__mips_isa_rev` macros value for Octeon CPU

This is one of the upstream changes needed for adding support for the
OCTEON+ CPU type, so that we can test Clang builds using the most
commonly available FreeBSD/mips64 reference platform, the Edge Router
Lite.

Requested by: kevans

MFC r354984:

Merge commit 0d14656b9 from llvm git (by Simon Atanasyan):

[mips] Set __OCTEON__ macros

This is one of the upstream changes needed for adding support for the
OCTEON+ CPU type, so that we can test Clang builds using the most
commonly available FreeBSD/mips64 reference platform, the Edge Router
Lite.

Requested by: kevans

MFC r354985:

Merge commit a751f557d from llvm git (by Simon Atanasyan):

[mips] Set macros for Octeon+ CPU

This is one of the upstream changes needed for adding support for the
OCTEON+ CPU type, so that we can test Clang builds using the most
commonly available FreeBSD/mips64 reference platform, the Edge Router
Lite.

Requested by: kevans

MFC r355397:

Merge commit 241cbf201 from llvm git (by Nemanja Ivanovic):

[PowerPC] Fix crash in peephole optimization

When converting reg+reg shifts to reg+imm rotates, we neglect to
consider the CodeGenOnly versions of the 32-bit shift mnemonics. This
means we produce a rotate with missing operands which causes a crash.

Committing this fix without review since it is non-controversial that
the list of mnemonics to consider should include the 64-bit aliases
for the exact mnemonics.

Fixes PR44183.

This should fix "Assertion failed: (idx < size()), function operator[],
file /usr/src/contrib/llvm/include/llvm/ADT/SmallVector.h, line 153"
when building the graphics/mesa-dri port for the PowerPC64 ELFv2 ABI.

Reported by: Alfredo Dal'Ava Júnior <alfredo.junior@eldorado.org.br>

MFC r355602:

Add a few missed source files to libllvm, for the MK_LLVM_TARGET_BPF=yes
case. Otherwise, linking of clang and other llvm based executables
would complain about missing symbols.

Reported by: rstone

MFC r355645 (by cem):

arm: libgcc_s: Fix ABI breakage introduced in r354347

Provide the symbol version for llvm-libunwind's _Unwind_Backtrace that libgcc
has historically provided on arm, in addition to the (default) standard version
used on all other arch.

Reported by: mmel

MFC r355803 (by mmel):

Fix LLVM libunwnwind _Unwind_Backtrace symbol version for ARM.
In original GNU libgcc, _Unwind_Backtrace is published with GCC_3.3 version
for all architectures but ARM. For ARM should be publishes with GCC_4.3.0
version. This was originally omitted in r255095, fixed in r318024 and omitted
aging in LLVM libunwind implementation in r354347.

For ARM _Unwind_Backtrace should be published as default with GCC_4.3.0
version , (because this is right original version) and again as
normal(not-default) with GCC_3.3 version (to maintain ABI compatibility
compiled/linked with wrong pre r318024 libgcc)

PR: 233664

Merge additions of LLVM libunwind libgcc_eh and libgcc_s. This is in
preparation of further LLVM merges.

MFC r307230 (by emaste):

Introduce lib/libgcc_eh and lib/libgcc_s for LLVM's implementation

They are not yet connected to the build, but I am adding them to allow
for easier testing, ports exp-runs, etc.

Reviewed by: ed
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D8188

MFC r307231 (by emaste):

libgcc_s: add libm dependencies from div{d,s,x}c3

compiler-rt's complex division support routines contain calls to
compiler builtins such as `__builtin_scalbnl`. Unfortunately Clang
turns these back into a call to `scalbnl`.

For now link libm's C version of the required support routines.

Reviewed by: ed
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D8190

MFC r307864 (by emaste):

Move the LLVM-based libgcc_s to /lib

When enabled, it should install in the same location as the existing
library.

Reported by: antoine

MFC r308001 (by emaste):

libgcc_eh/libgcc_s: apply hidden visibility only to static libs

MFC r308100 (by emaste):

compile libunwind c source with -fexceptions

When an exception is thrown the unwinder must unwind its own C source
(starting with _Unwind_RaiseException in UnwindLevel1.c), so it needs to
be built with unwinding data.

MFC r308294 (by emaste):

libgcc_s: make unspecified shlib symbols local

We want only symbols explicitly specified in the Version.map.

Sponsored by: The FreeBSD Foundation

MFC r308308 (by emaste):

Connect new LLVM-based libgcc_eh & libgcc_s to the build

Compiler-rt and LLVM's libunwind provide a suitable replacement for
libgcc.a, libgcc_eh.a, and libgcc_s.so.

Remove the now-unused LLVM_LIBUNWIND block from gnu/lib/libgcc.

PR: 213480 [exp-run]
Reviewed by: brooks, ed
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D8189

MFC r308379 (by emaste):

add __divdi3 and __udivdi3 to libgcc_s symbol version map

After r308294 they were missing on i386 (and previously were exported
only accidentally).

Reported by: antoine

MFC r308445 (by emaste):

add missing i386 symbols libgcc_s symbol version map

After r308294 they were missing on i386 (and previously were exported
only accidentally).

Reported by: antoine

MFC r312076 (by emaste):

libgcc_s: add libc DT_NEEDED to fix underlinking

PR: 216012
Reported by: jbeich
Sponsored by: The FreeBSD Foundation

MFC r316101 (by ngie):

Apply r315689 to lib/libgcc_s as well to unbreak the gcc xtoolchain build

lib/libgcc_s consumes lib/libcompiler_rt/Makefile*. The NO_WERROR.gcc in
lib/libcompiler_rt/Makefile doesn't seem to have made a difference in being
able to build this, so sprinkle NO_WERROR.gcc here as well.

Reported by: Jenkins (FreeBSD-head-amd64-gcc)
Tested with: amd64-gcc-6.3.0 (devel/amd64-xtoolchain-gcc)
Sponsored by: Dell EMC Isilon

MFC r320673 (by emaste):

Sort entries in libgcc_s Version.map

MFC r337585 (by dim):

In r308100, an explicit -fexceptions flag was added for the C sources
from LLVM's libunwind, which end up in libgcc_eh.a and libgcc_s.so.
This is because the unwinder needs the unwinder data for its own
functions.

However, for the C++ sources in libunwind, -fexceptions is already the
default, and this can have the side effect of generating a reference to
__gxx_personality_v0, the so-called personality function, which is
normally provided by the C++ ABI library (libcxxrt or libsupc++).

If the reference ends up in the eventual libgcc_s.so, linking any
non-C++ programs against it will fail with "undefined reference to
`__gxx_personality_v0'".

Note that at high optimization levels, the reference is usually
optimized away, which is why we have never noticed this problem before.

With clang 7.0.0 though, higher optimization levels don't help anymore,
since the addition of address-significance tables [1] in
<https://reviews.llvm.org/rL337339>. Effectively, this always causes a
reference to __gxx_personality_v0.

After discussion with the upstream author of that change, it turns out
that we should compile libunwind sources with the -fno-exceptions
-funwind-tables flags instead. This ensures unwind tables are
generated, but no references to any personality functions are emitted.

[1] https://lists.llvm.org/pipermail/llvm-dev/2018-May/123514.html

Reported by: jbeich
PR: 230399

MFC r356356, r356358, r356422: replace gcclibs' libssp

r356356:
Provide libssp based on libc

For libssp.so, rebuild stack_protector.c with FORTIFY_SOURCE stubs that just
abort built into it.

For libssp_nonshared.a, steal stack_protector_compat.c from
^/lib/libc/secure and massage it to maintain that __stack_chk_fail_local
is a hidden symbol.

libssp is now built unconditionally regardless of {WITH,WITHOUT}_SSP in the
build environment, and the gcclibs version has been disconnected from the
build in favor of this one.

r356358:
libssp: fix FORTIFY_SOURCE stub declarations

The LSB 4.1 that I referenced omitted the varargs, and I failed to catch it.
The __vsnprintf_chk error was from just downright misreading the page. GCC6
caught all of these, but I had only tested GCC4.2.

r356422:
Update libssp paths in various Makefile.depend* files

I've been advised that the model that uses these are fairly resilient, but
we do know the proper path to use (or remove, in the case of ^/targets/...),
so go ahead and update them to reflect that.

MFC bectl(8)/libbe(3): r337663-337664,337667,337697-337699,337800,337805,
337915-337918,337921,337924,337947,337993-337995,338221-338222,338303,
338417,339047,339972,339994,340334,340507-340508,340592-340594,
340635-340636,340722-340723,340974,342466,342849,342903,342911,343335,
343543,343977,343993-343994,344034,344067,344084,345302,345769,
345845-345846,345848,346082

There are simply too many small changes to enumerate; in summary:

bectl(8)/libbe(3) has been introduced from current state in -CURRENT and
added to the stable/11 rescue build. bectl(8) is a tool for managing ZFS
boot environments, largely inspired by beadm. It includes features such as
being able to jail a boot environment or easily mount it for modification.

Relnotes: probably

Merge llvm, clang, compiler-rt, libc++, libunwind, lld, lldb and openmp
8.0.0 final release r356365.

MFC r306265 (by emaste):

Force LLVM_LIBUNWIND off if we don't have a C++11 compiler

Tested by: bde
Differential Revision: https://reviews.freebsd.org/D7746

MFC r308100 (by emaste):

compile libunwind c source with -fexceptions

When an exception is thrown the unwinder must unwind its own C source
(starting with _Unwind_RaiseException in UnwindLevel1.c), so it needs to
be built with unwinding data.

MFC r324998 (by bdrewery):

Prefix {TARGET,BUILD}_TRIPLE with LLVM_ to avoid Makefile.inc1 collision.

The Makefile.inc1 TARGET_TRIPLE is for specifying which -target is used
during the build of world.

Reviewed by: dim, imp
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D12792

MFC r329093 (by emaste):

Promote llvm-cov to a standalone option

Introduce WITH_/WITHOUT_LLVM_COV to match GCC's WITH_/WITHOUT_GCOV.
It is intended to provide a superset of the interface and functionality
of gcov.

It is enabled by default when building Clang, similarly to gcov and GCC.

This change moves one file in libllvm to be compiled unconditionally.
Previously it was included only when WITH_CLANG_EXTRAS was set, but the
complexity of a new special case for (CLANG_EXTRAS | LLVM_COV) is not
worth avoiding a tiny increase in build time.

Reviewed by: dim, imp
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D142645

MFC r331244 (by jhb):

Add support for MIPS to LLVM's libunwind.

This is originally based on a patch from David Chisnall for soft-float
N64 but has since been updated to support O32, N32, and hard-float ABIs.
The soft-float O32, N32, and N64 support has been committed upstream.
The hard-float changes are still in review upstream.

Enable LLVM_LIBUNWIND on mips when building with a suitable (C+11-capable)
toolchain. This has been tested with external GCC for all ABIs and
O32 and N64 with clang.

Reviewed by: emaste
Obtained from: CheriBSD (original N64 patch)
Sponsored by: DARPA / AFRL
Differential Revision: https://reviews.freebsd.org/D14701

MFC r336691 (by emaste):

llvm: remove __FreeBSD_version conditionals

All supported FreeBSD build host versions have backtrace.h, so we can
just eliminate that test. For futimes() we can test the compiler's
built-in __FreeBSD__ major version rather than relying on including
osreldate.h. This should reduce the frequency with which Clang gets
rebuilt when building world.

Reviewed by: dim
Sponsored by: The FreeBSD Foundation

MFC r337379 (by andrew):

Default to armv5te in LINT on arm. This should fix building LINT there.

MFC r337552:

Add optional LLVM BPF target support

BPF (eBPF) is an independent instruction set architecture which is
introduced in Linux a few years ago. Originally, eBPF execute
environment was only inside Linux kernel. However, recent years there
are some user space implementation (https://github.com/iovisor/ubpf,
https://doc.dpdk.org/guides/prog_guide/bpf_lib.html) and kernel space
implementation for FreeBSD is going on
(https://github.com/YutaroHayakawa/generic-ebpf).

The BPF target support can be enabled using WITH_LLVM_TARGET_BPF, as it
is not built by default.

Submitted by: Yutaro Hayakawa <yhayakawa3720@gmail.com>
Reviewed by: dim, bdrewery
Differential Revision: https://reviews.freebsd.org/D16033

MFC r337585:

In r308100, an explicit -fexceptions flag was added for the C sources
from LLVM's libunwind, which end up in libgcc_eh.a and libgcc_s.so.
This is because the unwinder needs the unwinder data for its own
functions.

However, for the C++ sources in libunwind, -fexceptions is already the
default, and this can have the side effect of generating a reference to
__gxx_personality_v0, the so-called personality function, which is
normally provided by the C++ ABI library (libcxxrt or libsupc++).

If the reference ends up in the eventual libgcc_s.so, linking any
non-C++ programs against it will fail with "undefined reference to
`__gxx_personality_v0'".

Note that at high optimization levels, the reference is usually
optimized away, which is why we have never noticed this problem before.

With clang 7.0.0 though, higher optimization levels don't help anymore,
since the addition of address-significance tables [1] in
<https://reviews.llvm.org/rL337339>. Effectively, this always causes a
reference to __gxx_personality_v0.

After discussion with the upstream author of that change, it turns out
that we should compile libunwind sources with the -fno-exceptions
-funwind-tables flags instead. This ensures unwind tables are
generated, but no references to any personality functions are emitted.

[1] https://lists.llvm.org/pipermail/llvm-dev/2018-May/123514.html

Reported by: jbeich
PR: 230399

MFC r340287 (by emaste):

Consolidate gcov entries in OptionalObsoleteFiles

Sponsored by: The FreeBSD Foundation

MFC r340289 (by emaste):

llvm-cov: also install as gcov (if GNU gcov is disabled)

llvm-cov provides a gcov-compatible interface when invoked as gcov.

Reviewed by: dim, markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D17923

MFC r340296 (by emaste):

Move llvm-profdata build into MK_LLVM_COV block

llvm-profdata is used with llvm-cov for code coverage (although llvm-cov
can also operate independently in a gcov-compatible mode).
Although llvm-profdata can be used independently of llvm-cov it makes
sense to group these under one option.

Also handle these in OptionalObsoleteFiles.inc while here.

Sponsored by: The FreeBSD Foundation

MFC r340300 (by emaste):

libllvm: Move SampleProfWriter to SRCS_MIN

It is required by llvm-profdata, now built by default under the
LLVM_COV knob. The additional complexity that would come from avoiding
building it if CLANG_EXTRAS and LLVM_COV are both disabled is not worth
the small savings in build time.

Sponsored by: The FreeBSD Foundation

MFC r340972 (by emaste):

llvm-objdump: initial man page

Based on llvm-objdump's online documentation and usage information.
This serves as a starting point; additional detail and cleanup still
required.

Also being submitted upstream in LLVM review D54864. I expect to use
this bespoke copy while we have LLVM 6.0 or 7.0 in FreeBSD; when we
update to LLVM 8.0 it should be upstream and we will switch to it.

PR: 233437
Reviewed by: bcr (man formatting)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D18309

MFC r340973 (by emaste):

llvm-objdump.1: remove invalid options

Some options appear in llvm-objdump's usage information as a side effect
of its option parsing implementation and are not actually llvm-objdump
options. Reported in LLVM review https://reviews.llvm.org/D54864.

Reported by: Fangrui Song
Sponsored by: The FreeBSD Foundation

MFC r340975 (by emaste):

llvm-objdump.1: fix igor / mandoc -Tlint warnings

Accidentally omitted from r340972.

MFC r341055 (by emaste):

llvm-objdump.1: remove more unintentional options

Some options come from static constructors in LLVM libraries and are
automatically added to llvm's usage output. They're not really supposed
to be llvm-objdump options.

Reported by: Fangrui Song in LLVM review D54864
Sponsored by: The FreeBSD Foundation

MFC r344779:

Upgrade our copies of clang, llvm, lld, lldb, compiler-rt and libc++ to
the upstream release_80 branch r355313 (effectively, 8.0.0 rc3). The
release will follow very soon, but no more functional changes are
expected.

Release notes for llvm, clang and lld 8.0.0 will soon be available here:
<https://releases.llvm.org/8.0.0/docs/ReleaseNotes.html>
<https://releases.llvm.org/8.0.0/tools/clang/docs/ReleaseNotes.html>
<https://releases.llvm.org/8.0.0/tools/lld/docs/ReleaseNotes.html>

PR: 236062
Relnotes: yes

MFC r344798 (by emaste):

libllvm: promote WithColor and xxhash to SRCS_MIN

The armv6 build failed in CI due to missing symbols (from these two
source files) in the bootstrap Clang.

This affected only armv6 because other Clang-using archs are using LLD
as the bootstrap linker, and thus include SRCS_MIW via LLD_BOOTSTRAP.

Reported by: CI, via lwhsu
Sponsored by: The FreeBSD Foundation

MFC r344825:

Add a few missed files to the MK_LLVM_TARGET_BPF=yes case, otherwise
clang and various other executables will fail to link with undefined
symbols.

Reported by: O. Hartmann <ohartmann@walstatt.org>

MFC r344852:

Put in a temporary workaround for what is likely a gcc 6 bug (it does
not occur with gcc 7 or later). This should prevent the following error
from breaking the head-amd64-gcc CI builds:

In file included from /workspace/src/contrib/llvm/tools/lldb/source/API/SBMemoryRegionInfo.cpp:14:0:
/workspace/src/contrib/llvm/tools/lldb/include/lldb/Target/MemoryRegionInfo.h:128:54: error: 'template<class _InputIterator> lldb_private::MemoryRegionInfos::MemoryRegionInfos(_InputIterator, _InputIterator, const allocator_type&)' inherited from 'std::__1::vector<lldb_private::MemoryRegionInfo>'
using std::vector<lldb_private::MemoryRegionInfo>::vector;
^~~~~~
/workspace/src/contrib/llvm/tools/lldb/include/lldb/Target/MemoryRegionInfo.h:128:54: error: conflicts with version inherited from 'std::__1::vector<lldb_private::MemoryRegionInfo>'

Reported by: CI

MFC r344896:

Pull in r354937 from upstream clang trunk (by Jörg Sonnenberger):

Fix inline assembler constraint validation

The current constraint logic is both too lax and too strict. It fails
for input outside the [INT_MIN..INT_MAX] range, but it also
implicitly accepts 0 as value when it should not. Adjust logic to
handle both correctly.

Differential Revision: https://reviews.llvm.org/D58649

Pull in r355491 from upstream clang trunk (by Hans Wennborg):

Inline asm constraints: allow ICE-like pointers for the "n"
constraint (PR40890)

Apparently GCC allows this, and there's code relying on it (see bug).

The idea is to allow expression that would have been allowed if they
were cast to int. So I based the code on how such a cast would be
done (the CK_PointerToIntegral case in
IntExprEvaluator::VisitCastExpr()).

Differential Revision: https://reviews.llvm.org/D58821

These should fix assertions and errors when using the inline assembly
"n" constraint in certain ways.

In case of devel/valgrind, a pointer was used as the input for the
constraint, which lead to "Assertion failed: (isInt() && "Invalid
accessor"), function getInt".

In case of math/secp256k1, a very large integer value was used as input
for the constraint, which lead to "error: value '4624529908474429119'
out of range for constraint 'n'".

PR: 236216, 236194

MFC r344951:

Merge llvm, clang, compiler-rt, libc++, lld, and lldb release_80 branch
r355677 (effectively, 8.0.0 rc4), resolve conflicts, and bump version
numbers.

PR: 236062

MFC r345018:

Merge LLVM libunwind trunk r351319, from just before upstream's
release_80 branch point. Afterwards, we will merge the rest of the
changes in the actual release_80 branch.

PR: 236062

MFC r345019:

Merge LLVM libunwind release_80 branch r355677 (effectively, 8.0.0 rc4).

PR: 236062

MFC r345021:

Pull in r355854 from upstream llvm trunk (by Jonas Paulsson):

[RegAlloc] Avoid compile time regression with multiple copy hints.

As a fix for https://bugs.llvm.org/show_bug.cgi?id=40986 ("excessive
compile time building opencollada"), this patch makes sure that no
phys reg is hinted more than once from getRegAllocationHints().

This handles the case were many virtual registers are assigned to the
same physreg. The previous compile time fix (r343686) in
weightCalcHelper() only made sure that physical/virtual registers are
passed no more than once to addRegAllocationHint().

Review: Dimitry Andric, Quentin Colombet
https://reviews.llvm.org/D59201

This should fix a hang when compiling certain generated .cpp files in
the graphics/opencollada port.

PR: 236313

MFC r345068 (by jhb):

Move libunwind out of contrib/llvm/projects.

Move LLVM's libunwind to its own contrib/ directory similar to other
runtime libraries like libc++ and libcxxrt.

Reviewed by: dim, emaste
Differential Revision: https://reviews.freebsd.org/D19534

MFC r345073:

Revert r308867 (which was originally committed in the clang390-import
project branch):

Work around LLVM PR30879, which is about a bad interaction between
X86 Call Frame Optimization on i386 and libunwind, by disallowing the
optimization for i386-freebsd12.

This should fix some instances of broken exception handling when
frame pointers are omitted, in particular some unittests run during
the build of editors/libreoffice.

This hack will be removed as soon as upstream has implemented a more
permanent fix for this problem.

And indeed, after r345018 and r345019, which updated LLVM libunwind to
the most recent version, the above workaround is no longer needed. The
upstream commit which fixed this is:

https://llvm.org/viewvc/llvm-project?view=revision&revision=292723

Specifically, 32 bit (i386-freebsd) executables optimized with omitted
frame pointers and Call Frame Optimization should now behave correctly
when a C++ exception is thrown, and the stack is unwound.

Upstream PR: https://llvm.org/bugs/show_bug.cgi?id=30879
PR: 236062

MFC r345152:

Merge llvm, clang, compiler-rt, libc++, libunwind, lld, and lldb
release_80 branch r356034 (effectively, 8.0.0 rc5), resolve conflicts,
and bump version numbers.

PR: 236062

MFC r345231:

Add LLVM openmp trunk r351319 (just before the release_80 branch point)
to contrib/llvm. This is not yet connected to the build, the glue for
that will come in a follow-up commit.

PR: 236062

MFC r345232:

Bootstrap svn:mergeinfo on contrib/openmp.

PR: 236062

MFC r345233:

Merge openmp release_80 branch r356034 (effectively, 8.0.0 rc5).

PR: 236062

MFC r345234:

Add openmp __kmp_gettid() wrapper, using pthread_getthreadid_np(3).
This has also been submitted upstream.

PR: 236062

MFC r345283:

Enable building libomp.so for 32-bit x86. This is done by selectively
enabling the functions that save and restore MXCSR, since access to this
register requires SSE support.

Note that you may run into other issues with OpenMP on i386, since this
*not* yet supported upstream, and certainly not extensively tested.

PR: 236062, 236582

MFC r345345:

Merge llvm, clang, compiler-rt, libc++, libunwind, lld, lldb and openmp
8.0.0 final release r356365. There were no functional changes since the
most recent merge, of 8.0.0 rc5.

Release notes for llvm, clang, lld and libc++ 8.0.0 are now available:

https://llvm.org/releases/8.0.0/docs/ReleaseNotes.html
https://llvm.org/releases/8.0.0/tools/clang/docs/ReleaseNotes.html
https://llvm.org/releases/8.0.0/tools/lld/docs/ReleaseNotes.html
https://llvm.org/releases/8.0.0/projects/libcxx/docs/ReleaseNotes.html

PR: 236062

MFC r345349:

Pull in r352826 from upstream lld trunk (by Fangrui Song):

[ELF] Support --{,no-}allow-shlib-undefined

Summary:
In ld.bfd/gold, --no-allow-shlib-undefined is the default when
linking an executable. This patch implements a check to error on
undefined symbols in a shared object, if all of its DT_NEEDED entries
are seen.

Our approach resembles the one used in gold, achieves a good balance
to be useful but not too smart (ld.bfd traces all DSOs and emulates
the behavior of a dynamic linker to catch more cases).

The error is issued based on the symbol table, different from
undefined reference errors issued for relocations. It is most
effective when there are DSOs that were not linked with -z defs (e.g.
when static sanitizers runtime is used).

gold has a comment that some system libraries on GNU/Linux may have
spurious undefined references and thus system libraries should be
excluded (https://sourceware.org/bugzilla/show_bug.cgi?id=6811). The
story may have changed now but we make --allow-shlib-undefined the
default for now. Its interaction with -shared can be discussed in the
future.

Reviewers: ruiu, grimar, pcc, espindola

Reviewed By: ruiu

Subscribers: joerg, emaste, arichardson, llvm-commits

Differential Revision: https://reviews.llvm.org/D57385

Pull in r352943 from upstream lld trunk (by Fangrui Song):

[ELF] Default to --no-allow-shlib-undefined for executables

Summary:
This follows the ld.bfd/gold behavior.

The error check is useful as it captures a common type of ld.so
undefined symbol errors as link-time errors:

// a.cc => a.so (not linked with -z defs)
void f(); // f is undefined
void g() { f(); }

// b.cc => executable with a DT_NEEDED entry on a.so
void g();
int main() { g(); }

// ld.so errors when g() is executed (lazy binding) or when the program is started (-z now)
// symbol lookup error: ... undefined symbol: f

Reviewers: ruiu, grimar, pcc, espindola

Reviewed By: ruiu

Subscribers: llvm-commits, emaste, arichardson

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D57569

Together, these add support for --no-allow-shlib-undefined, and make it
the default for executables, so they will fail to link if any symbols
from needed shared libraries are undefined.

Reported by: jbeich
PR: 236062, 236141

MFC r345449:

Pull in r356809 from upstream llvm trunk (by Eli Friedman):

[ARM] Don't form "ands" when it isn't scheduled correctly.

In r322972/r323136, the iteration here was changed to catch cases at
the beginning of a basic block... but we accidentally deleted an
important safety check. Restore that check to the way it was.

Fixes https://bugs.llvm.org/show_bug.cgi?id=41116

Differential Revision: https://reviews.llvm.org/D59680

This should fix "Assertion failed: (LiveCPSR && "CPSR liveness tracking
is wrong!"), function UpdateCPSRUse" errors when building the devel/xwpe
port for armv7.

PR: 236062, 236568

Merge clang 7.0.1 and several follow-up changes

MFC r318594:

Add libc++experimental.a for std::experimental support

This adds a separate library for supporting std::experimental features.
It is purposefully static, and must be explicitly linked into programs
using -lc++experimental.

PLEASE NOTE: there is NO WARRANTY as to any stability or continuing
existence of the features in the std::experimental parts of the C++
library!

Reviewed by: ed
Differential Revision: https://reviews.freebsd.org/D10840

MFC r318598:

Add PICFLAG to build libc++experimental.a, so it can be used in all
situations.

Noticed by: kib

r336969 | emaste | 2018-07-31 16:12:09 +0200 (Tue, 31 Jul 2018) | 13 lines

llvm: [ELF][ARM] Add Arm ABI names for float ABI ELF Header flags

The ELF for the Arm architecture document defines, for EF_ARM_EABI_VER5
and above, the flags EF_ARM_ABI_FLOAT_HARD and EF_ARM_ABI_FLOAT_SOFT.
These have been defined to be compatible with the existing
EF_ARM_VFP_FLOAT and EF_ARM_SOFT_FLOAT used by gcc for
EF_ARM_EABI_UNKNOWN.

This patch adds the flags in addition to the existing ones so that any
code depending on the old names will still work.

Obtained from: llvm r338370 by Peter Smith

r336970 | emaste | 2018-07-31 16:14:41 +0200 (Tue, 31 Jul 2018) | 9 lines

llvm: [ARM] Complete enumeration values for Tag_ABI_VFP_args

The LLD implementation of Tag_ABI_VFP_args needs to check the rarely
seen values of 3 (toolchain specific) and 4 compatible with both Base
and VFP. Add the missing enumeration values so that LLD can refer to
them without having to use the raw numbers.

Obtained from: llvm r338373 by Peter Smith

r336972 | emaste | 2018-07-31 17:25:03 +0200 (Tue, 31 Jul 2018) | 37 lines

lld: [ELF][ARM] Implement support for Tag_ABI_VFP_args

The Tag_ABI_VFP_args build attribute controls the procedure call
standard used for floating point parameters on ARM. The values are:

0 - Base AAPCS (FP Parameters passed in Core (Integer) registers
1 - VFP AAPCS (FP Parameters passed in FP registers)
2 - Toolchain specific (Neither Base or VFP)
3 - Compatible with all (No use of floating point parameters)

If the Tag_ABI_VFP_args build attribute is missing it has an implicit
value of 0.

We use the attribute in two ways:

* Detect a clash in calling convention between Base, VFP and Toolchain.

we follow ld.bfd's lead and do not error if there is a clash between an
implicit Base AAPCS caused by a missing attribute. Many projects
including the hard-float (VFP AAPCS) version of glibc contain assembler
files that do not use floating point but do not have Tag_ABI_VFP_args.

* Set the EF_ARM_ABI_FLOAT_SOFT or EF_ARM_ABI_FLOAT_HARD ELF header flag

for Base or VFP AAPCS respectively. This flag is used by some ELF
loaders.

References:
* Addenda to, and Errata in, the ABI for the ARM Architecture for
Tag_ABI_VFP_args
* Elf for the ARM Architecture for ELF header flags

Fixes LLVM PR36009

PR: 229050
Obtained from: llvm r338377 by Peter Smith

r337282 | alc | 2018-08-04 04:30:51 +0200 (Sat, 04 Aug 2018) | 7 lines

Set the default image base on arm64 and i386 to a superpage-aligned
address.

Reviewed by: emaste, markj
Discussed with: dim
Differential Revision: https://reviews.freebsd.org/D16385

r339304 | emaste | 2018-10-11 15:19:17 +0200 (Thu, 11 Oct 2018) | 13 lines

lld: set sh_link and sh_info for .rela.plt sections

ELF spec says that for SHT_REL and SHT_RELA sh_link should reference the
associated string table and sh_info should reference the "section to
which the relocation applies." ELF Tool Chain's elfcopy / strip use
this (in part) to control whether or not the relocation entry is copied
to the output.

LLVM PR 37538 https://bugs.llvm.org/show_bug.cgi?id=37538

Approved by: re (kib)
Obtained from: llvm r344226 (backported for 6.0)

MFC r341825:

Upgrade our copies of clang, llvm, lld, lldb, compiler-rt and libc++ to
the upstream release_70 branch r348686 (effectively, 7.0.1 rc3). The
release will follow very soon, but no more functional changes are
expected.

Release notes for llvm, clang and lld 7.0.0 are available here:
<http://releases.llvm.org/7.0.0/docs/ReleaseNotes.html>
<http://releases.llvm.org/7.0.0/tools/clang/docs/ReleaseNotes.html>
<http://releases.llvm.org/7.0.0/tools/lld/docs/ReleaseNotes.html>

PR: 230240, 230355
Relnotes: yes

MFC r342123:

Update clang, llvm, lld, lldb, compiler-rt and libc++ version number to
7.0.1 release r349250. There were no functional changes since the 7.0.1
rc3 import.

PR: 230240, 230355
Relnotes: yes

r343429 | emaste | 2019-01-25 15:46:13 +0100 (Fri, 25 Jan 2019) | 16 lines

clang: default to DWARF 4 as of FreeBSD 13

FreeBSD previously defaulted to DWARF 2 because several tools (gdb,
ctfconvert, etc.) did not support later versions. These have either
been fixed or are deprecated.

Note that gdb 6 still exists but has been moved out of $PATH into
/usr/libexec and is intended only for use by crashinfo(8). The kernel
build sets the DWARF version explicitly via -gdwarf2, so this should
have no effect there.

PR: 234887 [exp-run]
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D17930

MFC r343916:

Pull in r352607 from upstream llvm trunk (by Craig Topper):

[X86] Add FPSW as a Def on some FP instructions that were missing it.

Pull in r353141 from upstream llvm trunk (by Craig Topper):

[X86] Connect the default fpsr and dirflag clobbers in inline
assembly to the registers we have defined for them.

Summary:
We don't currently map these constraints to physical register numbers
so they don't make it to the MachineIR representation of inline
assembly.

This could have problems for proper dependency tracking in the
machine schedulers though I don't have a test case that shows that.

Reviewers: rnk

Reviewed By: rnk

Subscribers: eraman, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D57641

Pull in r353489 from upstream llvm trunk (by Craig Topper):

[X86] Add FPCW as a register and start using it as an implicit use on
floating point instructions.

Summary:
FPCW contains the rounding mode control which we manipulate to
implement fp to integer conversion by changing the roudning mode,
storing the value to the stack, and then changing the rounding mode
back. Because we didn't model FPCW and its dependency chain, other
instructions could be scheduled into the middle of the sequence.

This patch introduces the register and adds it as an implciit def of
FLDCW and implicit use of the FP binary arithmetic instructions and
store instructions. There are more instructions that need to be
updated, but this is a good start. I believe this fixes at least the
reduced test case from PR40529.

Reviewers: RKSimon, spatel, rnk, efriedma, andrew.w.kaylor

Subscribers: dim, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D57735

These should fix a problem in clang 7.0 where it would sometimes emit
long double floating point instructions in a slightly wrong order,
leading to failures in our libm tests. In particular, the cbrt_test
test case 'cbrtl_powl' and the trig_test test case 'reduction'.

Also bump __FreeBSD_cc_version, to be able to detect this in our test
suite.

Reported by: lwhsu
PR: 234040
Upstream PR: https://bugs.llvm.org/show_bug.cgi?id=40206

MFC r344056:

Pull in r339734 from upstream llvm trunk (by Eli Friedman):

[ARM] Make PerformSHLSimplify add nodes to the DAG worklist correctly.

Intentionally excluding nodes from the DAGCombine worklist is likely
to lead to weird optimizations and infinite loops, so it's generally
a bad idea.

To avoid the infinite loops, fix DAGCombine to use the
isDesirableToCommuteWithShift target hook before performing the
transforms in question, and implement the target hook in the ARM
backend disable the transforms in question.

Fixes https://bugs.llvm.org/show_bug.cgi?id=38530 . (I don't have a
reduced testcase for that bug. But we should have sufficient test
coverage for PerformSHLSimplify given that we're not playing weird
tricks with the worklist. I can try to bugpoint it if necessary,
though.)

Differential Revision: https://reviews.llvm.org/D50667

This should fix a possible hang when compiling sys/dev/nxge/if_nxge.c
(which exists now only in the stable/11 branch) for arm.

Revert r342807, mistakenly including libbe(3)/bectl(8) MFC

MFC r342362-r342363: config(8) duplicate option handling

r342362:
config(8): Allow duplicate options to be specified

config(8)'s option handling has been written to allow duplicate options; if
the value changes, then the latest value is used and an informative message
is printed to stderr like so:

/usr/src/sys/amd64/conf/TEST: option "VERBOSE_SYSINIT" redefined from 0 to 1

Currently, this is only a possibility for cpu types, MAXUSERS, and
MACHINE_ARCH. Anything else duplicated in a config file will use the first
value set and error about duplicated options on subsequent appearances,
which is arguably unfriendly since one could specify:

include GENERIC
nooptions VERBOSE_SYSINIT
options VERBOSE_SYSINIT

to redefine the value later anyways.

Reported by: mmacy

r342363:
config(8): Remove all instances of an option when opting out

Quick follow-up to r342362: options can appear multiple times now, so
clean up all of them as needed. For non-OPTIONS options, this has no effect
since they're already de-duplicated.

MFC r335253:
Rework ofed build.

Sponsored by: Mellanox Technologies

MFC r334277, r334376, r334378 and r334418:

MFV r333789: libpcap 1.9.0 (pre-release)

- Update local copy of dlt.h with new DLT types.

Sponsored by: Mellanox Technologies

MFC r320872:
Create libdl.so.1 as a filter for libc.so.7 which exports public dl*
functions.

MFC r306657, r306673, r306726, r307737, r309366, r310135, r323990, r324414

r306657:
libcapsicum: introduce Capsicum helpers

Capsicum helpers are a set of inline functions which goal is to reduce
duplicated patterns used to Capsicumize applications.

Reviewed by: cem, AllanJude, bapt, ed, emaste
Differential Revision: https://reviews.freebsd.org/D8013

r306673:
libcapsicum: limit stderr

Don't limit stdout twice, instead limit stderr.

Pointed out by: rpokala@

r306726:
Add man pages for Capsicum helpers.

Reviewed by: cem
Differential Revision: https://reviews.freebsd.org/D8154

r307737:
Fix few sentence in the man page.

Pointed out by: wblock

r309366:
capsicum_helpers: Squash errors from closed fds

Squash EBADF from closed stdin, stdout, or stderr in caph_limit_stdio().
Any program used during special shell scripts may commonly be forked
from a parent process with closed standard stream. Do the common sense
thing for this common use.

Reported by: Iblis Lin <iblis AT hs.ntnu.edu.tw>
Reviewed by: oshogbo@ (earlier version)
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D8657

r310135:
capsicum_helpers: Add LOOKUP flag

Add a helper routine for opening a directory that is restricted to being
used for opening relative files as stdio streams.

I think this will really help basic adaptation of multi-file programs to
Capsicum. Rather than having each program initialize a rights object and
ioctl/fcntl arrays for their root fd for relative opens, consolidate in the
logical place.

Reviewed by: oshogbo@
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D8743

r323990:
capsicum_helpers: Add SEEK to default stdio rights set

PR: 219173
Sponsored by: Dell EMC Isilon

r324414:
capsicum_helpers: Add EVENT to default stdio rights set

Without it, calling caph_limit_stdio(3) breaks Irssi.

Reviewed by: oshogbo
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D12622

MFC Loader Fixes 2017q4p2: r324453, r324454

r324453: Create sys/boot/libsa and build libstand.a there

r324454: Disconnect libstand from the build.

MFC r326030: Install strings unconditionally

Previously it was enabled by WITH_/WITHOUT_TOOLCHAIN, but it is commonly
expected to be available and may have non-toolchain consumers. As it
is now taken from the BSD-licensed ELF Tool Chain project, just install
it unconditionally.

PR: 213665, 223725
Sponsored by: The FreeBSD Foundation

MFC r306375,r307802:

r306375 (by emaste):

Add a WITHOUT_DIALOG src.conf(5) knob

It also turns off dependencies (bsdinstall, bsdconfig, dpv, tzsetup).

r307802 (by bapt):

Fix build of tzsetup when WITHOUT_DIALOG is set

Hide dialog specific code behind HAVE_DIALOG. It allows to build a stripped
down version (missing the dialog UI) but perfectly function tzsetup when
world is built WITHOUT_DIALOG

Reorganise a bit the code to limit the number of blocks under HAVE_DIALOG

MFC efivar(8) (by imp):

List of revisions merged:
r307070
r307071
r307072
r307074
r307189
r307224
r307339
r307390
r307391
r309776
r314231
r314232
r314615
r314616
r314617
r314618
r314619
r314620
r314621
r314623
r314890
r314925
r314926
r314927
r314928
r315770
r315771

Discussed with: gjb (re), imp
Sponsored by: The FreeBSD Foundation

MFC r312937:

Fix typo in lib/Makefile

The SUBDIR_DEPEND variable should be for librpcsec_gss, not
liblibrpc_gss

PR: 216409

MFC r304647:

Rename ORDERED to BOOTSTRAP since no order is respected in the list.

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Separate BLACKLIST vs BLACKLIST_SUPPORT properly

Sponsored by: The FreeBSD Foundation

Remove libstdc++ again.

This was mis-merged in r298107 which missed r289389.

Add basic blacklist build support

Reviewed by: rpaulo
Approved by: rpaulo
Relnotes: YES
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D5913

zfsd(8), the ZFS fault management daemon

Add zfsd, which deals with hard drive faults in ZFS pools. It manages
hotspares and replements in drive slots that publish physical paths.

cddl/usr.sbin/zfsd
Add zfsd(8) and its unit tests

cddl/usr.sbin/Makefile
Add zfsd to the build

lib/libdevdctl
A C++ library that helps devd clients process events

lib/Makefile
share/mk/bsd.libnames.mk
share/mk/src.libnames.mk
Add libdevdctl to the build. It's a private library, unusable by
out-of-tree software.

etc/defaults/rc.conf
By default, set zfsd_enable to NO

etc/mtree/BSD.include.dist
Add a directory for libdevdctl's include files

etc/mtree/BSD.tests.dist
Add a directory for zfsd's unit tests

etc/mtree/BSD.var.dist
Add /var/db/zfsd/cases, where zfsd stores case files while it's shut
down.

etc/rc.d/Makefile
etc/rc.d/zfsd
Add zfsd's rc script

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c
Fix the resource.fs.zfs.statechange message. It had a number of
problems:

It was only being emitted on a transition to the HEALTHY state.
That made it impossible for zfsd to take actions based on drives
getting sicker.

It compared the new state to vdev_prevstate, which is the state that
the vdev had the last time it was opened. That doesn't make sense,
because a vdev can change state multiple times without being
reopened.

vdev_set_state contains logic that will change the device's new
state based on various conditions. However, the statechange event
was being posted _before_ that logic took effect. Now it's being
posted after.

Submitted by: gibbs, asomers, mav, allanjude
Reviewed by: mav, delphij
Relnotes: yes
Sponsored by: Spectra Logic Corp, iX Systems
Differential Revision: https://reviews.freebsd.org/D6564

Merge the projects/release-pkg branch to head.

This allows packaging the base system with pkg(8), including
but not limited to providing the ability to provide upstream
binary update possibilities for non-tier-1 architectures.

This merge is a requirement of the 11.0-RELEASE, and as such,
thank you to everyone that has tested the project branch.

Documentation in build(7) etc. is still somewhat sparse, but
updates to those parts will follow.

Sponsored by: The FreeBSD Foundation

Use the newly minted Makefile.libcompat to implement libsoft libraries
for the armv6 ABI switch. This also make WITH_LIBSOFT functional on
the arm platform. As a transition thing, this seems to work even
without switching the ABI (we basically build the same libraries
twice when MK_LIBSOFT=yes until the ABI cut over next
month). MK_LIBSOFT remains default no.

Revert r296395.

This mistakenly removed the SUBDIR_PARALLEL but even worse is that the install
(and build) order is not correct due to the lack of SUBDIR_DEPEND on the
most critical libraries. The only reason they build correctly now is because
buildworld's 'make libraries' orders them properly.

Pointyhat to: bdrewery
Sponsored by: EMC / Isilon Storage Division

Parallel installation has been safe here since r267511 added SUBDIR_DEPEND.

Sponsored by: EMC / Isilon Storage Division

Convert casperd(8) daemon to the libcasper.
After calling the cap_init(3) function Casper will fork from it's original
process, using pdfork(2). Forking from a process has a lot of advantages:
1. We have the same cwd as the original process.
2. The same uid, gid and groups.
3. The same MAC labels.
4. The same descriptor table.
5. The same routing table.
6. The same umask.
7. The same cpuset(1).
From now services are also in form of libraries.
We also removed libcapsicum at all and converts existing program using Casper
to new architecture.

Discussed with: pjd, jonathan, ed, drysdale@google.com, emaste
Partially reviewed by: drysdale@google.com, bdrewery
Approved by: pjd (mentor)
Differential Revision: https://reviews.freebsd.org/D4277

Add libpe for elfcopy(1) PE/COFF support

Sponsored by: The FreeBSD Foundation

Style.

Do build libproc and librtld_db for RISC-V as well.

Make the building of libsmb and mount_smbfs unconditional, now that r292552
has eliminated alignment and endian problems that were making it fail on
some platforms.

PR: 180438
PR: 189415

Start on a new library (libsysdecode) that provides routines for decoding
system call information such as system call arguments. Initially this
will consist of pulling duplicated code out of truss and kdump though it
may prove useful for other utilities in the future.

This commit moves the shared utrace(2) record parser out of kdump into
the library and updates kdump and truss to use it. One difference from
the previous version is that the library version treats unknown events
that start with the "RTLD" signature as unknown events. This simplifies
the interface and allows the consumer to decide how to handle all
non-recognized events. Instead, this function only generates a string
description for known malloc() and RTLD records.

Reviewed by: bdrewery
Differential Revision: https://reviews.freebsd.org/D4537

Move obscure lib/ installation of /usr/lib/include symlink to include/.

This avoids the need for an afterinstall: hook and a check for LIBRARIES_ONLY.
It also now respects INCLUDEDIR.

This came in r249484.

Sponsored by: EMC / Isilon Storage Division

Add lib80211, a small (but hopefully soon to grow) set of library
routines to interface with net80211.

This is all from the ifconfig program; the duplicate code from ifconfig
will be removed when it starts using this API.

Differential Revision: https://reviews.freebsd.org/D4290

Always build libevent, which fixes the build for usr.sbin/ypldap after r291230.

The dependency for ypldap is only with WITH_NIS, but libevent is small
enough to not warrant the extra logic and maintenance cost here so just
always build it.

Sponsored by: EMC / Isilon Storage Division

Rename libohash to libopenbsd.

libopenbsd is an internal library which
to bring in compatibility stuff from OpenBSD.
This will allow us to bring in more
OpenBSD utilities into the FreeBSD base system.

We similarly use libnetbsd for bringing in stuff from NetBSD.

Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D4078

libstdc++ also snook in incorrectly in r267511, despite not being a real
subdir.

Sponsored by: EMC / Isilon Storage Division

Remove unneeded libg++ reference that came in with r267511 based on a removed
comment.

Sponsored by: EMC / Isilon Storage Division

Fix another ++= parsed as '+=', missed in r289384.

Sponsored by: EMC / Isilon Storage Division

Let the SUBDIR_DEPEND*c++ variables actually work rather than being parsed
as a +=. These were safe due to a .WAIT very early on.

Sponsored by: EMC / Isilon Storage Division

Roll WITHOUT_ELFTOOLCHAIN_TOOLS into WITHOUT_TOOLCHAIN

The option was added only to ease the transition from GNU Binutils to
ELF Tool Chain tools, and that process is now complete (for the viable
replacements). Noting the removal in UPDATING is sufficient as we have
not shipped a release with the option.

Reviewed by: brooks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3240

First cut of DTrace for AArch64.

Reviewed by: andrew, emaste
Sponsored by: ARM Limited
Differential Revision: https://reviews.freebsd.org/D2738

Remove SUBDIR_DEPEND for non-existent libreadline directory

libreadline lives under gnu/lib

MFC after: 1 week

Drop libmandoc and incorporate it into the main mandoc Makefile

This simplifies maintainance of mandoc(1).
Note that the same direction was taken on OpenBSD

Remove redundant csu subdir logic

The appropriate subdirectories are handled by lib/csu/Makefile. There's
no need to duplicate this logic in Makefile.inc1 and lib/Makefile.

Reviewed by: imp
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D2523

Significant upgrades to sa(4) and mt(1).

The primary focus of these changes is to modernize FreeBSD's
tape infrastructure so that we can take advantage of some of the
features of modern tape drives and allow support for LTFS.

Significant changes and new features include:

o sa(4) driver status and parameter information is now exported via an
XML structure. This will allow for changes and improvements later
on that will not break userland applications. The old MTIOCGET
status ioctl remains, so applications using the existing interface
will not break.

o 'mt status' now reports drive-reported tape position information
as well as the previously available calculated tape position
information. These numbers will be different at times, because
the drive-reported block numbers are relative to BOP (Beginning
of Partition), but the block numbers calculated previously via
sa(4) (and still provided) are relative to the last filemark.
Both numbers are now provided. 'mt status' now also shows the
drive INQUIRY information, serial number and any position flags
(BOP, EOT, etc.) provided with the tape position information.
'mt status -v' adds information on the maximum possible I/O size,
and the underlying values used to calculate it.

o The extra sa(4) /dev entries (/dev/saN.[0-3]) have been removed.

The extra devices were originally added as place holders for
density-specific device nodes. Some OSes (NetBSD, NetApp's OnTap
and Solaris) have had device nodes that, when you write to them,
will automatically select a given density for particular tape drives.

This is a convenient way of switching densities, but it was never
implemented in FreeBSD. Only the device nodes were there, and that
sometimes confused users.

For modern tape devices, the density is generally not selectable
(e.g. with LTO) or defaults to the highest availble density when
the tape is rewritten from BOT (e.g. TS11X0). So, for most users,
density selection won't be necessary. If they do need to select
the density, it is easy enough to use 'mt density' to change it.

o Protection information is now supported. This is either a
Reed-Solomon CRC or CRC32 that is included at the end of each block
read and written. On write, the tape drive verifies the CRC, and
on read, the tape drive provides a CRC for the userland application
to verify.

o New, extensible tape driver parameter get/set interface.

o Density reporting information. For drives that support it,
'mt getdensity' will show detailed information on what formats the
tape drive supports, and what formats the tape drive supports.

o Some mt(1) functionality moved into a new mt(3) library so that
external applications can reuse the code.

o The new mt(3) library includes helper routines to aid in parsing
the XML output of the sa(4) driver, and build a tree of driver
metadata.

o Support for the MTLOAD (load a tape in the drive) and MTWEOFI
(write filemark immediate) ioctls needed by IBM's LTFS
implementation.

o Improve device departure behavior for the sa(4) driver. The previous
implementation led to hangs when the device was open.

o This has been tested on the following types of drives:
IBM TS1150
IBM TS1140
IBM LTO-6
IBM LTO-5
HP LTO-2
Seagate DDS-4
Quantum DLT-4000
Exabyte 8505
Sony DDS-2

contrib/groff/tmac/doc-syms,
share/mk/bsd.libnames.mk,
lib/Makefile,
Add libmt.

lib/libmt/Makefile,
lib/libmt/mt.3,
lib/libmt/mtlib.c,
lib/libmt/mtlib.h,
New mt(3) library that contains functions moved from mt(1) and
new functions needed to interact with the updated sa(4) driver.

This includes XML parser helper functions that application writers
can use when writing code to query tape parameters.

rescue/rescue/Makefile:
Add -lmt to CRUNCH_LIBS.

src/share/man/man4/mtio.4
Clarify this man page a bit, and since it contains what is
essentially the mtio.h header file, add new ioctls and structure
definitions from mtio.h.

src/share/man/man4/sa.4
Update BUGS and maintainer section.

sys/cam/scsi/scsi_all.c,
sys/cam/scsi/scsi_all.h:
Add SCSI SECURITY PROTOCOL IN/OUT CDB definitions and CDB building
functions.

sys/cam/scsi/scsi_sa.c
sys/cam/scsi/scsi_sa.h
Many tape driver changes, largely outlined above.

Increase the sa(4) driver read/write timeout from 4 to 32
minutes. This is based on the recommended values for IBM LTO
5/6 drives. This may also avoid timeouts for other tape
hardware that can take a long time to do retries and error
recovery. Longer term, a better way to handle this is to ask
the drive for recommended timeout values using the REPORT
SUPPORTED OPCODES command. Modern IBM and Oracle tape drives
at least support that command, and it would allow for more
accurate timeout values.

Add XML status generation. This is done with a series of
macros to eliminate as much duplicate code as possible. The
new XML-based status values are reported through the new
MTIOCEXTGET ioctl.

Add XML driver parameter reporting, using the new MTIOCPARAMGET
ioctl.

Add a new driver parameter setting interface, using the new
MTIOCPARAMSET and MTIOCSETLIST ioctls.

Add a new MTIOCRBLIM ioctl to get block limits information.

Add CCB/CDB building routines scsi_locate_16, scsi_locate_10,
and scsi_read_position_10().

scsi_locate_10 implements the LOCATE command, as does the
existing scsi_set_position() command. It just supports
additional arguments and features. If/when we figure out a
good way to provide backward compatibility for older
applications using the old function API, we can just revamp
scsi_set_position(). The same goes for
scsi_read_position_10() and the existing scsi_read_position()
function.

Revamp sasetpos() to take the new mtlocate structure as an
argument. It now will use either scsi_locate_10() or
scsi_locate_16(), depending upon the arguments the user
supplies. As before, once we change position we don't have a
clear idea of what the current logical position of the tape
drive is.

For tape drives that support long form position data, we
read the current position and store that for later reporting
after changing the position. This should help applications
like Bacula speed tape access under FreeBSD once they are
modified to support the new ioctls.

Add a new quirk, SA_QUIRK_NO_LONG_POS, that is set for all
drives that report SCSI-2 or older, as well as drives that
report an Illegal Request type error for READ POSITION with
the long format. So we should automatically detect drives
that don't support the long form and stop asking for it after
an initial try.

Add a partition number to the sa(4) softc.

Improve device departure handling. The previous implementation
led to hangs when the device was open.

If an application had the sa(4) driver open, and attempted to
close it after it went away, the cam_periph_release() call in
saclose() would cause the periph to get destroyed because that
was the last reference to it. Because destroy_dev() was
called from the sa(4) driver's cleanup routine (sacleanup()),
and would block waiting for the close to happen, a deadlock
would result.

So instead of calling destroy_dev() from the cleanup routine,
call destroy_dev_sched_cb() from saoninvalidate() and wait for
the callback.

Acquire a reference for devfs in saregister(), and release it
in the new sadevgonecb() routine when all devfs devices for
the particular sa(4) driver instance are gone.

Add a new function, sasetupdev(), to centralize setting
per-instance devfs device parameters instead of repeating the
code in saregister().

Add an open count to the softc, so we know how many
peripheral driver references are a result of open
sessions.

Add the D_TRACKCLOSE flag to the cdevsw flags so
that we get a 1:1 mapping of open to close calls
instead of a N:1 mapping.

This should be a no-op for everything except the
control device, since we don't allow more than one
open on non-control devices.

However, since we do allow multiple opens on the
control device, the combination of the open count
and the D_TRACKCLOSE flag should result in an
accurate peripheral driver reference count, and an
accurate open count.

The accurate open count allows us to release all
peripheral driver references that are the result
of open contexts once we get the callback from devfs.

sys/sys/mtio.h:
Add a number of new mt(4) ioctls and the requisite data
structures. None of the existing interfaces been removed
or changed.

This includes definitions for the following new ioctls:

MTIOCRBLIM /* get block limits */
MTIOCEXTLOCATE /* seek to position */
MTIOCEXTGET /* get tape status */
MTIOCPARAMGET /* get tape params */
MTIOCPARAMSET /* set tape params */
MTIOCSETLIST /* set N params */

usr.bin/mt/Makefile:
mt(1) now depends on libmt, libsbuf and libbsdxml.

usr.bin/mt/mt.1:
Document new mt(1) features and subcommands.

usr.bin/mt/mt.c:
Implement support for mt(1) subcommands that need to
use getopt(3) for their arguments.

Implement a new 'mt status' command to replace the old
'mt status' command. The old status command has been
renamed 'ostatus'.

The new status function uses the MTIOCEXTGET ioctl, and
therefore parses the XML data to determine drive status.
The -x argument to 'mt status' allows the user to dump out
the raw XML reported by the kernel.

The new status display is mostly the same as the old status
display, except that it doesn't print the redundant density
mode information, and it does print the current partition
number and position flags.

Add a new command, 'mt locate', that will supersede the
old 'mt setspos' and 'mt sethpos' commands. 'mt locate'
implements all of the functionality of the MTIOCEXTLOCATE
ioctl, and allows the user to change the logical position
of the tape drive in a number of ways. (Partition,
block number, file number, set mark number, end of data.)
The immediate bit and the explicit address bits are
implemented, but not documented in the man page.

Add a new 'mt weofi' command to use the new MTWEOFI ioctl.
This allows the user to ask the drive to write a filemark
without waiting around for the operation to complete.

Add a new 'mt getdensity' command that gets the XML-based
tape drive density report from the sa(4) driver and displays
it. This uses the SCSI REPORT DENSITY SUPPORT command
to get comprehensive information from the tape drive about
what formats it is able to read and write.

Add a new 'mt protect' command that allows getting and setting
tape drive protection information. The protection information
is a CRC tacked on to the end of every read/write from and to
the tape drive.

Sponsored by: Spectra Logic
MFC after: 1 month

Initial version of DTrace on ARM32.

Submitted by: Howard Su based on work by Oleksandr Tymoshenko
Reviewed by: ian, andrew, rpaulo, markj

Merge xz 5.2.0.

This brings support for multi-threaded compression. This brings close
N times faster compression where N is the number of CPU cores.
Because of this, liblzma now depends on libthr.

Soon libarchive will be modified to use the new lzma API.

Thanks to antoine@ for the exp-run.

Differential Revision: https://reviews.freebsd.org/D1786
Reviewed by: bapt

Add a new device control utility for new-bus devices called devctl. This
allows the user to request administrative changes to individual devices
such as attach or detaching drivers or disabling and re-enabling devices.
- Add a new /dev/devctl2 character device which uses ioctls for device
requests. The ioctls use a common 'struct devreq' which is somewhat
similar to 'struct ifreq'.
- The ioctls identify the device to operate on via a string. This
string can either by the device's name, or it can be a bus-specific
address. (For unattached devices, a bus address is the only way to
locate a device.) Bus drivers register an eventhandler to claim
unrecognized device names that the driver recognizes as a valid address.
Two buses currently support addresses: ACPI recognizes any device
in the ACPI namespace via its full path starting with "\" and
the PCI bus driver recognizes an address specification of
'pci[<domain>:]<bus>:<slot>:<func>' (identical to the PCI selector
strings supported by pciconf).
- To make it easier to cut and paste, change the PnP location string
in the PCI bus driver to output a full PCI selector string rather
than 'slot=<slot> function=<func>'.
- Add a devctl(3) interface in libdevctl which provides a wrapper around
the ioctls and is the preferred interface for other userland code.
- Add a devctl(8) program which is a simple wrapper around the requests
supported by devctl(3).
- Add a device_is_suspended() function to check DF_SUSPENDED.
- Add a resource_unset_value() function that can be used to remove a
hint from the kernel environment. This is used to clear a
hint.<driver>.<unit>.disabled hint when re-enabling a boot-time
disabled device.

Reviewed by: imp (parts)
Requested by: imp (changing PCI location string)
Relnotes: yes

Add MK_FILE to control whether or not to build file(1), libmagic(3), etc

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

Conditionalize building radius support into libpam, ppp, etc via
MK_RADIUS_SUPPORT

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Add MK_BHYVE knob for building and installing bhyve(4), et al

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

Build lib/libgpio if MK_GPIO != no

Fill in corresponding entries for MK_GPIO == no in OptionalObsoleteFiles.inc

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

Connect libclang_rt to the build, for specific architectures. This
contains the libraries for Address Sanitizer (asan), Undefined Behavior
Sanitizer (ubsan) and Profile Guided Optimization.

ASan is a fast memory error detector. It can detect the following types
of bugs:

Out-of-bounds accesses to heap, stack and globals
Use-after-free
Use-after-return (to some extent)
Double-free, invalid free
Memory leaks (experimental)

Typical slowdown introduced by AddressSanitizer is 2x.

UBSan is a fast and compatible undefined behavior checker. It enables a
number of undefined behavior checks that have small runtime cost and no
impact on address space layout or ABI.

PLEASE NOTE: the sanitizers still have some rough edges on FreeBSD,
particularly on i386. These will hopefully be smoothed out in the
coming time.

Differential Revision: https://reviews.freebsd.org/D1505

Deorbit the IEEE-488/GPIB support.

Build infrastructure for elftoolchain tools

Set WITH_ELFTOOLCHAIN_TOOLS in src.conf to use the elftoolchain version
of the following tools:

* addr2line
* elfcopy (strip / mcs)
* nm
* size
* strings

Reviewed by: bapt (earlier version)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D1224

Import libgpio.

This is a thin wrapper around the kernel interface which should make
it easier to write GPIO applications. gpioctl(8) will be converted to
use this library in a separate commit.

Differential Revision: https://reviews.freebsd.org/D1183
Reviewed by: adrian, loos
Discussed on: arm@, embedded@
Relnotes: yes

Promote SQLite3 as a privatelib as it will also be used by mandoc

While here ensure sqlite3 is using pread(2) and enable the suppot for FTS4

Re-enable dpv(1,3): Introduced via r274116; temporarily disabled
shortly thereafter via r274124 until I could get the right recipe
down w/respect to SUBDIR_DEPEND.

Thanks to: ngie, ian
Reviewed by: ian
MFC after: 21 days
X-MFC-to: stable/10 stable/9
X-MFC-with: 274116 274120 274121 274123 274144 274146

Upon second-thought (following r274144), remove spurious (unused)
line-noise (libdialog never lived in lib/ -- but rather the noise
came from translating a comment that was introduced 16 years ago
via r40306; translation from comment to code occurred via r267511).

MFC after: 3 days
Reviewed by: ngie
X-MFC-to: stable/10

Fix code-typo; introduced by r267511

MFC after: 3 days
X-MFC-to: stable/10

Temporarily _disable_ compilation of dpv(3) and dpv(1).
Will revisit this to find out how to solve the ordering
issue in buildworld (potentially `make -j' specific).

Reviewed by: shurd

Add new libraries/utilities for data throughput visualization.
dpv(3): dialog progress view library
dpv(1): stream data from stdin or multiple paths with dialog progress view
figpar(3): configuration file parsing library

Reviews: D714
Reviewed by: jelischer, shurd
Discussed at: MeetBSD California 2014 Vendor/Dev Summit
Discussed on: -current
MFC after: 21 days
X-MFC-to: stable/10 stable/9

Hook libxo to the build.

Sponsored by: Juniper Networks, Inc.

My previous commit exposed an issue as it fixed a different
issue. lib/atf isn't a prereq_lib, since it isn't required for other
libraries to build. Remove it. The old kludge of always building it
had effectively been retired. Since we don't want to build the
libraries with the tests when we're bootstrapping, invent
MK_TESTS_SUPPORT which normally defaults to the current MK_TESTS
value, except when explicitly defined. Make lib/atf depend on it being
yes. When building the libraries set MK_TESTS to no, and
MK_TESTS_SUPPORT to the current value of MK_TESTS so that later stages
of the build work correctly. This should fix (and does for me)
people's issues with parallel builds racing between lib/atf and
libexec/atf. Since lib/atf is built during the libraries phase, the
race disappears.

Promote libevent to lib/ level and fold ftp-proxy into its parent Makefile.
This allows us to use libevent for other application in the future.

For now libevent is still INTERNALLIB and no shared library is installed.

MFC after: 1 month

Make libohash a proper internallib instead of hidding it in m4

mandoc development seems to be also using ohash so prepare to share the code when
needed

Remove ia64.

This includes:
o All directories named *ia64*
o All files named *ia64*
o All ia64-specific code guarded by __ia64__
o All ia64-specific makefile logic
o Mention of ia64 in comments and documentation

This excludes:
o Everything under contrib/
o Everything under crypto/
o sys/xen/interface
o sys/sys/elf_common.h

Discussed at: BSDcan

Create a mechanism for providing fine-grained build order dependencies
during SUBDIR_PARALLEL builds. This augments the coarse .WAIT mechanism,
which is still useful if you've got a situation such as "almost everything
depends on A and B".

Because the parallel subdir mechanism uses non-obvious mangling of
target names, which should probably remain a private detail of the
implementation, it's not easy to do things like "libfoo: libbar", so
instead the new mechanism lets you set a variable that lists dependencies:

SUBDIR_DEPEND_libfoo= libgroodah libpouet

Note that while I'm using libraries as an example here, it really has
nothing to do with the generated library files. This is really saying
"build in directory libfoo after building in the libgroodah and libpouet
directories."

This updates lib/Makefile with dependency information based on the old
almost-accurate comment block and by combing through lib/* makefiles
looking for LDADD dependencies to other libraries within lib/*.

Reviewed by: Jia-Shiun Li <jiashiun@gmail.com>

Attach the CUSE library and kernel module to the default FreeBSD
builds. Bump the FreeBSD version number.

Sprinkle a few more .WAITs into the mix after csu, libc, msun and the
early built libraries. This should be sufficient for most cases and
has eliminated the issues I've seen with high -j builds. Races likely
still remain, but this knocks the problem down a notch.

Use src.opts.mk in preference to bsd.own.mk except where we need stuff
from the latter.

lib: Remove duplicate SUBDIR libnv.

With SUBDIR_PARALLEL, duplicates in ${SUBDIR} cause harmless but verbose
warnings.

Add a SUBDIR_PARALLEL option to bsd.subdir.mk, to allow make to process
all the SUBDIR entries in parallel, instead of serially. Apply this
option to a selected number of Makefiles, which can greatly speed up the
build on multi-core machines, when using make -j.

This can be extended to more Makefiles later on, whenever they are
verified to work correctly with parallel building.

I tested this on a 24-core machine, with make -j48 buildworld (N = 6):

before stddev after stddev
======= ====== ======= ======
real time 1741.1 16.5 959.8 2.7
user time 12468.7 16.4 14393.0 16.8
sys time 1825.0 54.8 2110.6 22.8

(user+sys)/real 8.2 17.1

E.g. the build was approximately 45% faster in real time. On machines
with less cores, or with lower -j settings, the speedup will not be as
impressive. But at least you can now almost max out a machine with
buildworld!

Submitted by: jilles
MFC after: 2 weeks

Remove IPX support.

IPX was a network transport protocol in Novell's NetWare network operating
system from late 80s and then 90s. The NetWare itself switched to TCP/IP
as default transport in 1998. Later, in this century the Novell Open
Enterprise Server became successor of Novell NetWare. The last release
that claimed to still support IPX was OES 2 in 2007. Routing equipment
vendors (e.g. Cisco) discontinued support for IPX in 2011.

Thus, IPX won't be supported in FreeBSD 11.0-RELEASE.

Add 'libstatfoo' from Sam Leffler.

Remove libyaml, it has been replaced by libucl

Branch libucl to the build

Please welcome casperd daemon. It (and its services) will be responsible for
giving access to functionality that is not available in capability mode
sandbox. The functionality can be precisely restricted.

Start with the following services:
- system.dns - provides API compatible to:
- gethostbyname(3),
- gethostbyname2(3),
- gethostbyaddr(3),
- getaddrinfo(3),
- getnameinfo(3),
- system.grp - provides getgrent(3)-compatible API,
- system.pwd - provides getpwent(3)-compatible API,
- system.random - allows to obtain entropy from /dev/random,
- system.sysctl - provides sysctlbyname(3-compatible API.

Sponsored by: The FreeBSD Foundation

Move my simple logging API to a separate library. It is now already used
by hastctl(8), hastd(8) and auditdistd(8) and will soon be also used
by casperd(8) and its services. There is no documentation and pjdlog.h
header file is not installed in /usr/include/ to keep it private.
Unfortunately we don't have /lib/private/ at this point, only
/usr/lib/private/, so the library is installed in /lib/.

Sponsored by: The FreeBSD Foundation

Attempt to move the POSIX iconv* symbols out of runtime linker space.
FreeBSD systems usually implemented this as a third party module and
our implementation hasn't played as nicely with the old way as it could
have.

To that end:
* Rename the iconv* symbols in libc.so.7 to have a __bsd_ prefix.
* Provide .symver compatability with existing 10.x+ binaries that
referenced the iconv symbols. All existing binaries should work.
* Like on Linux/glibc systems, add a libc_nonshared.a to the ldscript
at /usr/lib/libc.so.
* Move the "iconv*" wrapper symbols to libc_nonshared.a

This should solve the runtime ambiguity about which symbols resolve
to where. If you compile against the iconv in libc, your runtime
dependencies will be unambiguous.

Old 9.x libraries and binaries will always resolve against their
libiconv.so.3 like they did on 9.x. They won't resolve against libc.

Old 10.x binaries will be satisified by the .symver helpers.

This should allow ports to selectively compile against the libiconv
port if needed and it should behave without ambiguity now.

Discussed with: kib

Bring in libnv library for managing name/value pairs. The following types
are currently supported:

- NV_TYPE_NULL - only name, no data;
- NV_TYPE_BOOL - boolean (true or false);
- NV_TYPE_NUMBER - 64bit unsigned integer;
- NV_TYPE_STRING - C string;
- NV_TYPE_NVLIST - nested nvlist;
- NV_TYPE_DESCRIPTOR - file descriptor;
- NV_TYPE_BINARY - binary data.

For detailed documentation and examples see nv(3) manual page.

Sponsored by: The FreeBSD Foundation

Install category Kyuafiles from their category directories.

Move the installation of /usr/tests/lib/Kyuafile from src/tests/lib/
to src/lib/. This is to keep the src/tests/ hierarchy unaware of the
rest of the tree, which makes things clearer in general. In particular:

1) Everything related to the construction of /usr/tests/lib/ is kept
in src/lib/. There is no need to think about different directories
and how they relate to each other. (The same applies for libexec,
usr.bin, etc. but these are not yet handled.)

2) src/tests becomes the place to keep cross-functional test programs
and nothing else, which also helps in simplifying things.

Reviewed by: freebsd-testing
Approved by: rpaulo (mentor)

Subsume the functionality of MK_ATF into MK_TESTS.

There is no reason to keep the two knobs separate: if tests are
enabled, the ATF libraries are required; and if tests are disabled,
the ATF libraries are not necessary. Keeping the two just serves
to complicate the build.

Reviewed by: freebsd-testing
Approved by: rpaulo (mentor)

Remove the WITH_LIBICONV_COMPAT hack that seems to do more harm than
good. This caused libc to spoof the ports libiconv namespace and
provide a colliding libiconv.so.3 to fool rtld. This should have
been removed some time ago.

Remove BIND.

Approved by: re (gjb)

Move libldns to the correct (ordered) library list.

Approved by: re (blanket)

Build and install the Unbound caching DNS resolver daemon.

Approved by: re (blanket)

Connect libexecinfo to the build

Sponsored by: DARPA, AFRL

The iconv in libc did two things - implement the standard APIs, the GNU
extensions and also tried to be link time compatible with ports libiconv.
This splits that functionality and enables the parts that shouldn't
interfere with the port by default.

WITH_ICONV (now on by default) - adds iconv.h, iconv_open(3) etc.
WITH_LIBICONV_COMPAT (off by default) adds the libiconv_open etc API, linker
symbols and even a stub libiconv.so.3 that are good enough to be able
to 'pkg delete -f libiconv' on a running system and reasonably expect it
to work.

I have tortured many machines over the last few days to try and reduce
the possibilities of foot-shooting as much as I can. I've successfully
recompiled to enable and disable the libiconv_compat modes, ports that use
libiconv alongside system iconv etc. If you don't enable the
WITH_LIBICONV_COMPAT switch, they don't share symbol space.

This is an extension of behavior on other system. iconv(3) is a standard
libc interface and libiconv port expects to be able to run alongside it on
systems that have it.

Bumped osreldate.

- Trim an unused and bogus Makefile for mount_smbfs.
- Reconnect with some minor modifications, in particular now selsocket()
internals are adapted to use sbintime units after recent'ish calloutng
switch.

Wrap the creation of the ../include link in /usr/lib in
!defined(LIBRARIES_ONLY) so it is only created once on architectures
with 32-bit compat support.

Replace ln -fhs with ${INSTALL_SYMLINK} to the link is logged in the
METALOG.

An even more refined version of r249484, until we can come up with
a good fix for the -print-file-name=include breakage.

As suggested by Andrey Chernov.

Fix the symlink creation from r249484 so that repeated
installs work correctly.

Suggested by Tijl Coosemans.

Install a symlink
/usr/lib/include ==> /usr/include

This fixes -print-file-name=include in clang (and is
arguably a better way to fix the same issue in GCC than
the change I made in r231336).

MFC after: 1 week

Import libyaml as libbsdyml (private brand name)

LibYAML is a YAML 1.1 parser and emitter under MIT license which will
soon be used by the pkg boostrap (usr.bin/pkg) and bhyve

Reviewed by: roberto, antoine

Import LDNS and build it as an internal library.

Sort by MK_* knob like the comment says

MFC after: 1 week

Merge projects/bhyve to head.

'bhyve' was developed by grehan@ and myself at NetApp (thanks!).

Special thanks to Peter Snyder, Joe Caradonna and Michael Dexter for their
support and encouragement.

Obtained from: NetApp

With the old sade removed, libdisk is no longer used by anything in HEAD
and uses a number of problematic pre-gpart interfaces. Since it has been
entirely obsoleted by interfaces in geom, remove it.

Add libnetbsd, a thin compatibility layer intended to allow a limited
set of NetBSD software to compile as part of the FreeBSD build with
little or no modifiction. It is built as a static library and not
installed for general use. Likewise, its header files are not
installed.

Sponsored by: DARPA, AFRL

Implement DTrace for PowerPC. This includes both 32-bit and 64-bit.

There is one known issue: Some probes will display an error message along the
lines of: "Invalid address (0)"

I tested this with both a simple dtrace probe and dtruss on a few different
binaries on 32-bit. I only compiled 64-bit, did not run it, but I don't expect
problems without the modules loaded. Volunteers are welcome.

MFC after: 1 month

Add ATF to the build. This is may be a bit rought around the egdes,
but committing it helps to get everyone on the same page and makes
sure we make progress.

Tinderbox breakages that are the result of this commit are entirely
the committer's fault -- in other words: buildworld testing on amd64
only.

Credits follow:

Submitted by: Garrett Cooper <yanegomi@gmail.com>
Sponsored by: Isilon Systems
Based on work by: keramida@
Thanks to: gnn@, mdf@, mlaier@, sjg@
Special thanks to: keramida@

Apply local patches to mandoc and connect it to the build.

- adds a couple more library strings used in the tree
- changes some more to the current groff spelling
- changes page footer to match groff style

Disconnect non-MPSAFE SMBFS from the build in preparation for dropping
GIANT from VFS. In addition, disconnect also netsmb, which is a base
requirement for SMBFS.

In the while SMBFS regular users can use FUSE interface and smbnetfs
port to work with their SMBFS partitions.

Also, there are ongoing efforts by vendor to support in-kernel smbfs,
so there are good chances that it will get relinked once properly locked.

This is not targeted for MFC.

Disconnect non-MPSAFE NWFS from the build in preparation for dropping
GIANT from VFS. In addition, disconnect also netncp, which is a base
requirement for NWFS.

In the possibility of a future maintenance of the code and later
readd to the FreeBSD base, maybe we should think about a better location
for netncp. I'm not entirely sure the / top location is actually right,
however I will let network people to comment on that more specifically.

This is not targeted for MFC.

Import work done under project/nand (@235533) into head.

The NAND Flash environment consists of several distinct components:
- NAND framework (drivers harness for NAND controllers and NAND chips)
- NAND simulator (NANDsim)
- NAND file system (NAND FS)
- Companion tools and utilities
- Documentation (manual pages)

This work is still experimental. Please use with caution.

Obtained from: Semihalf
Supported by: FreeBSD Foundation, Juniper Networks

Import stdbuf(1) and the shared library it relies on.
This tool changes the default buffering behaviour of standard
stdio streams.

It only works on dynamic binaries. To make it work for static
ones it would require cluttering stdio because there no single
entry point.

PR: 166660
Reviewed by: current@, jhb
Approved by: kib (mentor)
MFC after: 1 week

Build libproc and librtld_db for MIPS

- Do not build libcom_err and compile_et when kerberos is disabled. They
depends on several heimdal libraries and not used by anything but kerberos
tools.

Fix typo in Makefile comment.

MFC after: 3 days

Add libstdthreads.

This library implements the C11 threads interface on top of the pthreads
library. As discussed on the lists, the preferred way to implement
this, is as a separate library.

It is unlikely that these functions will be used a lot in the future. It
would have been easier if the C11 working group standardized (a subset
of) pthreads and clock_nanosleep(). Having it as a separate library
allows the embedded people to omit it from their system.

Discussed on: arch@, threads@

Fix breakage after r227983; lib/libcxxrt still got built, because it was
not disabled in the usual way (by adding it to __DEFAULT_NO_OPTIONS in
share/mk/bsd.own.mk), and because the test for MK_LIBCPLUSPLUS in
Makefile.inc1 was incorrect.

Pointy hat to: dim

Import libc++ / libcxxrt into base. Not build by default yet (use
MK_LIBCPLUSPLUS=yes to enable). This is a work-in-progress. It works for
me, but is not guaranteed to work for anyone else and may eat your dog.

To build C++ using libc++, add -stdlib=libc++ to your CXX and LD flags.

Bug reports welcome, bug fixes even more welcome...

Approved by: dim (mentor)

Now that its only consumer is gone (sysinstall), remove libftpio as well.

Remove a redundant check for libncp.

Submitted by: Alexander Sack (asack at niksun dot com)

Backout libinstall.a -> libpkg commit.

Discussed with: erwin, brooks, bapt

- Commit work from libprocstat project. These patches add support for runtime
file and processes information retrieval from the running kernel via sysctl
in the form of new library, libprocstat. The library also supports KVM backend
for analyzing memory crash dumps. Both procstat(1) and fstat(1) utilities have
been modified to take advantage of the library (as the bonus point the fstat(1)
utility no longer need superuser privileges to operate), and the procstat(1)
utility is now able to display information from memory dumps as well.

The newly introduced fuser(1) utility also uses this library and able to operate
via sysctl and kvm backends.

The library is by no means complete (e.g. KVM backend is missing vnode name
resolution routines, and there're no manpages for the library itself) so I
plan to improve it further. I'm commiting it so it will get wider exposure
and review.

We won't be able to MFC this work as it relies on changes in HEAD, which
was introduced some time ago, that break kernel ABI. OTOH we may be able
to merge the library with KVM backend if we really need it there.

Discussed with: rwatson

Add the BSD-licensed Citrus iconv to the base system with default off
setting. It can be built by setting the WITH_ICONV knob. While this
knob is unset, the library part, the binaries, the header file and
the metadata files will not be built or installed so it makes no impact
on the system if left turned off.

This work is based on the iconv implementation in NetBSD but a great
number of improvements and feature additions have been included:

- Some utilities have been added. There is a conversion table generator,
which can compare conversion tables to reference data generated by
GNU libiconv. This helps ensuring conversion compatibility.
- UTF-16 surrogate support and some endianness issues have been fixed.
- The rather chaotic Makefiles to build metadata have been refactored
and cleaned up, now it is easy to read and it is also easier to add
support for new encodings.
- A bunch of new encodings and encoding aliases have been added.
- Support for 1->2, 1->3 and 1->4 mappings, which is needed for
transliterating with flying accents as GNU does, like "u.
- Lots of warnings have been fixed, the major part of the code is
now WARNS=6 clean.
- New section 1 and section 5 manual pages have been added.
- Some GNU-specific calls have been implemented:
iconvlist(), iconvctl(), iconv_canonicalize(), iconv_open_into()
- Support for GNU's //IGNORE suffix has been added.
- The "-" argument for stdin is now recognized in iconv(1) as per POSIX.
- The Big5 conversion module has been fixed.
- The iconv.h header files is supposed to be compatible with the
GNU version, i.e. sources should build with base iconv.h and
GNU libiconv. It also includes a macro magic to deal with the
char ** and const char ** incompatibility.
- GNU compatibility: "" or "char" means the current local
encoding in use
- Various cleanups and style(9) fixes.

Approved by: delphij (mentor)
Obtained from: The NetBSD Project
Sponsored by: Google Summer of Code 2009

Replace libgcc.a by libcompiler_rt.a.

libcompiler_rt.a is a BSD licensed C language runtime, which implements
many routines which are linked into binaries on architectures where
certain functionality is missing (e.g. 64 bits mul/div on i386).

Unfortunately, libcompiler_rt cannot replace libgcc entirely. Certain
features, such as an unwinder for exception handling, are missing.
That's why only libgcc.a is replaced for now, because this one does seem
to be complete.

Tested by: rene (amd64), nwhitehorn (powerpc), droso (i386 exprun)
and many others. Thanks!
Obtained from: user/ed/compiler-rt

Repair some build breakage introduced in r211725 and garbage collect some
code made obsolete in the same commit.

Change the logic here to match Makefile.inc1. Having it in two places
suggests an opportunity for refactoring :)

Submitted by: nathanw@

MFtbemd:

Prefer MACHNE_CPUARCH to MACHINE_ARCH in most contexts where you want
to test of all the CPUs of a given family conform.

Build librtld_db.

Sponsored by: The FreeBSD Foundation

Simplify the Makefile. The i386 and amd64 sections are equal.

Sponsored by: The FreeBSD Foundation

libproc is going to be i386/amd64 only with the next update.

Sponsored by: The FreeBSD Foundation

Build libsmb should be on powerpc64 as well.

Don't build Clang libs during lib32 build.

This should massively reduce the buildworld time on amd64.

Pointy hat to: me

Hook clang into the build on i386/amd64/powerpc.

Approved by: ed (mentor)

Import of liblzma, xz, xzdec, lzmainfo from vendor branch
Add support for xz and lzma to lesspipe.sh (xzless, lzless)
Bump __FreeBSD_version

Approved by: delphij (mentor)
MFC after: 2 weeks

- Take libinstall.a out of pkg_install and make it a proper shared library.
- Rework the wrapper support to check libpkg version as well as pkg_install
version.
- Add libfetch to _prebuild_libs.
- There are no new features introduced.

Notes: the API is not stable, so basically, do not use libpkg in your
projects for now. Also there's no manpage for libpkg yet, because the API
will change drastically. I repeat, do not use libpkg for now.

Make this conform to the other top-level Makefile subdir listings with
one file per line.

Suggested by: bde

Remove stale references to libkrb5.
Rejigger the SUBDIR setting a smidge: we now set all the libraries that depend
on something else, and then SUBDIR+= the rest.

A separate commit will fix the SUBDIR style to be the same as the rest
of the tree.

Introduce libefi -- a library around EFI runtime services and protocols.
This first commit brings 3 functions for enumerating, retrieving,
adding, removing and modifying EFI variables. The immediate use of these
include the insertion of a new boot option as part of the installation
process.

This library uses ioctl(2) requests implemented by io(4) to pass the
requests down through the kernel to EFI. These ioctl requests are only
implemented on ia64, so libefi is currently only enabled on ia64. The
interface is generic and io(4) on mad64/i386 can easily be taught to
handle these once EFI support has been added to the kernel there.

Connect smbfs related libraries and tool on sparc64.

Reviewed by: marius

Remove libulog from the bootstrap again.

libulog now only provides functions that are used by various packages
from the ports tree, namely the libutempter ones. There is no reason to
link it into the crunch/fixit binaries anymore.

Convert pam_lastlog(8) to libulog.

The information used by the "Last login:"-line is obtained by using
ulog_setutxfile(3) to switch to the lastlog database. Login and logout
are performed using the utility functions ulog_login(3) and
ulog_logout(3).

This also means we must build libulog during bootstrap.

Approved by: des

Add a new library: libulog.

One of the things I really want to do, is to get rid of the limitations
of our current utmp(5) mechanism:

- It only allows 8 byte TTY device names.
- The hostname only allows 16 bytes of storage.

I'm not a big fan of <utmpx.h>, but I think we should at least try to
add parts of it. Unfortunately we cannot implement <utmpx.h>, because we
miss various fields, such as ut_id, ut_pid, etc. The API provided by
libulog shares some similarities with <utmpx.h>, so it shouldn't be too
hard to port these applications eventually. In most simple cases, it
should just be a matter of removing the ulog_ prefix everywhere.

As a bonus, it also implements a function called ulog_login_pseudo(),
which allows unprivileged applications to write log entries, provided
they have a valid file descriptor to a pseudo-terminal master device.

libulog will allow a smoother transition to a new file format by adding
a library interface to deal with utmp/wtmp/lastlog files. I initially
thought about adding the functionality to libutil, but because I'm not
planning on keeping this library around forever, we'd better keep it
separated.

Next items on the todo list:

1. Port applications in the base system (and ports) to libulog, instead
of letting them use <utmp.h>.
2. Remove <utmp.h>, implement <utmpx.h> and reimplement this library on
top.
3. Port as many applications as possible back to <utmpx.h>.

- Teach vesa(4) and dpms(4) about x86emu. [1]
- Add vesa kernel options for amd64.
- Connect libvgl library and splash kernel modules to amd64 build.
- Connect manual page dpms(4) to amd64 build.
- Remove old vesa/dpms files.

Submitted by: paradox <ddkprog yahoo com> [1], swell k at gmail.com
(with some minor tweaks)

Add libjail, a (somewhat) simpler interface to the jail_set and jail_get
system calls and the security.jail.param sysctls.

Approved by: bz (mentor)

Commit missed file in r189587, update directory name for libusb.

Spotted by: rdivacky

Merge OpenBSM alpha 4 from OpenBSM vendor branch to head, both
contrib/openbsm (svn merge) and src/sys/{bsm,security/audit} (manual
merge). Add libauditd build parts and add to auditd's linkage;
force libbsm to build before libauditd.

OpenBSM history for imported revisions below for reference.

MFC after: 1 month
Sponsored by: Apple Inc.
Obtained from: TrustedBSD Project

OpenBSM 1.1 alpha 4

- With the addition of BSM error number mapping, we also need to map the
local error number passed to audit_submit(3) to a BSM error number,
rather than have the caller perform that conversion.
- Reallocate user audit events to avoid collisions with Solaris; adopt a
more formal allocation scheme, and add some events allocated in Solaris
that will be of immediate use on other platforms.
- Add an event for Calife.
- Add au_strerror(3), which allows generating strings for BSM errors
directly, rather than requiring applications to map to the local error
space, which might not be able to entirely represent the BSM error
number space.
- Major auditd rewrite for launchd(8) support. Add libauditd library
that is shared between launchd and auditd.
- Add AUDIT_TRIGGER_INITIALIZE trigger (sent via 'audit -i') for
(re)starting auditing under launchd(8) on Mac OS X.
- Add 'current' symlink to active audit trail.
- Add crash recovery of previous audit trail file when detected on audit
startup that it has not been properly terminated.
- Add the event AUE_audit_recovery to indicated when an audit trail file
has been recovered from not being properly terminated. This event is
stored in the new audit trail file and includes the path of recovered
audit trail file.
- Mac OS X and FreeBSD dependent code in auditd.c is separated into
auditd_darwin.c and auditd_fbsd.c files.
- Add an event for the posix_spawn(2) and fsgetpath(2) Mac OS X system
calls.
- For Mac OS X, we use ASL(3) instead of syslog(3) for logging.
- Add support for NOTICE level logging.

OpenBSM 1.1 alpha 3

- Add two new functions, au_bsm_to_errno() and au_errno_to_bsm(), to map
between BSM error numbers (largely the Solaris definitions) and local
errno(2) values for 32-bit and 64-bit return tokens. This is required
as operating systems don't agree on some of the values of more recent
error numbers.
- Fix a bug how au_to_exec_args(3) and au_to_exec_env(3) calculates the
total size for the token. This buge.
- Deprecated Darwin constants, such as TRAILER_PAD_MAGIC, removed.

src/sys/dev/usb2/controller/uss820dci_pccard.c
src/sys/dev/usb2/core/usbdevs
src/sys/dev/usb2/include/urio2_ioctl.h
src/sys/dev/usb2/storage/ustorage2_fs.h

These files are not used any more.

src/usr.sbin/Makefile
src/etc/mtree/BSD.include.dist
src/include/Makefile
src/lib/Makefile
src/share/man/man7/hier.7
src/share/mk/bsd.libnames.mk
src/etc/mtree/BSD.include.dist

Make "usbconfig" and "libusb20" a part of the default build.

src/sys/dev/usb/rio500_usb.h
src/sys/dev/usb2/storage/urio2.c

Use common include file.

src/sys/dev/usb2/bluetooth/ng_ubt2.c

Make USB bluetooth depend on "ng_hci" module.

src/sys/dev/usb2/controller/ehci2.c
src/sys/dev/usb2/controller/ehci2.h

Patches for Marvell EHCI.

src/sys/dev/usb2/core/usb2_busdma.c

Bugfix for 64-bit platforms. Need to unload the previously loaded DMA
map and some cleanup regarding some corner cases.

src/sys/dev/usb2/core/usb2_core.h
src/sys/dev/usb2/core/usb2_dev.c
src/sys/dev/usb2/core/usb2_dev.h

Bugfix for libusb filesystem interface.

New feature: Add support for filtering device data at the expense of the
userland process.

Add some more comments.

Some minor code styling.

Remove unused function, usb2_fifo_get_data_next().

Fix an issue about "fifo_index" being used instead of "ep_index".

src/sys/dev/usb2/core/usb2_device.c
src/sys/dev/usb2/core/usb2_generic.c

Bugfix for Linux USB compat layer. Do not free non-generic FIFOs when
doing an alternate setting.

Cleanup USB IOCTL and USB reference handling.
Fix a corner case where USB-FS was left initialised after
setting a new configuration or alternate setting.

src/sys/dev/usb2/core/usb2_hub.c

Improvement: Check all USB HUB ports by default at least one time.

src/sys/dev/usb2/core/usb2_request.c

Bugfix: Make sure destination ASCII string is properly zero terminated
in all cases.

Improvement: Skip invalid characters instead of replacing with a dot.

src/sys/dev/usb2/core/usb2_util.c
src/sys/dev/usb2/image/uscanner2.c

Spelling.

src/sys/dev/usb2/include/Makefile

Share "usbdevs" with the old USB stack.

src/sys/dev/usb2/include/usb2_devid.h
src/sys/dev/usb2/include/usb2_devtable.h

Regenerate files.

Alfred: Please fix the RCS tag at the top.

src/sys/dev/usb2/include/usb2_ioctl.h

Fix compilation of "kdump".

src/sys/dev/usb2/serial/ubsa2.c
src/sys/dev/usb2/serial/ugensa2.c

Remove device ID's which will end up in a new 3G driver.

src/sys/dev/usb2/sound/uaudio2.c

Correct a debug printout.

src/sys/dev/usb2/storage/umass2.c

Sync with old USB stack.

src/lib/libusb20/libusb20.3

Add more documentation.

src/lib/libusb20/libusb20.c

Various bugfixes and improvements.

src/usr.sbin/usbconfig/dump.c
src/usr.sbin/usbconfig/usbconfig.c

New commands for dumping strings and doing custom USB requests from
the command line.

Remove keyword requirements from generated files:
"head/sys/dev/usb2/include/usb2_devid.h"
"head/sys/dev/usb2/include/usb2_devtable.h"

add new build knobs and jigger some existing controls to improve
control over the result of buildworld and installworld; this especially
helps packaging systems such as nanobsd

Reviewed by: various (posted to arch)
MFC after: 1 month

Add an implementation of the RPCSEC_GSS authentication protocol for RPC. This
is based on an old implementation from the University of Michigan with lots of
changes and fixes by me and the addition of a Solaris-compatible API.

Sponsored by: Isilon Systems
Reviewed by: alfred

Remove netatm from HEAD as it is not MPSAFE and relies on the now removed
NET_NEEDS_GIANT. netatm has been disconnected from the build for ten
months in HEAD/RELENG_7. Specifics:

- netatm include files
- netatm command line management tools
- libatm
- ATM parts in rescue and sysinstall
- sample configuration files and documents
- kernel support as a module or in NOTES
- netgraph wrapper nodes for netatm
- ctags data for netatm.
- netatm-specific device drivers.

MFC after: 3 weeks
Reviewed by: bz
Discussed with: bms, bz, harti

Hook libdwarf and libproc into the build.

- Remove libkse and related support code in libpthread from the build.
Don't remove the files yet. Kernel support will be removed shortly.

- Fix setting of MK_GSSAPI option by bsd.own.mk; its value should
default to the value of MK_KERBEROS unless set explicitly by
WITH_GSSAPI/WITHOUT_GSSAPI. (This introduces another type of
MK_* variables which itself is questionable.)

- Teach tools/build/options/makeman script that generates the
src.conf(5) manpage about the new type of MK_* variables.

- Fix broken logic in lib/Makefile.

Hide the building and installation of libgssapi behind the
WITHOUT_KERBEROS knob. While GSS can be used for other things
some third party software (most notably ports/x11/kdelibs3)
takes the presence of libgssapi as an indication that kerberos
is available, and attempts to link with the kerberos libs. If
they are not available, the build will fail.

Because you might want to use GSS but not kerberos, add a knob
to re-enable it if WITHOUT_KERBEROS is present.

Document the new knob, and the new behavior of WITHOUT_KERBEROS.

Not objected and/or generally agreed to by: freebsd-arch

Problem discussed/analyzed in:
PR: ports/116484

Build libthread_db for arm as well.

MFC after: 1 week

Add a period for yar@.

Tweak the handling of "WITHOUT_LIBPTHREAD". Also remove the accidental
treatment of 'LIBKSE' as an "old style" knob.

Submitted by: ru
Approved by: re(kensmith)

Repo copy libpthreads to libkse.
This introduces the WITHOUT_LIBKSE nob,
and changes WITHOUT_LIBPTHREADS to mean with neither threading libs.
Approved by: re(kensmith)

Disconnect netatm from the build as it is not MPSAFE and relies on
NET_NEEDS_GIANT, which will shortly be removed. This is done in a
away that it may be easily reattached to the build before 7.1 if
appropriate locking is added. Specifics:

- Don't install netatm include files
- Disconnect netatm command line management tools
- Don't build libatm
- Don't include ATM parts in rescue or sysinstall
- Don't install sample configuration files and documents
- Don't build kernel support as a module or in NOTES
- Don't build netgraph wrapper nodes for netatm

This removes the last remaining consumer of NET_NEEDS_GIANT.

Reviewed by: harti
Discussed with: bz, bms
Approved by: re (kensmith)

Make sure libc is installed before all other shared libraries. Other
libraries might depend on versioned symbols from libc.

Switch to new ncurses build glue

Approved by: delphij

MFP4: Add an implementation of the ELF(3) and GELF(3) API set.

Bump __FreeBSD_version.

Reviewed by: jb

Removed libc_r build support.

Remove alpha left-overs.

Build libsmb and smbutil on ia64.

Build libthread_db on PowerPC.

Extend coverage of the MK_IPX build option to the following:

- <netipx> headers [1]
- IPX library (libipx)
- IPX support in ifconfig(8)
- IPXrouted(8)
- new MK_NCP option

New MK_NCP build option controls:

- <netncp> and <fs/nwfs> headers
- NCP library (libncp)
- ncplist(1) and ncplogin(1)
- mount_nwfs(8)
- ncp and nwfs kernel modules

User knobs: WITHOUT_IPX, WITHOUT_IPX_SUPPORT, WITHOUT_NCP.

[1] <netsmb/netbios.h> unconditionally uses <netipx> headers
so they are still installed. This needs to be dealt with.

Reimplementation of world/kernel build options. For details, see:

http://lists.freebsd.org/pipermail/freebsd-current/2006-March/061725.html

The src.conf(5) manpage is to follow in a few days.

Brought to you by: imp, jhb, kris, phk, ru (all bugs are mine)

Connect librt to buildworld.

Add libbsm to the library build.

Obtained from: TrustedBSD Project

Add a dummy arm Write_Disk() function, and compile libdisk on arm.

Add a new extensible GSS-API layer which can support GSS-API plugins,
similar the the Solaris implementation. Repackage the krb5 GSS mechanism
as a plugin library for the new implementation. This also includes a
comprehensive set of manpages for the GSS-API functions with text mostly
taken from the RFC.

Reviewed by: Love Hörnquist Åstrand <lha@it.su.se>, ru (build system), des (openssh parts)

Disconnect libc_r from buildworld, it is still kept in the tree to
provide some baseline references, but users are encouraged to use
libpthread or libthr in real world.

Discussed on: arch@

Connect smbfs build on powerpc.

Hook libmemstat(3) up to the build.

Build libpmc on all architectures (FWIW :-)

Ok'd by: jkoshy@

Bring a working snapshot of hwpmc(4), its associated libraries, userland utilities
and documentation into -CURRENT.

Bump FreeBSD_version.

Reviewed by: alc, jhb (kernel changes)

Connect libthr and libthread_db to make buildworld.

Prepare for importing my 1:1 threading work, disconnect libthr and
libthread_db from make buildworld.

Tie libgpib into the build (unless NO_GPIB)

NOCRYPT -> NO_CRYPT

NOATM -> NO_ATM

NOLIBC_R -> NO_LIBC_R
NOLIBPTHREAD -> NO_LIBPTHREAD
NOLIBTHR -> NO_LIBTHR

Remove stub libxpg4. All functionality was merged into libc a long time ago.

Add knob NO_NIS (fka NO_YP_LIBC) and make world compileable when set.
If turned on no NIS support and related programs will be built.

Lost parts rediscovered by: Danny Braniss <danny at cs.huji.ac.il>
PR: bin/68303
No objections: des, gshapiro, nectar
Reviewed by: ru
Approved by: rwatson (mentor)
MFC after: 2 weeks

Help Tinderbox and remove libautofs

Fix build in the !NOATM case by using the begemot library in place of
using libisc which was a part of BIND8.

Discussed with: des, re, dougb
Submitted by: harti (one part)
Reviewed by: harti (previous version)

Switch from BIND 8 to BIND 9.

Submitted by: (in part) dougb@, trhodes@
Reviewed by: dougb@, trhodes@, re@
MFC after: 5 days

Hook libautofs to the build.

Start the de-orbital burn of our previous FreeBSD version compat libs
living in usr/src. We need to use them from ports to record dependencies.

Discussed with: re(scottl)

build libthr on amd64.

Turn on the magic.

Enable libpthread build for powerpc

Hook libthread_db into the build, except for arm and powerpc.
Porting libthread_db to arm and/or powerpc is easy enough, but
we don't build gdb on those platforms yet.

Make bluetooth compile on all platforms

Reviewed by: imp, ru

Style fixes:
Main ones: mostly use conditional expressions in ifdefs instead of a
mixture of conditional expressions and nested ifdefs.
Nearby ones:
- don't do less than echo the code in the comment about libc_r
- fixed some internal insertion sort errors and indentation errors.

Fixed some insertion sort errors (external ones only).

Do not attempt to build libdisk, libthr and libc_r for arm.

Enable libdisk for powerpc build.

Put libypclnt behind NO_YP_LIBC

Hook libthr to the build for PowerPC

Moving forward, there's no reason to exclude libpthread on sparc64.

Connect libarchive to the build.

Add NO_BLUETOOTH knob to the build process

Requested by: phk
Reviewed by: imp (mentor), ru

The snmp_netgraph module depends on libnetgraph. So add a dependency and
add libnetgraph to the list of prebuilt libraries in the main Makefile.

Reviewed by: ru

Link libbsnmp to the build.

Link libngatm to the build (unless NOATM is set).

Update Bluetooth code.

Reviewed by: M. Warner Losh <imp@bsdimp.com>; John Hay <jhay@freebsd.org>
Approved by: M. Warner Losh <imp@bsdimp.com> (mentor)

- Support for multibyte charsets in LIBICONV.
- CD9660_ICONV, NTFS_ICONV and MSDOSFS_ICONV kernel options
(with corresponding modules).
- kiconv(3) for loadable charset conversion tables support.

Submitted by: Ryuichiro Imura <imura@ryu16.org>

Introduce more knobs to slim down FreeBSD userland

NO_TOOLCHAIN skips Compilers and Binutils
NO_USB skips USB stuff
NO_VINUM skips Vinum stuff
NO_ACPI skips ACPI stuff

Make build of libatm depend on existing NOATM conditional.

Imply NOLIBC_R for PowerPC.

Add libpthread to the alpha build.

Requested by ru: Since the majority of archs can now support the
build of libpthread, rearrange the Makefile to treat libpthread
as an exception.

Since it builds and seems to work OK, add libpthread to the amd64 build.

Don't check for the existance of src/crypto/ for building items that
may contain crypto. The days of ITAR paranoia are over, and the simple
macro tests that remain are sufficient.

Connect libncp/libsmb to the build. They compile, but have a couple of
silly bugs that probably wont quite make a segfault. eg: passing a pointer
to an int to sysctl instead of a pointer to a size_t.

Now that we have the stubs for alpha and we can build it
on that platform, invert the test for the platforms on
which libthr is built. Amd64 and powerpc are the only
platforms excluded.

Compile tested on: amd64, alpha

Very big makeover in the way telnet, telnetd and libtelnet are built.

Previously, there were two copies of telnet; a non-crypto version
that lived in the usual places, and a crypto version that lived in
crypto/telnet/. The latter was built in a broken manner somewhat akin
to other "contribified" sources. This meant that there were 4 telnets
competing with each other at build time - KerberosIV, Kerberos5,
plain-old-secure and base. KerberosIV is no longer in the running, but
the other three took it in turns to jump all over each other during a
"make buildworld".

As the crypto issue has been clarified, and crypto _calls_ are not
a problem, crypto/telnet has been repo-copied to contrib/telnet,
and with this commit, all telnets are now "contribified". The contrib
path was chosen to not destroy history in the repository, and differs
from other contrib/ entries in that it may be worked on as "normal"
BSD code. There is no dangerous crypto in these sources, only a
very weak system less strong than enigma(1).

Kerberos5 telnet and Secure telnet are now selected by using the usual
macros in /etc/make.conf, and the build process is unsurprising and
less treacherous.

o disconnect libc_r from the build on ia64. The library is now
obsolete. The intend is to add glue to either libthr or
libpthread to create the necessary compat links.
o Hook libpthread to the build on ia64. This is slightly out of
order, because the kernel still doesn't have all the support,
but that's not a problem in this case.

Re-enable libstand on all platforms including amd64.

Port libc_r to amd64, and turn it back on for amd64. It passes all of
the same src/lib/libc_r/test/* tests that the other platforms pass.

Build libthr by default on ia64.

Hook libthr to the build. It's currently built only for i386 and
a NOLIBTHR knob will prevent it from being built.

Retire the useless NOSECURE knob.

Approved by: re (scottl)

Turn MAKE_KERBEROS5 into NO_KERBEROS by negating the logic. Some extra
cleanups were necessary in release/Makefile, and the tinderbox code
was syntax checked, not run checked.

Build non-crypto telnet(1) and telnetd(8) if NO_OPENSSL is defined.

Submitted by: Marius Strobl <marius@alchemy.franken.de>

Turn off libstand for amd64 for the time being. It is built in i386
mode, and we do not need the complications for now.

No libc_r for AMD64.

Fixed alignment.

Connect libpthread to the (i386) world. It is currently installed
as libkse and will once again be renamed libpthread after more
testing.

Approved by: re

KerberosIV deorbit sequence continues: Look for K5 instead of KIV

Some things don't build for PowerPC yet.

List from: benno

Hook libgeom in.

Port libc_r to ia64. We need to do things slightly different
because we have 2 stacks per thread: the regular downward
memory stack and the irregular upward register stack. This
implementation lets both stacks grow toward each other. An
alternative scheme is to have them grow away from each other.
The alternate scheme has the advantage that both stack grow
toward guard pages. Since libc_r is virtually dead and we
really want the *context stuff for thread switching, we don't
try to be perfect, just functional.

Build libc_r on sparc64, it should work now.

Connect libbsdxml (nee libexpat) to the build.

reconnect libc_r
Hint from: deischen@freebsd.org

Add i386 to the list of architectures that libc_r is broken on. This
effectively removes pppctl from the build for now. It only compiles on
alpha now (now ironic).

Initiate deorbit burn for the i386-only a.out related support. Moves are
under way to move the remnants of the a.out toolchain to ports. As the
comment in src/Makefile said, this stuff is deprecated and one should not
expect this to remain beyond 4.0-REL. It has already lasted WAY beyond
that.

Notable exceptions:
gcc - I have not touched the a.out generation stuff there.
ldd/ldconfig - still have some code to interface with a.out rtld.
old as/ld/etc - I have not removed these yet, pending their move to ports.
some includes - necessary for ldd/ldconfig for now.

Tested on: i386 (extensively), alpha

Hook up libugidfw to the build.

Obtained from: TrustedBSD Project
Sponsored by: DARPA, NAI Labs

Add libufs, a library for dealing with UFS filesystems from userland to
the build. It is here to compartmentalise functionality currently duplicated
in many notable programs in the base system. It currently handles block
reads and writes, as well as reading and writing of the filesystem superblock,
and the reading/lookup of inode data. It supports both UFS and UFS2. I
will be maintaining it, and porting programs to use it, however for now, it
is simply being built as part of world.

Don't try to build libc_r on sparc64, the _atomic_lock.S file isn't implemented.

Do not build libmp if NOSECURE is set (NO_OPENSSL is on a subset of NOSECURE)

Removed the a.out compatibility cruft.

libgnumalloc.so.2 and libresolv.so.2 should be put under
lib/compat/compat2?/ but I don't have the 2.x releases.

We don't need libgnumalloc or libresolv on ELF systems.

Connect libypclnt to the build.

Follow NetBSD and rename libusb to libusbhid.

MFC after: 7 days

Add two new libraries which are part of sendmail 8.12. libsm is used only
for building sendmail and the associated utilities. libmilter is a new
mail filtering API for sendmail.

Kernel support for smbfs is only built on the i386 at the moment, so
limit the building and installation of the userland utilities to that
architecture for now.

Reported by: bmah

Add bmake glue for src/contrib/smbfs and connect userland smbfs
support to the build.

The MFC reminder below is subject to <re@FreeBSD.org> approval
prior to 4.5-RELEASE.

Reviewed by: bp, fjoe
MFC: 1 week

Fix NO_SENDMAIL knob. When FreeBSD's old BSD version of vacation was
replaced with the new version in sendmail's distribution, vacation and
the necessary libraries (libsmdb and libsmutil) were changed so they
were always compiled. This broke people who didn't checkout
src/contrib/sendmail/. I don't know if it's best to think of NO_SENDMAIL
as no sendmail sources available or no sendmail binary. It is now the former.

Also, remove the sendmail chapter from System Managers Manual (SMM) if
NO_SENDMAIL is defined (for similar reasons -- source not available).

PR: 31863, 31865
Submitted by: matusita, Joe Kelsey <joe@zircon.seattle.wa.us>
MFC after: 3 days

Do not ``build'' (ie remove, and put in stub libraries) for libresolv
and/or libgnumalloc on anything but i386. The other platforms
post-date this mistake.

Do not build libc_r for ia64. There are some fundamental issues that
need to be resolved (ie: it cannot use setjmp/longjmp for thread
switching, which isn't likely to be fixed soon. libc_r has to be
reimplemented using something like makecontext()/swapcontext() etc
in order to work in ia64.)

Adjust dependancies and build order. PAM needs RPC.

Say goodbye to libss, which somehow managed to crouch hidden in the tree
for long after it was used.

Add some features to libdevstat, and overhaul the interface a bit:

1.) prefix all functions in the library with devstat_ (compatability
functions are available for all functions that were chaned in an
incompatible way, but are deprecated).
2.) Add a pointer to a kvm_t as the first argument to functions that
used to get their information via sysctl; they behave the same
as before when NULL is passed as this argument, otherwise, the
information is obtained via libkvm using the supplied handle.
3.) Add a new function, devstat_compute_statistics(), that is intended
to replace the old compute_stats() function. It offers more
statistics data, and has a more flexible interface.

libdevstat does now require libkvm; a library depedency is added, so
that libkvm only needs to be explicitely specified for statically linked
programs.
The library major version number is bumped.

Submitted by: Sergey A. Osokin <osa@freebsd.org.ru>, ken (3)
Reviewed by: ken

Fix a cryptoless world by disconnecting libmp from the build when there is no
crypto bits installed and/or NOCRYPTO/NO_OPENSSL is defined. This unfortunately
meants that usr.bin/chkey, usr.bin/newkey and usr.sbin/keyserv have also to
be disconnected.

IMO it is merely a workaround, the proper solution is to move libmp to
src/crypto where it belongs and use libgmp for the cryptoless builds instead.

Missed by: dd

Enable the new libmp in the build, and disable libgmp and its
henchmen.

Axe S/Key. OPIE is the legal successor.

Fix long lines.

Add libbz2 to provide Bzip2 capabilities to the system.

Turn on libdevinfo

Activate build of posix1e extensions in libc and libc_r that have been
moved in from libposix1e, and deactivate build of the soon-to-be-removed
libposix1e.

Approved by: rwatson
Obtained from: TrustedBSD Project

Rewrite of the CAM error recovery code.

Some of the major changes include:

- The SCSI error handling portion of cam_periph_error() has
been broken out into a number of subfunctions to better
modularize the code that handles the hierarchy of SCSI errors.
As a result, the code is now much easier to read.

- String handling and error printing has been significantly
revamped. We now use sbufs to do string formatting instead
of using printfs (for the kernel) and snprintf/strncat (for
userland) as before.

There is a new catchall error printing routine,
cam_error_print() and its string-based counterpart,
cam_error_string() that allow the kernel and userland
applications to pass in a CCB and have errors printed out
properly, whether or not they're SCSI errors. Among other
things, this helped eliminate a fair amount of duplicate code
in camcontrol.

We now print out more information than before, including
the CAM status and SCSI status and the error recovery action
taken to remedy the problem.

- sbufs are now available in userland, via libsbuf. This
change was necessary since most of the error printing code
is shared between libcam and the kernel.

- A new transfer settings interface is included in this checkin.
This code is #ifdef'ed out, and is primarily intended to aid
discussion with HBA driver authors on the final form the
interface should take. There is example code in the ahc(4)
driver that implements the HBA driver side of the new
interface. The new transfer settings code won't be enabled
until we're ready to switch all HBA drivers over to the new
interface.

src/Makefile.inc1,
lib/Makefile: Add libsbuf. It must be built before libcam,
since libcam uses sbuf routines.

libcam/Makefile: libcam now depends on libsbuf.

libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the
sbuf sources from sys/kern.

bsd.libnames.mk: Add LIBSBUF.

camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically
linked, we can't depend on the dynamic linker
to pull in libsbuf.

camcontrol.c: Use cam_error_print() instead of checking for
CAM_SCSI_STATUS_ERROR on every failed CCB.

sbuf.9: Change the prototypes for sbuf_cat() and
sbuf_cpy() so that the source string is now a
const char *. This is more in line wth the
standard system string functions, and helps
eliminate warnings when dealing with a const
source buffer.

Fix a typo.

cam.c: Add description strings for the various CAM
error status values, as well as routines to
look up those strings.

Add new cam_error_string() and
cam_error_print() routines for userland and
the kernel.

cam.h: Add a new CAM flag, CAM_RETRY_SELTO.

Add enumerated types for the various options
available with cam_error_print() and
cam_error_string().

cam_ccb.h: Add new transfer negotiation structures/types.

Change inq_len in the ccb_getdev structure to
be "reserved". This field has never been
filled in, and will be removed when we next
bump the CAM version.

cam_debug.h: Fix typo.

cam_periph.c: Modularize cam_periph_error(). The SCSI error
handling part of cam_periph_error() is now
in camperiphscsistatuserror() and
camperiphscsisenseerror().

In cam_periph_lock(), increase the reference
count on the periph while we wait for our lock
attempt to succeed so that the periph won't go
away while we're sleeping.

cam_xpt.c: Add new transfer negotiation code. (ifdefed
out)

Add a new function, xpt_path_string(). This
is a string/sbuf analog to xpt_print_path().

scsi_all.c: Revamp string handing and error printing code.
We now use sbufs for much of the string
formatting code. More of that code is shared
between userland the kernel.

scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly
useful in the first place.

Add a new error action, SS_REQSENSE. (Send a
request sense and then retry the command.)
This is useful when the controller hasn't
performed autosense for some reason.

Change the default actions around a bit.

scsi_cd.c,
scsi_da.c,
scsi_pt.c,
scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection
timeouts shouldn't be covered by a sense flag.

scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO.

Get rid of the last vestiges of a read/write
interface.

libkern/bsearch.c,
sys/libkern.h,
conf/files: Add bsearch.c, which is needed for some of the
new table lookup routines.

aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if
CAM_NEW_TRAN_CODE is defined.

sbuf.h,
subr_sbuf.c: Add the appropriate #ifdefs so sbufs can
compile and run in userland.

Change sbuf_printf() to use vsnprintf()
instead of kvprintf(), which is only available
in the kernel.

Change the source string for sbuf_cpy() and
sbuf_cat() to be a const char *.

Add __BEGIN_DECLS and __END_DECLS around
function prototypes since they're now exported
to userland.

kdump/mkioctls: Include stdio.h before cam.h since cam.h now
includes a function with a FILE * argument.

Submitted by: gibbs (mostly)
Reviewed by: jdp, marcel (libsbuf makefile changes)
Reviewed by: des (sbuf changes)
Reviewed by: ken

Do not build (and install) both secure/ and standard versions
of libtelnet, telnetd, and telnet. This only worked because
secure/ was listed late in SUBDIR in Makefile.inc1.

Reviewed by: markm

There are now compat libs for the Alpha too.

Merge into a single US-exportable libcrypt, which only provides
one-way hash functions for authentication purposes. There is no more
"set the libcrypt->libXXXcrypt" nightmare.
- Undo the libmd.so hack, use -D to hide the md5c.c internals.
- Remove the symlink hacks in release/Makefile
- the algorthm is set by set_crypt_format() as before. If this is
not called, it tries to heuristically figure out the hash format, and
if all else fails, it uses the optional auth.conf entry to chose the
overall default hash.
- Since source has non-hidden crypto in it there may be some issues with
having the source it in some countries, so preserve the "secure/*"
division. You can still build a des-free libcrypt library if you want
to badly enough. This should not be a problem in the US or exporting
from the US as freebsd.org had notified BXA some time ago. That makes
this stuff re-exportable by anyone.
- For consistancy, the default in absence of any other clues is md5. This
is to try and minimize POLA across buildworld where folk may suddenly
be activating des-crypt()-hash support. Since the des hash may not
always be present, it seemed sensible to make the stronger md5 algorithm
the default.
All things being equal, no functionality is lost.

Reviewed-by: jkh

(flame-proof suit on)

Build the ISC library as libisc. This library comes as part of the
bind distribution, but until now was not being built as a separate
entity. For documentation, see these man pages:

assertions(3), eventlib(3), heap(3), logging(3), memcluster(3), tree(3).

Reviewed by: jdp