MFC r315698:

libkvm: bump WARNS to 6 after recent commits done to resolve warnings issues

Tested with: make tinderbox; clang 4.0.0 (amd64), gcc 4.2.1/6.3.0 (amd64)

MFC r316099:

lib/libkvm: start adding basic tests for kvm(3)

- kvm_close: add a testcase to verify support for errno = EINVAL / -1
(see D10065) when kd == NULL is provided to the libcall.
- kvm_geterr:
-- Add a negative testcase for kd == NULL returning "" (see D10022).
-- Add two positive testcases:
--- test the error case using kvm_write on a O_RDONLY descriptor.
--- test the "no error" case using kvm_read(3) and kvm_nlist(3) as
helper routines and by injecting a bogus error message via
_kvm_err (an internal API) _kvm_err was used as there isn't a
formalized way to clear the error output, and because
kvm_nlist always returns ENOENT with the NULL terminator today.
- kvm_open, kvm_open2:
-- Add some basic negative tests for kvm_open(3) and kvm_open2(3).
Testing positive cases with a specific
`corefile`/`execfile`/`resolver` requires more work and would require
user intervention today in order to reliably test this out.

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Remove kvm_getfiles(3)

This libcall has been broken since (at least) r174989/8.0-RELEASE.

Bump SHLIB_MAJOR for the change

Differential Revision: https://reviews.freebsd.org/D6052
Relnotes: yes
Reviewed by: jhb, markj
Sponsored by: EMC / Isilon Storage Division

Merge the projects/release-pkg branch to head.

This allows packaging the base system with pkg(8), including
but not limited to providing the ability to provide upstream
binary update possibilities for non-tier-1 architectures.

This merge is a requirement of the 11.0-RELEASE, and as such,
thank you to everyone that has tested the project branch.

Documentation in build(7) etc. is still somewhat sparse, but
updates to those parts will follow.

Sponsored by: The FreeBSD Foundation

Add support to libkvm for reading vmcores from other architectures.
- Add a kvaddr_type to represent kernel virtual addresses instead of
unsigned long.
- Add a struct kvm_nlist which is a stripped down version of struct nlist
that uses kvaddr_t for n_value.
- Add a kvm_native() routine that returns true if an open kvm descriptor
is for a native kernel and memory image.
- Add a kvm_open2() function similar to kvm_openfiles(). It drops the
unused 'swapfile' argument and adds a new function pointer argument for
a symbol resolving function. Native kernels still use _fdnlist() from
libc to resolve symbols if a resolver function is not supplied, but cross
kernels require a resolver.
- Add a kvm_nlist2() function similar to kvm_nlist() except that it uses
struct kvm_nlist instead of struct nlist.
- Add a kvm_read2() function similar to kvm_read() except that it uses
kvaddr_t instead of unsigned long for the kernel virtual address.
- Add a new kvm_arch switch of routines needed by a vmcore backend.
Each backend is responsible for implementing kvm_read2() for a given
vmcore format.
- Use libelf to read headers from ELF kernels and cores (except for
powerpc cores).
- Add internal helper routines for the common page offset hash table used
by the minidump backends.
- Port all of the existing kvm backends to implement a kvm_arch switch and
to be cross-friendly by using private constants instead of ones that
vary by platform (e.g. PAGE_SIZE). Static assertions are present when
a given backend is compiled natively to ensure the private constants
match the real ones.
- Enable all of the existing vmcore backends on all platforms. This means
that libkvm on any platform should be able to perform KVA translation
and read data from a vmcore of any platform.

Tested on: amd64, i386, sparc64 (marius)
Differential Revision: https://reviews.freebsd.org/D3341

Revert r284417 it is not necessary anymore

Enforce overwritting SHLIBDIR

Since METAMODE has been added, sys.mk loads bsd.mkopt.mk which ends load loading
bsd.own.mk which then defines SHLIBDIR before all the Makefile.inc everywhere.

This makes /lib being populated again.

Reported by: many

Add a new ARM TARGET_ARCH, armv6hf. This is considered experimental.

This targets the existing ARMv6 and ARMv7 SoCs that contain a VFP unit.
This is an optional coprocessors may not be present in all devices, however
it appears to be in all current SoCs we support.

armv6hf targets the VFP variant of the ARM EABI and our copy of gcc is too
old to support this. Because of this there are a number of WITH/WITHOUT
options that are unsupported and must be left as the default value. The
options and their required value are:
* WITH_ARM_EABI
* WITHOUT_GCC
* WITHOUT_GNUCXX

In addition, without an external toolchain, the following need to be left
as their default:
* WITH_CLANG
* WITH_CLANG_IS_CC

As there is a different method of passing float and double values to
functions the ABI is incompatible with existing armv6 binaries. To use
this a full rebuild of world is required. Because no floating point values
are passed into the kernel an armv6 kernel with VFP enabled will work with
an armv6hf userland and vice versa.

Re-apply the part of r260022 that was reverted by r260030 with
one significant difference: for LIB32 builds both TARGET_ARCH
and MACHINE_ARCH are defined. TARGET_ARCH confusingly holds the
architecture of the host (e.g. amd64), while MACHINE_ARCH holds
the architecture were trying to build (e.g. i386). With both
set and different, r260022 changed the behaviour to interpret
the condition as building a cross-amd64 libkvm on i386, when
obviously we're trying to build an i386 version on amd64. When
COMPAT_32BIT is defined, we're building LIB32 and ignore the
value of TARGET_ARCH as we did before.

Revert part of r260022: LIB32 builds were broken by it.

Allow building a cross libkvm by setting TARGET_ARCH. The library so
produced will be called libkvm-${ARCH} instead of libkvm. This allows
installing it alongside the native version.
For symbol lookups, use ps_pglobal_lookup() instead of __fdnlist()
when building a cross libkvm. It is assumed that the cross tool that
uses the cross libkvm also provides an implementation for this
proc_services function.

Note that this commit does not change any of the architecture-specific
code for cross-compilation.

In r227839, when removing libkvm dependency on procfs(5),
kvm_uread() function, used for reading from /proc/pid/mem, was
removed too. But the function declaration remained in kvm.h
public header and the soname was not bumped.

Remove kvm_uread() from kvm.h and bump the soname.

Reported by: rmh
Discussed on: arch

Fix typo.

o Provide function kvm_read_zpcpu() to access UMA_ZONE_PCPU allocations.
o Provide function kvm_counter_u64_fetch() to fetch valut of a counter(9).

Sponsored by: Nginx, Inc.

Disconnect sun4v architecture from the three.

Some files keep the SUN4V tags as a code reference, for the future,
if any rewamped sun4v support wants to be added again.

Reviewed by: marius
Tested by: sbruno
Approved by: re

libkvm: fix process runtime calculation on crashdumps

Fix a long standing bug, where the procs ticks where assumed to be in
us. Instead, read cpu_tick_frequency from the kernel and use the same
logic to convert runtime. This is still too optimistic in that it
assumes cpu_tick_frequency is available and fixed. Since this function
is only called on crashdumps, I think we can live with that. Testing has
shown the values to be correct for different kern.hz inside Virtualbox.

Bump WARNS. Alignment issues on some archs mean this is still at 3.

Reviewed by: bde

Add minidump support for MIPS

Repair some build breakage introduced in r211725 and garbage collect some
code made obsolete in the same commit.

MFtbemd:

Prefer MACHNE_CPUARCH to MACHINE_ARCH in most contexts where you want
to test of all the CPUs of a given family conform.

A first cut at teaching libkvm how to deal with dynamic per-CPU storage
(DPCPU):

A new API, kvm_dpcpu_setcpu(3), selects the active CPU for the purposes
of DPCPU. Calls to kvm_nlist(3) will automatically translate DPCPU
symbols and return a pointer to the current CPU's version of the data.
Consumers needing to read the same symbol on several CPUs will invoke a
series of setcpu/nlist calls, one per CPU of interest.

This addition makes it possible for tools like netstat(1) to query the
values of DPCPU variables during crashdump analysis, and is based on
similar code handling virtualized global variables.

MFC after: 1 week
Sponsored by: Juniper Networks, Inc.

Build lib/ with WARNS=6 by default.

Similar to libexec/, do the same with lib/. Make WARNS=6 the norm and
lower it when needed.

I'm setting WARNS?=0 for secure/. It seems secure/ includes the
Makefile.inc provided by lib/. I'm not going to touch that directory.
Most of the code there is contributed anyway.

Make libkvm work on live systems and crashdumps with and
without VIMAGE virtualization in the kernel.

If we cannot resolve a symbol try to see if we can find it with
prefix of the virtualized subsystem, currently only "vnet_entry"
by identifying either the vnet of the current process for a
live system or the vnet of proc0 (or of dumptid if compiled
in a non-default way).

The way this is done currently allows us to only touch libkvm
but no single application. Once we are going to virtualize more
subsystems we will have to review this decision for better scaling.

Submitted by: rwatson (initial version of kvm_vnet.c, lots of ideas)
Reviewed by: rwatson
Approved by: re (kib)

Support kernel crash mini dumps on ARM architecture.

Obtained from: Juniper Networks, Semihalf

Add a new routine kvm_getcptime(3) for fetching the equivalent of
'kern.cp_time'. For a live kernel it uses the sysctl. For a crashdump,
it first checks to see if the kernel has a 'cp_time' global symbol. If
it does, it uses that. If that doesn't work, when it uses the recently
added kvm_getmaxcpu(3) and kvm_getpcpu(3) routines to walk all the CPUs
and sum up their counters.

MFC after: 1 week

Add two new routines to libkvm for working with per-CPU data:
kvm_getmaxcpu() and kvm_getpcpu().

MFC after: 1 week

unbreak build for sparc64 etc.
TARGET won't be defined on non subarches

Approved by: rwatson (mentor)
Reviewed by: jmg

buildworld fixes for sun4v
not sure why pmap.c is included as it is unchanged

Approved by: rwatson (mentor)
Reviewed by: jmg

Teach libkvm how to read minidumps. It turns out that minidumps are
far more convenient for libkvm to work with because of the page table
block at the beginning. As a result, the MD code is smaller.

libkvm will automatically detect old vs mini dumps on i386 and amd64.

libkvm will handle i386 PAE and non-PAE modes. There is a PAE flag in
the i386 minidump header to signal the width of the entries in the
page table block.

Other convenient values are also present, such as kernbase and the direct
map addresses on amd64.

style.Makefile(5)

Stage 3 of dynamic root support. Make all the libraries needed to run
binaries in /bin and /sbin installed in /lib. Only the versioned files
reside in /lib, the .so symlink continues to live /usr/lib so the
toolchain doesn't need to be modified.

Removed -I${.CURDIR}/.../sys from CFLAGS.

MAN[1-9] -> MAN.

Find <libkvm.h> in the source tree. This helps standalone builds.

Replace beforeinstall target with new variables used by .mk system.

Reviewed by: marcel, and make world

Install <kvm.h> from here (after repo copy) so it's all in one package.

Fixed unsorting of SRCS and MAN3 in previous commit.

Sorted MLINKS.

Implement kvm_getswapinfo() libkvm function. Will be used by
pstat, top, and systat.

Change ${MACHINE} into ${MACHINE_ARCH} to support MACHINE=pc98.

-I${DESTDIR}/sys -> -I${.CURDIR}/../../sys.

Add ${DESTDIR} in front of absolute path.

Make it work with our make macros.

This commit was generated by cvs2svn to compensate for changes in r1573,
which included commits to RCS files with non-trunk default branches.

BSD 4.4 Lite Lib Sources