MFC r360077:

zfsd.8: fix orphan .Xr

Though ZFS is a kernel module, it has no man page in section 4.

Reported by: phk

MFC r337160:
Do not blindly include illumos kernel headers instead of user-space.
It is not needed now, and I doubt it much helped at all, creating more
confusions then good.

MFC r334594: dwatch(1): Update manual to reference actual release

Sponsored by: Smule, Inc.

MFC r319792:
Override the locale so that file lists get a consistent sort order.

MFC SVN r334261-334262,334359: dwatch(1) touch-ups

r334261: Guard against error when given -t "*..."
r334262: Eliminate ANSI dimming in developer mode
r334359: Fix "-t test" for post-processing profiles

Bump FreeBSD_version directly in stable/11 for ports IGNORE (as in r334290)

Reviewed by: gjb
Approved by: re (gjb)
Sponsored by: Smule, Inc.

MFC SVN r329188,329334,329353,329914,329995-329996: DTrace Enhancements

r329188: Use tabs in io.d, fix alignment issues, remove extra newlines
r329334: Add errno definitions to /usr/lib/dtrace/errno.d
r329353: Add inline to errno.d for translating int to string
r329914: Updates and enhancements to io.d to aid DTrace scripting
r329995: Updates and enhancements to signal.d to aid DTrace scripting
r329996: Consistent casing for fallback SIGCHLD (s/Unknown/unknown/)

MFC SVN r330559-330560,330672,332865-332867,333513-333519: dwatch(1)

r330559: Introduce dwatch(1) as a tool for making DTrace more useful
r330560: Bump dwatch(1) internal version from 1.0-beta-91 to 1.0
r330672: Fix display of wrong pid from dtrace_sched(4)
r332865: Add `-dev' option to aid debugging of profiles
r332866: Add profile for send(2)/recv(2) syscalls
r332867: Remove the line used to demonstrate `-dev' option
r333513: Bugfix, usage displayed with `-1Q'
r333514: Separate default values so `-[BK] num' don't affect usage
r333515: Simplify info message test
r333516: Export ARGV to profiles loaded via load_profile()
r333517: Allow `-E code' to override profile EVENT_DETAILS
r333518: Expose process for ip/tcp/udp
r333519: Refactor sendrecv profile

Reviewed by: markj, gnn, bdrewery (head; earlier version)
Approved by: re (gjb)
Relnotes: yes
Sponsored by: Smule, Inc.
Differential Revision: https://reviews.freebsd.org/D15418

MFC r329681: MFV r318941: 7446 zpool create should support efi system partition

illumos/illumos-gate@7855d95b30fd903e3918bad5a29b777e765db821
https://github.com/illumos/illumos-gate/commit/7855d95b30fd903e3918bad5a29b777e765db821

https://www.illumos.org/issues/7446
Since we support whole-disk configuration for boot pool, we also will need
whole disk support with UEFI boot and for this, zpool create should create efi-
system partition.
I have borrowed the idea from oracle solaris, and introducing zpool create -
B switch to provide an way to specify that boot partition should be created.
However, there is still an question, how big should the system partition be.
For time being, I have set default size 256MB (thats minimum size for FAT32
with 4k blocks). To support custom size, the set on creation "bootsize"
property is created and so the custom size can be set as: zpool create B -
o bootsize=34MB rpool c0t0d0
After pool is created, the "bootsize" property is read only. When -B switch is
not used, the bootsize defaults to 0 and is shown in zpool get output with
value ''. Older zfs/zpool implementations are ignoring this property.
https://www.illumos.org/rb/r/219/

Reviewed by: Andrew Stormont <andyjstormont@gmail.com>
Reviewed by: Yuri Pankov <yuri.pankov@gmail.com>
Approved by: Dan McDonald <danmcd@kebe.com>
Author: Toomas Soome <tsoome@me.com>

This commit makes no sense for FreeBSD, that is why I blocked the option,
but it should be good to stay closer to upstream.

MFC r329273, r329275, r329277, r329284, r329344

r329273:
geli: append "/eli" to the underlying provider's physical path

If the underlying provider's physical path is null, then the geli device's
physical path will be, too. Otherwise, it will append "/eli". This will make
geli work better with zfsd(8).

PR: 224962
Differential Revision: https://reviews.freebsd.org/D13979

r329275:
gpart: append partition name to the underlying provider's physical path

If the underlying provider's physical path is null, then the gpart device's
physical path will be, too. Otherwise, it will append the partition name,
such as "/p1" or "/s1/a". This will make gpart work better with zfsd(8).

PR: 224965
Differential Revision: https://reviews.freebsd.org/D14010

r329277:
Add mtree entry for 329275

X-MFC-With: 329275
Sponsored by: Spectra Logic Corp

r329284:
zfsd: Allow zfsd to work on any type of GEOM provider

cddl/usr.sbin/zfsd/zfsd_event.cc
Remove the check for da and ada devices. This way zfsd can work on md,
geli, glabel, gstripe, etc devices. geli in particular is useful
combined with ZFS. gnop is also useful for simulating drive pulls in
the ZFSD test suite.

Also, eliminate the DevfsEvent class entirely. Move its
responsibilities into GeomEvent. We can get everything we need to know
just from listening to GEOM events.

lib/libdevdctl/event.cc
Fix GeomEvent::DevName for CREATE events. Oddly, the relevant field is
named "cdev" for CREATE events but "devname" for disk events.

Relnotes: Yes (probably worth mentioning the geli part)
Sponsored by: Spectra Logic Corp

r329344:
Optimize zfsd for the happy case

If there are no damaged pools, then ignore all GEOM events. We only use
them to fix damaged pools. However, still pay attention to ZFS events.

X-MFC-With: 329284
Sponsored by: Spectra Logic Corp

MFC r328605:

zfsd: Don't spare a vdev that's being replaced

If a zfs pool contains a replacing vdev (either created manually by "zpool
replace" or by zfsd(8) via autoreplace by physical path) and then new spares
get added to the pool, zfsd shouldn't use one to replace the drive that is
already being replaced. That's a waste of resources that just slows down
the rebuild.

PR: 225547
Sponsored by: Spectra Logic Corp

MFC r327888, r327972, r327973:
Add "jid" and "jailname" variables to DTrace.

MFC r326878:
Mark uctf/err.user64mode.ksh as EXFAIL for now.

MFC r326096:
Annotate pragma/err.invalidlibdep.ksh as EXFAIL.

MFC r325011, r325016

r325011:
zfsd should be able to online an L2ARC that disappears and returns

Previously, this didn't work because L2ARC devices' labels don't contain
pool GUIDs. Modify zfsd so that the pool GUID won't be required:

lib/libdevdctl/guid.h
Change INVALID_GUID from a uint64_t constant to a function that
returns an invalid Guid object. Remove the void constructor.
Nothing uses it, and it violates RAII.

cddl/usr.sbin/zfsd/case_file.h
cddl/usr.sbin/zfsd/case_file.cc
Allow CaseFile::Find to match a CaseFile based on Vdev GUID alone.
In CaseFile::ReEvaluate, attempt to online devices even if the newly
arrived device has no pool GUID.

cddl/usr.sbin/zfsd/vdev_iterator.cc
Iterate through a pool's cache devices as well as its regular
devices.

Reported by: avg
Reviewed by: avg
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D12791

r325016:
Partially revert r325011: restore Guid's default constructor

Reported by: ohartmann
X-MFC-With: 325011
Sponsored by: Spectra Logic Corp

MFC r326061, r326063:
DTrace test fixups.

MFC r322854, r323995, r324568, r324991

r322854:
zfsd(8): Close a race condition when onlining a disk paritition

When inserting a partitioned disk, devfs and geom will announce the whole
disk before they announce the partition. If the partition containing ZFS
extends to one of the disk's extents, then zfsd will see a ZFS label on the
whole disk and attempt to online it. ZFS is smart enough to activate the
partition instead of the whole disk, but only if GEOM has already created
the partition's provider.

cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h
cddl/contrib/opensolaris/lib/libzfs/common/libzfs_import.c
Add a zpool_read_all_labels method. It's similar to
zpool_read_label, but it will return the number of labels found.

cddl/usr.sbin/zfsd/zfsd_event.cc
When processing a DevFS CREATE event, only online a VDEV if we can
read all four ZFS labels.

Reviewed by: mav
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D11920

r323995:
Close a memory leak when using zpool_read_all_labels

X-MFC-With: 322854
Sponsored by: Spectra Logic Corp

r324568:
Optimize zpool_read_all_labels with AIO

Read all labels in parallel instead of sequentially

X-MFC-With: 322854
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D12495

r324991:
Fix zpool_read_all_labels when vfs.aio.enable_unsafe=0

Previously, zpool_read_all_labels was trying to do 256KB reads, which are
greater than the default MAXPHYS and therefore must go through the slow,
unsafe AIO path. Shrink these reads to 112KB so they can use the safe, fast
AIO path instead.

X-MFC-With: 324568
Sponsored by: Spectra Logic Corp

MFC r325035: MFV r325013,r325034: 640 number_to_scaled_string is duplicated in several commands

FreeBSD note: of all libcmdutils functionality ZFS (and other illumos
contrib code) currently uses only nicenum() function (which is similar
to humanize_number but has some formatting differences). For this
reason I decided to not port the whole library. As a result, nicenum.c
from libcmdutils is compiled into libzfs and libzpool. This is a bit
ugly, but works. If one day we are forced to create libillumos, then
the file should be moved to that library.

MFC r319746,r319747,r319769: 8269 dtrace stddev aggregation is normalized incorrectly

illumos/illumos-gate@79809f9cf402f130667349b2d4007ecd65d63c6f
https://github.com/illumos/illumos-gate/commit/79809f9cf402f130667349b2d4007ecd65d63c6f

https://www.illumos.org/issues/8269
It seems that currently normalization of stddev aggregation is done
incorrectly.
We divide both the sum of values and the sum of their squares by the
normalization factor. But we should divide the sum of squares by the
normalization factor squared to scale the original values properly.

FreeBSD note: the actual change was committed in r316853, this commit
adds the test files and record merge information.

Reviewed by: Bryan Cantrill <bryan@joyent.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Andriy Gapon <avg@FreeBSD.org>

Sponsored by: Panzura

MFC r322773-r322775:
Fix an off-by-two in the llquantize() action parameter validation.

MFC r314654:

cddl: normalize paths using SRCTOP-relative paths or :H when possible

This simplifies make logic/output

While here, remove bogus CFLAGS which look for headers in cddl/lib/libumem.
There aren't any source files there (just Makefiles)

MFC r320165-r320167

r320165:
devd(8): Remove pidfile on shutdown

Sponsored by: Spectra Logic Corp

r320166:
Require devd to be running for its ATF tests to run

The ATF tests communicate with the system's running devd

PR: 220169
Reported by: gjb
Sponsored by: Spectra Logic Corp

r320167:
zfsd(8): Remove pidfile on shutdown

Sponsored by: Spectra Logic Corp

MFC r312396:

Fix an unchecked return value in zfsd

It's pretty unlikely to actually hit this, but good to check it anyway

Reported by: Coverity
CID: 1362018
MFC after: 4 weeks
Sponsored by: Spectra Logic Corp

MFC r307400, r307401:
DTrace test fixes.

MFC r309698, r309699, r309700:
DTrace test fixes.

MFC r305148:

DIRDEPS_BUILD: Add some missing dirctories to the build.

MFC r305209: MFV r302660: 6314 buffer overflow in dsl_dataset_name

illumos/illumos-gate@9adfa60d484ce2435f5af77cc99dcd4e692b6660
https://github.com/illumos/illumos-gate/commit/9adfa60d484ce2435f5af77cc99dcd4e6
92b6660

https://www.illumos.org/issues/6314
Callers of dsl_dataset_name pass a buffer of size ZFS_MAXNAMELEN, but
dsl_dataset_name copies the datasets' name PLUS the snapshot name to it,
resulting in a max of 2 * ZFS_MAXNAMELEN + '@'.

Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Igor Kozhukhov <ikozhukhov@gmail.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Matthew Ahrens <mahrens@delphix.com>

MFC r305018,r305019,r305020:

r305018:

Use SRCTOP instead of a homegrown definition for it (SRCDIR)

r305019:

Remove unnecessary variable (SRCDIR) replaced by SRCTOP in Makefile.common

r305020:

Remove redundant declarations and simplify ../ in pathing

- TESTSBASE and LOCALBASE are already defined in bsd.tests.mk
- TESTSDIR is automatically divined as ${TESTSBASE}${RELDIR:H} after
r289158.
- Replace SRCDIR with SRCTOP

MFC r305013:

Add an empty virtual destructor to zfsd's Vdev class. This is needed
because the class has virtual functions, and the compiler-generated
default destructor is non-virtual.

Reviewed by: asomers

MFC r305016:

Fix the zfsd unittest:
* TESTSDIR is supposed to be under cddl/usr.sbin, not cddl/sbin
* DevdCtl::EventBuffer no longer exists, so remove its forward
declaration

MFC r303900:

Highball memory requirement (4GB) with common/{raise,safety}

Both test suites require more memory than my amd64 VM using
GENERIC-NODEBUG can provide and reliably panic it with OOM issues in
dtrace(4).

Some of the testcases fail, but this at least bypasses the panic behavior
on platforms that don't have enough resources

Discussed with: markj

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Coverity fixes for r300906

lib/libdevdctl/consumer.cc
In Consumer::DisconnectFromDevd, don't close the socket if it's
already closed.

cddl/usr.sbin/zfsd/case_file.cc
lib/libdevdctl/consumer.h
Delete dead code leftover from before devd(8) gained SOCK_SEQPACKET
support

Reported by: Coverity
CID: 1356155, 1356169
Sponsored by: Spectra Logic Corp

zfsd: Remove a redundant semicolon,

zfsd: minor spelling fix.

Avoid more literal-suffix errors with C++11

zfsd(8), the ZFS fault management daemon

Add zfsd, which deals with hard drive faults in ZFS pools. It manages
hotspares and replements in drive slots that publish physical paths.

cddl/usr.sbin/zfsd
Add zfsd(8) and its unit tests

cddl/usr.sbin/Makefile
Add zfsd to the build

lib/libdevdctl
A C++ library that helps devd clients process events

lib/Makefile
share/mk/bsd.libnames.mk
share/mk/src.libnames.mk
Add libdevdctl to the build. It's a private library, unusable by
out-of-tree software.

etc/defaults/rc.conf
By default, set zfsd_enable to NO

etc/mtree/BSD.include.dist
Add a directory for libdevdctl's include files

etc/mtree/BSD.tests.dist
Add a directory for zfsd's unit tests

etc/mtree/BSD.var.dist
Add /var/db/zfsd/cases, where zfsd stores case files while it's shut
down.

etc/rc.d/Makefile
etc/rc.d/zfsd
Add zfsd's rc script

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c
Fix the resource.fs.zfs.statechange message. It had a number of
problems:

It was only being emitted on a transition to the HEALTHY state.
That made it impossible for zfsd to take actions based on drives
getting sicker.

It compared the new state to vdev_prevstate, which is the state that
the vdev had the last time it was opened. That doesn't make sense,
because a vdev can change state multiple times without being
reopened.

vdev_set_state contains logic that will change the device's new
state based on various conditions. However, the statechange event
was being posted _before_ that logic took effect. Now it's being
posted after.

Submitted by: gibbs, asomers, mav, allanjude
Reviewed by: mav, delphij
Relnotes: yes
Sponsored by: Spectra Logic Corp, iX Systems
Differential Revision: https://reviews.freebsd.org/D6564

Add initial DTrace support for RISC-V.

Sponsored by: DARPA, AFRL
Sponsored by: HEIF5

Remove the old version of the DTraceToolkit from the source tree.
The DTraceToolkit is part of the Open DTrace effort and is supported
on FreeBSD as a port (sysutils/DTraceToolkit) which has been updated
to properly track toolkit development upstream.

Sponsored by: DARPA, AFRL

Fix DTrace test ATF wrapper generation.

Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed
after r298107

Summary of changes:

- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
namespacing is kept with FILES appropriately, and that this shouldn't need
to be repeated if the namespace changes -- only the definition of PACKAGE
needs to be changed
- Allow PACKAGE to be overridden by callers instead of forcing it to always be
`tests`. In the event we get to the point where things can be split up
enough in the base system, it would make more sense to group the tests
with the blocks they're a part of, e.g. byacc with byacc-tests, etc
- Remove PACKAGE definitions where possible, i.e. where FILES wasn't used
previously.
- Remove unnecessary TESTSPACKAGE definitions; this has been elided into
bsd.tests.mk
- Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES;
${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk.
- Fix installation of files under data/ subdirectories in lib/libc/tests/hash
and lib/libc/tests/net/getaddrinfo
- Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup)

Document the proposed changes in share/examples/tests/tests/... via examples
so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of
replacing FILES. share/mk/bsd.README didn't seem like the appropriate method
of communicating that info.

MFC after: never probably
X-MFC with: r298107
PR: 209114
Relnotes: yes
Tested with: buildworld, installworld, checkworld; buildworld, packageworld
Sponsored by: EMC / Isilon Storage Division

Fix including Kyuafile in packaged base system.

Fix a related typo while here.

Note, this change results in the Kyuafile inclusion in the runtime
package, which needs to be fixed, however addresses the PR as far
as I can tell in my tests.

PR: 209114
Submitted by: ngie
Sponsored by: The FreeBSD Foundation

Merge the projects/release-pkg branch to head.

This allows packaging the base system with pkg(8), including
but not limited to providing the ability to provide upstream
binary update possibilities for non-tier-1 architectures.

This merge is a requirement of the 11.0-RELEASE, and as such,
thank you to everyone that has tested the project branch.

Documentation in build(7) etc. is still somewhat sparse, but
updates to those parts will follow.

Sponsored by: The FreeBSD Foundation

DIRDEPS_BUILD: Connect MK_TESTS.

Sponsored by: EMC / Isilon Storage Division

Test directories can build in parallel fine.

Sponsored by: EMC / Isilon Storage Division

Reduce zfs utilities overlinking

Fix a discrepancy in r291738.

The script that generates these makefiles was changed to modify LIBADD
rather than LDADD/DPADD, but the makefile itself was also changed in a
slightly different way.

Update DTrace test suite makefiles after r291963.

Fix LDADD/DPADD that should be LIBADD.

Sponsored by: EMC / Isilon Storage Division

Reduce overlinking of libdtrace consumers with libctf, libelf, libproc.

The proper place for this list is _DP_dtrace.

Due to removing the LDADD_dtrace, more LIBADD are needed in
cddl/usr.sbin/dtrace to prevent underlinking.

This fixes overlinking in cddl/usr.sbin/lockstat and
cddl/usr.sbin/plockstat.

Sponsored by: EMC / Isilon Storage Division

Fully connect cddl/usr.sbin/plockstat.

There seems to be no reason to keep this so private.

Also add missing optional MK_CDDL files.

Sponsored by: EMC / Isilon Storage Division

META MODE: Fix userland/cddl.

MK_CTF controls whether ctfmerge is ran against the binaries, not
whether CTF tools should be built.

Sponsored by: EMC / Isilon Storage Division

Rename Makefile.inc1 to dtrace.test.mk to avoid clash with top-level Makefile.inc1.

Discussed with: ngie, markj
Sponsored by: EMC / Isilon Storage Division

Reduce the Makefile snippet complexity a bit

- Set BINDIR to TESTSDIR globally (and subsequently, remove all
`${FILESGROUP}DIR` setting because BINDIR is set to `TESTSDIR`)
- Set MAN to "" globally, instead of per-PROG

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Remove unneeded _RECURSING_PROGS check.

It is definitely not needed after r288158, and is a private variable as well
that should not be checked here.

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Refactor the test/ Makefiles after recent changes to bsd.test.mk (r289158) and
netbsd-tests.test.mk (r289151)

- Eliminate explicit OBJTOP/SRCTOP setting
- Convert all ad hoc NetBSD test integration over to netbsd-tests.test.mk
- Remove unnecessary TESTSDIR setting
- Use SRCTOP where possible for clarity

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Divison

Update DTrace test makefiles after r288415.

META_MODE: Remove DEP_RELDIR from Makefile.depend files.

This has not been needed since r284171 in projects/bmake.

Sponsored by: EMC / Isilon Storage Division

Add a src.conf option to build and install the DTrace test suite.

Reviewed by: gnn, ngie
Differential Revision: https://reviews.freebsd.org/D3195

First cut of DTrace for AArch64.

Reviewed by: andrew, emaste
Sponsored by: ARM Limited
Differential Revision: https://reviews.freebsd.org/D2738

Add META_MODE support.

Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.

Still need to add real targets under targets/ to build packages.

Differential Revision: D2796
Reviewed by: brooks imp

Build cddl/{sbin,usr.bin,usr.sbin} in parallel as all of the applications are
freestanding (they require libraries build via make libraries in buildworld)

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Reduce overlinking.
Because of libdtrace there is still a bit a overlinking but nothing we can deal
with easily

Correctly link libdtrace and convert to LIBADD
Make dtrace only link to libdtrace

Fix a typo that snuck in with r283024, and remove the EXFAIL annotation from
a test which now passes as a result of that change.

Replace dtest.pl, the upstream DTrace test suite harness, with a shell
script. This reimplementation is much simpler than dtest.pl and is more
amenable to being run under Kyua - dtest.pl writes error output to a
temporary directory that is deleted when the run finishes, making it hard
to debug test failures. This change also removes the test suite's dependency
on perl.

Add infrastructure to integrate the DTrace test suite with Kyua.

For each test category, we generate a script containing ATF test cases for
the tests under that category. Each test case simply runs dtest.pl (the
upstream test harness) with the corresponding test files. The exclude.sh
script is used to record info about tests which should be skipped or are
expected to fail; it is used to generate atf_skip and atf_expect_fail calls.
The genmakefiles.sh script can be used to regenerate the test makefiles when
new tests are brought it from upstream.

The test suite is currently not connected to the build as there is a small
number of lingering test issues which still need to be worked out. In the
meantime however, the test suite can be easily built and installed
manually from cddl/usr.sbin/dtrace/tests.

Reviewed by: ngie
Sponsored by: EMC / Isilon Storage Division

Initial version of DTrace on ARM32.

Submitted by: Howard Su based on work by Oleksandr Tymoshenko
Reviewed by: ian, andrew, rpaulo, markj

Build plockstat if WITH_PLOCKSTAT is defined.

Make DTrace stuff compile with C99 standard.

Fix powerpc build:

Chase r270227 and compile lockstat with C99 standard.

Suggested by: bde

MFV r267565:

4757 ZFS embedded-data block pointers ("zero block compression")
4913 zfs release should not be subject to space checks

MFC after: 2 weeks

Use src.opts.mk in preference to bsd.own.mk except where we need stuff
from the latter.

Add placeholder Kyuafiles for various top-level hierarchies.

This change adds tests/ directories in the source tree to create various
subdirectories in /usr/tests/ and to install placeholder Kyuafiles for
them.

the relevant hierarchies are: cddl, etc, games, gnu and secure.

The reason for this is to simplify the addition of new test programs for
utilities or libraries under any of these directories. Doing so on a
case by case basis is unnecessary and is quite an obscure process.

NO_MAN= has been deprecated in favor of MAN= for some time, go ahead
and finish the job. ncurses is now the only Makefile in the tree that
uses it since it wasn't a simple mechanical change, and will be
addressed in a future commit.

Convert the lockstat(1) man page to mdoc and make sure that it gets
installed. Additionally, remove Solaris-specific sections and references,
and replace example outputs with output from lockstat on FreeBSD, since
lockstat's output contains stack traces.

This change also removes some examples that don't seem to work properly on
FreeBSD. The examples should be re-added when lockstat is fixed.

Reported by: avg
MFC after: 1 week

Build all ZFS testing & debugging tools with -g.

These programs and everything using libzpool rely on the embedded asserts to
verify the correctness of operations. Given that, the core dumps would be
useless without debug symbols.

Merge libzfs_core branch:
includes MFV 238590, 238592, 247580

MFV 238590, 238592:
In the first zfs ioctl restructuring phase, the libzfs_core library was
introduced. It is a new thin library that wraps around kernel ioctl's.
The idea is to provide a forward-compatible way of dealing with new
features. Arguments are passed in nvlists and not random zfs_cmd fields,
new-style ioctls are logged to pool history using a new method of
history logging.

http://blog.delphix.com/matt/2012/01/17/the-future-of-libzfs/

MFV 247580 [1]:
To address issues of several deadlocks and race conditions the locking
code around dsl_dataset was rewritten and the interface to synctasks
was changed.

User-Visible Changes:
"zfs snapshot" can create more arbitrary snapshots at once (atomically)
"zfs destroy" destroys multiple snapshots at once
"zfs recv" has improved performance

Backward Compatibility:
I have extended the compatibility layer to support full backward
compatibility by remapping or rewriting the responsible ioctl arguments.
Old utilities are fully supported by the new kernel module.

Forward Compatibility:
New utilities work with old kernels with the following restrictions:
- creating, destroying, holding and releasing of multiple snapshots
at once is not supported, this includes recursive (-r) commands

Illumos ZFS issues:
2882 implement libzfs_core
2900 "zfs snapshot" should be able to create multiple,
arbitrary snapshots at once
3464 zfs synctask code needs restructuring

References:
https://www.illumos.org/issues/2882
https://www.illumos.org/issues/2900
https://www.illumos.org/issues/3464 [1]

MFC after: 1 month
Sponsored by: Hybrid Logic Inc. [1]

Implement DTrace for PowerPC. This includes both 32-bit and 64-bit.

There is one known issue: Some probes will display an error message along the
lines of: "Invalid address (0)"

I tested this with both a simple dtrace probe and dtruss on a few different
binaries on 32-bit. I only compiled 64-bit, did not run it, but I don't expect
problems without the modules loaded. Volunteers are welcome.

MFC after: 1 month

Merge recent zfs vendor changes, sync code and adjust userland DEBUG.

Illumos issued covered:
1884 Empty "used" field for zfs *space commands
3006 VERIFY[S,U,P] and ASSERT[S,U,P] frequently check if first argument
is zero
3028 zfs {group,user}space -n prints (null) instead of numeric GID/UID
3048 zfs {user,group}space [-s|-S] is broken
3049 zfs {user,group}space -t doesn't really filter the results
3060 zfs {user,group}space -H output isn't tab-delimited
3061 zfs {user,group}space -o doesn't use specified fields order
3064 usr/src/cmd/zpool/zpool_main.c misspells "successful"
3093 zfs {user,group}space's -i is noop
3098 zfs userspace/groupspace fail without saying why when run as non-root

References:
https://www.illumos.org/issues/ + [issue_id]

Obtained from: illumos (vendor/illumos, vendor/illumos-sys)
MFC after: 2 weeks

Remove trailing whitespace.

Remove trailing whitespace.

Introduce "feature flags" for ZFS pools (bump SPA version to 5000).
Add first feature "com.delphix:async_destroy" (asynchronous destroy
of ZFS datasets).
Implement features support in ZFS boot code.

Illumos revisions merged:
13700:2889e2596bd6
13701:1949b688d5fb
2619 asynchronous destruction of ZFS file systems
2747 SPA versioning with zfs feature flags

References:
https://www.illumos.org/issues/2619
https://www.illumos.org/issues/2747

Obtained from: illumos (issue #2619, #2747)
MFC after: 1 month

Enable build of DTrace-related userland parts for MIPS

For now, remove MANFILTER from ZFS manpages (bad effect on html output).

MFC after: 4 days

Update and desolarization of zdb(8) and zstreamdump(1) manual pages:
- synchronized to match new vendor code [1]
- removed ATTRIBUTES sections
- updated SEE ALSO sections
- properly updated copyright information (required by CDDL)
- remove empty lines via MANFILTER

Obtained from: Illumos [1]
MFC after: 5 days

Fix a few gratuitous library dependencies. Some of the ZFS utilities
are linked with libraries they don't use:

- zinject doesn't use libavl
- ztest doesn't use libz
- zdb uses neither libavl nor libz
- zfs uses neither libbsdxml nor libm, nor libsbuf
- zpool uses neither libbsdxml nor libm, nor libsbuf

In addition, libzfs needs libm because it uses pow(), however it isn't
linked with -lm. This went unnoticed because all its users had -lm before.

Reviewed by: pjd, mm
Approved by: kib (mentor)
MFC after: 1 week

Finally... Import the latest open-source ZFS version - (SPA) 28.

Few new things available from now on:

- Data deduplication.
- Triple parity RAIDZ (RAIDZ3).
- zfs diff.
- zpool split.
- Snapshot holds.
- zpool import -F. Allows to rewind corrupted pool to earlier
transaction group.
- Possibility to import pool in read-only mode.

MFC after: 1 month

Minor grammar fixes.

Add a man page for plockstat.

Sponsored by: The FreeBSD Foundation

Install dtruss.

Sponsored by: The FreeBSD Foundation

Add a man page for dtruss.

Sponsored by: The FreeBSD Foundation

Use double quotes when checking the value of MACHINE_ARCH.

Sponsored by: The FreeBSD Foundation

Add a Makefile for dtruss.

Sponsored by: The FreeBSD Foundation

Port plockstat to FreeBSD. This will be connected to the build later.

Sponsored by: The FreeBSD Foundation

lockstat(1) now depends on libutil and librtld_db.

Sponsored by: The FreeBSD Foundation

libdtrace, dtrace(1) and lockstat(1) are for i386 and amd64 only right
now.

Sponsored by: The FreeBSD Foundation

dtrace(1) now depends on libutil and librtld_db.

Sponsored by: The FreeBSD Foundation

Remove manual .includes in cddl Makefiles

- Break the dependency on ../Makefile.inc for .PATH, and include
../Makefile.inc implicitly. This is required to ...
- Set WARNS?=6 in top-level Makefile.inc
- Remove now redundant WARNS settings, add WARNS?=0 where appropriate
- Remove redundant SHLIB_MAJOR overrides
- Use NO_MAN, not MK_MAN=no
- Remove redundant inclusion of bsd.own.mk
- Order Makefiles more according to style.Makefile(9)
- Reduce diff of cddl Makefiles against each other

No objection: pjd
Approved by: ed (co-mentor)

Fixed dependencies (make checkdpadd).

Add the OpenSolaris lockstat(1M) command. Requires the dtrace driver,
the lockstat provider, and the ksyms(4) pseudo driver kernel modules.

Approved by: gnn (mentor)

Switch over to gnu99 compilation on default for userland.

Tested by: make universe
Tested by: ports exp build (done by pav)
Reviewed by: ru
Reviewed by: silence on arch
Approved by: ed (mentor)

Update ZFS from version 6 to 13 and bring some FreeBSD-specific changes.

This bring huge amount of changes, I'll enumerate only user-visible changes:

- Delegated Administration

Allows regular users to perform ZFS operations, like file system
creation, snapshot creation, etc.

- L2ARC

Level 2 cache for ZFS - allows to use additional disks for cache.
Huge performance improvements mostly for random read of mostly
static content.

- slog

Allow to use additional disks for ZFS Intent Log to speed up
operations like fsync(2).

- vfs.zfs.super_owner

Allows regular users to perform privileged operations on files stored
on ZFS file systems owned by him. Very careful with this one.

- chflags(2)

Not all the flags are supported. This still needs work.

- ZFSBoot

Support to boot off of ZFS pool. Not finished, AFAIK.

Submitted by: dfr

- Snapshot properties

- New failure modes

Before if write requested failed, system paniced. Now one
can select from one of three failure modes:
- panic - panic on write error
- wait - wait for disk to reappear
- continue - serve read requests if possible, block write requests

- Refquota, refreservation properties

Just quota and reservation properties, but don't count space consumed
by children file systems, clones and snapshots.

- Sparse volumes

ZVOLs that don't reserve space in the pool.

- External attributes

Compatible with extattr(2).

- NFSv4-ACLs

Not sure about the status, might not be complete yet.

Submitted by: trasz

- Creation-time properties

- Regression tests for zpool(8) command.

Obtained from: OpenSolaris

Add the dtrace client app to the build.

Remove options MK_LIBKSE and DEFAULT_THREAD_LIB now that we no longer
build libkse. This should fix WITHOUT_LIBTHR builds as a side effect.

The sources covered by Sun's CDDL have been repo copied below the
src/cddl and src/sys/cddl directories per the core@ decision following
the license review.

This change modifies the affected Makefiles to reference the sources
in their new location.

Tweak the handling of "WITHOUT_LIBPTHREAD". Also remove the accidental
treatment of 'LIBKSE' as an "old style" knob.

Submitted by: ru
Approved by: re(kensmith)

Repo copy libpthreads to libkse.
This introduces the WITHOUT_LIBKSE nob,
and changes WITHOUT_LIBPTHREADS to mean with neither threading libs.
Approved by: re(kensmith)

Don't build bits that depend on the pthreads support if a
system was configured without such support.

Approved by: re (kensmith)

Fixed static linkage (build with -DNO_SHARED).

Approved by: re (kensmith)

- Reduce number of atomic operations needed to be implemented in asm by
implementing some of them using existing ones.
- Allow to compile ZFS on all archs and use atomic operations surrounded
by global mutex on archs we don't have or can't have all atomic
operations needed by ZFS.

Link libzpool, ztest and zdb against libpthread.

Requested by: ru

Apply a set of style.Makefile(5) changes to src/cddl/ makefiles.
There are some insignificant non-style changes as well.

Not fixed: makefiles use ${LIBTHR} that doesn't exist, thus
breaking "make checkdpadd" and not tracking dependencies
properly.

Approved by: pjd

Build ZFS on amd64 and pc98.

Approved by: pjd@

Please welcome ZFS - The last word in file systems.

ZFS file system was ported from OpenSolaris operating system. The code in under
CDDL license.

I'd like to thank all SUN developers that created this great piece of software.

Supported by: Wheel LTD (http://www.wheel.pl/)
Supported by: The FreeBSD Foundation (http://www.freebsdfoundation.org/)
Supported by: Sentex (http://www.sentex.net/)