Remove svn:mergeinfo from the releng/10.0 branch.

After branch creation from stable/10, the stable/10 branch mergeinfo
was moved to the root of the branch.

Since there have not been any merges from stable/10 to releng/10.0
yet, we do not need to track any of the existing mergeinfo here.

Merges to releng/10.0 should now be done to the root of the branch.

For future branches during the release cycle, unless otherwise noted,
this change will be done as part of the stable/ and releng/ branch
creation.

Discussed with: peter
Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

- Copy stable/10 (r259064) to releng/10.0 as part of the
10.0-RELEASE cycle.
- Update __FreeBSD_version [1]
- Set branch name to -RC1

[1] 10.0-CURRENT __FreeBSD_version value ended at '55', so
start releng/10.0 at '100' so the branch is started with
a value ending in zero.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Convert the dtrace(1) man page to mdoc and fix up some aspects of it that
don't make sense on FreeBSD. In particular,

- remove the ATTRIBUTES section,
- remove references to the Solaris Dynamic Tracing Guide, except in the
SEE ALSO section,
- update the description of the -A option for FreeBSD's implementation,
- remove references to Solaris-specific programs and configuration files,
and replace them with FreeBSD equivalents where possible.

The content has not changed aside from this.

Approved by: re (joel)
MFC after: 1 week

Fix implicit declaration of jail_getid()

Approved by: re

Add a separate translator for headers passed to the TCP probes in the
input path. These probes get some of the fields in host order, whereas the
output probes get them in network order, so a single translator isn't
enough. This workaround ensures that the problem is essentially invisble
to users: none of the probe arguments or their fields have changed.

Approved by: re (hrs)

MFV r254750:

Add support of Illumos dumps on zvol over RAID-Z.

Note that this only adds the features. FreeBSD would
still need more work to support dumping on zvols.

Illumos ZFS issues:
2932 support crash dumps to raidz, etc. pools

MFC after: 1 month
Approved by: re (ZFS blanket)

Use the address of the inpcb rather than the tcpcb to identify TCP
connections. This keeps the tcp provider consistent with the other network
providers.

Approved by: re (delphij)

MFV r247844 (illumos-gate 13975:ef6409bc370f)

Illumos ZFS issues:
3582 zfs_delay() should support a variable resolution
3584 DTrace sdt probes for ZFS txg states

Provide a compatibility shim for Solaris's cv_timedwait_hires
to help aid future porting.

Approved by: re (ZFS blanket)

Build all ZFS testing & debugging tools with -g.

These programs and everything using libzpool rely on the embedded asserts to
verify the correctness of operations. Given that, the core dumps would be
useless without debug symbols.

Merge various CTF fixes from illumos

2942 CTF tools need to handle files which legitimately lack data
2978 ctfconvert still needs to ignore legitimately dataless files on SPARC

Illumos Revisions: 13745:6b3106b4250f
13754:7231b684c18b

Reference:

https://www.illumos.org/issues/2942
https://www.illumos.org/issues/2978

MFC after: 3 weeks

Implement the ip, tcp, and udp DTrace providers. The probe definitions use
dynamic translation so that their arguments match the definitions for
these providers in Solaris and illumos. Thus, existing scripts for these
providers should work unmodified on FreeBSD.

Tested by: gnn, hiren
MFC after: 1 month

MFV r254751:

Don't treat the parameter as a number (pool GUID) when there is
error converting it from string, instead, treat it as the pool
name.

Illumos ZFS issues:
1765 assert triggered in libzfs_import.c trying to import pool
name beginning with a number

MFV r254748:

Fix memory leak in libzfs's iter_dependents_cb().

Illumos ZFS issues:
4061 libzfs: memory leak in iter_dependents_cb()

MFV r254746:

To quote original Illumos ticket:

libctf thinks that any ELF file containing more than 65536 sections is
corrupt, because it doesn't understand the SHN_XINDEX magic.

Illumos DTrace issues:
4005 libctf can't deal with extended sections

MFV r254422:

Illumos DTrace issues:
3089 want ::typedef
3094 libctf should support removing a dynamic type
3095 libctf does not validate arrays correctly
3096 libctf does not validate function types correctly

Enhance the ZFS vdev layer to maintain both a logical and a physical
minimum allocation size for devices. Use this information to
automatically increase ZFS's minimum allocation size for new top-level
vdevs to a value that more closely matches the optimum device
allocation size.

Use GEOM's stripesize attribute, if set, as the physical sector
size of the GEOM.

Calculate the minimum blocksize of each metaslab class. Use the
calculated value instead of SPA_MINBLOCKSIZE (512b) when determining
the likelyhood of compression yeilding a reduction in physical space
usage.

Report devices with sub-optimal block size configuration in "zpool
status". Also properly fail attempts to attach devices with a
logical block size greater than 8kB, since this will cause corruption
to ZFS's label area.

Sponsored by: Spectra Logic Corporaion
MFC after: 2 weeks

Background
==========
Many modern devices use physical allocation units that are much
larger than the minimum logical allocation size accessible by
external commands. Two prevalent examples of this are 512e disk
drives (512b logical sector, 4K physical sector) and flash devices
(512b logical sector, 4K or larger allocation block size, and 128k
or larger erase block size). Operations that modify less than the
physical sector size result in a costly read-modify-write or garbage
collection sequence on these devices.

Simply exporting the true physical sector of the device to ZFS would
yield optimal performance, but has two serious drawbacks:

1) Existing pools created with devices that have different logical
and physical block sizes, but were configured to use the logical
block size (e.g. because the OS version used for pool construction
reported the logical block size instead of the physical block
size) will suddenly find that the vdev allocation size has
increased. This can be easily tolerated for active members of
the array, but ZFS would prevent replacement of a vdev with
another identical device because it now appears that the smaller
allocation size required by the pool is not supported by the new
device.

2) The device's physical block size may be too large to be supported
by ZFS. The optimal allocation size for the vdev may be quite
large. For example, a RAID controller may export a vdev that
requires read-modify-write cycles unless accessed using 64k
aligned/sized requests. ZFS currently has an 8k minimum block
size limit.

Reporting both the logical and physical allocation sizes for vdevs
solves these problems. A device may be used so long as the logical
block size is compatible with the configuration. By comparing the
logical and physical block sizes, new configurations can be optimized
and administrators can be notified of any existing pools that are
sub-optimal.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h:
Add the SPA_ASHIFT constant. ZFS currently has a hard upper
limit of 13 (8k) for ashift and this constant is used to
both document and enforce this limit.

sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h:
Add the VDEV_AUX_ASHIFT_TOO_BIG error code.

Add fields for exporting the configured, logical, and
physical ashift to the vdev_stat_t structure.

Add VDEV_STAT_VALID() macro which can be used to verify the
presence of required vdev_stat_t fields in nvlist data.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c:
Provide a SYSCTL_PROC handler for "max_auto_ashift". Since
the limit is only referenced long after boot when a create
operation occurs, there's no compelling need for it to be
a boot time configurable tunable. This also allows the
validation code for the max_auto_ashift value to be contained
within the sysctl handler.

Populate the new fields in the vdev_stat_t structure.

Fail vdev opens if the vdev reports an ashift larger than
SPA_MAXASHIFT.

Propogate vdev_logical_ashift and vdev_physical_ashift between
child and parent vdevs as is done for vdev_ashift.

In vdev_open(), restore code that fails opens for devices
where vdev_ashift grows. This can only happen now if the
device's logical ashift grows, which means it really isn't
safe to use the device.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_file.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_missing.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_raidz.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_root.c:
Update the vdev_open() API so that both logical (what was
just ashift before) and physical ashift are reported.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h:
Add two new fields, vdev_physical_ashift and vdev_logical_ashift,
to vdev_t.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_config.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:
Add vdev_ashift_optimize(). Call it anytime a new top-level
vdev is allocated.

cddl/contrib/opensolaris/cmd/zpool/zpool_main.c:
Add text for the VDEV_AUX_ASHIFT_TOO_BIG error.

For each sub-optimally configured leaf vdev, report configured
and native block sizes.

cddl/contrib/opensolaris/cmd/zpool/zpool_main.c:
cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h:
cddl/contrib/opensolaris/lib/libzfs/common/libzfs_status.c:
Introduce a new zpool status: ZPOOL_STATUS_NON_NATIVE_ASHIFT.
This status is reported on healthy pools containing vdevs
configured to use a block size smaller than their reported
physical block size.

cddl/contrib/opensolaris/lib/libzfs/common/libzfs_status.c:
Update find_vdev_problem() and supporting functions to
provide the full vdev_stat_t structure to problem checking
routines, and to allow decent into replacing vdevs.

Add a vdev_non_native_ashift() validator which is used on
the full vdev tree to check for ZPOOL_STATUS_NON_NATIVE_ASHIFT.

cddl/contrib/opensolaris/lib/libzpool/common/kernel.c:
cddl/contrib/opensolaris/lib/libzpool/common/sys/zfs_context.h:
Enhance sysctl userland stubs now that a SYSCTL_PROC handler
is used in vdev.c.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab_impl.h:
When the group membership of a metaslab class changes (i.e.
when a vdev is added or removed from a pool), walk the group
list to determine the smallest block size currently available
and record this in the metaslab class.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab.h:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c:
Add the metaslab_class_get_minblocksize() accessor.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio_compress.h:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio_compress.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:
In zio_compress_data(), take the minimum blocksize as an
input parameter instead of assuming SPA_MINBLOCKSIZE.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:
In l2arc_compress_buf(), pass SPA_MINBLOCKSIZE as the minimum
blocksize of the device. The l2arc code performs has it's own
code for deciding if compression is worth while, so this
effectively disables zio_compress_data() from second guessing
the original decision.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:
In zio_write_bp_init(), use the minimum blocksize of the
normal metaslab class when compressing data.

MFV r254421:

Illumos ZFS issues:
3996 want a libzfs_core API to rollback to latest snapshot

Load the dtraceall module if /dev/dtrace/dtrace doesn't exist.

MFC after: 3 days

MFV r254079:

Illumos ZFS issues:
3957 ztest should update the cachefile before killing itself
3958 multiple scans can lead to partial resilvering
3959 ddt entries are not always resilvered
3960 dsl_scan can skip over dedup-ed blocks if
physical birth != logical birth
3961 freed gang blocks are not resilvered and can cause pool to suspend
3962 ztest should print out zfs debug buffer before exiting

MFV r254071:

Fix a regression introduced by fix for Illumos bug #3834. Quote from
Matthew Ahrens on the Illumos issue:

ztest fails this assertion because ztest_dmu_read_write() does
dmu_tx_hold_free(tx, bigobj, bigoff, bigsize);
and then
dmu_object_set_checksum(os, bigobj,
(enum zio_checksum)ztest_random_dsl_prop(ZFS_PROP_CHECKSUM), tx);

If the region to free is past the end of the file, the DMU assumes that there
will be nothing to do for this object. However, ztest does set_checksum(),
which must modify the dnode. The fix is for ztest to also call

dmu_tx_hold_bonus(tx, bigobj);

so we can account for the dirty data associated with setting the checksum

Illumos ZFS issues:
3955 ztest failure: assertion refcount_count(&tx->tx_space_written)
+ delta <= tx->tx_space_towrite

MFV r254070:

Merge vendor bugfix for ZFS test suite that triggers false positives.

Illumos ZFS issues:
3949 ztest fault injection should avoid resilvering devices
3950 ztest: deadman fires when we're doing a scan
3951 ztest hang when running dedup test
3952 ztest: ztest_reguid test and ztest_fault_inject don't place nice together

Fix implicit declaration of warnx().

MFV r253781 + r253871:

Illumos ZFS issues:
3894 zfs should not allow snapshot of inconsistent dataset

MFC after: 2 weeks

MFV r253784:

Fix zfs send -D hang after processing requiring a CTRL+C to interrupt due to
pthread_join prior to fd close.

This was introduced by r251646 (MFV r251644)

Illumos ZFS issue:
3909 "zfs send -D" does not work

MFC after: 1 day

DTrace: re-apply r249426 now that the underlying issues have been solved.

Merge change from illumos:

3519 DTrace fails to resolve const types from fbt
3520 dtrace internal error -- token type 316 is not a valid D
compilation token
3521 clean up dtrace unit tests

Illumos Revision: e98f46c

Reference:
https://www.illumos.org/issues/3519
https://www.illumos.org/issues/3520
https://www.illumos.org/issues/3521

Tested by: Fabian Keil
Obtained from: Illumos
MFC after: 1 month

DTrace: re-merge remainder of r249367 (original from Illumos).

Bring back some important fixes from Illumos:

3022 DTrace: keys should not affect the sort order when sorting by value
3023 it should be possible to dereference dynamic variables
3024 D integer narrowing needs some work

We particularly avoid the LD_NOLAZYLOAD changes that Illumos made
as those don't apply to FreeBSD and were causing problems in
interactive mode.

Illumos Revision: 13758:23432da34147

Reference:

https://www.illumos.org/issues/3022
https://www.illumos.org/issues/3023
https://www.illumos.org/issues/3024

MFC after: 1 month
Tested by: markj

Use kern_ioctl() rather than ioctl() for testing the FBT provider, since the
latter doesn't exist in FreeBSD. All the tests under fbtprovider pass now.

Style issue in r253661.

Pointed out by: avg
MFC after: 1 month

Fix a segfault in ctfmerge due to a bug in gcc.

GCC can generate bogus dwarf attributes with DW_AT_byte_size
set to 0xFFFFFFFF.
The issue was originaly detected in NetBSD but it has been
adapted for portability and to avoid compiler warnings.

Reference:
https://www.illumos.org/issues/3776

Obtained from: NetBSD
MFC after: 1 month

Manually merge part of vendor import r238583 from Illumos.

Illumos changeset: 13680:2bd022a765e2
Illumos ZFS issue:

2671 zpool import should not fail if vdev ashift has increased

MFC after: 3 days

Fix misleading or remove irrelevant illumos messages and manpage references
in the zfs command.

PR: bin/178996
Submitted by: Peter Schaefer <peter.schaefer@wilhelmheinrichs.de>
MFC after: 3 days

MFV r252215:

Restore a previous behavior before r251646, where when destructing
ZFS snapshot, the ioctl would return ENOENT when it hit any of
them in the errlist (the new behavior was only return ENOENT when
all returns error).

Illumos ZFS issues:
3829 fix for 3740 changed behavior of zfs destroy/hold/release ioctl

MFC after: 1 week

Diff reduction against Illumos, no real change to code itself.

This marks vendor branch revision 252213 as merged, the actual code was
committed in r245479.

MFC after: 1 week

Fixed ZFS zpool freeze (debug command) not processing due to invalid ioctl call syntax.

MFC after: 1 week

MFV r251644:

Poor ZFS send / receive performance due to snapshot
hold / release processing (by smh@)

Illumos ZFS issues:
3740 Poor ZFS send / receive performance due to snapshot
hold / release processing

MFC after: 2 weeks

MFV r251624:

txg commit callbacks don't work

Illumos ZFS issues:
3747 txg commit callbacks don't work

MFC after: 2 weeks

MFV r251623:

zpool create should treat -O mountpoint and -m the same

Illumos ZFS issues:
3745 zpool create should treat -O mountpoint and -m the same

MFC after: 2 weeks

MFV r251619:

ZFS needs better comments.

Illumos ZFS issues:
3741 zfs needs better comments

MFC after: 2 weeks

Port the SDT test now that it's possible to create SDT probes that take
seven arguments.

The original test uses Solaris' uadmin system call to trigger the test
probe; this change adds a sysctl to the dtrace_test module and gets the test
program to trigger the test probe via the sysctl handler.

The test is currently failing on amd64 because of some bugs in the way that
probe arguments beyond the first five are obtained - these bugs will be
fixed in a separate change.

Remove a block of code that was not intended to be part of the previous
revision.

X-MFC with: r250812

Re-introduce another part of r249367. This commit fixes a register leak in
dt_cg_ptrsize() and generally cleans up some of the error handling around
register allocation.

This change corresponds to part of illumos-gate commit e5803b76927480:
3025 register leak in D code generation

Reviewed by: pfg
Obtained from: illumos
MFC after: 1 month

Convert a couple of helper scripts used to test the ip provider to work on
FreeBSD. In the IPv6 case, try each interface before returning an error;
each IPv6-enabled interface will have a link-local address even if the link
isn't up.

MFC after: 1 week

head -n doesn't take negative arguments on FreeBSD, so instead use sed '$d'
to remove the last line of a file.

Bring back part of r249367 by adding DTrace's temporal option, which allows
users to guarantee that the output of DTrace scripts will be time-ordered.
This option is enabled by adding the line

#pragma D option temporal

to the beginning of a script, or by adding '-x temporal' to the arguments of
dtrace(1).

This change fixes a bug in the original port of the temporal option. This
bug was causing some assertions to fail, so they had been disabled; in this
revision the assertions are working properly and are enabled.

The DTrace version number has been bumped from 1.9.0 to 1.9.1 to reflect
the language change that's being introduced.

This change corresponds to part of illumos-gate commit e5803b76927480:
3021 option for time-ordered output from dtrace(1M)

Reviewed by: pfg
Obtained from: illumos
MFC after: 1 month

Work around the implementation of LIBRARIES_ONLY. It causes drti.o to
not be installed in /usr/lib32 on systems with compat-32 support.

This fix has two parts. First, the build is forced by linking drti.o
into a dummy internal library. Second, the object file is installed
manually in the LIBRARIES_ONLY case.

MFC after: 3 days

revert r248644 because of the regression for usdt probes

USDT probes are advertised to kernel by initialization code with
atrribute((constructor))). It seems that on Solaris the .init-ish code
of the main object is executed before RD_PREINIT point is hit. On
FreeBSD that is not the case. And because on FreeBSD there is no other
well-defined point between RD_PREINIT and main() we have to parse a
DTrace script when main is hit, for time being.

A footnote: currently we actually post RD_POSTINIT event, but that's a
bug because the event is triggered by hitting r_debug_state which
happens before any init code is executed.

Reported by: markj

Respect the enoent_ok flag if reporting error for holding an non-existing
snapshot.

Related illumos ZFS issue:
3699 zfs hold or release of a non-existent snapshot does not output error

Reported by: Steven Hartland <smh@FreeBSD.org>
MFC after: 3 days

Fix -Wmissing-variable-declarations compiler warnings.

References:
https://www.illumos.org/issues/3700

DTrace: NFS translators should be split into client/server pieces

Belatedly update the provider with changes from OpenSolaris:

PSARC 2008/050 DTrace NFS v3 Provider
6696397 NFS v3 provider reports all UDP clients as 0.0.0.0
6711844 assert: vp->v_shrlocks

Obtained from: Illumos

DTrace: Revert r249426

This change actually depends on r249367 which had to be reverted

Pointy Hat: pfg

DTrace: Revert r249367

The following change from illumos brought caused DTrace to
pause in an interactive environment:

3026 libdtrace should set LD_NOLAZYLOAD=1 to help the pid provider

This was not detected during testing because it doesn't
affect scripts.

We shouldn't be changing the environment, especially since the
LD_NOLAZYLOAD option doesn't apply to our (GNU) ld.
Unfortunately the change from upstream was made in such a way
that it is very difficult to separate this change from the
others so, at least for now, it's better to just revert
everything.

Reference:
https://www.illumos.org/issues/3026

Reported by: Navdeep Parhar and Mark Johnston

DTrace: print() should try to resolve function pointers

Merge changes from illumos:

3675 DTrace print() should try to resolve function pointers
3676 dt_print_enum hardcodes a value of zero

Illumos Revision: b1fa6326238973aeaf12c34fcda75985b6c06be1

Reference:
https://www.illumos.org/issues/3675
https://www.illumos.org/issues/3676

Obtained from: Illumos
MFC after: 1 month

Correct error message.

Reported by: Dirk Engling <erdgeist@erdgeist.org>

DTrace: NFS translators should be split into client/server pieces

Merge change from illumos:

1731 DTrace NFS translators should be split into client/server pieces

Illumos Revision: 13523:6763769941d2

This code seems to be currently unused on FreeBSD.

Reference:
https://www.illumos.org/issues/1731

Obtained from: Illumos
MFC after: 1 week

Dtrace: resolve const types from fbt and other fixes.

Merge change from illumos:

3519 DTrace fails to resolve const types from fbt
3520 dtrace internal error -- token type 316 is not a valid D
compilation token
3521 clean up dtrace unit tests

Illumos Revision: e98f46c

Reference:
https://www.illumos.org/issues/3519
https://www.illumos.org/issues/3520
https://www.illumos.org/issues/3521

Tested by: Fabian Keil
Obtained from: Illumos
MFC after: 1 month

DTrace: option for time-ordered output

Merge changes from illumos:

3021 option for time-ordered output from dtrace(1M)
3022 DTrace: keys should not affect the sort order when sorting by value
3023 it should be possible to dereference dynamic variables
3024 D integer narrowing needs some work
3025 register leak in D code generation
3026 libdtrace should set LD_NOLAZYLOAD=1 to help the pid provider

This brings yet another feature implemented in upstream DTrace.
A complete description is available here:
http://dtrace.org/blogs/ahl/2012/07/28/my-new-dtrace-favorite/

This change bumps the DT_VERS_* number to 1.9.1 in
accordance to what is done in illumos.

This change was somewhat complicated because upstream is mixed many
changes in an individual commit and some of the tests don't really
apply to us.

There are also appear to be differences in timestamping with Solaris
so we had to workaround some assertions making sure no regression
happened.

Special thanks to Fabian Keil for changes and testing.

Illumos Revisions: 13758:23432da34147

Reference:
https://www.illumos.org/issues/3021
https://www.illumos.org/issues/3022
https://www.illumos.org/issues/3023
https://www.illumos.org/issues/3024
https://www.illumos.org/issues/3025
https://www.illumos.org/issues/1694

Tested by: Fabian Keil
Obtained from: Illumos
MFC after: 1 months

Fix libzfs to report error instead of returning zero if trying to hold or
release a non-existing snapshot of a existing dataset. In recursive case
error is reported if no snapshots with the requested name have been found.

Problem and proposed solution reported to illumos:
3699 zfs hold or release of a non-existent snapshot does not output error

MFC after: 8 days

ZFS expects a copyout of zfs_cmd_t on an ioctl error. Our sys_ioctl()
doesn't copyout in this case.

To solve this issue a new struct zfs_iocparm_t is introduced consisting of:
- zfs_ioctl_version (future backwards compatibility purposes)
- user space pointer to zfs_cmd_t (copyin and copyout)
- size of zfs_cmd_t (verification purposes)

The copyin and copyout of zfs_cmd_t is now done the illumos (vendor) way
what makes porting of new changes easier and ensures correct behavior if
returning an error.

MFC after: 10 days

MFV r249185:
Allow zdb to output a histogram of compressed block sizes.

Illumos ZFS issues:
3641 want a histogram of compressed block sizes

MFC after: 3 weeks

MFV r248217:
Merge change from vendor to reduce diff only.
ZFS dtrace probes are not supported on FreeBSD yet.

Illumos ZFS issues:
3598 want to dtrace when errors are generated in zfs

MFC after: 3 weeks

Follow up to r247960 and rr247960 by also amending ctfmerge. For the
only other case where STT_FILE symbols are used, in symit_next() in
cddl/contrib/opensolaris/tools/ctf/cvt/input.c, save the basename of the
symbol, instead of the full pathname.

Reported by: avg
Tested by: avg, jimharris
MFC after: 1 week

Dtrace: enablings on defunct providers prevent providers from unregistering

Merge change from illumos:

1368 enablings on defunct providers prevent providers from unregistering

We try to address some underlying differences between the Solaris
and FreeBSD implementations: dtrace_attach() / dtrace_detach() are
currently unimplemented in FreeBSD but the new code from illumos
makes use of taskq so some adaptations were made to dtrace_open()
and dtrace_close() to handle them appropriately.

Illumos Revision: r13430:8e6add739e38

Reference:
https://www.illumos.org/issues/1368

Reviewed by: gnn
Tested by: Fabian Keil
Obtained from: Illumos
MFC after: 3 weeks

Commit a patch that fixes a problem in the #pragma statement when searching
for and loading dependent modules. This addresses a bug seen with
io.d where it was being doubly included.

PR: 171678
Submitted by: Mark Johnston
MFC after: 2 weeks

Revert previous change in favor of a fix to the actual dtrace
libraries that addresses a #pragma issue.

Remove dependency code that caused a double inclusion.

Pointed out by: rpaulo and others

Dtrace: dtrace.c erroneously checks for memory alignment on amd64.

Merge change from illumos:

3511 dtrace.c erroneously checks for memory alignment on amd64

Illumos Revision: c93cc65

Reference:
https://www.illumos.org/issues/3511

Obtained from: Illumos
MFC after: 3 weeks

Dtrace: Add SUN MDB-like type-aware print() action.

Merge change from illumos:

1694 Add type-aware print() action

This is a very nice feature implemented in upstream Dtrace.
A complete description is available here:
http://dtrace.org/blogs/eschrock/2011/10/26/your-mdb-fell-into-my-dtrace/

This change bumps the DT_VERS_* number to 1.9.0 in
accordance to what is done in illumos.

While here also include some minor cleanups to ease further merging
and appease clang with a fix by Fabian Keil.

Illumos Revisions: 13501:c3a7090dbc16
13483:f413e6c5d297

Reference:
https://www.illumos.org/issues/1560
https://www.illumos.org/issues/1694

Tested by: Fabian Keil
Obtained from: Illumos
MFC after: 1 month

Dtrace: add toupper()/tolower() and enhancements to lltostr().

Merge changes from illumos:

1451 DTrace needs toupper()/tolower() subroutines
1457 lltostr() D subroutine should take an optional base

This change bumps the DT_VERS_* number to 1.8.1 in
accordance to what is done in illumos.

The test suite we currently include is outdated and
doesnt support some updates in tst.subr.d which had to
be left out for now.

Illumos Revisions: r13458 5e394d8db762
r13459 c3454574dd1a

Reference:
https://www.illumos.org/issues/1451
https://www.illumos.org/issues/1457

Tested by: Fabian Keil
Obtained from: Illumos
MFC after: 1 month

Dtrace: add optional size argument to tracemem().

Merge change from illumos:

1455 DTrace tracemem() should take an optional size argument

Our local enhancements to dt_print_bytes were equivalent to
those in illumos but we made it match the illumos version
to ease further code merges.

For now leave out tst.smallsize.d and tst.smallsize.d.out
since those don't seem to work cleanly on FreeBSD.

This change bumps the DT_VERS_* number to 1.7.1 in accordance
to what is done in illumos.

Illumos Revision: 13457:571b0355c2e3

Reference:
https://www.illumos.org/issues/1455

Tested by: Fabian Keil
Obtained from: Illumos
MFC after: 1 month

dtrace: ensure that we can always catch a process (e.g. when -c is used)

It is not guaranteed that a program has a symbol table entry for main
and thus that it would be possible to set a breakpoint on it.

Reviewed by: rpaulo
Discussed with: rpaulo
MFC after: 13 days

Merge libzfs_core branch:
includes MFV 238590, 238592, 247580

MFV 238590, 238592:
In the first zfs ioctl restructuring phase, the libzfs_core library was
introduced. It is a new thin library that wraps around kernel ioctl's.
The idea is to provide a forward-compatible way of dealing with new
features. Arguments are passed in nvlists and not random zfs_cmd fields,
new-style ioctls are logged to pool history using a new method of
history logging.

http://blog.delphix.com/matt/2012/01/17/the-future-of-libzfs/

MFV 247580 [1]:
To address issues of several deadlocks and race conditions the locking
code around dsl_dataset was rewritten and the interface to synctasks
was changed.

User-Visible Changes:
"zfs snapshot" can create more arbitrary snapshots at once (atomically)
"zfs destroy" destroys multiple snapshots at once
"zfs recv" has improved performance

Backward Compatibility:
I have extended the compatibility layer to support full backward
compatibility by remapping or rewriting the responsible ioctl arguments.
Old utilities are fully supported by the new kernel module.

Forward Compatibility:
New utilities work with old kernels with the following restrictions:
- creating, destroying, holding and releasing of multiple snapshots
at once is not supported, this includes recursive (-r) commands

Illumos ZFS issues:
2882 implement libzfs_core
2900 "zfs snapshot" should be able to create multiple,
arbitrary snapshots at once
3464 zfs synctask code needs restructuring

References:
https://www.illumos.org/issues/2882
https://www.illumos.org/issues/2900
https://www.illumos.org/issues/3464 [1]

MFC after: 1 month
Sponsored by: Hybrid Logic Inc. [1]

MFV r248266:
Import minor ZFS changes from vendor

Illumos ZFS issues:
3604 zdb should print bpobjs more verbosely (fix zdb hang)
3606 zpool status -x shouldn't warn about old on-disk format

MFC after: 3 days

Update zfs.8 manpage date (missing in r247585)

MFC: together with r247585

Fix error in r247960: actually assign the basename to match.iim_file.

Pointed out by: avg
Pointy hat to: dim
MFC after: 1 week
X-MFC-With: r247960

Make ctfconvert work correctly on clang-compiled object files. Clang
puts the full original source filename in the STT_FILE entry of the ELF
symbol table, while gcc saves only the basename.

Since the DWARF DW_AT_name attribute contains the full source filename,
both for clang and gcc, ctfconvert takes just the basename of it, for
matching with the STT_FILE entry. So when attempting to match with such
an entry, use its basename, if necessary.

Reported by: avg
MFC after: 1 week

MFV r247845:
Import ZFS bpobj bugfix from vendor.

Illumos ZFS issues:
3603 panic from bpobj_enqueue_subobj()
3604 zdb should print bpobjs more verbosely

References:
https://www.illumos.org/issues/3603
https://www.illumos.org/issues/3604

MFC after: 1 week

MFV r247316:
Merge new read-only zfs properties from vendor (illumos)

Illumos ZFS issues:
3588 provide zfs properties for logical (uncompressed) space used and
referenced

References:
https://www.illumos.org/issues/3588

MFC after: 2 weeks

Fix the zfs_ioctl compat layer to support zfs_cmd size change introduced
in r247265 (ZFS deadman thread). Both new utilities now support the old
kernel and new kernel properly detects old utilities.

For future backwards compatibility, the vfs.zfs.version.ioctl read-only
sysctl has been introduced. With this sysctl zfs utilities will be able
to detect the ioctl interface version of the currently loaded zfs module.

As a side effect, the zfs utilities between r247265 and this revision don't
support the old kernel module. If you are using HEAD newer or equal than
r247265, install the new kernel module (or whole kernel) first.

MFC after: 10 days

MFV 247176, 247178, 247315:
Import metaslab_sync() speedup from vendor (illumos).

Illumos ZFS issues:
3552 condensing one space map burns 3 seconds of CPU in spa_sync() thread
3564 spa_sync() spends 5-10% of its time in metaslab_sync() (when not
condensing)
3578 transferring the freed map to the defer map should be constant time
3579 ztest trips assertion in metaslab_weight()

References:
https://www.illumos.org/issues/3552
https://www.illumos.org/issues/3564
https://www.illumos.org/issues/3578
https://www.illumos.org/issues/3579

MFC after: 2 weeks

MFV v242732:

Merge the ZFS I/O deadman thread from vendor (illumos).
This feature panics the system on hanging ZFS I/O, helps debugging
and resumes failed service.

The panic behavior can be controlled with the loader-only tunables:
vfs.zfs.deadman_enabled (enable or disable panic on stalled ZFS I/O)
vfs.zfs.deadman_synctime (expiration time for stalled ZFS I/O)

By default, ZFS I/O deadman is enabled by default on amd64 and i386
excluding virtual guest machines.

Illumos ZFS issues:
3246 ZFS I/O deadman thread

References:
https://www.illumos.org/issues/3246

MFC after: 2 weeks

Orphaned processes that are being traced are killed by the
kernel. Properly restore, continue, and detach from processes
being DTraced when DTrace exits with an error so the program
being inspected is not terminated.

cddl/contrib/opensolaris/cmd/dtrace/dtrace.c:
In fatal(), the generic error handler, close the DTrace
handle as is done in the "probe/script" error handler
dfatal(). fatal() can be invoked after DTrace attaches
to processes (e.g. a script specified by command line
argument can't be found) and closing the handle will
release them.

Submitted by: Spectra Logic Corporation
Reviewed by: rpaulo, gnn

Use the right year.

Noticed by: Denis Ahrens <denis h3q com>

MFV r246392:
Import vendor ZFS bugfix fixing a possible deadlock in arc_read().

Illumos ZFS issues:
3498 panic in arc_read(): !refcount_is_zero(&pbuf->b_hdr->b_refcnt)

References:
https://www.illumos.org/issues/3498

MFC after: 2 weeks

MFV r246388:

Import vendor bugfixes

Illumos ZFS issues:
3422 zpool create/syseventd race yield non-importable pool
3425 first write to a new zvol can fail with EFBIG

References:
https://www.illumos.org/issues/3422
https://www.illumos.org/issues/3425

MFC after: 2 weeks

Backport vendor changes in zfs(8) manual page (MFV r246389)

Illumos ZFS issues:
3380 zfs man page: documentation for zfs allow is confusing

References:
https://www.illumos.org/issues/3380

MFC after: 2 weeks

Correct spelling of "daemon". No .Dd bump.

Noticed by: Nathan Rich <Nathan.Rich dynastysystems com>
MFC after: 3 days

mdoc: Remove EOL whitespace.

MFV r245512:

* Illumos zfs issue #3035 [1] LZ4 compression support in ZFS.

LZ4 is a new high-speed BSD-licensed compression algorithm created
by Yann Collet that delivers very high compression and decompression
performance compared to lzjb (>50% faster on compression, >80% faster
on decompression and around 3x faster on compression of incompressible
data), while giving better compression ratio [1].

This version of LZ4 corresponds to upstream's [2] revision 85.

Please note that for obvious reasons this is not backward read
compatible. This means once a pool have LZ4 compressed data, these
data can no longer be read by older ZFS implementations.

Local changes:

- On-stack hash table disabled and using kernel slab allocator
instead, at this time. This requires larger kernel thread stack
for zio workers. This may change in the future should we adjusted
the zio workers' thread stack size.
- likely and unlikely will be undefined if they are already defined,
this is required for i386 XEN build.
- Removed De Bruijn sequence based __builtin_ctz family of builtins
in favor of the latter. Both GCC and clang supports these builtins.
- Changed the way the LZ4 code detects endianness.
- Manual pages modifications to mention the feature based on Illumos
counterpart.
- Boot loader changes to make it support LZ4 decompression.

[1] https://www.illumos.org/issues/3035
[2] http://code.google.com/p/lz4/source/list

Obtained from: Illumos (13921:9d721847e469)
Tested on: FreeBSD/amd64
MFC after: 1 month

Add a new LIBRARIES_ONLY make variable to disable the build and install
of files other than the actual libraries.

Use LIBRARIES_ONLY to supress the inclusion of files in the lib32
distribution that are duplicates of files in base.

Sponsored by: DARPA, AFRL
Reviewed by: emaste

Reports pools which have a removed l2cache disk under -x as this is what
happens when a cache device is dropped for any reason.

Reviewed by: pjd
Approved by: pjd (mentor)
MFC after: 2 weeks

The current ZFS code expects ddt_zap_count to always succeed by asserting
the underlying zap_count() to return no errors. However, it is possible
that the pool reaches to such a state where zap_count would return error,
leading to panics when a pool is imported.

This commit changes the ddt_zap_count to return error returned from
zap_count and handle the error appropriately. With this change, it's now
possible to let zpool rollback damaged transaction groups and import the
pool.

Obtained from: ZFS on Linux github (e8fd45a0f975c6b8ae8cd644714fc21f14fac2bf)
MFC after: 1 month

Allow to create pool even if mount point directory is not empty if -f is given.

Obtained from: WHEEL Systems

MFV r244245:

Merge two zdb bugfixes from vendor (illumos)

illumos-gate 13894:f4af77f6bbd2
3397 zdb <pool> <objnum> output is too verbose
3398 zdb can't dump feature flags zap objects

References:
https://www.illumos.org/issues/3397
https://www.illumos.org/issues/3398

MFC after: 1 week

Fixes zfs receive errors caused by snapshot replication being processed in a
random order instead of creation order.

Eliminates needless filesystem renames caused by removed parent snapshots
which subsequently causes many more errors.

PR: kern/172259
Submitted by: Steven Hartland
Reviewed by: pjd (mentor)
Approved by: pjd (mentor)
MFC after: 2 weeks

Update manpage dates in zfs.8 and zpool.8

MFC after: 2 weeks

MFV r243395:

Introduce a new dataset aclmode setting "restricted" to protect ACL's
being destroyed or corrupted by a drive-by chmod.

illumos-gate 13889:a67716f16746
3254 add support in zfs for aclmode=restricted

References:
https://www.illumos.org/issues/3254

MFC after: 2 weeks

MFV r243013 and r243267:

Import the zio nop-write improvement from Illumos. To reduce I/O,
nop-write omits overwriting data if the checksum (cryptographically
secure) of new data matches the checksum of existing data.
It also saves space if snapshots are in use.

It currently works only on datasets with enabled compression, disabled
deduplication and sha256 checksums.

IllumOS 13887:196932ec9e6a and 13888:7204b3392a58
3236 zio nop-write

References:
https://www.illumos.org/issues/3236

MFC after: 2 weeks

zfs sha256 checksum is missing in zfs.8 manpage

MFC after: 3 days

MFV r243012:

Illumos 13886:e3261d03efbf

3349 zpool upgrade -V bumps the on disk version number, but leaves
the in core version

References:
https://www.illumos.org/issues/3349

MFC after: 1 week

Sort SEE ALSO xrefs by the manual section number and add a missing comma.

Move zpool-features manual page from section 5 to section 7
and fix references

Reported by: pluknet
MFC after: 1 week

Fix an apparent typo in the manual section number in .Dt: it should be 5.

MFV r242729 (mm):

Illumos r13840:97fd5cdf328a:

3145 single-copy arc
3212 ztest: race condition between vdev_online() and spa_vdev_remove()

Illumos r13849:3468a95b27cd:

3258 ztest's use of file descriptors is unstable

Implement DTrace for PowerPC. This includes both 32-bit and 64-bit.

There is one known issue: Some probes will display an error message along the
lines of: "Invalid address (0)"

I tested this with both a simple dtrace probe and dtruss on a few different
binaries on 32-bit. I only compiled 64-bit, did not run it, but I don't expect
problems without the modules loaded. Volunteers are welcome.

MFC after: 1 month

Remove the support for using non-mpsafe filesystem modules.

In particular, do not lock Giant conditionally when calling into the
filesystem module, remove the VFS_LOCK_GIANT() and related
macros. Stop handling buffers belonging to non-mpsafe filesystems.

The VFS_VERSION is bumped to indicate the interface change which does
not result in the interface signatures changes.

Conducted and reviewed by: attilio
Tested by: pho

Add missing initialization for do_prefix.
Corrects porting error in r238391

Vendor issue and changeset reference:
2883 changing "canmount" property to "on" should not always remount dataset
https://www.illumos.org/issues/2883
Changeset 13743:95aba6e49b9f

Reported by: Guido Falsi <mad@madpilot.net>, avg
Obtained from: illumos (issue #2883)
MFC after: 1 week

Make sure that each va_start has one and only one matching va_end,
especially in error cases.

Merge recent vendor changes in ZFS.

Illumos issued covered:
2811 missing implementation: zfs send -r
3139 zdb dies when it tries to determine path of unlinked file
3189 kernel panic in ZFS test suite during hotspare_onoffline_004_neg
3208 moving zpool cross-endian results in incorrect user/group accounting

References:
https://www.illumos.org/issues/ + [issue_id]

Obtained from: illumos (vendor/illumos, vendor/illumos-sys)
MFC after: 2 weeks

It is possible to recursively destroy snapshots even if the snapshot
doesn't exist on a dataset we are starting from. For example if we
have the following configuration:

tank
tank/foo
tank/foo@snap
tank/bar
tank/bar@snap

We can execute:

# zfs destroy -t tank@snap

eventhough tank@snap doesn't exit.

Unfortunately it is not possible to do the same with recursive rename:

# zfs rename -r tank@snap tank@pans
cannot open 'tank@snap': dataset does not exist

...until now. This change allows to recursively rename snapshots even if
snapshot doesn't exist on the starting dataset.

Sponsored by: rsync.net
MFC after: 2 weeks

Add TRIM support.

The code builds a map of regions that were freed. On every write the
code consults the map and eventually removes ranges that were freed
before, but are now overwritten.

Freed blocks are not TRIMed immediately. There is a tunable that defines
how many txg we should wait with TRIMming freed blocks (64 by default).

There is a low priority thread that TRIMs ranges when the time comes.
During TRIM we keep in-flight ranges on a list to detect colliding
writes - we have to delay writes that collide with in-flight TRIMs in
case something will be reordered and write will reached the disk before
the TRIM. We don't have to do the same for in-flight writes, as
colliding writes just remove ranges to TRIM.

Sponsored by: multiplay.co.uk

This work includes some important fixes and some improvements obtained
from the zfsonlinux project, including TRIMming entire vdevs on pool
create/add/attach and on pool import for spare and cache vdevs.

Obtained from: zfsonlinux
Submitted by: Etienne Dechamps <etienne.dechamps@ovh.net>

Update usage to show the possiblity to use jail name

Reported by: bdrewery
MFC after: 1 month

Allow zfs jail and zfs unjail to use both jailnames and jailid

Reviewed by: pjd
MFC after: 1 month

Merge recent zfs vendor changes, sync code and adjust userland DEBUG.

Illumos issued covered:
1884 Empty "used" field for zfs *space commands
3006 VERIFY[S,U,P] and ASSERT[S,U,P] frequently check if first argument
is zero
3028 zfs {group,user}space -n prints (null) instead of numeric GID/UID
3048 zfs {user,group}space [-s|-S] is broken
3049 zfs {user,group}space -t doesn't really filter the results
3060 zfs {user,group}space -H output isn't tab-delimited
3061 zfs {user,group}space -o doesn't use specified fields order
3064 usr/src/cmd/zpool/zpool_main.c misspells "successful"
3093 zfs {user,group}space's -i is noop
3098 zfs userspace/groupspace fail without saying why when run as non-root

References:
https://www.illumos.org/issues/ + [issue_id]

Obtained from: illumos (vendor/illumos, vendor/illumos-sys)
MFC after: 2 weeks

Remove trailing whitespace.

Typo fix and minor word swap.

PR: 171356
Submitted by: bdrewery
MFC After: 3 days

Merge recent vendor changes and sync code:
1862 incremental zfs receive fails for sparse file > 8PB
3112 ztest does not honor ZFS_DEBUG
3122 zfs destroy filesystem should prefetch blocks
3129 'zpool reopen' restarts resilvers
3130 ztest failure: Assertion failed:
0 == dmu_objset_destroy(name, B_FALSE) (0x0 == 0x10)

References:
https://www.illumos.org/issues/1862
https://www.illumos.org/issues/3112
https://www.illumos.org/issues/3122
https://www.illumos.org/issues/3129
https://www.illumos.org/issues/3130

Obtained from: illumos (vendor/illumos, vendor/illumos-sys)
MFC after: 2 weeks

Add myself to copyright sections, per CDDL license.

Requested by: mm

Finish porting execsnoop to FreeBSD. This includes replacing the zonename
with a jail ID and removing the project ID from the list of options.

Remove trailing whitespace.

Remove trailing whitespace.

Mdoc fixes.

Minor mdoc fixes.

Mdoc fixes.

Merge recent vendor changes:
3100 zvol rename fails with EBUSY when dirty
3104 eliminate empty bpobjs
3120 zinject hangs in zfsdev_ioctl() due to uninitialized zc

References:
https://www.illumos.org/issues/3100
https://www.illumos.org/issues/3104
https://www.illumos.org/issues/3120

Obtained from: illumos (vendor/illumos, vendor/illumos-sys)
MFC after: 2 weeks

Add missing parts to zpool-features.5 manual page

Obtained from: vendor/illumos/dist
MFC after: 2 weeks

Merge recent vendor changes:
3086 unnecessarily setting DS_FLAG_INCONSISTENT on async destroyed datasets
3090 vdev_reopen() during reguid causes vdev to be treated as corrupt
3102 vdev_uberblock_load() and vdev_validate() may read the wrong label

Referenes:
https://www.illumos.org/issues/3086
https://www.illumos.org/issues/3090
https://www.illumos.org/issues/3102

PR: kern/170912, kern/170914
Obtained from: illumos (changeset #13776, #13777)
MFC after: 2 weeks

Update zfs(8) manpage with illumos version of "zfs diff"

Illumos issue:
2399 zfs manual page does not document use of "zfs diff"

References:
https://www.illumos.org/issues/2399

PR: docs/170764
Obtained from: ssh://anonhg@hg.illumos.org/illumos-gate
MFC after: 1 week

- Fix source dataset snapshot name in Example 15.
- Bump date.

MFC after: 3 days
X-MFC-With: r239216

Remove a leading space that breaks rendering.

MFC after: 3 days

When we return with an error we cannot unlock the mutex, because
it's been freed. Protect against that, hopefully unlikely, case.

Reviewed by: rpaulo
MFC after: 2 weeks

Fix wrong indent according to style(9)

MFC after: 2 weeks
> Description of fields to fill in above: 76 columns --|
> PR: If a GNATS PR is affected by the change.
> Submitted by: If someone else sent in the change.
> Reviewed by: If someone else reviewed your modification.
> Approved by: If you needed approval for this commit.
> Obtained from: If the change is from a third party.
> MFC after: N [day[s]|week[s]|month[s]]. Request a reminder email.
> Security: Vulnerability reference (one per line) or description.
> Empty fields above will be automatically removed.

M zpool_main.c

Fix reporting of root pool upgrade notice.

MFC after: 2 weeks

Partial MFV (illumos-gate 13753:2aba784c276b)
2762 zpool command should have better support for feature flags

References:
https://www.illumos.org/issues/2762

MFC after: 2 weeks

Revert previous commit. The bug was actually caused by an issue
in pre 1.8.5 versions of sudo which were sending too many
SIGINTs to processes when the user hit Ctrl-C.

Pointed out by: avg@, rpaulo@, sbruno@

Fix a bug in interrupt handling so that we're only considered
impatient if we sent more than 2 INT signals. This fixes a bug where
we wouldn't see aggregations print on the command line if we Ctrl-C'd
a dtrace script or command line invocation.

MFC after: 2 weeks

Dtrace: improve handling of library paths.

Merge changes from illumos

906 dtrace depends_on pragma should search all library paths, not just the
current one

949 dtrace should only include the first instance of a library found on
its library path

Illumos Revisions: 13353:936a1e45726c
13354:2b2c36a81512

Reference:
https://www.illumos.org/issues/906
https://www.illumos.org/issues/949

Tested by: Fabian Keil
Obtained from: Illumos
MFC after: 3 weeks

Use libc's strndup() instead of Dtrace's reimplementation.

Corresponds partially to OpenSolaris change:
PSARC 2010/299 GNU/Linux/BSD compatibility functions
6901783 strndup would be nice

MFC after: 2 weeks

Merge illumos commit 13455:7205f7794835

1458 D compiler fails to generate error on sizeof() an undefined struct

Reference:
https://www.illumos.org/issues/1458

Tested by: Fabian Keil
MFC after: 3 weeks

Merge illumos commit 13749:df4cd82e2b60

1796 "ZFS HOLD" should not be used when doing "ZFS SEND" froma read-only pool
2871 support for __ZFS_POOL_RESTRICT used by ZFS test suite
2903 zfs destroy -d does not work
2957 zfs destroy -R/r sometimes fails when removing defer-destroyed snapshot

References:
https://www.illumos.org/issues/1796
https://www.illumos.org/issues/2871
https://www.illumos.org/issues/2903
https://www.illumos.org/issues/2957

MFC after: 1 week

Change behavior introduced in r237119 to vendor solution

References:
https://www.illumos.org/issues/2883

PR: 167905
Obtained from: illumos (issue #2883)
MFC after: 2 weeks

Initial commit of an I/O provider for DTrace on FreeBSD.

These probes are most useful when looking into the structures
they provide, which are listed in io.d. For example:

dtrace -n 'io:genunix::start { printf("%d\n", args[0]->bio_bcount); }'

Note that the I/O systems in FreeBSD and Solaris/Illumos are sufficiently
different that there is not a 1:1 mapping from scripts that work
with one to the other.
MFC after: 1 month

Restore r211786 by rpaulo:

Port dtruss to FreeBSD.

Sponsored by: The FreeBSD Foundation

It appears the change was reverted by r235380.

Fix an obvious typo.

MFC after: 3 days

Fix clang warning, introduced in the recent dtrace import.

MFC after: 3 days

Import the dtrace llquantize test files from the vendor area.

Now that we have an illumos vendor area, import the llquantize
test files directly from there to make it easier to track the
code provenance.

Special thanks to Martin Matuska for populating the illumos
vendor area.

Requested by: obrien

Safer fix for building with Clang.

error: format specifies type 'long long' but the argument
has type 'int64_t' (aka 'long')

Reported by: Ed Maste

Fix build with Clang.

error: format specifies type 'long long' but the argument
has type 'int64_t' (aka 'long')

Reported by: Ed Maste

Bring llquantize support into Dtrace.

Bryan Cantrill implemented the equivalent of semi-log graph
paper for Dtrace so llquantize will use one logarithmic and
one linear scale.

Special thanks to Mark Peek for providing fix to an
assertion and to Fabian Keill for testing the port.

Illumos Revision: 13355:15b74a2a9a9d

Reference:
https://www.illumos/issues/905

Obtained from: Illumos
Tested by: Fabian Keill, mp
MFC after: 4 days

Do not remount ZFS dataset if changing canmount property to "on" and
dataset is already mounted.

PR: 167905
Submitted by: Bryan Drewery <bryan@shatow.net>
MFC after: 1 week

Document the -v flag for zpool list.

PR: 168970
Suggested by: Marcelo Araujo <araujo@FreeBSD.org>
MFC after: 3 days

Introduce "feature flags" for ZFS pools (bump SPA version to 5000).
Add first feature "com.delphix:async_destroy" (asynchronous destroy
of ZFS datasets).
Implement features support in ZFS boot code.

Illumos revisions merged:
13700:2889e2596bd6
13701:1949b688d5fb
2619 asynchronous destruction of ZFS file systems
2747 SPA versioning with zfs feature flags

References:
https://www.illumos.org/issues/2619
https://www.illumos.org/issues/2747

Obtained from: illumos (issue #2619, #2747)
MFC after: 1 month

Clean up trailing whitespace.

MFC after: 3 days
X-MFC-With: r236776

Fix a typo: s/deafult/default

MFC after: 3 days

Import Illumos revision 13715:351036203e4b
2803 zfs get guid pretty-prints the output

References:
https://www.illumos.org/issues/2803

Obtained from: illumos (issue #2803)
MFC after: 3 days

Add DTrace's io.d, which handles tranlsations for file, buffer and
device info structures as well as the fds[] array. This is a raw
version of the file, unmodified, to be used as a baseline.

Import illumos changeset 13570:3411fd5f1589
1948 zpool list should show more detailed pool information

Display per-vdev information with "zpool list -v".
The added expandsize property has currently no value on FreeBSD.
This changeset allows adding expansion support to individual vdevs
in the future.

References:
https://www.illumos.org/issues/1948

Obtained from: illumos (issue #1948)
MFC after: 2 weeks

Import illumos changeset 13605:b5c2b5db80d6 (partial)
763 FMD msg URLs should refer to something visible

Replace sun.com URL's with illumos.org

References:
https://www.illumos.org/issues/763

Obtained from: illumos (issue #763)
MFC after: 1 week

Import illumos changeset 13564:cf89c0c60496
1946 incorrect formatting when listing output of multiple pools with
zpool iostat -v

References:
https://www.illumos.org/issues/1946

Obtained from: illumos (issue #1946)
MFC after: 1 week

Import illumos changeset 13571:a5771a96228c
1950 ztest backwards compatibility testing option

References:
https://www.illumos.org/issues/1950

Obtained from: illumos (issue #1950)
MFC after: 2 weeks

Correct error message.

MFC after: 3 days

zpool_find_import_impl: another /dev/dsk -> /dev fix

This seems to fix zdb -e behavior.

PR: bin/155104
Submitted by: swell.k@gmail.com
MFC after: 2 weeks

zpool_do_import: use /dev instead of /dev/dsk as a default

This affects behavior of zpool import without -d option.

Reported by: Bruce Cran <bruce@cran.org.uk>
Submitted by: Fabian Keil <freebsd-listen@fabiankeil.de>
MFC after: 2 weeks

Move sh to the correct path for FreeBSD.

Move sh to the correct location for FreeBS (/bin/sh)

Fix execsnoop by changing exece and exec to be FreeBSD's execve.

Reference sh in the correct location (/bin/sh)

Fix opensnoop for FreeBSD by removing probes with 64 at the end as
these are unnecessary.

Reference sh in the correct location (/bin/sh)

Import dtracetoolkit into cddl/contrib

Revert previous failed cp.

Import dtracetoolkit into cddl/contrib

Import illumos changeset 13686:4bc0783f6064
2703 add mechanism to report ZFS send progress

If the zfs send command is used with the -v flag, the amount of bytes
transmitted is reported in per second updates.

References:
https://www.illumos.org/issues/2703

Obtained from: illumos (issue #2703)
MFC after: 2 weeks

Remove unreachable break in zfs_main.c

Partial import of illumos changeset 13622:e5889df1eaac
2077 lots of unreachable breaks in illumos gate

References:
https://www.illumos.org/issues/2077

PR: kern/167370
Submitted by: Marcelo Araujo <araujo@FreeBSD.org>
Obtained from: illumos (issue #2077)
MFC after: 1 week

Add support for force unmounting ZFS filesystems during "zfs rename"
with the -f flag.

Reimplementation of the illumos changeset 13677:a0cbef703c12
2635 'zfs rename -f' to perform force unmount

References:
https://www.illumos.org/issues/2635

PR: kern/164447
Suggested by: Marcelo Araujo <araujo@FreeBSD.org>
Obtained from: illumos (issue #2635)
MFC after: 1 week

Import illumos changeset 13618:c6ae14a341e8 [1]:
2088 zdb could use a reasonable manual page

Rewrite new zdb manpage to mdoc(7)

References:
https://www.illumos.org/issues/2088

PR: kern/167467 [1]
Submitted by: Marcelo Araujo <araujo@FreeBSD.org> [1]
Reviewed by: mm
Obtained from: Illumos (issue #2088)
MFC after: 1 week

General mdoc(7) and typo fixes.

PR: 167696
Submitted by: Nobuyuki Koganemaru (kogane!jp.freebsd.org)
MFC after: 3 days

GNU/kFreeBSD portability fix. This glue <stdlib.h> overrides header protection
in the system-wide version of <stdlib.h> by wrapping the #include_next
<stdlib.h> within the scope of its own header protection. On FreeBSD this has
no effect, since both header protections are equivalent. However the GNU version
of <stdlib.h> implements a special header protection mechanism which allows it
to be included multiple times (in different modes).

Simply by moving the #include_next off the header protection, we allow
system-wide <stdlib.h> to implement its own protection policy, whichever that
may be.

Implement the D "cpu" variable, which returns curcpu. I have chosen not
to follow the example of OpenSolaris and its descendants, which implemented
cpu as an inline that took a value out of curthread. At certain points in
the FreeBSD scheduler curthread->td_oncpu will no longer be valid (in
particukar, just before the thread gets descheduled) so instead I have
implemented this as its own built-in variable.

Sponsored by: Sandvine Inc.
MFC after: 1 week

Add -u option to 'zfs create' that prevents file system from being
automatically mounted. This is similar to the 'zfs receive -u'.

MFC after: 1 week

- Correct a typo which prevents 'lzjb' be displayed;
- Use quotes when tab is used.

Fix typo miror -> mirror

Reported by: Glen Barber <gjb@FreeBSD.org>
MFC after: 3 days

Change SIGUSR1 to SIGTHR to properly wake up a process that is being
traced. The use of SIGUSR1 caused traced processes (those attached to
with dtrace -p) to exit when dtrace exited.

MFC in: 1 week

Enable build of DTrace-related userland parts for MIPS

Add stub file for pid probe. It's required although pid probe is not supported
on MIPS yet

Maintain target's byte order for multi-byte fields in CTF structures.

CTF format is not cross-platform by design, e.g. it is not guaranteed
that data generated by ctfconvert/ctfmerge on one architecture will
be successfuly read on another. CTF structures are saved/restored
using naive approach. Roughly it looks like:
write(fd, &ctf_struct, sizeof(ctf_struct))
read(fd, &ctf_struct, sizeof(ctf_struct))

By sheer luck memory layout of all type-related CTF structures is the same
on amd64/i386/mips32/mips64. It's different on ARM though. sparc, ia64,
powerpc, and powerpc64 were not tested. So in order to get file compatible
with dtrace on ARM it should be compiled on ARM. Alternative solution would
be to have "signatures" for every platform and ctfmerge should convert host's
reperesentation of CTF structure to target's one using "signature" as template.

This patch checks byte order of ELF files used for generating CTF record
and makes sure that byte order of data written to resulting files is the same
as target's byte order.

Analogous to r232059, add a parameter for the ZFS file system:

allow.mount.zfs:
allow mounting the zfs filesystem inside a jail

This way the permssions for mounting all current VFCF_JAIL filesystems
inside a jail are controlled wia allow.mount.* jail parameters.

Update sysctl descriptions.
Update jail(8) and zfs(8) manpages.

TODO: document the connection of allow.mount.* and VFCF_JAIL for kernel
developers

MFC after: 10 days

Import illumos changeset 13608 [1]:

add support for "-t <datatype>" argument to zfs get

References:
https://www.illumos.org/issues/1936

Update zfs(8) manpage in respect of [1].
Fix typo in zfs(8) manpage.

Obtained from: illumos (issue #1936)
MFC after: 1 week

Merge multi-FIB IPv6 support from projects/multi-fibv6/head/:

Extend the so far IPv4-only support for multiple routing tables (FIBs)
introduced in r178888 to IPv6 providing feature parity.

This includes an extended rtalloc(9) KPI for IPv6, the necessary
adjustments to the network stack, and user land support as in netstat.

Sponsored by: Cisco Systems, Inc.
Reviewed by: melifaro (basically)
MFC after: 10 days

Merge illumos revision 13594:

uninitialized variables in zfs(8) may make snapshots undestroyable [1]

References:
https://www.illumos.org/issues/2067 [1]

Obtained from: illumos (issue #2067)
MFC after: 3 days

Set SVN text/plain property for some shell scripts that
happen to have a .exe extension.

While here fix the shebang of a shell script that
was looking for /bin/bash.

Reviewed by: gnn
Approved by: jhb (mentor)
MFC after: 2 weeks

add KM_NODEBUG needed by ARC buffer core dump exclusion change

Merge illumos revisions 13572, 13573, 13574:

Rev. 13572:
disk sync write perf regression when slog is used post oi_148 [1]

Rev. 13573:
crash during reguid causes stale config [2]
allow and unallow missing from zpool history since removal of pyzfs [5]

Rev. 13574:
leaking a vdev when removing an l2cache device [3]
memory leak when adding a file-based l2arc device [4]
leak in ZFS from metaslab_group_create and zfs_ereport_checksum [6]

References:
https://www.illumos.org/issues/1909 [1]
https://www.illumos.org/issues/1949 [2]
https://www.illumos.org/issues/1951 [3]
https://www.illumos.org/issues/1952 [4]
https://www.illumos.org/issues/1953 [5]
https://www.illumos.org/issues/1954 [6]

Obtained from: illumos (issues #1909, #1949, #1951, #1952, #1953, #1954)
MFC after: 2 weeks

Merge illumos revisions 13540, 13562:

illumos rev 13540 [1]:
Removal of pyzfs broke delegation for volumes

illumos rev 13562 [2]:
zfs allow arguments not parsed correctly after pyzfs removal

References:
https://www.illumos.org/issues/1726 [1]
https://www.illumos.org/issues/1977 [2]

Obtained from: illumos (issues #1726, #1977)
MFC after: 1 week

Dramatically optimize listing snapshots when user requests only snapshot
names and wants to sort them by name, ie. when executes:

# zfs list -t snapshot -o name -s name

Because only name is needed we don't have to read all snapshot properties.

Below you can find how long does it take to list 34509 snapshots from a single
disk pool before and after this change with cold and warm cache:

before:

# time zfs list -t snapshot -o name -s name > /dev/null
cold cache: 525s
warm cache: 218s

after:

# time zfs list -t snapshot -o name -s name > /dev/null
cold cache: 1.7s
warm cache: 1.1s

MFC after: 1 week

Add one more copyright line accidentially removed in r228103

MFC after: 3 days

Add accidentially removed copyright lines in r228103

Reported by: pjd
MFC after: 3 days

Add missing file permission to open(O_CREAT) call. This doesn't really matter
here, as we open /dev/null for test purposes, but it is useful for consistency
and further grepping for such bugs.

In cddl/contrib/opensolaris/cmd/dtrace/dtrace.c, fix some obtuse
snprintf tricks.

MFC after: 1 week

In cddl/contrib/opensolaris/tools/ctf/cvt/ctfmerge.c, a size_t is passed
to fprintf as a field width. It should be an int instead, so cast it.

MFC after: 1 week

In cddl/contrib/opensolaris/lib/libdtrace/common/dt_handle.c, some
uint64_t values are snprintf'd using %llx. On amd64, uint64_t is
typedef'd as unsigned long, so cast the values to u_longlong_t, as is
done similarly in the rest of the file.

MFC after: 1 week

In cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c, some
uint64_t values are snprintf'd using %llx. On amd64, uint64_t is
typedef'd as unsigned long, so cast the values to u_longlong_t, as is
done similarly in the rest of the file.

MFC after: 1 week

In cddl/contrib/opensolaris/lib/libdtrace/common/dt_subr.c, the
dt_popc() function assumes that either _ILP32 or _LP64 is defined,
otherwise it has no suitable implementation.

However, the _ILP32 and _LP64 macros come from isa_defs.h, which is not
included in this file. Add the include now, to get the macros defined.

MFC after: 1 week

In cddl/contrib/opensolaris/lib/libdtrace/common/dt_link.c, use the
correct printf format for a key_t (aka long).

MFC after: 1 week

In cddl/contrib/opensolaris/lib/libdtrace/i386/dt_isadep.c, use the
correct printf format for an unsigned long.

MFC after: 1 week

Cast away a clang alignment warning in drti.c's fixsymbol() function.
This code only runs on i386 and amd64, so there should be no problems if
buf + sec->dofs_offset is not aligned (which is unlikely anyway).

MFC after: 1 week

Some mdoc(7) style and typo fixes to zfs(8).

Submitted by: Nobuyuki Koganemaru <n-kogane@syd.odn.ne.jp>
MFC after: 3 days

Remove unnecesary "Ns" macros and add missing command example to zpool(8).

Reported by: Nobuyuki Koganemaru <kogane@FreeBSD.org>
MFC after: 3 days

Merge new ZFS features from illumos:

1644 add ZFS "clones" property
https://www.illumos.org/issues/1644

1645 add ZFS "written" and "written@..." properties
https://www.illumos.org/issues/1645

1646 "zfs send" should estimate size of stream
https://www.illumos.org/issues/1646

1647 "zfs destroy" should determine space reclaimed by destroying multiple
snapshots
https://www.illumos.org/issues/1647

1693 persistent 'comment' field for a zpool
https://www.illumos.org/issues/1693

1708 adjust size of zpool history data
https://www.illumos.org/issues/1708

1748 desire support for reguid in zfs
https://www.illumos.org/issues/1748

Obtained from: illumos (changesets 13514, 13524, 13525)
MFC after: 1 month

Change the Makefile in cddl/lib/drti to use bsd.lib.mk instead of
bsd.prog.mk -- we need to compile PIC, which requires a library build.
With this change, USDT (userspace DTrace probes) work from within
shared libraries.

PR: kern/159046
Submitted by: Alex Samorukov <samm at os2.kiev.ua>
Comments by: Scott Lystig Fritchie <slfritchie at snookles.com>
MFC after: 3 days

Use singular form for zfs destroy snapshot in zfs(8).

MFC after: 6 days

Add missing warning to zfs(8) for using "zfs destroy" with -r and -R flags.

Obtained from: illumos
MFC after: 6 days

Add missing -n flag to "zpool import" description.

MFC after: 1 week

Fix zfs(8) and zpool(8) context help to repport supported flags.

MFC after: 3 days

Update ZFS manual pages to a mdoc(7) reimplementation.

The zfs(8) and zpool(8) manual pages now match the state of the ZFS module
and have been customized for FreeBSD.

The new texts of the "Deduplication" subsection in zfs(8), the zpool "split"
command, the zfs "dedup" property and several other missing parts have been
added from illumos or OpenSolaris snv_134 (CDDL-licensed).

The mdoc(7) reimplementation of whole manual pages, the descriptions of the
zpool "readonly" property, "zfs diff" command and descriptions of several
other missing command flags and/or options were authored by myself.

MFC after: 1 week

Revert back to revision 227649 because of license uncertainity.

Manual pages from OpenSolaris svn_134 are still properly CDDL licensed
but I have been informed that the parts from s11ex are uncertain even
if they contain a CDDL header.

Fix mismerge in copyright of zpool(8).

MFC after: 4 days

Update zfs(8) and zpool(8) manpages from CDDL-licensed sources [1].
Improved alignment for a maximum width of 80 characters.
Mark unsupported parts as such.

Reported to vendor: Illumos issue #1801

References:
https://www.illumos.org/issues/1801

Obtained from: OpenSolaris CDDL manual pages (snv_134, s11express) [1]
MFC after: 4 days

For now, remove MANFILTER from ZFS manpages (bad effect on html output).

MFC after: 4 days

Update and desolarization of zdb(8) and zstreamdump(1) manual pages:
- synchronized to match new vendor code [1]
- removed ATTRIBUTES sections
- updated SEE ALSO sections
- properly updated copyright information (required by CDDL)
- remove empty lines via MANFILTER

Obtained from: Illumos [1]
MFC after: 5 days

More zfs(8) manpage fixes:
- remove shareiscsi property
- mark casesensitivity property as unsupported
- remove reference to Solaris Administration Guide

MFC after: 1 week

Fix reference to fsync(2).
Add more references to SEE ALSO section.

MFC after: 1 week

Update and desolarization of zfs(8) and zpool(8) manual pages:
- synchronized to match new vendor code (Illumos rev. 13513) [1]
- removed references to sun commands (replaced with FreeBSD commands)
- removed ATTRIBUTES sections
- updated SEE ALSO sections
- properly updated copyright information (required by CDDL)
- remove empty lines via MANFILTER

zfs(8) only:
- replaced "Zones" section with new "Jails" section
- removed misleading "ZFS Volumes as Swap or Dump Devices" section
- updated shareiscsi and sharesmb option information (not supported on FreeBSD)
- replace zoned property with jailed property

zpool(8) only:
- updated device names in examples

Obtained from: Illumos (as of rev. 13513:f84d4672fdbd) [1]
MFC after: 1 week

Import upstream changesets for the output of the "zpool" command:

952 separate intent logs should be obvious in 'zpool iostat' output
1337 `zpool status -D' should tell if there are no DDT entries

References:
https://www.illumos.org/issues/952
https://www.illumos.org/issues/1337

Obtained from: Illumos (issues 952, 1337; changesets 13384, 13432)
MFC after: 1 week

Fix a few gratuitous library dependencies. Some of the ZFS utilities
are linked with libraries they don't use:

- zinject doesn't use libavl
- ztest doesn't use libz
- zdb uses neither libavl nor libz
- zfs uses neither libbsdxml nor libm, nor libsbuf
- zpool uses neither libbsdxml nor libm, nor libsbuf

In addition, libzfs needs libm because it uses pow(), however it isn't
linked with -lm. This went unnoticed because all its users had -lm before.

Reviewed by: pjd, mm
Approved by: kib (mentor)
MFC after: 1 week

Update copyright to include myself.

MFC after: 2 weeks

Extend r226676 to allow rename without unmount even for file systems with
non-legacy mountpoints. It is better to be able to rename such file systems and
let them be mounted in old places until next reboot than using live CD, etc. to
rename with remount.

This is implemented by adding -u option to 'zfs rename'. If file system's
mountpoint property is set to 'legacy' or 'none', there is no need to specify -u.

Update zfs(8) manual page to reflect this addition.

MFC after: 2 weeks

Allow to rename file systems without remounting if it is possible.
It is possible for file systems with 'mountpoint' preperty set to 'legacy'
or 'none' - we don't have to change mount directory for them.
Currently such file systems are unmounted on rename and not even mounted back.

This introduces layering violation, as we need to update 'f_mntfromname'
field in statfs structure related to mountpoint (for the dataset we are
renaming and all its children).

In my opinion it is worth it, as it allow to update FreeBSD in even cleaner
way - in ZFS-only configuration root file system is ZFS file system with
'mountpoint' property set to 'legacy'. If root dataset is named system/rootfs,
we can snapshot it (system/rootfs@upgrade), clone it (system/oldrootfs),
update FreeBSD and if it doesn't boot we can boot back from system/oldrootfs
and rename it back to system/rootfs while it is mounted as /. Before it was
not possible, because unmounting / was not possible.

MFC after: 2 weeks

zdb: access dp_free_bpobj only if pool version is >= SPA_VERSION_DEADLISTS

Submitted by: avg
MFC after: 3 days

thr_create: new_thread_ID may be NULL

Submitted by: avg
MFC after: 3 days

libzpool task_alloc: pass only valid flags to kmem_alloc

tqflags may contain other flags besided those that are suitable for
kmem_alloc == umem_alloc

Submitted by: avg
MFC after: 3 days

Make all the lines align properly.

MFC after: 3 days

Remove assertion that prevents zfs rename of datasets with mountpoint=none
or mountpoint=legacy that have children datasets. This also fixes dataset
rename when receiving incremental snapshots as reported on freebsd-fs@

This assertion was made triggerable by opensolaris change #10196.

PR: bin/160400
Reviewed by: pjd
MFC after: 1 week

Fix serious bug in ZIL that can lead to pool corruption
in the case of a held dataset during remount.

Detailed description is available at:
https://www.illumos.org/issues/883

illumos-gate revision: 13380:161b964a0e10

Reviewed by: pjd
Approved by: re (kib)
Obtained from: Illumos (Bug #883)
MFC after: 3 days

Fix wrong initialization of "cmd" for calling the jail/unjail ioctl.

Reviewed by: pjd@, delphij@
Approved by: re (kib)
MFC after: 3 days

ZFS tries to allocate blocks evenly across all devices. This means when
devices are imbalanced zfs will lots of CPU searching for space on devices
which tend to be pretty full. It should instead fail quickly on the full
devices and move onto devices which have more availability.

New loader tunable: vfs.zfs.mg_alloc_failures (min = 8)

Illumos-gate changeset: 13379:4df42cc92254

Obtained from: Illumos (Bug #1051)
MFC after: 2 weeks

Resurrect the ZFS "aclmode" property
Change default of "aclmode" to "discard".

Illumos-gate changeset: 13370:8c04143bd318

Obtained from: Illumos (Feature #742)
MFC after: 2 weeks

cddl/contrib/opensolaris/cmd/zpool/zpool_main.c:
cddl/contrib/opensolaris/cmd/zpool/zpool.8:
cddl/contrib/opensolaris/lib/libzfs/common/libzfs_import.c:
Add the "zpool labelclear" command. This command can be
used to wipe the label data from a drive that is not
active in a pool. The optional "-f" argument can be
used to treat an exported or foreign vdev as "inactive"
thus allowing its label information to be cleared.

Correct reporting of missing leaf vdevs so that the GUID required to
perform pool actions is always displayed.

cddl/contrib/opensolaris/cmd/zpool/zpool_main.c:
The "zpool status" command reports the "last seen at"
device node path when the vdev name is being reported
by GUID. Augment this code to assume a GUID is reported
when a device goes missing after initial boot in addition
to the previous behavior of doing this for devices that
aren't seen at boot.

cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c:
In zpool_vdev_name(), report recently missing devices
by GUID. There is no guarantee they will return at
their previous location.

cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h:
cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c:
o Add zpool_pool_state_to_name() API to libzfs which converts a
pool_state_t into a user consumable string.
o While here, correct constness of make zpool_state_to_name()
and zpool_label_disk().

MFD after: 1 week

With retirement of cpumask_t and usage of cpuset_t for representing a
mask of CPUs, pc_other_cpus and pc_cpumask become highly inefficient.

Remove them and replace their usage with custom pc_cpuid magic (as,
atm, pc_cpumask can be easilly represented by (1 << pc_cpuid) and
pc_other_cpus by (all_cpus & ~(1 << pc_cpuid))).

This change is not targeted for MFC because of struct pcpu members
removal and dependency by cpumask_t retirement.

MD review by: marcel, marius, alc
Tested by: pluknet
MD testing by: marcel, marius, gonzo, andreast

Add a new "REFCOMPRESSRATIO" property.

For snapshots, this is the same as COMPRESSRATIO, but for
filesystems/volumes, the COMPRESSRATIO is based on the data "USED" (ie,
includes blocks in children, but not blocks shared with the origin).

This is needed to figure out how much space a filesystem would use if it
were not compressed (ignoring snapshots).

Illumos-gate revision: 13387

Obtained from: Illumos (Feature #1092)
MFC after: 2 weeks

Allow mountpoints as arguments for the 'zfs get' command.

Illumos-gate revision: 13295

Obtained from: Illumos (Feature #510)
MFC after: 1 week

Do not use #warning to warn about missing implementation of dt_popc(),
but just have a comment that this is broken.

This is just a bandaid until somebody can fix this correctly. The code
is just a broken as it was before r223262 - now buildworld just doesn't
fail.

Tested by: i386 + amd64 buildworld
With hat: benl co-mentor

Fix clang warnings.

Approved by: philip (mentor)

Remove C constructs that are incompatible with C++ from various
OpenSolaris and ZFS header files. These changes are sufficient
to allow a C++ program to use the libzfs library.

Note: The majority of these files already included 'extern "C"'
declarations, so the intention of providing C++ compatibility
already existed even if it wasn't provided.

cddl/compat/opensolaris/include/assert.h:
Wrap our compatibility assert implementation in
'extern "C"'. Since this is a compatibility header
I matched the Solaris style of doing this explicitly
rather than rely on FreeBSD's __BEGIN/END_DECLS macro.

sys/cddl/compat/opensolaris/sys/kstat.h:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/arc.h:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dsl_pool.h:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/ddt.h:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio.h:
Rename parameters in function declarations that conflict
with C++ keywords. This was the solution preferred by
members of the Illumos community.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_ioctl.h:
In C, nested structures are visible in the global namespace,
but in C++, they take on the namespace of the structure in
which they are contained. Flatten nested structure
definitions within struct zfs_cmd so these structures are
visible in the global namespace when compiled in both
languages.

Sponsored by: Spectra Logic Corporation

etire the cpumask_t type and replace it with cpuset_t usage.

This is intended to fix the bug where cpu mask objects are
capped to 32. MAXCPU, then, can now arbitrarely bumped to whatever
value. Anyway, as long as several structures in the kernel are
statically allocated and sized as MAXCPU, it is suggested to keep it
as low as possible for the time being.

Technical notes on this commit itself:
- More functions to handle with cpuset_t objects are introduced.
The most notable are cpusetobj_ffs() (which calculates a ffs(3)
for a cpuset_t object), cpusetobj_strprint() (which prepares a string
representing a cpuset_t object) and cpusetobj_strscan() (which
creates a valid cpuset_t starting from a string representation).
- pc_cpumask and pc_other_cpus are target to be removed soon.
With the moving from cpumask_t to cpuset_t they are now inefficient
and not really useful. Anyway, for the time being, please note that
access to pcpu datas is protected by sched_pin() in order to avoid
migrating the CPU while reading more than one (possible) word
- Please note that size of cpuset_t objects may differ between kernel
and userland. While this is not directly related to the patch itself,
it is good to understand that concept and possibly use the patch
as a reference on how to deal with cpuset_t objects in userland, when
accessing kernland members.
- KTR_CPUMASK is changed and now is represented through a string, to be
set as the example reported in NOTES.

Please additively note that no MAXCPU is bumped in this patch, but
private testing has been done until to MAXCPU=128 on a real 8x8x2(htt)
machine (amd64).

Please note that the FreeBSD version is not yet bumped because of
the upcoming pcpu changes. However, note that this patch is not
targeted for MFC.

People to thank for the time spent on this patch:
- sbruno, pluknet and Nicholas Esborn (nick AT desert DOT net) tested
several revision of the patches and really helped in improving
stability of this work.
- marius fixed several bugs in the sparc64 implementation and reviewed
patches related to ktr.
- jeff and jhb discussed the basic approach followed.
- kib and marcel made targeted review on some specific part of the
patch.
- marius, art, nwhitehorn and andreast reviewed MD specific part of
the patch.
- marius, andreast, gonzo, nwhitehorn and jceel tested MD specific
implementations of the patch.
- Other people have made contributions on other patches that have been
already committed and have been listed separately.

Companies that should be mentioned for having participated at several
degrees:
- Yahoo! for having offered the machines used for testing on big
count of CPUs.
- The FreeBSD Foundation for having sponsored my devsummit attendance,
which has been instrumental.
- Sandvine for having offered offices and infrastructure during
development.

(I really hope I didn't forget anyone, if it happened I apologize in
advance).

Close a race between libzfs and mountd when updating NFS exports.
- Flush the file descriptor for the new ZFS exports file before
sending a SIGHUP to mountd.

Reviewed by: pjd
Approved by: ken
MFC after: 3 days

Don't pass empty mount options to nmount(2).

Reviewed by: pjd
MFC after: 2 weeks

Fix 'zfs list <path>' handling. If the path was found, the 'ret' variable was
uninitialized.

PR: kern/155940
Submitted by: KOIE Hidetaka <koie@suri.co.jp>
MFC after: 1 week

Properly print characters larger than 127.

Submitted by: noordsij <noordsij@cs.helsinki.fi>
Reviewed by: Eric Schrock <eric.schrock@delphix.com>
MFC after: 1 month

Finally... Import the latest open-source ZFS version - (SPA) 28.

Few new things available from now on:

- Data deduplication.
- Triple parity RAIDZ (RAIDZ3).
- zfs diff.
- zpool split.
- Snapshot holds.
- zpool import -F. Allows to rewind corrupted pool to earlier
transaction group.
- Possibility to import pool in read-only mode.

MFC after: 1 month

When building libzpool on ia64 or sparc64, don't add the .note.GNU-stack
section.

Submitted by: kib

Fix some leftover binaries and shared libraries in the system that still
have an executable stack, due to linking in hand-assembled .S or .s
files, that have no .GNU-stack sections:

RWX --- --- /lib/libcrypto.so.6
RWX --- --- /lib/libmd.so.5
RWX --- --- /lib/libz.so.6
RWX --- --- /lib/libzpool.so.2
RWX --- --- /usr/lib/liblzma.so.5

These were found using scanelf, from the sysutils/pax-utils port.

Reviewed by: kib

Re-commit the zfs sysctl(9) type-safety changes.

Thanks to dim and pjd for the pointer to zfs_context.h for building
userland.

Print message with information about updating the boot code if a new
vdev is attached to a root pool (e.g. when creating a mirrored boot pool).

Reviewed by: pav
Approved by: delphij (mentor)
MFC after: 3 days

Do not print OpenSolaris hint to use (non-existing) installgrub(1) command
if creating a mirror by attaching a new vdev to a root pool.

Reported by: James R. Van Artsdalen (on freebsd-fs@freebsd.org)
Approved by: delphij (mentor)
MFC after: 3 days

Do not lint code beyond necessity (with apologies to Wiliam of Ockham).

Don't lint externally maintained CDDL code, or relint the 32-bit libraries
in amd64 mode.

Fix ztest when it is executed by just 'ztest' and not by full path
'/usr/bin/ztest'.

1. Remove invalid assertion.
2. Properly recalculate delta in case pthread_cond_timedwait() is interrupted.
3. Style fix.

Reported by: [1] App Deb <appdebgr@gmail.com>

Use ${PICFLAG} instead of -fpic.

Properly handle IO with B_FAILFAST
Retry IO once with ZIO_FLAG_TRYHARD before declaring a pool faulted

OpenSolaris revision and Bug IDs:

9725:0bf7402e8022
6843014 ZFS B_FAILFAST handling is broken

Approved by: delphij (mentor)
Obtained from: OpenSolaris (Bug ID 6843014)
MFC after: 3 weeks

Enable offlining of log devices.

OpenSolaris revision and Bug IDs:

9701:cc5b64682e64
6803605 should be able to offline log devices
6726045 vdev_deflate_ratio is not set when offlining a log device
6599442 zpool import has faults in the display

Approved by: delphij (mentor)
Obtained from: OpenSolaris (Bug ID 6803605, 6726045, 6599442)
MFC after: 3 weeks

Minor grammar fixes.

Remove duplicate include of <strings.h>

Approved by: delphij (mentor)
MFC after: 3 days

Add a man page for plockstat.

Sponsored by: The FreeBSD Foundation

Give a chance to the target binary to run the ctors by waiting until it
has reached main(). This allows plockstat to work.

Sponsored by: The FreeBSD Foundation

Disable debug by default.

Sponsored by: The FreeBSD Foundation

Don't try to map the USDT probes. This is necessary because there is no
__SUNW_dof symbol present in FreeBSD binaries.

Sponsored by: The FreeBSD Foundation

Don't clobber an existing target object file when doing the DTrace
linking process. This is needed because we change the source object
files and the second this dtrace -G is run, no probes will be found.
This hack allows us to build postgres with DTrace probes enabled. I'll
try to find a way to fix this without needing this hack.

Sponsored by: The FreeBSD Foundation

Include <sys/sdt.h> on a generated header.

Sponsored by: The FreeBSD Foundation

Eliminate confusing while () loop. In the first version of the code it was
there to avoid gotos, but in the current version it serves no purpose.

MFC after: 2 weeks

When upgrading a pool which contain root file system, give user a hint that
he should update boot code.

MFC after: 2 weeks

Give user a hint what to do when /usr/lib/zfs/pyzfs.py is missing.

MFC after: 2 weeks

Print errors on stderr.

MFC after: 2 weeks

Fix 'zfs allow' (maybe not only) returning:

cannot access dataset system/usr/home: Operation not supported

by including libzfs_impl.h. What libzfs_impl.h does is to redefine ioctl() to
be compatible with OpenSolaris. More specifically OpenSolaris returns ENOMEM
when buffer is too small and sets field zc_nvlist_dst_size to the size that
will be big enough for the data. In FreeBSD case ioctl() doesn't copy data
structure back in case of a failure. We work-around it in kernel and libzfs by
returning 0 from ioctl() and always checking if zc_nvlist_dst_size hasn't
changed. For this work-around to work in pyzfs we need this compatible ioctl()
which is implemented in libzfs_impl.h.

MFC after: 2 weeks

Update ZFS metaslab code from OpenSolaris.
This provides a noticeable write speedup, especially on pools with
less than 30% of free space.

Detailed information (OpenSolaris onnv changesets and Bug IDs):

11146:7e58f40bcb1c
6826241 Sync write IOPS drops dramatically during TXG sync
6869229 zfs should switch to shiny new metaslabs more frequently

11728:59fdb3b856f6
6918420 zdb -m has issues printing metaslab statistics

12047:7c1fcc8419ca
6917066 zfs block picking can be improved

Approved by: delphij (mentor)
Obtained from: OpenSolaris (Bug ID 6826241, 6869229, 6918420, 6917066)
MFC after: 2 weeks

Install dtruss.

Sponsored by: The FreeBSD Foundation

Add a man page for dtruss.

Sponsored by: The FreeBSD Foundation

Port dtruss to FreeBSD.

Sponsored by: The FreeBSD Foundation

Merge dtracetoolkit/dtruss from the vendor.

Sponsored by: The FreeBSD Foundation

Remove this directory for an attempt to merge from the vendor tree.

MFtbemd:

Prefer MACHNE_CPUARCH to MACHINE_ARCH in most contexts where you want
to test of all the CPUs of a given family conform.

Use real atomic operations for sparc64.

MFC after: 1 week

Add powerpc64 to the list of architectures with real atomic operations.

Submitted by: imp

Don't link drti.o with libelf_pic.a. This means that all software with
userland SDT probes must be linked with libelf explicitly.

Requested by: kib
Sponsored by: The FreeBSD Foundation

Use double quotes when checking the value of MACHINE_ARCH.

Sponsored by: The FreeBSD Foundation

Add a Makefile for dtruss.

Sponsored by: The FreeBSD Foundation

Port plockstat to FreeBSD. This will be connected to the build later.

Sponsored by: The FreeBSD Foundation

Link drti.o with a PIC version of libelf. This is needed because
drti.o depends on libelf and this avoids linking every other drti.o
program (namely programs with USDT probes) with libelf.

Sponsored by: The FreeBSD Foundation

Add a directory for the DTraceToolkit.

Sponsored by: The FreeBSD Foundation

Add libdtrace support for tracing userland programs.

Summary of changes:
* Implement a compatibility shim between Solaris libproc and our
libproc and remove several ifdefs because of this.
* Port the drti to FreeBSD.
* Implement the missing DOODAD sections
* Link with libproc and librtld_db
* Support for ustack, jstack and uregs (by sson@)
* Misc bugfixing

When writing the SUWN_dof section, we had to resort to building the ELF
file layout by "hand". This is the job of libelf, but our libelf doesn't
support this yet. When libelf is fixed, we can remove the code under
#ifdef BROKEN_LIBELF.

Sponsored by: The FreeBSD Foundation

Use double quotes when checking the value of MACHINE_ARCH.

Sponsored by: The FreeBSD Foundation

The DTrace instrumentation object is going to be i386/amd64 only.

Sponsored by: The FreeBSD Foundation

Port most of the DTrace tests to FreeBSD.

Sponsored by: The FreeBSD Foundation

Add man pages for the CTF tools.

Sponsored by: The FreeBSD Foundation

Fix another mismerge: bring back the definition of DT_MUTEX_HELD().

Fix the result of a mismerge. MUTEX_HELD should be DT_MUTEX_HELD() and
Plmid() is not supposed to be called.

MFV OpenSolaris DTrace userland bits.

lockstat(1) now depends on libutil and librtld_db.

Sponsored by: The FreeBSD Foundation

proc_wait() was replaced by proc_wstatus(). Right now this call is
useless because userland support is not working on HEAD, but this makes
libdtrace compile again.

Sponsored by: The FreeBSD Foundation

libdtrace, dtrace(1) and lockstat(1) are for i386 and amd64 only right
now.

Sponsored by: The FreeBSD Foundation

Update to the new proc_detach() function.

Sponsored by: The FreeBSD Foundation

dtrace(1) now depends on libutil and librtld_db.

Sponsored by: The FreeBSD Foundation

Catch up with r210324. d_buf will be NULL for SHT_NOBITS sections, do not
attempt to copy from it in that case.

MFC after: 1 month

dtrace: correctly map sections to addresses in elf object modules (amd64)

Unlike for modules with dso type, in elf object modules all the sections
have virtual address of zero. So, it is insufficient to add module base
address to section virtual address (as recorded in section header) to
get section address in kernel memory.
Instead, we should apply the same calculations that are performed by
kernel loaders (in boot code and in kernel) when they lay out sections
in memory.
Also, unlike OpenSolaris, the sections are not collapsed into just .text,
.data and .bss by module loaders, so we need to take additional care
about other sections.

Note that in-kernel symbol-to-address mapping worked just fine, e.g. fbt
provider could correctly find the functions, etc. It's only in userland
that the mapping in both direction worked incorrectly, e.g. in stack()
output addresses of functions in kernel modules were not translated to
their names.

Reviewed by: rpaulo
MFC after: 3 weeks

Enable fake resolving of SMB RIDs by using nulldomain and UID_NOBODY
- fixes panics when Solaris/OpenSolaris pools that contain files
uploaded with the SMB protocol are accessed

Enable seting/unsetting the sharesmb property (dummy action)
- allows users who import pools from Solaris/Opensolaris to unset
the sharesmb property and get rid of annoying messages

PR: kern/145778, kern/148709
Approved by: pjd, delphij (mentor)
MFC after: 7 weeks

MFV:
OpenSolaris' plockstat utility.

Sponsored by: The FreeBSD Foundation

Merge from vendor: libdtrace MD parts needed by fasttrap.

Sponsored by: The FreeBSD Foundation

MFp4 180933:

Initialize rw_count properly so that zdb(8) doesn't trigger assertion in
rw_enter():

ASSERT(rwlp->rw_count == 0);

While here, assert that rw_count is 0 when destroying the lock.

MFC after: 1 week

Merge ZFS version 15 and almost all OpenSolaris bugfixes referenced
in Solaris 10 updates 141445-09 and 142901-14.

Detailed information:
(OpenSolaris revisions and Bug IDs, Solaris 10 patch numbers)

7844:effed23820ae
6755435 zfs_open() and zfs_close() needs to use ZFS_ENTER/ZFS_VERIFY_ZP (141445-01)

7897:e520d8258820
6748436 inconsistent zpool.cache in boot_archive could panic a zfs root filesystem upon boot-up (141445-01)

7965:b795da521357
6740164 zpool attach can create an illegal root pool (141909-02)

8084:b811cc60d650
6769612 zpool_import() will continue to write to cachefile even if altroot is set (N/A)

8121:7fd09d4ebd9c
6757430 want an option for zdb to disable space map loading and leak tracking (141445-01)

8129:e4f45a0bfbb0
6542860 ASSERT: reason != VDEV_LABEL_REMOVE||vdev_inuse(vd, crtxg, reason, 0) (141445-01)

8188:fd00c0a81e80
6761100 want zdb option to select older uberblocks (141445-01)

8190:6eeea43ced42
6774886 zfs_setattr() won't allow ndmp to restore SUNWattr_rw (141445-01)

8225:59a9961c2aeb
6737463 panic while trying to write out config file if root pool import fails (141445-01)

8227:f7d7be9b1f56
6765294 Refactor replay (141445-01)

8228:51e9ca9ee3a5
6572357 libzfs should do more to avoid mnttab lookups (141909-01)
6572376 zfs_iter_filesystems and zfs_iter_snapshots get objset stats twice (141909-01)

8241:5a60f16123ba
6328632 zpool offline is a bit too conservative (141445-01)
6739487 ASSERT: txg <= spa_final_txg due to scrub/export race (141445-01)
6767129 ASSERT: cvd->vdev_isspare, in spa_vdev_detach() (141445-01)
6747698 checksum failures after offline -t / export / import / scrub (141445-01)
6745863 ZFS writes to disk after it has been offlined (141445-01)
6722540 50% slowdown on scrub/resilver with certain vdev configurations (141445-01)
6759999 resilver logic rewrites ditto blocks on both source and destination (141445-01)
6758107 I/O should never suspend during spa_load() (141445-01)
6776548 codereview(1) runs off the page when faced with multi-line comments (N/A)
6761406 AMD errata 91 workaround doesn't work on 64-bit systems (141445-01)

8242:e46e4b2f0a03
6770866 GRUB/ZFS should require physical path or devid, but not both (141445-01)

8269:03a7e9050cfd
6674216 "zfs share" doesn't work, but "zfs set sharenfs=on" does (141445-01)
6621164 $SRC/cmd/zfs/zfs_main.c seems to have a syntax error in the translation note (141445-01)
6635482 i18n problems in libzfs_dataset.c and zfs_main.c (141445-01)
6595194 "zfs get" VALUE column is as wide as NAME (141445-01)
6722991 vdev_disk.c: error checking for ddi_pathname_to_dev_t() must test for NODEV (141445-01)
6396518 ASSERT strings shouldn't be pre-processed (141445-01)

8274:846b39508aff
6713916 scrub/resilver needlessly decompress data (141445-01)

8343:655db2375fed
6739553 libzfs_status msgid table is out of sync (141445-01)
6784104 libzfs unfairly rejects numerical values greater than 2^63 (141445-01)
6784108 zfs_realloc() should not free original memory on failure (141445-01)

8525:e0e0e525d0f8
6788830 set large value to reservation cause core dump (141445-01)
6791064 want sysevents for ZFS scrub (141445-01)
6791066 need to be able to set cachefile on faulted pools (141445-01)
6791071 zpool_do_import() should not enable datasets on faulted pools (141445-01)
6792134 getting multiple properties on a faulted pool leads to confusion (141445-01)

8547:bcc7b46e5ff7
6792884 Vista clients cannot access .zfs (141445-01)

8632:36ef517870a3
6798384 It can take a village to raise a zio (141445-01)

8636:7e4ce9158df3
6551866 deadlock between zfs_write(), zfs_freesp(), and zfs_putapage() (141909-01)
6504953 zfs_getpage() misunderstands VOP_GETPAGE() interface (141909-01)
6702206 ZFS read/writer lock contention throttles sendfile() benchmark (141445-01)
6780491 Zone on a ZFS filesystem has poor fork/exec performance (141445-01)
6747596 assertion failed: DVA_EQUAL(BP_IDENTITY(&zio->io_bp_orig), BP_IDENTITY(zio->io_bp))); (141445-01)

8692:692d4668b40d
6801507 ZFS read aggregation should not mind the gap (141445-01)

8697:e62d2612c14d
6633095 creating a filesystem with many properties set is slow (141445-01)

8768:dfecfdbb27ed
6775697 oracle crashes when overwriting after hitting quota on zfs (141909-01)

8811:f8deccf701cf
6790687 libzfs mnttab caching ignores external changes (141445-01)
6791101 memory leak from libzfs_mnttab_init (141445-01)

8845:91af0d9c0790
6800942 smb_session_create() incorrectly stores IP addresses (N/A)
6582163 Access Control List (ACL) for shares (141445-01)
6804954 smb_search - shortname field should be space padded following the NULL terminator (N/A)
6800184 Panic at smb_oplock_conflict+0x35() (N/A)

8876:59d2e67b4b65
6803822 Reboot after replacement of system disk in a ZFS mirror drops to grub> prompt (141445-01)

8924:5af812f84759
6789318 coredump when issue zdb -uuuu poolname/ (141445-01)
6790345 zdb -dddd -e poolname coredump (141445-01)
6797109 zdb: 'zdb -dddddd pool_name/fs_name inode' coredump if the file with inode was deleted (141445-01)
6797118 zdb: 'zdb -dddddd poolname inum' coredump if I miss the fs name (141445-01)
6803343 shareiscsi=on failed, iscsitgtd failed request to share (141445-01)

9030:243fd360d81f
6815893 hang mounting a dataset after booting into a new boot environment (141445-01)

9056:826e1858a846
6809691 'zpool create -f' no longer overwrites ufs infomation (141445-01)

9179:d8fbd96b79b3
6790064 zfs needs to determine uid and gid earlier in create process (141445-01)

9214:8d350e5d04aa
6604992 forced unmount + being in .zfs/snapshot/<snap1> = not happy (141909-01)
6810367 assertion failed: dvp->v_flag & VROOT, file: ../../common/fs/gfs.c, line: 426 (141909-01)

9229:e3f8b41e5db4
6807765 ztest_dsl_dataset_promote_busy needs to clean up after ENOSPC (141445-01)

9230:e4561e3eb1ef
6821169 offlining a device results in checksum errors (141445-01)
6821170 ZFS should not increment error stats for unavailable devices (141445-01)
6824006 need to increase issue and interrupt taskqs threads in zfs (141445-01)

9234:bffdc4fc05c4
6792139 recovering from a suspended pool needs some work (141445-01)
6794830 reboot command hangs on a failed zfs pool (141445-01)

9246:67c03c93c071
6824062 System panicked in zfs_mount due to NULL pointer dereference when running btts and svvs tests (141909-01)

9276:a8a7fc849933
6816124 System crash running zpool destroy on broken zpool (141445-03)

9355:09928982c591
6818183 zfs snapshot -r is slow due to set_snap_props() doing txg_wait_synced() for each new snapshot (141445-03)

9391:413d0661ef33
6710376 log device can show incorrect status when other parts of pool are degraded (141445-03)

9396:f41cf682d0d3 (part already merged)
6501037 want user/group quotas on ZFS (141445-03)
6827260 assertion failed in arc_read(): hdr == pbuf->b_hdr (141445-03)
6815592 panic: No such hold X on refcount Y from zfs_znode_move (141445-03)
6759986 zfs list shows temporary %clone when doing online zfs recv (141445-03)

9404:319573cd93f8
6774713 zfs ignores canmount=noauto when sharenfs property != off (141445-03)

9412:4aefd8704ce0
6717022 ZFS DMU needs zero-copy support (141445-03)

9425:e7ffacaec3a8
6799895 spa_add_spares() needs to be protected by config lock (141445-03)
6826466 want to post sysevents on hot spare activation (141445-03)
6826468 spa 'allowfaulted' needs some work (141445-03)
6826469 kernel support for storing vdev FRU information (141445-03)
6826470 skip posting checksum errors from DTL regions of leaf vdevs (141445-03)
6826471 I/O errors after device remove probe can confuse FMA (141445-03)
6826472 spares should enjoy some of the benefits of cache devices (141445-03)

9443:2a96d8478e95
6833711 gang leaders shouldn't have to be logical (141445-03)

9463:d0bd231c7518
6764124 want zdb to be able to checksum metadata blocks only (141445-03)

9465:8372081b8019
6830237 zfs panic in zfs_groupmember() (141445-03)

9466:1fdfd1fed9c4
6833162 phantom log device in zpool status (141445-03)

9469:4f68f041ddcd
6824968 add ZFS userquota support to rquotad (141445-03)

9470:6d827468d7b5
6834217 godfather I/O should reexecute (141445-03)

9480:fcff33da767f
6596237 Stop looking and start ganging (141909-02)

9493:9933d599bc93
6623978 lwb->lwb_buf != NULL, file ../../../uts/common/fs/zfs/zil.c, line 787, function zil_lwb_commit (141445-06)

9512:64cafcbcc337
6801810 Commit of aligned streaming rewrites to ZIL device causes unwanted disk reads (N/A)

9515:d3b739d9d043
6586537 async zio taskqs can block out userland commands (142901-09)

9554:787363635b6a
6836768 zfs_userspace() callback has no way to indicate failure (N/A)

9574:1eb6a6ab2c57
6838062 zfs panics when an error is encountered in space_map_load() (141909-02)

9583:b0696cd037cc
6794136 Panic BAD TRAP: type=e when importing degraded zraid pool. (141909-03)

9630:e25a03f552e0
6776104 "zfs import" deadlock between spa_unload() and spa_async_thread() (141445-06)

9653:a70048a304d1
6664765 Unable to remove files when using fat-zap and quota exceeded on ZFS filesystem (141445-06)

9688:127be1845343
6841321 zfs userspace / zfs get userused@ doesn't work on mounted snapshot (N/A)
6843069 zfs get userused@S-1-... doesn't work (N/A)

9873:8ddc892eca6e
6847229 assertion failed: refcount_count(&tx->tx_space_written) + delta <= tx->tx_space_towrite in dmu_tx.c (141445-06)

9904:d260bd3fd47c
6838344 kernel heap corruption detected on zil while stress testing (141445-06)

9951:a4895b3dd543
6844900 zfs_ioc_userspace_upgrade leaks (N/A)

10040:38b25aeeaf7a
6857012 zfs panics on zpool import (141445-06)

10000:241a51d8720c
6848242 zdb -e no longer works as expected (N/A)

10100:4a6965f6bef8
6856634 snv_117 not booting: zfs_parse_bootfs: error2 (141445-07)

10160:a45b03783d44
6861983 zfs should use new name <-> SID interfaces (N/A)
6862984 userquota commands can hang (141445-06)

10299:80845694147f
6696858 zfs receive of incremental replication stream can dereference NULL pointer and crash (N/A)

10302:a9e3d1987706
6696858 zfs receive of incremental replication stream can dereference NULL pointer and crash (fix lint) (N/A)

10575:2a8816c5173b (partial merge)
6882227 spa_async_remove() shouldn't do a full clear (142901-14)

10800:469478b180d9
6880764 fsync on zfs is broken if writes are greater than 32kb on a hard crash and no log attached (142901-09)
6793430 zdb -ivvvv assertion failure: bp->blk_cksum.zc_word[2] == dmu_objset_id(zilog->zl_os) (N/A)

10801:e0bf032e8673 (partial merge)
6822816 assertion failed: zap_remove_int(ds_next_clones_obj) returns ENOENT (142901-09)

10810:b6b161a6ae4a
6892298 buf->b_hdr->b_state != arc_anon, file: ../../common/fs/zfs/arc.c, line: 2849 (142901-09)

10890:499786962772
6807339 spurious checksum errors when replacing a vdev (142901-13)

11249:6c30f7dfc97b
6906110 bad trap panic in zil_replay_log_record (142901-13)
6906946 zfs replay isn't handling uid/gid correctly (142901-13)

11454:6e69bacc1a5a
6898245 suspended zpool should not cause rest of the zfs/zpool commands to hang (142901-10)

11546:42ea6be8961b (partial merge)
6833999 3-way deadlock in dsl_dataset_hold_ref() and dsl_sync_task_group_sync() (142901-09)

Discussed with: pjd
Approved by: delphij (mentor)
Obtained from: OpenSolaris (multiple Bug IDs)
MFC after: 2 months

- Add support for the "sec" option to "zfs set sharenfs"

Submitted by: Dmitry Morozovsky <marck@FreeBSD.org>
Approved by: delphij (mentor)
MFC after: 3 days

Set svn:executable property so we can run the DTrace test suite.

Unbreak platforms with char unsigned by default. Oddly enough, GCC isn't
satisfied with a simple cast to int in the check against EOF, so the fix
is a bit involved by actually having to go through a temporary variable.

Do not allow EOF token to be put back into input buffer.

This reimplements previous change from r20930 in more generic way.

MFC after: 1 week

In dtrace lexer, do not unput token if it is EOF.

This is harmless with OpenSolaris fex and yacc, but with FreeBSD
causes '\xff' symbol to be physically put into input buffer, causing
subsequent syntax error.

MFC after: 1 week

Allow to use 'jailed' property again.

Reported by: Eugene Mitrofanov <eugene@imedia.ru>
MFC after: 3 days

Fix zfs receive temporarily changing unchanged stream properties.
Fix possible panic with zfs_enable_datasets.

OpenSolaris onnv revision: 8536:33bd5de3260e

Approved by: pjd, delphij (mentor)
Obtained from: OpenSolaris (Bug ID 6748561, 6757075)
MFC after: 3 days

Fix userland build by making io_task available only for the kernel and by
providing taskq_dispatch_safe() macro.

MFC after: 1 week

Import OpenSolaris revision 7837:001de5627df3
It includes the following changes:
- parallel reads in traversal code (Bug ID 6333409)
- faster traversal for zfs send (Bug ID 6418042)
- traversal code cleanup (Bug ID 6725675)
- fix for two scrub related bugs (Bug ID 6729696, 6730101)
- fix assertion in dbuf_verify (Bug ID 6752226)
- fix panic during zfs send with i/o errors (Bug ID 6577985)
- replace P2CROSS with P2BOUNDARY (Bug ID 6725680)

List of OpenSolaris Bug IDs:
6333409, 6418042, 6757112, 6725668, 6725675, 6725680,
6725698, 6729696, 6730101, 6752226, 6577985, 6755042

Approved by: pjd, delphij (mentor)
Obtained from: OpenSolaris (multiple Bug IDs)
MFC after: 1 week

Fix possible panic with zfs destroy.

OpenSolaris onnv revision: 8779:f164e0e90508

PR: kern/146471
Approved by: pjd, delphij (mentor)
Obtained from: OpenSolaris (Bug ID 6784924)
MFC after: 3 days

Merger of the quota64 project into head.

This joint work of Dag-Erling Smørgrav and myself updates the
FFS quota system to support both traditional 32-bit and new 64-bit
quotas (for those of you who want to put 2+Tb quotas on your users).

By default quotas are not compiled into the kernel. To include them
in your kernel configuration you need to specify:

options QUOTA # Enable FFS quotas

If you are already running with the current 32-bit quotas, they
should continue to work just as they have in the past. If you
wish to convert to using 64-bit quotas, use `quotacheck -c 64';
if you wish to revert from 64-bit quotas back to 32-bit quotas,
use `quotacheck -c 32'.

There is a new library of functions to simplify the use of the
quota system, do `man quotafile' for details. If your application
is currently using the quotactl(2), it is highly recommended that
you convert your application to use the quotafile interface.
Note that existing binaries will continue to work.

Special thanks to John Kozubik of rsync.net for getting me
interested in pursuing 64-bit quota support and for funding
part of my development time on this project.

Introduce hardforce export option (-F) for "zpool export".
When exporting with this flag, zpool.cache remains untouched.

OpenSolaris onnv revision: 8211:32722be6ad3b

Approved by: pjd, delphij (mentor)
Obtained from: OpenSolaris (Bug ID: 6775357)

Enable "zfs list" to list explicitly requested snapshots.

Partial import of OpenSolaris onnv revision:
8415:8809e849f63e

PR: kern/146297
Submitted by: myself
Approved by: pjd, delphij (mentor)
Obtained from: OpenSolaris (Bug ID 6758338)
MFC after: 2 weeks

Do not encode more than CTF_MAX_VLEN(1023) enum members.

CTF can not represent enums with more than CTF_MAX_VLEN members, but
ctfconvert will happily ignore that limitation and create CTF section no
other tool can interpret.

This change is different from similar change from upstream, which just
returns an error if big enum is encountered. Doing that means that
every FreeBSD kernel with compiled in hwpmc will have no useable CTF
information due to pmc_event enum having 1236+ members.

Refine previous partial merge of OpenSolaris onnv revision 9396:f41cf682d0d3.
This fixes a regression that zfs list would crash on zfs having user properties.

PR: kern/145377
Submitted by: mm
Approved by: pjd
Obtained from: OpenSolaris
MFC after: 10 days

- update zfs and zpool manual pages to match the current state of the source

PR: 144984
Submitted by: mm@
Approved by: pjd@
Obtained from: OpenSolaris
MFC after: 4 days

Merge OpenSolaris revision 9396:f41cf682d0d3:

6830813 zfs list -t all fails assertion

*Note that this is only a partial merge of this revision addressing only
this one issue.*

PR: bin/144720
Submitted by: mm
Approved by: pjd
Obtained from: OpenSolaris
MFC after: 1 month

Merge OpenSolaris revision 9365:7838a22eccd6:

PSARC/2009/171 zfs list -d and zfs get -d
6762432 zfs list --depth

PR: bin/144720
Submitted by: mm
Approved by: pjd
Obtained from: OpenSolaris
MFC after: 1 month

Merge OpenSolaris revision 8802:010b31dd4c53:

6773366 "zfs list" memory consumption can be further reduced

PR: bin/144720
Submitted by: mm
Approved by: pjd
Obtained from: OpenSolaris
MFC after: 1 month

Remove manual .includes in cddl Makefiles

- Break the dependency on ../Makefile.inc for .PATH, and include
../Makefile.inc implicitly. This is required to ...
- Set WARNS?=6 in top-level Makefile.inc
- Remove now redundant WARNS settings, add WARNS?=0 where appropriate
- Remove redundant SHLIB_MAJOR overrides
- Use NO_MAN, not MK_MAN=no
- Remove redundant inclusion of bsd.own.mk
- Order Makefiles more according to style.Makefile(9)
- Reduce diff of cddl Makefiles against each other

No objection: pjd
Approved by: ed (co-mentor)

Fixed dependencies (make checkdpadd).

Apply OpenSolaris revision 8012 which brings our zpool to version 14,
making it possible for zpools created on OpenSolaris 2009.06 be used
on FreeBSD.

PR: kern/141800
Submitted by: mm
Reviewed by: pjd, trasz
Obtained from: OpenSolaris
MFC after: 2 weeks

Apply fix Solaris bug 6462803 zfs snapshot -r failed because
filesystem was busy.

Submitted by: mm
Approved by: pjd
MFC after: 2 weeks

Add an option to specify that the received ZFS should not be automatically
mounted (receive -u).

Obtained from: OpenSolaris (onnv revision 8584:327a1b6dd944)
Approved by: pjd

Properly mark ZFS properties which are not changeable under FreeBSD.

Reviewed by: pjd

'aclmode' and 'aclinherit' properties should work as advertised; don't
refuse to set them.

Fletcher4 is not the default checksum algorithm.

MFC after: 3 days

Implement __assert() for Solaris-specific code. Until now Solaris code was
using Solaris prototype for __assert(), but FreeBSD's implementation.
Both take different arguments, so we were either core-dumping in assert()
or printing garbage.

Reported by: avg
MFC after: 1 week

Fix detection of file system being shared. After this change commands like:

# zfs unshare -a
# zfs destroy foo/bar
# zfs rename foo/bar foo/baz

should properly remove exported file systems.

MFC after: 3 days

Teach zdb(8) how to obtain GEOM provider size.

PR: kern/133134
Reported by: Philipp Wuensche <cryx-freebsd@h3q.com>
MFC after: 3 days

Fix receive when dataset has no / in its name.

Submitted by: James R. Van Artsdalen <james-freebsd-current@jrv.org>
Approved by: re (kib)

Remove files that are no longer used.

Discussed with: kmacy
Approved by: re (kib)

Bump the version of all non-symbol-versioned shared libraries in
preparation for 8.0-RELEASE. Add the previous version of those
libraries to ObsoleteFiles.inc and bump __FreeBSD_Version.

Reviewed by: kib
Approved by: re (rwatson)

The mutex_owned() macro should operate on kmutex_t and not on mutex_t.
This fixes 'zdb <poolname>' crash.

Reported by: avg
Approved by: re (kib)

Bootstrap mergeinfo for the OpenSolaris contrib bits.

Fix lockstat breakage to arm/powerpc buildworld.
Thanks to IBM char's are unsigned on arm/powerpc.

Approved by: gnn (mentor)

Add the OpenSolaris lockstat(1M) command. Requires the dtrace driver,
the lockstat provider, and the ksyms(4) pseudo driver kernel modules.

Approved by: gnn (mentor)

atomic.S has been renamed opensolaris_atomic.S to avoid collisions

define VN_RELE_ASYNC for use by libzpool

Switch over to gnu99 compilation on default for userland.

Tested by: make universe
Tested by: ports exp build (done by pav)
Reviewed by: ru
Reviewed by: silence on arch
Approved by: ed (mentor)

Correct the sense of the "use32" terenary operator so that it uses
'lib32' for 32-bit binaries on amd64 and 'lib' for 64-bit binaries.
Remove an #ifdef __i386__ that defined use32 to the wrong value on
i386 to workaround the bug in the terenary operator.

MFC after: 1 month

Include param.h instead of types.h before mount.h so that the nested
include of param.h can be removed from audit.h.

MFC after: 3 days

Including mount.h requires including param.h.

MFC after: 3 weeks

IFp4: Don't rely on disk IDs and always use vdev guids, which means always look
up for components by reading metadata. This might be slower when there are big
number of disks in the system, but is definiately more reliable.

Fix a warning on amd64 caused by using int for request argument instead of
unsigned long:

WARNING pid 12888 (zfs/zpool): ioctl sign-extension ioctl ffffffffcc285aXX

Reported by: kris

Update ZFS from version 6 to 13 and bring some FreeBSD-specific changes.

This bring huge amount of changes, I'll enumerate only user-visible changes:

- Delegated Administration

Allows regular users to perform ZFS operations, like file system
creation, snapshot creation, etc.

- L2ARC

Level 2 cache for ZFS - allows to use additional disks for cache.
Huge performance improvements mostly for random read of mostly
static content.

- slog

Allow to use additional disks for ZFS Intent Log to speed up
operations like fsync(2).

- vfs.zfs.super_owner

Allows regular users to perform privileged operations on files stored
on ZFS file systems owned by him. Very careful with this one.

- chflags(2)

Not all the flags are supported. This still needs work.

- ZFSBoot

Support to boot off of ZFS pool. Not finished, AFAIK.

Submitted by: dfr

- Snapshot properties

- New failure modes

Before if write requested failed, system paniced. Now one
can select from one of three failure modes:
- panic - panic on write error
- wait - wait for disk to reappear
- continue - serve read requests if possible, block write requests

- Refquota, refreservation properties

Just quota and reservation properties, but don't count space consumed
by children file systems, clones and snapshots.

- Sparse volumes

ZVOLs that don't reserve space in the pool.

- External attributes

Compatible with extattr(2).

- NFSv4-ACLs

Not sure about the status, might not be complete yet.

Submitted by: trasz

- Creation-time properties

- Regression tests for zpool(8) command.

Obtained from: OpenSolaris

Merge latest DTrace changes from Perforce.

Approved by: jb

Mips doesn't have a long double that's different in size from double.

Add a missing file change from the VOP_GETATTR() argument axing.

Don't need to include vmem.h anymore.

Add the dtrace client app to the build.

Add the DTrace build tools.

Oops, these files belonged under src/sys, not src. Sorry.

This commit was generated by cvs2svn to compensate for changes in r179191,
which included commits to RCS files with non-trunk default branches.

Add the DTrace libraries and D library scripts.

Add Solaris compatibility definitions for things we don't really want in
FreeBSD headers.

* Handle the different ioctl design.
* Add a couple of FreeBSD action extensions.

Make this file compile cleanly.

Leave out some header files.

* Handle building on FreeBSD for i386 and amd64.
* Add stubs for the other FreeBSD arches.
* Make the file compile cleanly.

* Make this file compile cleanly.
* Add FreeBSD constructor/destructor definitions.

* Add FreeBSD action extensions.
* Get the kernel module file name rather than hard-coding it like on Solaris.
* Use FreeBSD's process library API.
* Handle FreeBSD's different lock types.
* Get the list of loaded providers via a syscall.

* Handle the different ioctl design.
* Make this file compile cleanly.

There is a lot of commented out code here which applies to future work
for userland tracing.

For now we need the basic functions to be in place so that DTrace can
be used for kernel tracing.

Leave out a header file.

Merge FreeBSD include file changes with some code that should have come
in on the vendor branch.

* Use FreeBSD's process library instead of the Solaris one.
* There are a few placeholders in here for which there isn't libproc
support code yet. This is relevent to userland tracing. This set of
commits is designed to get kernel tracing up and running, with the
userland stuff to follow later.

Leave out the hook that Solaris has into it's rtld.

* Add a couple of action definitions for FreeBSD extensions.
* Handle the different ioctl design.
* Add support for the get and set error location.
* Add support for freopen().

* Handle the different ioctl design.
* Make the file compile cleanly.

There are many places in libdtrace where errno can be set. When an
error is returned all the way back to the dtrace app, it's hard to
figure out where that error came from.

Add a couple of functions to get and set the error location which can
be optionally compiled into the library.

Add some error definitions for FreeBSD action extensions.

* Make this file compile cleanly.
* Use FreeBSD's kld syscalls to get kernel module information instead
of using an object file system like Solaris uses.

* Handle the different ioctl design.
* Use FreeBSD's process library API.
* Make the file compile cleanly.

Just leave out a few includes to compile this on FreeBSD.

* Get the maximum number of CPUs via a sysctl.
* Handle the different ioctl design.
* Support the freopen() changes.
* Use functions in FreeBSD's process library rather than the CDDL
library that Solaris has which sits on top of their process file
system and is therefore unsuitable for use on FreeBSD. The libproc
API for FreeBSD is deliberately different to that on Solaris because
Sun wouldn't release the libproc.h header under a BSD license.

* Define YY_INPUT for flex since input() can't be re-defined.
* On FreeBSD define both LITTLE_ENDIAN and BIG_ENDIAN and then set the
BYTE_ORDER to the one we are using. On Solaris they define one or
the other but not both. For us to keep using FreeBSD header files,
we need to use endian definitions the same way we do in pure BSD
code.

Make this compile cleanly.

Handle the different level of indirection between ioctl on SYSV vs BSD.

freopen() on FreeBSD behaves differently to the Solaris one.

* Use a portable POSIX timed wait.
* Handle the different ioctl format.

* Fix a function prototype.
* Allow for the different level of indirection between SYSV's ioctl and BSD's.
* Map a couple of sysconf definitions to one that FreeBSD has.

Simplify for FreeBSD for now, bypassing the multibyte char variables
that Solaris has. We may need to revisit this issue later.

Solaris has code which makes zlib optional, so they have to jump through
some hoops to get the compression functions. On FreeBSD have libz and
can just link against it.

'echo' behaviour differs between SYSV and BSD.

A lot of changes to make this code compile cleanly on FreeBSD.

Use FreeBSD's libdwarf which is BSD licensed instead of the GPL'd one that
is used in Solaris.

We need to be consistent with prototype definitions. It isn't OK to
use 'const' and just override it whenever we feel like it. If we use
it at all, then we need to do it properly.

Add a couple of functions that were useful in getting this code ported.

Make this file compile cleanly.

Apply FreeBSD changes.

On FreeBSD we can't afford to have a build fail simply because there
was some sort of CTF conversion error.

Add a missing include file. We care about function prototype definitions.

Apply FreeBSD changes.

There is an incomplete piece of code in here which requires the process
handling library which is still under development.

* Set the path to perl on FreeBSD.
* Use the FreeBSD shell.
* On FreeBSD the tests run from the OBJDIR, so output files go there
rather than in the source tree like they do on Solaris.
* FreeBSD doesn't need a special path to the compiler.

Set the path to perl on FreeBSD.

Port these test files to work under FreeBSD.

This commit was generated by cvs2svn to compensate for changes in r178528,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r178525,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r178481,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r178479,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r178476,
which included commits to RCS files with non-trunk default branches.

Add FreeBSD IDs to files that originate in FreeBSD.

Remove options MK_LIBKSE and DEFAULT_THREAD_LIB now that we no longer
build libkse. This should fix WITHOUT_LIBTHR builds as a side effect.

The sources covered by Sun's CDDL have been repo copied below the
src/cddl and src/sys/cddl directories per the core@ decision following
the license review.

This change modifies the affected Makefiles to reference the sources
in their new location.

Remove _SOLARIS_C_SOURCE now that it doesn't do anything in FreeBSD
headers. All OpenSolaris compatibility comes via the set of specific
compatibility headers in src/compat/opensolaris and
src/sys/compat/opensolaris.

Remove a couple of definitions which now exist in compatibility headers.

Add more OpenSolaris compatibilty headers.

Tweak the handling of "WITHOUT_LIBPTHREAD". Also remove the accidental
treatment of 'LIBKSE' as an "old style" knob.

Submitted by: ru
Approved by: re(kensmith)

Repo copy libpthreads to libkse.
This introduces the WITHOUT_LIBKSE nob,
and changes WITHOUT_LIBPTHREADS to mean with neither threading libs.
Approved by: re(kensmith)

Don't build bits that depend on the pthreads support if a
system was configured without such support.

Approved by: re (kensmith)

Fixed static linkage (build with -DNO_SHARED).

Approved by: re (kensmith)

Now that we have CDDLed code in the tree, add CDDL license.

Discussed with: core
Approved by: re (kensmith)

Use ioctl() to get correct media size so that we can locate
ZFS labels in the tail of the media.

Approved by: re (kensmith), pjd

- Reduce number of atomic operations needed to be implemented in asm by
implementing some of them using existing ones.
- Allow to compile ZFS on all archs and use atomic operations surrounded
by global mutex on archs we don't have or can't have all atomic
operations needed by ZFS.

Use provider's ident to handle situations when disks are moved around
and show up with different names: first try to open provider using
remembered name and compare its ident, if equal, this is our provider,
if not equal or there is no provider with such name, find provider with
remembered ident and don't care about the name.