Remove svn:mergeinfo carried over from stable/9.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC r266520:

Explicitly link libzfs against libavl as it is done in OpenSolaris
(4543:12bb2876a62e). Without this, some third party applications
may break because the lack of AVL related symbols.

FreeBSD base system are not affected because the FreeBSD ZFS command
line tools were all linked against libavl and thus hide the underlying
issue.

PR: bin/183081
Approved by: re (gjb)

MFC r264835: MFV r264829:

3897 zfs filesystem and snapshot limits

MFC r264669: MFV r264666:

4374 dn_free_ranges should use range_tree_t

MFC r263889: MFV r263887:

3993 zpool(1M) and zfs(1M) should support -p for "list" and "get"
4700 "zpool get" doesn't support -H or -o options

MFC r263459 (MFV 263436-263438):

3947 zpool(1M) references nonexistent zfs-features(5)
4540 zpool(1M) man page doesn't describe "readonly" property
3948 zfs sync=default is not accepted
4611 zfs(1M) still mentions 'send -r' in synopsis
4415 zpool(1M) man page missing "import -m" description
4570 Document dedupditto pool property
4572 Dedup-related documentation additions for zpool and zdb.
1371 Add -D option description to zpool(1M) manpage
4571 Add documentation for -T and interval to "zpool list"

MFC r264851

Eliminated optarg global being used outside of the function which called getopt

Sponsored by: Multiplay

MFC r262542:
Move some files that are identical on i386 and amd64 to an x86 subdirectory
rather than keeping duplicate copies.

MFC r264040:

4248 dtrace(1M) should never create DOF with empty probes section
4249 Only probes from the first DTrace object file will be included

Illumos Revision: 4a20ab41aadcb81c53e72fc65886e964e9add59

Reference:
https://www.illumos.org/issues/4248
https://www.illumos.org/issues/4249

Obtained from: Illumos

MFC r264467:

Take into account when zpool history block grows exceeding 128KB in zpool(8)
and zdb(8) by growing the buffer on demand with a cap of 1GB (specified in
spa_history_create_obj()).

PR: bin/186574
Submitted by: Andrew Childs <lorne cons org nz> (with changes)

MFC r262596:
4478 dtrace_dof_maxsize is far too small

illumos/illumos-gate@d339a29bb4765c4b6883a935cf69b669cd05bca0

MFC r263385:

Remove unused option -r from zpool.

Submitted by: Richard Yao <ryao gentoo org>

MFC r260880 (by kaiw, from projects/elftoolchain):

* Make die_mem_offset() be able to handle DW_AT_data_member_location
attributes generated by Clang 3.4.
* Document how different compilers generate DW_AT_data_member_location
attributes differently.
* Document the quirks about DW_FORM_data[48].

This is a slightly modified version, adapted to work with the old
libdwarf in stable/9 and stable/10. It should fix DTrace on these
branches, when the kernel is compiled with clang 3.4.

Note that you have to build *and* install the CTF tools first, before
building the kernel. Otherwise you can possibly still get error
messages similar to "failed to copy type of 'pr_uid': Type information
is in parent and unavailable", when attempting to run dtrace(1).

Submitted by: kaiw

MFC r262912

cddl/contrib/opensolaris/lib/libuutil/common/uu_avl.c
Fix a memory leak in uu_avl_pool_create: pthread_mutex_init without
a corresponding pthread_mutex_destroy. It shows up, among other
places, when doing "zfs list

MFC r262613:

Merge the projects/clang-sparc64 branch back to head. This brings in
several updates from the llvm and clang trunks to make the sparc64
backend fully functional.

Apart from one patch to sys/sparc64/include/pcpu.h which is still under
discussion, this makes it possible to let clang fully build world and
kernel for sparc64.

Any assistance with testing this on actual sparc64 hardware is greatly
appreciated, as there will unavoidably be bugs left.

Many thanks go to Roman Divacky for his upstream work on getting the
sparc64 backend into shape.

MFC r262985:

Repair a few minor mismerges from r262261 in the clang-sparc64 project
branch. This is also to minimize differences with upstream.

MFC r260183:

MFV r260154 + 260182:

4369 implement zfs bookmarks
4368 zfs send filesystems from readonly pools

Illumos/illumos-gate@78f171005391b928aaf1642b3206c534ed644332

MFC r254755: MFV r254748:

Fix memory leak in libzfs's iter_dependents_cb().

Illumos ZFS issues:
4061 libzfs: memory leak in iter_dependents_cb()

MFC r259850: MFV r258384:

2583 Add -p (parsable) option to zfs list

illumos/illumos-gate@43d68d68c1ce08fb35026bebfb141af422e7082e

MFC r256999 (smh):

Added support for the 'zfs list -t snap' and 'zfs snap' aliases which are
available under Oracle Solaris 11.

This includes an update to the ZFS(8) man page to reflect all the
available alias (snap, umount, and recv).

Initial changes obtained from ZFS On Linux + fixes for man page and cmd
help:
https://github.com/zfsonlinux/zfs/commit/10b75496bb0cb7a7b8146c263164adc37f1d176a
https://github.com/zfsonlinux/zfs/commit/cf81b00a73fe47fdb21586ac1cc179b734540973

Obtained from: https://github.com/zfsonlinux/zfs

MFC r260150: MFV r259170:

4370 avoid transmitting holes during zfs send

4371 DMU code clean up

illumos/illumos-gate@43466aae47bfcd2ad9bf501faec8e75c08095e4f

NOTE: Make sure the boot code is updated if a zpool upgrade is
done on boot zpool.

MFC r260138: MFV r242733:

3306 zdb should be able to issue reads in parallel
3321 'zpool reopen' command should be documented in the man page
and help message

illumos/illumos-gate@31d7e8fa33fae995f558673adb22641b5aa8b6e1

FreeBSD porting notes: the kernel part of this changeset depends
on Solaris buf(9S) interfaces and are not really applicable for
our use. vdev_disk.c is patched as-is to reduce diverge from
upstream, but vdev_file.c is left intact.

MFC r259813 + r259816: MFV r258374:

4171 clean up spa_feature_*() interfaces

4172 implement extensible_dataset feature for use by other zpool
features

illumos/illumos-gate@2acef22db7808606888f8f92715629ff3ba555b9

MFC r259811: MFV r258373:

4168 ztest assertion failure in dbuf_undirty

4169 verbatim import causes zdb to segfa
4170 zhack leaves pool in ACTIVE state

illumos/illumos-gate@7fdd916c474ea52896c671bbe7b56ba34a1ca132

MFC r254587: MFV r254421:

Illumos ZFS issues:
3996 want a libzfs_core API to rollback to latest snapshot

MFC r262577: MFV r262570:

4626 libzfs memleak in zpool_in_use()

MFC r261774 by feld:

Add caveat to zpool manpage indicating that we do not automatically activate
hot spares. This should be MFC'd to all STABLE branches.

Upon the availability of zfsd, the zpool manpage on relevant branches should
be updated to remove this caveat and document hot spare's reliance on zfsd.

Requested by: feld

MFC r259168:
Don't even try to read vdev labels from devices smaller then SPA_MINDEVSIZE
(64MB). Even if we would find one somehow, ZFS kernel code rejects such
devices. It is funny to look on attempts to read 4 256K vdev labels from
1.44MB floppy, though it is not very practical and quite slow.

MFC r253819: MFV r253781 + r253871: 3894 zfs should not allow snapshot of inconsistent dataset

MFC slacker: delphij

MFC r260703: zinject must use ioctl(2) compatibility wrapper

MFC r260811: zdb -R: do not treat numeric parameters to a flag as more flags

MFC r260703: zinject must use ioctl(2) compatibility wrapper

MFC r254591,255753: Enhance the ZFS vdev layer to maintain both a
logical and a physical minimum allocation size for devices

Missed userland bits.

MFC r258717: MFV r258371,r258372: 4101 metaslab_debug should allow for
fine-grained control

MFC r255750: MFV r254750: Add support of Illumos dumps on zvol over RAID-Z.

Note that this only adds the features. FreeBSD would
still need more work to support dumping on zvols.

MFC slacker: delphij

MFC r254112: MFV r254079: multiple ZFS issues

MFC r254077: MFV r254071: Fix a regression introduced by fix for Illumos bug #3834

MFC r261122: dtrace: remove unexplained 16MB limitation from dt_alloc/dt_zalloc

MFC r260051:
When clearing relocations to __dtrace* symbols, handle both SHT_REL and
SHT_RELA sections properly instead of assuming that the relocation section
is of type SHT_REL.

MFC r253720: Use kern_ioctl() rather than ioctl() for testing the FBT provider

MFC r251237: Port the SDT test

MFC r258632,258704: MFV r255255: 4045 zfs write throttle & i/o scheduler
performance work

Note a change in dmu_tx_delay: pause_sbt is not available in this
branch.

Sponsored by: HybridCluster [merge]

MFC r254074: MFV r254070: Merge vendor bugfix for ZFS test suite that
triggers false positives

MFC r255437: MFV r247844 (illumos-gate 13975:ef6409bc370f)

Note that a different kind of cv_timedwait_hires shim is provided in
this branch because cv_timedwait_sbt is not available for better
emulation.

Sponsored by: HybridCluster [merge]

MFC r258630: 734 taskq_dispatch_prealloc() desired

MFC r254941;

Merge a couple of CTF fixes from illumos:

2942 CTF tools need to handle files which legitimately lack data
2978 ctfconvert still needs to ignore legitimately dataless files on SPARC

Illumos Revisions: 13745:6b3106b4250f
13754:7231b684c18b

Reference:

https://www.illumos.org/issues/2942
https://www.illumos.org/issues/2978

Obtained from: Illumos

MFC r249426;

Merge change from illumos:

3519 DTrace fails to resolve const types from fbt
3520 dtrace internal error -- token type 316 is not a valid D
compilation token
3521 clean up dtrace unit tests

Illumos Revision: e98f46c

Reference:
https://www.illumos.org/issues/3519
https://www.illumos.org/issues/3520
https://www.illumos.org/issues/3521

Tested by: Fabian Keil
Obtained from: Illumos

MFC r250574, r250812, r253725;

DTrace: option for time-ordered output

Merge changes from illumos:

3021 option for time-ordered output from dtrace(1M)
3022 DTrace: keys should not affect the sort order when sorting by value
3023 it should be possible to dereference dynamic variables
3024 D integer narrowing needs some work
3025 register leak in D code generation

This brings yet another feature implemented in upstream DTrace.
A complete description is available here:
http://dtrace.org/blogs/ahl/2012/07/28/my-new-dtrace-favorite/

This change bumps the DT_VERS_* number to 1.9.1 in
accordance to what is done in illumos.

Special thanks to Mark Johnston for fixes and testing.

Illumos Revisions: 13758:23432da34147

Reference:
https://www.illumos.org/issues/3021
https://www.illumos.org/issues/3022
https://www.illumos.org/issues/3023
https://www.illumos.org/issues/3024
https://www.illumos.org/issues/3025
https://www.illumos.org/issues/1694

Obtained from: Illumos

MFC r253818:

Fix zfs send -D hang after processing requiring a CTRL+C to interrupt due to
pthread_join prior to fd close.

Approved by: re (delphij)

MFC r252732:
Fix misleading or remove irrelevant illumos messages and manpage references
in the zfs command.

PR: bin/178996
Submitted by: Peter Schaefer <peter.schaefer@wilhelmheinrichs.de>

MFC r252218:

Diff reduction against Illumos, no real change to code itself.

This marks vendor branch revision 252213 as merged, the actual code was
committed in r245479.

MFC r251646 + r252219:

MFV r251644:

Poor ZFS send / receive performance due to snapshot
hold / release processing (by smh@)

Illumos ZFS issues:
3740 Poor ZFS send / receive performance due to snapshot
hold / release processing

MFV r252215:

Restore a previous behavior before r251646, where when destructing
ZFS snapshot, the ioctl would return ENOENT when it hit any of
them in the errlist (the new behavior was only return ENOENT when
all returns error).

Illumos ZFS issues:
3829 fix for 3740 changed behavior of zfs destroy/hold/release ioctl

MFC r251635: illumos #3747 txg commit callbacks don't work

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c:
Fix commit callbacks by moving them to the task's list.
Previously, list_move_tail() returned without doing anything because
the task list was passed as the source rather than destination.

cddl/contrib/opensolaris/cmd/ztest/ztest.c:
Check the commit callback threshold correctly.

Submitted by: will
Reviewed by: Matthew Ahrens <mahrens@delphix.com>,
Christopher Siden <christopher.siden@delphix.com>
Sponsored by: Spectra Logic

MFC r251634: illumos #3745 zpool create should treat -O mountpoint and -m the same

cddl/contrib/opensolaris/cmd/zpool/zpool_main.c: (change 644608)
This allows specifying a mountpoint using the latter form and having
its value checked and used as it would be using the former form.

As a consequence of this change:

1. The mountpoint property is set in the fsprops nvlist prior
to creating the pool, rather than being set after creating
the pool. To me, this is the proper approach, since it
avoids creating the pool if the mountpoint setting would
cause the command to fail.
2. The mountpoint property, unlike all others, can be specified
more than once. Only the last setting takes effect. This
is to avoid breaking potential existing users that specify
-m more than once.

Submitted by: will

cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c
Fix "zpool create -R <whatever> -m <whatever>". Ever since
change 644608, this has been broken. The problem is that some
old code in libzfs_pool.c would force a pool's mountpoint to
"/" when creating a pool with an altroot. That probably
implemented some old policy decision regarding altroots, but it
conflicts with the current manpage. It also had no effect
until 644608, because the zpool command would _always_ change
the pool's mountpoint after creating it. The solution is to
delete the old code from libzfs_pool.c.

Submitted by: asomers

Reviewed by: Matthew Ahrens <mahrens@delphix.com>,
Christopher Siden <christopher.siden@delphix.com>
Sponsored by: Spectra Logic

MFC r251629: 3741 zfs needs better comments

Embellish the comments in various components of ZFS. Move some comments
around closer to what they describe. Specifically, answer the questions:

- What are some of the edge cases of the dbuf state machine?
- What does a txg quiesce do?
- When does the DMU notify threads waiting on txg's that they may
proceed?
- How do the calculations for RAIDZ map allocations work?
- What process do the RAIDZ I/O start and done callbacks follow?

While here, adjust the function prototype of dmu_zfetch.c:dmu_zfetch_colinear()
to match its comment which describes its return as a boolean.

Submitted by: asomers, gibbs, will
Reviewed by: Matthew Ahrens <mahrens@delphix.com>,
Eric Schrock <eric.schrock@delphix.com>,
Christopher Siden <christopher.siden@delphix.com>
Sponsored by: Spectra Logic

MFC r252056:
Fix destroyed ZFS pools failing to import

MFC r252059:
Fix ZFS zpool freeze (debug command) not processing due to invalid ioctl call syntax.

MFC r252060:
Fix intermittent ZFS lock panic

MFC r252061:
Switch ZFS mutex_owner macro to use sx_xholder as its now exported via sx.h

MFC: 248848

Commit a patch that fixes a problem in the #pragma statement when searching
for and loading dependent modules. This addresses a bug seen with
io.d where it was being doubly included.

PR: 171678
Submitted by: Mark Johnston

MFC: 235380,235454,235455,235456,235457,235613,235614,235628,236567,
238125,238535,238660,238979,239972

Bring all changes to the DTrace Toolkit into FreeBSD 9.

Added ZFS TRIM support which is enabled by default. To disable
ZFS TRIM support set vfs.zfs.trim.enabled=0 in loader.conf.

Creating new ZFS pools and adding new devices to existing pools
first performs a full device level TRIM which can take a significant
amount of time. The sysctl vfs.zfs.vdev.trim_on_init can be set to 0
to disable this behaviour.

ZFS TRIM requires the underlying device support BIO_DELETE which
is currently provided by methods such as ATA TRIM and SCSI UNMAP
via CAM, which are typically supported by SSD's.

Stats for ZFS TRIM can be monitored by looking at the sysctl's
under kstat.zfs.misc.zio_trim.

MFC r240868: Add TRIM support
MFC r244155: Renamed zfs trim stats
MFC r244187: Upgrade TRIM free request sizes optimisation
MFC r244188: Added vfs.zfs.vdev.trim_on_init sysctl
MFC r248572: Add TRIM support for L2ARC
MFC r248574: Improve TXG handling in the TRIM module
MFC r248575: TRIM cache devices based on time instead of TXGs
MFC r248576: Names the ZFS TRIM thread
MFC r248577: Optimisation of TRIM processing
MFC r248602: Fix for building libzpool under i386
MFC r249921: Enabled ZFS TRIM by default

MFC r244194:
Fix zfs receive errors caused by snapshot replication being processed in
random order.

PR: kern/172259

MFC r245479:
Report pools which have a removed l2cache disk under zpool status -x

MFC r250337

Work around the implementation of LIBRARIES_ONLY. It causes drti.o to
not be installed in /usr/lib32 on systems with compat-32 support.

This fix has two parts. First, the build is forced by linking drti.o
into a dummy internal library. Second, the object file is installed
manually in the LIBRARIES_ONLY case.

MFC r249526, r249600:
DTrace: NFS translators should be split into client/server pieces

Merge change from illumos:

1731 DTrace NFS translators should be split into client/server pieces

Include Changes from OpenSolaris:
PSARC 2008/050 DTrace NFS v3 Provider
6696397 NFS v3 provider reports all UDP clients as 0.0.0.0
6711844 assert: vp-->v_shrlocks

Illumos Revision: 13523:6763769941d2

This code seems to be currently unused on FreeBSD.

Reference:
https://www.illumos.org/issues/1731

Obtained from: Illumos

MFC r249526:
DTrace: print() should try to resolve function pointers

Merge changes from illumos:

3675 DTrace print() should try to resolve function pointers
3676 dt_print_enum hardcodes a value of zero

Illumos Revision: b1fa6326238973aeaf12c34fcda75985b6c06be1

Reference:
https://www.illumos.org/issues/3675
https://www.illumos.org/issues/3676

Obtained from: Illumos

MFC r248983:
Dtrace: enablings on defunct providers prevent providers from unregistering

Merge change from illumos:

1368 enablings on defunct providers prevent providers from unregistering

Illumos Revision: 13430:8e6add739e38

Reference:
https://www.illumos.org/issues/1368

Reviewed by: gnn
Tested by: Fabian Keil
Obtained from: Illumos

Merge r246619 from head:
Correct spelling of "daemon". No .Dd bump.
Noticed by: Nathan Rich <Nathan.Rich dynastysystems com>

MFC r249883:
Respect the enoent_ok flag if reporting error for holding an non-existing
snapshot.

Related illumos ZFS issue:
3699 zfs hold or release of a non-existent snapshot does not output error

Reported by: Steven Hartland <smh@FreeBSD.org>

MFC r248690, r248706, 248708, r248752:
Dtrace: merge new functions from Illumos.

This covers illumos issues:

1455 DTrace tracemem() should take an optional size argument
1451 DTrace needs toupper()/tolower() subroutines
1457 lltostr() D subroutine should take an optional base
1694 Add type-aware print() action
3511 dtrace.c erroneously checks for memory alignment on amd64

This includes an enhanced print() described here:

http://dtrace.org/blogs/eschrock/2011/10/26/your-mdb-fell-into-my-dtrace/

This change bumps the DT_VERS_* number to 1.9.0 in
accordance to what is done in illumos.

Illumos Revisions: 13457:571b0355c2e3
13458:5e394d8db762
13459:c3454574dd1a
13501:c3a7090dbc16
13483:f413e6c5d297

Reference:
https://www.illumos.org/issues/1455
https://www.illumos.org/issues/1451
https://www.illumos.org/issues/1457
https://www.illumos.org/issues/1560
https://www.illumos.org/issues/1694
https://www.illumos.org/issues/3511

Tested by: Fabian Keil
Obtained from: Illumos

MFC 248571,248976,249004,249042,249188,249195-249196,249206,249207,249319,
249326,249356-249357

Merge libzfs_core and other ZFS bugfixes and improvements.

MFC r248571:
MFV 238590, 238592:
In the first zfs ioctl restructuring phase, the libzfs_core library was
introduced. It is a new thin library that wraps around kernel ioctl's.
The idea is to provide a forward-compatible way of dealing with new
features. Arguments are passed in nvlists and not random zfs_cmd fields,
new-style ioctls are logged to pool history using a new method of
history logging.

http://blog.delphix.com/matt/2012/01/17/the-future-of-libzfs/

MFV 247580 [1]:
To address issues of several deadlocks and race conditions the locking
code around dsl_dataset was rewritten and the interface to synctasks
was changed.

User-Visible Changes:
"zfs snapshot" can create more arbitrary snapshots at once (atomically)
"zfs destroy" destroys multiple snapshots at once
"zfs recv" has improved performance

Backward Compatibility:
I have extended the compatibility layer to support full backward
compatibility by remapping or rewriting the responsible ioctl arguments.
Old utilities are fully supported by the new kernel module.

Forward Compatibility:
New utilities work with old kernels with the following restrictions:
- creating, destroying, holding and releasing of multiple snapshots
at once is not supported, this includes recursive (-r) commands

Illumos ZFS issues:
2882 implement libzfs_core
2900 "zfs snapshot" should be able to create multiple,
arbitrary snapshots at once
3464 zfs synctask code needs restructuring

MFC r248976:
Call dmu_snapshot_list_next() in zvol.c with dsl_pool_config lock held

MFC r249004:
Do not check against uninitialized rc and comment out vendor code

MFC r249042:
Fix possible pool hold leak in dmu_send_impl()

Illumos ZFS issues:
3645 dmu_send_impl: possibilty of pool hold leak

MFC r249188:
Import vendor change to reduce diff, no effect on FreeBSD.

Illumos ZFS issues:
3517 importing pool with autoreplace=on and "hole" vdevs crashes
syseventd

MFC r249195:
Merge change from vendor to reduce diff only.
ZFS dtrace probes are not supported on FreeBSD yet.

Illumos ZFS issues:
3598 want to dtrace when errors are generated in zfs

MFC r249196:
Provide a fix for kernel panic if receiving recursive deduplicated
streams. Problem reported to vendor.

Illumos ZFS issues:
3692 Panic on zfs receive of a recursive deduplicated stream

MFC r249206:
Merge vendor change - modify time processing in deadman thread.

Illumos ZFS issues:
3618 ::zio dcmd does not show timestamp data

MFC r249207:
Allow zdb to output a histogram of compressed block sizes.

Illumos ZFS issues:
3641 want a histogram of compressed block sizes

MFC r249319:
ZFS expects a copyout of zfs_cmd_t on an ioctl error. Our sys_ioctl()
doesn't copyout in this case.

To solve this a new struct zfs_iocparm_t is introduced consisting of:
- zfs_ioctl_version (future backwards compatibility purposes)
- user space pointer to zfs_cmd_t (copyin and copyout)
- size of zfs_cmd_t (verification purposes)

The copyin and copyout of zfs_cmd_t is now done the illumos (vendor) way
what makes porting of new changes easier and ensures correct behavior if
returning an error.

MFC r249326:
Cast (void *)(uintptr_t) on copyout and copyin of zfs_iocparm_t.zfs_cmd

MFC r249356:
Merge bugfixes accepted and integrated by vendor. Underlying problems
have been reported by us and fixed in r240942 and r249196.

Illumos ZFS issues:
3645 dmu_send_impl: possibilty of pool hold leak
3692 Panic on zfs receive of a recursive deduplicated stream

MFC r249357:
Fix libzfs to report error instead of returning zero if trying to hold or
release a non-existing snapshot of a existing dataset. In recursive case
error is reported if no snapshots with the requested name have been found.

Illumos ZFS issues:
3699 zfs hold or release of a non-existent snapshot does not output
error

MFC r248991:

Follow up to r247960 and rr247960 by also amending ctfmerge. For the
only other case where STT_FILE symbols are used, in symit_next() in
cddl/contrib/opensolaris/tools/ctf/cvt/input.c, save the basename of the
symbol, instead of the full pathname.

Reported by: avg
Tested by: avg, jimharris

MFC r240870 (pjd):
It is possible to recursively destroy snapshots even if the snapshot
doesn't exist on a dataset we are starting from. For example if we
have the following configuration:

tank
tank/foo
tank/foo@snap
tank/bar
tank/bar@snap

We can execute:

# zfs destroy -t tank@snap

eventhough tank@snap doesn't exit.

Unfortunately it is not possible to do the same with recursive rename:

# zfs rename -r tank@snap tank@pans
cannot open 'tank@snap': dataset does not exist

...until now. This change allows to recursively rename snapshots even if
snapshot doesn't exist on the starting dataset.

Sponsored by: rsync.net

MFC r242845, r247592:

MFC r242845 (delphij):
Illumos r13840:97fd5cdf328a:
3145 single-copy arc
3212 ztest: race condition between vdev_online() and spa_vdev_remove()

Illumos r13849:3468a95b27cd:
3258 ztest's use of file descriptors is unstable

MFC r247592 (delphij):
Import a fix tighten assertion on SPA versions from vendor (Illumos).

Illumos ZFS issue:
3543 Feature flags causes assertion in spa.c to miss certain cases

Approved by: delphij

MFC r247187,247265,247348,247398,247540,247585,247852,248265,248267
Merge various ZFS improvements and bugfixes

MFC r247187:
Import vendor change to avoid "unitialized variable" warnings.

Illumos ZFS issues:
3522 zfs module should not allow uninitialized variables

MFC r247265:
Merge the ZFS I/O deadman thread from vendor (illumos).
This feature panics the system on hanging ZFS I/O, helps debugging
and resumes failed service.

The panic behavior can be controlled with the loader-only tunables:
vfs.zfs.deadman_enabled (enable or disable panic on stalled ZFS I/O)
vfs.zfs.deadman_synctime (expiration time for stalled ZFS I/O)

By default, ZFS I/O deadman is enabled by default on amd64 and i386
excluding virtual guest machines.

MFC r247348:
Be more verbose on ZFS deadman I/O panic
Patch suggested upstream.

MFC r247398:
Import metaslab_sync() speedup from vendor (illumos).

Illumos ZFS issues:
3552 condensing one space map burns 3 seconds of CPU in spa_sync() thread
3564 spa_sync() spends 5-10% of its time in metaslab_sync() (when not
condensing)
3578 transferring the freed map to the defer map should be constant time
3579 ztest trips assertion in metaslab_weight()

MFC r247540:
Fix the zfs_ioctl compat layer to support zfs_cmd size change introduced
in r247265 (ZFS deadman thread). Both new utilities now support the old
kernel and new kernel properly detects old utilities.

For future backwards compatibility, the vfs.zfs.version.ioctl read-only
sysctl has been introduced. With this sysctl zfs utilities will be able
to detect the ioctl interface version of the currently loaded zfs module.

MFC r247585:
Merge new read-only zfs properties from vendor (illumos)

Illumos ZFS issues:
3588 provide zfs properties for logical (uncompressed) space used and
referenced

MFC r247852:
Import ZFS bpobj bugfix from vendor.

Illumos ZFS issues:
3603 panic from bpobj_enqueue_subobj()
3604 zdb should print bpobjs more verbosely

MFC r248265:
Update zfs.8 manpage date (missing in r247585)

MFC r248267:
Import minor ZFS changes from vendor

Illumos ZFS issues:
3604 zdb should print bpobjs more verbosely (fix zdb hang)
3606 zpool status -x shouldn't warn about old on-disk format

MFC r245561,245885:

Add a new LIBRARIES_ONLY make variable to disable the build and install
of files other than the actual libraries.

Use LIBRARIES_ONLY to supress the inclusion of files in the lib32
distribution that are duplicates of files in base.

Sponsored by: DARPA, AFRL
Reviewed by: emaste

MFC r247960:

Make ctfconvert work correctly on clang-compiled object files. Clang
puts the full original source filename in the STT_FILE entry of the ELF
symbol table, while gcc saves only the basename.

Since the DWARF DW_AT_name attribute contains the full source filename,
both for clang and gcc, ctfconvert takes just the basename of it, for
matching with the STT_FILE entry. So when attempting to match with such
an entry, use its basename, if necessary.

Reported by: avg

MFC r247962:

Fix error in r247960: actually assign the basename to match.iim_file.

Pointed out by: avg
Pointy hat to: dim

MFC r247048.

Orphaned processes that are being traced are killed by the
kernel. Properly restore, continue, and detach from processes
being DTraced when DTrace exits with an error so the program
being inspected is not terminated.

cddl/contrib/opensolaris/cmd/dtrace/dtrace.c:
In fatal(), the generic error handler, close the DTrace
handle as is done in the "probe/script" error handler
dfatal(). fatal() can be invoked after DTrace attaches
to processes (e.g. a script specified by command line
argument can't be found) and closing the handle will
release them.

Submitted by: Spectra Logic Corporation
Reviewed by: rpaulo, gnn

MFC r246631,246651,246666,246675,246678,246688:
Merge various ZFS bugfixes

MFC r246631:
Import vendor bugfixes

Illumos ZFS issues:
3422 zpool create/syseventd race yield non-importable pool
3425 first write to a new zvol can fail with EFBIG

MFC r246651:
Import minor type change in refcount.h header from vendor (illumos).

MFC r246666:
Import vendor ZFS bugfix fixing a problem in arc_read().

Illumos ZFS issues:
3498 panic in arc_read(): !refcount_is_zero(&pbuf->b_hdr->b_refcnt)

MFC r246675:
Add tunable to allow block allocation on degraded vdevs.

Illumos ZFS issues:
3507 Tunable to allow block allocation even on degraded vdevs

MFC r246678:
Import vendor bugfixes regarding SA rounding, header size and layout.
This was already partially fixed by avg.

Illumos ZFS issues:
3512 rounding discrepancy in sa_find_sizes()
3513 mismatch between SA header size and layout

MFC r246688 [1]:
Merge zfs_ioctl.c code that should have been merged together with ZFS v28.
Fixes several problems if working with read-only pools.

Changed code originaly introduced in onnv-gate 13061:bda0decf867b
Contains changes up to illumos-gate 13700:4bc0783f6064

PR: kern/175897 [1]
Suggested by: avg [1]

MFC r246586,246587,246619,246624,246768,246808:

LZ4 compression support in ZFS.

(Note: sys/conf/files change omitted from this changeset).

MFC: r240696, r240698, r240740

Allow zfs jail and zfs unjail to use both jailnames and jailid

MFC r245264:

The current ZFS code expects ddt_zap_count to always succeed by asserting
the underlying zap_count() to return no errors. However, it is possible
that the pool reaches to such a state where zap_count would return error,
leading to panics when a pool is imported.

This commit changes the ddt_zap_count to return error returned from
zap_count and handle the error appropriately. With this change, it's now
possible to let zpool rollback damaged transaction groups and import the
pool.

Obtained from: ZFS on Linux github (e8fd45a0f975c6b8ae8cd644714fc21f14fac2bf)

Clean some 'svn:executable' properties in the tree.

Submitted by: Christoph Mallon

While here, merge some other mergeinfo properties that
were left behind from my commits

/head/include:r241008,241141,241181
/head/contrib/gcc:r244776,244792
/head/cddl:r238457,238509,238558

MFC r243102:
Sort SEE ALSO xrefs by the manual section number and add a missing comma.

MFC r244246:

illumos-gate 13894:f4af77f6bbd2
3397 zdb <pool> <objnum> output is too verbose
3398 zdb can't dump feature flags zap objects

References:
https://www.illumos.org/issues/3397
https://www.illumos.org/issues/3398

MFC: 238979

When we return with an error we cannot unlock the mutex, because
it's been freed. Protect against that, hopefully unlikely, case.

Reviewed by: rpaulo

MFC recent ZFS changes from illumos:
243503, 243524, 243525, 243560, 243561

MFC r243503:
Illumos 13879:4eac7a87eff2
3329 spa_sync() spends 10-20% of its time in spa_free_sync_cb()
3330 space_seg_t should have its own kmem_cache
3331 deferred frees should happen after sync_pass 1
3335 make SYNC_PASS_* constants tunable

New loader-only tunables:
vfs.zfs.sync_pass_deferred_free
vfs.zfs.sync_pass_dont_compress
vfs.zfs.sync_pass_rewrite

References:
https://www.illumos.org/issues/3329
https://www.illumos.org/issues/3330
https://www.illumos.org/issues/3331
https://www.illumos.org/issues/3335

MFC r243524:
Import the zio nop-write improvement from Illumos. To reduce I/O,
nop-write omits overwriting data if the checksum (cryptographically
secure) of new data matches the checksum of existing data.
It also saves space if snapshots are in use.

It currently works only on datasets with enabled compression, disabled
deduplication and sha256 checksums.

IllumOS 13887:196932ec9e6a and 13888:7204b3392a58
3236 zio nop-write

References:
https://www.illumos.org/issues/3236

MFC r243525:
Add loader(8) tunable to enable/disable nopwrite functionality:
vfs.zfs.nopwrite_enabled

MFC r243560:
Introduce a new dataset aclmode setting "restricted" to protect ACL's
being destroyed or corrupted by a drive-by chmod.

illumos-gate 13889:a67716f16746
3254 add support in zfs for aclmode=restricted

MFC r243561:
Update manpage dates in zfs.8 and zpool.8

MFC: 238366, 240924

Initial commit of an I/O provider for DTrace on FreeBSD.

These probes are most useful when looking into the structures
they provide, which are listed in io.d. For example:

dtrace -n 'io:kernel::start { printf("%d\n", args[0]->bio_bcount); }'

Note that the I/O systems in FreeBSD and Solaris/Illumos are sufficiently
different that there is not a 1:1 mapping from scripts that work
with one to the other.

This commit includes the fix so that our probes use "kernel"
instead of the Solaris specific "genunix"

Merge ZFS feature flags support and related bugfixes:
236884, 237001, 237119, 237458, 237972, 238113, 238391, 238422, 238926,
238950, 238951, 239389, 239394, 239620, 239774, 239953, 239958, 239967,
239968, 240063, 240133, 240153, 240303, 240345, 240415, 240955, 241655,
243014, 243505, 243506

MFC r236884:
Introduce "feature flags" for ZFS pools (bump SPA version to 5000).
Add first feature "com.delphix:async_destroy" (asynchronous destroy
of ZFS datasets).
Implement features support in ZFS boot code.

Illumos revisions merged:
13700:2889e2596bd6
13701:1949b688d5fb
2619 asynchronous destruction of ZFS file systems
2747 SPA versioning with zfs feature flags

References:
https://www.illumos.org/issues/2619
https://www.illumos.org/issues/2747

MFC r237001:
Fix ZFS boot with pre-features pools (version <= 28) broken in r236884

MFC r237119 [1]:
Do not remount ZFS dataset if changing canmount property to "on" and
dataset is already mounted.

MFC r237458:
Import Illumos revision 13736:9f1d48e1681f
2901 ZFS receive fails for exabyte sparse files

References:
https://www.illumos.org/issues/2901

MFC r237972:
Expose scrub and resilver tunables.
This allows the user to tune the priority trade-off between scrub/resilver
and other ZFS I/O.

MFC r238113 (pjd):
vdev_io_done stage is not used for ioctls.

MFC r238391:
Change behavior introduced in r237119 to vendor solution

References:
https://www.illumos.org/issues/2883

MFC r238422:
Merge illumos commit 13749:df4cd82e2b60

1796 "ZFS HOLD" should not be used when doing "ZFS SEND" froma read-only pool
2871 support for __ZFS_POOL_RESTRICT used by ZFS test suite
2903 zfs destroy -d does not work
2957 zfs destroy -R/r sometimes fails when removing defer-destroyed snapshot

References:
https://www.illumos.org/issues/1796
https://www.illumos.org/issues/2871
https://www.illumos.org/issues/2903
https://www.illumos.org/issues/2957

MFC r238926:
Partial MFV (illumos-gate 13753:2aba784c276b)
2762 zpool command should have better support for feature flags

References:
https://www.illumos.org/issues/2762

MFC r238950:
Fix reporting of root pool upgrade notice.

MFC r238951:
Fix wrong indent according to style(9)

MFC r239389:
Backport fix for vendor issue #3085
3085 zfs diff panics, then panics in a loop on booting

References:
https://www.illumos.org/issues/3085

MFC r239394:
Update zfs(8) manpage with illumos version of "zfs diff"

Illumos issue:
2399 zfs manual page does not document use of "zfs diff"

References:
https://www.illumos.org/issues/2399

MFC r239620 [2]:
Merge recent vendor changes:
3086 unnecessarily setting DS_FLAG_INCONSISTENT on async destroyed datasets
3090 vdev_reopen() during reguid causes vdev to be treated as corrupt
3102 vdev_uberblock_load() and vdev_validate() may read the wrong label

Referenes:
https://www.illumos.org/issues/3086
https://www.illumos.org/issues/3090
https://www.illumos.org/issues/3102

MFC r239774:
Merge recent vendor changes:
3100 zvol rename fails with EBUSY when dirty
3104 eliminate empty bpobjs
3120 zinject hangs in zfsdev_ioctl() due to uninitialized zc

References:
https://www.illumos.org/issues/3100
https://www.illumos.org/issues/3104
https://www.illumos.org/issues/3120

MFC r239953 (joel):
Mdoc fixes.

MFC r239958 (joel):
Minor mdoc fixes.

MFC r239967 (joel):
Mdoc fixes.

MFC r239968 (joel):
Remove trailing whitespace.

MFC r240063 (gjb):
Add myself to copyright sections, per CDDL license.

MFC r240133:
Merge recent vendor changes and sync code:
1862 incremental zfs receive fails for sparse file > 8PB
3112 ztest does not honor ZFS_DEBUG
3122 zfs destroy filesystem should prefetch blocks
3129 'zpool reopen' restarts resilvers
3130 ztest failure: Assertion failed:
0 == dmu_objset_destroy(name, B_FALSE) (0x0 == 0x10)

References:
https://www.illumos.org/issues/1862
https://www.illumos.org/issues/3112
https://www.illumos.org/issues/3122
https://www.illumos.org/issues/3129
https://www.illumos.org/issues/3130

MFC r240153 (gjb) [3]:
Typo fix and minor word swap.

MFC r240303:
Add assfail() and assfail3() to the opensolaris module.
Remove obsoleted intermediate cddl/compat/opensolaris/sys/debug.h.

MFC r240345 (avg):
zfs: fix sa_modify_attrs handling of variable-sized attributes

- skip length_idx index for a replaced variable-sized attribute
- skip length_idx index for a removed variable-sized attribute
- also re-arranged code to make sure that length_idx is always
incremented for variable-sized attributes
- additionally add an assertion that the number of actually produced
attributes is the same as the expected number of resulting
attributes

MFC r240415:
Merge recent zfs vendor changes, sync code and adjust userland DEBUG.

Illumos issued covered:
1884 Empty "used" field for zfs *space commands
3006 VERIFY[S,U,P] and ASSERT[S,U,P] frequently check if first argument
is zero
3028 zfs {group,user}space -n prints (null) instead of numeric GID/UID
3048 zfs {user,group}space [-s|-S] is broken
3049 zfs {user,group}space -t doesn't really filter the results
3060 zfs {user,group}space -H output isn't tab-delimited
3061 zfs {user,group}space -o doesn't use specified fields order
3064 usr/src/cmd/zpool/zpool_main.c misspells "successful"
3093 zfs {user,group}space's -i is noop
3098 zfs userspace/groupspace fail without saying why when run as non-root

References:
https://www.illumos.org/issues/ + [issue_id]

MFC r240955 (partial):
Merge recent vendor changes in ZFS.

Illumos issued covered:
3139 zdb dies when it tries to determine path of unlinked file
3189 kernel panic in ZFS test suite during hotspare_onoffline_004_neg
3208 moving zpool cross-endian results in incorrect user/group accounting

References:
https://www.illumos.org/issues/ + [issue_id]

MFC r241655:
Add missing initialization for do_prefix.
Corrects porting error in r238391

Vendor issue and changeset reference:
2883 changing "canmount" property to "on" should not always remount dataset
https://www.illumos.org/issues/2883
Changeset 13743:95aba6e49b9f

MFC r243014:
Move zpool-features manual page from section 5 to section 7
and fix references

Reported by: pluknet

MFC r243505:
Illumos 13886:e3261d03efbf

3349 zpool upgrade -V bumps the on disk version number, but leaves
the in core version

References:
https://www.illumos.org/issues/3349

MFC r243506:
zfs sha256 checksum is missing in zfs.8 manpage

PR: kern/167905 [1], kern/170912 [2], kern/170914 [2], doc/171356 [3]

MFC r240955 (partial):
Remove unsupported "-r" option from zfs send.

Illumos issued covered:
2811 missing implementation: zfs send -r

PR: kern/171761

MFC r226940 (rmh):

Fix a few gratuitous library dependencies. Some of the ZFS utilities
are linked with libraries they don't use:

- zinject doesn't use libavl
- ztest doesn't use libz
- zdb uses neither libavl nor libz
- zfs uses neither libbsdxml nor libm, nor libsbuf
- zpool uses neither libbsdxml nor libm, nor libsbuf

In addition, libzfs needs libm because it uses pow(), however it isn't
linked with -lm. This went unnoticed because all its users had -lm before.

MFC r240153:
- Typo fix and minor word swap.

PR: 171356

MFC 238457:

1458 D compiler fails to generate error on sizeof() an undefined struct

Reference:
https://www.illumos.org/issues/1458

Obtained from: Illumos

MFC r238558:

Dtrace: improve handling of library paths.

Merge changes from illumos

906 dtrace depends_on pragma should search all library paths, not just the
current one

949 dtrace should only include the first instance of a library found on
its library path

Obtained from: Illumos

MFC r238509:

Use libc's strndup() instead of Dtrace's reimplementation.

Corresponds partially to OpenSolaris change:
PSARC 2010/299 GNU/Linux/BSD compatibility functions
6901783 strndup would be nice.

MFC r239216, r239217:

r239216:
- Remove a leading space that breaks rendering

r239217:
- Fix source dataset snapshot name in Example 15.
- Bump date.

MFC r238071:

Fix clang warning, introduced in the recent dtrace import.

Approved by: re (kib)

MFC r237624, r237714, r237716, r237860:

Bring llquantize support into Dtrace.

Bryan Cantrill implemented the equivalent of semi-log graph
paper for Dtrace so llquantize will use one logarithmic and
one linear scale.

MFC r237119:

Do not remount ZFS dataset if changing canmount property to "on" and
dataset is already mounted.

PR: 167905
Submitted by: Bryan Drewery <bryan@shatow.net>

MFC r236776, r236777:

r236776:
Fix a typo: s/deafult/default

r236777:
Clean up trailing whitespace.

MFC r236960:
Document the -v flag for zpool list.

PR: 168970
Suggested by: Marcelo Araujo <araujo@FreeBSD.org>

MFC r236155:
Import illumos changeset 13570:3411fd5f1589
1948 zpool list should show more detailed pool information

Display per-vdev information with "zpool list -v".
The added expandsize property has currently no value on FreeBSD.
This changeset allows adding expansion support to individual vdevs
in the future.

References:
https://www.illumos.org/issues/1948

Obtained from: illumos (issue #1948)

MFC r236143, r236705:

MFC r236143 [1]:
Import illumos changeset 13571:a5771a96228c
1950 ztest backwards compatibility testing option

MFC r236705 [2]:
Import Illumos revision 13715:351036203e4b
2803 zfs get guid pretty-prints the output

References:
https://www.illumos.org/issues/1950 [1]
https://www.illumos.org/issues/2803 [2]

Obtained from: illumos (issue #1950 [1], #2803 [2])

MFC r236145, r236146:

MFC r236145 [1]:
Import illumos changeset 13564:cf89c0c60496
1946 incorrect formatting when listing output of multiple pools with
zpool iostat -v

MFC r236146 [2]:
Import illumos changeset 13605:b5c2b5db80d6 (partial)
763 FMD msg URLs should refer to something visible
Replace sun.com URL's with illumos.org

References:
https://www.illumos.org/issues/1946 [1]
https://www.illumos.org/issues/763 [2]

Obtained from: illumos (issue #1946 [1], #763 [2])

MFC r235479: zpool_find_import_impl: another /dev/dsk -> /dev fix

PR: bin/155104

MFC r235478: zpool_do_import: use /dev instead of /dev/dsk as a default

MFC r234691

Implement the D "cpu" variable, which returns curcpu. I have chosen not
to follow the example of OpenSolaris and its descendants, which implemented
cpu as an inline that took a value out of curthread. At certain points in
the FreeBSD scheduler curthread->td_oncpu will no longer be valid (in
particular, just before the thread gets descheduled) so instead I have
implemented this as its own built-in variable.

Sponsored by: Sandvine Inc.

MFC r235222:
Import illumos changeset 13686:4bc0783f6064
2703 add mechanism to report ZFS send progress

If the zfs send command is used with the -v flag, the amount of bytes
transmitted is reported in per second updates.

References:
https://www.illumos.org/issues/2703

Obtained from: illumos (issue #2703)

MFC r235212, r235216, r235217:

MFC r325212 [1]:
Import illumos changeset 13618:c6ae14a341e8
2088 zdb could use a reasonable manual page

Rewrite new zdb manpage to mdoc(7)

MFC r235216 [2]:
Add support for force unmounting ZFS filesystems during "zfs rename"
with the -f flag.

Reimplementation of the illumos changeset 13677:a0cbef703c12
2635 'zfs rename -f' to perform force unmount

MFC r235217 [3]:
Remove unreachable break in zfs_main.c

Partial import of illumos changeset 13622:e5889df1eaac
2077 lots of unreachable breaks in illumos gate

References:
https://www.illumos.org/issues/2088 [1]
https://www.illumos.org/issues/2635 [2]
https://www.illumos.org/issues/2077 [3]

PR: 164447 [1], 164447 [2], 167370 [3]
Suggested by: Marcelo Araujo <araujo@FreeBSD.org>
Obtained from: illumos (issue #2088 [1], #2635 [2], #2077 [3])

MFC r235211:

General mdoc(7) and typo fixes.

PR: 167696

MFC: 234234

Change SIGUSR1 to SIGTHR to properly wake up a process that is being
traced. The use of SIGUSR1 caused traced processes (those attached to
with dtrace -p) to exit when dtrace exited.

MFC r234336:
Fix typo miror -> mirror

Reported by: Glen Barber <gjb@FreeBSD.org>

MFC r232064:
Import illumos changeset 13608 [1]:

add support for "-t <datatype>" argument to zfs get

References:
https://www.illumos.org/issues/1936

Update zfs(8) manpage in respect of [1].
Fix typo in zfs(8) manpage.

Obtained from: illumos (issue #1936)

MFC r230623

exclude kmem_alloc'ed ARC data buffers from kernel minidumps on amd64
excluding other allocations including UMA now entails the addition of
a single flag to kmem_alloc or uma zone create

Reviewed by: alc,avg

MFC r230397, r230438:

MFC r230397 (pjd):
By default turn off prefetch when listing snapshots.
In my tests it makes listing snapshots 19% faster with cold cache and
47% faster with warm cache.

MFC r230438 (pjd):
Dramatically optimize listing snapshots when user requests only snapshot
names and wants to sort them by name, ie. when executes:

# zfs list -t snapshot -o name -s name

Because only name is needed we don't have to read all snapshot properties.

Below you can find how long does it take to list 34509 snapshots from a single
disk pool before and after this change with cold and warm cache:

before:

# time zfs list -t snapshot -o name -s name > /dev/null
cold cache: 525s
warm cache: 218s

after:

# time zfs list -t snapshot -o name -s name > /dev/null
cold cache: 1.7s
warm cache: 1.1s

MFC: r230635

Set SVN text/plain property for some shell scripts that
happen to have a .exe extension.

While here fix the shebang of a shell script that
was looking for /bin/bash.

Reviewed by: gnn
Approved by: jhb (mentor)

MFC r231144:
Merge illumos revision 13594:

uninitialized variables in zfs(8) may make snapshots undestroyable [1]

References:
https://www.illumos.org/issues/2067 [1]

Obtained from: illumos (issue #2067)
PR: bin/164802

MFC r230514:
Merge illumos revisions 13572, 13573, 13574:

Rev. 13572:
disk sync write perf regression when slog is used post oi_148 [1]

Rev. 13573:
crash during reguid causes stale config [2]
allow and unallow missing from zpool history since removal of pyzfs [5]

Rev. 13574:
leaking a vdev when removing an l2cache device [3]
memory leak when adding a file-based l2arc device [4]
leak in ZFS from metaslab_group_create and zfs_ereport_checksum [6]

References:
https://www.illumos.org/issues/1909 [1]
https://www.illumos.org/issues/1949 [2]
https://www.illumos.org/issues/1951 [3]
https://www.illumos.org/issues/1952 [4]
https://www.illumos.org/issues/1953 [5]
https://www.illumos.org/issues/1954 [6]

Obtained from: illumos (issues #1909, #1949, #1951, #1952, #1953, #1954)

MFC r230449:
Merge illumos revisions 13540, 13562:

illumos rev 13540 [1]:
Removal of pyzfs broke delegation for volumes

illumos rev 13562 [2]:
zfs allow arguments not parsed correctly after pyzfs removal

References:
https://www.illumos.org/issues/1726 [1]
https://www.illumos.org/issues/1977 [2]

Obtained from: illumos (issues #1726, #1977)

MFC r230402, r230404:
Add accidentially removed copyright lines in r228103

Reported by: pjd

MFC r228206, r228353:

MFC r228206:
Remove unnecesary "Ns" macros and add missing command example to zpool(8).

MFC r228353:
Some mdoc(7) style and typo fixes to zfs(8).

Submitted by: Nobuyuki Koganemaru <n-kogane@syd.odn.ne.jp>

MFC r228103, r228104:

MFC r228103:
Merge new ZFS features from illumos:

1644 add ZFS "clones" property
https://www.illumos.org/issues/1644

1645 add ZFS "written" and "written@..." properties
https://www.illumos.org/issues/1645

1646 "zfs send" should estimate size of stream
https://www.illumos.org/issues/1646

1647 "zfs destroy" should determine space reclaimed by destroying multiple
snapshots
https://www.illumos.org/issues/1647

1693 persistent 'comment' field for a zpool
https://www.illumos.org/issues/1693

1708 adjust size of zpool history data
https://www.illumos.org/issues/1708

1748 desire support for reguid in zfs
https://www.illumos.org/issues/1748

MFC r228104:
Fix typo in copyright notice.

Obtained from: illumos (changesets 13514, 13524, 13525)

MFC zfs manpage update
(r227646, r227648, r227649, r227752, r228019, r228045, r228054, r228055)

MFC r227646 (partial):
Update and desolarization of zfs(8) and zpool(8) manual pages:
- synchronized to match new vendor code (Illumos rev. 13513) [1]
- removed references to sun commands (replaced with FreeBSD commands)
- removed ATTRIBUTES sections
- updated SEE ALSO sections
- properly updated copyright information (required by CDDL)

zfs(8) only:
- replaced "Zones" section with new "Jails" section
- removed misleading "ZFS Volumes as Swap or Dump Devices" section
- updated shareiscsi and sharesmb option information (not supported on FreeBSD)
- replace zoned property with jailed property

zpool(8) only:
- updated device names in examples

MFC r227648:
Fix reference to fsync(2).
Add more references to SEE ALSO section.

MFC r227649:
More zfs(8) manpage fixes:
- remove shareiscsi property
- mark casesensitivity property as unsupported
- remove reference to Solaris Administration Guide

MFC r227752 (partial):
Update and desolarization of zdb(8) and zstreamdump(1) manual pages:
- synchronized to match new vendor code [1]
- removed ATTRIBUTES sections
- updated SEE ALSO sections
- properly updated copyright information (required by CDDL)

MFC r228019:
Update ZFS manual pages to a mdoc(7) reimplementation.

The zfs(8) and zpool(8) manual pages now match the state of the ZFS module
and have been customized for FreeBSD.

The new texts of the "Deduplication" subsection in zfs(8), the zpool "split"
command, the zfs "dedup" property and several other missing parts have been
added from illumos or OpenSolaris snv_134 (CDDL-licensed).

The mdoc(7) reimplementation of whole manual pages, the descriptions of the
zpool "readonly" property, "zfs diff" command and descriptions of several
other missing command flags and/or options were authored by myself.

MFC r228045:
Add missing -n flag to "zpool import" description.

MFC r228054:
Add missing warning to zfs(8) for using "zfs destroy" with -r and -R flags.

MFC r228055:
Use singular form for zfs destroy snapshot in zfs(8).

Obtained from: Illumos (as of rev. 13513:f84d4672fdbd) [1]

MFC r226676, r226678, r226700, r226705, r226706, r226707:

MFC r226676 (pjd):
Allow to rename file systems without remounting if it is possible.
It is possible for file systems with 'mountpoint' preperty set to 'legacy'
or 'none' - we don't have to change mount directory for them.
Currently such file systems are unmounted on rename and not even mounted back.

This introduces layering violation, as we need to update 'f_mntfromname'
field in statfs structure related to mountpoint (for the dataset we are
renaming and all its children).

In my opinion it is worth it, as it allow to update FreeBSD in even cleaner
way - in ZFS-only configuration root file system is ZFS file system with
'mountpoint' property set to 'legacy'. If root dataset is named system/rootfs,
we can snapshot it (system/rootfs@upgrade), clone it (system/oldrootfs),
update FreeBSD and if it doesn't boot we can boot back from system/oldrootfs
and rename it back to system/rootfs while it is mounted as /. Before it was
not possible, because unmounting / was not possible.

MFC r227768 (pjd):
Include <sys/zfs_vfsops.h> only when compiling kernel module.

MFC r226700 (pjd):
Don't forget to rename mounted snapshots of the file system being renamed.

MFC r226705 (pjd):
Extend r226676 to allow rename without unmount even for file systems with
non-legacy mountpoints. It is better to be able to rename such file systems and
let them be mounted in old places until next reboot than using live CD, etc. to
rename with remount.

This is implemented by adding -u option to 'zfs rename'. If file system's
mountpoint property is set to 'legacy' or 'none', there is no need to specify -u.

Update zfs(8) manual page to reflect this addition.

MFC r226706 (pjd):
Update copyright to include myself.

MFC r226707 (pjd):
- Use better naming now that we allow to rename any mounted file system (not
only legacy).
- Update copyright to include myself.

Approved by: pjd

MFC r227497, r228020:

MFC r227497 [1]:
Import upstream changesets for the output of the "zpool" command:

952 separate intent logs should be obvious in 'zpool iostat' output
1337 `zpool status -D' should tell if there are no DDT entries

References:
https://www.illumos.org/issues/952
https://www.illumos.org/issues/1337

MFC r228020:
Fix zfs(8) and zpool(8) context help to repport supported flags.

Obtained from: Illumos (issues 952, 1337; changesets 13384, 13432) [1]

MFC r228598:

In cddl/contrib/opensolaris/cmd/dtrace/dtrace.c, fix some obtuse
snprintf tricks.

MFC r228597:

In cddl/contrib/opensolaris/tools/ctf/cvt/ctfmerge.c, a size_t is passed
to fprintf as a field width. It should be an int instead, so cast it.

MFC r228580:

In cddl/contrib/opensolaris/lib/libdtrace/common/dt_handle.c, some
uint64_t values are snprintf'd using %llx. On amd64, uint64_t is
typedef'd as unsigned long, so cast the values to u_longlong_t, as is
done similarly in the rest of the file.

MFC r228579:

In cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c, some
uint64_t values are snprintf'd using %llx. On amd64, uint64_t is
typedef'd as unsigned long, so cast the values to u_longlong_t, as is
done similarly in the rest of the file.

MFC r228550:

In cddl/contrib/opensolaris/lib/libdtrace/common/dt_subr.c, the
dt_popc() function assumes that either _ILP32 or _LP64 is defined,
otherwise it has no suitable implementation.

However, the _ILP32 and _LP64 macros come from isa_defs.h, which is not
included in this file. Add the include now, to get the macros defined.

MFC r228549:

In cddl/contrib/opensolaris/lib/libdtrace/common/dt_link.c, use the
correct printf format for a key_t (aka long).

MFC r228548:

In cddl/contrib/opensolaris/lib/libdtrace/i386/dt_isadep.c, use the
correct printf format for an unsigned long.

MFC r228547:

Cast away a clang alignment warning in drti.c's fixsymbol() function.
This code only runs on i386 and amd64, so there should be no problems if
buf + sec->dofs_offset is not aligned (which is unlikely anyway).

Merge r228057 from head to stable/9:

Change the Makefile in cddl/lib/drti to use bsd.lib.mk instead of
bsd.prog.mk -- we need to compile PIC, which requires a library build.
With this change, USDT (userspace DTrace probes) work from within
shared libraries.

PR: kern/159046
Submitted by: Alex Samorukov <samm at os2.kiev.ua>
Comments by: Scott Lystig Fritchie <slfritchie at snookles.com>

Approved by: re (xxx)

MFC r226613,r226615,r226616:

r226613:

libzpool task_alloc: pass only valid flags to kmem_alloc

tqflags may contain other flags besided those that are suitable for
kmem_alloc == umem_alloc

Submitted by: avg

r226615:

thr_create: new_thread_ID may be NULL

Submitted by: avg

r226616:

zdb: access dp_free_bpobj only if pool version is >= SPA_VERSION_DEADLISTS

Submitted by: avg

Approved by: re (kib)

MFC: r227006, r227281, r227282

Add a PCI front-end to esp(4) allowing it to support AMD Am53C974 and
replace amd(4) with the former in the amd64, i386 and pc98 GENERIC kernel
configuration files. Besides duplicating functionality, amd(4), which
previously also supported the AMD Am53C974, unlike esp(4) is no longer
maintained and has accumulated enough bit rot over time to always cause
a panic during boot as long as at least one target is attached to it
(see PR 124667).

PR: 124667
Approved by: re (kib)
Obtained from: NetBSD (based on)

MFC r225828:
Remove assertion that prevents zfs rename of datasets with mountpoint=none
or mountpoint=legacy that have children datasets. This also fixes dataset
rename when receiving incremental snapshots as reported on freebsd-fs@

This assertion was made triggerable by opensolaris change #10196.

PR: bin/160400
Reviewed by: pjd
Approved by: re (kib)

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

Fix serious bug in ZIL that can lead to pool corruption
in the case of a held dataset during remount.

Detailed description is available at:
https://www.illumos.org/issues/883

illumos-gate revision: 13380:161b964a0e10

Reviewed by: pjd
Approved by: re (kib)
Obtained from: Illumos (Bug #883)
MFC after: 3 days

Fix wrong initialization of "cmd" for calling the jail/unjail ioctl.

Reviewed by: pjd@, delphij@
Approved by: re (kib)
MFC after: 3 days

ZFS tries to allocate blocks evenly across all devices. This means when
devices are imbalanced zfs will lots of CPU searching for space on devices
which tend to be pretty full. It should instead fail quickly on the full
devices and move onto devices which have more availability.

New loader tunable: vfs.zfs.mg_alloc_failures (min = 8)

Illumos-gate changeset: 13379:4df42cc92254

Obtained from: Illumos (Bug #1051)
MFC after: 2 weeks

Resurrect the ZFS "aclmode" property
Change default of "aclmode" to "discard".

Illumos-gate changeset: 13370:8c04143bd318

Obtained from: Illumos (Feature #742)
MFC after: 2 weeks

cddl/contrib/opensolaris/cmd/zpool/zpool_main.c:
cddl/contrib/opensolaris/cmd/zpool/zpool.8:
cddl/contrib/opensolaris/lib/libzfs/common/libzfs_import.c:
Add the "zpool labelclear" command. This command can be
used to wipe the label data from a drive that is not
active in a pool. The optional "-f" argument can be
used to treat an exported or foreign vdev as "inactive"
thus allowing its label information to be cleared.

Correct reporting of missing leaf vdevs so that the GUID required to
perform pool actions is always displayed.

cddl/contrib/opensolaris/cmd/zpool/zpool_main.c:
The "zpool status" command reports the "last seen at"
device node path when the vdev name is being reported
by GUID. Augment this code to assume a GUID is reported
when a device goes missing after initial boot in addition
to the previous behavior of doing this for devices that
aren't seen at boot.

cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c:
In zpool_vdev_name(), report recently missing devices
by GUID. There is no guarantee they will return at
their previous location.

cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h:
cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c:
o Add zpool_pool_state_to_name() API to libzfs which converts a
pool_state_t into a user consumable string.
o While here, correct constness of make zpool_state_to_name()
and zpool_label_disk().

MFD after: 1 week

With retirement of cpumask_t and usage of cpuset_t for representing a
mask of CPUs, pc_other_cpus and pc_cpumask become highly inefficient.

Remove them and replace their usage with custom pc_cpuid magic (as,
atm, pc_cpumask can be easilly represented by (1 << pc_cpuid) and
pc_other_cpus by (all_cpus & ~(1 << pc_cpuid))).

This change is not targeted for MFC because of struct pcpu members
removal and dependency by cpumask_t retirement.

MD review by: marcel, marius, alc
Tested by: pluknet
MD testing by: marcel, marius, gonzo, andreast

Add a new "REFCOMPRESSRATIO" property.

For snapshots, this is the same as COMPRESSRATIO, but for
filesystems/volumes, the COMPRESSRATIO is based on the data "USED" (ie,
includes blocks in children, but not blocks shared with the origin).

This is needed to figure out how much space a filesystem would use if it
were not compressed (ignoring snapshots).

Illumos-gate revision: 13387

Obtained from: Illumos (Feature #1092)
MFC after: 2 weeks

Allow mountpoints as arguments for the 'zfs get' command.

Illumos-gate revision: 13295

Obtained from: Illumos (Feature #510)
MFC after: 1 week

Do not use #warning to warn about missing implementation of dt_popc(),
but just have a comment that this is broken.

This is just a bandaid until somebody can fix this correctly. The code
is just a broken as it was before r223262 - now buildworld just doesn't
fail.

Tested by: i386 + amd64 buildworld
With hat: benl co-mentor

Fix clang warnings.

Approved by: philip (mentor)

Remove C constructs that are incompatible with C++ from various
OpenSolaris and ZFS header files. These changes are sufficient
to allow a C++ program to use the libzfs library.

Note: The majority of these files already included 'extern "C"'
declarations, so the intention of providing C++ compatibility
already existed even if it wasn't provided.

cddl/compat/opensolaris/include/assert.h:
Wrap our compatibility assert implementation in
'extern "C"'. Since this is a compatibility header
I matched the Solaris style of doing this explicitly
rather than rely on FreeBSD's __BEGIN/END_DECLS macro.

sys/cddl/compat/opensolaris/sys/kstat.h:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/arc.h:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dsl_pool.h:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/ddt.h:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio.h:
Rename parameters in function declarations that conflict
with C++ keywords. This was the solution preferred by
members of the Illumos community.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_ioctl.h:
In C, nested structures are visible in the global namespace,
but in C++, they take on the namespace of the structure in
which they are contained. Flatten nested structure
definitions within struct zfs_cmd so these structures are
visible in the global namespace when compiled in both
languages.

Sponsored by: Spectra Logic Corporation

etire the cpumask_t type and replace it with cpuset_t usage.

This is intended to fix the bug where cpu mask objects are
capped to 32. MAXCPU, then, can now arbitrarely bumped to whatever
value. Anyway, as long as several structures in the kernel are
statically allocated and sized as MAXCPU, it is suggested to keep it
as low as possible for the time being.

Technical notes on this commit itself:
- More functions to handle with cpuset_t objects are introduced.
The most notable are cpusetobj_ffs() (which calculates a ffs(3)
for a cpuset_t object), cpusetobj_strprint() (which prepares a string
representing a cpuset_t object) and cpusetobj_strscan() (which
creates a valid cpuset_t starting from a string representation).
- pc_cpumask and pc_other_cpus are target to be removed soon.
With the moving from cpumask_t to cpuset_t they are now inefficient
and not really useful. Anyway, for the time being, please note that
access to pcpu datas is protected by sched_pin() in order to avoid
migrating the CPU while reading more than one (possible) word
- Please note that size of cpuset_t objects may differ between kernel
and userland. While this is not directly related to the patch itself,
it is good to understand that concept and possibly use the patch
as a reference on how to deal with cpuset_t objects in userland, when
accessing kernland members.
- KTR_CPUMASK is changed and now is represented through a string, to be
set as the example reported in NOTES.

Please additively note that no MAXCPU is bumped in this patch, but
private testing has been done until to MAXCPU=128 on a real 8x8x2(htt)
machine (amd64).

Please note that the FreeBSD version is not yet bumped because of
the upcoming pcpu changes. However, note that this patch is not
targeted for MFC.

People to thank for the time spent on this patch:
- sbruno, pluknet and Nicholas Esborn (nick AT desert DOT net) tested
several revision of the patches and really helped in improving
stability of this work.
- marius fixed several bugs in the sparc64 implementation and reviewed
patches related to ktr.
- jeff and jhb discussed the basic approach followed.
- kib and marcel made targeted review on some specific part of the
patch.
- marius, art, nwhitehorn and andreast reviewed MD specific part of
the patch.
- marius, andreast, gonzo, nwhitehorn and jceel tested MD specific
implementations of the patch.
- Other people have made contributions on other patches that have been
already committed and have been listed separately.

Companies that should be mentioned for having participated at several
degrees:
- Yahoo! for having offered the machines used for testing on big
count of CPUs.
- The FreeBSD Foundation for having sponsored my devsummit attendance,
which has been instrumental.
- Sandvine for having offered offices and infrastructure during
development.

(I really hope I didn't forget anyone, if it happened I apologize in
advance).

Close a race between libzfs and mountd when updating NFS exports.
- Flush the file descriptor for the new ZFS exports file before
sending a SIGHUP to mountd.

Reviewed by: pjd
Approved by: ken
MFC after: 3 days

Don't pass empty mount options to nmount(2).

Reviewed by: pjd
MFC after: 2 weeks

Fix 'zfs list <path>' handling. If the path was found, the 'ret' variable was
uninitialized.

PR: kern/155940
Submitted by: KOIE Hidetaka <koie@suri.co.jp>
MFC after: 1 week

Properly print characters larger than 127.

Submitted by: noordsij <noordsij@cs.helsinki.fi>
Reviewed by: Eric Schrock <eric.schrock@delphix.com>
MFC after: 1 month

Finally... Import the latest open-source ZFS version - (SPA) 28.

Few new things available from now on:

- Data deduplication.
- Triple parity RAIDZ (RAIDZ3).
- zfs diff.
- zpool split.
- Snapshot holds.
- zpool import -F. Allows to rewind corrupted pool to earlier
transaction group.
- Possibility to import pool in read-only mode.

MFC after: 1 month

When building libzpool on ia64 or sparc64, don't add the .note.GNU-stack
section.

Submitted by: kib

Fix some leftover binaries and shared libraries in the system that still
have an executable stack, due to linking in hand-assembled .S or .s
files, that have no .GNU-stack sections:

RWX --- --- /lib/libcrypto.so.6
RWX --- --- /lib/libmd.so.5
RWX --- --- /lib/libz.so.6
RWX --- --- /lib/libzpool.so.2
RWX --- --- /usr/lib/liblzma.so.5

These were found using scanelf, from the sysutils/pax-utils port.

Reviewed by: kib

Re-commit the zfs sysctl(9) type-safety changes.

Thanks to dim and pjd for the pointer to zfs_context.h for building
userland.

Print message with information about updating the boot code if a new
vdev is attached to a root pool (e.g. when creating a mirrored boot pool).

Reviewed by: pav
Approved by: delphij (mentor)
MFC after: 3 days

Do not print OpenSolaris hint to use (non-existing) installgrub(1) command
if creating a mirror by attaching a new vdev to a root pool.

Reported by: James R. Van Artsdalen (on freebsd-fs@freebsd.org)
Approved by: delphij (mentor)
MFC after: 3 days

Do not lint code beyond necessity (with apologies to Wiliam of Ockham).

Don't lint externally maintained CDDL code, or relint the 32-bit libraries
in amd64 mode.

Fix ztest when it is executed by just 'ztest' and not by full path
'/usr/bin/ztest'.

1. Remove invalid assertion.
2. Properly recalculate delta in case pthread_cond_timedwait() is interrupted.
3. Style fix.

Reported by: [1] App Deb <appdebgr@gmail.com>

Use ${PICFLAG} instead of -fpic.

Properly handle IO with B_FAILFAST
Retry IO once with ZIO_FLAG_TRYHARD before declaring a pool faulted

OpenSolaris revision and Bug IDs:

9725:0bf7402e8022
6843014 ZFS B_FAILFAST handling is broken

Approved by: delphij (mentor)
Obtained from: OpenSolaris (Bug ID 6843014)
MFC after: 3 weeks

Enable offlining of log devices.

OpenSolaris revision and Bug IDs:

9701:cc5b64682e64
6803605 should be able to offline log devices
6726045 vdev_deflate_ratio is not set when offlining a log device
6599442 zpool import has faults in the display

Approved by: delphij (mentor)
Obtained from: OpenSolaris (Bug ID 6803605, 6726045, 6599442)
MFC after: 3 weeks

Minor grammar fixes.

Remove duplicate include of <strings.h>

Approved by: delphij (mentor)
MFC after: 3 days

Add a man page for plockstat.

Sponsored by: The FreeBSD Foundation

Give a chance to the target binary to run the ctors by waiting until it
has reached main(). This allows plockstat to work.

Sponsored by: The FreeBSD Foundation

Disable debug by default.

Sponsored by: The FreeBSD Foundation

Don't try to map the USDT probes. This is necessary because there is no
__SUNW_dof symbol present in FreeBSD binaries.

Sponsored by: The FreeBSD Foundation

Don't clobber an existing target object file when doing the DTrace
linking process. This is needed because we change the source object
files and the second this dtrace -G is run, no probes will be found.
This hack allows us to build postgres with DTrace probes enabled. I'll
try to find a way to fix this without needing this hack.

Sponsored by: The FreeBSD Foundation

Include <sys/sdt.h> on a generated header.

Sponsored by: The FreeBSD Foundation

Eliminate confusing while () loop. In the first version of the code it was
there to avoid gotos, but in the current version it serves no purpose.

MFC after: 2 weeks

When upgrading a pool which contain root file system, give user a hint that
he should update boot code.

MFC after: 2 weeks

Give user a hint what to do when /usr/lib/zfs/pyzfs.py is missing.

MFC after: 2 weeks

Print errors on stderr.

MFC after: 2 weeks

Fix 'zfs allow' (maybe not only) returning:

cannot access dataset system/usr/home: Operation not supported

by including libzfs_impl.h. What libzfs_impl.h does is to redefine ioctl() to
be compatible with OpenSolaris. More specifically OpenSolaris returns ENOMEM
when buffer is too small and sets field zc_nvlist_dst_size to the size that
will be big enough for the data. In FreeBSD case ioctl() doesn't copy data
structure back in case of a failure. We work-around it in kernel and libzfs by
returning 0 from ioctl() and always checking if zc_nvlist_dst_size hasn't
changed. For this work-around to work in pyzfs we need this compatible ioctl()
which is implemented in libzfs_impl.h.

MFC after: 2 weeks

Update ZFS metaslab code from OpenSolaris.
This provides a noticeable write speedup, especially on pools with
less than 30% of free space.

Detailed information (OpenSolaris onnv changesets and Bug IDs):

11146:7e58f40bcb1c
6826241 Sync write IOPS drops dramatically during TXG sync
6869229 zfs should switch to shiny new metaslabs more frequently

11728:59fdb3b856f6
6918420 zdb -m has issues printing metaslab statistics

12047:7c1fcc8419ca
6917066 zfs block picking can be improved

Approved by: delphij (mentor)
Obtained from: OpenSolaris (Bug ID 6826241, 6869229, 6918420, 6917066)
MFC after: 2 weeks

Install dtruss.

Sponsored by: The FreeBSD Foundation

Add a man page for dtruss.

Sponsored by: The FreeBSD Foundation

Port dtruss to FreeBSD.

Sponsored by: The FreeBSD Foundation

Merge dtracetoolkit/dtruss from the vendor.

Sponsored by: The FreeBSD Foundation

Remove this directory for an attempt to merge from the vendor tree.

MFtbemd:

Prefer MACHNE_CPUARCH to MACHINE_ARCH in most contexts where you want
to test of all the CPUs of a given family conform.

Use real atomic operations for sparc64.

MFC after: 1 week

Add powerpc64 to the list of architectures with real atomic operations.

Submitted by: imp

Don't link drti.o with libelf_pic.a. This means that all software with
userland SDT probes must be linked with libelf explicitly.

Requested by: kib
Sponsored by: The FreeBSD Foundation

Use double quotes when checking the value of MACHINE_ARCH.

Sponsored by: The FreeBSD Foundation

Add a Makefile for dtruss.

Sponsored by: The FreeBSD Foundation

Port plockstat to FreeBSD. This will be connected to the build later.

Sponsored by: The FreeBSD Foundation

Link drti.o with a PIC version of libelf. This is needed because
drti.o depends on libelf and this avoids linking every other drti.o
program (namely programs with USDT probes) with libelf.

Sponsored by: The FreeBSD Foundation

Add a directory for the DTraceToolkit.

Sponsored by: The FreeBSD Foundation

Add libdtrace support for tracing userland programs.

Summary of changes:
* Implement a compatibility shim between Solaris libproc and our
libproc and remove several ifdefs because of this.
* Port the drti to FreeBSD.
* Implement the missing DOODAD sections
* Link with libproc and librtld_db
* Support for ustack, jstack and uregs (by sson@)
* Misc bugfixing

When writing the SUWN_dof section, we had to resort to building the ELF
file layout by "hand". This is the job of libelf, but our libelf doesn't
support this yet. When libelf is fixed, we can remove the code under
#ifdef BROKEN_LIBELF.

Sponsored by: The FreeBSD Foundation

Use double quotes when checking the value of MACHINE_ARCH.

Sponsored by: The FreeBSD Foundation

The DTrace instrumentation object is going to be i386/amd64 only.

Sponsored by: The FreeBSD Foundation

Port most of the DTrace tests to FreeBSD.

Sponsored by: The FreeBSD Foundation

Add man pages for the CTF tools.

Sponsored by: The FreeBSD Foundation

Fix another mismerge: bring back the definition of DT_MUTEX_HELD().

Fix the result of a mismerge. MUTEX_HELD should be DT_MUTEX_HELD() and
Plmid() is not supposed to be called.

MFV OpenSolaris DTrace userland bits.

lockstat(1) now depends on libutil and librtld_db.

Sponsored by: The FreeBSD Foundation

proc_wait() was replaced by proc_wstatus(). Right now this call is
useless because userland support is not working on HEAD, but this makes
libdtrace compile again.

Sponsored by: The FreeBSD Foundation

libdtrace, dtrace(1) and lockstat(1) are for i386 and amd64 only right
now.

Sponsored by: The FreeBSD Foundation

Update to the new proc_detach() function.

Sponsored by: The FreeBSD Foundation

dtrace(1) now depends on libutil and librtld_db.

Sponsored by: The FreeBSD Foundation

Catch up with r210324. d_buf will be NULL for SHT_NOBITS sections, do not
attempt to copy from it in that case.

MFC after: 1 month

dtrace: correctly map sections to addresses in elf object modules (amd64)

Unlike for modules with dso type, in elf object modules all the sections
have virtual address of zero. So, it is insufficient to add module base
address to section virtual address (as recorded in section header) to
get section address in kernel memory.
Instead, we should apply the same calculations that are performed by
kernel loaders (in boot code and in kernel) when they lay out sections
in memory.
Also, unlike OpenSolaris, the sections are not collapsed into just .text,
.data and .bss by module loaders, so we need to take additional care
about other sections.

Note that in-kernel symbol-to-address mapping worked just fine, e.g. fbt
provider could correctly find the functions, etc. It's only in userland
that the mapping in both direction worked incorrectly, e.g. in stack()
output addresses of functions in kernel modules were not translated to
their names.

Reviewed by: rpaulo
MFC after: 3 weeks

Enable fake resolving of SMB RIDs by using nulldomain and UID_NOBODY
- fixes panics when Solaris/OpenSolaris pools that contain files
uploaded with the SMB protocol are accessed

Enable seting/unsetting the sharesmb property (dummy action)
- allows users who import pools from Solaris/Opensolaris to unset
the sharesmb property and get rid of annoying messages

PR: kern/145778, kern/148709
Approved by: pjd, delphij (mentor)
MFC after: 7 weeks

MFV:
OpenSolaris' plockstat utility.

Sponsored by: The FreeBSD Foundation

Merge from vendor: libdtrace MD parts needed by fasttrap.

Sponsored by: The FreeBSD Foundation

MFp4 180933:

Initialize rw_count properly so that zdb(8) doesn't trigger assertion in
rw_enter():

ASSERT(rwlp->rw_count == 0);

While here, assert that rw_count is 0 when destroying the lock.

MFC after: 1 week

Merge ZFS version 15 and almost all OpenSolaris bugfixes referenced
in Solaris 10 updates 141445-09 and 142901-14.

Detailed information:
(OpenSolaris revisions and Bug IDs, Solaris 10 patch numbers)

7844:effed23820ae
6755435 zfs_open() and zfs_close() needs to use ZFS_ENTER/ZFS_VERIFY_ZP (141445-01)

7897:e520d8258820
6748436 inconsistent zpool.cache in boot_archive could panic a zfs root filesystem upon boot-up (141445-01)

7965:b795da521357
6740164 zpool attach can create an illegal root pool (141909-02)

8084:b811cc60d650
6769612 zpool_import() will continue to write to cachefile even if altroot is set (N/A)

8121:7fd09d4ebd9c
6757430 want an option for zdb to disable space map loading and leak tracking (141445-01)

8129:e4f45a0bfbb0
6542860 ASSERT: reason != VDEV_LABEL_REMOVE||vdev_inuse(vd, crtxg, reason, 0) (141445-01)

8188:fd00c0a81e80
6761100 want zdb option to select older uberblocks (141445-01)

8190:6eeea43ced42
6774886 zfs_setattr() won't allow ndmp to restore SUNWattr_rw (141445-01)

8225:59a9961c2aeb
6737463 panic while trying to write out config file if root pool import fails (141445-01)

8227:f7d7be9b1f56
6765294 Refactor replay (141445-01)

8228:51e9ca9ee3a5
6572357 libzfs should do more to avoid mnttab lookups (141909-01)
6572376 zfs_iter_filesystems and zfs_iter_snapshots get objset stats twice (141909-01)

8241:5a60f16123ba
6328632 zpool offline is a bit too conservative (141445-01)
6739487 ASSERT: txg <= spa_final_txg due to scrub/export race (141445-01)
6767129 ASSERT: cvd->vdev_isspare, in spa_vdev_detach() (141445-01)
6747698 checksum failures after offline -t / export / import / scrub (141445-01)
6745863 ZFS writes to disk after it has been offlined (141445-01)
6722540 50% slowdown on scrub/resilver with certain vdev configurations (141445-01)
6759999 resilver logic rewrites ditto blocks on both source and destination (141445-01)
6758107 I/O should never suspend during spa_load() (141445-01)
6776548 codereview(1) runs off the page when faced with multi-line comments (N/A)
6761406 AMD errata 91 workaround doesn't work on 64-bit systems (141445-01)

8242:e46e4b2f0a03
6770866 GRUB/ZFS should require physical path or devid, but not both (141445-01)

8269:03a7e9050cfd
6674216 "zfs share" doesn't work, but "zfs set sharenfs=on" does (141445-01)
6621164 $SRC/cmd/zfs/zfs_main.c seems to have a syntax error in the translation note (141445-01)
6635482 i18n problems in libzfs_dataset.c and zfs_main.c (141445-01)
6595194 "zfs get" VALUE column is as wide as NAME (141445-01)
6722991 vdev_disk.c: error checking for ddi_pathname_to_dev_t() must test for NODEV (141445-01)
6396518 ASSERT strings shouldn't be pre-processed (141445-01)

8274:846b39508aff
6713916 scrub/resilver needlessly decompress data (141445-01)

8343:655db2375fed
6739553 libzfs_status msgid table is out of sync (141445-01)
6784104 libzfs unfairly rejects numerical values greater than 2^63 (141445-01)
6784108 zfs_realloc() should not free original memory on failure (141445-01)

8525:e0e0e525d0f8
6788830 set large value to reservation cause core dump (141445-01)
6791064 want sysevents for ZFS scrub (141445-01)
6791066 need to be able to set cachefile on faulted pools (141445-01)
6791071 zpool_do_import() should not enable datasets on faulted pools (141445-01)
6792134 getting multiple properties on a faulted pool leads to confusion (141445-01)

8547:bcc7b46e5ff7
6792884 Vista clients cannot access .zfs (141445-01)

8632:36ef517870a3
6798384 It can take a village to raise a zio (141445-01)

8636:7e4ce9158df3
6551866 deadlock between zfs_write(), zfs_freesp(), and zfs_putapage() (141909-01)
6504953 zfs_getpage() misunderstands VOP_GETPAGE() interface (141909-01)
6702206 ZFS read/writer lock contention throttles sendfile() benchmark (141445-01)
6780491 Zone on a ZFS filesystem has poor fork/exec performance (141445-01)
6747596 assertion failed: DVA_EQUAL(BP_IDENTITY(&zio->io_bp_orig), BP_IDENTITY(zio->io_bp))); (141445-01)

8692:692d4668b40d
6801507 ZFS read aggregation should not mind the gap (141445-01)

8697:e62d2612c14d
6633095 creating a filesystem with many properties set is slow (141445-01)

8768:dfecfdbb27ed
6775697 oracle crashes when overwriting after hitting quota on zfs (141909-01)

8811:f8deccf701cf
6790687 libzfs mnttab caching ignores external changes (141445-01)
6791101 memory leak from libzfs_mnttab_init (141445-01)

8845:91af0d9c0790
6800942 smb_session_create() incorrectly stores IP addresses (N/A)
6582163 Access Control List (ACL) for shares (141445-01)
6804954 smb_search - shortname field should be space padded following the NULL terminator (N/A)
6800184 Panic at smb_oplock_conflict+0x35() (N/A)

8876:59d2e67b4b65
6803822 Reboot after replacement of system disk in a ZFS mirror drops to grub> prompt (141445-01)

8924:5af812f84759
6789318 coredump when issue zdb -uuuu poolname/ (141445-01)
6790345 zdb -dddd -e poolname coredump (141445-01)
6797109 zdb: 'zdb -dddddd pool_name/fs_name inode' coredump if the file with inode was deleted (141445-01)
6797118 zdb: 'zdb -dddddd poolname inum' coredump if I miss the fs name (141445-01)
6803343 shareiscsi=on failed, iscsitgtd failed request to share (141445-01)

9030:243fd360d81f
6815893 hang mounting a dataset after booting into a new boot environment (141445-01)

9056:826e1858a846
6809691 'zpool create -f' no longer overwrites ufs infomation (141445-01)

9179:d8fbd96b79b3
6790064 zfs needs to determine uid and gid earlier in create process (141445-01)

9214:8d350e5d04aa
6604992 forced unmount + being in .zfs/snapshot/<snap1> = not happy (141909-01)
6810367 assertion failed: dvp->v_flag & VROOT, file: ../../common/fs/gfs.c, line: 426 (141909-01)

9229:e3f8b41e5db4
6807765 ztest_dsl_dataset_promote_busy needs to clean up after ENOSPC (141445-01)

9230:e4561e3eb1ef
6821169 offlining a device results in checksum errors (141445-01)
6821170 ZFS should not increment error stats for unavailable devices (141445-01)
6824006 need to increase issue and interrupt taskqs threads in zfs (141445-01)

9234:bffdc4fc05c4
6792139 recovering from a suspended pool needs some work (141445-01)
6794830 reboot command hangs on a failed zfs pool (141445-01)

9246:67c03c93c071
6824062 System panicked in zfs_mount due to NULL pointer dereference when running btts and svvs tests (141909-01)

9276:a8a7fc849933
6816124 System crash running zpool destroy on broken zpool (141445-03)

9355:09928982c591
6818183 zfs snapshot -r is slow due to set_snap_props() doing txg_wait_synced() for each new snapshot (141445-03)

9391:413d0661ef33
6710376 log device can show incorrect status when other parts of pool are degraded (141445-03)

9396:f41cf682d0d3 (part already merged)
6501037 want user/group quotas on ZFS (141445-03)
6827260 assertion failed in arc_read(): hdr == pbuf->b_hdr (141445-03)
6815592 panic: No such hold X on refcount Y from zfs_znode_move (141445-03)
6759986 zfs list shows temporary %clone when doing online zfs recv (141445-03)

9404:319573cd93f8
6774713 zfs ignores canmount=noauto when sharenfs property != off (141445-03)

9412:4aefd8704ce0
6717022 ZFS DMU needs zero-copy support (141445-03)

9425:e7ffacaec3a8
6799895 spa_add_spares() needs to be protected by config lock (141445-03)
6826466 want to post sysevents on hot spare activation (141445-03)
6826468 spa 'allowfaulted' needs some work (141445-03)
6826469 kernel support for storing vdev FRU information (141445-03)
6826470 skip posting checksum errors from DTL regions of leaf vdevs (141445-03)
6826471 I/O errors after device remove probe can confuse FMA (141445-03)
6826472 spares should enjoy some of the benefits of cache devices (141445-03)

9443:2a96d8478e95
6833711 gang leaders shouldn't have to be logical (141445-03)

9463:d0bd231c7518
6764124 want zdb to be able to checksum metadata blocks only (141445-03)

9465:8372081b8019
6830237 zfs panic in zfs_groupmember() (141445-03)

9466:1fdfd1fed9c4
6833162 phantom log device in zpool status (141445-03)

9469:4f68f041ddcd
6824968 add ZFS userquota support to rquotad (141445-03)

9470:6d827468d7b5
6834217 godfather I/O should reexecute (141445-03)

9480:fcff33da767f
6596237 Stop looking and start ganging (141909-02)

9493:9933d599bc93
6623978 lwb->lwb_buf != NULL, file ../../../uts/common/fs/zfs/zil.c, line 787, function zil_lwb_commit (141445-06)

9512:64cafcbcc337
6801810 Commit of aligned streaming rewrites to ZIL device causes unwanted disk reads (N/A)

9515:d3b739d9d043
6586537 async zio taskqs can block out userland commands (142901-09)

9554:787363635b6a
6836768 zfs_userspace() callback has no way to indicate failure (N/A)

9574:1eb6a6ab2c57
6838062 zfs panics when an error is encountered in space_map_load() (141909-02)

9583:b0696cd037cc
6794136 Panic BAD TRAP: type=e when importing degraded zraid pool. (141909-03)

9630:e25a03f552e0
6776104 "zfs import" deadlock between spa_unload() and spa_async_thread() (141445-06)

9653:a70048a304d1
6664765 Unable to remove files when using fat-zap and quota exceeded on ZFS filesystem (141445-06)

9688:127be1845343
6841321 zfs userspace / zfs get userused@ doesn't work on mounted snapshot (N/A)
6843069 zfs get userused@S-1-... doesn't work (N/A)

9873:8ddc892eca6e
6847229 assertion failed: refcount_count(&tx->tx_space_written) + delta <= tx->tx_space_towrite in dmu_tx.c (141445-06)

9904:d260bd3fd47c
6838344 kernel heap corruption detected on zil while stress testing (141445-06)

9951:a4895b3dd543
6844900 zfs_ioc_userspace_upgrade leaks (N/A)

10040:38b25aeeaf7a
6857012 zfs panics on zpool import (141445-06)

10000:241a51d8720c
6848242 zdb -e no longer works as expected (N/A)

10100:4a6965f6bef8
6856634 snv_117 not booting: zfs_parse_bootfs: error2 (141445-07)

10160:a45b03783d44
6861983 zfs should use new name <-> SID interfaces (N/A)
6862984 userquota commands can hang (141445-06)

10299:80845694147f
6696858 zfs receive of incremental replication stream can dereference NULL pointer and crash (N/A)

10302:a9e3d1987706
6696858 zfs receive of incremental replication stream can dereference NULL pointer and crash (fix lint) (N/A)

10575:2a8816c5173b (partial merge)
6882227 spa_async_remove() shouldn't do a full clear (142901-14)

10800:469478b180d9
6880764 fsync on zfs is broken if writes are greater than 32kb on a hard crash and no log attached (142901-09)
6793430 zdb -ivvvv assertion failure: bp->blk_cksum.zc_word[2] == dmu_objset_id(zilog->zl_os) (N/A)

10801:e0bf032e8673 (partial merge)
6822816 assertion failed: zap_remove_int(ds_next_clones_obj) returns ENOENT (142901-09)

10810:b6b161a6ae4a
6892298 buf->b_hdr->b_state != arc_anon, file: ../../common/fs/zfs/arc.c, line: 2849 (142901-09)

10890:499786962772
6807339 spurious checksum errors when replacing a vdev (142901-13)

11249:6c30f7dfc97b
6906110 bad trap panic in zil_replay_log_record (142901-13)
6906946 zfs replay isn't handling uid/gid correctly (142901-13)

11454:6e69bacc1a5a
6898245 suspended zpool should not cause rest of the zfs/zpool commands to hang (142901-10)

11546:42ea6be8961b (partial merge)
6833999 3-way deadlock in dsl_dataset_hold_ref() and dsl_sync_task_group_sync() (142901-09)

Discussed with: pjd
Approved by: delphij (mentor)
Obtained from: OpenSolaris (multiple Bug IDs)
MFC after: 2 months

- Add support for the "sec" option to "zfs set sharenfs"

Submitted by: Dmitry Morozovsky <marck@FreeBSD.org>
Approved by: delphij (mentor)
MFC after: 3 days

Set svn:executable property so we can run the DTrace test suite.

Unbreak platforms with char unsigned by default. Oddly enough, GCC isn't
satisfied with a simple cast to int in the check against EOF, so the fix
is a bit involved by actually having to go through a temporary variable.

Do not allow EOF token to be put back into input buffer.

This reimplements previous change from r20930 in more generic way.

MFC after: 1 week

In dtrace lexer, do not unput token if it is EOF.

This is harmless with OpenSolaris fex and yacc, but with FreeBSD
causes '\xff' symbol to be physically put into input buffer, causing
subsequent syntax error.

MFC after: 1 week

Allow to use 'jailed' property again.

Reported by: Eugene Mitrofanov <eugene@imedia.ru>
MFC after: 3 days

Fix zfs receive temporarily changing unchanged stream properties.
Fix possible panic with zfs_enable_datasets.

OpenSolaris onnv revision: 8536:33bd5de3260e

Approved by: pjd, delphij (mentor)
Obtained from: OpenSolaris (Bug ID 6748561, 6757075)
MFC after: 3 days

Fix userland build by making io_task available only for the kernel and by
providing taskq_dispatch_safe() macro.

MFC after: 1 week

Import OpenSolaris revision 7837:001de5627df3
It includes the following changes:
- parallel reads in traversal code (Bug ID 6333409)
- faster traversal for zfs send (Bug ID 6418042)
- traversal code cleanup (Bug ID 6725675)
- fix for two scrub related bugs (Bug ID 6729696, 6730101)
- fix assertion in dbuf_verify (Bug ID 6752226)
- fix panic during zfs send with i/o errors (Bug ID 6577985)
- replace P2CROSS with P2BOUNDARY (Bug ID 6725680)

List of OpenSolaris Bug IDs:
6333409, 6418042, 6757112, 6725668, 6725675, 6725680,
6725698, 6729696, 6730101, 6752226, 6577985, 6755042

Approved by: pjd, delphij (mentor)
Obtained from: OpenSolaris (multiple Bug IDs)
MFC after: 1 week

Fix possible panic with zfs destroy.

OpenSolaris onnv revision: 8779:f164e0e90508

PR: kern/146471
Approved by: pjd, delphij (mentor)
Obtained from: OpenSolaris (Bug ID 6784924)
MFC after: 3 days

Merger of the quota64 project into head.

This joint work of Dag-Erling Smørgrav and myself updates the
FFS quota system to support both traditional 32-bit and new 64-bit
quotas (for those of you who want to put 2+Tb quotas on your users).

By default quotas are not compiled into the kernel. To include them
in your kernel configuration you need to specify:

options QUOTA # Enable FFS quotas

If you are already running with the current 32-bit quotas, they
should continue to work just as they have in the past. If you
wish to convert to using 64-bit quotas, use `quotacheck -c 64';
if you wish to revert from 64-bit quotas back to 32-bit quotas,
use `quotacheck -c 32'.

There is a new library of functions to simplify the use of the
quota system, do `man quotafile' for details. If your application
is currently using the quotactl(2), it is highly recommended that
you convert your application to use the quotafile interface.
Note that existing binaries will continue to work.

Special thanks to John Kozubik of rsync.net for getting me
interested in pursuing 64-bit quota support and for funding
part of my development time on this project.

Introduce hardforce export option (-F) for "zpool export".
When exporting with this flag, zpool.cache remains untouched.

OpenSolaris onnv revision: 8211:32722be6ad3b

Approved by: pjd, delphij (mentor)
Obtained from: OpenSolaris (Bug ID: 6775357)

Enable "zfs list" to list explicitly requested snapshots.

Partial import of OpenSolaris onnv revision:
8415:8809e849f63e

PR: kern/146297
Submitted by: myself
Approved by: pjd, delphij (mentor)
Obtained from: OpenSolaris (Bug ID 6758338)
MFC after: 2 weeks

Do not encode more than CTF_MAX_VLEN(1023) enum members.

CTF can not represent enums with more than CTF_MAX_VLEN members, but
ctfconvert will happily ignore that limitation and create CTF section no
other tool can interpret.

This change is different from similar change from upstream, which just
returns an error if big enum is encountered. Doing that means that
every FreeBSD kernel with compiled in hwpmc will have no useable CTF
information due to pmc_event enum having 1236+ members.

Refine previous partial merge of OpenSolaris onnv revision 9396:f41cf682d0d3.
This fixes a regression that zfs list would crash on zfs having user properties.

PR: kern/145377
Submitted by: mm
Approved by: pjd
Obtained from: OpenSolaris
MFC after: 10 days

- update zfs and zpool manual pages to match the current state of the source

PR: 144984
Submitted by: mm@
Approved by: pjd@
Obtained from: OpenSolaris
MFC after: 4 days

Merge OpenSolaris revision 9396:f41cf682d0d3:

6830813 zfs list -t all fails assertion

*Note that this is only a partial merge of this revision addressing only
this one issue.*

PR: bin/144720
Submitted by: mm
Approved by: pjd
Obtained from: OpenSolaris
MFC after: 1 month

Merge OpenSolaris revision 9365:7838a22eccd6:

PSARC/2009/171 zfs list -d and zfs get -d
6762432 zfs list --depth

PR: bin/144720
Submitted by: mm
Approved by: pjd
Obtained from: OpenSolaris
MFC after: 1 month

Merge OpenSolaris revision 8802:010b31dd4c53:

6773366 "zfs list" memory consumption can be further reduced

PR: bin/144720
Submitted by: mm
Approved by: pjd
Obtained from: OpenSolaris
MFC after: 1 month

Remove manual .includes in cddl Makefiles

- Break the dependency on ../Makefile.inc for .PATH, and include
../Makefile.inc implicitly. This is required to ...
- Set WARNS?=6 in top-level Makefile.inc
- Remove now redundant WARNS settings, add WARNS?=0 where appropriate
- Remove redundant SHLIB_MAJOR overrides
- Use NO_MAN, not MK_MAN=no
- Remove redundant inclusion of bsd.own.mk
- Order Makefiles more according to style.Makefile(9)
- Reduce diff of cddl Makefiles against each other

No objection: pjd
Approved by: ed (co-mentor)

Fixed dependencies (make checkdpadd).

Apply OpenSolaris revision 8012 which brings our zpool to version 14,
making it possible for zpools created on OpenSolaris 2009.06 be used
on FreeBSD.

PR: kern/141800
Submitted by: mm
Reviewed by: pjd, trasz
Obtained from: OpenSolaris
MFC after: 2 weeks

Apply fix Solaris bug 6462803 zfs snapshot -r failed because
filesystem was busy.

Submitted by: mm
Approved by: pjd
MFC after: 2 weeks

Add an option to specify that the received ZFS should not be automatically
mounted (receive -u).

Obtained from: OpenSolaris (onnv revision 8584:327a1b6dd944)
Approved by: pjd

Properly mark ZFS properties which are not changeable under FreeBSD.

Reviewed by: pjd

'aclmode' and 'aclinherit' properties should work as advertised; don't
refuse to set them.

Fletcher4 is not the default checksum algorithm.

MFC after: 3 days

Implement __assert() for Solaris-specific code. Until now Solaris code was
using Solaris prototype for __assert(), but FreeBSD's implementation.
Both take different arguments, so we were either core-dumping in assert()
or printing garbage.

Reported by: avg
MFC after: 1 week

Fix detection of file system being shared. After this change commands like:

# zfs unshare -a
# zfs destroy foo/bar
# zfs rename foo/bar foo/baz

should properly remove exported file systems.

MFC after: 3 days

Teach zdb(8) how to obtain GEOM provider size.

PR: kern/133134
Reported by: Philipp Wuensche <cryx-freebsd@h3q.com>
MFC after: 3 days

Fix receive when dataset has no / in its name.

Submitted by: James R. Van Artsdalen <james-freebsd-current@jrv.org>
Approved by: re (kib)

Remove files that are no longer used.

Discussed with: kmacy
Approved by: re (kib)

Bump the version of all non-symbol-versioned shared libraries in
preparation for 8.0-RELEASE. Add the previous version of those
libraries to ObsoleteFiles.inc and bump __FreeBSD_Version.

Reviewed by: kib
Approved by: re (rwatson)

The mutex_owned() macro should operate on kmutex_t and not on mutex_t.
This fixes 'zdb <poolname>' crash.

Reported by: avg
Approved by: re (kib)

Bootstrap mergeinfo for the OpenSolaris contrib bits.

Fix lockstat breakage to arm/powerpc buildworld.
Thanks to IBM char's are unsigned on arm/powerpc.

Approved by: gnn (mentor)

Add the OpenSolaris lockstat(1M) command. Requires the dtrace driver,
the lockstat provider, and the ksyms(4) pseudo driver kernel modules.

Approved by: gnn (mentor)

atomic.S has been renamed opensolaris_atomic.S to avoid collisions

define VN_RELE_ASYNC for use by libzpool

Switch over to gnu99 compilation on default for userland.

Tested by: make universe
Tested by: ports exp build (done by pav)
Reviewed by: ru
Reviewed by: silence on arch
Approved by: ed (mentor)

Correct the sense of the "use32" terenary operator so that it uses
'lib32' for 32-bit binaries on amd64 and 'lib' for 64-bit binaries.
Remove an #ifdef __i386__ that defined use32 to the wrong value on
i386 to workaround the bug in the terenary operator.

MFC after: 1 month

Include param.h instead of types.h before mount.h so that the nested
include of param.h can be removed from audit.h.

MFC after: 3 days

Including mount.h requires including param.h.

MFC after: 3 weeks

IFp4: Don't rely on disk IDs and always use vdev guids, which means always look
up for components by reading metadata. This might be slower when there are big
number of disks in the system, but is definiately more reliable.

Fix a warning on amd64 caused by using int for request argument instead of
unsigned long:

WARNING pid 12888 (zfs/zpool): ioctl sign-extension ioctl ffffffffcc285aXX

Reported by: kris

Update ZFS from version 6 to 13 and bring some FreeBSD-specific changes.

This bring huge amount of changes, I'll enumerate only user-visible changes:

- Delegated Administration

Allows regular users to perform ZFS operations, like file system
creation, snapshot creation, etc.

- L2ARC

Level 2 cache for ZFS - allows to use additional disks for cache.
Huge performance improvements mostly for random read of mostly
static content.

- slog

Allow to use additional disks for ZFS Intent Log to speed up
operations like fsync(2).

- vfs.zfs.super_owner

Allows regular users to perform privileged operations on files stored
on ZFS file systems owned by him. Very careful with this one.

- chflags(2)

Not all the flags are supported. This still needs work.

- ZFSBoot

Support to boot off of ZFS pool. Not finished, AFAIK.

Submitted by: dfr

- Snapshot properties

- New failure modes

Before if write requested failed, system paniced. Now one
can select from one of three failure modes:
- panic - panic on write error
- wait - wait for disk to reappear
- continue - serve read requests if possible, block write requests

- Refquota, refreservation properties

Just quota and reservation properties, but don't count space consumed
by children file systems, clones and snapshots.

- Sparse volumes

ZVOLs that don't reserve space in the pool.

- External attributes

Compatible with extattr(2).

- NFSv4-ACLs

Not sure about the status, might not be complete yet.

Submitted by: trasz

- Creation-time properties

- Regression tests for zpool(8) command.

Obtained from: OpenSolaris

Merge latest DTrace changes from Perforce.

Approved by: jb

Mips doesn't have a long double that's different in size from double.

Add a missing file change from the VOP_GETATTR() argument axing.

Don't need to include vmem.h anymore.

Add the dtrace client app to the build.

Add the DTrace build tools.

Oops, these files belonged under src/sys, not src. Sorry.

This commit was generated by cvs2svn to compensate for changes in r179191,
which included commits to RCS files with non-trunk default branches.

Add the DTrace libraries and D library scripts.

Add Solaris compatibility definitions for things we don't really want in
FreeBSD headers.

* Handle the different ioctl design.
* Add a couple of FreeBSD action extensions.

Make this file compile cleanly.

Leave out some header files.

* Handle building on FreeBSD for i386 and amd64.
* Add stubs for the other FreeBSD arches.
* Make the file compile cleanly.

* Make this file compile cleanly.
* Add FreeBSD constructor/destructor definitions.

* Add FreeBSD action extensions.
* Get the kernel module file name rather than hard-coding it like on Solaris.
* Use FreeBSD's process library API.
* Handle FreeBSD's different lock types.
* Get the list of loaded providers via a syscall.

* Handle the different ioctl design.
* Make this file compile cleanly.

There is a lot of commented out code here which applies to future work
for userland tracing.

For now we need the basic functions to be in place so that DTrace can
be used for kernel tracing.

Leave out a header file.

Merge FreeBSD include file changes with some code that should have come
in on the vendor branch.

* Use FreeBSD's process library instead of the Solaris one.
* There are a few placeholders in here for which there isn't libproc
support code yet. This is relevent to userland tracing. This set of
commits is designed to get kernel tracing up and running, with the
userland stuff to follow later.

Leave out the hook that Solaris has into it's rtld.

* Add a couple of action definitions for FreeBSD extensions.
* Handle the different ioctl design.
* Add support for the get and set error location.
* Add support for freopen().

* Handle the different ioctl design.
* Make the file compile cleanly.

There are many places in libdtrace where errno can be set. When an
error is returned all the way back to the dtrace app, it's hard to
figure out where that error came from.

Add a couple of functions to get and set the error location which can
be optionally compiled into the library.

Add some error definitions for FreeBSD action extensions.

* Make this file compile cleanly.
* Use FreeBSD's kld syscalls to get kernel module information instead
of using an object file system like Solaris uses.

* Handle the different ioctl design.
* Use FreeBSD's process library API.
* Make the file compile cleanly.

Just leave out a few includes to compile this on FreeBSD.

* Get the maximum number of CPUs via a sysctl.
* Handle the different ioctl design.
* Support the freopen() changes.
* Use functions in FreeBSD's process library rather than the CDDL
library that Solaris has which sits on top of their process file
system and is therefore unsuitable for use on FreeBSD. The libproc
API for FreeBSD is deliberately different to that on Solaris because
Sun wouldn't release the libproc.h header under a BSD license.

* Define YY_INPUT for flex since input() can't be re-defined.
* On FreeBSD define both LITTLE_ENDIAN and BIG_ENDIAN and then set the
BYTE_ORDER to the one we are using. On Solaris they define one or
the other but not both. For us to keep using FreeBSD header files,
we need to use endian definitions the same way we do in pure BSD
code.

Make this compile cleanly.

Handle the different level of indirection between ioctl on SYSV vs BSD.

freopen() on FreeBSD behaves differently to the Solaris one.

* Use a portable POSIX timed wait.
* Handle the different ioctl format.

* Fix a function prototype.
* Allow for the different level of indirection between SYSV's ioctl and BSD's.
* Map a couple of sysconf definitions to one that FreeBSD has.

Simplify for FreeBSD for now, bypassing the multibyte char variables
that Solaris has. We may need to revisit this issue later.

Solaris has code which makes zlib optional, so they have to jump through
some hoops to get the compression functions. On FreeBSD have libz and
can just link against it.

'echo' behaviour differs between SYSV and BSD.

A lot of changes to make this code compile cleanly on FreeBSD.

Use FreeBSD's libdwarf which is BSD licensed instead of the GPL'd one that
is used in Solaris.

We need to be consistent with prototype definitions. It isn't OK to
use 'const' and just override it whenever we feel like it. If we use
it at all, then we need to do it properly.

Add a couple of functions that were useful in getting this code ported.

Make this file compile cleanly.

Apply FreeBSD changes.

On FreeBSD we can't afford to have a build fail simply because there
was some sort of CTF conversion error.

Add a missing include file. We care about function prototype definitions.

Apply FreeBSD changes.

There is an incomplete piece of code in here which requires the process
handling library which is still under development.

* Set the path to perl on FreeBSD.
* Use the FreeBSD shell.
* On FreeBSD the tests run from the OBJDIR, so output files go there
rather than in the source tree like they do on Solaris.
* FreeBSD doesn't need a special path to the compiler.

Set the path to perl on FreeBSD.

Port these test files to work under FreeBSD.

This commit was generated by cvs2svn to compensate for changes in r178528,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r178525,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r178481,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r178479,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r178476,
which included commits to RCS files with non-trunk default branches.

Add FreeBSD IDs to files that originate in FreeBSD.

Remove options MK_LIBKSE and DEFAULT_THREAD_LIB now that we no longer
build libkse. This should fix WITHOUT_LIBTHR builds as a side effect.

The sources covered by Sun's CDDL have been repo copied below the
src/cddl and src/sys/cddl directories per the core@ decision following
the license review.

This change modifies the affected Makefiles to reference the sources
in their new location.

Remove _SOLARIS_C_SOURCE now that it doesn't do anything in FreeBSD
headers. All OpenSolaris compatibility comes via the set of specific
compatibility headers in src/compat/opensolaris and
src/sys/compat/opensolaris.

Remove a couple of definitions which now exist in compatibility headers.

Add more OpenSolaris compatibilty headers.

Tweak the handling of "WITHOUT_LIBPTHREAD". Also remove the accidental
treatment of 'LIBKSE' as an "old style" knob.

Submitted by: ru
Approved by: re(kensmith)

Repo copy libpthreads to libkse.
This introduces the WITHOUT_LIBKSE nob,
and changes WITHOUT_LIBPTHREADS to mean with neither threading libs.
Approved by: re(kensmith)

Don't build bits that depend on the pthreads support if a
system was configured without such support.

Approved by: re (kensmith)

Fixed static linkage (build with -DNO_SHARED).

Approved by: re (kensmith)

Now that we have CDDLed code in the tree, add CDDL license.

Discussed with: core
Approved by: re (kensmith)

Use ioctl() to get correct media size so that we can locate
ZFS labels in the tail of the media.

Approved by: re (kensmith), pjd

- Reduce number of atomic operations needed to be implemented in asm by
implementing some of them using existing ones.
- Allow to compile ZFS on all archs and use atomic operations surrounded
by global mutex on archs we don't have or can't have all atomic
operations needed by ZFS.

Use provider's ident to handle situations when disks are moved around
and show up with different names: first try to open provider using
remembered name and compare its ident, if equal, this is our provider,
if not equal or there is no provider with such name, find provider with
remembered ident and don't care about the name.

Clean up a little.

Obtained from: OpenSolaris

Improve sharenfs option handling, so it is possible to give hosts list.
Before the change the command above:

# zfs set sharenfs=freefall.freebsd.org,69.147.83.54 tank/foo

was translated to:

/tank/foo -freefall.freebsd.org -69.147.83.54

instead of:

/tank/foo freefall.freebsd.org 69.147.83.54

This commit corrects this.

MFp4:

@118370 Correct typo.

@118371 Integrate changes from vendor.

@118491 Show backtrace on unexpected code paths.

@118494 Integrate changes from vendor.

@118504 Fix sendfile(2). I had two ways of fixing it:
1. Fixing sendfile(2) itself to use VOP_GETPAGES() instead of
hacking around with vn_rdwr(UIO_NOCOPY), which was suggested
by ups.
2. Modify ZFS behaviour to handle this special case.

Although 1 is more correct, I've choosen 2, because hack from 1
have a side-effect of beeing faster - it reads ahead MAXBSIZE
bytes instead of reading page by page. This is not easy to implement
with VOP_GETPAGES(), at least not for me in this very moment.

Reported by: Andrey V. Elsukov <bu7cher@yandex.ru>

@118525 Reorganize the code to reduce diff.

@118526 This code path is expected. It is simply when file is opened with
O_FSYNC flag.

Reported by: kris
Reported by: Michal Suszko <dry@dry.pl>

Fix build breakage. Most of ZFS code is also compiled in userland and I
should really stop forgetting about that.

Link libzpool, ztest and zdb against libpthread.

Requested by: ru

Apply a set of style.Makefile(5) changes to src/cddl/ makefiles.
There are some insignificant non-style changes as well.

Not fixed: makefiles use ${LIBTHR} that doesn't exist, thus
breaking "make checkdpadd" and not tracking dependencies
properly.

Approved by: pjd

In FreeBSD,
- cv_timedwait() takes delta instead of absolute time,
- we need ioctl() for mediasize.

Reviewed by: pjd

MFp4: Synchronize with vendor (mostly 'zfs rename -r').

MFp4: Add missing kmem_size() definition.

MFp4: Hide under '#ifdef _KERNEL' only what's really needed.

MFp4: Synchronize with recent OpenSolaris changes.

If we cannot open /dev/zfs try to load zfs.ko automatically and reopen.

Move atomic.S files to directories that better fit OpenSolaris directory
layout.

Build ZFS on amd64 and pc98.

Approved by: pjd@

Please welcome ZFS - The last word in file systems.

ZFS file system was ported from OpenSolaris operating system. The code in under
CDDL license.

I'd like to thank all SUN developers that created this great piece of software.

Supported by: Wheel LTD (http://www.wheel.pl/)
Supported by: The FreeBSD Foundation (http://www.freebsdfoundation.org/)
Supported by: Sentex (http://www.sentex.net/)