#
364843 |
|
26-Aug-2020 |
jhb |
MFC 361220: Correct the order of arguments to copyin() for Q_SETQUOTA.
|
#
351807 |
|
04-Sep-2019 |
avg |
MFC r351168: zfs_vget: fix vnode reference count leak in error path
|
#
343624 |
|
31-Jan-2019 |
sef |
MFC r342928: Change ZFS quotas to return EINVAL when not present (matches man page).
Approved by: mav Reviewed by: markj PR: 234413 Sponsored by: iXsystems Inc Reported by: Emrion <kmachine@free.fr>
|
#
339109 |
|
03-Oct-2018 |
mav |
MFC r336959: MFV r336958: 9337 zfs get all is slow due to uncached metadata
This project's goal is to make read-heavy channel programs and zfs(1m) administrative commands faster by caching all the metadata that they will need in the dbuf layer. This will prevent the data from being evicted, so that any future call to i.e. zfs get all won't have to go to disk (very much).
illumos/illumos-gate@adb52d9262f45a04318fc6e188fe2b7f59d989a5
Reviewed by: Prakash Surya <prakash.surya@delphix.com> Reviewed by: George Wilson <george.wilson@delphix.com> Reviewed by: Thomas Caputi <tcaputi@datto.com> Approved by: Richard Lowe <richlowe@richlowe.net> Author: Matthew Ahrens <mahrens@delphix.com>
|
#
339008 |
|
29-Sep-2018 |
sef |
MFC r336017,r338799
r336017 This exposes ZFS user and group quotas via the normal quatactl(2) mechanism. (Read-only at this point, however.) In particular, this is to allow rpc.rquotad query quotas for NFS mounts, allowing users to see their quotas on the hosts using the datasets.
The changes specifically:
* Add new RPC entry points for querying quotas. * Changes the library routines to allow non-UFS quotas. * Changes rquotad to check for quotas on mounted filesystems, rather than being limited to entries in /etc/fstab * Lastly, adds a VFS entry-point for ZFS to query quotas.
Note that this makes one unavoidable behavioural change: if quotas are enabled, then they can be queried, as opposed to the current method of checking for quotas being specified in fstab. (With ZFS, if there are user or group quotas, they're used, always.)
r338799 Author: kib
Fix ZFS VFS op quotactl to follow busy protocol.
Approved by: mav Sponsored by: iXsystems, inc
|
#
338975 |
|
27-Sep-2018 |
mav |
MFC r334810 (by benno), r338205, r338206: r334810: Break recursion involving getnewvnode and zfs_rmnode.
When we're at our vnode limit, getnewvnode will call into the vnode LRU cache to free up vnodes. If the vnode we try to recycle is a ZFS vnode we end up, eventually, in zfs_rmnode. If the ZFS vnode we're recycling represents something with extended attributes, zfs_rmnode will call zfs_zget which will attempt to allocate another vnode. If the next vnode we try to recycle is also a ZFS vnode representing something with extended attributes we can recurse further. This ends up being unbounded and can end up overflowing the stack.
In order to avoid this, restructure zfs_rmnode to simply add the extended attribute directory's object ID to the unlinked set, thus not requiring the allocation of a vnode. We then schedule a task that calls zfs_unlinked_drain which will do the work of properly marking the vnodes for unlinking. zfs_unlinked_drain is also called on mount so these will be cleaned up there.
r338205: Create separate taskqueue to call zfs_unlinked_drain().
r334810 introduced zfs_unlinked_drain() dispatch to taskqueue on every deletion of a file with extended attributes. Using system_taskq for that with its multiple threads in case of multiple files deletion caused all available CPU threads to uselessly spin on busy locks, completely blocking the system.
Use of single dedicated taskqueue is the only easy solution I've found, while in would be great if we could specify that some task should be executed only once at a time, but never in parallel, while many tasks could use different threads same time.
r338206: Add dmu_tx_assign() error handling in zfs_unlinked_drain().
The error handling got lost during r334810, while according to the report error there may happen in case of dataset being over quota. In such case just leave the node in the unlinked list to be freed sometimes later.
|
#
332525 |
|
16-Apr-2018 |
mav |
MFC r329732: MFV r329502: 7614 zfs device evacuation/removal
illumos/illumos-gate@5cabbc6b49070407fb9610cfe73d4c0e0dea3e77
https://www.illumos.org/issues/7614: This project allows top-level vdevs to be removed from the storage pool with “zpool remove”, reducing the total amount of storage in the pool. This operation copies all allocated regions of the device to be removed onto other devices, recording the mapping from old to new location. After the removal is complete, read and free operations to the removed (now “indirect”) vdev must be remapped and performed at the new location on disk. The indirect mapping table is kept in memory whenever the pool is loaded, so there is minimal performance overhead when doing operations on the indirect vdev.
The size of the in-memory mapping table will be reduced when its entries become “obsolete” because they are no longer used by any block pointers in the pool. An entry becomes obsolete when all the blocks that use it are freed. An entry can also become obsolete when all the snapshots that reference it are deleted, and the block pointers that reference it have been “remapped” in all filesystems/zvols (and clones). Whenever an indirect block is written, all the block pointers in it will be “remapped” to their new (concrete) locations if possible. This process can be accelerated by using the “zfs remap” command to proactively rewrite all indirect blocks that reference indirect (removed) vdevs.
Note that when a device is removed, we do not verify the checksum of the data that is copied. This makes the process much faster, but if it were used on redundant vdevs (i.e. mirror or raidz vdevs), it would be possible to copy the wrong data, when we have the correct data on e.g. the other side of the mirror. Therefore, mirror and raidz devices can not be removed.
Reviewed by: Alex Reece <alex@delphix.com> Reviewed by: George Wilson <george.wilson@delphix.com> Reviewed by: John Kennedy <john.kennedy@delphix.com> Reviewed by: Prakash Surya <prakash.surya@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Richard Laager <rlaager@wiktel.com> Reviewed by: Tim Chase <tim@chase2k.com> Approved by: Garrett D'Amore <garrett@damore.org> Author: Prashanth Sreenivasa <pks@delphix.com>
|
#
331384 |
|
22-Mar-2018 |
mav |
MFC r329625: MFV r307315: 7301 zpool export -f should be able to interrupt file freeing
Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Sanjay Nadkarni <sanjay.nadkarni@nexenta.com> Reviewed by: Saso Kiselkov <saso.kiselkov@nexenta.com> Reviewed by: John Kennedy <john.kennedy@delphix.com> Author: Alek Pinchuk <alek@nexenta.com>
Closes #175
|
#
328049 |
|
16-Jan-2018 |
avg |
MFC r327725: zfs_mount: restore a bit of ifdef-out illumos code
|
#
325534 |
|
08-Nov-2017 |
avg |
MFC r324163: MFV r323530,r323533,r323534: 7431 ZFS Channel Programs, and followups
Also MFC-ed are the following fixes: - r324164 Add several new files to the files enabled by ZFS kernel option - r324178 unbreak kernel builds on sparc64 and powerpc - r324194 fix incorrect use of getzfsvfs_impl in r324163 - r324292 really unbreak kernel builds on sparc64 and powerpc64
|
#
321578 |
|
26-Jul-2017 |
mav |
MFC r319949: MFV r319948: 5428 provide fts(), reallocarray(), and strtonum()
illumos/illumos-gate@4585130b259133a26efae68275dbe56b08366deb https://github.com/illumos/illumos-gate/commit/4585130b259133a26efae68275dbe56b08366deb
https://www.illumos.org/issues/5428
Most of the upstream change is not applicable to FreeBSD. Only the renaming of strtonum to zfs_strtonum is relevant to us. And we already had it partially done.
Reviewed by: Robert Mustacchi <rm@joyent.com> Approved by: Joshua M. Clulow <josh@sysmgr.org> Author: Yuri Pankov <yuri.pankov@nexenta.com>
|
#
321556 |
|
26-Jul-2017 |
mav |
MFC r318833: MFV r316925: 6101 attempt to lzc_create() a filesystem under a volume results in a panic
illumos/illumos-gate@b127fe3c059af7adf772735498680b4f2e1405ef https://github.com/illumos/illumos-gate/commit/b127fe3c059af7adf772735498680b4f2e1405ef
https://www.illumos.org/issues/6101 lzc_create(), or more correctly, zfs_ioc_create() does not reject an attempt to create a filesystem as a child of a volume, instead it proceeds to a crash. A crash stack obtained on FreeBSD: page fault while in kernel mode
zap_leaf_lookup() fzap_lookup() zap_lookup_norm() zap_lookup() zfs_get_zplprop() zfs_fill_zplprops_impl() zfs_ioc_create() zfsdev_ioctl() devfs_ioctl_f() kern_ioctl() sys_ioctl() This crash happened with a kernel without debugging assertions. The immediate cause of crash appears to an attempt to interpret a zvol object as a zap object. For filesystems: #define MASTER_NODE_OBJ 1 For zvols: #define ZVOL_OBJ 1ULL #define ZVOL_ZAP_OBJ 2ULL So, I see two problems here: 1. an attempt to create a filesystem under a zvol should be rejected as early as possible, maybe in zfs_fill_zplprops() 2. maybe zap_lookup / zap_lockdir should reject objects that are not of one of the zap object types
Reviewed by: Matthew Ahrens <mahrens@delphix.com> Approved by: Dan McDonald <danmcd@omniti.com> Author: Andriy Gapon <avg@FreeBSD.org>
|
#
315842 |
|
23-Mar-2017 |
avg |
MFC r314048,r314194: reimplement zfsctl (.zfs) support
|
#
310959 |
|
31-Dec-2016 |
mjg |
MFC r305378,r305379,r305386,r305684,r306224,r306608,r306803,r307650,r307685, r308407,r308665,r308667,r309067:
cache: put all negative entry management code into dedicated functions
== cache: manage negative entry list with a dedicated lock
Since negative entries are managed with a LRU list, a hit requires a modificaton.
Currently the code tries to upgrade the global lock if needed and is forced to retry the lookup if it fails.
Provide a dedicated lock for use when the cache is only shared-locked.
==
cache: defer freeing entries until after the global lock is dropped
This also defers vdrop for held vnodes.
==
cache: improve scalability by introducing bucket locks
An array of bucket locks is added.
All modifications still require the global cache_lock to be held for writing. However, most readers only need the relevant bucket lock and in effect can run concurrently to the writer as long as they use a different lock. See the added comment for more details.
This is an intermediate step towards removal of the global lock.
==
cache: get rid of the global lock
Add a table of vnode locks and use them along with bucketlocks to provide concurrent modification support. The approach taken is to preserve the current behaviour of the namecache and just lock all relevant parts before any changes are made.
Lookups still require the relevant bucket to be locked.
==
cache: ignore purgevfs requests for filesystems with few vnodes
purgevfs is purely optional and induces lock contention in workloads which frequently mount and unmount filesystems.
In particular, poudriere will do this for filesystems with 4 vnodes or less. Full cache scan is clearly wasteful.
Since there is no explicit counter for namecache entries, the number of vnodes used by the target fs is checked.
The default limit is the number of bucket locks.
== (by kib)
Limit scope of the optimization in r306608 to dounmount() caller only. Other uses of cache_purgevfs() do rely on the cache purge for correct operations, when paths are invalidated without unmount.
==
cache: split negative entry LRU into multiple lists
This splits the ncneg_mtx lock while preserving the hit ratio at least during buildworld.
Create N dedicated lists for new negative entries.
Entries with at least one hit get promoted to the hot list, where they get requeued every M hits.
Shrinking demotes one hot entry and performs a round-robin shrinking of regular lists.
==
cache: fix up a corner case in r307650
If no negative entry is found on the last list, the ncp pointer will be left uninitialized and a non-null value will make the function assume an entry was found.
Fix the problem by initializing to NULL on entry.
== (by kib)
vn_fullpath1() checked VV_ROOT and then unreferenced vp->v_mount->mnt_vnodecovered unlocked. This allowed unmount to race. Lock vnode after we noticed the VV_ROOT flag. See comments for explanation why unlocked check for the flag is considered safe.
==
cache: fix a race between entry removal and demotion
The negative list shrinker can demote an entry with only hotlist + neglist locks held. On the other hand entry removal possibly sets the NCF_DVDROP without aformentioned locks held prior to detaching it from the respective netlist., which can lose the update made by the shrinker.
==
cache: plug a write-only variable in cache_negative_zap_one
==
cache: ensure that the number of bucket locks does not exceed hash size
The size can be changed by side effect of modifying kern.maxvnodes.
Since numbucketlocks was not modified, setting a sufficiently low value would give more locks than actual buckets, which would then lead to corruption.
Force the number of buckets to be not smaller.
Note this should not matter for real world cases.
|
#
310513 |
|
24-Dec-2016 |
avg |
MFC r309099: MFV r308990: 7181 race between zfs_mount and zfs_ioc_rollback
|
#
310509 |
|
24-Dec-2016 |
avg |
MFC r309097: MFV r308987: 7180 potential race between zfs_suspend_fs+zfs_resume_fs and zfs_ioc_rename
|
#
307108 |
|
11-Oct-2016 |
mav |
MFC r305209: MFV r302660: 6314 buffer overflow in dsl_dataset_name
illumos/illumos-gate@9adfa60d484ce2435f5af77cc99dcd4e692b6660 https://github.com/illumos/illumos-gate/commit/9adfa60d484ce2435f5af77cc99dcd4e6 92b6660
https://www.illumos.org/issues/6314 Callers of dsl_dataset_name pass a buffer of size ZFS_MAXNAMELEN, but dsl_dataset_name copies the datasets' name PLUS the snapshot name to it, resulting in a max of 2 * ZFS_MAXNAMELEN + '@'.
Reviewed by: George Wilson <george.wilson@delphix.com> Reviewed by: Prakash Surya <prakash.surya@delphix.com> Reviewed by: Igor Kozhukhov <ikozhukhov@gmail.com> Approved by: Dan McDonald <danmcd@omniti.com> Author: Matthew Ahrens <mahrens@delphix.com>
|
#
303970 |
|
11-Aug-2016 |
avg |
MFC r303763,303791,303869: zfs: honour and make use of vfs vnode locking protocol
ZFS POSIX Layer is originally written for Solaris VFS which is very different from FreeBSD VFS. Most importantly many things that FreeBSD VFS manages on behalf of all filesystems are implemented in ZPL in a different way. Thus, ZPL contains code that is redundant on FreeBSD or duplicates VFS functionality or, in the worst cases, badly interacts / interferes with VFS.
The most prominent problem is a deadlock caused by the lock order reversal of vnode locks that may happen with concurrent zfs_rename() and lookup(). The deadlock is a result of zfs_rename() not observing the vnode locking contract expected by VFS.
This commit removes all ZPL internal locking that protects parent-child relationships of filesystem nodes. These relationships are protected by vnode locks and the code is changed to take advantage of that fact and to properly interact with VFS.
Removal of the internal locking allowed all ZPL dmu_tx_assign calls to use TXG_WAIT mode.
Another victim, disputable perhaps, is ZFS support for filesystems with mixed case sensitivity. That support is not provided by the OS anyway, so in ZFS it was a buch of dead code.
To do: - replace ZFS_ENTER mechanism with VFS managed / visible mechanism - replace zfs_zget with zfs_vget[f] as much as possible - get rid of not really useful now zfs_freebsd_* adapters - more cleanups of unneeded / unused code - fix / replace .zfs support
PR: 209158 Approved by: re (gjb)
|
#
303969 |
|
11-Aug-2016 |
avg |
MFC r302836: 6874 rollback and receive need to reset ZPL state to what's on disk
Approved by: re (gjb)
|