#
719fcafe |
|
11-Mar-2024 |
Josef Bacik <josef@toxicpanda.com> |
nfs: fix panic when nfs4_ff_layout_prepare_ds() fails We've been seeing the following panic in production BUG: kernel NULL pointer dereference, address: 0000000000000065 PGD 2f485f067 P4D 2f485f067 PUD 2cc5d8067 PMD 0 RIP: 0010:ff_layout_cancel_io+0x3a/0x90 [nfs_layout_flexfiles] Call Trace: <TASK> ? __die+0x78/0xc0 ? page_fault_oops+0x286/0x380 ? __rpc_execute+0x2c3/0x470 [sunrpc] ? rpc_new_task+0x42/0x1c0 [sunrpc] ? exc_page_fault+0x5d/0x110 ? asm_exc_page_fault+0x22/0x30 ? ff_layout_free_layoutreturn+0x110/0x110 [nfs_layout_flexfiles] ? ff_layout_cancel_io+0x3a/0x90 [nfs_layout_flexfiles] ? ff_layout_cancel_io+0x6f/0x90 [nfs_layout_flexfiles] pnfs_mark_matching_lsegs_return+0x1b0/0x360 [nfsv4] pnfs_error_mark_layout_for_return+0x9e/0x110 [nfsv4] ? ff_layout_send_layouterror+0x50/0x160 [nfs_layout_flexfiles] nfs4_ff_layout_prepare_ds+0x11f/0x290 [nfs_layout_flexfiles] ff_layout_pg_init_write+0xf0/0x1f0 [nfs_layout_flexfiles] __nfs_pageio_add_request+0x154/0x6c0 [nfs] nfs_pageio_add_request+0x26b/0x380 [nfs] nfs_do_writepage+0x111/0x1e0 [nfs] nfs_writepages_callback+0xf/0x30 [nfs] write_cache_pages+0x17f/0x380 ? nfs_pageio_init_write+0x50/0x50 [nfs] ? nfs_writepages+0x6d/0x210 [nfs] ? nfs_writepages+0x6d/0x210 [nfs] nfs_writepages+0x125/0x210 [nfs] do_writepages+0x67/0x220 ? generic_perform_write+0x14b/0x210 filemap_fdatawrite_wbc+0x5b/0x80 file_write_and_wait_range+0x6d/0xc0 nfs_file_fsync+0x81/0x170 [nfs] ? nfs_file_mmap+0x60/0x60 [nfs] __x64_sys_fsync+0x53/0x90 do_syscall_64+0x3d/0x90 entry_SYSCALL_64_after_hwframe+0x46/0xb0 Inspecting the core with drgn I was able to pull this >>> prog.crashed_thread().stack_trace()[0] #0 at 0xffffffffa079657a (ff_layout_cancel_io+0x3a/0x84) in ff_layout_cancel_io at fs/nfs/flexfilelayout/flexfilelayout.c:2021:27 >>> prog.crashed_thread().stack_trace()[0]['idx'] (u32)1 >>> prog.crashed_thread().stack_trace()[0]['flseg'].mirror_array[1].mirror_ds (struct nfs4_ff_layout_ds *)0xffffffffffffffed This is clear from the stack trace, we call nfs4_ff_layout_prepare_ds() which could error out initializing the mirror_ds, and then we go to clean it all up and our check is only for if (!mirror->mirror_ds). This is inconsistent with the rest of the users of mirror_ds, which have if (IS_ERR_OR_NULL(mirror_ds)) to keep from tripping over this exact scenario. Fix this up in ff_layout_cancel_io() to make sure we don't panic when we get an error. I also spot checked all the other instances of checking mirror_ds and we appear to be doing the correct checks everywhere, only unconditionally dereferencing mirror_ds when we know it would be valid. Signed-off-by: Josef Bacik <josef@toxicpanda.com> Fixes: b739a5bd9d9f ("NFSv4/flexfiles: Cancel I/O if the layout is recalled or revoked") Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
e1c6cfbb |
|
08-Oct-2023 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
pNFS/flexfiles: Check the layout validity in ff_layout_mirror_prepare_stats Ensure that we check the layout pointer and validity after dereferencing it in ff_layout_mirror_prepare_stats. Fixes: 08e2e5bc6c9a ("pNFS/flexfiles: Clean up layoutstats") Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
dd7d7ee3 |
|
03-Sep-2023 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
NFS/pNFS: Report EINVAL errors from connect() to the server With IPv6, connect() can occasionally return EINVAL if a route is unavailable. If this happens during I/O to a data server, we want to report it using LAYOUTERROR as an inability to connect. Fixes: dd52128afdde ("NFSv4.1/pnfs Ensure flexfiles reports all connection related errors") Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
5a17f040 |
|
26-Oct-2022 |
Kees Cook <keescook@chromium.org> |
cred: Do not default to init_cred in prepare_kernel_cred() A common exploit pattern for ROP attacks is to abuse prepare_kernel_cred() in order to construct escalated privileges[1]. Instead of providing a short-hand argument (NULL) to the "daemon" argument to indicate using init_cred as the base cred, require that "daemon" is always set to an actual task. Replace all existing callers that were passing NULL with &init_task. Future attacks will need to have sufficiently powerful read/write primitives to have found an appropriately privileged task and written it to the ROP stack as an argument to succeed, which is similarly difficult to the prior effort needed to escalate privileges before struct cred existed: locate the current cred and overwrite the uid member. This has the added benefit of meaning that prepare_kernel_cred() can no longer exceed the privileges of the init task, which may have changed from the original init_cred (e.g. dropping capabilities from the bounding set). [1] https://google.com/search?q=commit_creds(prepare_kernel_cred(0)) Cc: "Eric W. Biederman" <ebiederm@xmission.com> Cc: David Howells <dhowells@redhat.com> Cc: "Rafael J. Wysocki" <rafael@kernel.org> Cc: Steve French <sfrench@samba.org> Cc: Ronnie Sahlberg <lsahlber@redhat.com> Cc: Shyam Prasad N <sprasad@microsoft.com> Cc: Tom Talpey <tom@talpey.com> Cc: Namjae Jeon <linkinjeon@kernel.org> Cc: Trond Myklebust <trond.myklebust@hammerspace.com> Cc: Anna Schumaker <anna@kernel.org> Cc: Chuck Lever <chuck.lever@oracle.com> Cc: Jeff Layton <jlayton@kernel.org> Cc: "David S. Miller" <davem@davemloft.net> Cc: Eric Dumazet <edumazet@google.com> Cc: Jakub Kicinski <kuba@kernel.org> Cc: Paolo Abeni <pabeni@redhat.com> Cc: "Michal Koutný" <mkoutny@suse.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: linux-cifs@vger.kernel.org Cc: samba-technical@lists.samba.org Cc: linux-nfs@vger.kernel.org Cc: netdev@vger.kernel.org Signed-off-by: Kees Cook <keescook@chromium.org> Acked-by: Luis Chamberlain <mcgrof@kernel.org> Reviewed-by: Sergey Senozhatsky <senozhatsky@chromium.org> Acked-by: Russ Weight <russell.h.weight@intel.com> Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Acked-by: Paulo Alcantara (SUSE) <pc@cjr.nz> Link: https://lore.kernel.org/r/20221026232943.never.775-kees@kernel.org
|
#
b739a5bd |
|
05-Oct-2022 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
NFSv4/flexfiles: Cancel I/O if the layout is recalled or revoked If the layout is recalled or revoked, we want to cancel I/O as quickly as possible so that we can return the layout. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
90377158 |
|
24-Aug-2022 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
NFSv4/pNFS: Always return layout stats on layout return for flexfiles We want to ensure that the server never misses the layout stats when we're closing the file, so that it knows whether or not to update its internal state. Otherwise, if we were racing with a layout stat, we might cause the server to invalidate its layout before the layout stat got processed. Fixes: 06946c6a3d8b ("pNFS/flexfiles: Only send layoutstats updates for mirrors that were updated") Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
7836d754 |
|
18-May-2022 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
pNFS/flexfiles: Report RDMA connection errors to the server The RPC/RDMA driver will return -EPROTO and -ENODEV as connection errors under certain circumstances. Make sure that we handle them and report them to the server. If not, we can end up cycling forever in a LAYOUTGET/LAYOUTRETURN loop. Fixes: a12f996d3413 ("NFSv4/pNFS: Use connections to a DS that are all of the same protocol family") Cc: stable@vger.kernel.org # 5.11.x Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
3e5f151e |
|
21-Mar-2022 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
pNFS/flexfiles: Ensure pNFS allocation modes are consistent with nfsiod Ensure that pNFS flexfile allocations in rpciod/nfsiod callbacks can fail in low memory mode, so that the threads don't block and loop forever. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
61345a42 |
|
29-Jan-2022 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
NFSv4/flexfiles: Convert GFP_NOFS to GFP_KERNEL Assume that the higher layers will have set memalloc_nofs_save/restore as appropriate. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
b40887e1 |
|
16-Oct-2021 |
Chuck Lever <chuck.lever@oracle.com> |
SUNRPC: Trace calls to .rpc_call_done Introduce a single tracepoint that can replace simple dprintk call sites in upper layer "rpc_call_done" callbacks. Example: kworker/u24:2-1254 [001] 771.026677: rpc_stats_latency: task:00000001@00000002 xid=0x16a6f3c0 rpcbindv2 GETPORT backlog=446 rtt=101 execute=555 kworker/u24:2-1254 [001] 771.026677: rpc_task_call_done: task:00000001@00000002 flags=ASYNC|DYNAMIC|SOFT|SOFTCONN|SENT runstate=RUNNING|ACTIVE status=0 action=rpcb_getport_done kworker/u24:2-1254 [001] 771.026678: rpcb_setport: task:00000001@00000002 status=0 port=20048 Signed-off-by: Chuck Lever <chuck.lever@oracle.com> Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
ed34695e |
|
29-Mar-2021 |
Nikola Livic <nlivic@gmail.com> |
pNFS/flexfiles: fix incorrect size check in decode_nfs_fh() We (adam zabrocki, alexander matrosov, alexander tereshkin, maksym bazalii) observed the check: if (fh->size > sizeof(struct nfs_fh)) should not use the size of the nfs_fh struct which includes an extra two bytes from the size field. struct nfs_fh { unsigned short size; unsigned char data[NFS_MAXFHSIZE]; } but should determine the size from data[NFS_MAXFHSIZE] so the memcpy will not write 2 bytes beyond destination. The proposed fix is to compare against the NFS_MAXFHSIZE directly, as is done elsewhere in fs code base. Fixes: d67ae825a59d ("pnfs/flexfiles: Add the FlexFile Layout Driver") Signed-off-by: Nikola Livic <nlivic@gmail.com> Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com> Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
52104f27 |
|
16-Dec-2020 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
NFS/pNFS: Fix a typo in ff_layout_resend_pnfs_read() Don't bump the index twice. Fixes: 563c53e73b8b ("NFS: Fix flexfiles read failover") Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
9bfffea3 |
|
16-Dec-2020 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
pNFS/flexfiles: Avoid spurious layout returns in ff_layout_choose_ds_for_read The callers of ff_layout_choose_ds_for_read() should decide whether or not they want to return the layout on error. Sometimes, we may just want to retry from the beginning. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
9a701631 |
|
09-Nov-2020 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
pNFS/flexfiles: Fix up layoutstats reporting for non-TCP transports Ensure that we report the correct netid when using UDP or RDMA transports to the DSes. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
0ae4c3e8 |
|
11-Nov-2020 |
Chuck Lever <chuck.lever@oracle.com> |
SUNRPC: Add xdr_set_scratch_page() and xdr_reset_scratch_buffer() Clean up: De-duplicate some frequently-used code. Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
|
#
63e2fffa |
|
15-Nov-2020 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
pNFS/flexfiles: Fix array overflow when flexfiles mirroring is enabled If the flexfiles mirroring is enabled, then the read code expects to be able to set pgio->pg_mirror_idx to point to the data server that is being used for this particular read. However it does not change the pg_mirror_count because we only need to send a single read. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
c754e137 |
|
18-Sep-2020 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
pNFS/flexfiles: Be consistent about mirror index types A mirror index is always of type u32. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
c0a1d129 |
|
18-Sep-2020 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
pNFS/flexfiles: Ensure we initialise the mirror bsizes correctly on read While it is true that reading from an unmirrored source always uses index 0, that is no longer true for mirrored sources when we fail over. Fixes: 563c53e73b8b ("NFS: Fix flexfiles read failover") Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
b9df46d0 |
|
18-Sep-2020 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
pNFS/flexfiles: Be consistent about mirror index types A mirror index is always of type u32. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
ee15c7b5 |
|
18-Sep-2020 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
pNFS/flexfiles: Ensure we initialise the mirror bsizes correctly on read While it is true that reading from an unmirrored source always uses index 0, that is no longer true for mirrored sources when we fail over. Fixes: 563c53e73b8b ("NFS: Fix flexfiles read failover") Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
df561f66 |
|
23-Aug-2020 |
Gustavo A. R. Silva <gustavoars@kernel.org> |
treewide: Use fallthrough pseudo-keyword Replace the existing /* fall through */ comments and its variants with the new pseudo-keyword macro fallthrough[1]. Also, remove unnecessary fall-through markings when it is the case. [1] https://www.kernel.org/doc/html/v5.7/process/deprecated.html?highlight=fallthrough#implicit-switch-case-fall-through Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org>
|
#
563c53e7 |
|
11-Aug-2020 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
NFS: Fix flexfiles read failover The current mirrored read failover code is correctly resetting the mirror index between failed reads, however it is not able to actually flip the RPC call over to the next RPC client. The end result is that we keep resending the RPC call to the same client over and over. The fix is to use the pnfs_read_resend_pnfs() mechanism to schedule a new RPC call, but we need to add the ability to pass in a mirror index so that we always retry the next mirror in the list. Fixes: 166bd5b889ac ("pNFS/flexfiles: Fix layoutstats handling during read failovers") Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
18eb87f4 |
|
18-Jun-2020 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
pNFS/flexfiles: The mirror count could depend on the layout segment range Make sure we specify the layout segment range when calculating the mirror count. In theory, that number could depend on the range to which we're writing. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
f97ff92b |
|
18-Jun-2020 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
pNFS/flexfiles: Clean up redundant calls to pnfs_put_lseg() Both nfs_pageio_reset_read_mds() and nfs_pageio_reset_write_mds() do call pnfs_generic_pg_cleanup() for us. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
8b040137 |
|
22-Jun-2020 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
pNFS/flexfiles: Fix list corruption if the mirror count changes If the mirror count changes in the new layout we pick up inside ff_layout_pg_init_write(), then we can end up adding the request to the wrong mirror and corrupting the mirror->pg_list. Fixes: d600ad1f2bdb ("NFS41: pop some layoutget errors to application") Cc: stable@vger.kernel.org Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
cbd7be43 |
|
23-Mar-2020 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
pNFS/flexfiles: Specify the layout segment range in LAYOUTGET Move from requesting only full file layout segments, to requesting layout segments that match our I/O size. This means the server is still free to return a full file layout, but we will no longer error out if it does not. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
e70430d9 |
|
23-Mar-2020 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
pNFS/flexfiles: remove requirement for whole file layouts Remove the requirement that the server always sends whole file layouts. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
e1e54ab7 |
|
23-Mar-2020 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
pNFS/flexfiles: Check the layout segment range before doing I/O When starting to read or write with a layout segment, check that the range matches our request. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
660d1eb2 |
|
23-Mar-2020 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
pNFS/flexfile: Don't merge layout segments if the mirrors don't match Check that the number of mirrors, and the mirror information matches before deciding to merge layout segments in pNFS/flexfiles. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
9c455a8c |
|
21-Mar-2020 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
NFS/pNFS: Clean up pNFS commit operations Move the pNFS commit related operations into a separate structure that can be carried by the pnfs_ds_commit_info. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
0aa647b7 |
|
21-Mar-2020 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
NFS: Remove bucket array from struct pnfs_ds_commit_info Remove the unused bucket array in struct pnfs_ds_commit_info. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
ba827c9a |
|
20-Mar-2020 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
pNFS: Enable per-layout segment commit structures Enable adding and lookup of per-layout segment commits in filelayout and flexfilelayout. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
a9901899 |
|
20-Mar-2020 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
pNFS: Add infrastructure for cleaning up per-layout commit structures Ensure that both the file and flexfiles layout types clean up when freeing the layout segments. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
c21e7168 |
|
19-Mar-2020 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
NFSv4/pnfs: Support a list of commit arrays in struct pnfs_ds_commit_info When we have multiple layout segments with different lists of mirrored data, we need to track the commits on a per layout segment basis. This patch adds a list to support this tracking in struct pnfs_ds_commit_info. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
329651b1 |
|
19-Mar-2020 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
pNFS/flexfiles: Simplify allocation of the mirror array Just allocate the array at the end of the layout segment structure, instead of allocating it as a separate array of pointers. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
cf6605d1 |
|
18-Feb-2020 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
NFSv4: Ensure layout headers are RCU safe Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
194a0dc8 |
|
10-Feb-2020 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
pNFS/flexfiles: Report DELAY and GRACE errors from the DS to the server Ensure that if the DS is returning too many DELAY and GRACE errors, we also report that to the MDS through the layouterror mechanism. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
088f3e68 |
|
06-Jan-2020 |
Trond Myklebust <trondmy@gmail.com> |
pNFS/flexfiles: Add tracing for layout errors Trace layout errors for pNFS/flexfiles on read/write/commit operations. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
0722dc9f |
|
06-Jan-2020 |
Trond Myklebust <trondmy@gmail.com> |
pNFS/flexfiles: Record resend attempts on I/O failure If the attempt to do pNFS fails, then record what action we take to recover (resend, reset to pnfs or reset to mds). Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
7af46292 |
|
23-Aug-2019 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
pNFS/flexfiles: Don't time out requests on hard mounts If the mount is hard, we should ignore the 'io_maxretrans' module parameter so that we always keep retrying. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
d5711920 |
|
16-Aug-2019 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
Revert "NFSv4/flexfiles: Abort I/O early if the layout segment was invalidated" This reverts commit a79f194aa4879e9baad118c3f8bb2ca24dbef765. The mechanism for aborting I/O is racy, since we are not guaranteed that the request is asleep while we're changing both task->tk_status and task->tk_action. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> Cc: stable@vger.kernel.org # v5.1
|
#
d5b9216f |
|
18-Jul-2019 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
pnfs/flexfiles: Add tracepoints for detecting pnfs fallback to MDS Add tracepoints to allow debugging of the event chain leading to a pnfs fallback to doing I/O through the MDS. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
09c434b8 |
|
19-May-2019 |
Thomas Gleixner <tglx@linutronix.de> |
treewide: Add SPDX license identifier for more missed files Add SPDX license identifiers to all files which: - Have no license information of any form - Have MODULE_LICENCE("GPL*") inside which was used in the initial scan/conversion to ignore the file These files fall under the project license, GPL v2 only. The resulting SPDX license identifier is: GPL-2.0-only Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
|
#
9fcd5960 |
|
07-Apr-2019 |
Trond Myklebust <trondmy@gmail.com> |
NFS: Add a helper to return a pointer to the open context of a struct nfs_page Add a helper for when we remove the explicit pointer to the open context. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
33344e0f |
|
07-Apr-2019 |
Trond Myklebust <trondmy@gmail.com> |
pNFS: Add tracking to limit the number of pNFS retries When the client is reading or writing using pNFS, and hits an error on the DS, then it typically sends a LAYOUTERROR and/or LAYOUTRETURN to the MDS, before redirtying the failed pages, and going for a new round of reads/writebacks. The problem is that if the server has no way to fix the DS, then we may need a way to interrupt this loop after a set number of attempts have been made. This patch adds an optional module parameter that allows the admin to specify how many times to retry the read/writeback process before failing with a fatal error. The default behaviour is to retry forever. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
166bd5b8 |
|
22-Mar-2019 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
pNFS/flexfiles: Fix layoutstats handling during read failovers During a read failover, we may end up changing the value of the pgio_mirror_idx, so make sure that we record the layout stats before that update. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
4cbc8a57 |
|
28-Feb-2019 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
NFS/flexfile: Simplify nfs4_ff_layout_select_ds_stateid() Pass in a pointer to the mirror rather than forcing another array access. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
626d48b1 |
|
28-Feb-2019 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
NFS/flexfile: Simplify nfs4_ff_layout_ds_version() Pass in a pointer to the mirror rather than forcing another array access. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
312cd4cb |
|
28-Feb-2019 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
NFS/flexfiles: Simplify ff_layout_get_ds_cred() Pass in a pointer to the mirror rather than forcing another array access. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
561d6f8a |
|
28-Feb-2019 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
NFS/flexfiles: Simplify nfs4_ff_find_or_create_ds_client() Pass in a pointer to the mirror rather than forcing another array access. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
749da527 |
|
28-Feb-2019 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
NFS/flexfiles: Simplify nfs4_ff_layout_select_ds_fh() Pass in a pointer to the mirror rather than having to retrieve it from the array and then verify the resulting pointer. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
76c66905 |
|
14-Feb-2019 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
NFS/flexfiles: Speed up read failover when DSes are down If we notice that a DS may be down, we should attempt to read from the other mirrors first before we go back to retry the dead DS. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
d082d4b5 |
|
25-Feb-2019 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
NFS/flexfiles: Remove bogus checks for invalid deviceids We already check the deviceids before we start the RPC call. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
2444ff27 |
|
14-Feb-2019 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
NFS/flexfiles: refactor calls to fs4_ff_layout_prepare_ds() While we may want to skip attempting to connect to a downed mirror when we're deciding which mirror to select for a read, we do not want to do so once we've committed to attempting the I/O in ff_layout_read/write_pagelist(), or ff_layout_initiate_commit() Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
f0922a6c |
|
10-Feb-2019 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
NFS/flexfiles: Send LAYOUTERROR when failing over mirrored reads When a read to the preferred mirror returns an error, the flexfiles driver records the error in the inode list and currently marks the layout for return before failing over the attempted read to the next mirror. What we actually want to do is fire off a LAYOUTERROR to notify the MDS that there is an issue with the preferred mirror, then we fail over. Only once we've failed to read from all mirrors should we return the layout. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
a79f194a |
|
27-Feb-2019 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
NFSv4/flexfiles: Abort I/O early if the layout segment was invalidated If a layout segment gets invalidated while a pNFS I/O operation is queued for transmission, then we ideally want to abort immediately. This is particularly the case when there is a large number of I/O related RPCs queued in the RPC layer, and the layout segment gets invalidated due to an ENOSPC error, or an EACCES (because the client was fenced). We may end up forced to spam the MDS with a lot of otherwise unnecessary LAYOUTERRORs after that I/O fails. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
762bb7e9 |
|
26-Feb-2019 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
NFS/flexfiles: Fix up sparse RCU annotations Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
0ccc61b1 |
|
11-Feb-2019 |
Chuck Lever <chuck.lever@oracle.com> |
SUNRPC: Add xdr_stream::rqst field Having access to the controlling rpc_rqst means a trace point in the XDR code can report: - the XID - the task ID and client ID - the p_name of RPC being processed Subsequent patches will introduce such trace points. Signed-off-by: Chuck Lever <chuck.lever@oracle.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
a52458b4 |
|
02-Dec-2018 |
NeilBrown <neilb@suse.com> |
NFS/NFSD/SUNRPC: replace generic creds with 'struct cred'. SUNRPC has two sorts of credentials, both of which appear as "struct rpc_cred". There are "generic credentials" which are supplied by clients such as NFS and passed in 'struct rpc_message' to indicate which user should be used to authorize the request, and there are low-level credentials such as AUTH_NULL, AUTH_UNIX, AUTH_GSS which describe the credential to be sent over the wires. This patch replaces all the generic credentials by 'struct cred' pointers - the credential structure used throughout Linux. For machine credentials, there is a special 'struct cred *' pointer which is statically allocated and recognized where needed as having a special meaning. A look-up of a low-level cred will map this to a machine credential. Signed-off-by: NeilBrown <neilb@suse.com> Acked-by: J. Bruce Fields <bfields@redhat.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
8276c902 |
|
02-Dec-2018 |
NeilBrown <neilb@suse.com> |
SUNRPC: remove uid and gid from struct auth_cred Use cred->fsuid and cred->fsgid instead. Signed-off-by: NeilBrown <neilb@suse.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
fc0664fd |
|
02-Dec-2018 |
NeilBrown <neilb@suse.com> |
SUNRPC: remove groupinfo from struct auth_cred. We can use cred->groupinfo (from the 'struct cred') instead. Signed-off-by: NeilBrown <neilb@suse.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
97f68c6b |
|
02-Dec-2018 |
NeilBrown <neilb@suse.com> |
SUNRPC: add 'struct cred *' to auth_cred and rpc_cred The SUNRPC credential framework was put together before Linux has 'struct cred'. Now that we have it, it makes sense to use it. This first step just includes a suitable 'struct cred *' pointer in every 'struct auth_cred' and almost every 'struct rpc_cred'. The rpc_cred used for auth_null has a NULL 'struct cred *' as nothing else really makes sense. For rpc_cred, the pointer is reference counted. For auth_cred it isn't. struct auth_cred are either allocated on the stack, in which case the thread owns a reference to the auth, or are part of 'struct generic_cred' in which case gc_base owns the reference, and "acred" shares it. Signed-off-by: NeilBrown <neilb@suse.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
320f35b7 |
|
26-Nov-2018 |
Tigran Mkrtchyan <tigran.mkrtchyan@desy.de> |
flexfiles: enforce per-mirror stateid only for v4 DSes Since commit bb21ce0ad227 we always enforce per-mirror stateid. However, this makes sense only for v4+ servers. Signed-off-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de> Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
bb21ce0a |
|
20-Nov-2018 |
Tigran Mkrtchyan <tigran.mkrtchyan@desy.de> |
flexfiles: use per-mirror specified stateid for IO rfc8435 says: For tight coupling, ffds_stateid provides the stateid to be used by the client to access the file. However current implementation replaces per-mirror provided stateid with by open or lock stateid. Ensure that per-mirror stateid is used by ff_layout_write_prepare_v4 and nfs4_ff_layout_prepare_ds. Signed-off-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de> Signed-off-by: Rick Macklem <rmacklem@uoguelph.ca> Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
28ced9a8 |
|
03-Sep-2018 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
pNFS: Don't allocate more pages than we need to fit a layoutget response For the 'files' and 'flexfiles' layout types, we do not expect the reply to be any larger than 4k. The block and scsi layout types are a little more greedy, so we keep allocating the maximum response size for now. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
1c1aeaf1 |
|
21-Aug-2018 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfiles: ff_layout_pg_init_read should exit on error If we get an error while retrieving the layout, then we should report it rather than falling back to I/O through the MDS. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
056f9ad6 |
|
23-Jun-2018 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
pNFS/flexfiles: Ensure we always return a layout if it has layoutstats If a layout segment is carrying layoutstats or layout error information, then we always want to return it rather than using a forgetful model. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
7b0df92a |
|
18-Jun-2018 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
pNFS/flexfiles: Process writeback resends from nfsiod context as well Although the writeback resends are more robust than the reads, since they are not immediately rescheduled by the same thread, we are better off processing them in the same place as the reads. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
42f86b44 |
|
18-Jun-2018 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
pNFS/flexfiles: Don't tie up all the rpciod threads in resends We do not want to have rpciod threads perform recursive calls into the RPC layer since that can deadlock. In particular, having to wait for a layoutget can be nasty... We want rather to defer scheduling those retries until we're in the rpc_release() callback, since that is called from the nfsiod workqueue. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
6396bb22 |
|
12-Jun-2018 |
Kees Cook <keescook@chromium.org> |
treewide: kzalloc() -> kcalloc() The kzalloc() function has a 2-factor argument form, kcalloc(). This patch replaces cases of: kzalloc(a * b, gfp) with: kcalloc(a * b, gfp) as well as handling cases of: kzalloc(a * b * c, gfp) with: kzalloc(array3_size(a, b, c), gfp) as it's slightly less ugly than: kzalloc_array(array_size(a, b), c, gfp) This does, however, attempt to ignore constant size factors like: kzalloc(4 * 1024, gfp) though any constants defined via macros get caught up in the conversion. Any factors with a sizeof() of "unsigned char", "char", and "u8" were dropped, since they're redundant. The Coccinelle script used for this was: // Fix redundant parens around sizeof(). @@ type TYPE; expression THING, E; @@ ( kzalloc( - (sizeof(TYPE)) * E + sizeof(TYPE) * E , ...) | kzalloc( - (sizeof(THING)) * E + sizeof(THING) * E , ...) ) // Drop single-byte sizes and redundant parens. @@ expression COUNT; typedef u8; typedef __u8; @@ ( kzalloc( - sizeof(u8) * (COUNT) + COUNT , ...) | kzalloc( - sizeof(__u8) * (COUNT) + COUNT , ...) | kzalloc( - sizeof(char) * (COUNT) + COUNT , ...) | kzalloc( - sizeof(unsigned char) * (COUNT) + COUNT , ...) | kzalloc( - sizeof(u8) * COUNT + COUNT , ...) | kzalloc( - sizeof(__u8) * COUNT + COUNT , ...) | kzalloc( - sizeof(char) * COUNT + COUNT , ...) | kzalloc( - sizeof(unsigned char) * COUNT + COUNT , ...) ) // 2-factor product with sizeof(type/expression) and identifier or constant. @@ type TYPE; expression THING; identifier COUNT_ID; constant COUNT_CONST; @@ ( - kzalloc + kcalloc ( - sizeof(TYPE) * (COUNT_ID) + COUNT_ID, sizeof(TYPE) , ...) | - kzalloc + kcalloc ( - sizeof(TYPE) * COUNT_ID + COUNT_ID, sizeof(TYPE) , ...) | - kzalloc + kcalloc ( - sizeof(TYPE) * (COUNT_CONST) + COUNT_CONST, sizeof(TYPE) , ...) | - kzalloc + kcalloc ( - sizeof(TYPE) * COUNT_CONST + COUNT_CONST, sizeof(TYPE) , ...) | - kzalloc + kcalloc ( - sizeof(THING) * (COUNT_ID) + COUNT_ID, sizeof(THING) , ...) | - kzalloc + kcalloc ( - sizeof(THING) * COUNT_ID + COUNT_ID, sizeof(THING) , ...) | - kzalloc + kcalloc ( - sizeof(THING) * (COUNT_CONST) + COUNT_CONST, sizeof(THING) , ...) | - kzalloc + kcalloc ( - sizeof(THING) * COUNT_CONST + COUNT_CONST, sizeof(THING) , ...) ) // 2-factor product, only identifiers. @@ identifier SIZE, COUNT; @@ - kzalloc + kcalloc ( - SIZE * COUNT + COUNT, SIZE , ...) // 3-factor product with 1 sizeof(type) or sizeof(expression), with // redundant parens removed. @@ expression THING; identifier STRIDE, COUNT; type TYPE; @@ ( kzalloc( - sizeof(TYPE) * (COUNT) * (STRIDE) + array3_size(COUNT, STRIDE, sizeof(TYPE)) , ...) | kzalloc( - sizeof(TYPE) * (COUNT) * STRIDE + array3_size(COUNT, STRIDE, sizeof(TYPE)) , ...) | kzalloc( - sizeof(TYPE) * COUNT * (STRIDE) + array3_size(COUNT, STRIDE, sizeof(TYPE)) , ...) | kzalloc( - sizeof(TYPE) * COUNT * STRIDE + array3_size(COUNT, STRIDE, sizeof(TYPE)) , ...) | kzalloc( - sizeof(THING) * (COUNT) * (STRIDE) + array3_size(COUNT, STRIDE, sizeof(THING)) , ...) | kzalloc( - sizeof(THING) * (COUNT) * STRIDE + array3_size(COUNT, STRIDE, sizeof(THING)) , ...) | kzalloc( - sizeof(THING) * COUNT * (STRIDE) + array3_size(COUNT, STRIDE, sizeof(THING)) , ...) | kzalloc( - sizeof(THING) * COUNT * STRIDE + array3_size(COUNT, STRIDE, sizeof(THING)) , ...) ) // 3-factor product with 2 sizeof(variable), with redundant parens removed. @@ expression THING1, THING2; identifier COUNT; type TYPE1, TYPE2; @@ ( kzalloc( - sizeof(TYPE1) * sizeof(TYPE2) * COUNT + array3_size(COUNT, sizeof(TYPE1), sizeof(TYPE2)) , ...) | kzalloc( - sizeof(TYPE1) * sizeof(THING2) * (COUNT) + array3_size(COUNT, sizeof(TYPE1), sizeof(TYPE2)) , ...) | kzalloc( - sizeof(THING1) * sizeof(THING2) * COUNT + array3_size(COUNT, sizeof(THING1), sizeof(THING2)) , ...) | kzalloc( - sizeof(THING1) * sizeof(THING2) * (COUNT) + array3_size(COUNT, sizeof(THING1), sizeof(THING2)) , ...) | kzalloc( - sizeof(TYPE1) * sizeof(THING2) * COUNT + array3_size(COUNT, sizeof(TYPE1), sizeof(THING2)) , ...) | kzalloc( - sizeof(TYPE1) * sizeof(THING2) * (COUNT) + array3_size(COUNT, sizeof(TYPE1), sizeof(THING2)) , ...) ) // 3-factor product, only identifiers, with redundant parens removed. @@ identifier STRIDE, SIZE, COUNT; @@ ( kzalloc( - (COUNT) * STRIDE * SIZE + array3_size(COUNT, STRIDE, SIZE) , ...) | kzalloc( - COUNT * (STRIDE) * SIZE + array3_size(COUNT, STRIDE, SIZE) , ...) | kzalloc( - COUNT * STRIDE * (SIZE) + array3_size(COUNT, STRIDE, SIZE) , ...) | kzalloc( - (COUNT) * (STRIDE) * SIZE + array3_size(COUNT, STRIDE, SIZE) , ...) | kzalloc( - COUNT * (STRIDE) * (SIZE) + array3_size(COUNT, STRIDE, SIZE) , ...) | kzalloc( - (COUNT) * STRIDE * (SIZE) + array3_size(COUNT, STRIDE, SIZE) , ...) | kzalloc( - (COUNT) * (STRIDE) * (SIZE) + array3_size(COUNT, STRIDE, SIZE) , ...) | kzalloc( - COUNT * STRIDE * SIZE + array3_size(COUNT, STRIDE, SIZE) , ...) ) // Any remaining multi-factor products, first at least 3-factor products, // when they're not all constants... @@ expression E1, E2, E3; constant C1, C2, C3; @@ ( kzalloc(C1 * C2 * C3, ...) | kzalloc( - (E1) * E2 * E3 + array3_size(E1, E2, E3) , ...) | kzalloc( - (E1) * (E2) * E3 + array3_size(E1, E2, E3) , ...) | kzalloc( - (E1) * (E2) * (E3) + array3_size(E1, E2, E3) , ...) | kzalloc( - E1 * E2 * E3 + array3_size(E1, E2, E3) , ...) ) // And then all remaining 2 factors products when they're not all constants, // keeping sizeof() as the second factor argument. @@ expression THING, E1, E2; type TYPE; constant C1, C2, C3; @@ ( kzalloc(sizeof(THING) * C2, ...) | kzalloc(sizeof(TYPE) * C2, ...) | kzalloc(C1 * C2 * C3, ...) | kzalloc(C1 * C2, ...) | - kzalloc + kcalloc ( - sizeof(TYPE) * (E2) + E2, sizeof(TYPE) , ...) | - kzalloc + kcalloc ( - sizeof(TYPE) * E2 + E2, sizeof(TYPE) , ...) | - kzalloc + kcalloc ( - sizeof(THING) * (E2) + E2, sizeof(THING) , ...) | - kzalloc + kcalloc ( - sizeof(THING) * E2 + E2, sizeof(THING) , ...) | - kzalloc + kcalloc ( - (E1) * E2 + E1, E2 , ...) | - kzalloc + kcalloc ( - (E1) * (E2) + E1, E2 , ...) | - kzalloc + kcalloc ( - E1 * E2 + E1, E2 , ...) ) Signed-off-by: Kees Cook <keescook@chromium.org>
|
#
f86c3ac5 |
|
20-Sep-2016 |
Fred Isaman <fred.isaman@gmail.com> |
pnfs: Add layout driver flag PNFS_LAYOUTGET_ON_OPEN Driver can set flag to allow LAYOUTGET to be sent with OPEN. Signed-off-by: Fred Isaman <fred.isaman@gmail.com> Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
212bf41d |
|
19-Oct-2017 |
Elena Reshetova <elena.reshetova@intel.com> |
fs, nfs: convert nfs_client.cl_count from atomic_t to refcount_t atomic_t variables are currently used to implement reference counters with the following properties: - counter is initialized to 1 using atomic_set() - a resource is freed upon counter reaching zero - once counter reaches zero, its further increments aren't allowed - counter schema uses basic atomic operations (set, inc, inc_not_zero, dec_and_test, etc.) Such atomic variables should be converted to a newly provided refcount_t type and API that prevents accidental counter overflows and underflows. This is important since overflows and underflows can lead to use-after-free situation and be exploitable. The variable nfs_client.cl_count is used as pure reference counter. Convert it to refcount_t and fix up the operations. Suggested-by: Kees Cook <keescook@chromium.org> Reviewed-by: David Windsor <dwindsor@gmail.com> Reviewed-by: Hans Liljestrand <ishkamiel@gmail.com> Signed-off-by: Elena Reshetova <elena.reshetova@intel.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
81a090b9 |
|
19-Oct-2017 |
Elena Reshetova <elena.reshetova@intel.com> |
fs, nfs: convert nfs4_ff_layout_mirror.ref from atomic_t to refcount_t atomic_t variables are currently used to implement reference counters with the following properties: - counter is initialized to 1 using atomic_set() - a resource is freed upon counter reaching zero - once counter reaches zero, its further increments aren't allowed - counter schema uses basic atomic operations (set, inc, inc_not_zero, dec_and_test, etc.) Such atomic variables should be converted to a newly provided refcount_t type and API that prevents accidental counter overflows and underflows. This is important since overflows and underflows can lead to use-after-free situation and be exploitable. The variable nfs4_ff_layout_mirror.ref is used as pure reference counter. Convert it to refcount_t and fix up the operations. Suggested-by: Kees Cook <keescook@chromium.org> Reviewed-by: David Windsor <dwindsor@gmail.com> Reviewed-by: Hans Liljestrand <ishkamiel@gmail.com> Signed-off-by: Elena Reshetova <elena.reshetova@intel.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
4b75053e |
|
12-Jul-2017 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfiles: Handle expired layout segments in ff_layout_initiate_commit() If the layout has expired due to a fencing event, then we should not attempt to commit to the DS. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
22368ff1 |
|
23-Jun-2017 |
Olga Kornievskaia <kolga@netapp.com> |
PNFS for stateid errors retry against MDS first Upon receiving a stateid error such as BAD_STATEID, the client should retry the operation against the MDS before deciding to do stateid recovery. Previously, the code would initiate state recovery and it could lead to a race in a state manager that could chose an incorrect recovery method which would lead to the EIO failure for the application. Signed-off-by: Olga Kornievskaia <kolga@netapp.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
662f9a10 |
|
19-May-2017 |
Dan Carpenter <dan.carpenter@oracle.com> |
pNFS/flexfiles: missing error code in ff_layout_alloc_lseg() If xdr_inline_decode() fails then we end up returning ERR_PTR(0). The caller treats NULL returns as -ENOMEM so it doesn't really hurt runtime, but obviously we intended to set an error code here. Fixes: d67ae825a59d ("pnfs/flexfiles: Add the FlexFile Layout Driver") Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com> Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
76b2a303 |
|
09-May-2017 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfiles: Always attempt to call layoutstats when flexfiles is enabled Layoutstats is always desirable when using the flexfiles driver, so we should enable it if that driver is being loaded. It is safe to do so, because even when the mount specifies NFSv4.1, we will turn it off if the server tells us it is unsupported. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
37f8aa16 |
|
28-Apr-2017 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfiles: Fix up the ff_layout_write_pagelist failure path If the attempt to write through pNFS fails, we need to use the same failure semantics as for the read path: If the FF_FLAGS_NO_IO_THRU_MDS flag is set or we have sufficient valid DSes, then we must retry through pNFS Fixes: d67ae825a59d ("pnfs/flexfiles: Add the FlexFile Layout Driver") Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
b3230e80 |
|
25-Apr-2017 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS: Ensure we check layout segment validity in the pg_init() callback If we have a layout segment cached in pgio->pg_lseg, we should check it for validity before reusing it in a new RPC request. Otherwise, if we recoalesce, we can end up looping forever. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
5b5e0928 |
|
27-Feb-2017 |
Alexey Dobriyan <adobriyan@gmail.com> |
lib/vsprintf.c: remove %Z support Now that %z is standartised in C99 there is no reason to support %Z. Unlike %L it doesn't even make format strings smaller. Use BUILD_BUG_ON in a couple ATM drivers. In case anyone didn't notice lib/vsprintf.o is about half of SLUB which is in my opinion is quite an achievement. Hopefully this patch inspires someone else to trim vsprintf.c more. Link: http://lkml.kernel.org/r/20170103230126.GA30170@avx2 Signed-off-by: Alexey Dobriyan <adobriyan@gmail.com> Cc: Andy Shevchenko <andy.shevchenko@gmail.com> Cc: Rasmus Villemoes <linux@rasmusvillemoes.dk> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
|
#
df3ab232 |
|
17-Feb-2017 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfiles: If the layout is invalid, it must be updated before retrying If we see that our pNFS READ/WRITE/COMMIT operation failed, but we also see that our layout segment is no longer valid, then we need to get a new layout segment before retrying. Fixes: 90816d1ddacf ("NFSv4.1/flexfiles: Don't mark the entire deviceid...") Cc: stable@vger.kernel.org # v4.2+ Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
ab6e9aaf |
|
19-Feb-2017 |
Trond Myklebust <trond.myklebust@primarydata.com> |
NFSv4: Replace ad-hoc xdr encode/decode helpers with xdr_stream_* generics Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com> Reviewed-by: Chuck Lever <chuck.lever@oracle.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
2e54b9b1 |
|
12-Jan-2017 |
Wei Yongjun <weiyongjun1@huawei.com> |
pNFS/flexfiles: Make local symbol layoutreturn_ops static Fixes the following sparse warning: fs/nfs/flexfilelayout/flexfilelayout.c:2114:34: warning: symbol 'layoutreturn_ops' was not declared. Should it be static? Signed-off-by: Wei Yongjun <weiyongjun1@huawei.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
6de7e12f |
|
09-Jan-2017 |
Anna Schumaker <Anna.Schumaker@Netapp.com> |
NFS: Use nfs4_setup_sequence() everywhere This does the right thing depending on if we have a session, rather than needing to handle this manually in multiple places. Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
1f3a8e49 |
|
24-Dec-2016 |
Thomas Gleixner <tglx@linutronix.de> |
ktime: Get rid of ktime_equal() No point in going through loops and hoops instead of just comparing the values. Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Peter Zijlstra <peterz@infradead.org>
|
#
2456e855 |
|
25-Dec-2016 |
Thomas Gleixner <tglx@linutronix.de> |
ktime: Get rid of the union ktime is a union because the initial implementation stored the time in scalar nanoseconds on 64 bit machine and in a endianess optimized timespec variant for 32bit machines. The Y2038 cleanup removed the timespec variant and switched everything to scalar nanoseconds. The union remained, but become completely pointless. Get rid of the union and just keep ktime_t as simple typedef of type s64. The conversion was done with coccinelle and some manual mopping up. Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Peter Zijlstra <peterz@infradead.org>
|
#
1c48cee8 |
|
14-Dec-2016 |
Weston Andros Adamson <dros@primarydata.com> |
pNFS/flexfiles: delete deviceid, don't mark inactive Instead of marking a device inactive, remove it from the cache entirely. Flexfiles has a way to report errors back to the server, so we don't want to stop devices from being tried again for 120 seconds. Signed-off-by: Weston Andros Adamson <dros@primarydata.com> Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
d9152114 |
|
09-Dec-2016 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfiles: Ensure we have enough buffer for layoutreturn The flexfiles client can piggyback both layout errors and layoutstats as part of the layoutreturn. Both these payloads can get large, with 20 layout error entries taking up about 1.2K, and 4 layoutstats entries taking up another 1K. This patch allows a maximum payload of 4k by allocating a full page. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
5ba6a09e |
|
09-Dec-2016 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfiles: Remove a redundant parameter in ff_layout_encode_ioerr() Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
65990d1a |
|
30-Sep-2016 |
Fred Isaman <fred.isaman@gmail.com> |
pNFS/flexfiles: Fix a deadlock on LAYOUTGET We encountered a deadlock where the SEQUENCE that accompanied the LAYOUTGET triggered a session drain, while ff_layout_alloc_lseg triggered a GETDEVICEINFO. The GETDEVICEINFO hung waiting for the session drain, while the LAYOUTGET held the slot waiting for alloc_lseg to finish. Avoid this by moving the call to nfs4_find_get_deviceid out of ff_layout_alloc_lseg and into nfs4_ff_layout_prepare_ds. Signed-off-by: Fred Isaman <fred.isaman@gmail.com> [dros@primarydata.com: pNFS/flexfiles: fix races in ff_layout_mirror_valid] Signed-off-by: Weston Andros Adamson <dros@primarydata.com> Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
230bc962 |
|
19-Oct-2016 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfiles: Support sending layoutstats in layoutreturn Add the ability to send an array of layoutstats entries as part of layoutreturn. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
422c93c8 |
|
06-Oct-2016 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfiles: Minor refactoring before adding iostats to layoutreturn Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
2f8220c1 |
|
03-Oct-2016 |
Trond Myklebust <trond.myklebust@primarydata.com> |
NFS: Fix up read of mirror stats Need to lock while reading in order to ensure 64-bit reads are correct. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
08e2e5bc |
|
29-Sep-2016 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfiles: Clean up layoutstats Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
5b9b3c85 |
|
02-Dec-2016 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfiles: Refactor encoding of the layoutreturn payload Add the layout error payload to the flexfiles layoutreturn private data, and set up the encoding mechanisms. This is a refactoring in preparation for adding the layout iostats payload. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
06946c6a |
|
25-Nov-2016 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfiles: Only send layoutstats updates for mirrors that were updated If there have been no reads or writes to a given mirror since the last layoutstats update, then don't resend the same data. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
46c98c6d |
|
25-Nov-2016 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfiles: Don't attempt to send layoutstats if there are no entries If the list of mirrors is empty, then don't send an RPC call. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
94e5c571 |
|
15-Sep-2016 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS: Get rid of unnecessary layout parameter in encode_layoutreturn callback The parameter is already present in the "args" structure. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
54e4a0df |
|
27-Nov-2016 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS: Fix a deadlock between read resends and layoutreturn We must not call nfs_pageio_init_read() on a new nfs_pageio_descriptor while holding a reference to a layout segment, as that can deadlock pnfs_update_layout(). Fixes: d67ae825a59d6 ("pnfs/flexfiles: Add the FlexFile Layout Driver") Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com> Cc: stable@vger.kernel.org # v4.0+
|
#
41020b67 |
|
22-Sep-2016 |
Trond Myklebust <trond.myklebust@primarydata.com> |
NFSv4.x: Allow callers of nfs_remove_bad_delegation() to specify a stateid Allow the callers of nfs_remove_bad_delegation() to specify the stateid that needs to be marked as bad. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com> Tested-by: Oleg Drokin <green@linuxhacker.ru> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
3dc14735 |
|
29-Aug-2016 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfiles: Fix an Oopsable condition when connection to the DS fails If the attempt to connect to a DS fails inside ff_layout_pg_init_read or ff_layout_pg_init_write, then we currently end up clearing the layout segment carried by the struct nfs_pageio_descriptor, causing an Oops when we later call into ff_layout_read_pagelist/ff_layout_write_pagelist. The fix is to ensure we return the layout and then retry. Fixes: 446ca2195303 ("pNFS/flexfiles: When initing reads or writes, we...") Cc: stable@vger.kernel.org # v4.7+ Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
1c8d477a |
|
13-Aug-2016 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfiles: Fix layoutstat periodic reporting Putting the periodicity timer in the mirror instances is causing non-scalable reporting behaviour and missed reporting intervals. When you recall layouts and/or implement client side mirroring, it leads to consecutive reports with only a few ms between RPC calls. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com> Fixes: d0379a5d066a9 ("pNFS/flexfiles: Support server-supplied...")
|
#
2e18d4d8 |
|
26-Jun-2016 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS: Files and flexfiles always need to commit before layoutcommit So ensure that we mark the layout for commit once the write is done, and then ensure that the commit to ds is finished before sending layoutcommit. Note that by doing this, we're able to optimise away the commit for the case of servers that don't need layoutcommit in order to return updated attributes. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
bc28e1c2 |
|
26-Jun-2016 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfiles: Clean up calls to pnfs_set_layoutcommit() Let's just have one place where we check ff_layout_need_layoutcommit(). Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
c001c87a |
|
25-Jun-2016 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfiles: Fix layoutcommit after a commit to DS We should always do a layoutcommit after commit to DS, except if the layout segment we're using has set FF_FLAGS_NO_LAYOUTCOMMIT. Fixes: d67ae825a59d ("pnfs/flexfiles: Add the FlexFile Layout Driver") Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
c7d73af2 |
|
25-May-2016 |
Tom Haynes <thomas.haynes@primarydata.com> |
pnfs: pnfs_update_layout needs to consider if strict iomode checking is on As flexfiles has FF_FLAGS_NO_READ_IO, there is a need to generically support enforcing that a IOMODE_RW segment will not allow READ I/O. Signed-off-by: Tom Haynes <loghyr@primarydata.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
602c4cd4 |
|
25-May-2016 |
Tom Haynes <thomas.haynes@primarydata.com> |
nfs/flexfiles: Use the layout segment for reading unless it a IOMODE_RW and reading is disabled Signed-off-by: Tom Haynes <loghyr@primarydata.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
094069f1 |
|
16-May-2016 |
Jeff Layton <jlayton@kernel.org> |
flexfiles: remove pointless setting of NFS_LAYOUT_RETURN_REQUESTED Setting just the NFS_LAYOUT_RETURN_REQUESTED flag doesn't do anything, unless there are lsegs that are also being marked for return. At the point where that happens this flag is also set, so these set_bit calls don't do anything useful. Signed-off-by: Jeff Layton <jeff.layton@primarydata.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
ee26bdd6 |
|
16-May-2016 |
Jeff Layton <jlayton@kernel.org> |
pnfs: don't merge new ff lsegs with ones that have LAYOUTRETURN bit set Otherwise, we'll end up returning layouts that we've just received if the client issues a new LAYOUTGET prior to the LAYOUTRETURN. Signed-off-by: Jeff Layton <jeff.layton@primarydata.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
446ca219 |
|
16-May-2016 |
Tom Haynes <loghyr@primarydata.com> |
pNFS/flexfiles: When initing reads or writes, we might have to retry connecting to DSes If we are initializing reads or writes and can not connect to a DS, then check whether or not IO is allowed through the MDS. If it is allowed, reset to the MDS. Else, fail the layout segment and force a retry of a new layout segment. Signed-off-by: Tom Haynes <loghyr@primarydata.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
3b13b4b3 |
|
16-May-2016 |
Tom Haynes <loghyr@primarydata.com> |
pNFS/flexfiles: When checking for available DSes, conditionally check for MDS io Whenever we check to see if we have the needed number of DSes for the action, we may also have to check to see whether IO is allowed to go to the MDS or not. [jlayton: fix merge conflict due to lack of localio patches here] Signed-off-by: Tom Haynes <loghyr@primarydata.com> Signed-off-by: Jeff Layton <jeff.layton@primarydata.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
75bf47eb |
|
16-May-2016 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfile: Fix erroneous fall back to read/write through the MDS This patch fixes a problem whereby the pNFS client falls back to doing reads and writes through the metadata server even when the layout flag FF_FLAGS_NO_IO_THRU_MDS is set. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
93b717fd |
|
16-May-2016 |
Trond Myklebust <trond.myklebust@primarydata.com> |
NFSv4: Label stateids with the type In order to more easily distinguish what kind of stateid we are dealing with, introduce a type that can be used to label the stateid structure. The label will be useful both for debugging, but also when dealing with operations like SETATTR, READ and WRITE that can take several different types of stateid as arguments. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
3064b686 |
|
21-Apr-2016 |
Jeff Layton <jlayton@kernel.org> |
nfs: have flexfiles mirror keep creds for both ro and rw layouts A mirror can be shared between multiple layouts, even with different iomodes. That makes stats gathering simpler, but it causes a problem when we get different creds in READ vs. RW layouts. The current code drops the newer credentials onto the floor when this occurs. That's problematic when you fetch a READ layout first, and then a RW. If the READ layout doesn't have the correct creds to do a write, then writes will fail. We could just overwrite the READ credentials with the RW ones, but that would break the ability for the server to fence the layout for reads if things go awry. We need to be able to revert to the earlier READ creds if the RW layout is returned afterward. The simplest fix is to just keep two sets of creds per mirror. One for READ layouts and one for RW, and then use the appropriate set depending on the iomode of the layout segment. Also fix up some RCU nits that sparse found. Signed-off-by: Jeff Layton <jeff.layton@primarydata.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
90a0be00 |
|
21-Apr-2016 |
Jeff Layton <jlayton@kernel.org> |
nfs: get a reference to the credential in ff_layout_alloc_lseg We're just as likely to have allocation problems here as we would if we delay looking up the credential like we currently do. Fix the code to get a rpc_cred reference early, as soon as the mirror is set up. This allows us to eliminate the mirror early if there is a problem getting an rpc credential. This also allows us to drop the uid/gid from the layout_mirror struct as well. In the event that we find an existing mirror where this one would go, we swap in the new creds unconditionally, and drop the reference to the old one. Note that the old ff_layout_update_mirror_cred function wouldn't set this pointer unless the DS version was 3, but we don't know what the DS version is at this point. I'm a little unclear on why it did that as you still need creds to talk to v4 servers as well. I have the code set it regardless of the DS version here. Also note the change to using generic creds instead of calling lookup_cred directly. With that change, we also need to populate the group_info pointer in the acred as some functions expect that to never be NULL. Instead of allocating one every time however, we can allocate one when the module is loaded and share it since the group_info is refcounted. Signed-off-by: Jeff Layton <jeff.layton@primarydata.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
57f3f4c0 |
|
21-Apr-2016 |
Jeff Layton <jlayton@kernel.org> |
nfs: have ff_layout_get_ds_cred take a reference to the cred In later patches, we're going to want to allow the creds to be updated when we get a new layout with updated creds. Have this function take a reference to the cred that is later put once the call has been dispatched. Also, prepare for this change by ensuring we follow RCU rules when getting a reference to the cred as well. Signed-off-by: Jeff Layton <jeff.layton@primarydata.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
fe238e60 |
|
01-Apr-2016 |
Dave Wysochanski <dwysocha@redhat.com> |
NFS: Save struct inode * inside nfs_commit_info to clarify usage of i_lock Commit ea2cf22 created nfs_commit_info and saved &inode->i_lock inside this NFS specific structure. This obscures the usage of i_lock. Instead, save struct inode * so later it's clear the spinlock taken is i_lock. Should be no functional change. Signed-off-by: Dave Wysochanski <dwysocha@redhat.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
2370abda |
|
27-Jan-2016 |
Trond Myklebust <trond.myklebust@primarydata.com> |
NFS: Cleanup - rename NFS_LAYOUT_RETURN_BEFORE_CLOSE NFS_LAYOUT_RETURN_BEFORE_CLOSE is being used to signal that a layoutreturn is needed, either due to a layout recall or to a layout error. Rename it to NFS_LAYOUT_RETURN_REQUESTED in order to clarify its purpose. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
082fa37d |
|
21-Jan-2016 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfiles: Fix an XDR encoding bug in layoutreturn We must not skip encoding the statistics, or the server will see an XDR encoding error. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com> Cc: stable@vger.kernel.org # 4.0+
|
#
dc602dd7 |
|
31-Dec-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
NFS/pNFS: Fix up pNFS write reschedule layering violations and bugs The flexfiles layout in particular, seems to want to poke around in the O_DIRECT flags when retransmitting. This patch sets up an interface to allow it to call back into O_DIRECT to handle retransmission correctly. It also fixes a potential bug whereby we could change the behaviour of O_DIRECT if an error is already pending. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
86fb449b |
|
30-Dec-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfiles: Fix an Oopsable typo in ff_mirror_match_fh() Jeff reports seeing an Oops in ff_layout_alloc_lseg. Turns out copy+paste has played cruel tricks on a nested loop. Reported-by: Jeff Layton <jeff.layton@primarydata.com> Cc: stable@vger.kernel.org # 4.3+ Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
4d0ac221 |
|
22-Dec-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfiles: Ensure we record layoutstats even if RPC is terminated early Currently, we will only record the layoutstats correctly if the RPC call successfully obtains a slot. If we exit before that happens, then we may find ourselves starting the busy timer through the call in ff_layout_(read|write)_prepare_layoutstats, but never stopping it. The same thing happens if we're doing DA-DS. The fix is to ensure that we catch these cases in the rpc_release() callback. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
37e9ed22 |
|
21-Dec-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS: Add flag to track if we've called nfs4_ff_layout_stat_io_start_read/write Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
7eeea167 |
|
17-Dec-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfiles: Fix a statistics gathering imbalance When we replay a failed read, write or commit to the dataserver, we need to ensure that we call ff_layout_read_prepare_v3(), ff_layout_write_prepare_v3 or ff_layout_commit_prepare_v3() so that we reset the statistics. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
2e5b29f0 |
|
14-Dec-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfiles: Don't prevent flexfiles client from retrying LAYOUTGET Fix a bug in which flexfiles clients are falling back to I/O through the MDS even when the FF_FLAGS_NO_IO_THRU_MDS flag is set. The flexfiles client will always report errors through the LAYOUTRETURN and/or LAYOUTERROR mechanisms, so it should normally be safe for it to retry the LAYOUTGET until it fails or succeeds. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
141b9b59 |
|
07-Dec-2015 |
Peng Tao <tao.peng@primarydata.com> |
pnfs/flexfiles: count io stat in rpc_count_stats callback If client ever restarts IO due to some errors, we'll endup mis-counting IO stats if we do the counting in .rpc_done callback. Move it to .rpc_count_stats callback that is only called when releasing RPC. Signed-off-by: Peng Tao <tao.peng@primarydata.com> Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
c22eeb86 |
|
07-Dec-2015 |
Peng Tao <tao.peng@primarydata.com> |
pnfs/flexfiles: do not mark delay-like status as DS failure We just need to delay and retry in these cases. Signed-off-by: Peng Tao <tao.peng@primarydata.com> Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
d600ad1f |
|
03-Dec-2015 |
Peng Tao <tao.peng@primarydata.com> |
NFS41: pop some layoutget errors to application For ERESTARTSYS/EIO/EROFS/ENOSPC/E2BIG in layoutget, we should just bail out instead of hiding the error and retrying inband IO. Change all the call sites to pop the error all the way up. Signed-off-by: Peng Tao <tao.peng@primarydata.com> Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
d0379a5d |
|
16-Nov-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfiles: Support server-supplied layoutstats sampling period Some servers want to be able to control the frequency with which clients report layoutstats, for instance, in order to monitor QoS for a particular file or set of file. In order to support this, the flexfiles layout allows the server to pass this info as a hint in the layout payload. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
260074cd |
|
02-Nov-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfiles: Add support for FF_FLAGS_NO_IO_THRU_MDS For loosely coupled pNFS/flexfiles systems, there is often no advantage at all in going through the MDS for I/O, since the MDS is subject to the same limitations as all other clients when talking to DSes. If a DS is unresponsive, I/O through the MDS will fail. For such systems, the only scalable solution is to have the pNFS clients retry doing pNFS, and so the protocol now provides a flag that allows the pNFS server to signal this. If LAYOUTGET returns FF_FLAGS_NO_IO_THRU_MDS, then we should assume that the MDS wants the client to retry using these devices, even if they were previously marked as being unavailable. To do so, we add a helper, ff_layout_mark_devices_valid() that will be called from layoutget. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
13544412 |
|
02-Nov-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfiles: When mirrored, retry failed reads by switching mirrors If the pNFS/flexfiles file is mirrored, and a read to one mirror fails, then we should bump the mirror index, so that we retry to a different mirror. Once we've iterated through all mirrors and all failed, we can return the layout and issue a new LAYOUTGET. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
7cc8c5cd |
|
02-Sep-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
NFSv4.1/flexfiles: Clean up ff_layout_write_done_cb/ff_layout_commit_done_cb Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
f95c03b2 |
|
02-Sep-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
NFSv4.1/flexfiles: Mark the layout for return in ff_layout_io_track_ds_error() When I/O cannot complete due to a fatal error on the DS, ensure that we invalidate the corresponding layout segment and return it. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
972398fa |
|
01-Sep-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
NFSv4.1/flexfiles: Fix freeing of mirrors Mirrors are now shared objects, so we should not be freeing them directly inside ff_layout_free_lseg(). We should already be doing the right thing in _ff_layout_free_lseg(), so just let it handle things. Also ensure that ff_layout_free_mirror() frees the RPC credential if it is set. Fixes: 28a0d72c6867 ("Add refcounting to struct nfs4_ff_layout_mirror") Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
90816d1d |
|
30-Aug-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
NFSv4.1/flexfiles: Don't mark the entire deviceid as bad for file errors If the file was fenced and/or has been deleted on the DS, then we want to retry pNFS after a layoutreturn with error report. If the server cannot fix the problem, then we rely on it to tell us so in the response to the LAYOUTGET. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
0762ed2c |
|
25-Aug-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
NFSv4.1/flexfiles: Allow coalescing of new layout segments and existing ones In order to ensure atomicity of updates, we merge the old layout segments into the new ones, and then invalidate the old ones. Also ensure that we order the list of layout segments so that RO segments are preferred over RW. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
e3b1df2d |
|
24-Aug-2015 |
kbuild test robot <fengguang.wu@intel.com> |
NFSv4.1/flexfile: ff_layout_remove_mirror can be static Signed-off-by: Fengguang Wu <fengguang.wu@intel.com> Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
bbf58bf3 |
|
24-Aug-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
NFSv4.2/pnfs: Make the layoutstats timer configurable Allow advanced users to set the layoutstats timer in order to lengthen or shorten the period between layoutstat transmissions to the server. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
266d12d4 |
|
24-Aug-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
NFSv4.1/flexfile: Ensure uniqueness of mirrors across layout segments Keep the full list of mirrors in the struct nfs4_ff_layout_mirror so that they can be shared among the layout segments that use them. Also ensure that we send out only one copy of the layoutstats per mirror. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
0b7baf94 |
|
24-Aug-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
NFSv4.1/flexfiles: Remove mirror backpointer to lseg. When we start sharing mirrors between several lsegs, we won't be able to keep it. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
28a0d72c |
|
24-Aug-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
NFSv4.1/flexfiles: Add refcounting to struct nfs4_ff_layout_mirror We do want to share mirrors between layout segments, so add a refcount to enable that. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
54204010 |
|
21-Aug-2015 |
Peng Tao <tao.peng@primarydata.com> |
NFS41/flexfiles: zero out DS write wcc We do not want to update inode attributes with DS values. Cc: stable@vger.kernel.org # v4.0+ Signed-off-by: Peng Tao <tao.peng@primarydata.com> Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
dd52128a |
|
20-Aug-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
NFSv4.1/pnfs Ensure flexfiles reports all connection related errors Make sure that we also handle RPC level connection and protocol negotiation errors. Reported-by: Tom Haynes <loghyr@primarydata.com> Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
e76d28dd |
|
20-Aug-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
NFSv4.1/pnfs: Ensure the flexfiles layoutstats timers are consistent We want to ensure that the stopwatches for the busy timer and the aggregate timer are consistent. This means that they need to use the same start/stop times. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
69f230d9 |
|
19-Aug-2015 |
Peng Tao <tao.peng@primarydata.com> |
NFS41/flexfiles: update inode after write finishes Otherwise we break fstest case tests/read_write/mctime.t Does files layout need the same fix as well? Cc: stable@vger.kernel.org # v4.0+ Cc: Anna Schumaker <anna.schumaker@netapp.com> Signed-off-by: Peng Tao <tao.peng@primarydata.com> Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
ce603281 |
|
10-Jul-2015 |
Jeff Layton <jlayton@kernel.org> |
nfs: remove some dead code in ff_layout_pg_get_mirror_count_write We already know that pg_lseg is NULL here. Signed-off-by: Jeff Layton <jeff.layton@primarydata.com> Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
c8ad8894 |
|
05-Aug-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
NFSv4.2/pnfs: Use GFP_NOIO for layoutstat reporting in the writeback path Prevent a potential deadlock. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
d099d7b8 |
|
10-Aug-2015 |
Peng Tao <tao.peng@primarydata.com> |
pnfs/flexfiles: LAYOUTSTATS ii_count should be ops instead of bytes Turned out I misinterpreted the spec... Cc: Tom Haynes <thomas.haynes@primarydata.com> Reported-by: Jean Spector <jean@primarydata.com> Signed-off-by: Peng Tao <tao.peng@primarydata.com> Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
690edcfa |
|
08-Jul-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
NFSv4.2/flexfiles: Fix a typo in the flexfiles layoutstats code Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
d6208769 |
|
26-Jun-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfiles: Fix the reset of struct pgio_header when resending hdr->good_bytes needs to be set to the length of the request, not zero. Cc: stable@vger.kernel.org # 4.0+ Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
c0f5f505 |
|
26-Jun-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfiles: Turn off layoutcommit for servers that don't need it This patch ensures that we record the value of 'ffl_flags' from the layout, and then checks for the presence of the FF_FLAGS_NO_LAYOUTCOMMIT flag before deciding whether or not to call pnfs_set_layoutcommit(). The effect is that servers now can decide whether or not they want the client to call layoutcommit before returning a writeable layout. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
9bbd9bb4 |
|
25-Jun-2015 |
Peng Tao <tao.peng@primarydata.com> |
pnfs/flexfiles: protect ktime manipulation with mirror lock It looks as if xchg() and cmpxchg() are not available for 64-bit integers on sparc32: > New breakage seen in linux-next today: > > ERROR: "__xchg_called_with_bad_pointer" [fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko] undefined! > ERROR: "__cmpxchg_called_with_bad_pointer" [fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko] undefined! > make[2]: *** [__modpost] Error 1 > make[1]: *** [modules] Error 2 Given that mirror ktime manipulation is already under mirror->lock, let's make use of the fact. Reported-by: Paul Gortmaker <paul.gortmaker@windriver.com> Signed-off-by: Peng Tao <tao.peng@primarydata.com> Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
97ba375b |
|
23-Jun-2015 |
Peng Tao <tao.peng@primarydata.com> |
pnfs/flexfiles: report layoutstat regularly As a simple scheme, report every minute if IO is still going on. Reviewed-by: Jeff Layton <jeff.layton@primarydata.com> Signed-off-by: Peng Tao <tao.peng@primarydata.com> Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
27c43064 |
|
23-Jun-2015 |
Peng Tao <tao.peng@primarydata.com> |
pnfs/flexfiles: encode LAYOUTSTATS flexfiles specific data Reviewed-by: Jeff Layton <jeff.layton@primarydata.com> Signed-off-by: Peng Tao <tao.peng@primarydata.com> Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
ad4dc53e |
|
23-Jun-2015 |
Peng Tao <tao.peng@primarydata.com> |
pnfs/flexfiles: add ff_layout_prepare_layoutstats It fills in the generic part of LAYOUTSTATS call. One thing to note is that we don't really track if IO is continuous or not. So just fake to use the completed bytes for it. Still missing flexfiles specific part, which will be included in the next patch. Reviewed-by: Jeff Layton <jeff.layton@primarydata.com> Signed-off-by: Peng Tao <tao.peng@primarydata.com> Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
d983803d |
|
23-Jun-2015 |
Peng Tao <tao.peng@primarydata.com> |
pNFS/flexfiles: track when layout is first used So that we can report cumulative time since the beginning of statistics collection of the layout. Reviewed-by: Jeff Layton <jeff.layton@primarydata.com> Signed-off-by: Peng Tao <tao.peng@primarydata.com> Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
abcb7bfc |
|
23-Jun-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
pNFS/flexfiles: add layoutstats tracking Reviewed-by: Jeff Layton <jeff.layton@primarydata.com> Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
455b6ee6 |
|
12-Jun-2015 |
Fabian Frederick <fabf@skynet.be> |
pnfs/flexfiles: use swap() in ff_layout_sort_mirrors() Use kernel.h macro definition. Thanks to Julia Lawall for Coccinelle scripting support. Signed-off-by: Fabian Frederick <fabf@skynet.be> Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
40c64c26 |
|
15-Apr-2015 |
Anna Schumaker <Anna.Schumaker@netapp.com> |
NFS: Move nfs_idmap.h into fs/nfs/ This file is only used internally to the NFS v4 module, so it doesn't need to be in the global include path. I also renamed it from nfs_idmap.h to nfs4idmap.h to emphasize that it's an NFSv4-only include file. Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com> Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
5bb89b47 |
|
25-Mar-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
NFSv4.1/pnfs: Separate out metadata and data consistency for pNFS The LAYOUTCOMMIT operation means different things to different layout types. For blocks and objects, it is both a data and metadata consistency operation. For files and flexfiles, it is only a metadata consistency operation. This patch separates out the 2 cases, allowing the files/flexfiles layout drivers to optimise away the data consistency calls to layoutcommit. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
67af7611 |
|
25-Mar-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
NFSv4.1/pnfs: Refactor pnfs_set_layoutcommit() pnfs_set_layoutcommit() and pnfs_commit_set_layoutcommit() are 100% identical except for the function arguments. Refactor to eliminate the difference. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
fc87701b |
|
09-Mar-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
NFS: Fix free_deveiceid -> free_deviceid Make it easier to grep for these functions by name. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
338d00cf |
|
17-Feb-2015 |
Tom Haynes <thomas.haynes@primarydata.com> |
pnfs: Refactor the *_layout_mark_request_commit to use pnfs_layout_mark_request_commit The File Layout's filelayout_mark_request_commit() is almost the Flex File Layout's ff_layout_mark_request_commit(). And that can be reduced by calling into nfs_request_add_commit_list(). Signed-off-by: Tom Haynes <loghyr@primarydata.com> Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
d15bc38d |
|
13-Feb-2015 |
Tom Haynes <thomas.haynes@primarydata.com> |
nfs: Provide and use helper functions for marking a page as unstable Signed-off-by: Tom Haynes <loghyr@primarydata.com> Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
d67ae825 |
|
11-Dec-2014 |
Tom Haynes <loghyr@primarydata.com> |
pnfs/flexfiles: Add the FlexFile Layout Driver The flexfile layout is a new layout that extends the file layout. It is currently being drafted as a specification at https://datatracker.ietf.org/doc/draft-ietf-nfsv4-layout-types/ Signed-off-by: Weston Andros Adamson <dros@primarydata.com> Signed-off-by: Tom Haynes <loghyr@primarydata.com> Signed-off-by: Tao Peng <bergwolf@primarydata.com>
|