#
2c35f43b |
|
30-Jan-2024 |
Dai Ngo <dai.ngo@oracle.com> |
SUNRPC: increase size of rpc_wait_queue.qlen from unsigned short to unsigned int When the NFS client is under extreme load the rpc_wait_queue.qlen counter can be overflowed. Here is an instant of the backlog queue overflow in a real world environment shown by drgn helper: rpc_task_stats(rpc_clnt): ------------------------- rpc_clnt: 0xffff92b65d2bae00 rpc_xprt: 0xffff9275db64f000 Queue: sending[64887] pending[524] backlog[30441] binding[0] XMIT task: 0xffff925c6b1d8e98 WRITE: 750654 __dta_call_status_580: 65463 __dta_call_transmit_status_579: 1 call_reserveresult: 685189 nfs_client_init_is_complete: 1 COMMIT: 584 call_reserveresult: 573 __dta_call_status_580: 11 ACCESS: 1 __dta_call_status_580: 1 GETATTR: 10 __dta_call_status_580: 4 call_reserveresult: 6 751249 tasks for server 111.222.333.444 Total tasks: 751249 count_rpc_wait_queues(xprt): ---------------------------- **** rpc_xprt: 0xffff9275db64f000 num_reqs: 65511 wait_queue: xprt_binding[0] cnt: 0 wait_queue: xprt_binding[1] cnt: 0 wait_queue: xprt_binding[2] cnt: 0 wait_queue: xprt_binding[3] cnt: 0 rpc_wait_queue[xprt_binding].qlen: 0 maxpriority: 0 wait_queue: xprt_sending[0] cnt: 0 wait_queue: xprt_sending[1] cnt: 64887 wait_queue: xprt_sending[2] cnt: 0 wait_queue: xprt_sending[3] cnt: 0 rpc_wait_queue[xprt_sending].qlen: 64887 maxpriority: 3 wait_queue: xprt_pending[0] cnt: 524 wait_queue: xprt_pending[1] cnt: 0 wait_queue: xprt_pending[2] cnt: 0 wait_queue: xprt_pending[3] cnt: 0 rpc_wait_queue[xprt_pending].qlen: 524 maxpriority: 0 wait_queue: xprt_backlog[0] cnt: 0 wait_queue: xprt_backlog[1] cnt: 685801 wait_queue: xprt_backlog[2] cnt: 0 wait_queue: xprt_backlog[3] cnt: 0 rpc_wait_queue[xprt_backlog].qlen: 30441 maxpriority: 3 [task cnt mismatch] There is no effect on operations when this overflow occurs. However it causes confusion when trying to diagnose the performance problem. Signed-off-by: Dai Ngo <dai.ngo@oracle.com> Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
57331a59 |
|
04-Jan-2024 |
Benjamin Coddington <bcodding@redhat.com> |
NFSv4.1: Use the nfs_client's rpc timeouts for backchannel For backchannel requests that lookup the appropriate nfs_client, use the state-management rpc_clnt's rpc_timeout parameters for the backchannel's response. When the nfs_client cannot be found, fall back to using the xprt's default timeout parameters. Signed-off-by: Benjamin Coddington <bcodding@redhat.com> Tested-by: Chuck Lever <chuck.lever@oracle.com> Tested-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
691d0b78 |
|
18-Apr-2023 |
Dai Ngo <dai.ngo@oracle.com> |
SUNRPC: remove the maximum number of retries in call_bind_status Currently call_bind_status places a hard limit of 3 to the number of retries on EACCES error. This limit was done to prevent NLM unlock requests from being hang forever when the server keeps returning garbage. However this change causes problem for cases when NLM service takes longer than 9 seconds to register with the port mapper after a restart. This patch removes this hard coded limit and let the RPC handles the retry based on the standard hard/soft task semantics. Fixes: 0b760113a3a1 ("NLM: Don't hang forever on NLM unlock requests") Reported-by: Helen Chao <helen.chao@oracle.com> Tested-by: Helen Chao <helen.chao@oracle.com> Signed-off-by: Dai Ngo <dai.ngo@oracle.com> Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
f8423909 |
|
05-Oct-2022 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
SUNRPC: Add a helper to allow pNFS drivers to selectively cancel RPC calls Add the helper rpc_cancel_tasks(), which uses a caller-defined selection function to define a set of in-flight RPC calls to cancel. This is mainly intended for pNFS drivers which are subject to a layout recall, and which may therefore want to cancel all pending I/O using that layout in order to redrive it after the layout recall has been satisfied. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
39494194 |
|
05-Oct-2022 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
SUNRPC: Fix races with rpc_killall_tasks() Ensure that we immediately call rpc_exit_task() after waking up, and that the tk_rpc_status cannot get clobbered by some other function. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
f5d39b02 |
|
22-Aug-2022 |
Peter Zijlstra <peterz@infradead.org> |
freezer,sched: Rewrite core freezer logic Rewrite the core freezer to behave better wrt thawing and be simpler in general. By replacing PF_FROZEN with TASK_FROZEN, a special block state, it is ensured frozen tasks stay frozen until thawed and don't randomly wake up early, as is currently possible. As such, it does away with PF_FROZEN and PF_FREEZER_SKIP, freeing up two PF_flags (yay!). Specifically; the current scheme works a little like: freezer_do_not_count(); schedule(); freezer_count(); And either the task is blocked, or it lands in try_to_freezer() through freezer_count(). Now, when it is blocked, the freezer considers it frozen and continues. However, on thawing, once pm_freezing is cleared, freezer_count() stops working, and any random/spurious wakeup will let a task run before its time. That is, thawing tries to thaw things in explicit order; kernel threads and workqueues before doing bringing SMP back before userspace etc.. However due to the above mentioned races it is entirely possible for userspace tasks to thaw (by accident) before SMP is back. This can be a fatal problem in asymmetric ISA architectures (eg ARMv9) where the userspace task requires a special CPU to run. As said; replace this with a special task state TASK_FROZEN and add the following state transitions: TASK_FREEZABLE -> TASK_FROZEN __TASK_STOPPED -> TASK_FROZEN __TASK_TRACED -> TASK_FROZEN The new TASK_FREEZABLE can be set on any state part of TASK_NORMAL (IOW. TASK_INTERRUPTIBLE and TASK_UNINTERRUPTIBLE) -- any such state is already required to deal with spurious wakeups and the freezer causes one such when thawing the task (since the original state is lost). The special __TASK_{STOPPED,TRACED} states *can* be restored since their canonical state is in ->jobctl. With this, frozen tasks need an explicit TASK_FROZEN wakeup and are free of undue (early / spurious) wakeups. Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Reviewed-by: Ingo Molnar <mingo@kernel.org> Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> Link: https://lore.kernel.org/r/20220822114649.055452969@infradead.org
|
#
ba8ec7a6 |
|
23-Jul-2022 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
SUNRPC: Shrink size of struct rpc_task Move the field 'tk_rpc_status' so that we eliminate a 4 byte hole in the structure. For x86_64, this shrinks the size of the struct by 8 bytes. 'pahole' output before the change: /* size: 232, cachelines: 4, members: 27 */ /* sum members: 222, holes: 1, sum holes: 4 */ /* sum bitfield members: 8 bits (1 bytes) */ /* padding: 5 */ /* last cacheline: 40 bytes */ 'pahole' output after the change: /* size: 224, cachelines: 4, members: 27 */ /* padding: 1 */ /* last cacheline: 32 bytes */ Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
33e5c765 |
|
14-Mar-2022 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
NFS: Fix memory allocation in rpc_malloc() When in a low memory situation, we do want rpciod to kick off direct reclaim in the case where that helps, however we don't want it looping forever in mempool_alloc(). So first try allocating from the slab using GFP_KERNEL | __GFP_NORETRY, and then fall back to a GFP_NOWAIT allocation from the mempool. Ditto for rpc_alloc_task() Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
89c2be8a |
|
06-Mar-2022 |
NeilBrown <neilb@suse.de> |
NFS: discard NFS_RPC_SWAPFLAGS and RPC_TASK_ROOTCREDS NFS_RPC_SWAPFLAGS is only used for READ requests. It sets RPC_TASK_SWAPPER which gives some memory-allocation priority to requests. This is not needed for swap READ - though it is for writes where it is set via a different mechanism. RPC_TASK_ROOTCREDS causes the 'machine' credential to be used. This is not needed as the root credential is saved when the swap file is opened, and this is used for all IO. So NFS_RPC_SWAPFLAGS isn't needed, and as it is the only user of RPC_TASK_ROOTCREDS, that isn't needed either. Remove both. Signed-off-by: NeilBrown <neilb@suse.de> Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
b9f8713f |
|
11-Jul-2021 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
SUNRPC: Remove unnecessary memory barriers The only check for RPC_TASK_RUNNING is the one in rpc_make_runnable(), which happens under the same spin lock held when we call rpc_clear_running(). Ditto, the last check for RPC_TASK_QUEUED in rpc_execute() is performed under the same lock as the one held when we call rpc_clear_queued(). Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
85e39fee |
|
23-Jun-2021 |
Olga Kornievskaia <kolga@netapp.com> |
NFSv4.1 identify and mark RPC tasks that can move between transports In preparation for when we can re-try a task on a different transport, identify and mark such RPC tasks as moveable. Only 4.1+ operarations can be re-tried on a different transport. Signed-off-by: Olga Kornievskaia <kolga@netapp.com> Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
7eac5264 |
|
07-Feb-2020 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
SUNRPC: Add a flag to avoid reference counts on credentials Add a flag to signal to the RPC layer that the credential is already pinned for the duration of the RPC call. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
d5711920 |
|
16-Aug-2019 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
Revert "NFSv4/flexfiles: Abort I/O early if the layout segment was invalidated" This reverts commit a79f194aa4879e9baad118c3f8bb2ca24dbef765. The mechanism for aborting I/O is racy, since we are not guaranteed that the request is asleep while we're changing both task->tk_status and task->tk_action. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> Cc: stable@vger.kernel.org # v5.1
|
#
691b45dd |
|
19-Aug-2019 |
Chuck Lever <chuck.lever@oracle.com> |
SUNRPC: Remove rpc_wake_up_queued_task_on_wq() Clean up: commit c544577daddb ("SUNRPC: Clean up transport write space handling") appears to have removed the last caller of rpc_wake_up_queued_task_on_wq(). Signed-off-by: Chuck Lever <chuck.lever@oracle.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
5a0c257f |
|
29-May-2019 |
NeilBrown <neilb@suse.com> |
NFS: send state management on a single connection. With NFSv4.1, different network connections need to be explicitly bound to a session. During session startup, this is not possible so only a single connection must be used for session startup. So add a task flag to disable the default round-robin choice of connections (when nconnect > 1) and force the use of a single connection. Then use that flag on all requests for session management - for consistence, include NFSv4.0 management (SETCLIENTID) and session destruction Reported-by: Chuck Lever <chuck.lever@oracle.com> Signed-off-by: NeilBrown <neilb@suse.com> Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
7e0a0e38 |
|
01-May-2019 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
SUNRPC: Replace the queue timer with a delayed work function The queue timer function, which walks the RPC queue in order to locate candidates for waking up is one of the current constraints against removing the bh-safe queue spin locks. Replace it with a delayed work queue, so that we can do the actual rpc task wake ups from an ordinary process context. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
5ad64b36 |
|
07-Apr-2019 |
Trond Myklebust <trondmy@gmail.com> |
SUNRPC: Add tracking of RPC level errors Add variables to track RPC level errors so that we can distinguish between issue that arose in the RPC transport layer as opposed to those arising from the reply message. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
24a9d9a2 |
|
07-Apr-2019 |
Trond Myklebust <trondmy@gmail.com> |
SUNRPC: Simplify queue timeouts using timer_reduce() Simplify the setting of queue timeouts by using the timer_reduce() function. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
5efd1876 |
|
07-Apr-2019 |
Trond Myklebust <trondmy@gmail.com> |
SUNRPC: Fix up tracking of timeouts Add a helper to ensure that debugfs and friends print out the correct current task timeout value. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
6b2e6856 |
|
07-Apr-2019 |
Trond Myklebust <trondmy@gmail.com> |
SUNRPC: Add function rpc_sleep_on_timeout() Clean up the RPC task sleep interfaces by replacing the task->tk_timeout 'hidden parameter' to rpc_sleep_on() with a new function that takes an absolute timeout. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
8357a9b6 |
|
07-Apr-2019 |
Trond Myklebust <trondmy@gmail.com> |
SUNRPC: Remove unused argument 'action' from rpc_sleep_on_priority() None of the callers set the 'action' argument, so let's just remove it. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
ae67bd38 |
|
07-Apr-2019 |
Trond Myklebust <trondmy@gmail.com> |
SUNRPC: Fix up task signalling The RPC_TASK_KILLED flag should really not be set from another context because it can clobber data in the struct task when task->tk_flags is changed non-atomically. Let's therefore swap out RPC_TASK_KILLED with an atomic flag, and add a function to set that flag and safely wake up the task. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
af6b61d7 |
|
11-Apr-2019 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
Revert "SUNRPC: Micro-optimise when the task is known not to be sleeping" This reverts commit 009a82f6437490c262584d65a14094a818bcb747. The ability to optimise here relies on compiler being able to optimise away tail calls to avoid stack overflows. Unfortunately, we are seeing reports of problems, so let's just revert. Reported-by: Daniel Mack <daniel@zonque.org> Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
009a82f6 |
|
08-Mar-2019 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
SUNRPC: Micro-optimise when the task is known not to be sleeping In cases where we know the task is not sleeping, try to optimise away the indirect call to task->tk_action() by replacing it with a direct call. Only change tail calls, to allow gcc to perform tail call elimination. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
a79f194a |
|
27-Feb-2019 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
NFSv4/flexfiles: Abort I/O early if the layout segment was invalidated If a layout segment gets invalidated while a pNFS I/O operation is queued for transmission, then we ideally want to abort immediately. This is particularly the case when there is a large number of I/O related RPCs queued in the RPC layer, and the layout segment gets invalidated due to an ENOSPC error, or an EACCES (because the client was fenced). We may end up forced to spam the MDS with a lot of otherwise unnecessary LAYOUTERRORs after that I/O fails. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
a52458b4 |
|
02-Dec-2018 |
NeilBrown <neilb@suse.com> |
NFS/NFSD/SUNRPC: replace generic creds with 'struct cred'. SUNRPC has two sorts of credentials, both of which appear as "struct rpc_cred". There are "generic credentials" which are supplied by clients such as NFS and passed in 'struct rpc_message' to indicate which user should be used to authorize the request, and there are low-level credentials such as AUTH_NULL, AUTH_UNIX, AUTH_GSS which describe the credential to be sent over the wires. This patch replaces all the generic credentials by 'struct cred' pointers - the credential structure used throughout Linux. For machine credentials, there is a special 'struct cred *' pointer which is statically allocated and recognized where needed as having a special meaning. A look-up of a low-level cred will map this to a machine credential. Signed-off-by: NeilBrown <neilb@suse.com> Acked-by: J. Bruce Fields <bfields@redhat.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
1de7eea9 |
|
02-Dec-2018 |
NeilBrown <neilb@suse.com> |
SUNRPC: add side channel to use non-generic cred for rpc call. The credential passed in rpc_message.rpc_cred is always a generic credential except in one instance. When gss_destroying_context() calls rpc_call_null(), it passes a specific credential that it needs to destroy. In this case the RPC acts *on* the credential rather than being authorized by it. This special case deserves explicit support and providing that will mean that rpc_message.rpc_cred is *always* generic, allowing some optimizations. So add "tk_op_cred" to rpc_task and "rpc_op_cred" to the setup data. Use this to pass the cred down from rpc_call_null(), and have rpcauth_bindcred() notice it and bind it in place. Credit to kernel test robot <fengguang.wu@intel.com> for finding a bug in earlier version of this patch. Signed-off-by: NeilBrown <neilb@suse.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
a68a72e1 |
|
02-Dec-2018 |
NeilBrown <neilb@suse.com> |
SUNRPC: introduce RPC_TASK_NULLCREDS to request auth_none In almost all cases the credential stored in rpc_message.rpc_cred is a "generic" credential. One of the two expections is when an AUTH_NULL credential is used such as for RPC ping requests. To improve consistency, don't pass an explicit credential in these cases, but instead pass NULL and set a task flag, similar to RPC_TASK_ROOTCREDS, which requests that NULL credentials be used by default. Signed-off-by: NeilBrown <neilb@suse.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
ecd5f97e |
|
02-Dec-2018 |
NeilBrown <neilb@suse.com> |
SUNRPC: discard RPC_DO_ROOTOVERRIDE() it is never used. Signed-off-by: NeilBrown <neilb@suse.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
f42f7c28 |
|
08-Sep-2018 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
SUNRPC: Fix priority queue fairness Fix up the priority queue to not batch by owner, but by queue, so that we allow '1 << priority' elements to be dequeued before switching to the next priority queue. The owner field is still used to wake up requests in round robin order by owner to avoid single processes hogging the RPC layer by loading the queues. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
359c48c0 |
|
29-Aug-2018 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
SUNRPC: Add a helper to wake up a sleeping rpc_task and set its status Add a helper that will wake up a task that is sleeping on a specific queue, and will set the value of task->tk_status. This is mainly intended for use by the transport layer to notify the task of an error condition. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
cf9946cd |
|
05-Aug-2018 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
SUNRPC: Refactor the transport request pinning We are going to need to pin for both send and receive. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
7ebbbc6e |
|
28-Aug-2018 |
Trond Myklebust <trond.myklebust@hammerspace.com> |
SUNRPC: Simplify identification of when the message send/receive is complete Add states to indicate that the message send and receive are not yet complete. Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
#
2275cde4 |
|
07-Feb-2018 |
Trond Myklebust <trond.myklebust@primarydata.com> |
SUNRPC: Queue latency-sensitive socket tasks to xprtiod The response to a write_space notification is very latency sensitive, so we should queue it to the lower latency xprtiod_workqueue. This is something we already do for the other cases where an rpc task holds the transport XPRT_LOCKED bitlock. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
b2441318 |
|
01-Nov-2017 |
Greg Kroah-Hartman <gregkh@linuxfoundation.org> |
License cleanup: add SPDX GPL-2.0 license identifier to files with no license Many source files in the tree are missing licensing information, which makes it harder for compliance tools to determine the correct license. By default all files without license information are under the default license of the kernel, which is GPL version 2. Update the files which contain no license information with the 'GPL-2.0' SPDX license identifier. The SPDX identifier is a legally binding shorthand, which can be used instead of the full boiler plate text. This patch is based on work done by Thomas Gleixner and Kate Stewart and Philippe Ombredanne. How this work was done: Patches were generated and checked against linux-4.14-rc6 for a subset of the use cases: - file had no licensing information it it. - file was a */uapi/* one with no licensing information in it, - file was a */uapi/* one with existing licensing information, Further patches will be generated in subsequent months to fix up cases where non-standard license headers were used, and references to license had to be inferred by heuristics based on keywords. The analysis to determine which SPDX License Identifier to be applied to a file was done in a spreadsheet of side by side results from of the output of two independent scanners (ScanCode & Windriver) producing SPDX tag:value files created by Philippe Ombredanne. Philippe prepared the base worksheet, and did an initial spot review of a few 1000 files. The 4.13 kernel was the starting point of the analysis with 60,537 files assessed. Kate Stewart did a file by file comparison of the scanner results in the spreadsheet to determine which SPDX license identifier(s) to be applied to the file. She confirmed any determination that was not immediately clear with lawyers working with the Linux Foundation. Criteria used to select files for SPDX license identifier tagging was: - Files considered eligible had to be source code files. - Make and config files were included as candidates if they contained >5 lines of source - File already had some variant of a license header in it (even if <5 lines). All documentation files were explicitly excluded. The following heuristics were used to determine which SPDX license identifiers to apply. - when both scanners couldn't find any license traces, file was considered to have no license information in it, and the top level COPYING file license applied. For non */uapi/* files that summary was: SPDX license identifier # files ---------------------------------------------------|------- GPL-2.0 11139 and resulted in the first patch in this series. If that file was a */uapi/* path one, it was "GPL-2.0 WITH Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was: SPDX license identifier # files ---------------------------------------------------|------- GPL-2.0 WITH Linux-syscall-note 930 and resulted in the second patch in this series. - if a file had some form of licensing information in it, and was one of the */uapi/* ones, it was denoted with the Linux-syscall-note if any GPL family license was found in the file or had no licensing in it (per prior point). Results summary: SPDX license identifier # files ---------------------------------------------------|------ GPL-2.0 WITH Linux-syscall-note 270 GPL-2.0+ WITH Linux-syscall-note 169 ((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21 ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17 LGPL-2.1+ WITH Linux-syscall-note 15 GPL-1.0+ WITH Linux-syscall-note 14 ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5 LGPL-2.0+ WITH Linux-syscall-note 4 LGPL-2.1 WITH Linux-syscall-note 3 ((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3 ((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1 and that resulted in the third patch in this series. - when the two scanners agreed on the detected license(s), that became the concluded license(s). - when there was disagreement between the two scanners (one detected a license but the other didn't, or they both detected different licenses) a manual inspection of the file occurred. - In most cases a manual inspection of the information in the file resulted in a clear resolution of the license that should apply (and which scanner probably needed to revisit its heuristics). - When it was not immediately clear, the license identifier was confirmed with lawyers working with the Linux Foundation. - If there was any question as to the appropriate license identifier, the file was flagged for further research and to be revisited later in time. In total, over 70 hours of logged manual review was done on the spreadsheet to determine the SPDX license identifiers to apply to the source files by Kate, Philippe, Thomas and, in some cases, confirmation by lawyers working with the Linux Foundation. Kate also obtained a third independent scan of the 4.13 code base from FOSSology, and compared selected files where the other two scanners disagreed against that SPDX file, to see if there was new insights. The Windriver scanner is based on an older version of FOSSology in part, so they are related. Thomas did random spot checks in about 500 files from the spreadsheets for the uapi headers and agreed with SPDX license identifier in the files he inspected. For the non-uapi files Thomas did random spot checks in about 15000 files. In initial set of patches against 4.14-rc6, 3 files were found to have copy/paste license identifier errors, and have been fixed to reflect the correct identifier. Additionally Philippe spent 10 hours this week doing a detailed manual inspection and review of the 12,461 patched files from the initial patch version early this week with: - a full scancode scan run, collecting the matched texts, detected license ids and scores - reviewing anything where there was a license detected (about 500+ files) to ensure that the applied SPDX license was correct - reviewing anything where there was no detection but the patch license was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied SPDX license was correct This produced a worksheet with 20 files needing minor correction. This worksheet was then exported into 3 different .csv files for the different types of files to be modified. These .csv files were then reviewed by Greg. Thomas wrote a script to parse the csv files and add the proper SPDX tag to the file, in the format that the file expected. This script was further refined by Greg based on the output to detect more types of files automatically and to distinguish between header and source .c files (which need different comment types.) Finally Greg ran the script using the .csv files to generate the patches. Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org> Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
|
#
729749bb |
|
13-Aug-2017 |
Trond Myklebust <trond.myklebust@primarydata.com> |
SUNRPC: Don't hold the transport lock across socket copy operations Instead add a mechanism to ensure that the request doesn't disappear from underneath us while copying from the socket. We do this by preventing xprt_release() from freeing the XDR buffers until the flag RPC_TASK_MSG_RECV has been cleared from the request. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com> Reviewed-by: Chuck Lever <chuck.lever@oracle.com>
|
#
511e936b |
|
12-May-2017 |
Christoph Hellwig <hch@lst.de> |
sunrpc: mark all struct rpc_procinfo instances as const struct rpc_procinfo contains function pointers, and marking it as constant avoids it being able to be used as an attach vector for code injections. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
5dd43ce2 |
|
19-Jun-2017 |
Ingo Molnar <mingo@kernel.org> |
sched/wait: Split out the wait_bit*() APIs from <linux/wait.h> into <linux/wait_bit.h> The wait_bit*() types and APIs are mixed into wait.h, but they are a pretty orthogonal extension of wait-queues. Furthermore, only about 50 kernel files use these APIs, while over 1000 use the regular wait-queue functionality. So clean up the main wait.h by moving the wait-bit functionality out of it, into a separate .h and .c file: include/linux/wait_bit.h for types and APIs kernel/sched/wait_bit.c for the implementation Update all header dependencies. This reduces the size of wait.h rather significantly, by about 30%. Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-kernel@vger.kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org>
|
#
499b4988 |
|
12-May-2017 |
Christoph Hellwig <hch@lst.de> |
sunrpc: mark all struct rpc_procinfo instances as const struct rpc_procinfo contains function pointers, and marking it as constant avoids it being able to be used as an attach vector for code injections. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
3435c74a |
|
15-Sep-2016 |
Chuck Lever <chuck.lever@oracle.com> |
SUNRPC: Generalize the RPC buffer release API xprtrdma needs to allocate the Call and Reply buffers separately. TBH, the reliance on using a single buffer for the pair of XDR buffers is transport implementation-specific. Instead of passing just the rq_buffer into the buf_free method, pass the task structure and let buf_free take care of freeing both XDR buffers at once. There's a micro-optimization here. In the common case, both xprt_release and the transport's buf_free method were checking if rq_buffer was NULL. Now the check is done only once per RPC. Signed-off-by: Chuck Lever <chuck.lever@oracle.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
5fe6eaa1 |
|
15-Sep-2016 |
Chuck Lever <chuck.lever@oracle.com> |
SUNRPC: Generalize the RPC buffer allocation API xprtrdma needs to allocate the Call and Reply buffers separately. TBH, the reliance on using a single buffer for the pair of XDR buffers is transport implementation-specific. Transports that want to allocate separate Call and Reply buffers will ignore the "size" argument anyway. Don't bother passing it. The buf_alloc method can't return two pointers. Instead, make the method's return value an error code, and set the rq_buffer pointer in the method itself. This gives call_allocate an opportunity to terminate an RPC instead of looping forever when a permanent problem occurs. If a request is just bogus, or the transport is in a state where it can't allocate resources for any request, there needs to be a way to kill the RPC right there and not loop. This immediately fixes a rare problem in the backchannel send path, which loops if the server happens to send a CB request whose call+reply size is larger than a page (which it shouldn't do yet). One more issue: looks like xprt_inject_disconnect was incorrectly placed in the failure path in call_allocate. It needs to be in the success path, as it is for other call-sites. Signed-off-by: Chuck Lever <chuck.lever@oracle.com> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
|
#
f1dc237c |
|
26-May-2016 |
Trond Myklebust <trond.myklebust@primarydata.com> |
SUNRPC: Reduce latency when send queue is congested Use the low latency transport workqueue to process the task that is next in line on the xprt->sending queue. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
40a5f1b1 |
|
27-May-2016 |
Trond Myklebust <trond.myklebust@primarydata.com> |
SUNRPC: RPC transport queue must be low latency rpciod can easily get congested due to the long list of queued rpc_tasks. Having the receive queue wait in turn for those tasks to complete can therefore be a bottleneck. Address the problem by separating the workqueues into: - rpciod: manages rpc_tasks - xprtiod: manages transport related work. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
9d61498d |
|
30-Jan-2016 |
Trond Myklebust <trond.myklebust@primarydata.com> |
SUNRPC: Allow caller to specify the transport to use This is needed in order to allow the NFSv4.1 backchannel and BIND_CONN_TO_SESSION function to work. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
fb43d172 |
|
30-Jan-2016 |
Trond Myklebust <trond.myklebust@primarydata.com> |
SUNRPC: Use the multipath iterator to assign a transport to each task Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
5edd1051 |
|
25-Feb-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
SUNRPC: Reorder rpc_task to put waitqueue related info in same cachelines Try to group all the data required by the waitqueues, their timers and timer callbacks into the same cachelines for performance. With this reordering, "pahole" reports the following structure on x86_64: struct rpc_task { atomic_t tk_count; /* 0 4 */ int tk_status; /* 4 4 */ struct list_head tk_task; /* 8 16 */ void (*tk_callback)(struct rpc_task *); /* 24 void (*tk_action)(struct rpc_task *); /* 32 long unsigned int tk_timeout; /* 40 8 */ long unsigned int tk_runstate; /* 48 8 */ struct rpc_wait_queue * tk_waitqueue; /* 56 8 */ /* --- cacheline 1 boundary (64 bytes) --- */ union { struct work_struct tk_work; /* 64 */ struct rpc_wait tk_wait; /* 56 */ } u; /* 64 64 */ /* --- cacheline 2 boundary (128 bytes) --- */ struct rpc_message tk_msg; /* 128 32 */ void * tk_calldata; /* 160 8 */ const struct rpc_call_ops * tk_ops; /* 168 8 */ struct rpc_clnt * tk_client; /* 176 8 */ struct rpc_rqst * tk_rqstp; /* 184 8 */ /* --- cacheline 3 boundary (192 bytes) --- */ struct workqueue_struct * tk_workqueue; /* 192 8 */ ktime_t tk_start; /* 200 8 */ pid_t tk_owner; /* 208 4 */ short unsigned int tk_flags; /* 212 2 */ short unsigned int tk_timeouts; /* 214 2 */ short unsigned int tk_pid; /* 216 2 */ unsigned char tk_priority:2; /* 218: 6 1 */ unsigned char tk_garb_retry:2; /* 218: 4 1 */ unsigned char tk_cred_retry:2; /* 218: 2 1 */ unsigned char tk_rebind_retry:2; /* 218: 0 1 */ /* size: 224, cachelines: 4, members: 24 */ /* padding: 5 */ /* last cacheline: 32 bytes */ }; whereas on i386, it reports everything fitting into the 1st cacheline. Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
3c87ef6e |
|
03-Jun-2015 |
Jeff Layton <jlayton@kernel.org> |
sunrpc: keep a count of swapfiles associated with the rpc_clnt Jerome reported seeing a warning pop when working with a swapfile on NFS. The nfs_swap_activate can end up calling sk_set_memalloc while holding the rcu_read_lock and that function can sleep. To fix that, we need to take a reference to the xprt while holding the rcu_read_lock, set the socket up for swapping and then drop that reference. But, xprt_put is not exported and having NFS deal with the underlying xprt is a bit of layering violation anyway. Fix this by adding a set of activate/deactivate functions that take a rpc_clnt pointer instead of an rpc_xprt, and have nfs_swap_activate and nfs_swap_deactivate call those. Also, add a per-rpc_clnt atomic counter to keep track of the number of active swapfiles associated with it. When the counter does a 0->1 transition, we enable swapping on the xprt, when we do a 1->0 transition we disable swapping on it. This also allows us to be a bit more selective with the RPC_TASK_SWAPPER flag. If non-swapper and swapper clnts are sharing a xprt, then we only need to flag the tasks from the swapper clnt with that flag. Acked-by: Mel Gorman <mgorman@suse.de> Reported-by: Jerome Marchand <jmarchan@redhat.com> Signed-off-by: Jeff Layton <jeff.layton@primarydata.com> Reviewed-by: Chuck Lever <chuck.lever@oracle.com> Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
0f419791 |
|
01-Jun-2015 |
Trond Myklebust <trond.myklebust@primarydata.com> |
SUNRPC: Remove unused argument 'tk_ops' in rpc_run_bc_task Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
1306729b |
|
17-Nov-2014 |
Jeff Layton <jlayton@kernel.org> |
sunrpc: eliminate RPC_TRACEPOINTS It's always set to the same value as CONFIG_TRACEPOINTS, so we can just use that instead. Signed-off-by: Jeff Layton <jlayton@primarydata.com> Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
f895b252 |
|
17-Nov-2014 |
Jeff Layton <jlayton@kernel.org> |
sunrpc: eliminate RPC_DEBUG It's always set to whatever CONFIG_SUNRPC_DEBUG is, so just use that. Signed-off-by: Jeff Layton <jlayton@primarydata.com> Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
|
#
c1221321 |
|
06-Jul-2014 |
NeilBrown <neilb@suse.de> |
sched: Allow wait_on_bit_action() functions to support a timeout It is currently not possible for various wait_on_bit functions to implement a timeout. While the "action" function that is called to do the waiting could certainly use schedule_timeout(), there is no way to carry forward the remaining timeout after a false wake-up. As false-wakeups a clearly possible at least due to possible hash collisions in bit_waitqueue(), this is a real problem. The 'action' function is currently passed a pointer to the word containing the bit being waited on. No current action functions use this pointer. So changing it to something else will be a little noisy but will have no immediate effect. This patch changes the 'action' function to take a pointer to the "struct wait_bit_key", which contains a pointer to the word containing the bit so nothing is really lost. It also adds a 'private' field to "struct wait_bit_key", which is initialized to zero. An action function can now implement a timeout with something like static int timed_out_waiter(struct wait_bit_key *key) { unsigned long waited; if (key->private == 0) { key->private = jiffies; if (key->private == 0) key->private -= 1; } waited = jiffies - key->private; if (waited > 10 * HZ) return -EAGAIN; schedule_timeout(waited - 10 * HZ); return 0; } If any other need for context in a waiter were found it would be easy to use ->private for some other purpose, or even extend "struct wait_bit_key". My particular need is to support timeouts in nfs_release_page() to avoid deadlocks with loopback mounted NFS. While wait_on_bit_timeout() would be a cleaner interface, it will not meet my need. I need the timeout to be sensitive to the state of the connection with the server, which could change. So I need to use an 'action' interface. Signed-off-by: NeilBrown <neilb@suse.de> Acked-by: Peter Zijlstra <peterz@infradead.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Steve French <sfrench@samba.org> Cc: David Howells <dhowells@redhat.com> Cc: Steven Whitehouse <swhiteho@redhat.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: http://lkml.kernel.org/r/20140707051604.28027.41257.stgit@notabene.brown Signed-off-by: Ingo Molnar <mingo@kernel.org>
|
#
4e857c58 |
|
17-Mar-2014 |
Peter Zijlstra <peterz@infradead.org> |
arch: Mass conversion of smp_mb__*() Mostly scripted conversion of the smp_mb__* barriers. Signed-off-by: Peter Zijlstra <peterz@infradead.org> Acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Link: http://lkml.kernel.org/n/tip-55dhyhocezdw1dg7u19hmh1u@git.kernel.org Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: linux-arch@vger.kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org>
|
#
8a19a0b6 |
|
23-Sep-2013 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Add RPC task and client level options to disable the resend timeout Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
8d1018c7 |
|
04-Sep-2013 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Ensure rpc_task->tk_pid is available for tracepoints Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
786615bc |
|
05-Aug-2013 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: If the rpcbind channel is disconnected, fail the call to unregister If rpcbind causes our connection to the AF_LOCAL socket to close after we've registered a service, then we want to be careful about reconnecting since the mount namespace may have changed. By simply refusing to reconnect the AF_LOCAL socket in the case of unregister, we avoid the need to somehow save the mount namespace. While this may lead to some services not unregistering properly, it should be safe. Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com> Cc: Nix <nix@esperi.org.uk> Cc: Jeff Layton <jlayton@redhat.com> Cc: stable@vger.kernel.org # 3.9.x
|
#
74fe5f7c |
|
21-May-2013 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Remove unused functions rpc_task_set/has_priority Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
64bbe3d6 |
|
20-May-2013 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Remove the unused helpers task_for_each() and task_for_first() Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
0053a8e6 |
|
20-May-2013 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Remove unused function rpc_queue_empty Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
77102893 |
|
08-Jan-2013 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Nuke the tk_xprt macro It is no longer in use Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
c05eecf6 |
|
30-Nov-2012 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Don't allow low priority tasks to pre-empt higher priority ones Currently, the priority queues attempt to be 'fair' to lower priority tasks by scheduling them after a certain number of higher priority tasks have run. The problem is that both the transport send queue and the NFSv4.1 session slot queue have strong ordering requirements. This patch therefore removes the fairness code in favour of strong ordering of task priorities. Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
0a702195 |
|
17-Feb-2012 |
Weston Andros Adamson <dros@netapp.com> |
NFS: include filelayout DS rpc stats in mountstats Include RPC statistics from all data servers in /proc/self/mountstats for pNFS filelayout mounts. Signed-off-by: Weston Andros Adamson <dros@netapp.com> Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
2f09c242 |
|
08-Feb-2012 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Ensure that we can trace waitqueues when !defined(CONFIG_SYSCTL) The tracepoint code relies on the queue->name being defined in order to be able to display the name of the waitqueue on which an RPC task is sleeping. Reported-by: Randy Dunlap <rdunlap@xenotime.net> Reported-by: Steven Rostedt <rostedt@goodmis.org> Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com> Acked-by: Steven Rostedt <rostedt@goodmis.org> Acked-by: Randy Dunlap <rdunlap@xenotime.net>
|
#
82b0a4c3 |
|
20-Jan-2012 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Add trace events to the sunrpc subsystem Add declarations to allow tracing of RPC call creation, running, sleeping, and destruction. Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
961a828d |
|
17-Jan-2012 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Fix potential races in xprt_lock_write_next() We have to ensure that the wake up from the waitqueue and the assignment of xprt->snd_task are atomic. We can do this by assigning the snd_task while under the waitqueue spinlock. Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
70abc49b |
|
12-Jan-2012 |
Stanislav Kinsbursky <skinsbursky@parallels.com> |
SUNRPC: make SUNPRC clients list per network namespace context This patch moves static SUNRPC clients list and it's lock to sunrpc_net structure. Currently this list is used only for debug purposes. But later it will be used also for selecting clients by networks namespace on PipeFS mount/umount events. Per-network namespace lists will make this faster and simplier. Note: client list is taken from "init_net" network namespace context in rpc_show_tasks(). This will be changed some day later with making SUNRPC sysctl's per network namespace context. Signed-off-by: Stanislav Kinsbursky <skinsbursky@parallels.com> Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
3b27bad7 |
|
17-Jul-2011 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Allow caller of rpc_sleep_on() to select priority levels Currently, the caller has to change the value of task->tk_priority if it wants to select on which priority level the task will sleep. This patch allows the caller to select a priority level at sleep time rather than always using task->tk_priority. Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
0b760113 |
|
31-May-2011 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
NLM: Don't hang forever on NLM unlock requests If the NLM daemon is killed on the NFS server, we can currently end up hanging forever on an 'unlock' request, instead of aborting. Basically, if the rpcbind request fails, or the server keeps returning garbage, we really want to quit instead of retrying. Tested-by: Vasily Averin <vvs@sw.ru> Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com> Cc: stable@kernel.org
|
#
7494d00c |
|
24-Apr-2011 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Allow RPC calls to return ETIMEDOUT instead of EIO On occasion, it is useful for the NFS layer to distinguish between soft timeouts and other EIO errors due to (say) encoding errors, or authentication errors. The following patch ensures that the default behaviour of the RPC layer remains to return EIO on soft timeouts (until we have audited all the callers). Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
468f8613 |
|
18-Apr-2011 |
Bryan Schumaker <bjschuma@netapp.com> |
NFSv4.1: Don't update sequence number if rpc_task is not sent If we fail to contact the gss upcall program, then no message will be sent to the server. The client still updated the sequence number, however, and this lead to NFS4ERR_SEQ_MISMATCH for the next several RPC calls. Signed-off-by: Bryan Schumaker <bjschuma@netapp.com> Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
bf294b41 |
|
21-Feb-2011 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Close a race in __rpc_wait_for_completion_task() Although they run as rpciod background tasks, under normal operation (i.e. no SIGKILL), functions like nfs_sillyrename(), nfs4_proc_unlck() and nfs4_do_close() want to be fully synchronous. This means that when we exit, we want all references to the rpc_task to be gone, and we want any dentry references etc. held by that task to be released. For this reason these functions call __rpc_wait_for_completion_task(), followed by rpc_put_task() in the expectation that the latter will be releasing the last reference to the rpc_task, and thus ensuring that the callback_ops->rpc_release() has been called synchronously. This patch fixes a race which exists due to the fact that rpciod calls rpc_complete_task() (in order to wake up the callers of __rpc_wait_for_completion_task()) and then subsequently calls rpc_put_task() without ensuring that these two steps are done atomically. In order to avoid adding new spin locks, the patch uses the existing waitqueue spin lock to order the rpc_task reference count releases between the waiting process and rpciod. The common case where nobody is waiting for completion is optimised for by checking if the RPC_TASK_ASYNC flag is cleared and/or if the rpc_task reference count is 1: in those cases we drop trying to grab the spin lock, and immediately free up the rpc_task. Those few processes that need to put the rpc_task from inside an asynchronous context and that do not care about ordering are given a new helper: rpc_put_task_async(). Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
d9b6cd94 |
|
31-Jul-2010 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Ensure that rpc_exit() always wakes up a sleeping task Make rpc_exit() non-inline, and ensure that it always wakes up a task that has been queued. Kill off the now unused rpc_wake_up_task(). Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
9bb0b813 |
|
12-May-2010 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Reorder the struct rpc_task fields This improves the packing of the rpc_task, and ensures that on 64-bit platforms the size reduces to 216 bytes. Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
d72b6cec |
|
12-May-2010 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Remove the 'tk_magic' debugging field It has not triggered in almost a decade. Time to get rid of it... Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
d60dbb20 |
|
12-May-2010 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Move the task->tk_bytes_sent and tk_rtt to struct rpc_rqst It seems strange to maintain stats for bytes_sent in one structure, and bytes received in another. Try to assemble all the RPC request-related stats in struct rpc_rqst Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
ff839970 |
|
07-May-2010 |
Chuck Lever <chuck.lever@oracle.com> |
SUNRPC: Replace jiffies-based metrics with ktime-based metrics Currently RPC performance metrics that tabulate elapsed time use jiffies time values. This is problematic on systems that use slow jiffies (for instance 100HZ systems built for paravirtualized environments). It is also a problem for computing precise latency statistics for advanced network transports, such as InfiniBand, that can have round-trip latencies significanly faster than a single clock tick. For the RPC client, adopt the high resolution time stamp mechanism already used by the network layer and blktrace: ktime. We use ktime format time stamps for all internal computations, and convert to milliseconds for presentation. As a result, we need only addition operations in the performance critical paths; multiply/divide is required only for presentation. We could report RTT metrics in microseconds. In fact the mountstats format is versioned to accomodate exactly this kind of interface improvement. For now, however, we'll stay with millisecond precision for presentation to maintain backwards compatibility with the handful of currently deployed user space tools. At a later point, we'll move to an API such as BDI_STATS where a finer timestamp precision can be reported. Signed-off-by: Chuck Lever <chuck.lever@oracle.com> Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
cf3b01b5 |
|
14-Dec-2009 |
Alexandros Batsakis <batsakis@netapp.com> |
rpc: add a new priority in RPC task Signed-off-by: Alexandros Batsakis <batsakis@netapp.com> Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
48f18612 |
|
14-Dec-2009 |
Alexandros Batsakis <batsakis@netapp.com> |
rpc: add rpc_queue_empty function Signed-off-by: Alexandros Batsakis <batsakis@netapp.com> Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
09a21c41 |
|
03-Dec-2009 |
Chuck Lever <chuck.lever@oracle.com> |
SUNRPC: Allow RPCs to fail quickly if the server is unreachable The kernel sometimes makes RPC calls to services that aren't running. Because the kernel's RPC client always assumes the hard retry semantic when reconnecting a connection-oriented RPC transport, the underlying reconnect logic takes a long while to time out, even though the remote may have responded immediately with ECONNREFUSED. In certain cases, like upcalls to our local rpcbind daemon, or for NFS mount requests, we'd like the kernel to fail immediately if the remote service isn't reachable. This allows another transport to be tried immediately, or the pending request can be abandoned quickly. Introduce a per-request flag which controls how call_transmit_status() behaves when request transmission fails because the server cannot be reached. We don't want soft connection semantics to apply to other errors. The default case of the switch statement in call_transmit_status() no longer falls through; the fall through code is copied to the default case, and a "break;" is added. The transport's connection re-establishment timeout is also ignored for such requests. We want the request to fail immediately, so the reconnect delay is skipped. Additionally, we don't want a connect failure here to further increase the reconnect timeout value, since this request will not be retried. Signed-off-by: Chuck Lever <chuck.lever@oracle.com> Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
55ae1aab |
|
01-Apr-2009 |
Ricardo Labiaga <Ricardo.Labiaga@netapp.com> |
nfs41: Add backchannel processing support to RPC state machine Adds rpc_run_bc_task() which is called by the NFS callback service to process backchannel requests. It performs similar work to rpc_run_task() though "schedules" the backchannel task to be executed starting at the call_trasmit state in the RPC state machine. It also introduces some miscellaneous updates to the argument validation, call_transmit, and transport cleanup functions to take into account that there are now forechannel and backchannel tasks. Backchannel requests do not carry an RPC message structure, since the payload has already been XDR encoded using the existing NFSv4 callback mechanism. Introduce a new transmit state for the client to reply on to backchannel requests. This new state simply reserves the transport and issues the reply. In case of a connection related error, disconnects the transport and drops the reply. It requires the forechannel to re-establish the connection and the server to retransmit the request, as stated in NFSv4.1 section 2.9.2 "Client and Server Transport Behavior". Note: There is no need to loop attempting to reserve the transport. If EAGAIN is returned by xprt_prepare_transmit(), return with tk_status == 0, setting tk_action to call_bc_transmit. rpc_execute() will invoke it again after the task is taken off the sleep queue. [nfs41: rpc_run_bc_task() need not be exported outside RPC module] [nfs41: New call_bc_transmit RPC state] Signed-off-by: Ricardo Labiaga <Ricardo.Labiaga@netapp.com> Signed-off-by: Benny Halevy <bhalevy@panasas.com> [nfs41: Backchannel: No need to loop in call_bc_transmit()] Signed-off-by: Andy Adamson <andros@netapp.com> Signed-off-by: Ricardo Labiaga <Ricardo.Labiaga@netapp.com> Signed-off-by: Benny Halevy <bhalevy@panasas.com> [rpc_count_iostats incorrectly exits early] Signed-off-by: Ricardo Labiaga <Ricardo.Labiaga@netapp.com> Signed-off-by: Benny Halevy <bhalevy@panasas.com> [Convert rpc_reply_expected() to inline function] [Remove unnecessary BUG_ON()] [Rename variable] Signed-off-by: Ricardo Labiaga <Ricardo.Labiaga@netapp.com> Signed-off-by: Benny Halevy <bhalevy@panasas.com>
|
#
aae2006e |
|
01-Apr-2009 |
Andy Adamson <andros@netapp.com> |
nfs41: sunrpc: Export the call prepare state for session reset Signed-off-by: Andy Adamson<andros@netapp.com> Signed-off-by: Benny Halevy <bhalevy@panasas.com> Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
a486aeda |
|
09-Jun-2008 |
\\\"J. Bruce Fields\\\ <bfields@citi.umich.edu> |
rpc: minor cleanup of scheduler callback code Try to make the comment here a little more clear and concise. Also, this macro definition seems unnecessary. Signed-off-by: J. Bruce Fields <bfields@citi.umich.edu> Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
5e4424af |
|
25-Feb-2008 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Remove now-redundant RCU-safe rpc_task free path Now that we've tightened up the locking rules for RPC queue wakeups, we can remove the RCU-safe kfree calls... Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
f5fb7b06 |
|
25-Feb-2008 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Eliminate the now-redundant rpc_start_wakeup() Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
eb276c0e |
|
22-Feb-2008 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Switch tasks to using the rpc_waitqueue's timer function Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
36df9aae |
|
18-Jul-2007 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Add a timer function to wait queues. This is designed to replace the timeout timer in the individual rpc_tasks. By putting the timer function in the wait queue, we will eventually be able to reduce the total number of timers in use by the RPC subsystem. Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
f6a1cc89 |
|
22-Feb-2008 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Add a (empty for the moment) destructor for rpc_wait_queues Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
5d00837b |
|
22-Feb-2008 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Run rpc timeout functions as callbacks instead of in softirqs An audit of the current RPC timeout functions shows that they don't really ever need to run in the softirq context. As long as the softirq is able to signal that the wakeup is due to a timeout (which it can do by setting task->tk_status to -ETIMEDOUT) then the callback functions can just run as standard task->tk_callback functions (in the rpciod/process context). The only possible border-line case would be xprt_timer() for the case of UDP, when the callback is used to reduce the size of the transport congestion window. In testing, however, the effect of moving that update to a callback would appear to be minor. Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
fda13939 |
|
22-Feb-2008 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Convert users of rpc_wake_up_task to use rpc_wake_up_queued_task Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
96ef13b2 |
|
22-Feb-2008 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Add a new helper rpc_wake_up_queued_task() In all cases where we currently use rpc_wake_up_task(), we almost always know on which waitqueue the rpc_task is actually sleeping. This will allows us to simplify the queue locking in a future patch. Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
32bfb5c0 |
|
19-Feb-2008 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Allow the rpc_release() callback to be run on another workqueue A lot of the work done by the rpc_release() callback is inappropriate for rpciod as it will often involve things like starting a new rpc call in order to clean up state after an interrupted NFSv4 open() call, or calls to mntput(), etc. This patch allows the caller of rpc_run_task() to specify that the rpc_release callback should run on a different workqueue than the default rpciod_workqueue. Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
c087567d |
|
18-Jul-2007 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Remove the obsolete RPC_WAITQ macro Now that we've killed off all the users. Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
47fe0648 |
|
25-Oct-2007 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Unexport rpc_init_task() and rpc_execute() Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
e8f5d77c |
|
25-Oct-2007 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: allow the caller of rpc_run_task to preallocate the struct rpc_task Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
3ff7576d |
|
14-Jul-2007 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Clean up the initialisation of priority queue scheduling info. We want the default scheduling priority (priority == 0) to remain RPC_PRIORITY_NORMAL. Also ensure that the priority wait queue scheduling is per process id instead of sometimes being per thread, and sometimes being per inode. Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
c970aa85 |
|
14-Jul-2007 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Clean up rpc_run_task Make it use the new task initialiser structure instead of acting as a wrapper. Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
84115e1c |
|
14-Jul-2007 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Cleanup of rpc_task initialisation Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
150030b7 |
|
06-Dec-2007 |
Matthew Wilcox <willy@infradead.org> |
NFS: Switch from intr mount option to TASK_KILLABLE By using the TASK_KILLABLE infrastructure, we can get rid of the 'intr' mount option. We have to use _killable everywhere instead of _interruptible as we get rid of rpc_clnt_sigmask/sigunmask. Signed-off-by: Liam R. Howlett <howlett@gmail.com> Signed-off-by: Matthew Wilcox <willy@linux.intel.com>
|
#
1be27f36 |
|
27-Jun-2007 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Remove the tk_auth macro... We should almost always be deferencing the rpc_auth struct by means of the credential's cr_auth field instead of the rpc_clnt->cl_auth anyway. Fix up that historical mistake, and remove the macro that propagated it. Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
6529eba0 |
|
14-Jun-2007 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Move rpc_task->tk_task list into struct rpc_clnt Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
c5a4dd8b |
|
29-Mar-2007 |
Chuck Lever <chuck.lever@oracle.com> |
SUNRPC: Eliminate side effects from rpc_malloc Currently rpc_malloc sets req->rq_buffer internally. Make this a more generic interface: return a pointer to the new buffer (or NULL) and make the caller set req->rq_buffer and req->rq_bufsize. This looks much more like kmalloc and eliminates the side effects. To fix a potential deadlock, this patch also replaces GFP_NOFS with GFP_NOWAIT in rpc_malloc. This prevents async RPCs from sleeping outside the RPC's task scheduler while allocating their buffer. Signed-off-by: Chuck Lever <chuck.lever@oracle.com> Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
7b8f850b |
|
20-Feb-2007 |
Ralf Baechle <ralf@linux-mips.org> |
[PATCH] Fix build errors if bitop functions are do {} while macros If one of clear_bit, change_bit or set_bit is defined as a do { } while (0) function usage of these functions in parenthesis like (foo_bit(23, &var)) while be expaned to something like (do { ... } while (0)}). resulting in a build error. This patch removes the useless parenthesis. Signed-off-by: Ralf Baechle <ralf@linux-mips.org> Cc: Trond Myklebust <trond.myklebust@fys.uio.no> Cc: Neil Brown <neilb@suse.de> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
|
#
2efef837 |
|
03-Feb-2007 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
RPC: Clean up rpc_execute... The error values are already propagated through task->tk_status, and none of the callers check one without checking the other, so we can drop the return value. Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
bde8f00c |
|
24-Jan-2007 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
[PATCH] NFS: Fix Oops in rpc_call_sync() Fix the Oops in http://bugzilla.linux-nfs.org/show_bug.cgi?id=138 We shouldn't be calling rpc_release_task() for tasks that are not active. Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
|
#
6cfd76a2 |
|
06-Dec-2006 |
Peter Zijlstra <a.p.zijlstra@chello.nl> |
[PATCH] lockdep: name some old style locks Name some of the remaning 'old_style_spin_init' locks Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl> Acked-by: Ingo Molnar <mingo@elte.hu> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
|
#
bbd5a1f9 |
|
18-Oct-2006 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Fix up missing BKL in asynchronous RPC callback functions Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
8aca67f0 |
|
13-Nov-2006 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Fix a potential race in rpc_wake_up_task() Use RCU to ensure that we can safely call rpc_finish_wakeup after we've called __rpc_do_wake_up_task. If not, there is a theoretical race, in which the rpc_task finishes executing, and gets freed first. Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
e6b3c4db |
|
11-Nov-2006 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
Fix a second potential rpc_wakeup race... Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
5b1eacbc |
|
22-Aug-2006 |
Chuck Lever <chuck.lever@oracle.com> |
SUNRPC: Support for RPC child tasks no longer needed The previous patches removed the last user of RPC child tasks, so we can remove support for child tasks from net/sunrpc/sched.c now. Signed-off-by: Chuck Lever <chuck.lever@oracle.com> Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
dead28da |
|
20-Mar-2006 |
Chuck Lever <cel@netapp.com> |
SUNRPC: eliminate rpc_call() Clean-up: replace rpc_call() helper with direct call to rpc_call_sync. This makes NFSv2 and NFSv3 synchronous calls more computationally efficient, and reduces stack consumption in functions that used to invoke rpc_call more than once. Test plan: Compile kernel with CONFIG_NFS enabled. Connectathon on NFS version 2, version 3, and version 4 mount points. Signed-off-by: Chuck Lever <cel@netapp.com> Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
ef759a2e |
|
20-Mar-2006 |
Chuck Lever <cel@netapp.com> |
SUNRPC: introduce per-task RPC iostats Account for various things that occur while an RPC task is executed. Separate timers for RPC round trip and RPC execution time show how long RPC requests wait in queue before being sent. Eventually these will be accumulated at xprt_release time in one place where they can be viewed from userland. Test plan: Compile kernel with CONFIG_NFS enabled. Signed-off-by: Chuck Lever <cel@netapp.com> Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
e19b63da |
|
20-Mar-2006 |
Chuck Lever <cel@netapp.com> |
SUNRPC: track length of RPC wait queues RPC wait queue length will eventually be exported to userland via the RPC iostats interface. Test plan: Compile kernel with CONFIG_NFS enabled. Signed-off-by: Chuck Lever <cel@netapp.com> Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
24c5d9d7 |
|
20-Mar-2006 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Run rpci->queue_timeout on the rpciod workqueue instead of generic Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
02107148 |
|
03-Jan-2006 |
Chuck Lever <cel@netapp.com> |
SUNRPC: switchable buffer allocation Add RPC client transport switch support for replacing buffer management on a per-transport basis. In the current IPv4 socket transport implementation, RPC buffers are allocated as needed for each RPC message that is sent. Some transport implementations may choose to use pre-allocated buffers for encoding, sending, receiving, and unmarshalling RPC messages, however. For transports capable of direct data placement, the buffers can be carved out of a pre-registered area of memory rather than from a slab cache. Test-plan: Millions of fsx operations. Performance characterization with "sio" and "iozone". Use oprofile and other tools to look for significant regression in CPU utilization. Signed-off-by: Chuck Lever <cel@netapp.com> Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
44c28873 |
|
03-Jan-2006 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
NFSv4: stateful NFSv4 RPC call interface The NFSv4 model requires us to complete all RPC calls that might establish state on the server whether or not the user wants to interrupt it. We may also need to schedule new work (including new RPC calls) in order to cancel the new state. The asynchronous RPC model will allow us to ensure that RPC calls always complete, but in order to allow for "synchronous" RPC, we want to add the ability to wait for completion. The waits are, of course, interruptible. Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
4ce70ada |
|
03-Jan-2006 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Further cleanups Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
963d8fe5 |
|
03-Jan-2006 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
RPC: Clean up RPC task structure Shrink the RPC task structure. Instead of storing separate pointers for task->tk_exit and task->tk_release, put them in a structure. Also pass the user data pointer as a parameter instead of passing it via task->tk_calldata. This enables us to nest callbacks. Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
abbcf28f |
|
03-Jan-2006 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
SUNRPC: Yet more RPC cleanups Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
96651ab3 |
|
22-Jun-2005 |
Trond Myklebust <Trond.Myklebust@netapp.com> |
[PATCH] RPC: Shrink struct rpc_task by switching to wait_on_bit() Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
|
#
1da177e4 |
|
16-Apr-2005 |
Linus Torvalds <torvalds@ppc970.osdl.org> |
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
|