1 ================================ 2 ASYNCHRONOUS OPERATIONS HANDLING 3 ================================ 4 5By: David Howells <dhowells@redhat.com> 6 7Contents: 8 9 (*) Overview. 10 11 (*) Operation record initialisation. 12 13 (*) Parameters. 14 15 (*) Procedure. 16 17 (*) Asynchronous callback. 18 19 20======== 21OVERVIEW 22======== 23 24FS-Cache has an asynchronous operations handling facility that it uses for its 25data storage and retrieval routines. Its operations are represented by 26fscache_operation structs, though these are usually embedded into some other 27structure. 28 29This facility is available to and expected to be be used by the cache backends, 30and FS-Cache will create operations and pass them off to the appropriate cache 31backend for completion. 32 33To make use of this facility, <linux/fscache-cache.h> should be #included. 34 35 36=============================== 37OPERATION RECORD INITIALISATION 38=============================== 39 40An operation is recorded in an fscache_operation struct: 41 42 struct fscache_operation { 43 union { 44 struct work_struct fast_work; 45 struct slow_work slow_work; 46 }; 47 unsigned long flags; 48 fscache_operation_processor_t processor; 49 ... 50 }; 51 52Someone wanting to issue an operation should allocate something with this 53struct embedded in it. They should initialise it by calling: 54 55 void fscache_operation_init(struct fscache_operation *op, 56 fscache_operation_release_t release); 57 58with the operation to be initialised and the release function to use. 59 60The op->flags parameter should be set to indicate the CPU time provision and 61the exclusivity (see the Parameters section). 62 63The op->fast_work, op->slow_work and op->processor flags should be set as 64appropriate for the CPU time provision (see the Parameters section). 65 66FSCACHE_OP_WAITING may be set in op->flags prior to each submission of the 67operation and waited for afterwards. 68 69 70========== 71PARAMETERS 72========== 73 74There are a number of parameters that can be set in the operation record's flag 75parameter. There are three options for the provision of CPU time in these 76operations: 77 78 (1) The operation may be done synchronously (FSCACHE_OP_MYTHREAD). A thread 79 may decide it wants to handle an operation itself without deferring it to 80 another thread. 81 82 This is, for example, used in read operations for calling readpages() on 83 the backing filesystem in CacheFiles. Although readpages() does an 84 asynchronous data fetch, the determination of whether pages exist is done 85 synchronously - and the netfs does not proceed until this has been 86 determined. 87 88 If this option is to be used, FSCACHE_OP_WAITING must be set in op->flags 89 before submitting the operation, and the operating thread must wait for it 90 to be cleared before proceeding: 91 92 wait_on_bit(&op->flags, FSCACHE_OP_WAITING, 93 fscache_wait_bit, TASK_UNINTERRUPTIBLE); 94 95 96 (2) The operation may be fast asynchronous (FSCACHE_OP_FAST), in which case it 97 will be given to keventd to process. Such an operation is not permitted 98 to sleep on I/O. 99 100 This is, for example, used by CacheFiles to copy data from a backing fs 101 page to a netfs page after the backing fs has read the page in. 102 103 If this option is used, op->fast_work and op->processor must be 104 initialised before submitting the operation: 105 106 INIT_WORK(&op->fast_work, do_some_work); 107 108 109 (3) The operation may be slow asynchronous (FSCACHE_OP_SLOW), in which case it 110 will be given to the slow work facility to process. Such an operation is 111 permitted to sleep on I/O. 112 113 This is, for example, used by FS-Cache to handle background writes of 114 pages that have just been fetched from a remote server. 115 116 If this option is used, op->slow_work and op->processor must be 117 initialised before submitting the operation: 118 119 fscache_operation_init_slow(op, processor) 120 121 122Furthermore, operations may be one of two types: 123 124 (1) Exclusive (FSCACHE_OP_EXCLUSIVE). Operations of this type may not run in 125 conjunction with any other operation on the object being operated upon. 126 127 An example of this is the attribute change operation, in which the file 128 being written to may need truncation. 129 130 (2) Shareable. Operations of this type may be running simultaneously. It's 131 up to the operation implementation to prevent interference between other 132 operations running at the same time. 133 134 135========= 136PROCEDURE 137========= 138 139Operations are used through the following procedure: 140 141 (1) The submitting thread must allocate the operation and initialise it 142 itself. Normally this would be part of a more specific structure with the 143 generic op embedded within. 144 145 (2) The submitting thread must then submit the operation for processing using 146 one of the following two functions: 147 148 int fscache_submit_op(struct fscache_object *object, 149 struct fscache_operation *op); 150 151 int fscache_submit_exclusive_op(struct fscache_object *object, 152 struct fscache_operation *op); 153 154 The first function should be used to submit non-exclusive ops and the 155 second to submit exclusive ones. The caller must still set the 156 FSCACHE_OP_EXCLUSIVE flag. 157 158 If successful, both functions will assign the operation to the specified 159 object and return 0. -ENOBUFS will be returned if the object specified is 160 permanently unavailable. 161 162 The operation manager will defer operations on an object that is still 163 undergoing lookup or creation. The operation will also be deferred if an 164 operation of conflicting exclusivity is in progress on the object. 165 166 If the operation is asynchronous, the manager will retain a reference to 167 it, so the caller should put their reference to it by passing it to: 168 169 void fscache_put_operation(struct fscache_operation *op); 170 171 (3) If the submitting thread wants to do the work itself, and has marked the 172 operation with FSCACHE_OP_MYTHREAD, then it should monitor 173 FSCACHE_OP_WAITING as described above and check the state of the object if 174 necessary (the object might have died whilst the thread was waiting). 175 176 When it has finished doing its processing, it should call 177 fscache_put_operation() on it. 178 179 (4) The operation holds an effective lock upon the object, preventing other 180 exclusive ops conflicting until it is released. The operation can be 181 enqueued for further immediate asynchronous processing by adjusting the 182 CPU time provisioning option if necessary, eg: 183 184 op->flags &= ~FSCACHE_OP_TYPE; 185 op->flags |= ~FSCACHE_OP_FAST; 186 187 and calling: 188 189 void fscache_enqueue_operation(struct fscache_operation *op) 190 191 This can be used to allow other things to have use of the worker thread 192 pools. 193 194 195===================== 196ASYNCHRONOUS CALLBACK 197===================== 198 199When used in asynchronous mode, the worker thread pool will invoke the 200processor method with a pointer to the operation. This should then get at the 201container struct by using container_of(): 202 203 static void fscache_write_op(struct fscache_operation *_op) 204 { 205 struct fscache_storage *op = 206 container_of(_op, struct fscache_storage, op); 207 ... 208 } 209 210The caller holds a reference on the operation, and will invoke 211fscache_put_operation() when the processor function returns. The processor 212function is at liberty to call fscache_enqueue_operation() or to take extra 213references. 214