1 2Handling of seeking on a transformation. 3 4______________________________________________________________________ 5Explained by example. Covers possible problems and solutions too. 6 7______________________________________________________________________ 8The transformation: oct (conversion of bytes into octal encoding) 9 10Ratio: 1 real character encoded into 3 characters 11 12Assumptions: Attached to a seekable channel, 13 Configured to encode written bytes and to decode 14 read bytes 15 16Wording: Transform is 'up', the channel below 'down'. 17 18______________________________________________________________________ 19Basics: Per up-character written, seek 3 characters in down. 20 So: tell.up = tell.down / 3 21 22Do: Create down-channel, seek 5 bytes bytes from start, 23 stack up, tell. 24 25 What is the position ? 26 27 tell.down % 3 == 2, tel.down / 3 = 1 ? 28 29 The problem is one of phases. The octal encoding may 30 start at arbitrary positions and not aligned to 0-phase, 31 so a simple division is not quite right. 32 33Solution: Use position in down current during stacking as zero-point, 34 calculate positions relative to this place. 35 36 So: tell.up = (tell.down - offset) / 3 37 38 In the example above: tell.up = 0 = (5-5)/3. Ok. 39 40 41Do: Create down channel, stack up, seek 2 characters, force 42 seek policy 'transform identity', seek 1 byte back, 43 restore old policy (*), seek 1 character forward, tell. 44 45 What is the position ? 46 47 Trace: Seek 2 chars up = 6 characters down. 48 Seek 1 char up back, identity! = Place 5 down. 49 Seek 1 char up, normal = Place 8 50 51 8 % 3 == 2, out of phase again, fractional position! 52 53 The problem is that we forced the down-channel into a 54 different phase relative to the one established by the first 55 two actions. 56 57Solution: Use the position in down just before restoration of natural 58 seek policy as *new* zero-point, again calculate positions 59 relative to this place. 60 61More complications: 62 63* A transform reads many bytes ahead if the user requests 64 more than it has buffered. It additionally remembers the 65 data not yet consumed by the caller. 66 67 This introduces an additional offset between up and down 68 positions. 69 70 Example: 71 Create down channel, stack up. 72 => up/down position == 0, empty buffers. 73 74 Read 10 characters. 75 => Transform reads 4K ahead, transforms them 76 into 1365 decoded characters and 1 character 77 untransformed. 10 decoded characters are 78 delivered. 79 80 => tell.down == 4096. 81 => tell.up == 10 82 83 offset = 4096 - 10*3 = 4066 84 85 Conclusions for the code 86 87 - Reading from the buffer has to decrement the offset. 88 89 - Seeking within the limits of the buffer changes only 90 the offset. Reading while within the limits of the 91 buffer has to take the data from the up position, 92 possibly from inside the buffers, and not from the 93 start. Of course, it may cause a reload too, as usual. 94 95 - Tell should not go to the down channel while within 96 the limits of the buffers. Hm, this way it will almost 97 never talk to the down channel, because the offsets 98 generated from the read-aheads are sufficient. 99 100 - Seeking behind or before the buffer discards it. And 101 has to seek the down channel to its new position too. 102 103 - Tell'ing could go down if there is no offset. But all 104 of the above nearly amounts to keeping our own 105 location, so we can use that. 106 107 - A write has to restore the real down position 108 associated with the up position, it can use the offset 109 for this. Read buffers must be discarded to prevent 110 the system from returning false data. The write 111 *could* try to update the buffers too, but I don't 112 think that this is worth the effort. 113 114* The above covered seeking from start and relative to the current 115 position. 116 117 Now how do the seek relative to the end ? 118 119 Especially if the end in the down-channel is a fractional position 120 upward. 121 122 Round to nearest non-fractional below end of down as logical end ? 123 Or the next non-fractional greater than the real 'end' ? 124 125 'oct' and similar transformations *are* able to handle incomplete 126 blocks at the end. 127 128 des in ecb and cbc mode on the other hand is not able to do this. 129 130 Hm, ... 131 132 133(*) Oops, not yet possible with the defined interface! 134 135 136Question: 137 138* Any real life examples of non-linear functions between seek locations 139 of up and down, still computable without effort ? 140 141 If no, I might reduce the basic transform information from a 142 function vector to two numbers simply specifying the ratio 143 between them. 2 numbers to express things like 3:4, f.e. for base64. 144 145 This should make it simpler for 'tcl level transforms' too. 146 147 And I am able to handle basically *all* stuff in the generic level 148 149* And I haven't thought about 'encode on read'. Returns 3 characters 150 per character below. So a seek position above is a fraction below! 151 152 OTOH, from the texts above (send earlier) it seems that I have to 153 keep separate up and down locations anyway so it shouldn't be that 154 much of hassle. Especially if I only allow seeking in multiples of 155 the ratio (3 here). 156