1
2Handling of seeking on a transformation.
3
4______________________________________________________________________
5Explained by example. Covers possible problems and solutions too.
6
7______________________________________________________________________
8The transformation:	oct	(conversion of bytes into octal encoding)
9
10Ratio:			1 real character encoded into 3 characters
11
12Assumptions:		Attached to a seekable channel,
13			Configured to encode written bytes and to decode
14			read bytes
15
16Wording:		Transform is 'up', the channel below 'down'.
17
18______________________________________________________________________
19Basics:		Per up-character written, seek 3 characters in down.
20		So:	tell.up = tell.down / 3
21
22Do:		Create down-channel, seek 5 bytes bytes from start,
23		stack up, tell.
24
25		What is the position ?
26
27			tell.down % 3 == 2, tel.down / 3 = 1 ?
28
29		The problem is one of phases. The octal encoding may
30		start at arbitrary positions and not aligned to 0-phase,
31		so a simple division is not quite right.
32
33Solution:	Use position in down current during stacking as zero-point,
34		calculate positions relative to this place.
35
36		So:	tell.up = (tell.down - offset) / 3 
37
38		In the example above:	tell.up = 0 = (5-5)/3. Ok.
39
40
41Do:		Create down channel, stack up, seek 2 characters, force
42		seek policy 'transform identity', seek 1 byte back,
43		restore old policy (*), seek 1 character forward, tell.
44
45		What is the position ?
46
47		Trace:	Seek 2 chars up			= 6 characters down.
48			Seek 1 char up back, identity!	= Place 5 down.
49			Seek 1 char up, normal		= Place 8
50
51			8 % 3 == 2, out of phase again, fractional position!
52
53		The problem is that we forced the down-channel into a
54		different phase relative to the one established by the first
55		two actions.
56
57Solution:	Use the position in down just before restoration of natural
58		seek policy as *new* zero-point, again calculate positions
59		relative to this place.
60
61More complications:
62
63*	A transform reads many bytes ahead if the user requests
64 	more than it has buffered. It additionally remembers the
65 	data not yet consumed by the caller.
66 
67 	This introduces an additional offset between up and down
68 	positions.
69 
70 	Example:
71 		Create down channel, stack up.
72 		=>	up/down position == 0, empty buffers.
73 
74 		Read 10 characters.
75 		=>	Transform reads 4K ahead, transforms them
76 			into 1365 decoded characters and 1 character
77 			untransformed. 10 decoded characters are
78 			delivered.
79 
80 		=>	tell.down == 4096.
81 		=>	tell.up == 10
82 		
83 			offset = 4096 - 10*3 = 4066
84 
85 	Conclusions for the code
86 
87 	-	Reading from the buffer has to decrement the offset.
88 
89 	-	Seeking within the limits of the buffer changes only
90 		the offset. Reading while within the limits of the
91 		buffer has to take the data from the up position,
92 		possibly from inside the buffers, and not from the
93 		start. Of course, it may cause a reload too, as usual.
94 
95 	-	Tell should not go to the down channel while within
96 		the limits of the buffers. Hm, this way it will almost
97 		never talk to the down channel, because the offsets
98 		generated from the read-aheads are sufficient.
99 
100 	-	Seeking behind or before the buffer discards it. And
101 		has to seek the down channel to its new position too.
102 
103 	-	Tell'ing could go down if there is no offset. But all
104 		of the above nearly amounts to keeping our own
105 		location, so we can use that.
106 
107 	-	A write has to restore the real down position
108 		associated with the up position, it can use the offset
109 		for this. Read buffers must be discarded to prevent
110 		the system from returning false data. The write
111 		*could* try to update the buffers too, but I don't
112 		think that this is worth the effort.
113
114*	The above covered seeking from start and relative to the current
115	position.
116
117	Now how do the seek relative to the end ?
118
119	Especially if the end in the down-channel is a fractional position
120	upward.
121
122	Round to nearest non-fractional below end of down as logical end ? 
123	Or the next non-fractional greater than the real 'end' ?
124
125	'oct' and similar transformations *are* able to handle incomplete
126	blocks at the end.
127
128	des in ecb and cbc mode on the other hand is not able to do this.
129
130	Hm, ...
131
132
133(*) Oops, not yet possible with the defined interface!
134
135
136Question:
137
138*	Any real life examples of non-linear functions between seek locations
139	of up and down, still computable without effort ?
140
141	If no, I might reduce the basic transform information from a
142	function vector to two numbers simply specifying the ratio
143	between them. 2 numbers to express things like 3:4, f.e. for base64.
144
145	This should make it simpler for 'tcl level transforms' too.
146
147	And I am able to handle basically *all* stuff in the generic level
148
149*	And I haven't thought about 'encode on read'. Returns 3 characters
150	per character below. So a seek position above is a fraction below!
151
152	OTOH, from the texts above (send earlier) it seems that I have to
153	keep separate up and down locations anyway so it shouldn't be that
154	much of hassle. Especially if I only allow seeking in multiples of
155	the ratio (3 here).
156