1<html> 2<head> 3<title>Metakit Tips and Tricks</title> 4<meta name="generator" content="Frontier 5.0.1 Win95"></head> 5<body bgcolor="#FFFFFF" alink="#008000" vlink="#800080" link="#0000FF" text="#000000"> 6 7<table cellspacing=0 cellpadding=0 border=0 width="100%"><tr valign=top> 8 <td width=100><a href="http://www.equi4.com/"> 9 <img src=e4w.gif alt="E q u i 4" align=left width=97 height=35 hspace=0 border=0></a> 10 </td><td align=center> 11 <font size=+2>Metakit Tips and Tricks</font><br> 12 <i>December 1999</i> 13 </td><td width=100></td></tr></table> 14<p> 15 16 17<i>This page list some ideas and background information 18related to the use of Metakit. 19If you have suggestions, please send them to <a href="mailto:jcw@equi4.com">jcw@equi4.com</a> ...</i> 20<br><br> 21 22<table border=0> 23 <tr> 24 <td valign=top align=center><a href="#streaming"><b>Streaming</b></a></td> 25 <td width=10></td> 26 <td>Some details about streaming and on-demand loading</td> 27 </tr> 28 <tr> 29 <td valign=top align=center><a href="#faster"><b>Faster</b></a></td> 30 <td width=10></td> 31 <td>What to do and what to avoid for maximum speed</td> 32 </tr> 33 <tr> 34 <td valign=top align=center><a href="#smaller"><b>Smaller</b></a></td> 35 <td width=10></td> 36 <td>How to manage data in the most compact format</td> 37 </tr> 38</table> 39</center> 40 41 42<dl> 43<dt><br><hr size=1><br><a name=streaming></a><b>Streaming</b> - 44Some details about streaming and on-demand loading<p><dd><font face=Times> 45 46When saving data, Metakit uses different formats, depending on whether 47the data was saved with c4_Storage::Commit or with c4_Storage::SaveToStream. 48This has a number of consequences on how and when data is loaded back in: 49 50<blockquote><table cellpadding=5 cellspacing=0 border=1> 51 <tr> 52 <th> Saved using </th> 53 <th> Loaded as </th> 54 <th align=left> Comments </th> 55 </tr> 56 <tr> 57 <td align=center> 58 Commit<br> 59 Commit<br> 60 SaveToStream<br> 61 SaveToStream</td> 62 <td align=center> 63 File<br> 64 Stream<br> 65 File<br> 66 Stream</td> 67 <td> 68 The normal case, loads data on-demand<br> 69 This combination is <b>not</b> allowed!<br> 70 Ok, will load all data during open<br> 71 Loads all data serially right away</td> 72 </tr> 73</table></blockquote> 74 75As you can see in this overview, random-access and streaming can be mixed 76in three out of four ways. Once loaded, you can again save using either 77c4_Storage::Commit or c4_Storage::SaveToStream. 78<p> 79When random-access and streaming are mixed on a storage object, the 80following will happen:<p> 81<ul><li>Files are only modified (conceptually) during a call to c4_Storage::Commit. 82<li>c4_Storage::SaveToStream makes a snapshot of the current (uncommitted) state. 83</ul><p> 84It is therefore possible to open a file, apply a few changes, save the 85changed contents to a stream, and then close the file without committing 86those changes. Streaming does not interfere with the commit/rollback mechanism. 87 88</font><br> 89<dt><br><hr size=1><br><a name=faster></a><b>Faster</b> - 90What to do and what to avoid for maximum speed<p><dd><font face=Times> 91 92The data structures used in Metakit can have surprising effects on performance, 93both positively and negatively. 94Here are a few suggestions which should help in most cases: 95<p> 96<li><b>Preallocate rows</b> - if the number or rows to add is known in advance, 97use c4_View::SetSize to set the view size accordingly. Fixed-size properties 98will need to do fewer internal reallocations if pre-allocated. 99<p> 100When inserting several rows, consider inserting them all with a dummy value 101first, and then using a loop to adjust each of the inserted values. 102<p> 103<li><b>Limit the use of c4_Row objects</b> - The construction of c4_Row objects is 104relatively slow, and can often easily be avoided. Here are three ways to 105accomplish the same thing: 106 107<blockquote><table cellpadding=5 cellspacing=0 border=1> 108 <tr> 109 <th>Slow</th> 110 <td> 111 for (int i = 0; i < n; ++i)<br> 112 {<br> 113 c4_Row temp;<br> 114 property (temp) = ...;<br> 115 view[i] = temp;<br> 116 } 117 </td> 118 </tr> 119 <tr> 120 <th>Faster</th> 121 <td> 122 c4_Row temp;<br> 123 for (int i = 0; i < n; ++i)<br> 124 {<br> 125 property (temp) = ...;<br> 126 view[i] = temp;<br> 127 } 128 </td> 129 </tr> 130 <tr> 131 <th>Fastest</th> 132 <td> 133 for (int i = 0; i < n; ++i)<br> 134 {<br> 135 c4_RowRef row = view[i];<br> 136 property (row) = ...;<br> 137 } 138 </td> 139 </tr> 140</table></blockquote> 141 142The last two statements could also have been combined as "property (view[i]) = ...;" 143<p> 144Note that "prop1 [value1] + prop2 [value2]" also creates several temporary 145c4_Row objects, and should be avoided when top performance is critical. 146<p> 147<li><b>Remove views before storage objects are destroyed</b> - If a view 148still exists when its underlying storage object is destroyed, Metakit has to 149create a full copy of the data in memory before releasing the storage object, 150to avoid even more confusing side-effects. 151 152The following example illustrates a common source of suprises in Metakit: 153 154<blockquote><table cellpadding=5 cellspacing=0 border=1> 155 <tr> 156 <th>Very slow<br>(on close)</th> 157 <td> 158 {<br> 159 c4_View view;<br> 160 c4_Storage storage (...);<br> 161 view = storage.View(...);<br> 162 ...<br> 163 } 164 </td> 165 </tr> 166 <tr> 167 <th>Instant</th> 168 <td> 169 {<br> 170 c4_Storage storage (...);<br> 171 c4_View view = storage.View(...);<br> 172 ...<br> 173 } 174 </td> 175 </tr> 176</table></blockquote> 177 178In the first case, the view still exists as the storage object goes out of scope. 179There is a simple way to detach views from their storage objects, since they 180act merely as (reference-counted) pointers: just set "view = c4_View ();" 181<b>before</b> the storage object is destroyed. 182 183</font><br> 184<dt><br><hr size=1><br><a name=smaller></a><b>Smaller</b> - 185How to manage data in the most compact format<p><dd><font face=Times> 186 187Metakit implements automatic "adaptive integer sizing" and will often create 188very compact data files as a result. Strings and binary properties also use a 189variable format, and require no "field width" setting. Some tips to 190allow you to make optimal use of these features: 191<p> 192<li><b>Attached views are more compact</b> - You can trade off size versus 193speed by selecting either views attached to a storage object (smallest), or 194unattached views (fastest). Note that "storage" does not imply the need to 195store data on file in this context. It is perfectly valid to use the following 196code to manage temporary data in memory without ever saving it to file: 197 198<blockquote><table cellpadding=5 cellspacing=0 border=1> 199 <tr> 200 <td> 201 c4_Storage storage;<br> 202 c4_View view = storage.GetAt("temp[prop1:I,prop2:I]");<br> 203 ...<br> 204 prop1 (view[i]) = ...;<br> 205 ... 206 </td> 207 </tr> 208</table></blockquote> 209 210In this example, the view would use "adaptive integer sizing" even though 211its data is valid only for the duration of each run. 212Unattached views always use 32 bits to store integers. 213<p> 214<li><b>Integer formats only increase in size</b> - The representation used to 215store integers in attached views (and on file) 216 is based on the largest value (in magnitude) ever stored in 217each property and view. Storing large values in one subview will not affect 218the storage used by that same property in another subview. 219<p> 220For integers, the number of bits used to store values is determined as follows: 221 222<blockquote><table cellpadding=5 cellspacing=0 border=1> 223 <tr> 224 <th> Value range </th> 225 <th> Bits used </th> 226 <th align=left> Comments </th> 227 </tr> 228 <tr> 229 <td align=center> 230 0<br> 231 0 .. 1<br> 232 0 .. 3<br> 233 0 .. 15<br> 234 -128 .. 127<br> 235 -32768 .. 32727<br> 236 all others</td> 237 <td align=center> 238 0<br> 239 1<br> 240 2<br> 241 4<br> 242 8<br> 243 16<br> 244 32</td> 245 <td> 246 If all rows are 0, <b>no</b> storage space is used<br> 247 Booleans are stored as 8 rows per byte<br> 248 Tiny positive codes<br> 249 A hex nibble<br> 250 One byte<br> 251 A short integer<br> 252 A long integer</td> 253 </tr> 254</table></blockquote> 255 256</font><br> 257</dl> 258 259<hr size=1> 260<center> 261 <a href="index.html" target="_top">Roadmap</a> 262 <a href="classes.html">Class Index</a> 263 <a href="samples.html">Sample Index</a> 264 <a href="tips.html">Tips and Tricks</a> 265</center> 266 267 268<hr size=1> 269<font color="#000099" size=-1>© 1998 Equi4 Software. All rights reserved.</font> 270 271</body> 272</html> 273
