1<?xml version="1.0" encoding="UTF-8" standalone="no"?> 2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 3<html xmlns="http://www.w3.org/1999/xhtml"> 4 <head> 5 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> 6 <title>Configuring the Logging Subsystem</title> 7 <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> 8 <meta name="generator" content="DocBook XSL Stylesheets V1.62.4" /> 9 <link rel="home" href="index.html" title="Getting Started with Berkeley DB Transaction Processing" /> 10 <link rel="up" href="filemanagement.html" title="Chapter��5.��Managing DB Files" /> 11 <link rel="previous" href="logfileremoval.html" title="Removing Log Files" /> 12 <link rel="next" href="wrapup.html" title="Chapter��6.��Summary and Examples" /> 13 </head> 14 <body> 15 <div class="navheader"> 16 <table width="100%" summary="Navigation header"> 17 <tr> 18 <th colspan="3" align="center">Configuring the Logging Subsystem</th> 19 </tr> 20 <tr> 21 <td width="20%" align="left"><a accesskey="p" href="logfileremoval.html">Prev</a>��</td> 22 <th width="60%" align="center">Chapter��5.��Managing DB Files</th> 23 <td width="20%" align="right">��<a accesskey="n" href="wrapup.html">Next</a></td> 24 </tr> 25 </table> 26 <hr /> 27 </div> 28 <div class="sect1" lang="en" xml:lang="en"> 29 <div class="titlepage"> 30 <div> 31 <div> 32 <h2 class="title" style="clear: both"><a id="logconfig"></a>Configuring the Logging Subsystem</h2> 33 </div> 34 </div> 35 <div></div> 36 </div> 37 <p> 38 You can configure the following aspects of the logging 39 subsystem: 40 </p> 41 <div class="itemizedlist"> 42 <ul type="disc"> 43 <li> 44 <p> 45 Size of the log files. 46 </p> 47 </li> 48 <li> 49 <p> 50 Size of the logging subsystem's region. 51 See <a href="logconfig.html#logregionsize">Configuring the Logging Region Size</a>. 52 </p> 53 </li> 54 <li> 55 <p> 56 Maintain logs entirely in-memory. 57 See <a href="logconfig.html#inmemorylogging">Configuring In-Memory Logging</a> 58 for more information. 59 </p> 60 </li> 61 <li> 62 <p> 63 Size of the log buffer in memory. 64 See <a href="logconfig.html#logbuffer">Setting the In-Memory Log Buffer Size</a>. 65 </p> 66 </li> 67 <li> 68 <p> 69 On-disk location of your log files. See 70 <a href="enabletxn.html#splittingdata">Identifying Specific File Locations</a>. 71 </p> 72 </li> 73 </ul> 74 </div> 75 <div class="sect2" lang="en" xml:lang="en"> 76 <div class="titlepage"> 77 <div> 78 <div> 79 <h3 class="title"><a id="logfilesize"></a>Setting the Log File Size</h3> 80 </div> 81 </div> 82 <div></div> 83 </div> 84 <p> 85 Whenever a pre-defined amount of data is written to a 86 log file (10 MB by default), DB stops using the 87 current log file and starts writing to a new file. 88 You can change the maximum amount of data contained in each 89 log file by using the 90 <span> 91 92 <tt class="methodname">DbEnv::set_lg_max()</tt> 93 method. 94 </span> 95 96 Note that this method can be used at any time 97 during an application's lifetime. 98 </p> 99 <p> 100 Setting the log file size to something larger than its 101 default value is largely a matter of 102 convenience and a reflection of the application's 103 preference in backup media and frequency. However, if 104 you set the log file size too low relative to your 105 application's traffic patterns, you can cause 106 yourself trouble. 107 </p> 108 <p> 109 From a performance perspective, setting the log file 110 size to a low value can cause your active transactions to 111 pause their writing activities more frequently than would 112 occur with larger log file sizes. Whenever a 113 transaction completes the log buffer is flushed to 114 disk. Normally other transactions can continue to 115 write to the log buffer while this flush is in 116 progress. However, when one log file is being closed 117 and another created, all transactions must cease 118 writing to the log buffer until the switch over is 119 completed. 120 </p> 121 <p> 122 Beyond performance concerns, using smaller log files 123 can cause you to use more physical files on disk. 124 As a result, your application could run out 125 of log sequence numbers, depending on how busy your 126 application is. 127 </p> 128 <p> 129 Every log file is identified with a 10 digit number. 130 Moreover, the maximum number of log files that your application 131 is allowed to create in its lifetime is 2,000,000,000. 132 </p> 133 <p> 134 For example, if your application performs 6,000 transactions per 135 second for 24 hours a day, and you are logging 500 bytes of 136 data per transaction into 10 MB log files, then you 137 will run out of log files in around 221 years: 138 </p> 139 <pre class="programlisting"> (10 * 2^20 * 2000000000) / (6000 * 500 * 365 * 60 *60 * 24) = 221 </pre> 140 <p> 141 However, if you were writing 2000 bytes of data per 142 transaction, and using 1 MB log files, then the same 143 formula shows you running out of log files in 5 years time. 144 </p> 145 <p> 146 All of these time frames are quite long, to be sure, 147 but if you do run out of log files after, say, 5 years 148 of continuous operations, then you must reset your log 149 sequence numbers. To do so: 150 </p> 151 <div class="orderedlist"> 152 <ol type="1"> 153 <li> 154 <p> 155 Backup your databases as if to prepare for 156 catastrophic failure. See 157 <a href="backuprestore.html">Backup Procedures</a> 158 for more information. 159 </p> 160 </li> 161 <li> 162 <p> 163 Reset the log file's sequence number using the 164 <span><b class="command">db_load</b></span> utility's 165 <tt class="literal">-r</tt> option. 166 </p> 167 </li> 168 <li> 169 <p> 170 Remove all of the log files from your 171 environment. Note that this is the only 172 situation in which all of the log files are 173 removed from an environment; in all other 174 cases, at least a single log file is retained. 175 </p> 176 </li> 177 <li> 178 <p> 179 Restart your application. 180 </p> 181 </li> 182 </ol> 183 </div> 184 </div> 185 <div class="sect2" lang="en" xml:lang="en"> 186 <div class="titlepage"> 187 <div> 188 <div> 189 <h3 class="title"><a id="logregionsize"></a>Configuring the Logging Region Size</h3> 190 </div> 191 </div> 192 <div></div> 193 </div> 194 <p> 195 The logging subsystem's default region size is 60 KB. The 196 logging region is used to store filenames, and so you may 197 need to increase its size if a large number of files (that 198 is, if you have a very large number of databases) will 199 be opened and registered with DB's log manager. 200 </p> 201 <p> 202 You can set the size of your logging region by using the 203 <span> 204 205 <tt class="methodname">DbEnv::set_lg_region()</tt> 206 </span> 207 208 method. Note that this method can only be called before the 209 first environment handle for your application is opened. 210 </p> 211 </div> 212 <div class="sect2" lang="en" xml:lang="en"> 213 <div class="titlepage"> 214 <div> 215 <div> 216 <h3 class="title"><a id="inmemorylogging"></a>Configuring In-Memory Logging</h3> 217 </div> 218 </div> 219 <div></div> 220 </div> 221 <p> 222 It is possible to configure your logging subsystem such 223 that logs are maintained entirely in memory. When 224 you do this, you give up your transactional durability 225 guarantee. Without log files, you have no way to run 226 recovery so any system or software failures that you might 227 experience can corrupt your databases. 228 </p> 229 <p> 230 However, by giving up your durability guarantees, you can 231 greatly improve your application's throughput by avoiding 232 the disk I/O necessary to write logging information to 233 disk. In this case, you still retain your transactional 234 atomicity, consistency, and isolation guarantees. 235 </p> 236 <p> 237 To configure your logging subsystem to maintain your logs 238 entirely in-memory: 239 </p> 240 <div class="itemizedlist"> 241 <ul type="disc"> 242 <li> 243 <p> 244 Make sure your log buffer is capable of holding all 245 log information that can accumulate during the longest 246 running transaction. See <a href="logconfig.html#logbuffer">Setting the In-Memory Log Buffer Size</a> for details. 247 </p> 248 </li> 249 <li> 250 <p> 251 Do not run normal recovery when you open your environment. In this configuration, there are no 252 log files available against which you can run recovery. As a result, if you specify recovery 253 when you open your environment, it is ignored. 254 </p> 255 </li> 256 <li> 257 <p> 258 Specify 259 <span> 260 <tt class="literal">DB_LOG_IN_MEMORY</tt> to the 261 262 <tt class="methodname">DbEnv::log_set_config()</tt> 263 </span> 264 265 266 method. Note that you must specify this before 267 your application opens its first environment 268 handle. 269 </p> 270 </li> 271 </ul> 272 </div> 273 <p> 274 For example: 275 </p> 276 <pre class="programlisting">#include "db_cxx.h" 277 278... 279 280int main(void) 281{ 282 // Set the normal flags for a transactional subsystem. Note that 283 // we DO NOT specify DB_RECOVER. 284 u_int32_t env_flags = DB_CREATE | // If the environment does not 285 // exist, create it. 286 DB_INIT_LOCK | // Initialize locking 287 DB_INIT_LOG | // Initialize logging 288 DB_INIT_MPOOL | // Initialize the cache 289 DB_THREAD | // Free-thread the env handle 290 DB_INIT_TXN; // Initialize transactions 291 292 std::string envHome("/export1/testEnv"); 293 DbEnv myEnv(0); 294 295 try { 296 297 // Indicate that logging is to be performed only in memory. 298 // Doing this means that we give up our transactional durability 299 // guarantee. 300 myEnv.log_set_config(DB_LOG_IN_MEMORY, 1); 301 302 // Configure the size of our log memory buffer. This must be 303 // large enough to hold all the logging information likely 304 // to be created for our longest running transaction. The 305 // default size for the logging buffer is 1 MB when logging 306 // is performed in-memory. For this example, we arbitrarily 307 // set the logging buffer to 5 MB. 308 myEnv.set_lg_bsize(5 * 1024 * 1024); 309 310 // Open the environment as normal. 311 myEnv.open(envHome.c_str(), env_flags, 0); 312 313 } catch(DbException &e) { 314 std::cerr << "Error opening database and environment: " 315 << file_name << ", " 316 << envHome << std::endl; 317 std::cerr << e.what() << std::endl; 318 } 319 320 // From here, you open databases, create transactions and 321 // perform database operations exactly as you would if you 322 // were logging to disk. This part is omitted for brevity. </pre> 323 </div> 324 <div class="sect2" lang="en" xml:lang="en"> 325 <div class="titlepage"> 326 <div> 327 <div> 328 <h3 class="title"><a id="logbuffer"></a>Setting the In-Memory Log Buffer Size</h3> 329 </div> 330 </div> 331 <div></div> 332 </div> 333 <p> 334 When your application is configured for on-disk logging 335 (the default behavior for transactional applications), log 336 information is stored in-memory until the storage space 337 fills up, or a transaction commit forces the log 338 information to be flushed to disk. 339 </p> 340 <p> 341 It is possible to increase the amount of memory available 342 to your file log buffer. Doing so improves throughput for 343 long-running transactions, or for transactions that produce 344 a large amount of data. 345 </p> 346 <p> 347 When you have your logging subsystem configured to maintain 348 your log entirely in memory (see 349 <a href="logconfig.html#inmemorylogging">Configuring In-Memory Logging</a>), it is very important 350 to configure your log buffer size because the log buffer 351 must be capable of holding all log information that can 352 accumulate during the longest running transaction. 353 You must make sure that the in-memory log buffer size is 354 large enough that no transaction will ever span the entire 355 buffer. You must also avoid a state where the in-memory 356 buffer is full and no space can be freed because a 357 transaction that started the first log "file" is still 358 active. 359 </p> 360 <p> 361 When your logging subsystem is configured for on-disk 362 logging, the default log buffer space is 32 KB. When 363 in-memory logging is configured, the default log buffer 364 space is 1 MB. 365 </p> 366 <p> 367 You can increase your log buffer space using the 368 <span> 369 370 <tt class="methodname">DbEnv::set_lg_bsize()</tt> 371 </span> 372 373 method. Note that this method can only be called before the 374 first environment handle for your application is opened. 375 </p> 376 </div> 377 </div> 378 <div class="navfooter"> 379 <hr /> 380 <table width="100%" summary="Navigation footer"> 381 <tr> 382 <td width="40%" align="left"><a accesskey="p" href="logfileremoval.html">Prev</a>��</td> 383 <td width="20%" align="center"> 384 <a accesskey="u" href="filemanagement.html">Up</a> 385 </td> 386 <td width="40%" align="right">��<a accesskey="n" href="wrapup.html">Next</a></td> 387 </tr> 388 <tr> 389 <td width="40%" align="left" valign="top">Removing Log Files��</td> 390 <td width="20%" align="center"> 391 <a accesskey="h" href="index.html">Home</a> 392 </td> 393 <td width="40%" align="right" valign="top">��Chapter��6.��Summary and Examples</td> 394 </tr> 395 </table> 396 </div> 397 </body> 398</html> 399