1<?xml version="1.0" encoding="UTF-8" standalone="no"?> 2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 3<html xmlns="http://www.w3.org/1999/xhtml"> 4 <head> 5 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> 6 <title>Configuring the Logging Subsystem</title> 7 <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> 8 <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> 9 <link rel="start" href="index.html" title="Getting Started with Berkeley DB Transaction Processing" /> 10 <link rel="up" href="filemanagement.html" title="Chapter��5.��Managing DB Files" /> 11 <link rel="prev" href="logfileremoval.html" title="Removing Log Files" /> 12 <link rel="next" href="wrapup.html" title="Chapter��6.��Summary and Examples" /> 13 </head> 14 <body> 15 <div class="navheader"> 16 <table width="100%" summary="Navigation header"> 17 <tr> 18 <th colspan="3" align="center">Configuring the Logging Subsystem</th> 19 </tr> 20 <tr> 21 <td width="20%" align="left"><a accesskey="p" href="logfileremoval.html">Prev</a>��</td> 22 <th width="60%" align="center">Chapter��5.��Managing DB Files</th> 23 <td width="20%" align="right">��<a accesskey="n" href="wrapup.html">Next</a></td> 24 </tr> 25 </table> 26 <hr /> 27 </div> 28 <div class="sect1" lang="en" xml:lang="en"> 29 <div class="titlepage"> 30 <div> 31 <div> 32 <h2 class="title" style="clear: both"><a id="logconfig"></a>Configuring the Logging Subsystem</h2> 33 </div> 34 </div> 35 </div> 36 <div class="toc"> 37 <dl> 38 <dt> 39 <span class="sect2"> 40 <a href="logconfig.html#logfilesize">Setting the Log File Size</a> 41 </span> 42 </dt> 43 <dt> 44 <span class="sect2"> 45 <a href="logconfig.html#logregionsize">Configuring the Logging Region Size</a> 46 </span> 47 </dt> 48 <dt> 49 <span class="sect2"> 50 <a href="logconfig.html#inmemorylogging">Configuring In-Memory Logging</a> 51 </span> 52 </dt> 53 <dt> 54 <span class="sect2"> 55 <a href="logconfig.html#logbuffer">Setting the In-Memory Log Buffer Size</a> 56 </span> 57 </dt> 58 </dl> 59 </div> 60 <p> 61 You can configure the following aspects of the logging 62 subsystem: 63 </p> 64 <div class="itemizedlist"> 65 <ul type="disc"> 66 <li> 67 <p> 68 Size of the log files. 69 </p> 70 </li> 71 <li> 72 <p> 73 Size of the logging subsystem's region. 74 See <a class="xref" href="logconfig.html#logregionsize" title="Configuring the Logging Region Size">Configuring the Logging Region Size</a>. 75 </p> 76 </li> 77 <li> 78 <p> 79 Maintain logs entirely in-memory. 80 See <a class="xref" href="logconfig.html#inmemorylogging" title="Configuring In-Memory Logging">Configuring In-Memory Logging</a> 81 for more information. 82 </p> 83 </li> 84 <li> 85 <p> 86 Size of the log buffer in memory. 87 See <a class="xref" href="logconfig.html#logbuffer" title="Setting the In-Memory Log Buffer Size">Setting the In-Memory Log Buffer Size</a>. 88 </p> 89 </li> 90 <li> 91 <p> 92 On-disk location of your log files. See 93 <a class="xref" href="enabletxn.html#splittingdata" title="Identifying Specific File Locations">Identifying Specific File Locations</a>. 94 </p> 95 </li> 96 </ul> 97 </div> 98 <div class="sect2" lang="en" xml:lang="en"> 99 <div class="titlepage"> 100 <div> 101 <div> 102 <h3 class="title"><a id="logfilesize"></a>Setting the Log File Size</h3> 103 </div> 104 </div> 105 </div> 106 <p> 107 Whenever a pre-defined amount of data is written to a 108 log file (10 MB by default), DB stops using the 109 current log file and starts writing to a new file. 110 You can change the maximum amount of data contained in each 111 log file by using the 112 <span> 113 <code class="methodname">DB_ENV->set_lg_max()</code> 114 115 method. 116 </span> 117 118 Note that this method can be used at any time 119 during an application's lifetime. 120 </p> 121 <p> 122 Setting the log file size to something larger than its 123 default value is largely a matter of 124 convenience and a reflection of the application's 125 preference in backup media and frequency. However, if 126 you set the log file size too low relative to your 127 application's traffic patterns, you can cause 128 yourself trouble. 129 </p> 130 <p> 131 From a performance perspective, setting the log file 132 size to a low value can cause your active transactions to 133 pause their writing activities more frequently than would 134 occur with larger log file sizes. Whenever a 135 transaction completes the log buffer is flushed to 136 disk. Normally other transactions can continue to 137 write to the log buffer while this flush is in 138 progress. However, when one log file is being closed 139 and another created, all transactions must cease 140 writing to the log buffer until the switch over is 141 completed. 142 </p> 143 <p> 144 Beyond performance concerns, using smaller log files 145 can cause you to use more physical files on disk. 146 As a result, your application could run out 147 of log sequence numbers, depending on how busy your 148 application is. 149 </p> 150 <p> 151 Every log file is identified with a 10 digit number. 152 Moreover, the maximum number of log files that your application 153 is allowed to create in its lifetime is 2,000,000,000. 154 </p> 155 <p> 156 For example, if your application performs 6,000 transactions per 157 second for 24 hours a day, and you are logging 500 bytes of 158 data per transaction into 10 MB log files, then you 159 will run out of log files in around 221 years: 160 </p> 161 <pre class="programlisting"> (10 * 2^20 * 2000000000) / (6000 * 500 * 365 * 60 *60 * 24) = 221 </pre> 162 <p> 163 However, if you were writing 2000 bytes of data per 164 transaction, and using 1 MB log files, then the same 165 formula shows you running out of log files in 5 years time. 166 </p> 167 <p> 168 All of these time frames are quite long, to be sure, 169 but if you do run out of log files after, say, 5 years 170 of continuous operations, then you must reset your log 171 sequence numbers. To do so: 172 </p> 173 <div class="orderedlist"> 174 <ol type="1"> 175 <li> 176 <p> 177 Backup your databases as if to prepare for 178 catastrophic failure. See 179 <a class="xref" href="backuprestore.html" title="Backup Procedures">Backup Procedures</a> 180 for more information. 181 </p> 182 </li> 183 <li> 184 <p> 185 Reset the log file's sequence number using the 186 <span class="command"><strong>db_load</strong></span> utility's 187 <code class="literal">-r</code> option. 188 </p> 189 </li> 190 <li> 191 <p> 192 Remove all of the log files from your 193 environment. Note that this is the only 194 situation in which all of the log files are 195 removed from an environment; in all other 196 cases, at least a single log file is retained. 197 </p> 198 </li> 199 <li> 200 <p> 201 Restart your application. 202 </p> 203 </li> 204 </ol> 205 </div> 206 </div> 207 <div class="sect2" lang="en" xml:lang="en"> 208 <div class="titlepage"> 209 <div> 210 <div> 211 <h3 class="title"><a id="logregionsize"></a>Configuring the Logging Region Size</h3> 212 </div> 213 </div> 214 </div> 215 <p> 216 The logging subsystem's default region size is 60 KB. The 217 logging region is used to store filenames, and so you may 218 need to increase its size if a large number of files (that 219 is, if you have a very large number of databases) will 220 be opened and registered with DB's log manager. 221 </p> 222 <p> 223 You can set the size of your logging region by using the 224 <span> 225 <code class="methodname">DB_ENV->set_lg_regionmax()</code> 226 227 </span> 228 229 method. Note that this method can only be called before the 230 first environment handle for your application is opened. 231 </p> 232 </div> 233 <div class="sect2" lang="en" xml:lang="en"> 234 <div class="titlepage"> 235 <div> 236 <div> 237 <h3 class="title"><a id="inmemorylogging"></a>Configuring In-Memory Logging</h3> 238 </div> 239 </div> 240 </div> 241 <p> 242 It is possible to configure your logging subsystem such 243 that logs are maintained entirely in memory. When 244 you do this, you give up your transactional durability 245 guarantee. Without log files, you have no way to run 246 recovery so any system or software failures that you might 247 experience can corrupt your databases. 248 </p> 249 <p> 250 However, by giving up your durability guarantees, you can 251 greatly improve your application's throughput by avoiding 252 the disk I/O necessary to write logging information to 253 disk. In this case, you still retain your transactional 254 atomicity, consistency, and isolation guarantees. 255 </p> 256 <p> 257 To configure your logging subsystem to maintain your logs 258 entirely in-memory: 259 </p> 260 <div class="itemizedlist"> 261 <ul type="disc"> 262 <li> 263 <p> 264 Make sure your log buffer is capable of holding all 265 log information that can accumulate during the longest 266 running transaction. See <a class="xref" href="logconfig.html#logbuffer" title="Setting the In-Memory Log Buffer Size">Setting the In-Memory Log Buffer Size</a> for details. 267 </p> 268 </li> 269 <li> 270 <p> 271 Do not run normal recovery when you open your environment. In this configuration, there are no 272 log files available against which you can run recovery. As a result, if you specify recovery 273 when you open your environment, it is ignored. 274 </p> 275 </li> 276 <li> 277 <p> 278 Specify 279 <span> 280 <code class="literal">DB_LOG_IN_MEMORY</code> to the 281 <code class="methodname">DB_ENV->log_set_config()</code> 282 283 </span> 284 285 method. Note that you must specify this before 286 your application opens its first environment 287 handle. 288 </p> 289 </li> 290 </ul> 291 </div> 292 <p> 293 For example: 294 </p> 295 <pre class="programlisting">#include <stdio.h> 296#include <stdlib.h> 297 298#include "db.h" 299 300int 301main(void) 302{ 303 int ret, ret_c; 304 u_int32_t db_flags, env_flags; 305 DB_ENV *envp; 306 const char *db_home_dir = "/tmp/myEnvironment"; 307 308 envp = NULL; 309 310 /* Create the environment */ 311 ret = db_env_create(&envp, 0); 312 if (ret != 0) { 313 fprintf(stderr, "Error creating environment handle: %s\n", 314 db_strerror(ret)); 315 return (EXIT_FAILURE); 316 } 317 318 /* 319 * Indicate that logging is to be performed only in memory. 320 * Doing this means that we give up our transactional durability 321 * guarantee. 322 */ 323 envp->log_set_config(envp, DB_LOG_IN_MEMORY, 1); 324 325 /* 326 * Configure the size of our log memory buffer. This must be 327 * large enough to hold all the logging information likely 328 * to be created for our longest running transaction. The 329 * default size for the logging buffer is 1 MB when logging 330 * is performed in-memory. For this example, we arbitrarily 331 * set the logging buffer to 5 MB. 332 */ 333 ret = envp->set_lg_bsize(envp, 5 * 1024 * 1024); 334 if (ret != 0) { 335 fprintf(stderr, "Error setting log buffer size: %s\n", 336 db_strerror(ret)); 337 goto err; 338 } 339 340 /* 341 * Set the normal flags for a transactional subsystem. Note that 342 * we DO NOT specify DB_RECOVER. Also, remember that the logging 343 * subsystem is automatically enabled when we initialize the 344 * transactional subsystem, so we do not explicitly enable 345 * logging here. 346 */ 347 env_flags = DB_CREATE | /* If the environment does not 348 * exist, create it. */ 349 DB_INIT_LOCK | /* Initialize locking */ 350 DB_INIT_LOG | /* Initialize logging */ 351 DB_INIT_MPOOL | /* Initialize the cache */ 352 DB_THREAD | /* Free-thread the env handle. */ 353 DB_INIT_TXN; /* Initialize transactions */ 354 355 /* Open the environment as normal */ 356 ret = envp->open(envp, db_home_dir, env_flags, 0); 357 if (ret != 0) { 358 fprintf(stderr, "Error opening environment: %s\n", 359 db_strerror(ret)); 360 goto err; 361 } 362 363 364 /* 365 * From here, you create transactions and perform database operations 366 * exactly as you would if you were logging to disk. This part is 367 * omitted for brevity. 368 */ 369 370 ... 371 372err: 373 /* Close the databases (omitted) */ 374 375 ... 376 377 /* Close the environment */ 378 if (envp != NULL) { 379 ret_c = envp->close(envp, 0); 380 if (ret_c != 0) { 381 fprintf(stderr, "environment close failed: %s\n", 382 db_strerror(ret_c)); 383 ret = ret_c; 384 } 385 } 386 387 return (ret == 0 ? EXIT_SUCCESS : EXIT_FAILURE); 388} </pre> 389 </div> 390 <div class="sect2" lang="en" xml:lang="en"> 391 <div class="titlepage"> 392 <div> 393 <div> 394 <h3 class="title"><a id="logbuffer"></a>Setting the In-Memory Log Buffer Size</h3> 395 </div> 396 </div> 397 </div> 398 <p> 399 When your application is configured for on-disk logging 400 (the default behavior for transactional applications), log 401 information is stored in-memory until the storage space 402 fills up, or a transaction commit forces the log 403 information to be flushed to disk. 404 </p> 405 <p> 406 It is possible to increase the amount of memory available 407 to your file log buffer. Doing so improves throughput for 408 long-running transactions, or for transactions that produce 409 a large amount of data. 410 </p> 411 <p> 412 When you have your logging subsystem configured to maintain 413 your log entirely in memory (see 414 <a class="xref" href="logconfig.html#inmemorylogging" title="Configuring In-Memory Logging">Configuring In-Memory Logging</a>), it is very important 415 to configure your log buffer size because the log buffer 416 must be capable of holding all log information that can 417 accumulate during the longest running transaction. 418 You must make sure that the in-memory log buffer size is 419 large enough that no transaction will ever span the entire 420 buffer. You must also avoid a state where the in-memory 421 buffer is full and no space can be freed because a 422 transaction that started the first log "file" is still 423 active. 424 </p> 425 <p> 426 When your logging subsystem is configured for on-disk 427 logging, the default log buffer space is 32 KB. When 428 in-memory logging is configured, the default log buffer 429 space is 1 MB. 430 </p> 431 <p> 432 You can increase your log buffer space using the 433 <span> 434 <code class="methodname">DB_ENV->set_lg_bsize()</code> 435 436 </span> 437 438 method. Note that this method can only be called before the 439 first environment handle for your application is opened. 440 </p> 441 </div> 442 </div> 443 <div class="navfooter"> 444 <hr /> 445 <table width="100%" summary="Navigation footer"> 446 <tr> 447 <td width="40%" align="left"><a accesskey="p" href="logfileremoval.html">Prev</a>��</td> 448 <td width="20%" align="center"> 449 <a accesskey="u" href="filemanagement.html">Up</a> 450 </td> 451 <td width="40%" align="right">��<a accesskey="n" href="wrapup.html">Next</a></td> 452 </tr> 453 <tr> 454 <td width="40%" align="left" valign="top">Removing Log Files��</td> 455 <td width="20%" align="center"> 456 <a accesskey="h" href="index.html">Home</a> 457 </td> 458 <td width="40%" align="right" valign="top">��Chapter��6.��Summary and Examples</td> 459 </tr> 460 </table> 461 </div> 462 </body> 463</html> 464