1<?xml version="1.0" encoding="UTF-8" standalone="no"?> 2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 3<html xmlns="http://www.w3.org/1999/xhtml"> 4 <head> 5 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> 6 <title>Configuring the Logging Subsystem</title> 7 <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> 8 <meta name="generator" content="DocBook XSL Stylesheets V1.62.4" /> 9 <link rel="home" href="index.html" title="Getting Started with Berkeley DB Transaction Processing" /> 10 <link rel="up" href="filemanagement.html" title="Chapter 5. Managing DB Files" /> 11 <link rel="previous" href="logfileremoval.html" title="Removing Log Files" /> 12 <link rel="next" href="wrapup.html" title="Chapter 6. Summary and Examples" /> 13 </head> 14 <body> 15 <div class="navheader"> 16 <table width="100%" summary="Navigation header"> 17 <tr> 18 <th colspan="3" align="center">Configuring the Logging Subsystem</th> 19 </tr> 20 <tr> 21 <td width="20%" align="left"><a accesskey="p" href="logfileremoval.html">Prev</a> </td> 22 <th width="60%" align="center">Chapter 5. Managing DB Files</th> 23 <td width="20%" align="right"> <a accesskey="n" href="wrapup.html">Next</a></td> 24 </tr> 25 </table> 26 <hr /> 27 </div> 28 <div class="sect1" lang="en" xml:lang="en"> 29 <div class="titlepage"> 30 <div> 31 <div> 32 <h2 class="title" style="clear: both"><a id="logconfig"></a>Configuring the Logging Subsystem</h2> 33 </div> 34 </div> 35 <div></div> 36 </div> 37 <p> 38 You can configure the following aspects of the logging 39 subsystem: 40 </p> 41 <div class="itemizedlist"> 42 <ul type="disc"> 43 <li> 44 <p> 45 Size of the log files. 46 </p> 47 </li> 48 <li> 49 <p> 50 Size of the logging subsystem's region. 51 See <a href="logconfig.html#logregionsize">Configuring the Logging Region Size</a>. 52 </p> 53 </li> 54 <li> 55 <p> 56 Maintain logs entirely in-memory. 57 See <a href="logconfig.html#inmemorylogging">Configuring In-Memory Logging</a> 58 for more information. 59 </p> 60 </li> 61 <li> 62 <p> 63 Size of the log buffer in memory. 64 See <a href="logconfig.html#logbuffer">Setting the In-Memory Log Buffer Size</a>. 65 </p> 66 </li> 67 <li> 68 <p> 69 On-disk location of your log files. See 70 <a href="enabletxn.html#splittingdata">Identifying Specific File Locations</a>. 71 </p> 72 </li> 73 </ul> 74 </div> 75 <div class="sect2" lang="en" xml:lang="en"> 76 <div class="titlepage"> 77 <div> 78 <div> 79 <h3 class="title"><a id="logfilesize"></a>Setting the Log File Size</h3> 80 </div> 81 </div> 82 <div></div> 83 </div> 84 <p> 85 Whenever a pre-defined amount of data is written to a 86 log file (10 MB by default), DB stops using the 87 current log file and starts writing to a new file. 88 You can change the maximum amount of data contained in each 89 log file by using the 90 <span> 91 <tt class="methodname">DB_ENV->set_lg_max()</tt> 92 93 method. 94 </span> 95 96 Note that this method can be used at any time 97 during an application's lifetime. 98 </p> 99 <p> 100 Setting the log file size to something larger than its 101 default value is largely a matter of 102 convenience and a reflection of the application's 103 preference in backup media and frequency. However, if 104 you set the log file size too low relative to your 105 application's traffic patterns, you can cause 106 yourself trouble. 107 </p> 108 <p> 109 From a performance perspective, setting the log file 110 size to a low value can cause your active transactions to 111 pause their writing activities more frequently than would 112 occur with larger log file sizes. Whenever a 113 transaction completes the log buffer is flushed to 114 disk. Normally other transactions can continue to 115 write to the log buffer while this flush is in 116 progress. However, when one log file is being closed 117 and another created, all transactions must cease 118 writing to the log buffer until the switch over is 119 completed. 120 </p> 121 <p> 122 Beyond performance concerns, using smaller log files 123 can cause you to use more physical files on disk. 124 As a result, your application could run out 125 of log sequence numbers, depending on how busy your 126 application is. 127 </p> 128 <p> 129 Every log file is identified with a 10 digit number. 130 Moreover, the maximum number of log files that your application 131 is allowed to create in its lifetime is 2,000,000,000. 132 </p> 133 <p> 134 For example, if your application performs 6,000 transactions per 135 second for 24 hours a day, and you are logging 500 bytes of 136 data per transaction into 10 MB log files, then you 137 will run out of log files in around 221 years: 138 </p> 139 <pre class="programlisting"> (10 * 2^20 * 2000000000) / (6000 * 500 * 365 * 60 *60 * 24) = 221 </pre> 140 <p> 141 However, if you were writing 2000 bytes of data per 142 transaction, and using 1 MB log files, then the same 143 formula shows you running out of log files in 5 years time. 144 </p> 145 <p> 146 All of these time frames are quite long, to be sure, 147 but if you do run out of log files after, say, 5 years 148 of continuous operations, then you must reset your log 149 sequence numbers. To do so: 150 </p> 151 <div class="orderedlist"> 152 <ol type="1"> 153 <li> 154 <p> 155 Backup your databases as if to prepare for 156 catastrophic failure. See 157 <a href="backuprestore.html">Backup Procedures</a> 158 for more information. 159 </p> 160 </li> 161 <li> 162 <p> 163 Reset the log file's sequence number using the 164 <span><b class="command">db_load</b></span> utility's 165 <tt class="literal">-r</tt> option. 166 </p> 167 </li> 168 <li> 169 <p> 170 Remove all of the log files from your 171 environment. Note that this is the only 172 situation in which all of the log files are 173 removed from an environment; in all other 174 cases, at least a single log file is retained. 175 </p> 176 </li> 177 <li> 178 <p> 179 Restart your application. 180 </p> 181 </li> 182 </ol> 183 </div> 184 </div> 185 <div class="sect2" lang="en" xml:lang="en"> 186 <div class="titlepage"> 187 <div> 188 <div> 189 <h3 class="title"><a id="logregionsize"></a>Configuring the Logging Region Size</h3> 190 </div> 191 </div> 192 <div></div> 193 </div> 194 <p> 195 The logging subsystem's default region size is 60 KB. The 196 logging region is used to store filenames, and so you may 197 need to increase its size if a large number of files (that 198 is, if you have a very large number of databases) will 199 be opened and registered with DB's log manager. 200 </p> 201 <p> 202 You can set the size of your logging region by using the 203 <span> 204 <tt class="methodname">DB_ENV->set_lg_regionmax()</tt> 205 206 </span> 207 208 method. Note that this method can only be called before the 209 first environment handle for your application is opened. 210 </p> 211 </div> 212 <div class="sect2" lang="en" xml:lang="en"> 213 <div class="titlepage"> 214 <div> 215 <div> 216 <h3 class="title"><a id="inmemorylogging"></a>Configuring In-Memory Logging</h3> 217 </div> 218 </div> 219 <div></div> 220 </div> 221 <p> 222 It is possible to configure your logging subsystem such 223 that logs are maintained entirely in memory. When 224 you do this, you give up your transactional durability 225 guarantee. Without log files, you have no way to run 226 recovery so any system or software failures that you might 227 experience can corrupt your databases. 228 </p> 229 <p> 230 However, by giving up your durability guarantees, you can 231 greatly improve your application's throughput by avoiding 232 the disk I/O necessary to write logging information to 233 disk. In this case, you still retain your transactional 234 atomicity, consistency, and isolation guarantees. 235 </p> 236 <p> 237 To configure your logging subsystem to maintain your logs 238 entirely in-memory: 239 </p> 240 <div class="itemizedlist"> 241 <ul type="disc"> 242 <li> 243 <p> 244 Make sure your log buffer is capable of holding all 245 log information that can accumulate during the longest 246 running transaction. See <a href="logconfig.html#logbuffer">Setting the In-Memory Log Buffer Size</a> for details. 247 </p> 248 </li> 249 <li> 250 <p> 251 Do not run normal recovery when you open your environment. In this configuration, there are no 252 log files available against which you can run recovery. As a result, if you specify recovery 253 when you open your environment, it is ignored. 254 </p> 255 </li> 256 <li> 257 <p> 258 Specify 259 <span> 260 <tt class="literal">DB_LOG_IN_MEMORY</tt> to the 261 <tt class="methodname">DB_ENV->log_set_config()</tt> 262 263 </span> 264 265 266 method. Note that you must specify this before 267 your application opens its first environment 268 handle. 269 </p> 270 </li> 271 </ul> 272 </div> 273 <p> 274 For example: 275 </p> 276 <pre class="programlisting">#include <stdio.h> 277#include <stdlib.h> 278 279#include "db.h" 280 281int 282main(void) 283{ 284 int ret, ret_c; 285 u_int32_t db_flags, env_flags; 286 DB_ENV *envp; 287 const char *db_home_dir = "/tmp/myEnvironment"; 288 289 envp = NULL; 290 291 /* Create the environment */ 292 ret = db_env_create(&envp, 0); 293 if (ret != 0) { 294 fprintf(stderr, "Error creating environment handle: %s\n", 295 db_strerror(ret)); 296 return (EXIT_FAILURE); 297 } 298 299 /* 300 * Indicate that logging is to be performed only in memory. 301 * Doing this means that we give up our transactional durability 302 * guarantee. 303 */ 304 envp->log_set_config(envp, DB_LOG_IN_MEMORY, 1); 305 306 /* 307 * Configure the size of our log memory buffer. This must be 308 * large enough to hold all the logging information likely 309 * to be created for our longest running transaction. The 310 * default size for the logging buffer is 1 MB when logging 311 * is performed in-memory. For this example, we arbitrarily 312 * set the logging buffer to 5 MB. 313 */ 314 ret = envp->set_lg_bsize(envp, 5 * 1024 * 1024); 315 if (ret != 0) { 316 fprintf(stderr, "Error setting log buffer size: %s\n", 317 db_strerror(ret)); 318 goto err; 319 } 320 321 /* 322 * Set the normal flags for a transactional subsystem. Note that 323 * we DO NOT specify DB_RECOVER. Also, remember that the logging 324 * subsystem is automatically enabled when we initialize the 325 * transactional subsystem, so we do not explicitly enable 326 * logging here. 327 */ 328 env_flags = DB_CREATE | /* If the environment does not 329 * exist, create it. */ 330 DB_INIT_LOCK | /* Initialize locking */ 331 DB_INIT_LOG | /* Initialize logging */ 332 DB_INIT_MPOOL | /* Initialize the cache */ 333 DB_THREAD | /* Free-thread the env handle. */ 334 DB_INIT_TXN; /* Initialize transactions */ 335 336 /* Open the environment as normal */ 337 ret = envp->open(envp, db_home_dir, env_flags, 0); 338 if (ret != 0) { 339 fprintf(stderr, "Error opening environment: %s\n", 340 db_strerror(ret)); 341 goto err; 342 } 343 344 345 /* 346 * From here, you create transactions and perform database operations 347 * exactly as you would if you were logging to disk. This part is 348 * omitted for brevity. 349 */ 350 351 ... 352 353err: 354 /* Close the databases (omitted) */ 355 356 ... 357 358 /* Close the environment */ 359 if (envp != NULL) { 360 ret_c = envp->close(envp, 0); 361 if (ret_c != 0) { 362 fprintf(stderr, "environment close failed: %s\n", 363 db_strerror(ret_c)); 364 ret = ret_c; 365 } 366 } 367 368 return (ret == 0 ? EXIT_SUCCESS : EXIT_FAILURE); 369} </pre> 370 </div> 371 <div class="sect2" lang="en" xml:lang="en"> 372 <div class="titlepage"> 373 <div> 374 <div> 375 <h3 class="title"><a id="logbuffer"></a>Setting the In-Memory Log Buffer Size</h3> 376 </div> 377 </div> 378 <div></div> 379 </div> 380 <p> 381 When your application is configured for on-disk logging 382 (the default behavior for transactional applications), log 383 information is stored in-memory until the storage space 384 fills up, or a transaction commit forces the log 385 information to be flushed to disk. 386 </p> 387 <p> 388 It is possible to increase the amount of memory available 389 to your file log buffer. Doing so improves throughput for 390 long-running transactions, or for transactions that produce 391 a large amount of data. 392 </p> 393 <p> 394 When you have your logging subsystem configured to maintain 395 your log entirely in memory (see 396 <a href="logconfig.html#inmemorylogging">Configuring In-Memory Logging</a>), it is very important 397 to configure your log buffer size because the log buffer 398 must be capable of holding all log information that can 399 accumulate during the longest running transaction. 400 You must make sure that the in-memory log buffer size is 401 large enough that no transaction will ever span the entire 402 buffer. You must also avoid a state where the in-memory 403 buffer is full and no space can be freed because a 404 transaction that started the first log "file" is still 405 active. 406 </p> 407 <p> 408 When your logging subsystem is configured for on-disk 409 logging, the default log buffer space is 32 KB. When 410 in-memory logging is configured, the default log buffer 411 space is 1 MB. 412 </p> 413 <p> 414 You can increase your log buffer space using the 415 <span> 416 <tt class="methodname">DB_ENV->set_lg_bsize()</tt> 417 418 </span> 419 420 method. Note that this method can only be called before the 421 first environment handle for your application is opened. 422 </p> 423 </div> 424 </div> 425 <div class="navfooter"> 426 <hr /> 427 <table width="100%" summary="Navigation footer"> 428 <tr> 429 <td width="40%" align="left"><a accesskey="p" href="logfileremoval.html">Prev</a> </td> 430 <td width="20%" align="center"> 431 <a accesskey="u" href="filemanagement.html">Up</a> 432 </td> 433 <td width="40%" align="right"> <a accesskey="n" href="wrapup.html">Next</a></td> 434 </tr> 435 <tr> 436 <td width="40%" align="left" valign="top">Removing Log Files </td> 437 <td width="20%" align="center"> 438 <a accesskey="h" href="index.html">Home</a> 439 </td> 440 <td width="40%" align="right" valign="top"> Chapter 6. Summary and Examples</td> 441 </tr> 442 </table> 443 </div> 444 </body> 445</html> 446