1<?xml version="1.0" encoding="UTF-8" standalone="no"?> 2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 3<html xmlns="http://www.w3.org/1999/xhtml"> 4 <head> 5 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> 6 <title>Chapter��5.��Managing DB Files</title> 7 <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> 8 <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> 9 <link rel="start" href="index.html" title="Getting Started with Berkeley DB Transaction Processing" /> 10 <link rel="up" href="index.html" title="Getting Started with Berkeley DB Transaction Processing" /> 11 <link rel="prev" href="reversesplit.html" title="Reverse BTree Splits" /> 12 <link rel="next" href="backuprestore.html" title="Backup Procedures" /> 13 </head> 14 <body> 15 <div class="navheader"> 16 <table width="100%" summary="Navigation header"> 17 <tr> 18 <th colspan="3" align="center">Chapter��5.��Managing DB Files</th> 19 </tr> 20 <tr> 21 <td width="20%" align="left"><a accesskey="p" href="reversesplit.html">Prev</a>��</td> 22 <th width="60%" align="center">��</th> 23 <td width="20%" align="right">��<a accesskey="n" href="backuprestore.html">Next</a></td> 24 </tr> 25 </table> 26 <hr /> 27 </div> 28 <div class="chapter" lang="en" xml:lang="en"> 29 <div class="titlepage"> 30 <div> 31 <div> 32 <h2 class="title"><a id="filemanagement"></a>Chapter��5.��Managing DB Files</h2> 33 </div> 34 </div> 35 </div> 36 <div class="toc"> 37 <p> 38 <b>Table of Contents</b> 39 </p> 40 <dl> 41 <dt> 42 <span class="sect1"> 43 <a href="filemanagement.html#checkpoints">Checkpoints</a> 44 </span> 45 </dt> 46 <dt> 47 <span class="sect1"> 48 <a href="backuprestore.html">Backup Procedures</a> 49 </span> 50 </dt> 51 <dd> 52 <dl> 53 <dt> 54 <span class="sect2"> 55 <a href="backuprestore.html#copyutilities">About Unix Copy Utilities</a> 56 </span> 57 </dt> 58 <dt> 59 <span class="sect2"> 60 <a href="backuprestore.html#standardbackup">Offline Backups</a> 61 </span> 62 </dt> 63 <dt> 64 <span class="sect2"> 65 <a href="backuprestore.html#hotbackup">Hot Backup</a> 66 </span> 67 </dt> 68 <dt> 69 <span class="sect2"> 70 <a href="backuprestore.html#incrementalbackups">Incremental Backups</a> 71 </span> 72 </dt> 73 </dl> 74 </dd> 75 <dt> 76 <span class="sect1"> 77 <a href="recovery.html">Recovery Procedures</a> 78 </span> 79 </dt> 80 <dd> 81 <dl> 82 <dt> 83 <span class="sect2"> 84 <a href="recovery.html#normalrecovery">Normal Recovery</a> 85 </span> 86 </dt> 87 <dt> 88 <span class="sect2"> 89 <a href="recovery.html#catastrophicrecovery">Catastrophic Recovery</a> 90 </span> 91 </dt> 92 </dl> 93 </dd> 94 <dt> 95 <span class="sect1"> 96 <a href="architectrecovery.html">Designing Your Application for Recovery</a> 97 </span> 98 </dt> 99 <dd> 100 <dl> 101 <dt> 102 <span class="sect2"> 103 <a href="architectrecovery.html#multithreadrecovery">Recovery for Multi-Threaded Applications</a> 104 </span> 105 </dt> 106 <dt> 107 <span class="sect2"> 108 <a href="architectrecovery.html#multiprocessrecovery">Recovery in Multi-Process Applications</a> 109 </span> 110 </dt> 111 </dl> 112 </dd> 113 <dt> 114 <span class="sect1"> 115 <a href="hotfailover.html">Using Hot Failovers</a> 116 </span> 117 </dt> 118 <dt> 119 <span class="sect1"> 120 <a href="logfileremoval.html">Removing Log Files</a> 121 </span> 122 </dt> 123 <dt> 124 <span class="sect1"> 125 <a href="logconfig.html">Configuring the Logging Subsystem</a> 126 </span> 127 </dt> 128 <dd> 129 <dl> 130 <dt> 131 <span class="sect2"> 132 <a href="logconfig.html#logfilesize">Setting the Log File Size</a> 133 </span> 134 </dt> 135 <dt> 136 <span class="sect2"> 137 <a href="logconfig.html#logregionsize">Configuring the Logging Region Size</a> 138 </span> 139 </dt> 140 <dt> 141 <span class="sect2"> 142 <a href="logconfig.html#inmemorylogging">Configuring In-Memory Logging</a> 143 </span> 144 </dt> 145 <dt> 146 <span class="sect2"> 147 <a href="logconfig.html#logbuffer">Setting the In-Memory Log Buffer Size</a> 148 </span> 149 </dt> 150 </dl> 151 </dd> 152 </dl> 153 </div> 154 <p> 155 DB is capable of storing several types of files on disk: 156 </p> 157 <div class="itemizedlist"> 158 <ul type="disc"> 159 <li> 160 <p> 161 Data files, which contain the actual data in your database. 162 </p> 163 </li> 164 <li> 165 <p> 166 Log files, which contain information required to recover your 167 database in the event of a system or application failure. 168 </p> 169 </li> 170 <li> 171 <p> 172 Region files, which contain information necessary for the 173 overall operation of your application. 174 </p> 175 </li> 176 <li> 177 <p> 178 Temporary files, which are created only under certain special circumstances. These files never need to 179 be backed up or otherwise managed and so they are not a consideration for the topics described in this 180 chapter. See <a class="xref" href="enabletxn.html#security" title="Security Considerations">Security Considerations</a> 181 for more information on temporary files. 182 </p> 183 </li> 184 </ul> 185 </div> 186 <p> 187 Of these, you must manage your data and log files by ensuring that they 188 are backed up. You should also pay attention to the amount of disk space 189 your log files are consuming, and periodically remove any unneeded 190 files. Finally, you can optionally tune your logging subsystem to best 191 suit your application's needs and requirements. 192 These topics are discussed in this chapter. 193 </p> 194 <div class="sect1" lang="en" xml:lang="en"> 195 <div class="titlepage"> 196 <div> 197 <div> 198 <h2 class="title" style="clear: both"><a id="checkpoints"></a>Checkpoints</h2> 199 </div> 200 </div> 201 </div> 202 <p> 203 Before we can discuss DB file management, we need to 204 describe checkpoints. When databases are modified (that is, a 205 transaction is committed), the modifications are recorded in 206 DB's logs, but they are <span class="emphasis"><em>not</em></span> 207 necessarily reflected in the actual database files on disk. 208 </p> 209 <p> 210 This means that as time goes on, increasingly 211 more data is contained in your log files that is not 212 contained in your data files. As a result, you must keep more 213 log files around than you might actually need. Also, any 214 recovery run from your log files will take increasingly longer 215 amounts of time, because there is more data in the log files 216 that must be reflected back into the data files during the 217 recovery process. 218 </p> 219 <p> 220 You can reduce these problems by periodically 221 running a checkpoint against your environment. The checkpoint: 222 </p> 223 <div class="itemizedlist"> 224 <ul type="disc"> 225 <li> 226 <p> 227 Flushes dirty pages from the in-memory cache. This means that data modifications found in your 228 in-memory cache are written to the database files on disk. Note that a checkpoint also causes data 229 dirtied by an uncommitted transaction to also be written to your database files on disk. In this latter 230 case, DB's normal recovery is used to remove any such modifications that were subsequently 231 abandoned by your application using a transaction abort. 232 </p> 233 <p> 234 Normal recovery is describe in <a class="xref" href="recovery.html" title="Recovery Procedures">Recovery Procedures</a>. 235 </p> 236 </li> 237 <li> 238 <p> 239 Writes a checkpoint record. 240 </p> 241 </li> 242 <li> 243 <p> 244 Flushes the log. This causes all log data that has not yet been written to disk to be written. 245 </p> 246 </li> 247 <li> 248 <p> 249 Writes a list of open databases. 250 </p> 251 </li> 252 </ul> 253 </div> 254 <p> 255 There are several ways to run a checkpoint. One way is to use 256 the <span class="command"><strong>db_checkpoint</strong></span> command line utility. (Note, however, that this command line utility 257 cannot be used if your environment was opened using 258 <span> 259 <code class="literal">DB_PRIVATE</code>.) 260 </span> 261 262 </p> 263 <p> 264 You can also run a thread that periodically checkpoints your 265 environment for you by calling the 266 <code class="methodname">DB_ENV->txn_checkpoint()</code> 267 268 269 method. 270 </p> 271 <p> 272 Note that you can prevent a checkpoint from occurring unless more 273 than a specified amount of log data has been written since the 274 last checkpoint. You can also prevent the checkpoint from 275 running unless more than a specified amount of time has 276 occurred since the last checkpoint. These conditions are 277 particularly interesting if you have multiple threads 278 <span>or processes</span> 279 running checkpoints. 280 </p> 281 <p> 282 For configuration information, see the 283 <a class="ulink" href="http://www.oracle.com/technology/documentation/berkeley-db/db/api_c/txn_checkpoint.html" target="_top"> 284 DB_ENV->txn_checkpoint() API reference page. 285 </a> 286 287 288 </p> 289 <p> 290 Note that running checkpoints can be quite expensive. DB must 291 flush every dirty page to the backing database files. On the 292 other hand, if you do not run checkpoints often enough, your 293 recovery time can be unnecessarily long and you may be using more 294 disk space than you really need. Also, you cannot remove log files 295 until a checkpoint is run. Therefore, deciding how frequently 296 to run a checkpoint is one of the most 297 common tuning activity for DB applications. 298 </p> 299 <p> 300 For example, to run a checkpoint from a separate thread of control: 301 </p> 302 <pre class="programlisting">#include <stdio.h> 303#include <pthread.h> 304#include <stdlib.h> 305#include "db.h" 306 307void *checkpoint_thread(void *); 308 309int 310main(void) 311{ 312 int ret; 313 u_int32_t env_flags; 314 DB_ENV *envp; 315 const char *db_home_dir = "/tmp/myEnvironment"; 316 pthread_t ptid; 317 318 envp = NULL; 319 320 /* Open the environment */ 321 ret = db_env_create(&envp, 0); 322 if (ret != 0) { 323 fprintf(stderr, "Error creating environment handle: %s\n", 324 db_strerror(ret)); 325 return (EXIT_FAILURE); 326 } 327 328 env_flags = DB_CREATE | /* If the environment does not 329 * exist, create it. */ 330 DB_INIT_LOCK | /* Initialize locking */ 331 DB_INIT_LOG | /* Initialize logging */ 332 DB_INIT_MPOOL | /* Initialize the cache */ 333 DB_THREAD | /* Free-thread the env handle. */ 334 DB_INIT_TXN; /* Initialize transactions */ 335 336 /* Open the environment. */ 337 ret = envp->open(envp, db_home_dir, env_flags, 0); 338 if (ret != 0) { 339 fprintf(stderr, "Error opening environment: %s\n", 340 db_strerror(ret)); 341 goto err; 342 } 343 344 345 /* Start a checkpoint thread. */ 346 if ((ret = pthread_create( 347 &ptid, NULL, checkpoint_thread, (void *)envp)) != 0) { 348 fprintf(stderr, 349 "txnapp: failed spawning checkpoint thread: %s\n", 350 strerror(ret)); 351 goto err; 352 } 353 354 /* 355 * All other threads and application shutdown code 356 * omitted for brevity. 357 */ 358 359 ... 360} 361 362 363void * 364checkpoint_thread(void *arg) { 365 DB_ENV *dbenv; 366 int ret; 367 368 dbenv = arg; 369 370 /* Checkpoint once a minute. */ 371 for (;; sleep(60)) 372 if ((ret = dbenv->txn_checkpoint(dbenv, 0, 0, 0)) != 0) { 373 dbenv->err(dbenv, ret, "checkpoint thread"); 374 exit (1); 375 } 376 377 /* NOTREACHED */ 378} </pre> 379 </div> 380 </div> 381 <div class="navfooter"> 382 <hr /> 383 <table width="100%" summary="Navigation footer"> 384 <tr> 385 <td width="40%" align="left"><a accesskey="p" href="reversesplit.html">Prev</a>��</td> 386 <td width="20%" align="center">��</td> 387 <td width="40%" align="right">��<a accesskey="n" href="backuprestore.html">Next</a></td> 388 </tr> 389 <tr> 390 <td width="40%" align="left" valign="top">Reverse BTree Splits��</td> 391 <td width="20%" align="center"> 392 <a accesskey="h" href="index.html">Home</a> 393 </td> 394 <td width="40%" align="right" valign="top">��Backup Procedures</td> 395 </tr> 396 </table> 397 </div> 398 </body> 399</html> 400