1<?xml version="1.0" encoding="UTF-8" standalone="no"?> 2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 3<html xmlns="http://www.w3.org/1999/xhtml"> 4 <head> 5 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> 6 <title>Chapter 5. Managing DB Files</title> 7 <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> 8 <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> 9 <link rel="start" href="index.html" title="Getting Started with Berkeley DB Transaction Processing" /> 10 <link rel="up" href="index.html" title="Getting Started with Berkeley DB Transaction Processing" /> 11 <link rel="prev" href="reversesplit.html" title="Reverse BTree Splits" /> 12 <link rel="next" href="backuprestore.html" title="Backup Procedures" /> 13 </head> 14 <body> 15 <div class="navheader"> 16 <table width="100%" summary="Navigation header"> 17 <tr> 18 <th colspan="3" align="center">Chapter 5. Managing DB Files</th> 19 </tr> 20 <tr> 21 <td width="20%" align="left"><a accesskey="p" href="reversesplit.html">Prev</a> </td> 22 <th width="60%" align="center"> </th> 23 <td width="20%" align="right"> <a accesskey="n" href="backuprestore.html">Next</a></td> 24 </tr> 25 </table> 26 <hr /> 27 </div> 28 <div class="chapter" lang="en" xml:lang="en"> 29 <div class="titlepage"> 30 <div> 31 <div> 32 <h2 class="title"><a id="filemanagement"></a>Chapter 5. Managing DB Files</h2> 33 </div> 34 </div> 35 </div> 36 <div class="toc"> 37 <p> 38 <b>Table of Contents</b> 39 </p> 40 <dl> 41 <dt> 42 <span class="sect1"> 43 <a href="filemanagement.html#checkpoints">Checkpoints</a> 44 </span> 45 </dt> 46 <dt> 47 <span class="sect1"> 48 <a href="backuprestore.html">Backup Procedures</a> 49 </span> 50 </dt> 51 <dd> 52 <dl> 53 <dt> 54 <span class="sect2"> 55 <a href="backuprestore.html#copyutilities">About Unix Copy Utilities</a> 56 </span> 57 </dt> 58 <dt> 59 <span class="sect2"> 60 <a href="backuprestore.html#standardbackup">Offline Backups</a> 61 </span> 62 </dt> 63 <dt> 64 <span class="sect2"> 65 <a href="backuprestore.html#hotbackup">Hot Backup</a> 66 </span> 67 </dt> 68 <dt> 69 <span class="sect2"> 70 <a href="backuprestore.html#incrementalbackups">Incremental Backups</a> 71 </span> 72 </dt> 73 </dl> 74 </dd> 75 <dt> 76 <span class="sect1"> 77 <a href="recovery.html">Recovery Procedures</a> 78 </span> 79 </dt> 80 <dd> 81 <dl> 82 <dt> 83 <span class="sect2"> 84 <a href="recovery.html#normalrecovery">Normal Recovery</a> 85 </span> 86 </dt> 87 <dt> 88 <span class="sect2"> 89 <a href="recovery.html#catastrophicrecovery">Catastrophic Recovery</a> 90 </span> 91 </dt> 92 </dl> 93 </dd> 94 <dt> 95 <span class="sect1"> 96 <a href="architectrecovery.html">Designing Your Application for Recovery</a> 97 </span> 98 </dt> 99 <dd> 100 <dl> 101 <dt> 102 <span class="sect2"> 103 <a href="architectrecovery.html#multithreadrecovery">Recovery for Multi-Threaded Applications</a> 104 </span> 105 </dt> 106 <dt> 107 <span class="sect2"> 108 <a href="architectrecovery.html#multiprocessrecovery">Recovery in Multi-Process Applications</a> 109 </span> 110 </dt> 111 </dl> 112 </dd> 113 <dt> 114 <span class="sect1"> 115 <a href="hotfailover.html">Using Hot Failovers</a> 116 </span> 117 </dt> 118 <dt> 119 <span class="sect1"> 120 <a href="logfileremoval.html">Removing Log Files</a> 121 </span> 122 </dt> 123 <dt> 124 <span class="sect1"> 125 <a href="logconfig.html">Configuring the Logging Subsystem</a> 126 </span> 127 </dt> 128 <dd> 129 <dl> 130 <dt> 131 <span class="sect2"> 132 <a href="logconfig.html#logfilesize">Setting the Log File Size</a> 133 </span> 134 </dt> 135 <dt> 136 <span class="sect2"> 137 <a href="logconfig.html#logregionsize">Configuring the Logging Region Size</a> 138 </span> 139 </dt> 140 <dt> 141 <span class="sect2"> 142 <a href="logconfig.html#inmemorylogging">Configuring In-Memory Logging</a> 143 </span> 144 </dt> 145 <dt> 146 <span class="sect2"> 147 <a href="logconfig.html#logbuffer">Setting the In-Memory Log Buffer Size</a> 148 </span> 149 </dt> 150 </dl> 151 </dd> 152 </dl> 153 </div> 154 <p> 155 DB is capable of storing several types of files on disk: 156 </p> 157 <div class="itemizedlist"> 158 <ul type="disc"> 159 <li> 160 <p> 161 Data files, which contain the actual data in your database. 162 </p> 163 </li> 164 <li> 165 <p> 166 Log files, which contain information required to recover your 167 database in the event of a system or application failure. 168 </p> 169 </li> 170 <li> 171 <p> 172 Region files, which contain information necessary for the 173 overall operation of your application. 174 </p> 175 </li> 176 <li> 177 <p> 178 Temporary files, which are created only under certain special circumstances. These files never need to 179 be backed up or otherwise managed and so they are not a consideration for the topics described in this 180 chapter. See <a class="xref" href="enabletxn.html#security" title="Security Considerations">Security Considerations</a> 181 for more information on temporary files. 182 </p> 183 </li> 184 </ul> 185 </div> 186 <p> 187 Of these, you must manage your data and log files by ensuring that they 188 are backed up. You should also pay attention to the amount of disk space 189 your log files are consuming, and periodically remove any unneeded 190 files. Finally, you can optionally tune your logging subsystem to best 191 suit your application's needs and requirements. 192 These topics are discussed in this chapter. 193 </p> 194 <div class="sect1" lang="en" xml:lang="en"> 195 <div class="titlepage"> 196 <div> 197 <div> 198 <h2 class="title" style="clear: both"><a id="checkpoints"></a>Checkpoints</h2> 199 </div> 200 </div> 201 </div> 202 <p> 203 Before we can discuss DB file management, we need to 204 describe checkpoints. When databases are modified (that is, a 205 transaction is committed), the modifications are recorded in 206 DB's logs, but they are <span class="emphasis"><em>not</em></span> 207 necessarily reflected in the actual database files on disk. 208 </p> 209 <p> 210 This means that as time goes on, increasingly 211 more data is contained in your log files that is not 212 contained in your data files. As a result, you must keep more 213 log files around than you might actually need. Also, any 214 recovery run from your log files will take increasingly longer 215 amounts of time, because there is more data in the log files 216 that must be reflected back into the data files during the 217 recovery process. 218 </p> 219 <p> 220 You can reduce these problems by periodically 221 running a checkpoint against your environment. The checkpoint: 222 </p> 223 <div class="itemizedlist"> 224 <ul type="disc"> 225 <li> 226 <p> 227 Flushes dirty pages from the in-memory cache. This means that data modifications found in your 228 in-memory cache are written to the database files on disk. Note that a checkpoint also causes data 229 dirtied by an uncommitted transaction to also be written to your database files on disk. In this latter 230 case, DB's normal recovery is used to remove any such modifications that were subsequently 231 abandoned by your application using a transaction abort. 232 </p> 233 <p> 234 Normal recovery is describe in <a class="xref" href="recovery.html" title="Recovery Procedures">Recovery Procedures</a>. 235 </p> 236 </li> 237 <li> 238 <p> 239 Writes a checkpoint record. 240 </p> 241 </li> 242 <li> 243 <p> 244 Flushes the log. This causes all log data that has not yet been written to disk to be written. 245 </p> 246 </li> 247 <li> 248 <p> 249 Writes a list of open databases. 250 </p> 251 </li> 252 </ul> 253 </div> 254 <p> 255 There are several ways to run a checkpoint. One way is to use 256 the <span class="command"><strong>db_checkpoint</strong></span> command line utility. (Note, however, that this command line utility 257 cannot be used if your environment was opened using 258 <span> 259 <code class="literal">DB_PRIVATE</code>.) 260 </span> 261 262 </p> 263 <p> 264 You can also run a thread that periodically checkpoints your 265 environment for you by calling the 266 267 <code class="methodname">DbEnv::txn_checkpoint()</code> 268 269 method. 270 </p> 271 <p> 272 Note that you can prevent a checkpoint from occurring unless more 273 than a specified amount of log data has been written since the 274 last checkpoint. You can also prevent the checkpoint from 275 running unless more than a specified amount of time has 276 occurred since the last checkpoint. These conditions are 277 particularly interesting if you have multiple threads 278 <span>or processes</span> 279 running checkpoints. 280 </p> 281 <p> 282 For configuration information, see the 283 284 <a class="ulink" href="http://www.oracle.com/technology/documentation/berkeley-db/db/api_cxx/txn_checkpoint.html" target="_top"> 285 DbEnv::txn_checkpoint() API reference page. 286 </a> 287 288 </p> 289 <p> 290 Note that running checkpoints can be quite expensive. DB must 291 flush every dirty page to the backing database files. On the 292 other hand, if you do not run checkpoints often enough, your 293 recovery time can be unnecessarily long and you may be using more 294 disk space than you really need. Also, you cannot remove log files 295 until a checkpoint is run. Therefore, deciding how frequently 296 to run a checkpoint is one of the most 297 common tuning activity for DB applications. 298 </p> 299 <p> 300 For example, to run a checkpoint from a separate thread of control: 301 </p> 302 <pre class="programlisting">#include <pthread.h> 303#include "db_cxx.h" 304 305... 306 307void *checkpoint_thread(void *); 308 309int main(void) 310{ 311 u_int32_t env_flags = DB_CREATE | // If the environment does not 312 // exist, create it. 313 DB_INIT_LOCK | // Initialize locking 314 DB_INIT_LOG | // Initialize logging 315 DB_INIT_MPOOL | // Initialize the cache 316 DB_THREAD | // Free-thread the env handle 317 DB_INIT_TXN; // Initialize transactions 318 319 std::string envHome("/export1/testEnv"); 320 DbEnv myEnv(0); 321 322 try { 323 324 myEnv.open(envHome.c_str(), env_flags, 0); 325 326 // Start a checkpoint thread. 327 pthread_t ptid; 328 int ret; 329 if ((ret = pthread_create( 330 &ptid, NULL, checkpoint_thread, (void *)&myEnv)) != 0) { 331 fprintf(stderr, 332 "txnapp: failed spawning checkpoint thread: %s\n", 333 strerror(errno)); 334 myEnv.close(0); 335 exit (1); 336 } 337 338 // All other threads and application shutdown code 339 // omitted for brevity. 340 341 ... 342} 343 344void * 345checkpoint_thread(void *arg) { 346 DbEnv *dbenv = arg; 347 348 // Checkpoint once a minute. 349 for (;; sleep(60)) { 350 try { 351 dbenv->txn_checkpoint(0, 0, 0)); 352 } catch(DbException &e) { 353 dbenv->err(e.get_errno(), "checkpoint thread"); 354 exit (e.get_errno()); 355 } 356 } 357 358 // NOTREACHED 359} </pre> 360 </div> 361 </div> 362 <div class="navfooter"> 363 <hr /> 364 <table width="100%" summary="Navigation footer"> 365 <tr> 366 <td width="40%" align="left"><a accesskey="p" href="reversesplit.html">Prev</a> </td> 367 <td width="20%" align="center"> </td> 368 <td width="40%" align="right"> <a accesskey="n" href="backuprestore.html">Next</a></td> 369 </tr> 370 <tr> 371 <td width="40%" align="left" valign="top">Reverse BTree Splits </td> 372 <td width="20%" align="center"> 373 <a accesskey="h" href="index.html">Home</a> 374 </td> 375 <td width="40%" align="right" valign="top"> Backup Procedures</td> 376 </tr> 377 </table> 378 </div> 379 </body> 380</html> 381