1<?xml version="1.0" encoding="UTF-8" standalone="no"?> 2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 3<html xmlns="http://www.w3.org/1999/xhtml"> 4 <head> 5 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> 6 <title>Chapter��5.��Managing DB Files</title> 7 <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> 8 <meta name="generator" content="DocBook XSL Stylesheets V1.62.4" /> 9 <link rel="home" href="index.html" title="Getting Started with Berkeley DB Transaction Processing" /> 10 <link rel="up" href="index.html" title="Getting Started with Berkeley DB Transaction Processing" /> 11 <link rel="previous" href="reversesplit.html" title="Reverse BTree Splits" /> 12 <link rel="next" href="backuprestore.html" title="Backup Procedures" /> 13 </head> 14 <body> 15 <div class="navheader"> 16 <table width="100%" summary="Navigation header"> 17 <tr> 18 <th colspan="3" align="center">Chapter��5.��Managing DB Files</th> 19 </tr> 20 <tr> 21 <td width="20%" align="left"><a accesskey="p" href="reversesplit.html">Prev</a>��</td> 22 <th width="60%" align="center">��</th> 23 <td width="20%" align="right">��<a accesskey="n" href="backuprestore.html">Next</a></td> 24 </tr> 25 </table> 26 <hr /> 27 </div> 28 <div class="chapter" lang="en" xml:lang="en"> 29 <div class="titlepage"> 30 <div> 31 <div> 32 <h2 class="title"><a id="filemanagement"></a>Chapter��5.��Managing DB Files</h2> 33 </div> 34 </div> 35 <div></div> 36 </div> 37 <div class="toc"> 38 <p> 39 <b>Table of Contents</b> 40 </p> 41 <dl> 42 <dt> 43 <span class="sect1"> 44 <a href="filemanagement.html#checkpoints">Checkpoints</a> 45 </span> 46 </dt> 47 <dt> 48 <span class="sect1"> 49 <a href="backuprestore.html">Backup Procedures</a> 50 </span> 51 </dt> 52 <dd> 53 <dl> 54 <dt> 55 <span class="sect2"> 56 <a href="backuprestore.html#copyutilities">About Unix Copy Utilities</a> 57 </span> 58 </dt> 59 <dt> 60 <span class="sect2"> 61 <a href="backuprestore.html#standardbackup">Offline Backups</a> 62 </span> 63 </dt> 64 <dt> 65 <span class="sect2"> 66 <a href="backuprestore.html#hotbackup">Hot Backup</a> 67 </span> 68 </dt> 69 <dt> 70 <span class="sect2"> 71 <a href="backuprestore.html#incrementalbackups">Incremental Backups</a> 72 </span> 73 </dt> 74 </dl> 75 </dd> 76 <dt> 77 <span class="sect1"> 78 <a href="recovery.html">Recovery Procedures</a> 79 </span> 80 </dt> 81 <dd> 82 <dl> 83 <dt> 84 <span class="sect2"> 85 <a href="recovery.html#normalrecovery">Normal Recovery</a> 86 </span> 87 </dt> 88 <dt> 89 <span class="sect2"> 90 <a href="recovery.html#catastrophicrecovery">Catastrophic Recovery</a> 91 </span> 92 </dt> 93 </dl> 94 </dd> 95 <dt> 96 <span class="sect1"> 97 <a href="architectrecovery.html">Designing Your Application for Recovery</a> 98 </span> 99 </dt> 100 <dd> 101 <dl> 102 <dt> 103 <span class="sect2"> 104 <a href="architectrecovery.html#multithreadrecovery">Recovery for Multi-Threaded Applications</a> 105 </span> 106 </dt> 107 <dt> 108 <span class="sect2"> 109 <a href="architectrecovery.html#multiprocessrecovery">Recovery in Multi-Process Applications</a> 110 </span> 111 </dt> 112 </dl> 113 </dd> 114 <dt> 115 <span class="sect1"> 116 <a href="hotfailover.html">Using Hot Failovers</a> 117 </span> 118 </dt> 119 <dt> 120 <span class="sect1"> 121 <a href="logfileremoval.html">Removing Log Files</a> 122 </span> 123 </dt> 124 <dt> 125 <span class="sect1"> 126 <a href="logconfig.html">Configuring the Logging Subsystem</a> 127 </span> 128 </dt> 129 <dd> 130 <dl> 131 <dt> 132 <span class="sect2"> 133 <a href="logconfig.html#logfilesize">Setting the Log File Size</a> 134 </span> 135 </dt> 136 <dt> 137 <span class="sect2"> 138 <a href="logconfig.html#logregionsize">Configuring the Logging Region Size</a> 139 </span> 140 </dt> 141 <dt> 142 <span class="sect2"> 143 <a href="logconfig.html#inmemorylogging">Configuring In-Memory Logging</a> 144 </span> 145 </dt> 146 <dt> 147 <span class="sect2"> 148 <a href="logconfig.html#logbuffer">Setting the In-Memory Log Buffer Size</a> 149 </span> 150 </dt> 151 </dl> 152 </dd> 153 </dl> 154 </div> 155 <p> 156 DB is capable of storing several types of files on disk: 157 </p> 158 <div class="itemizedlist"> 159 <ul type="disc"> 160 <li> 161 <p> 162 Data files, which contain the actual data in your database. 163 </p> 164 </li> 165 <li> 166 <p> 167 Log files, which contain information required to recover your 168 database in the event of a system or application failure. 169 </p> 170 </li> 171 <li> 172 <p> 173 Region files, which contain information necessary for the 174 overall operation of your application. 175 </p> 176 </li> 177 <li> 178 <p> 179 Temporary files, which are created only under certain special circumstances. These files never need to 180 be backed up or otherwise managed and so they are not a consideration for the topics described in this 181 chapter. See <a href="enabletxn.html#security">Security Considerations</a> 182 for more information on temporary files. 183 </p> 184 </li> 185 </ul> 186 </div> 187 <p> 188 Of these, you must manage your data and log files by ensuring that they 189 are backed up. You should also pay attention to the amount of disk space 190 your log files are consuming, and periodically remove any unneeded 191 files. Finally, you can optionally tune your logging subsystem to best 192 suit your application's needs and requirements. 193 These topics are discussed in this chapter. 194 </p> 195 <div class="sect1" lang="en" xml:lang="en"> 196 <div class="titlepage"> 197 <div> 198 <div> 199 <h2 class="title" style="clear: both"><a id="checkpoints"></a>Checkpoints</h2> 200 </div> 201 </div> 202 <div></div> 203 </div> 204 <p> 205 Before we can discuss DB file management, we need to 206 describe checkpoints. When databases are modified (that is, a 207 transaction is committed), the modifications are recorded in 208 DB's logs, but they are <span class="emphasis"><em>not</em></span> 209 necessarily reflected in the actual database files on disk. 210 </p> 211 <p> 212 This means that as time goes on, increasingly 213 more data is contained in your log files that is not 214 contained in your data files. As a result, you must keep more 215 log files around than you might actually need. Also, any 216 recovery run from your log files will take increasingly longer 217 amounts of time, because there is more data in the log files 218 that must be reflected back into the data files during the 219 recovery process. 220 </p> 221 <p> 222 You can reduce these problems by periodically 223 running a checkpoint against your environment. The checkpoint: 224 </p> 225 <div class="itemizedlist"> 226 <ul type="disc"> 227 <li> 228 <p> 229 Flushes dirty pages from the in-memory cache. This means that data modifications found in your 230 in-memory cache are written to the database files on disk. Note that a checkpoint also causes data 231 dirtied by an uncommitted transaction to also be written to your database files on disk. In this latter 232 case, DB's normal recovery is used to remove any such modifications that were subsequently 233 abandoned by your application using a transaction abort. 234 </p> 235 <p> 236 Normal recovery is describe in <a href="recovery.html">Recovery Procedures</a>. 237 </p> 238 </li> 239 <li> 240 <p> 241 Writes a checkpoint record. 242 </p> 243 </li> 244 <li> 245 <p> 246 Flushes the log. This causes all log data that has not yet been written to disk to be written. 247 </p> 248 </li> 249 <li> 250 <p> 251 Writes a list of open databases. 252 </p> 253 </li> 254 </ul> 255 </div> 256 <p> 257 There are several ways to run a checkpoint. One way is to use 258 the <span><b class="command">db_checkpoint</b></span> command line utility. (Note, however, that this command line utility 259 cannot be used if your environment was opened using 260 261 <span> 262 <tt class="methodname">EnvironmentConfig.setPrivate()</tt>.) 263 </span> 264 </p> 265 <p> 266 You can also run a thread that periodically checkpoints your 267 environment for you by calling the 268 269 270 <tt class="methodname">Environment.checkpoint()</tt> 271 method. 272 </p> 273 <p> 274 Note that you can prevent a checkpoint from occurring unless more 275 than a specified amount of log data has been written since the 276 last checkpoint. You can also prevent the checkpoint from 277 running unless more than a specified amount of time has 278 occurred since the last checkpoint. These conditions are 279 particularly interesting if you have multiple threads 280 <span>or processes</span> 281 running checkpoints. 282 </p> 283 <p> 284 For configuration information, see the 285 286 287 <a href="http://www.oracle.com/technology/documentation/berkeley-db/db/java/com/sleepycat/db/CheckpointConfig.html" target="_top"> 288 CheckpointConfig Javadoc page. 289 </a> 290 </p> 291 <p> 292 Note that running checkpoints can be quite expensive. DB must 293 flush every dirty page to the backing database files. On the 294 other hand, if you do not run checkpoints often enough, your 295 recovery time can be unnecessarily long and you may be using more 296 disk space than you really need. Also, you cannot remove log files 297 until a checkpoint is run. Therefore, deciding how frequently 298 to run a checkpoint is one of the most 299 common tuning activity for DB applications. 300 </p> 301 <p> 302 For example, the following class performs a checkpoint every 60 seconds, so long as 500 kb of logging data has been 303 written since the last checkpoint: 304</p> 305 <pre class="programlisting"><span>package db.txn;</span> 306 307import com.sleepycat.db.CheckpointConfig; 308import com.sleepycat.db.DatabaseException; 309import com.sleepycat.db.Environment; 310 311public class CheckPointer extends Thread 312{ 313 private CheckpointConfig cpc = new CheckpointConfig(); 314 private Environment myEnv = null; 315 private static boolean canRun = true; 316 317 318 // Constructor. 319 CheckPointer(Environment env) { 320 myEnv = env; 321 // Run a checkpoint only if 500 kbytes of log data has been 322 // written. 323 cpc.setKBytes(500); 324 } 325 326 // Thread method that performs a checkpoint every 327 // 60 seconds 328 public void run () { 329 while (canRun) { 330 try { 331 myEnv.checkpoint(cpc); 332 sleep(60000); 333 } catch (DatabaseException de) { 334 System.err.println("Checkpoint error: " + 335 de.toString()); 336 } catch (InterruptedException e) { 337 // Should never get here 338 System.err.println("got interrupted exception"); 339 } 340 } 341 } 342 343 public static void stopRunning() { 344 canRun = false; 345 } 346} </pre> 347 <p> 348 And you use this class as follows. Note that we add the call to shutdown the checkpoint thread in our application's 349 shutdown code: 350</p> 351 <pre class="programlisting">package db.txn; 352 353import com.sleepycat.db.DatabaseException; 354import com.sleepycat.db.Environment; 355import com.sleepycat.db.EnvironmentConfig; 356 357import java.io.File; 358import java.io.FileNotFoundException; 359 360 361public class TryCheckPoint { 362 363 private static String myEnvPath = "./"; 364 365 private static Environment myEnv = null; 366 367 368 private static void usage() { 369 System.out.println("TxnGuide [-h <env directory>]"); 370 System.exit(-1); 371 } 372 373 public static void main(String args[]) { 374 try { 375 // Parse the arguments list 376 parseArgs(args); 377 // Open the environment and databases 378 openEnv(); 379 380 // Start the checkpoint thread 381 CheckPointer cp = new CheckPointer(myEnv); 382 cp.start(); 383 384 ////////////////////////////////// 385 // Do database work here as normal 386 ////////////////////////////////// 387 388 // Once all database work is completed, stop the checkpoint 389 // thread. 390 CheckPointer.stopRunning(); 391 392 // Join the checkpoint thread in case it needs some time to 393 // cleanly shutdown. 394 cp.join(); 395 396 } catch (Exception e) { 397 System.err.println("TryCheckPoint: " + e.toString()); 398 e.printStackTrace(); 399 } finally { 400 closeEnv(); 401 } 402 System.out.println("All done."); 403 } 404 405 // Open an environment and databases 406 private static void openEnv() throws DatabaseException { 407 System.out.println("opening env"); 408 409 // Set up the environment. 410 EnvironmentConfig myEnvConfig = new EnvironmentConfig(); 411 myEnvConfig.setAllowCreate(true); 412 myEnvConfig.setInitializeCache(true); 413 myEnvConfig.setInitializeLocking(true); 414 myEnvConfig.setInitializeLogging(true); 415 myEnvConfig.setTransactional(true); 416 // EnvironmentConfig.setThreaded(true) is the default behavior 417 // in Java, so we do not have to do anything to cause the 418 // environment handle to be free-threaded. 419 420 try { 421 // Open the environment 422 myEnv = new Environment(new File(myEnvPath), // Env home 423 myEnvConfig); 424 425 // Skipping the database opens and closes for brevity 426 427 } catch (FileNotFoundException fnfe) { 428 System.err.println("openEnv: " + fnfe.toString()); 429 System.exit(-1); 430 } 431 } 432 433 // Close the environment and databases 434 private static void closeEnv() { 435 System.out.println("Closing env"); 436 if (myEnv != null ) { 437 try { 438 myEnv.close(); 439 } catch (DatabaseException e) { 440 System.err.println("closeEnv: " + e.toString()); 441 e.printStackTrace(); 442 } 443 } 444 } 445 446 private TryCheckPoint() {} 447 448 private static void parseArgs(String args[]) { 449 for(int i = 0; i < args.length; ++i) { 450 if (args[i].startsWith("-")) { 451 switch(args[i].charAt(1)) { 452 case 'h': 453 myEnvPath = new String(args[++i]); 454 break; 455 default: 456 usage(); 457 } 458 } 459 } 460 } 461} </pre> 462 </div> 463 </div> 464 <div class="navfooter"> 465 <hr /> 466 <table width="100%" summary="Navigation footer"> 467 <tr> 468 <td width="40%" align="left"><a accesskey="p" href="reversesplit.html">Prev</a>��</td> 469 <td width="20%" align="center"> 470 <a accesskey="u" href="index.html">Up</a> 471 </td> 472 <td width="40%" align="right">��<a accesskey="n" href="backuprestore.html">Next</a></td> 473 </tr> 474 <tr> 475 <td width="40%" align="left" valign="top">Reverse BTree Splits��</td> 476 <td width="20%" align="center"> 477 <a accesskey="h" href="index.html">Home</a> 478 </td> 479 <td width="40%" align="right" valign="top">��Backup Procedures</td> 480 </tr> 481 </table> 482 </div> 483 </body> 484</html> 485