1<?xml version="1.0" encoding="UTF-8" standalone="no"?> 2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 3<html xmlns="http://www.w3.org/1999/xhtml"> 4 <head> 5 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> 6 <title>Writing In-Memory Berkeley DB Applications</title> 7 <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> 8 <meta name="generator" content="DocBook XSL Stylesheets V1.62.4" /> 9 <link rel="home" href="index.html" title="Writing In-Memory Berkeley DB Applications" /> 10 </head> 11 <body> 12 <div class="navheader"> 13 <table width="100%" summary="Navigation header"> 14 <tr> 15 <th colspan="3" align="center">Writing In-Memory Berkeley DB Applications</th> 16 </tr> 17 </table> 18 <hr /> 19 </div> 20 <div class="article" lang="en" xml:lang="en"> 21 <div class="titlepage"> 22 <div> 23 <div> 24 <h1 class="title"><a id="id793397"></a>Writing In-Memory Berkeley DB Applications</h1> 25 </div> 26 <div> 27 <div class="legalnotice"> 28 <p class="legalnotice-title"> 29 <b>Legal Notice</b> 30 </p> 31 <p> 32 This documentation is distributed under an open source license. 33 You may review the terms of this license at: 34 <a href="http://www.oracle.com/technology/software/products/berkeley-db/htdocs/oslicense.html" target="_top">http://www.oracle.com/technology/software/products/berkeley-db/htdocs/oslicense.html</a> 35 36 37 </p> 38 <p> 39 Oracle, Berkeley DB, 40 41 42 and 43 Sleepycat are trademarks or registered trademarks of 44 Oracle. All rights to these marks are reserved. 45 No third-party use is permitted without the 46 express prior written consent of Oracle. 47 </p> 48 <p> 49 To obtain a copy of this document's original source code, please 50 submit a request to the Oracle Technology Network forum at: 51 <a href="http://forums.oracle.com/forums/forum.jspa?forumID=271" target="_top">http://forums.oracle.com/forums/forum.jspa?forumID=271</a> 52 53 54 </p> 55 </div> 56 </div> 57 <div> 58 <p class="pubdate">4/25/2008</p> 59 </div> 60 </div> 61 <div></div> 62 <hr /> 63 </div> 64 <div class="toc"> 65 <p> 66 <b>Table of Contents</b> 67 </p> 68 <dl> 69 <dt> 70 <span class="sect1"> 71 <a href="index.html#intro">Introduction</a> 72 </span> 73 </dt> 74 <dt> 75 <span class="sect1"> 76 <a href="index.html#resources">Resources to be Managed</a> 77 </span> 78 </dt> 79 <dt> 80 <span class="sect1"> 81 <a href="index.html#strategies">Strategies</a> 82 </span> 83 </dt> 84 <dt> 85 <span class="sect1"> 86 <a href="index.html#dbfiles">Keeping the Database in Memory</a> 87 </span> 88 </dt> 89 <dt> 90 <span class="sect1"> 91 <a href="index.html#env">Keeping Environments in Memory</a> 92 </span> 93 </dt> 94 <dt> 95 <span class="sect1"> 96 <a href="index.html#cachesize">Sizing the Cache</a> 97 </span> 98 </dt> 99 <dd> 100 <dl> 101 <dt> 102 <span class="sect2"> 103 <a href="index.html#dbcachesize-db">Specifying a Cache Size using the 104 Database Handle</a> 105 </span> 106 </dt> 107 <dt> 108 <span class="sect2"> 109 <a href="index.html#dbcachesize-env">Specifying a Cache Size using the 110 Environment Handle</a> 111 </span> 112 </dt> 113 </dl> 114 </dd> 115 <dt> 116 <span class="sect1"> 117 <a href="index.html#mpool-nofile">Keeping Temporary Overflow Pages in Memory</a> 118 </span> 119 </dt> 120 <dt> 121 <span class="sect1"> 122 <a href="index.html#logs">Keeping Logs in Memory</a> 123 </span> 124 </dt> 125 <dt> 126 <span class="sect1"> 127 <a href="index.html#example_in-mem">Example In-Memory Application</a> 128 </span> 129 </dt> 130 </dl> 131 </div> 132 <div class="sect1" lang="en" xml:lang="en"> 133 <div class="titlepage"> 134 <div> 135 <div> 136 <h2 class="title" style="clear: both"><a id="intro"></a>Introduction</h2> 137 </div> 138 </div> 139 <div></div> 140 </div> 141 <p> 142 This document describes how to write a DB application 143 that keeps its data entirely in memory. That is, the 144 application writes no data to disk. For this reason, 145 in-memory only applications typically discard all 146 data durability guarantees. 147 </p> 148 <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"> 149 <h3 class="title">Note</h3> 150 <p> 151 This document assume familiarity with the 152 <i class="citetitle">Getting Started with Berkeley DB</i> guide. If 153 you are using environments or transactions, then 154 you should also have an understanding of the 155 concepts in <i class="citetitle">Berkeley DB Getting Started with Transaction Processing</i> 156 guide. 157 </p> 158 </div> 159 <p> 160 There are several reasons why you might want to write an 161 in-memory only DB application. For platforms on which a 162 disk drive is available to back your data, an in-memory 163 application might be desirable from a performance 164 perspective. In this case, the data that your application 165 manages might be generated during run-time and so is of no 166 interest across application startups. 167 </p> 168 <p> 169 Other platforms are disk-less. In this case, an in-memory only 170 configuration is the only possible choice. Note that this 171 document's primary focus is disk-less systems for which an 172 on-disk filesystem is not available. 173 </p> 174 </div> 175 <div class="sect1" lang="en" xml:lang="en"> 176 <div class="titlepage"> 177 <div> 178 <div> 179 <h2 class="title" style="clear: both"><a id="resources"></a>Resources to be Managed</h2> 180 </div> 181 </div> 182 <div></div> 183 </div> 184 <p> 185 Before continuing, it is worthwhile to briefly 186 describe the DB resources that must be managed 187 if you are going to configure an in-memory only 188 DB application. These are resources that are by 189 default persisted on disk, or backed by a 190 filesystem on disk. Some configuration is therefore 191 required to keep these resources in-memory only. 192 </p> 193 <p> 194 Note that you can configure only some of these 195 resources to be held in-memory, and allow others to 196 be backed by disk. This might be desireable for 197 some applications that wish to improve application 198 performance by, for example, eliminating disk I/O 199 for some, but not all, of these resources. However, 200 for the purpose of this document, we assume you 201 want to configure all of these resources to be held 202 in memory. 203 </p> 204 <p> 205 Managing these resources for an in-memory 206 application is described in detail later in this 207 article. 208 </p> 209 <div class="itemizedlist"> 210 <ul type="disc"> 211 <li> 212 <p> 213 Database files 214 </p> 215 <p> 216 Normally, DB stores your 217 database data within on-disk files. 218 For an entirely in-memory 219 application, you are required to turn 220 off this behavior. 221 </p> 222 </li> 223 <li> 224 <p> 225 Environment region files 226 </p> 227 <p> 228 DB environments manage region 229 files for a variety of purposes. 230 Normally these are backed by the 231 filesystem, but by using the 232 appropriate configuration option 233 you can cause region files to 234 reside in memory only. 235 </p> 236 </li> 237 <li> 238 <p> 239 Database cache 240 </p> 241 <p> 242 The DB cache must be configured 243 large enough to hold all your data 244 in memory. If you do not size your 245 cache large enough, then DB 246 will attempt to write pages to 247 disk. In a disk-less system, this 248 will result in an abnormal 249 termination of your program. 250 </p> 251 </li> 252 <li> 253 <p> 254 Logs 255 </p> 256 <p> 257 DB logs describe the write activity 258 that has occurred in your application. 259 They are used for a number of purposes, 260 such as recovery operations for 261 applications that are seeking data 262 durability guarantees. 263 </p> 264 <p> 265 For in-memory applications that do not 266 care about durability guarantees, logs 267 are still required if you want 268 transactional benefits other than 269 durability (such as isolation 270 and atomicity). This is because DB's 271 transactional subsystem requires logs, 272 even if you want to discard all data 273 durability guarantees. 274 </p> 275 <p> 276 If this describes your application, you 277 must enable logs but configure them to 278 reside only within memory. 279 </p> 280 </li> 281 <li> 282 <p> 283 Temporary overflow pages 284 </p> 285 <p> 286 You must disallow backing temporary 287 database files with the 288 filesystem. This is mostly a 289 configuration option, but it is 290 also dependent upon sizing your 291 cache correctly. 292 </p> 293 </li> 294 </ul> 295 </div> 296 </div> 297 <div class="sect1" lang="en" xml:lang="en"> 298 <div class="titlepage"> 299 <div> 300 <div> 301 <h2 class="title" style="clear: both"><a id="strategies"></a>Strategies</h2> 302 </div> 303 </div> 304 <div></div> 305 </div> 306 <p> 307 DB is an extremely flexible product that can be 308 adapted to suit almost any data management 309 requirements. This means that you can configure 310 DB to operate entirely within memory, but still 311 retain some data durability guarantees or even 312 throw away all durability guarantees. 313 </p> 314 <p> 315 Data durability guarantees describe how persistent 316 your data is. That is, once you have made a change 317 to the data stored in your database, how much of a 318 guarantee do you require that that modification 319 will persist (not be lost)? There are a great many 320 options here. For the absolute best durability guarantee, you 321 should fully transaction-protect your data and 322 allow your data to be written to disk upon each 323 transaction commit. Of course, this guarantee is 324 not available for disk-less systems. 325 </p> 326 <p> 327 At the opposite end of the spectrum, you can throw 328 away all your data once your application is done 329 with it (for example, at application shutdown). 330 This is a good option if you are using DB only 331 as a kind of caching mechanism. In this case, you 332 obviously must either generate your data entirely 333 during runtime, or obtain it from some remote 334 location during application startup. 335 </p> 336 <p> 337 There are also durability options that exist somewhere in 338 between these two extremes. For example, disk-less 339 systems are sometimes backed by some kind of flash 340 memory (e.g. compact flash cards). These 341 platforms may want to limit the number of writes 342 applied to the backing media because it is capable 343 of accepting only a limited number of writes before it must 344 be replaced. For this reason, you might want to 345 limit data writes to the flash media only during 346 specific moments during your application's 347 runtime; for example, only at application shutdown. 348 </p> 349 <p> 350 The point here is to be aware that a great many 351 options are available to you when writing an 352 in-memory only application. That said, the focus of 353 this document is strictly disk-less systems; that 354 is, systems that provide no means by which 355 data can be written to persistent media. 356 </p> 357 </div> 358 <div class="sect1" lang="en" xml:lang="en"> 359 <div class="titlepage"> 360 <div> 361 <div> 362 <h2 class="title" style="clear: both"><a id="dbfiles"></a>Keeping the Database in Memory</h2> 363 </div> 364 </div> 365 <div></div> 366 </div> 367 <p> 368 Normally DB databases are backed by the 369 filesystem. For in-memory applications, such as 370 is required on disk-less systems, you can cause your 371 database to only reside in-memory. That is, 372 their contents are stored entirely within DB's 373 cache. 374 </p> 375 <p> 376 There are two requirements for keeping your 377 database(s) in-memory. 378 The first is to size your cache such that it is big 379 enough to hold all your data in-memory. If your 380 cache fills up, then DB will return 381 <tt class="literal">ENOMEM</tt> on the next operation 382 that requests additional pages in the cache. As with all errors while 383 updating a database, the current transaction must be aborted. If the update 384 was being done without a transaction, then the application must close its 385 environment and database handles, reopen them, and then refresh the database 386 from some backup data source. 387 </p> 388 <p> 389 For information on setting the cache size, see 390 <a href="index.html#cachesize">Sizing the Cache</a>. 391 </p> 392 <p> 393 Beyond cache sizing, you also must tell DB not 394 to back your database with an on-disk file. You do 395 this by NOT providing a database file name 396 when you open the database. Note that the database 397 file name is different from the database name; you 398 can name your in-memory databases even if you 399 are not storing them in an on-disk file. 400 </p> 401 <p> 402 For example: 403 </p> 404 <pre class="programlisting">#include "db.h" 405 406... 407 408 int ret, ret_c; 409 const char *db_name = "in_mem_db1"; 410 u_int32_t db_flags; /* For open flags */ 411 DB *dbp; /* Database handle */ 412 413... 414 415 /* Initialize the DB handle */ 416 ret = db_create(&dbp, NULL, 0); 417 if (ret != 0) { 418 fprintf(stderr, "Error creating database handle: %s\n", 419 db_strerror(ret)); 420 goto err; 421 } 422 423 db_flags = DB_CREATE; /* If it doesn't exist, create it */ 424 425 /* 426 * Open the database. Note that the file name is NULL. 427 * This forces the database to be stored in the cache only. 428 * Also note that the database has a name, even though its 429 * file name is NULL. 430 */ 431 ret = dbp->open(dbp, /* Pointer to the database */ 432 NULL, /* Txn pointer */ 433 <b class="userinput"><tt>NULL,</tt></b> /* File name is not specified on purpose */ 434 db_name, /* Logical db name. */ 435 DB_BTREE, /* Database type (using btree) */ 436 db_flags, /* Open flags */ 437 0); /* File mode. Using defaults */ 438 if (ret != 0) { 439 dbp->err(dbp, ret, "Database open failed"); 440 goto err; 441 } 442 443err: 444 /* Close the database */ 445 if (dbp != NULL) { 446 ret_c = dbp->close(dbp, 0); 447 if (ret_c != 0) { 448 fprintf(stderr, "%s database close failed.\n", 449 db_strerror(ret_c)); 450 ret = ret_c 451 } 452 } </pre> 453 </div> 454 <div class="sect1" lang="en" xml:lang="en"> 455 <div class="titlepage"> 456 <div> 457 <div> 458 <h2 class="title" style="clear: both"><a id="env"></a>Keeping Environments in Memory</h2> 459 </div> 460 </div> 461 <div></div> 462 </div> 463 <p> 464 Like databases, DB environments are usually backed by the filesystem. In 465 fact, a big part of what environments do is identify the location on disk 466 where resources (such as log and database files) are kept. 467 </p> 468 <p> 469 However, environments are also used for managing resources, such as 470 obtaining new transactions, so they are useful even when building an 471 in-memory application. Therefore, if you are going to use an environment for 472 your in-memory DB application, you must configure it such that it does 473 not want to use the filesystem. There are two things you need to do here. 474 </p> 475 <p> 476 First, when you open your environment, do NOT identify a home directory. To 477 accomplish this, you must: 478 </p> 479 <div class="itemizedlist"> 480 <ul type="disc"> 481 <li> 482 <p> 483 NOT provide a value for the <tt class="literal">db_home</tt> 484 parameter on the 485 <tt class="methodname">DB_ENV->open()</tt> 486 method. 487 </p> 488 </li> 489 <li> 490 <p> 491 NOT have a DB_HOME environment variable set. 492 </p> 493 </li> 494 <li> 495 <p> 496 NOT call any of the methods that affect file naming 497 (<tt class="methodname">DB_ENV->set_data_dir()</tt>, 498 <tt class="methodname">DB_ENV->set_lg_dir()</tt>, or 499 <tt class="methodname">DB_ENV->set_tmp_dir()</tt>). 500 </p> 501 </li> 502 </ul> 503 </div> 504 <p> 505 Beyond this, you must also ensure that regions are 506 backed by heap memory 507 instead of by the filesystem or system shared memory. You do this when you 508 open your environment by specifying the <tt class="literal">DB_PRIVATE</tt> flag. 509 Note that the use of <tt class="literal">DB_PRIVATE</tt> means that other processes 510 cannot share the environment. Consequently, your in-memory only application 511 must be a single-process, although it can be multi-threaded. 512 </p> 513 <p> 514 For example: 515 </p> 516 <pre class="programlisting">#include "db.h" 517 518... 519 520 int ret, ret_c; 521 u_int32_t env_flags; /* For open flags */ 522 DB_ENV *envp; /* Environment handle */ 523 524... 525 526 /* Initialize the ENV handle */ 527 ret = db_env_create(&envp, 0); 528 if (ret != 0) { 529 fprintf(stderr, "Error creating environment handle: %s\n", 530 db_strerror(ret)); 531 goto err; 532 } 533 534 535 /* 536 * Environment flags. These are for a non-threaded 537 * in-memory application. 538 */ 539 env_flags = 540 DB_CREATE | /* Create the environment if it does not exist */ 541 DB_INIT_LOCK | /* Initialize the locking subsystem */ 542 DB_INIT_LOG | /* Initialize the logging subsystem */ 543 DB_INIT_TXN | /* Initialize the transactional subsystem. This 544 * also turns on logging. */ 545 DB_INIT_MPOOL | /* Initialize the memory pool (in-memory cache) */ 546 <b class="userinput"><tt>DB_PRIVATE | /* Region files are not backed by the filesystem. 547 * Instead, they are backed by heap memory. */</tt></b> 548 549 /* 550 * Now open the environment. Notice that we do not provide a location 551 * for the environment's home directory. This is required for an 552 * in-memory only application. 553 */ 554 ret = envp->open(envp, <b class="userinput"><tt>NULL</tt></b>, env_flags, 0); 555 if (ret != 0) { 556 fprintf(stderr, "Error opening environment: %s\n", 557 db_strerror(ret)); 558 goto err; 559 } 560 561err: 562 /* Close the environment */ 563 if (envp != NULL) { 564 ret_c = envp->close(envp, 0); 565 if (ret_c != 0) { 566 fprintf(stderr, "environment close failed: %s\n", 567 db_strerror(ret_c)); 568 ret = ret_c 569 } 570 }</pre> 571 </div> 572 <div class="sect1" lang="en" xml:lang="en"> 573 <div class="titlepage"> 574 <div> 575 <div> 576 <h2 class="title" style="clear: both"><a id="cachesize"></a>Sizing the Cache</h2> 577 </div> 578 </div> 579 <div></div> 580 </div> 581 <p> 582 One of the most important considerations for an 583 in-memory application is to ensure that your 584 database cache is large enough. In a normal 585 application that is not in-memory, the cache 586 provides a mechanism by which frequently-used 587 data can be accessed without resorting to disk I/O. 588 For an in-memory application, the cache is the only 589 location your data can exist so it is critical that 590 you make the cache large enough for your data set. 591 </p> 592 <p> 593 You specify the size of your cache at application 594 startup. Obviously you should not specify a size 595 that is larger than available memory. Note that the 596 size you specify for your cache is actually a 597 <span class="emphasis"><em>maximum</em></span> size; DB will only 598 use memory as required so if you specify a cache 599 size of 1 GB but your data set is only ever 10 MB 600 in size, then 10 MB is what DB will use. 601 </p> 602 <p> 603 Note that if you specify a cache size less than 604 500 MB, then the cache size is automatically 605 increased by 25% to account for internal overhead 606 purposes. 607 </p> 608 <p> 609 There are two ways to specify a cache size, 610 depending on whether you are using a database 611 environment. 612 </p> 613 <div class="sect2" lang="en" xml:lang="en"> 614 <div class="titlepage"> 615 <div> 616 <div> 617 <h3 class="title"><a id="dbcachesize-db"></a>Specifying a Cache Size using the 618 Database Handle</h3> 619 </div> 620 </div> 621 <div></div> 622 </div> 623 <p> 624 To select a cache size using the database 625 handle, use the 626 <tt class="methodname">DB->set_cachesize()</tt> 627 method. Note that you cannot use this 628 method after the database has been opened. 629 </p> 630 <p> 631 Also, if you are using a database environment, 632 it is an error to use this method. See 633 the next section for details on 634 selecting your cache size. 635 </p> 636 <p> 637 The following code fragment creates a 638 database handle, sets the cache size to 639 10 MB and then opens the database: 640 </p> 641 <pre class="programlisting">#include "db.h" 642 643... 644 645 int ret, ret_c; 646 const char *db_name = "in_mem_db1"; 647 u_int32_t db_flags; /* For open flags */ 648 DB *dbp; /* Database handle */ 649 650... 651 652 /* Initialize the DB handle */ 653 ret = db_create(&dbp, NULL, 0); 654 if (ret != 0) { 655 fprintf(stderr, "Error creating database handle: %s\n", 656 db_strerror(ret)); 657 goto err; 658 } 659 660 <b class="userinput"><tt>/*************************************************************/ 661 /*************************************************************/ 662 /************* Set the cache size here **********************/ 663 /*************************************************************/ 664 /*************************************************************/ 665 666 ret = dbp->set_cachesize(dbp, 667 0, /* 0 gigabytes */ 668 10 * 1024 * 1024, /* 10 megabytes */ 669 1); /* Create 1 cache. All memory will 670 * be allocated contiguously. */ 671 if (ret != 0) { 672 dbp->err(dbp, ret, "Database open failed"); 673 goto err; 674 }</tt></b> 675 676 677 db_flags = DB_CREATE; /* If it doesn't exist, create it */ 678 ret = dbp->open(dbp, /* Pointer to the database */ 679 NULL, /* Txn pointer */ 680 NULL, /* File name is not specified on purpose */ 681 db_name, /* Logical db name */ 682 DB_BTREE, /* Database type (using btree) */ 683 db_flags, /* Open flags */ 684 0); /* File mode. Using defaults */ 685 if (ret != 0) { 686 dbp->err(dbp, ret, "Database open failed"); 687 goto err; 688 } 689 690err: 691 /* Close the database */ 692 if (dbp != NULL) { 693 ret_c = dbp->close(dbp, 0); 694 if (ret_c != 0) { 695 fprintf(stderr, "%s database close failed.\n", 696 db_strerror(ret_c)); 697 ret = ret_c 698 } 699 } </pre> 700 </div> 701 <div class="sect2" lang="en" xml:lang="en"> 702 <div class="titlepage"> 703 <div> 704 <div> 705 <h3 class="title"><a id="dbcachesize-env"></a>Specifying a Cache Size using the 706 Environment Handle</h3> 707 </div> 708 </div> 709 <div></div> 710 </div> 711 <p> 712 To select a cache size using the 713 environment handle, use the 714 <tt class="methodname">ENV->set_cachesize()</tt> 715 method. Note that you cannot use this 716 method after the environment has been opened. 717 </p> 718 <p> 719 The following code fragment creates an 720 environment handle, sets the cache size to 721 10 MB and then opens the environment: Once 722 opened, you can use the environment when 723 you open your database(s). This means all 724 your databases will use the same 725 cache. 726 </p> 727 <pre class="programlisting">#include "db.h" 728 729... 730 731 int ret, ret_c; 732 u_int32_t env_flags; /* For open flags */ 733 DB_ENV *envp; /* Environment handle */ 734 735... 736 737 /* Initialize the ENV handle */ 738 ret = db_env_create(&envp, 0); 739 if (ret != 0) { 740 fprintf(stderr, "Error creating environment handle: %s\n", 741 db_strerror(ret)); 742 goto err; 743 } 744 745 <b class="userinput"><tt>/*************************************************************/ 746 /*************************************************************/ 747 /************* Set the cache size here **********************/ 748 /*************************************************************/ 749 /*************************************************************/ 750 751 ret = 752 envp->set_cachesize(envp, 753 0, /* 0 gigabytes */ 754 10 * 1024 * 1024, /* 10 megabytes */ 755 1); /* Create 1 cache. All memory will 756 * be allocated contiguously. */ 757 if (ret != 0) { 758 envp->err(envp, ret, "Environment open failed"); 759 goto err; 760 }</tt></b> 761 762 /* 763 * Environment flags. These are for a non-threaded 764 * in-memory application. 765 */ 766 env_flags = 767 DB_CREATE | /* Create the environment if it does not exist */ 768 DB_INIT_LOCK | /* Initialize the locking subsystem */ 769 DB_INIT_LOG | /* Initialize the logging subsystem */ 770 DB_INIT_TXN | /* Initialize the transactional subsystem. This 771 * also turns on logging. */ 772 DB_INIT_MPOOL | /* Initialize the memory pool (in-memory cache) */ 773 DB_PRIVATE | /* Region files are not backed by the filesystem. 774 * Instead, they are backed by heap memory. */ 775 776 /* 777 * Now open the environment. Notice that we do not provide a location 778 * for the environment's home directory. This is required for an 779 * in-memory only application. 780 */ 781 ret = envp->open(envp, NULL, env_flags, 0); 782 if (ret != 0) { 783 fprintf(stderr, "Error opening environment: %s\n", 784 db_strerror(ret)); 785 goto err; 786 } 787 788err: 789 /* Close the environment */ 790 if (envp != NULL) { 791 ret_c = envp->close(envp, 0); 792 if (ret_c != 0) { 793 fprintf(stderr, "environment close failed: %s\n", 794 db_strerror(ret_c)); 795 ret = ret_c 796 } 797 } </pre> 798 </div> 799 </div> 800 <div class="sect1" lang="en" xml:lang="en"> 801 <div class="titlepage"> 802 <div> 803 <div> 804 <h2 class="title" style="clear: both"><a id="mpool-nofile"></a>Keeping Temporary Overflow Pages in Memory</h2> 805 </div> 806 </div> 807 <div></div> 808 </div> 809 <p> 810 Normally, when a database is opened, a temporary 811 file is opened on-disk to back the database. This 812 file is used if the database grows so large that it 813 fills the entire cache. At that time, database 814 pages that do not fit into the in-memory cache file 815 are written temporarily to this file. 816 </p> 817 <p> 818 For disk-less systems, you should configure your 819 databases so that this temporary file is not 820 created. When you do this, any attempt to create 821 new database pages once the cache is full will 822 fail. 823 </p> 824 <p> 825 You configure this option on a per-database handle 826 basis. That means you must configure this for every 827 in-memory database that your application uses. 828 </p> 829 <p> 830 To set this option, obtain the 831 <tt class="literal">DB_MPOOLFILE</tt> field from you 832 <tt class="literal">DB</tt> and then configure 833 <tt class="literal">DB_MPOOL_NOFILE</tt> using the 834 <tt class="methodname">DB_MPOOLFILE->set_flags()</tt> 835 method. 836 </p> 837 <p> 838 For example: 839 </p> 840 <pre class="programlisting">#include "db.h" 841 842... 843 844 int ret, ret_c; 845 u_int32_t env_flags; /* For open flags */ 846 DB_ENV *envp; /* Environment handle */ 847 848... 849 850 /* 851 * Configure the cache file. This can be done 852 * at any point in the application's life once the 853 * DB handle has been created. 854 */ 855 mpf = dbp->get_mpf(dbp); 856 ret = mpf->set_flags(mpf, DB_MPOOL_NOFILE, 1); 857 858 if (ret != 0) { 859 fprintf(stderr, 860 "Attempt failed to configure for no backing of temp files: %s\n", 861 db_strerror(ret)); 862 goto err; 863 } </pre> 864 </div> 865 <div class="sect1" lang="en" xml:lang="en"> 866 <div class="titlepage"> 867 <div> 868 <div> 869 <h2 class="title" style="clear: both"><a id="logs"></a>Keeping Logs in Memory</h2> 870 </div> 871 </div> 872 <div></div> 873 </div> 874 <p> 875 DB logs describe the write activity that has occurred in 876 your application. For a purely in-memory application, logs 877 should be used only if you wish to transaction-protect your 878 database writes as logs are required by the DB 879 transactional subsystem. 880 </p> 881 <p> 882 Note that transactions provide a number of guarantees. One 883 of these is not interesting to a purely in-memory 884 application (data durability). However, other transaction 885 guarantees such as isolation and atomicity might be of 886 interest to your application. 887 </p> 888 <p> 889 If this is the case for your application, then you must 890 configure your logs to be kept entirely in-memory. You 891 do this by setting a configuration option that prevents 892 DB from writing log data to disk. 893 Do this by setting the <tt class="literal">DB_LOG_IN_MEMORY</tt> flag using the 894 <tt class="methodname">DB_ENV->log_set_config()</tt> method. 895 </p> 896 <p> 897 In addition, you must configure your log buffer size so that it is capable 898 of holding all log information that can accumulate during your longest 899 running transaction. That is, make sure the in-memory log buffer is large 900 enough that no transaction will ever span the entire buffer. Also, avoid a 901 state where the in-memory buffer is full and no space can be freed because a 902 transaction that started the first log "file" is still active. 903 </p> 904 <p> 905 How much log buffer space is required is a function of the number of 906 transactions you have running concurrently, how long they last, and how much 907 write activity occurs within them. When in-memory logging is configured, the 908 default log buffer space is 1 MB. 909 </p> 910 <p> 911 You set your log buffer space using the 912 <tt class="methodname">DB_ENV->set_lg_bsize()</tt>. 913 </p> 914 <p> 915 For example, the following code fragment configure in-memory log usage, and 916 it configures the log buffer size to 10 MB: 917 </p> 918 <pre class="programlisting">#include "db.h" 919 920... 921 922 int ret, ret_c; 923 u_int32_t env_flags; /* For open flags */ 924 DB_ENV *envp; /* Environment handle */ 925 926... 927 928 /* Initialize the ENV handle */ 929 ret = db_env_create(&envp, 0); 930 if (ret != 0) { 931 fprintf(stderr, "Error creating environment handle: %s\n", 932 db_strerror(ret)); 933 goto err; 934 } 935 936 /* 937 * Environment flags. These are for a non-threaded 938 * in-memory application. 939 */ 940 env_flags = 941 DB_CREATE | /* Create the environment if it does not exist */ 942 DB_INIT_LOCK | /* Initialize the locking subsystem */ 943 DB_INIT_LOG | /* Initialize the logging subsystem */ 944 DB_INIT_TXN | /* Initialize the transactional subsystem. This 945 * also turns on logging. */ 946 DB_INIT_MPOOL | /* Initialize the memory pool (in-memory cache) */ 947 DB_PRIVATE | /* Region files are not backed by the filesystem. 948 * Instead, they are backed by heap memory. */ 949 950 951 <b class="userinput"><tt>/* Specify in-memory logging */ 952 ret = envp->log_set_config(envp, DB_LOG_IN_MEMORY, 1); 953 if (ret != 0) { 954 fprintf(stderr, "Error setting log subsystem to in-memory: %s\n", 955 db_strerror(ret)); 956 goto err; 957 } 958 959 /* 960 * Specify the size of the in-memory log buffer. 961 */ 962 ret = envp->set_lg_bsize(envp, 10 * 1024 * 1024); 963 if (ret != 0) { 964 fprintf(stderr, "Error increasing the log buffer size: %s\n", 965 db_strerror(ret)); 966 goto err; 967 }</tt></b> 968 969 /* 970 * Now open the environment. Notice that we do not provide a location 971 * for the environment's home directory. This is required for an 972 * in-memory only application. 973 */ 974 ret = envp->open(envp, NULL, env_flags, 0); 975 if (ret != 0) { 976 fprintf(stderr, "Error opening environment: %s\n", 977 db_strerror(ret)); 978 goto err; 979 } 980 981err: 982 /* Close the environment */ 983 if (envp != NULL) { 984 ret_c = envp->close(envp, 0); 985 if (ret_c != 0) { 986 fprintf(stderr, "environment close failed: %s\n", 987 db_strerror(ret_c)); 988 ret = ret_c 989 } 990 } </pre> 991 </div> 992 <div class="sect1" lang="en" xml:lang="en"> 993 <div class="titlepage"> 994 <div> 995 <div> 996 <h2 class="title" style="clear: both"><a id="example_in-mem"></a>Example In-Memory Application</h2> 997 </div> 998 </div> 999 <div></div> 1000 </div> 1001 <p> 1002 The following brief example illustrates how to open 1003 an application that is entirely in-memory. The 1004 application opens an environment and a single 1005 database, and does this in a way that the database 1006 is transaction-protected. 1007 </p> 1008 <p> 1009 Transactions can be 1010 desirable for an in-memory application even though 1011 you discard your durability guarantees, because of 1012 the other things that transactions offer such as 1013 atomicity and isolation. 1014 </p> 1015 <p> 1016 Notice that the example does nothing other than 1017 open and close the environment and database. DB 1018 database reads and writes work identically between 1019 in-memory-only and durable applications (that is, 1020 applications that write database application to 1021 durable storage). Consequently, there is no point 1022 in illustrating those actions here. 1023 </p> 1024 <pre class="programlisting">/* We assume an ANSI-compatible compiler */ 1025#include <stdio.h> 1026#include <stdlib.h> 1027#include <string.h> 1028#include <db.h> 1029 1030 1031int 1032main(void) 1033{ 1034 /* Initialize our handles */ 1035 DB *dbp = NULL; 1036 DB_ENV *envp = NULL; 1037 DB_MPOOLFILE *mpf = NULL; 1038 1039 int ret, ret_t; 1040 const char *db_name = "in_mem_db1"; 1041 u_int32_t open_flags; 1042 1043 /* Create the environment */ 1044 ret = db_env_create(&envp, 0); 1045 if (ret != 0) { 1046 fprintf(stderr, "Error creating environment handle: %s\n", 1047 db_strerror(ret)); 1048 goto err; 1049 } 1050 1051 open_flags = 1052 DB_CREATE | /* Create the environment if it does not exist */ 1053 DB_INIT_LOCK | /* Initialize the locking subsystem */ 1054 DB_INIT_LOG | /* Initialize the logging subsystem */ 1055 DB_INIT_MPOOL | /* Initialize the memory pool (in-memory cache) */ 1056 DB_INIT_TXN | 1057 DB_PRIVATE; /* Region files are not backed by the filesystem. 1058 * Instead, they are backed by heap memory. */ 1059 1060 /* Specify in-memory logging */ 1061 ret = envp->log_set_config(envp, DB_LOG_IN_MEMORY, 1); 1062 if (ret != 0) { 1063 fprintf(stderr, "Error setting log subsystem to in-memory: %s\n", 1064 db_strerror(ret)); 1065 goto err; 1066 } 1067 /* 1068 * Specify the size of the in-memory log buffer. 1069 */ 1070 ret = envp->set_lg_bsize(envp, 10 * 1024 * 1024); 1071 if (ret != 0) { 1072 fprintf(stderr, "Error increasing the log buffer size: %s\n", 1073 db_strerror(ret)); 1074 goto err; 1075 } 1076 1077 /* 1078 * Specify the size of the in-memory cache. 1079 */ 1080 ret = envp->set_cachesize(envp, 0, 10 * 1024 * 1024, 1); 1081 if (ret != 0) { 1082 fprintf(stderr, "Error increasing the cache size: %s\n", 1083 db_strerror(ret)); 1084 goto err; 1085 } 1086 1087 /* 1088 * Now actually open the environment. Notice that the environment home 1089 * directory is NULL. This is required for an in-memory only 1090 * application. 1091 */ 1092 ret = envp->open(envp, NULL, open_flags, 0); 1093 if (ret != 0) { 1094 fprintf(stderr, "Error opening environment: %s\n", 1095 db_strerror(ret)); 1096 goto err; 1097 } 1098 1099 1100 /* Initialize the DB handle */ 1101 ret = db_create(&dbp, envp, 0); 1102 if (ret != 0) { 1103 envp->err(envp, ret, 1104 "Attempt to create db handle failed."); 1105 goto err; 1106 } 1107 1108 1109 /* 1110 * Set the database open flags. Autocommit is used because we are 1111 * transactional. 1112 */ 1113 open_flags = DB_CREATE | DB_AUTO_COMMIT; 1114 ret = dbp->open(dbp, /* Pointer to the database */ 1115 NULL, /* Txn pointer */ 1116 NULL, /* File name -- Must be NULL for inmemory! */ 1117 db_name, /* Logical db name */ 1118 DB_BTREE, /* Database type (using btree) */ 1119 open_flags, /* Open flags */ 1120 0); /* File mode. Using defaults */ 1121 1122 if (ret != 0) { 1123 envp->err(envp, ret, 1124 "Attempt to open db failed."); 1125 goto err; 1126 } 1127 1128 /* Configure the cache file */ 1129 mpf = dbp->get_mpf(dbp); 1130 ret = mpf->set_flags(mpf, DB_MPOOL_NOFILE, 1); 1131 1132 if (ret != 0) { 1133 envp->err(envp, ret, 1134 "Attempt failed to configure for no backing of temp files."); 1135 goto err; 1136 } 1137 1138err: 1139 /* Close our database handle, if it was opened. */ 1140 if (dbp != NULL) { 1141 ret_t = dbp->close(dbp, 0); 1142 if (ret_t != 0) { 1143 fprintf(stderr, "%s database close failed.\n", 1144 db_strerror(ret_t)); 1145 ret = ret_t; 1146 } 1147 } 1148 1149 /* Close our environment, if it was opened. */ 1150 if (envp != NULL) { 1151 ret_t = envp->close(envp, 0); 1152 if (ret_t != 0) { 1153 fprintf(stderr, "environment close failed: %s\n", 1154 db_strerror(ret_t)); 1155 ret = ret_t; 1156 } 1157 } 1158 1159 /* Final status message and return. */ 1160 printf("I'm all done.\n"); 1161 return (ret == 0 ? EXIT_SUCCESS : EXIT_FAILURE); 1162} </pre> 1163 </div> 1164 </div> 1165 <div class="navfooter"> 1166 <hr /> 1167 </div> 1168 </body> 1169</html> 1170