1<?xml version="1.0" encoding="UTF-8" standalone="no"?> 2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 3<html xmlns="http://www.w3.org/1999/xhtml"> 4 <head> 5 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> 6 <title>Base API In-Memory Transaction Example</title> 7 <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> 8 <meta name="generator" content="DocBook XSL Stylesheets V1.62.4" /> 9 <link rel="home" href="index.html" title="Getting Started with Berkeley DB Transaction Processing" /> 10 <link rel="up" href="wrapup.html" title="Chapter��6.��Summary and Examples" /> 11 <link rel="previous" href="txnexample_dpl.html" title="DPL Transaction Example" /> 12 </head> 13 <body> 14 <div class="navheader"> 15 <table width="100%" summary="Navigation header"> 16 <tr> 17 <th colspan="3" align="center">Base API In-Memory Transaction Example</th> 18 </tr> 19 <tr> 20 <td width="20%" align="left"><a accesskey="p" href="txnexample_dpl.html">Prev</a>��</td> 21 <th width="60%" align="center">Chapter��6.��Summary and Examples</th> 22 <td width="20%" align="right">��</td> 23 </tr> 24 </table> 25 <hr /> 26 </div> 27 <div class="sect1" lang="en" xml:lang="en"> 28 <div class="titlepage"> 29 <div> 30 <div> 31 <h2 class="title" style="clear: both"><a id="inmem_txnexample_java"></a>Base API In-Memory Transaction Example</h2> 32 </div> 33 </div> 34 <div></div> 35 </div> 36 <p> 37 DB is sometimes used for applications that simply need to cache 38 data retrieved from some other location (such as a remote database 39 server). DB is also often used in embedded systems. 40 </p> 41 <p> 42 In both cases, applications may still want to use transactions for 43 atomicity, consistency, and isolation guarantees, but they may want 44 to forgo the durability guarantee entirely. That is, they may want 45 their DB environment and databases kept entirely in-memory so 46 as to avoid the performance impact of unneeded disk I/O. 47 </p> 48 <p> 49 To do this: 50 </p> 51 <div class="itemizedlist"> 52 <ul type="disc"> 53 <li> 54 <p> 55 Refrain from specifying a home directory when you open your 56 environment. The exception to this is if you are using the 57 <tt class="literal">DB_CONFIG</tt> configuration file ��� in 58 that case you must identify the environment's home 59 directory so that the configuration file can be found. 60 </p> 61 </li> 62 <li> 63 <p> 64 Configure your environment to back your regions from 65 system memory instead of the filesystem. 66 </p> 67 </li> 68 <li> 69 <p> 70 Configure your logging subsystem such that log files are kept 71 entirely in-memory. 72 </p> 73 </li> 74 <li> 75 <p> 76 Increase the size of your in-memory log buffer so that it 77 is large enough to hold the largest set of concurrent write operations. 78 </p> 79 </li> 80 <li> 81 <p> 82 Increase the size of your in-memory cache so that it can 83 hold your entire data set. You do not want your cache to 84 page to disk. 85 </p> 86 </li> 87 <li> 88 <p> 89 Do not specify a file name when you open your database(s). 90 </p> 91 </li> 92 </ul> 93 </div> 94 <p> 95 As an example, this section takes the transaction example provided 96 in <a href="txnexample_java.html">Base API Transaction Example</a> 97 and it updates that example so that the environment, database, log 98 files, and regions are all kept entirely in-memory. 99 </p> 100 <p> 101 For illustration purposes, we also modify this example so that 102 uncommitted reads are no longer used to enable the <tt class="methodname">countRecords()</tt> 103 method. Instead, we simply provide a transaction handle to 104 <tt class="methodname">countRecords()</tt> so as to avoid the 105 self-deadlock. 106 </p> 107 <p> 108 The majority of the modifications to the original example are performed in the <tt class="classname">TxnGuide</tt> 109 example class (see <a href="txnexample_java.html#txnguideexample">TxnGuide.java</a>). 110 This is because the majority of the work that we need to do is performed when the environment and 111 databases are opened. 112 </p> 113 <p> 114 To begin, we simplify the beginning of the class a bit. We eliminate some variables that the example no longer 115 needs ��� specifically variables having to do with the location of the environment and the names of the 116 database files. 117 We can also remove our <tt class="function">usage()</tt> method because we no 118 longer require any command line arguments. 119 </p> 120 <pre class="programlisting">// File TxnGuideInMemory.java 121 122package db.txn; 123 124import com.sleepycat.bind.serial.StoredClassCatalog; 125 126import com.sleepycat.db.Database; 127import com.sleepycat.db.DatabaseConfig; 128import com.sleepycat.db.DatabaseException; 129import com.sleepycat.db.DatabaseType; 130import com.sleepycat.db.LockDetectMode; 131 132import com.sleepycat.db.Environment; 133import com.sleepycat.db.EnvironmentConfig; 134 135import java.io.File; 136import java.io.FileNotFoundException; 137 138<b class="userinput"><tt>public class TxnGuideInMemory {</tt></b> 139 140 // DB handles 141 private static Database myDb = null; 142 private static Database myClassDb = null; 143 private static Environment myEnv = null; 144 145 private static final int NUMTHREADS = 5; </pre> 146 <p> 147 Next, in our <tt class="function">main()</tt> method, we 148 remove the call to <tt class="methodname">parseArgs()</tt> because that only existed in the previous example for 149 collecting the environment home location. Everything else is essentially the same. 150</p> 151 <pre class="programlisting"> public static void main(String args[]) { 152 try { 153 154 // Open the environment and databases 155 openEnv(); 156 157 // Get our class catalog (used to serialize objects) 158 StoredClassCatalog classCatalog = 159 new StoredClassCatalog(myClassDb); 160 161 // Start the threads 162 DBWriter[] threadArray; 163 threadArray = new DBWriter[NUMTHREADS]; 164 for (int i = 0; i < NUMTHREADS; i++) { 165 threadArray[i] = new DBWriter(myEnv, myDb, classCatalog); 166 threadArray[i].start(); 167 } 168 169 for (int i = 0; i < NUMTHREADS; i++) { 170 threadArray[i].join(); 171 } 172 } catch (Exception e) { 173 System.err.println("<b class="userinput"><tt>TxnGuideInMemory</tt></b>: " + e.toString()); 174 e.printStackTrace(); 175 } finally { 176 closeEnv(); 177 } 178 System.out.println("All done."); 179 } </pre> 180 <p> 181 Next we open our environment as always. However, in doing so we: 182 </p> 183 <div class="itemizedlist"> 184 <ul type="disc"> 185 <li> 186 <p> 187 Set <tt class="methodname">EnvironmentConfig.setPrivate()</tt> 188 to <tt class="literal">true</tt>. 189 This causes our environment to back regions using our 190 application's heap memory rather than by using the filesystem. 191 This is the first important step to keeping our DB data 192 entirely in-memory. 193 </p> 194 </li> 195 <li> 196 <p> 197 Remove <tt class="methodname">runRecovery()</tt> 198 from the environment configuration. Because all our data will be held entirely in memory, recovery is a 199 non-issue. Note that if we had left the call to <tt class="methodname">runRecovery()</tt> 200 in, it would be silently ignored. 201 </p> 202 </li> 203 </ul> 204 </div> 205 <pre class="programlisting"> private static void openEnv() throws DatabaseException { 206 System.out.println("opening env"); 207 208 // Set up the environment. 209 EnvironmentConfig myEnvConfig = new EnvironmentConfig(); 210 211 <b class="userinput"><tt>// Region files are not backed by the filesystem, they are 212 // backed by heap memory. 213 myEnvConfig.setPrivate(true);</tt></b> 214 215 myEnvConfig.setAllowCreate(true); 216 myEnvConfig.setInitializeCache(true); 217 myEnvConfig.setInitializeLocking(true); 218 myEnvConfig.setInitializeLogging(true); 219 myEnvConfig.setTransactional(true); 220 // EnvironmentConfig.setThreaded(true) is the default behavior 221 // in Java, so we do not have to do anything to cause the 222 // environment handle to be free-threaded. 223 224 // Indicate that we want db to internally perform deadlock 225 // detection. Also indicate that the transaction that has 226 // performed the least amount of write activity to 227 // receive the deadlock notification, if any. 228 myEnvConfig.setLockDetectMode(LockDetectMode.MINWRITE); </pre> 229 <p> 230 Now we configure our environment to keep the log files in memory, 231 increase the log buffer size to 10 MB, and increase our in-memory 232 cache to 10 MB. These values should be more than enough for our 233 application's workload. 234 </p> 235 <pre class="programlisting"> 236 <b class="userinput"> 237 <tt> // Specify in-memory logging 238 myEnvConfig.setLogInMemory(true); 239 // Specify the size of the in-memory log buffer 240 // Must be large enough to handle the log data created by 241 // the largest transaction. 242 myEnvConfig.setLogBufferSize(10 * 1024 * 1024); 243 // Specify the size of the in-memory cache 244 // Set it large enough so that it won't page. 245 myEnvConfig.setCacheSize(10 * 1024 * 1024); </tt> 246 </b> 247 </pre> 248 <p> 249 Our database configuration is identical to the original example, except that we do not specify 250 <tt class="methodname">setReadUncomitted()</tt> here. We will be causing our <tt class="methodname">countRecords()</tt> 251 method to join the transaction rather than perform uncommitted reads, so we do not need our database to support them. 252</p> 253 <pre class="programlisting"> // Set up the database 254 DatabaseConfig myDbConfig = new DatabaseConfig(); 255 myDbConfig.setType(DatabaseType.BTREE); 256 myDbConfig.setAllowCreate(true); 257 myDbConfig.setTransactional(true); 258 myDbConfig.setSortedDuplicates(true); 259 // no DatabaseConfig.setThreaded() method available. 260 // db handles in java are free-threaded so long as the 261 // env is also free-threaded. </pre> 262 <p> 263 Next, we open the environment. This is 264 identical to how the example previously worked, except that we do not 265 provide a location for the environment's home directory. 266 </p> 267 <pre class="programlisting"> try { 268 // Open the environment 269 myEnv = new Environment(<b class="userinput"><tt>null</tt></b>, // Env home 270 myEnvConfig); </pre> 271 <p> 272 When we open our databases, we also specify <tt class="literal">null</tt> for the file names. The causes the database 273 to not be backed by the filesystem; that is, the databases are held entirely in memory. 274 </p> 275 <pre class="programlisting"> // Open the database. Do not provide a txn handle. This open 276 // is auto committed because DatabaseConfig.setTransactional() 277 // is true. 278 myDb = myEnv.openDatabase(null, // txn handle 279 <b class="userinput"><tt>null</tt></b>, // Database file name 280 null, // Database name 281 myDbConfig); 282 283 // Used by the bind API for serializing objects 284 // Class database must not support duplicates 285 myDbConfig.setSortedDuplicates(false); 286 myClassDb = myEnv.openDatabase(null, // txn handle 287 <b class="userinput"><tt>null</tt></b>, // Database file name 288 null, // Database name, 289 myDbConfig); 290 } catch (FileNotFoundException fnfe) { 291 System.err.println("openEnv: " + fnfe.toString()); 292 System.exit(-1); 293 } 294 } </pre> 295 <p> 296 After that, our class is unchanged, except for some very minor modifications. 297 Most notably, we remove the <tt class="methodname">parseArgs()</tt> 298 method from the application, because we no longer need it. 299 </p> 300 <pre class="programlisting"> private static void closeEnv() { 301 System.out.println("Closing env"); 302 if (myDb != null ) { 303 try { 304 myDb.close(); 305 } catch (DatabaseException e) { 306 System.err.println("closeEnv: myDb: " + 307 e.toString()); 308 e.printStackTrace(); 309 } 310 } 311 312 if (myClassDb != null ) { 313 try { 314 myClassDb.close(); 315 } catch (DatabaseException e) { 316 System.err.println("closeEnv: myClassDb: " + 317 e.toString()); 318 e.printStackTrace(); 319 } 320 } 321 322 if (myEnv != null ) { 323 try { 324 myEnv.close(); 325 } catch (DatabaseException e) { 326 System.err.println("closeEnv: " + e.toString()); 327 e.printStackTrace(); 328 } 329 } 330 } 331 332 <b class="userinput"><tt>private TxnGuideInMemory() {}</tt></b> 333} </pre> 334 <p> 335 That completes our modifications to this class. 336 We now turn our attention to our <tt class="classname">DBWriter</tt> 337 class (see <a href="txnexample_java.html#dbwriter">DBWriter.java</a>). 338 It is unchanged, except for one small modification. In the 339 <tt class="methodname">run()</tt> method, we call <tt class="methodname">countRecords()</tt> 340 with a transaction handle, rather than configuring our entire 341 application for uncommitted reads. Both mechanisms work well-enough 342 for preventing a self-deadlock. However, the individual count 343 in this example will tend to be lower than the counts seen in 344 the previous transaction example, because 345 <tt class="function">countRecords()</tt> can no longer see records 346 created but not yet committed by other threads. 347 Additionally, the usage of the transaction handle here will 348 probably cause more deadlocks than using read-uncommitted does, because more locking is being performed in 349 this case. 350 </p> 351 <pre class="programlisting">package db.txn; 352 353import com.sleepycat.bind.EntryBinding; 354import com.sleepycat.bind.serial.StoredClassCatalog; 355import com.sleepycat.bind.serial.SerialBinding; 356import com.sleepycat.bind.tuple.StringBinding; 357 358import com.sleepycat.db.Cursor; 359import com.sleepycat.db.CursorConfig; 360import com.sleepycat.db.Database; 361import com.sleepycat.db.DatabaseEntry; 362import com.sleepycat.db.DatabaseException; 363import com.sleepycat.db.DeadlockException; 364import com.sleepycat.db.Environment; 365import com.sleepycat.db.LockMode; 366import com.sleepycat.db.OperationStatus; 367import com.sleepycat.db.Transaction; 368 369import java.io.UnsupportedEncodingException; 370import java.util.Random; 371 372public class DBWriter extends Thread 373{ 374 private Database myDb = null; 375 private Environment myEnv = null; 376 private EntryBinding dataBinding = null; 377 private Random generator = new Random(); 378 379 private static final int MAX_RETRY = 20; 380 381 private static String[] keys = {"key 1", "key 2", "key 3", 382 "key 4", "key 5", "key 6", 383 "key 7", "key 8", "key 9", 384 "key 10"}; 385 386 387 // Constructor. Get our DB handles from here 388 DBWriter(Environment env, Database db, StoredClassCatalog scc) 389 throws DatabaseException { 390 myDb = db; 391 myEnv = env; 392 dataBinding = new SerialBinding(scc, PayloadData.class); 393 } 394 395 396 // Thread method that writes a series of records 397 // to the database using transaction protection. 398 // Deadlock handling is demonstrated here. 399 public void run () { 400 Transaction txn = null; 401 402 // Perform 50 transactions 403 for (int i=0; i<50; i++) { 404 405 boolean retry = true; 406 int retry_count = 0; 407 // while loop is used for deadlock retries 408 while (retry) { 409 // try block used for deadlock detection and 410 // general db exception handling 411 try { 412 413 // Get a transaction 414 txn = myEnv.beginTransaction(null, null); 415 // Write 10 records to the db 416 // for each transaction 417 for (int j = 0; j < 10; j++) { 418 // Get the key 419 DatabaseEntry key = new DatabaseEntry(); 420 StringBinding.stringToEntry(keys[j], key); 421 422 // Get the data 423 PayloadData pd = new PayloadData(i+j, getName(), 424 generator.nextDouble()); 425 DatabaseEntry data = new DatabaseEntry(); 426 dataBinding.objectToEntry(pd, data); 427 428 // Do the put 429 myDb.put(txn, key, data); 430 } 431 432 // commit 433 System.out.println(getName() + 434 " : committing txn : " + i); 435 436 System.out.println(getName() + " : Found " + 437 countRecords(<b class="userinput"><tt>txn</tt></b>) + " records in the database."); 438 try { 439 txn.commit(); 440 txn = null; 441 } catch (DatabaseException e) { 442 System.err.println("Error on txn commit: " + 443 e.toString()); 444 } 445 retry = false; 446 447 } catch (DeadlockException de) { 448 System.out.println("################# " + getName() + 449 " : caught deadlock"); 450 // retry if necessary 451 if (retry_count < MAX_RETRY) { 452 System.err.println(getName() + 453 " : Retrying operation."); 454 retry = true; 455 retry_count++; 456 } else { 457 System.err.println(getName() + 458 " : out of retries. Giving up."); 459 retry = false; 460 } 461 } catch (DatabaseException e) { 462 // abort and don't retry 463 retry = false; 464 System.err.println(getName() + 465 " : caught exception: " + e.toString()); 466 System.err.println(getName() + 467 " : errno: " + e.getErrno()); 468 e.printStackTrace(); 469 } finally { 470 if (txn != null) { 471 try { 472 txn.abort(); 473 } catch (Exception e) { 474 System.err.println( 475 "Error aborting transaction: " + 476 e.toString()); 477 e.printStackTrace(); 478 } 479 } 480 } 481 } 482 } 483 } </pre> 484 <p> 485 Next we update <tt class="methodname">countRecords()</tt>. The only difference 486 here is that we no longer specify <tt class="methodname">CursorConfig.setReadUncomitted()</tt> when 487 we open our cursor. Note that even this minor change is not required. 488 If we do not configure our database to support uncommitted reads, 489 <tt class="methodname">CursorConfig.setReadUncomitted()</tt> is silently 490 ignored. However, we remove the property anyway from the cursor open so as to 491 avoid confusion. 492</p> 493 <pre class="programlisting"> // This simply counts the number of records contained in the 494 // database and returns the result. You can use this method 495 // in three ways: 496 // 497 // First call it with an active txn handle. 498 // Secondly, configure the cursor for uncommitted reads 499 // Third, call count_records AFTER the writer has committed 500 // its transaction. 501 // 502 // If you do none of these things, the writer thread will 503 // self-deadlock. 504 // 505 // Note that this method exists only for illustrative purposes. 506 // A more straight-forward way to count the number of records in 507 // a database is to use the Database.getStats() method. 508 private int countRecords(Transaction txn) throws DatabaseException { 509 DatabaseEntry key = new DatabaseEntry(); 510 DatabaseEntry data = new DatabaseEntry(); 511 int count = 0; 512 Cursor cursor = null; 513 514 try { 515 // Get the cursor 516 CursorConfig cc = new CursorConfig(); 517 cc.setReadUncomitted(true); 518 cursor = myDb.openCursor(txn, cc); 519 while (cursor.getNext(key, data, LockMode.DEFAULT) == 520 OperationStatus.SUCCESS) { 521 522 count++; 523 } 524 } finally { 525 if (cursor != null) { 526 cursor.close(); 527 } 528 } 529 530 return count; 531 532 } 533} </pre> 534 <p> 535 This completes our in-memory transactional example. If you would like to 536 experiment with this code, you can find the example in the following 537 location in your DB distribution: 538</p> 539 <pre class="programlisting"><span class="emphasis"><em>DB_INSTALL</em></span>/examples_java/src/db/txn</pre> 540 </div> 541 <div class="navfooter"> 542 <hr /> 543 <table width="100%" summary="Navigation footer"> 544 <tr> 545 <td width="40%" align="left"><a accesskey="p" href="txnexample_dpl.html">Prev</a>��</td> 546 <td width="20%" align="center"> 547 <a accesskey="u" href="wrapup.html">Up</a> 548 </td> 549 <td width="40%" align="right">��</td> 550 </tr> 551 <tr> 552 <td width="40%" align="left" valign="top">DPL Transaction Example��</td> 553 <td width="20%" align="center"> 554 <a accesskey="h" href="index.html">Home</a> 555 </td> 556 <td width="40%" align="right" valign="top">��</td> 557 </tr> 558 </table> 559 </div> 560 </body> 561</html> 562