1<?xml version="1.0" encoding="UTF-8" standalone="no"?> 2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 3<html xmlns="http://www.w3.org/1999/xhtml"> 4 <head> 5 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> 6 <title>Base API In-Memory Transaction Example</title> 7 <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> 8 <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> 9 <link rel="start" href="index.html" title="Getting Started with Berkeley DB Transaction Processing" /> 10 <link rel="up" href="wrapup.html" title="Chapter��6.��Summary and Examples" /> 11 <link rel="prev" href="txnexample_dpl.html" title="DPL Transaction Example" /> 12 </head> 13 <body> 14 <div class="navheader"> 15 <table width="100%" summary="Navigation header"> 16 <tr> 17 <th colspan="3" align="center">Base API In-Memory Transaction Example</th> 18 </tr> 19 <tr> 20 <td width="20%" align="left"><a accesskey="p" href="txnexample_dpl.html">Prev</a>��</td> 21 <th width="60%" align="center">Chapter��6.��Summary and Examples</th> 22 <td width="20%" align="right">��</td> 23 </tr> 24 </table> 25 <hr /> 26 </div> 27 <div class="sect1" lang="en" xml:lang="en"> 28 <div class="titlepage"> 29 <div> 30 <div> 31 <h2 class="title" style="clear: both"><a id="inmem_txnexample_java"></a>Base API In-Memory Transaction Example</h2> 32 </div> 33 </div> 34 </div> 35 <p> 36 DB is sometimes used for applications that simply need to cache 37 data retrieved from some other location (such as a remote database 38 server). DB is also often used in embedded systems. 39 </p> 40 <p> 41 In both cases, applications may still want to use transactions for 42 atomicity, consistency, and isolation guarantees, but they may want 43 to forgo the durability guarantee entirely. That is, they may want 44 their DB environment and databases kept entirely in-memory so 45 as to avoid the performance impact of unneeded disk I/O. 46 </p> 47 <p> 48 To do this: 49 </p> 50 <div class="itemizedlist"> 51 <ul type="disc"> 52 <li> 53 <p> 54 Refrain from specifying a home directory when you open your 55 environment. The exception to this is if you are using the 56 <code class="literal">DB_CONFIG</code> configuration file ��� in 57 that case you must identify the environment's home 58 directory so that the configuration file can be found. 59 </p> 60 </li> 61 <li> 62 <p> 63 Configure your environment to back your regions from 64 system memory instead of the filesystem. 65 </p> 66 </li> 67 <li> 68 <p> 69 Configure your logging subsystem such that log files are kept 70 entirely in-memory. 71 </p> 72 </li> 73 <li> 74 <p> 75 Increase the size of your in-memory log buffer so that it 76 is large enough to hold the largest set of concurrent write operations. 77 </p> 78 </li> 79 <li> 80 <p> 81 Increase the size of your in-memory cache so that it can 82 hold your entire data set. You do not want your cache to 83 page to disk. 84 </p> 85 </li> 86 <li> 87 <p> 88 Do not specify a file name when you open your database(s). 89 </p> 90 </li> 91 </ul> 92 </div> 93 <p> 94 As an example, this section takes the transaction example provided 95 in <a class="xref" href="txnexample_java.html" title="Base API Transaction Example">Base API Transaction Example</a> 96 and it updates that example so that the environment, database, log 97 files, and regions are all kept entirely in-memory. 98 </p> 99 <p> 100 For illustration purposes, we also modify this example so that 101 uncommitted reads are no longer used to enable the <code class="methodname">countRecords()</code> 102 method. Instead, we simply provide a transaction handle to 103 <code class="methodname">countRecords()</code> so as to avoid the 104 self-deadlock. 105 </p> 106 <p> 107 The majority of the modifications to the original example are performed in the <code class="classname">TxnGuide</code> 108 example class (see <a class="xref" href="txnexample_java.html#txnguideexample" title="TxnGuide.java">TxnGuide.java</a>). 109 This is because the majority of the work that we need to do is performed when the environment and 110 databases are opened. 111 </p> 112 <p> 113 To begin, we simplify the beginning of the class a bit. We eliminate some variables that the example no longer 114 needs ��� specifically variables having to do with the location of the environment and the names of the 115 database files. 116 We can also remove our <code class="function">usage()</code> method because we no 117 longer require any command line arguments. 118 </p> 119 <pre class="programlisting">// File TxnGuideInMemory.java 120 121package db.txn; 122 123import com.sleepycat.bind.serial.StoredClassCatalog; 124 125import com.sleepycat.db.Database; 126import com.sleepycat.db.DatabaseConfig; 127import com.sleepycat.db.DatabaseException; 128import com.sleepycat.db.DatabaseType; 129import com.sleepycat.db.LockDetectMode; 130 131import com.sleepycat.db.Environment; 132import com.sleepycat.db.EnvironmentConfig; 133 134import java.io.File; 135import java.io.FileNotFoundException; 136 137<strong class="userinput"><code>public class TxnGuideInMemory {</code></strong> 138 139 // DB handles 140 private static Database myDb = null; 141 private static Database myClassDb = null; 142 private static Environment myEnv = null; 143 144 private static final int NUMTHREADS = 5; </pre> 145 <p> 146 Next, in our <code class="function">main()</code> method, we 147 remove the call to <code class="methodname">parseArgs()</code> because that only existed in the previous example for 148 collecting the environment home location. Everything else is essentially the same. 149</p> 150 <pre class="programlisting"> public static void main(String args[]) { 151 try { 152 153 // Open the environment and databases 154 openEnv(); 155 156 // Get our class catalog (used to serialize objects) 157 StoredClassCatalog classCatalog = 158 new StoredClassCatalog(myClassDb); 159 160 // Start the threads 161 DBWriter[] threadArray; 162 threadArray = new DBWriter[NUMTHREADS]; 163 for (int i = 0; i < NUMTHREADS; i++) { 164 threadArray[i] = new DBWriter(myEnv, myDb, classCatalog); 165 threadArray[i].start(); 166 } 167 168 for (int i = 0; i < NUMTHREADS; i++) { 169 threadArray[i].join(); 170 } 171 } catch (Exception e) { 172 System.err.println("<strong class="userinput"><code>TxnGuideInMemory</code></strong>: " + e.toString()); 173 e.printStackTrace(); 174 } finally { 175 closeEnv(); 176 } 177 System.out.println("All done."); 178 } </pre> 179 <p> 180 Next we open our environment as always. However, in doing so we: 181 </p> 182 <div class="itemizedlist"> 183 <ul type="disc"> 184 <li> 185 <p> 186 Set <code class="methodname">EnvironmentConfig.setPrivate()</code> 187 to <code class="literal">true</code>. 188 This causes our environment to back regions using our 189 application's heap memory rather than by using the filesystem. 190 This is the first important step to keeping our DB data 191 entirely in-memory. 192 </p> 193 </li> 194 <li> 195 <p> 196 Remove <code class="methodname">runRecovery()</code> 197 from the environment configuration. Because all our data will be held entirely in memory, recovery is a 198 non-issue. Note that if we had left the call to <code class="methodname">runRecovery()</code> 199 in, it would be silently ignored. 200 </p> 201 </li> 202 </ul> 203 </div> 204 <pre class="programlisting"> private static void openEnv() throws DatabaseException { 205 System.out.println("opening env"); 206 207 // Set up the environment. 208 EnvironmentConfig myEnvConfig = new EnvironmentConfig(); 209 210 <strong class="userinput"><code>// Region files are not backed by the filesystem, they are 211 // backed by heap memory. 212 myEnvConfig.setPrivate(true);</code></strong> 213 214 myEnvConfig.setAllowCreate(true); 215 myEnvConfig.setInitializeCache(true); 216 myEnvConfig.setInitializeLocking(true); 217 myEnvConfig.setInitializeLogging(true); 218 myEnvConfig.setTransactional(true); 219 // EnvironmentConfig.setThreaded(true) is the default behavior 220 // in Java, so we do not have to do anything to cause the 221 // environment handle to be free-threaded. 222 223 // Indicate that we want db to internally perform deadlock 224 // detection. Also indicate that the transaction that has 225 // performed the least amount of write activity to 226 // receive the deadlock notification, if any. 227 myEnvConfig.setLockDetectMode(LockDetectMode.MINWRITE); </pre> 228 <p> 229 Now we configure our environment to keep the log files in memory, 230 increase the log buffer size to 10 MB, and increase our in-memory 231 cache to 10 MB. These values should be more than enough for our 232 application's workload. 233 </p> 234 <pre class="programlisting"> 235 <strong class="userinput"> 236 <code> // Specify in-memory logging 237 myEnvConfig.setLogInMemory(true); 238 // Specify the size of the in-memory log buffer 239 // Must be large enough to handle the log data created by 240 // the largest transaction. 241 myEnvConfig.setLogBufferSize(10 * 1024 * 1024); 242 // Specify the size of the in-memory cache 243 // Set it large enough so that it won't page. 244 myEnvConfig.setCacheSize(10 * 1024 * 1024); </code> 245 </strong> 246 </pre> 247 <p> 248 Our database configuration is identical to the original example, except that we do not specify 249 <code class="methodname">setReadUncomitted()</code> here. We will be causing our <code class="methodname">countRecords()</code> 250 method to join the transaction rather than perform uncommitted reads, so we do not need our database to support them. 251</p> 252 <pre class="programlisting"> // Set up the database 253 DatabaseConfig myDbConfig = new DatabaseConfig(); 254 myDbConfig.setType(DatabaseType.BTREE); 255 myDbConfig.setAllowCreate(true); 256 myDbConfig.setTransactional(true); 257 myDbConfig.setSortedDuplicates(true); 258 // no DatabaseConfig.setThreaded() method available. 259 // db handles in java are free-threaded so long as the 260 // env is also free-threaded. </pre> 261 <p> 262 Next, we open the environment. This is 263 identical to how the example previously worked, except that we do not 264 provide a location for the environment's home directory. 265 </p> 266 <pre class="programlisting"> try { 267 // Open the environment 268 myEnv = new Environment(<strong class="userinput"><code>null</code></strong>, // Env home 269 myEnvConfig); </pre> 270 <p> 271 When we open our databases, we also specify <code class="literal">null</code> for the file names. The causes the database 272 to not be backed by the filesystem; that is, the databases are held entirely in memory. 273 </p> 274 <pre class="programlisting"> // Open the database. Do not provide a txn handle. This open 275 // is auto committed because DatabaseConfig.setTransactional() 276 // is true. 277 myDb = myEnv.openDatabase(null, // txn handle 278 <strong class="userinput"><code>null</code></strong>, // Database file name 279 null, // Database name 280 myDbConfig); 281 282 // Used by the bind API for serializing objects 283 // Class database must not support duplicates 284 myDbConfig.setSortedDuplicates(false); 285 myClassDb = myEnv.openDatabase(null, // txn handle 286 <strong class="userinput"><code>null</code></strong>, // Database file name 287 null, // Database name, 288 myDbConfig); 289 } catch (FileNotFoundException fnfe) { 290 System.err.println("openEnv: " + fnfe.toString()); 291 System.exit(-1); 292 } 293 } </pre> 294 <p> 295 After that, our class is unchanged, except for some very minor modifications. 296 Most notably, we remove the <code class="methodname">parseArgs()</code> 297 method from the application, because we no longer need it. 298 </p> 299 <pre class="programlisting"> private static void closeEnv() { 300 System.out.println("Closing env"); 301 if (myDb != null ) { 302 try { 303 myDb.close(); 304 } catch (DatabaseException e) { 305 System.err.println("closeEnv: myDb: " + 306 e.toString()); 307 e.printStackTrace(); 308 } 309 } 310 311 if (myClassDb != null ) { 312 try { 313 myClassDb.close(); 314 } catch (DatabaseException e) { 315 System.err.println("closeEnv: myClassDb: " + 316 e.toString()); 317 e.printStackTrace(); 318 } 319 } 320 321 if (myEnv != null ) { 322 try { 323 myEnv.close(); 324 } catch (DatabaseException e) { 325 System.err.println("closeEnv: " + e.toString()); 326 e.printStackTrace(); 327 } 328 } 329 } 330 331 <strong class="userinput"><code>private TxnGuideInMemory() {}</code></strong> 332} </pre> 333 <p> 334 That completes our modifications to this class. 335 We now turn our attention to our <code class="classname">DBWriter</code> 336 class (see <a class="xref" href="txnexample_java.html#dbwriter" title="DBWriter.java">DBWriter.java</a>). 337 It is unchanged, except for one small modification. In the 338 <code class="methodname">run()</code> method, we call <code class="methodname">countRecords()</code> 339 with a transaction handle, rather than configuring our entire 340 application for uncommitted reads. Both mechanisms work well-enough 341 for preventing a self-deadlock. However, the individual count 342 in this example will tend to be lower than the counts seen in 343 the previous transaction example, because 344 <code class="function">countRecords()</code> can no longer see records 345 created but not yet committed by other threads. 346 Additionally, the usage of the transaction handle here will 347 probably cause more deadlocks than using read-uncommitted does, because more locking is being performed in 348 this case. 349 </p> 350 <pre class="programlisting">package db.txn; 351 352import com.sleepycat.bind.EntryBinding; 353import com.sleepycat.bind.serial.StoredClassCatalog; 354import com.sleepycat.bind.serial.SerialBinding; 355import com.sleepycat.bind.tuple.StringBinding; 356 357import com.sleepycat.db.Cursor; 358import com.sleepycat.db.CursorConfig; 359import com.sleepycat.db.Database; 360import com.sleepycat.db.DatabaseEntry; 361import com.sleepycat.db.DatabaseException; 362import com.sleepycat.db.DeadlockException; 363import com.sleepycat.db.Environment; 364import com.sleepycat.db.LockMode; 365import com.sleepycat.db.OperationStatus; 366import com.sleepycat.db.Transaction; 367 368import java.io.UnsupportedEncodingException; 369import java.util.Random; 370 371public class DBWriter extends Thread 372{ 373 private Database myDb = null; 374 private Environment myEnv = null; 375 private EntryBinding dataBinding = null; 376 private Random generator = new Random(); 377 378 private static final int MAX_RETRY = 20; 379 380 private static String[] keys = {"key 1", "key 2", "key 3", 381 "key 4", "key 5", "key 6", 382 "key 7", "key 8", "key 9", 383 "key 10"}; 384 385 386 // Constructor. Get our DB handles from here 387 DBWriter(Environment env, Database db, StoredClassCatalog scc) 388 throws DatabaseException { 389 myDb = db; 390 myEnv = env; 391 dataBinding = new SerialBinding(scc, PayloadData.class); 392 } 393 394 395 // Thread method that writes a series of records 396 // to the database using transaction protection. 397 // Deadlock handling is demonstrated here. 398 public void run () { 399 Transaction txn = null; 400 401 // Perform 50 transactions 402 for (int i=0; i<50; i++) { 403 404 boolean retry = true; 405 int retry_count = 0; 406 // while loop is used for deadlock retries 407 while (retry) { 408 // try block used for deadlock detection and 409 // general db exception handling 410 try { 411 412 // Get a transaction 413 txn = myEnv.beginTransaction(null, null); 414 // Write 10 records to the db 415 // for each transaction 416 for (int j = 0; j < 10; j++) { 417 // Get the key 418 DatabaseEntry key = new DatabaseEntry(); 419 StringBinding.stringToEntry(keys[j], key); 420 421 // Get the data 422 PayloadData pd = new PayloadData(i+j, getName(), 423 generator.nextDouble()); 424 DatabaseEntry data = new DatabaseEntry(); 425 dataBinding.objectToEntry(pd, data); 426 427 // Do the put 428 myDb.put(txn, key, data); 429 } 430 431 // commit 432 System.out.println(getName() + 433 " : committing txn : " + i); 434 435 System.out.println(getName() + " : Found " + 436 countRecords(<strong class="userinput"><code>txn</code></strong>) + " records in the database."); 437 try { 438 txn.commit(); 439 txn = null; 440 } catch (DatabaseException e) { 441 System.err.println("Error on txn commit: " + 442 e.toString()); 443 } 444 retry = false; 445 446 } catch (DeadlockException de) { 447 System.out.println("################# " + getName() + 448 " : caught deadlock"); 449 // retry if necessary 450 if (retry_count < MAX_RETRY) { 451 System.err.println(getName() + 452 " : Retrying operation."); 453 retry = true; 454 retry_count++; 455 } else { 456 System.err.println(getName() + 457 " : out of retries. Giving up."); 458 retry = false; 459 } 460 } catch (DatabaseException e) { 461 // abort and don't retry 462 retry = false; 463 System.err.println(getName() + 464 " : caught exception: " + e.toString()); 465 System.err.println(getName() + 466 " : errno: " + e.getErrno()); 467 e.printStackTrace(); 468 } finally { 469 if (txn != null) { 470 try { 471 txn.abort(); 472 } catch (Exception e) { 473 System.err.println( 474 "Error aborting transaction: " + 475 e.toString()); 476 e.printStackTrace(); 477 } 478 } 479 } 480 } 481 } 482 } </pre> 483 <p> 484 Next we update <code class="methodname">countRecords()</code>. The only difference 485 here is that we no longer specify <code class="methodname">CursorConfig.setReadUncomitted()</code> when 486 we open our cursor. Note that even this minor change is not required. 487 If we do not configure our database to support uncommitted reads, 488 <code class="methodname">CursorConfig.setReadUncomitted()</code> is silently 489 ignored. However, we remove the property anyway from the cursor open so as to 490 avoid confusion. 491</p> 492 <pre class="programlisting"> // This simply counts the number of records contained in the 493 // database and returns the result. You can use this method 494 // in three ways: 495 // 496 // First call it with an active txn handle. 497 // Secondly, configure the cursor for uncommitted reads 498 // Third, call count_records AFTER the writer has committed 499 // its transaction. 500 // 501 // If you do none of these things, the writer thread will 502 // self-deadlock. 503 // 504 // Note that this method exists only for illustrative purposes. 505 // A more straight-forward way to count the number of records in 506 // a database is to use the Database.getStats() method. 507 private int countRecords(Transaction txn) throws DatabaseException { 508 DatabaseEntry key = new DatabaseEntry(); 509 DatabaseEntry data = new DatabaseEntry(); 510 int count = 0; 511 Cursor cursor = null; 512 513 try { 514 // Get the cursor 515 CursorConfig cc = new CursorConfig(); 516 cc.setReadUncomitted(true); 517 cursor = myDb.openCursor(txn, cc); 518 while (cursor.getNext(key, data, LockMode.DEFAULT) == 519 OperationStatus.SUCCESS) { 520 521 count++; 522 } 523 } finally { 524 if (cursor != null) { 525 cursor.close(); 526 } 527 } 528 529 return count; 530 531 } 532} </pre> 533 <p> 534 This completes our in-memory transactional example. If you would like to 535 experiment with this code, you can find the example in the following 536 location in your DB distribution: 537</p> 538 <pre class="programlisting"><span class="emphasis"><em>DB_INSTALL</em></span>/examples_java/src/db/txn</pre> 539 </div> 540 <div class="navfooter"> 541 <hr /> 542 <table width="100%" summary="Navigation footer"> 543 <tr> 544 <td width="40%" align="left"><a accesskey="p" href="txnexample_dpl.html">Prev</a>��</td> 545 <td width="20%" align="center"> 546 <a accesskey="u" href="wrapup.html">Up</a> 547 </td> 548 <td width="40%" align="right">��</td> 549 </tr> 550 <tr> 551 <td width="40%" align="left" valign="top">DPL Transaction Example��</td> 552 <td width="20%" align="center"> 553 <a accesskey="h" href="index.html">Home</a> 554 </td> 555 <td width="40%" align="right" valign="top">��</td> 556 </tr> 557 </table> 558 </div> 559 </body> 560</html> 561