1<?xml version="1.0" encoding="UTF-8" standalone="no"?> 2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 3<html xmlns="http://www.w3.org/1999/xhtml"> 4 <head> 5 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> 6 <title>Example Processing Loop</title> 7 <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> 8 <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> 9 <link rel="start" href="index.html" title="Getting Started with Replicated Berkeley DB Applications" /> 10 <link rel="up" href="fwrkmasterreplica.html" title="Chapter��4.��Replica versus Master Processes" /> 11 <link rel="prev" href="processingloop.html" title="Processing Loop" /> 12 <link rel="next" href="addfeatures.html" title="Chapter��5.��Additional Features" /> 13 </head> 14 <body> 15 <div class="navheader"> 16 <table width="100%" summary="Navigation header"> 17 <tr> 18 <th colspan="3" align="center">Example Processing Loop</th> 19 </tr> 20 <tr> 21 <td width="20%" align="left"><a accesskey="p" href="processingloop.html">Prev</a>��</td> 22 <th width="60%" align="center">Chapter��4.��Replica versus Master Processes</th> 23 <td width="20%" align="right">��<a accesskey="n" href="addfeatures.html">Next</a></td> 24 </tr> 25 </table> 26 <hr /> 27 </div> 28 <div class="sect1" lang="en" xml:lang="en"> 29 <div class="titlepage"> 30 <div> 31 <div> 32 <h2 class="title" style="clear: both"><a id="exampledoloop"></a>Example Processing Loop</h2> 33 </div> 34 </div> 35 </div> 36 <div class="toc"> 37 <dl> 38 <dt> 39 <span class="sect2"> 40 <a href="exampledoloop.html#runningit">Running It</a> 41 </span> 42 </dt> 43 </dl> 44 </div> 45 <p> 46 In this section we take the example 47 processing loop that we presented in the 48 previous section and we flesh it out to 49 provide a more complete example. We do this 50 by updating the 51 <code class="function">doloop()</code> 52 function that our original transaction 53 application used 54 55 <span>(see <a class="xref" href="simpleprogramlisting.html#doloop_cxx" title="Method: SimpleTxn::doloop()">Method: SimpleTxn::doloop()</a>)</span> 56 57 to fully support our replicated application. 58 </p> 59 <p> 60 In the following example code, code that we 61 add to the original example is presented in 62 <strong class="userinput"><code>bold</code></strong>. 63 </p> 64 <p> 65 To begin, we include a new header file into 66 our application so that we can check for the 67 <code class="literal">ENOENT</code> return value later 68 in our processing loop. We also define our 69 <code class="literal">APP_DATA</code> 70 structure, and we define a 71 <code class="literal">sleeptime</code> value. 72 73 74 75 <span> 76 Finally, we update <code class="classname">RepMgrGSG</code> 77 to have a new method for our event notification 78 callback, and to add a new data member for our 79 <code class="literal">APP_DATA</code> data member. 80 </span> 81 </p> 82 <pre class="programlisting">#include <db_cxx.h> 83#include <iostream> 84<strong class="userinput"><code>#include <errno.h></code></strong> 85 86... 87// Skipping all the RepHostInfoObj and RepConfigInfo code, which does not 88// change. 89... 90 91using std::cout; 92using std::cin; 93using std::cerr; 94using std::endl; 95using std::flush; 96 97#define CACHESIZE (10 * 1024 * 1024) 98#define DATABASE "quote.db" 99<strong class="userinput"><code>#define SLEEPTIME 3</code></strong> 100 101const char *progname = "RepMgrGSG"; 102 103<strong class="userinput"><code>// Struct used to store information in Db app_private field. 104typedef struct { 105 int is_master; 106} APP_DATA;</code></strong> 107 108class RepMgrGSG 109{ 110public: 111 // Constructor. 112 RepMgrGSG(); 113 // Initialization method. Creates and opens our environment handle. 114 int init(RepConfigInfo* config); 115 // The doloop is where all the work is performed. 116 int doloop(); 117 // terminate() provides our shutdown code. 118 int terminate(); 119 120 <strong class="userinput"><code>// event notification callback 121 static void 122 event_callback(DbEnv * dbenv, u_int32_t which, void *info);</code></strong> 123 124private: 125 // disable copy constructor. 126 RepMgrGSG(const RepMgrGSG &); 127 void operator = (const RepMgrGSG &); 128 129 // internal data members. 130 <strong class="userinput"><code>APP_DATA app_data;</code></strong> 131 RepConfigInfo *app_config; 132 DbEnv dbenv; 133 134 // private methods. 135 // print_stocks() is used to display the contents of our database. 136 static int print_stocks(Db *dbp); 137}; </pre> 138 <p> 139 That done, we can skip the 140 141 <span><code class="methodname">main()</code> method, because it does not change.</span> 142 143 144 Instead, we skip down to our 145 <span><code class="classname">RepMgrGSG</code> constructor where we initialize our 146 <code class="literal">APP_DATA is_master</code> data member:</span> 147 148 149</p> 150 <pre class="programlisting"> 151RepMgrGSG::RepMgrGSG() : app_config(0), dbenv(0) 152{ 153 <strong class="userinput"><code>app_data.is_master = 0; // assume I start out as client</code></strong> 154} 155</pre> 156 <p> 157 That done, we must also 158 update <code class="methodname">RepMgrGSG::init()</code> to do a couple 159 of things. First, we need to register our event callback with 160 the environment handle. We also need to make our 161 <code class="literal">APP_DATA</code> data member available through our 162 environment handle's <code class="methodname">app_private</code> 163 field. This is a fairly trivial update, and it happens at the 164 top of the method (we skip the rest of the method's listing 165 since it does not change): 166 167 </p> 168 <pre class="programlisting">int RepMgrGSG::init(RepConfigInfo *config) 169{ 170 int ret = 0; 171 172 app_config = config; 173 174 dbenv.set_errfile(stderr); 175 dbenv.set_errpfx(progname); 176 <strong class="userinput"><code>dbenv.set_app_private(&app_data); 177 dbenv.set_event_notify(event_callback);</code></strong> 178 179 ... </pre> 180 <p> 181 That done, we need to implement our 182 <code class="function">event_callback()</code> callback. Note that what we use 183 here is no different from the callback that we described in 184 the previous section. However, for the sake of completeness we 185 provide the implementation here again. 186</p> 187 <pre class="programlisting"> 188 <strong class="userinput"> 189 <code>/* 190 * A callback used to determine whether the local environment is a 191 * replica or a master. This is called by the Replication Manager 192 * when the local replication environment changes state. 193 */ 194void RepMgrGSG::event_callback(DbEnv *dbenv, u_int32_t which, void *info) 195{ 196 APP_DATA *app = dbenv->get_app_private(); 197 198 info = NULL; /* Currently unused. */ 199 200 switch (which) { 201 case DB_EVENT_REP_MASTER: 202 app->is_master = 1; 203 break; 204 205 case DB_EVENT_REP_CLIENT: 206 app->is_master = 0; 207 break; 208 209 case DB_EVENT_REP_STARTUPDONE: /* fallthrough */ 210 case DB_EVENT_REP_NEWMASTER: 211 /* Ignore. */ 212 break; 213 214 default: 215 dbenv->errx(dbenv, "ignoring event %d", which); 216 } 217}</code> 218 </strong> 219 </pre> 220 <p> 221 That done, we need to update our 222 <code class="function">doloop()</code> 223 224 225 <span>method.</span> 226</p> 227 <p> 228 We begin by updating our database handle open flags to 229 determine which flags to use, depending on whether the 230 application is running as a master. 231 </p> 232 <pre class="programlisting">#define BUFSIZE 1024 233int RepMgrGSG::doloop() 234{ 235 Db *dbp; 236 Dbt key, data; 237 char buf[BUFSIZE], *rbuf; 238 int ret; 239 240 dbp = 0; 241 memset(&key, 0, sizeof(key)); 242 memset(&data, 0, sizeof(data)); 243 ret = 0; 244 245 for (;;) { 246 if (dbp == 0) { 247 dbp = new Db(&dbenv, 0); 248 249 try { 250 dbp->open(NULL, DATABASE, NULL, DB_BTREE, 251 <strong class="userinput"><code>app_data.is_master ? DB_CREATE | DB_AUTO_COMMIT : 252 DB_AUTO_COMMIT</code></strong>, 0); </pre> 253 <p> 254 When we open the database, we modify our error handling to 255 account for the case where the database does not yet exist. This can 256 happen if our code is running as a replica and the Replication Manager has not 257 yet had a chance to create the databases for us. Recall that replicas never 258 write to their own databases directly, and so they cannot 259 create databases on their own. 260</p> 261 <p> 262 If we detect that the database does not yet exist, we simply 263 close the database handle, sleep for a short period of time 264 and then continue processing. This gives the Replication Manager a chance to 265 create the database so that our replica can continue 266 operations. 267</p> 268 <pre class="programlisting"> } catch(DbException dbe) { 269 <strong class="userinput"><code>/* It is expected that this condition will be triggered 270 * when client sites start up. 271 * It can take a while for the master site to be found 272 * and synced, and no DB will be available until then. 273 */ 274 if (dbe.get_errno() == ENOENT) { 275 cout << "No stock db available yet - " 276 << "retrying." << endl; 277 try { 278 dbp->close(0); 279 } catch (DbException dbe2) { 280 cout << "Unexpected error closing after failed" 281 << " open, message: " << dbe2.what() << endl; 282 dbp = NULL; 283 goto err; 284 } 285 dbp = NULL; 286 sleep(SLEEPTIME); 287 continue; 288 } else {</code></strong> 289 dbenv.err(ret, "DB->open"); 290 throw dbe; 291 <strong class="userinput"><code>}</code></strong> 292 } 293 } </pre> 294 <p> 295 Next we modify our prompt, so that if the local process is running 296 as a replica, we can tell from the shell that the prompt is for a 297 read-only process. 298 </p> 299 <pre class="programlisting"> <strong class="userinput"><code>cout << "QUOTESERVER" ; 300 if (!app_data.is_master) 301 cout << "(read-only)"; 302 cout << "> " << flush; </code></strong></pre> 303 <p> 304 When we collect data from the prompt, there is a case that says 305 if no data is entered then show the entire stocks database. 306 This display is performed by our 307 <code class="function">print_stocks()</code> 308 309 <span>method</span> 310 (which has not 311 required a modification since we first introduced it in 312 <a class="xref" href="simpleprogramlisting.html#printstocks_c" title="Method: SimpleTxn::print_stocks()"> 313 314 <span>Method: SimpleTxn::print_stocks()</span> 315 316 </a>). 317 </p> 318 <p> 319 When we call 320 <span><code class="function">print_stocks()</code>, </span> 321 322 we check for a dead replication handle. Dead 323 replication handles happen whenever a replication election 324 results in a previously committed transaction becoming 325 invalid. This is an error scenario caused by a new master having a 326 slightly older version of the data than the original 327 master and so all replicas must modify their database(s) to 328 reflect that of the new master. In this situation, some 329 number of previously committed transactions may have to be 330 unrolled. From the replica's perspective, the database 331 handles should all be closed and then opened again. 332 </p> 333 <pre class="programlisting"> 334 if (fgets(buf, sizeof(buf), stdin) == NULL) 335 break; 336 if (strtok(&buf[0], " \t\n") == NULL) { 337 switch ((ret = print_stocks(dbp))) { 338 case 0: 339 continue; 340 <strong class="userinput"><code>case DB_REP_HANDLE_DEAD: 341 (void)dbp->close(DB_NOSYNC); 342 cout << "closing db handle due to rep handle dead" << endl; 343 dbp = NULL; 344 continue;</code></strong> 345 default: 346 dbp->err(ret, "Error traversing data"); 347 goto err; 348 } 349 } 350 rbuf = strtok(NULL, " \t\n"); 351 if (rbuf == NULL || rbuf[0] == '\0') { 352 if (strncmp(buf, "exit", 4) == 0 || 353 strncmp(buf, "quit", 4) == 0) 354 break; 355 dbenv.errx("Format: TICKER VALUE"); 356 continue; 357 } </pre> 358 <p> 359 That done, we need to add a little error checking to our 360 command prompt to make sure the user is not attempting to 361 modify the database at a replica. Remember, replicas must never 362 modify their local databases on their own. This guards against 363 that happening due to user input at the prompt. 364 </p> 365 <pre class="programlisting"> <strong class="userinput"><code>if (!app_data.is_master) { 366 dbenv->errx(dbenv, "Can't update at client"); 367 continue; 368 }</code></strong> 369 370 key.set_data(buf); 371 key.set_size((u_int32_t)strlen(buf)); 372 373 data.set_data(rbuf); 374 data.set_size((u_int32_t)strlen(rbuf)); 375 376 if ((ret = dbp->put(NULL, &key, &data, 0)) != 0) 377 { 378 dbp->err(ret, "DB->put"); 379 if (ret != DB_KEYEXIST) 380 goto err; 381 } 382 } 383 384err: if (dbp != 0) 385 (void)dbp->close(dbp, DB_NOSYNC); 386 387 return (ret); 388} </pre> 389 <p> 390 With that completed, we are all done updating our application 391 for replication. 392 393 The only remaining 394 395 <span>method, <code class="function">print_stocks()</code>,</span> 396 397 398 is unmodified from when we 399 originally introduced it. For details on that function, see 400 <a class="xref" href="simpleprogramlisting.html#printstocks_c" title="Method: SimpleTxn::print_stocks()"> 401 402 <span>Method: SimpleTxn::print_stocks()</span> 403 404 </a>. 405</p> 406 <div class="sect2" lang="en" xml:lang="en"> 407 <div class="titlepage"> 408 <div> 409 <div> 410 <h3 class="title"><a id="runningit"></a>Running It</h3> 411 </div> 412 </div> 413 </div> 414 <p> 415 To run our replicated application, we need to make 416 sure each participating environment has its own unique 417 home directory. We can do this by running 418 each site on a separate networked machine, but that 419 is not strictly necessary; multiple instances of this 420 code can run on the same machine provided the 421 environment home restriction is observed. 422 </p> 423 <p> 424 To run a process, make sure the environment home 425 exists and then start the process using the 426 <code class="literal">-h</code> option to specify that 427 directory. You must also use the <code class="literal">-l</code> 428 option to identify the local host and port that this 429 process will use to listen for replication messages, the 430 <code class="literal">-n</code> option to specify the number of sites 431 in the replication group, and 432 the <code class="literal">-r</code> option to identify the other 433 processes in the replication group. Finally, use the 434 <code class="literal">-p</code> option to specify a priority. 435 The process that you designate to have the highest priority will become 436 the master. 437 </p> 438 <pre class="programlisting">> mkdir env1 439> ./RepMgrGSG -h env1 -n 2 -l localhost:8080 -r localhost:8081 -p 10 440No stock database yet available. 441No stock database yet available. </pre> 442 <p> 443 Now, start another process. This time, change the environment 444 home to something else, use the <code class="literal">-l</code> flag to at 445 least change the port number the process is listening on, and 446 use the <code class="literal">-r</code> option to identify the host and 447 port of the other replication process: 448</p> 449 <pre class="programlisting">> mkdir env2 450> ./RepMgrGSG -h env2 -n 2 -l localhost:8081 -r localhost:8080 -p 20</pre> 451 <p> 452 After a short pause, the second process should display the master 453 prompt: 454</p> 455 <pre class="programlisting"> 456QUOTESERVER > </pre> 457 <p> 458 And the first process should 459 display the read-only prompt: 460</p> 461 <pre class="programlisting"> 462QUOTESERVER (read-only)> </pre> 463 <p> 464 Now go to the master process and give it a couple of stocks and stock 465 prices: 466</p> 467 <pre class="programlisting">QUOTESERVER> FAKECO 9.87 468QUOTESERVER> NOINC .23 469QUOTESERVER> </pre> 470 <p> 471 Then, go to the replica and hit <strong class="userinput"><code>return</code></strong> at the prompt to 472 see the new values: 473</p> 474 <pre class="programlisting">QUOTESERVER (read-only)> 475 Symbol Price 476 ====== ===== 477 FAKECO 9.87 478 NOINC .23 479QUOTESERVER (read-only)> </pre> 480 <p> 481 Doing the same at the master results in the same thing: 482</p> 483 <pre class="programlisting">QUOTESERVER> 484 Symbol Price 485 ====== ===== 486 FAKECO 9.87 487 NOINC .23 488QUOTESERVER> </pre> 489 <p> 490 You can change a stock by simply entering the stock value and 491 new price at the master's prompt: 492</p> 493 <pre class="programlisting">QUOTESERVER> FAKECO 10.01 494QUOTESERVER> </pre> 495 <p> 496 Then, go to either the master or the replica to see the updated 497 database. On the master: 498</p> 499 <pre class="programlisting">QUOTESERVER> 500 Symbol Price 501 ====== ===== 502 FAKECO 10.01 503 NOINC .23 504QUOTESERVER> </pre> 505 <p> 506 And on the replica: 507</p> 508 <pre class="programlisting">QUOTESERVER (read-only)> 509 Symbol Price 510 ====== ===== 511 FAKECO 10.01 512 NOINC .23 513QUOTESERVER (read-only)> </pre> 514 <p> 515 Finally, to quit the applications, simply type 516 <code class="literal">quit</code> at both prompts. On the replica: 517</p> 518 <pre class="programlisting">QUOTESERVER (read-only)> quit 519> </pre> 520 <p> 521 And on the master as well: 522</p> 523 <pre class="programlisting">QUOTESERVER> quit 524> </pre> 525 </div> 526 </div> 527 <div class="navfooter"> 528 <hr /> 529 <table width="100%" summary="Navigation footer"> 530 <tr> 531 <td width="40%" align="left"><a accesskey="p" href="processingloop.html">Prev</a>��</td> 532 <td width="20%" align="center"> 533 <a accesskey="u" href="fwrkmasterreplica.html">Up</a> 534 </td> 535 <td width="40%" align="right">��<a accesskey="n" href="addfeatures.html">Next</a></td> 536 </tr> 537 <tr> 538 <td width="40%" align="left" valign="top">Processing Loop��</td> 539 <td width="20%" align="center"> 540 <a accesskey="h" href="index.html">Home</a> 541 </td> 542 <td width="40%" align="right" valign="top">��Chapter��5.��Additional Features</td> 543 </tr> 544 </table> 545 </div> 546 </body> 547</html> 548