1<?xml version="1.0" encoding="UTF-8" standalone="no"?> 2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 3<html xmlns="http://www.w3.org/1999/xhtml"> 4 <head> 5 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> 6 <title>Example Processing Loop</title> 7 <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> 8 <meta name="generator" content="DocBook XSL Stylesheets V1.62.4" /> 9 <link rel="home" href="index.html" title="Getting Started with Replicated Berkeley DB Applications" /> 10 <link rel="up" href="fwrkmasterreplica.html" title="Chapter 4. Replica versus Master Processes" /> 11 <link rel="previous" href="processingloop.html" title="Processing Loop" /> 12 <link rel="next" href="addfeatures.html" title="Chapter 5. Additional Features" /> 13 </head> 14 <body> 15 <div class="navheader"> 16 <table width="100%" summary="Navigation header"> 17 <tr> 18 <th colspan="3" align="center">Example Processing Loop</th> 19 </tr> 20 <tr> 21 <td width="20%" align="left"><a accesskey="p" href="processingloop.html">Prev</a> </td> 22 <th width="60%" align="center">Chapter 4. Replica versus Master Processes</th> 23 <td width="20%" align="right"> <a accesskey="n" href="addfeatures.html">Next</a></td> 24 </tr> 25 </table> 26 <hr /> 27 </div> 28 <div class="sect1" lang="en" xml:lang="en"> 29 <div class="titlepage"> 30 <div> 31 <div> 32 <h2 class="title" style="clear: both"><a id="exampledoloop"></a>Example Processing Loop</h2> 33 </div> 34 </div> 35 <div></div> 36 </div> 37 <p> 38 In this section we take the example 39 processing loop that we presented in the 40 previous section and we flesh it out to 41 provide a more complete example. We do this 42 by updating the 43 <tt class="function">doloop()</tt> 44 function that our original transaction 45 application used 46 47 <span>(see <a href="simpleprogramlisting.html#doloop_cxx">Method: RepMgr::doloop()</a>)</span> 48 49 to fully support our replicated application. 50 </p> 51 <p> 52 In the following example code, code that we 53 add to the original example is presented in 54 <b class="userinput"><tt>bold</tt></b>. 55 </p> 56 <p> 57 To begin, we include a new header file into 58 our application so that we can check for the 59 <tt class="literal">ENOENT</tt> return value later 60 in our processing loop. We also define our 61 <tt class="literal">APP_DATA</tt> 62 structure, and we define a 63 <tt class="literal">sleeptime</tt> value. 64 65 66 67 <span> 68 Finally, we update <tt class="classname">RepMgr</tt> 69 to have a new method for our event notification 70 callback, and to add a new data member for our 71 <tt class="literal">APP_DATA</tt> data member. 72 </span> 73 </p> 74 <pre class="programlisting">#include <db_cxx.h> 75#include <iostream> 76<b class="userinput"><tt>#include <errno.h></tt></b> 77 78... 79// Skipping all the RepHostInfoObj and RepConfigInfo code, which does not 80// change. 81... 82 83using std::cout; 84using std::cin; 85using std::cerr; 86using std::endl; 87using std::flush; 88 89#define CACHESIZE (10 * 1024 * 1024) 90#define DATABASE "quote.db" 91<b class="userinput"><tt>#define SLEEPTIME 3</tt></b> 92 93const char *progname = "RepMgr"; 94 95<b class="userinput"><tt>// Struct used to store information in Db app_private field. 96typedef struct { 97 int is_master; 98} APP_DATA;</tt></b> 99 100class RepMgr 101{ 102public: 103 // Constructor. 104 RepMgr(); 105 // Initialization method. Creates and opens our environment handle. 106 int init(RepConfigInfo* config); 107 // The doloop is where all the work is performed. 108 int doloop(); 109 // terminate() provides our shutdown code. 110 int terminate(); 111 112 <b class="userinput"><tt>// event notification callback 113 static void 114 event_callback(DbEnv * dbenv, u_int32_t which, void *info);</tt></b> 115 116private: 117 // disable copy constructor. 118 RepMgr(const RepMgr &); 119 void operator = (const RepMgr &); 120 121 // internal data members. 122 <b class="userinput"><tt>APP_DATA app_data;</tt></b> 123 RepConfigInfo *app_config; 124 DbEnv dbenv; 125 126 // private methods. 127 // print_stocks() is used to display the contents of our database. 128 static int print_stocks(Db *dbp); 129}; </pre> 130 <p> 131 That done, we can skip the 132 133 <span><tt class="methodname">main()</tt> method, because it does not change.</span> 134 135 136 Instead, we skip down to our 137 <span><tt class="classname">RepMgr</tt> constructor where we initialize our 138 <tt class="literal">APP_DATA is_master</tt> data member:</span> 139 140 141</p> 142 <pre class="programlisting"> 143RepMgr::RepMgr() : app_config(0), dbenv(0) 144{ 145 <b class="userinput"><tt>app_data.is_master = 0; // assume I start out as client</tt></b> 146} 147</pre> 148 <p> 149 That done, we must also 150 update <tt class="methodname">RepMgr::init()</tt> to do a couple 151 of things. First, we need to register our event callback with 152 the environment handle. We also need to make our 153 <tt class="literal">APP_DATA</tt> data member available through our 154 environment handle's <tt class="methodname">app_private</tt> 155 field. This is a fairly trivial update, and it happens at the 156 top of the method (we skip the rest of the method's listing 157 since it does not change): 158 159 </p> 160 <pre class="programlisting">int RepMgr::init(RepConfigInfo *config) 161{ 162 int ret = 0; 163 164 app_config = config; 165 166 dbenv.set_errfile(stderr); 167 dbenv.set_errpfx(progname); 168 <b class="userinput"><tt>dbenv.set_app_private(&app_data); 169 dbenv.set_event_notify(event_callback);</tt></b> 170 171 ... </pre> 172 <p> 173 That done, we need to implement our 174 <tt class="function">event_callback()</tt> callback. Note that what we use 175 here is no different from the callback that we described in 176 the previous section. However, for the sake of completeness we 177 provide the implementation here again. 178</p> 179 <pre class="programlisting"> 180 <b class="userinput"> 181 <tt>/* 182 * A callback used to determine whether the local environment is a replica 183 * or a master. This is called by the replication framework 184 * when the local replication environment changes state. 185 */ 186void RepMgr::event_callback(DbEnv *dbenv, u_int32_t which, void *info) 187{ 188 APP_DATA *app = dbenv->get_app_private(); 189 190 info = NULL; /* Currently unused. */ 191 192 switch (which) { 193 case DB_EVENT_REP_MASTER: 194 app->is_master = 1; 195 break; 196 197 case DB_EVENT_REP_CLIENT: 198 app->is_master = 0; 199 break; 200 201 case DB_EVENT_REP_STARTUPDONE: /* fallthrough */ 202 case DB_EVENT_REP_NEWMASTER: 203 /* Ignore. */ 204 break; 205 206 default: 207 dbenv->errx(dbenv, "ignoring event %d", which); 208 } 209}</tt> 210 </b> 211 </pre> 212 <p> 213 That done, we need to update our 214 <tt class="function">doloop()</tt> 215 216 217 <span>method.</span> 218</p> 219 <p> 220 We begin by updating our database handle open flags to 221 determine which flags to use, depending on whether the 222 application is running as a master. 223 </p> 224 <pre class="programlisting">#define BUFSIZE 1024 225int RepMgr::doloop() 226{ 227 Db *dbp; 228 Dbt key, data; 229 char buf[BUFSIZE], *rbuf; 230 int ret; 231 232 dbp = NULL; 233 memset(&key, 0, sizeof(key)); 234 memset(&data, 0, sizeof(data)); 235 ret = 0; 236 237 for (;;) { 238 if (dbp == NULL) { 239 dbp = new Db(&dbenv, 0); 240 241 // Set page size small so page allocation is cheap. 242 if ((ret = dbp->set_pagesize(512)) != 0) 243 goto err; 244 245 try { 246 dbp->open(NULL, DATABASE, NULL, DB_BTREE, 247 <b class="userinput"><tt>app_data.is_master ? DB_CREATE | DB_AUTO_COMMIT : 248 DB_AUTO_COMMIT</tt></b>, 0); </pre> 249 <p> 250 When we open the database, we modify our error handling to 251 account for the case where the database does not yet exist. This can 252 happen if our code is running as a replica and the replication framework has not 253 yet had a chance to create the databases for us. Recall that replicas never 254 write to their own databases directly, and so they cannot 255 create databases on their own. 256</p> 257 <p> 258 If we detect that the database does not yet exist, we simply 259 close the database handle, sleep for a short period of time 260 and then continue processing. This gives the replication framework a chance to 261 create the database so that our replica can continue 262 operations. 263</p> 264 <pre class="programlisting"> } catch(DbException dbe) { 265 <b class="userinput"><tt>/* It is expected that this condition will be triggered 266 * when client sites start up. 267 * It can take a while for the master site to be found 268 * and synced, and no DB will be available until then. 269 */ 270 if (dbe.get_errno() == ENOENT) { 271 cout << "No stock db available yet - retrying." << endl; 272 try { 273 dbp->close(0); 274 } catch (DbException dbe2) { 275 cout << "Unexpected error closing after failed" 276 << " open, message: " << dbe2.what() << endl; 277 dbp = NULL; 278 goto err; 279 } 280 dbp = NULL; 281 sleep(SLEEPTIME); 282 continue; 283 } else {</tt></b> 284 dbenv.err(ret, "DB->open"); 285 throw dbe; 286 <b class="userinput"><tt>}</tt></b> 287 } 288 } </pre> 289 <p> 290 Next we modify our prompt, so that if the local process is running 291 as a replica, we can tell from the shell that the prompt is for a 292 read-only process. 293 </p> 294 <pre class="programlisting"> <b class="userinput"><tt>cout << "QUOTESERVER" ; 295 if (!app_data.is_master) 296 cout << "(read-only)"; 297 cout << "> " << flush; </tt></b></pre> 298 <p> 299 When we collect data from the prompt, there is a case that says 300 if no data is entered then show the entire stocks database. 301 This display is performed by our 302 <tt class="function">print_stocks()</tt> 303 304 <span>method</span> 305 (which has not 306 required a modification since we first introduced it in 307 <a href="simpleprogramlisting.html#printstocks_c"> 308 309 <span>Method: RepMgr::print_stocks()</span> 310 311 </a>). 312 </p> 313 <p> 314 When we call 315 <span><tt class="function">print_stocks()</tt>, </span> 316 317 we check for a dead replication handle. Dead 318 replication handles happen whenever a replication election 319 results in a previously committed transaction becoming 320 invalid. This is an error scenario caused by a new master having a 321 slightly older version of the data than the original 322 master and so all replicas must modify their database(s) to 323 reflect that of the new master. In this situation, some 324 number of previously committed transactions may have to be 325 unrolled. From the replica's perspective, the database 326 handles should all be closed and then opened again. 327 </p> 328 <pre class="programlisting"> 329 if (fgets(buf, sizeof(buf), stdin) == NULL) 330 break; 331 if (strtok(&buf[0], " \t\n") == NULL) { 332 switch ((ret = print_stocks(dbp))) { 333 case 0: 334 continue; 335 <b class="userinput"><tt>case DB_REP_HANDLE_DEAD: 336 (void)dbp->close(DB_NOSYNC); 337 cout << "closing db handle due to rep handle dead" << endl; 338 dbp = NULL; 339 continue;</tt></b> 340 default: 341 dbp->err(ret, "Error traversing data"); 342 goto err; 343 } 344 } 345 rbuf = strtok(NULL, " \t\n"); 346 if (rbuf == NULL || rbuf[0] == '\0') { 347 if (strncmp(buf, "exit", 4) == 0 || 348 strncmp(buf, "quit", 4) == 0) 349 break; 350 dbenv.errx("Format: TICKER VALUE"); 351 continue; 352 } </pre> 353 <p> 354 That done, we need to add a little error checking to our 355 command prompt to make sure the user is not attempting to 356 modify the database at a replica. Remember, replicas must never 357 modify their local databases on their own. This guards against 358 that happening due to user input at the prompt. 359 </p> 360 <pre class="programlisting"> <b class="userinput"><tt>if (!app_data.is_master) { 361 dbenv->errx(dbenv, "Can't update at client"); 362 continue; 363 }</tt></b> 364 365 key.set_data(buf); 366 key.set_size((u_int32_t)strlen(buf)); 367 368 data.set_data(rbuf); 369 data.set_size((u_int32_t)strlen(rbuf)); 370 371 if ((ret = dbp->put(NULL, &key, &data, 0)) != 0) 372 { 373 dbp->err(ret, "DB->put"); 374 if (ret != DB_KEYEXIST) 375 goto err; 376 } 377 } 378 379err: if (dbp != NULL) 380 (void)dbp->close(dbp, DB_NOSYNC); 381 382 return (ret); 383} </pre> 384 <p> 385 With that completed, we are all done updating our application 386 for replication. 387 388 The only remaining 389 390 <span>method, <tt class="function">print_stocks()</tt>,</span> 391 392 393 is unmodified from when we 394 originally introduced it. For details on that function, see 395 <a href="simpleprogramlisting.html#printstocks_c"> 396 397 <span>Method: RepMgr::print_stocks()</span> 398 399 </a>. 400</p> 401 <div class="sect2" lang="en" xml:lang="en"> 402 <div class="titlepage"> 403 <div> 404 <div> 405 <h3 class="title"><a id="runningit"></a>Running It</h3> 406 </div> 407 </div> 408 <div></div> 409 </div> 410 <p> 411 To run our replicated application, we need to make 412 sure each participating environment has its own unique 413 home directory. We can do this by running 414 each site on a separate networked machine, but that 415 is not strictly necessary; multiple instances of this 416 code can run on the same machine provided the 417 environment home restriction is observed. 418 </p> 419 <p> 420 To run a process, make sure the environment home 421 exists and then start the process using the 422 <tt class="literal">-h</tt> option to specify that 423 directory. You must also use the <tt class="literal">-m</tt> 424 option to identify the local host and port that this 425 process will use to listen for replication messages, and 426 the <tt class="literal">-o</tt> option to identify the other 427 processes in the replication group. Finally, use the 428 <tt class="literal">-p</tt> option to specify a priority. 429 The process that you designate to have the highest priority will become 430 the master. 431 </p> 432 <pre class="programlisting">> mkdir env1 433> /RepMgr -h env1 -n 2 -m localhost:8080 -o localhost:8081 -p 10 434No stock database yet available. 435No stock database yet available. </pre> 436 <p> 437 Now, start another process. This time, change the environment 438 home to something else, use the <tt class="literal">-m</tt> to at 439 least change the port number the process is listening on, and 440 use the <tt class="literal">-o</tt> option to identify the host and 441 port of the other replication process: 442</p> 443 <pre class="programlisting">> mkdir env2 444> /RepMgr -h env2 -n 2 -m localhost:8081 -o localhost:8080 -p 20</pre> 445 <p> 446 After a short pause, the second process should display the master 447 prompt: 448</p> 449 <pre class="programlisting"> 450QUOTESERVER > </pre> 451 <p> 452 And the first process should 453 display the read-only prompt: 454</p> 455 <pre class="programlisting"> 456QUOTESERVER (read-only)> </pre> 457 <p> 458 Now go to the master process and give it a couple of stocks and stock 459 prices: 460</p> 461 <pre class="programlisting">QUOTESERVER> FAKECO 9.87 462QUOTESERVER> NOINC .23 463QUOTESERVER> </pre> 464 <p> 465 Then, go to the replica and hit <b class="userinput"><tt>return</tt></b> at the prompt to 466 see the new values: 467</p> 468 <pre class="programlisting">QUOTESERVER (read-only)> 469 Symbol Price 470 ====== ===== 471 FAKECO 9.87 472 NOINC .23 473QUOTESERVER (read-only)> </pre> 474 <p> 475 Doing the same at the master results in the same thing: 476</p> 477 <pre class="programlisting">QUOTESERVER> 478 Symbol Price 479 ====== ===== 480 FAKECO 9.87 481 NOINC .23 482QUOTESERVER> </pre> 483 <p> 484 You can change a stock by simply entering the stock value and 485 new price at the master's prompt: 486</p> 487 <pre class="programlisting">QUOTESERVER> FAKECO 10.01 488QUOTESERVER> </pre> 489 <p> 490 Then, go to either the master or the replica to see the updated 491 database: 492</p> 493 <pre class="programlisting">QUOTESERVER> 494 Symbol Price 495 ====== ===== 496 FAKECO 10.01 497 NOINC .23 498QUOTESERVER> </pre> 499 <p> 500 And on the replica: 501</p> 502 <pre class="programlisting">QUOTESERVER (read-only)> 503 Symbol Price 504 ====== ===== 505 FAKECO 10.01 506 NOINC .23 507QUOTESERVER (read-only)> </pre> 508 <p> 509 Finally, to quit the applications, simply type 510 <tt class="literal">quit</tt> at both prompts: 511</p> 512 <pre class="programlisting">QUOTESERVER (read-only)> quit 513> </pre> 514 <p> 515 And on the master as well: 516</p> 517 <pre class="programlisting">QUOTESERVER> quit 518> </pre> 519 </div> 520 </div> 521 <div class="navfooter"> 522 <hr /> 523 <table width="100%" summary="Navigation footer"> 524 <tr> 525 <td width="40%" align="left"><a accesskey="p" href="processingloop.html">Prev</a> </td> 526 <td width="20%" align="center"> 527 <a accesskey="u" href="fwrkmasterreplica.html">Up</a> 528 </td> 529 <td width="40%" align="right"> <a accesskey="n" href="addfeatures.html">Next</a></td> 530 </tr> 531 <tr> 532 <td width="40%" align="left" valign="top">Processing Loop </td> 533 <td width="20%" align="center"> 534 <a accesskey="h" href="index.html">Home</a> 535 </td> 536 <td width="40%" align="right" valign="top"> Chapter 5. Additional Features</td> 537 </tr> 538 </table> 539 </div> 540 </body> 541</html> 542