1<?xml version="1.0" encoding="UTF-8" standalone="no"?> 2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 3<html xmlns="http://www.w3.org/1999/xhtml"> 4 <head> 5 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> 6 <title>Example Processing Loop</title> 7 <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> 8 <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> 9 <link rel="start" href="index.html" title="Getting Started with Replicated Berkeley DB Applications" /> 10 <link rel="up" href="fwrkmasterreplica.html" title="Chapter 4. Replica versus Master Processes" /> 11 <link rel="prev" href="processingloop.html" title="Processing Loop" /> 12 <link rel="next" href="addfeatures.html" title="Chapter 5. Additional Features" /> 13 </head> 14 <body> 15 <div class="navheader"> 16 <table width="100%" summary="Navigation header"> 17 <tr> 18 <th colspan="3" align="center">Example Processing Loop</th> 19 </tr> 20 <tr> 21 <td width="20%" align="left"><a accesskey="p" href="processingloop.html">Prev</a> </td> 22 <th width="60%" align="center">Chapter 4. Replica versus Master Processes</th> 23 <td width="20%" align="right"> <a accesskey="n" href="addfeatures.html">Next</a></td> 24 </tr> 25 </table> 26 <hr /> 27 </div> 28 <div class="sect1" lang="en" xml:lang="en"> 29 <div class="titlepage"> 30 <div> 31 <div> 32 <h2 class="title" style="clear: both"><a id="exampledoloop"></a>Example Processing Loop</h2> 33 </div> 34 </div> 35 </div> 36 <div class="toc"> 37 <dl> 38 <dt> 39 <span class="sect2"> 40 <a href="exampledoloop.html#runningit">Running It</a> 41 </span> 42 </dt> 43 </dl> 44 </div> 45 <p> 46 In this section we take the example 47 processing loop that we presented in the 48 previous section and we flesh it out to 49 provide a more complete example. We do this 50 by updating the 51 <code class="function">doloop()</code> 52 function that our original transaction 53 application used 54 <span>(see <a class="xref" href="simpleprogramlisting.html#doloop_c" title="Function: doloop()">Function: doloop()</a>)</span> 55 56 57 to fully support our replicated application. 58 </p> 59 <p> 60 In the following example code, code that we 61 add to the original example is presented in 62 <strong class="userinput"><code>bold</code></strong>. 63 </p> 64 <p> 65 To begin, we include a new header file into 66 our application so that we can check for the 67 <code class="literal">ENOENT</code> return value later 68 in our processing loop. We also define our 69 <code class="literal">APP_DATA</code> 70 structure, and we define a 71 <code class="literal">sleeptime</code> value. 72 73 <span> 74 Finally, we 75 add a new forward declaration for our event 76 callback. 77 </span> 78 79 80 </p> 81 <pre class="programlisting">/* 82 * File: ex_rep_gsg_repmgr.c 83 */ 84 85#include <stdlib.h> 86#include <string.h> 87<strong class="userinput"><code>#include <errno.h></code></strong> 88#ifndef _WIN32 89#include <unistd.h> 90#endif 91 92#include <db.h> 93 94#ifdef _WIN32 95extern int getopt(int, char * const *, const char *); 96#endif 97 98 99#define CACHESIZE (10 * 1024 * 1024) 100#define DATABASE "quote.db" 101<strong class="userinput"><code>#define SLEEPTIME 3</code></strong> 102 103const char *progname = "ex_rep_gsg_repmgr"; 104 105<strong class="userinput"><code>typedef struct { 106 int is_master; 107} APP_DATA;</code></strong> 108 109int create_env(const char *, DB_ENV **); 110int env_init(DB_ENV *, const char *); 111int doloop (DB_ENV *); 112static int print_stocks(DBC *); 113<strong class="userinput"><code>void *event_callback(DB_ENV *, u_int32_t, void *);</code></strong> </pre> 114 <p> 115 In our <code class="function">main()</code> function, most of what we 116 have to add to it is some new variable declarations and 117 initializations: 118</p> 119 <pre class="programlisting">int 120main(int argc, char *argv[]) 121{ 122 extern char *optarg; 123 DB_ENV *dbenv; 124 const char *home; 125 char ch, *host, *portstr; 126 int ret, local_is_set, totalsites; 127 u_int32_t port; 128 <strong class="userinput"><code>/* Used to track whether this is a replica or a master */ 129 APP_DATA my_app_data; 130 131 my_app_data.is_master = 0; /* Assume that we start as a replica */</code></strong> 132 dbenv = NULL; 133 134 ret = local_is_set = totalsites = 0; 135 home = NULL;</pre> 136 <p> 137 The rest of our <code class="function">main()</code> function 138 is unchanged, except that we make our 139 <code class="literal">APP_DATA</code> structure available through our 140 environment handle's <code class="methodname">app_private</code> 141 field: 142 </p> 143 <pre class="programlisting"> 144 if ((ret = create_env(progname, &dbenv)) != 0) 145 goto err; 146 147 <strong class="userinput"><code>/* Make APP_DATA available through the environment handle */ 148 dbenv->app_private = &my_app_data;</code></strong> 149 150 /* Default priority is 100 */ 151 dbenv->rep_set_priority(dbenv, 100); 152 /* Permanent messages require at least one ack */ 153 dbenv->repmgr_set_ack_policy(dbenv, DB_REPMGR_ACKS_ONE); 154 /* Give 500 microseconds to receive the ack */ 155 dbenv->rep_set_timeout(dbenv, DB_REP_ACK_TIMEOUT, 500); 156 157 /* Collect the command line options */ 158 while ((ch = getopt(argc, argv, "h:l:m:p:r:")) != EOF) 159 switch (ch) { 160 case 'h': 161 home = optarg; 162 break; 163 /* Set the host and port used by this environment */ 164 case 'l': 165 host = strtok(optarg, ":"); 166 if ((portstr = strtok(NULL, ":")) == NULL) { 167 fprintf(stderr, "Bad host specification.\n"); 168 goto err; 169 } 170 port = (unsigned short)atoi(portstr); 171 if (dbenv->repmgr_set_local_site(dbenv, host, port, 0) 172 != 0) { 173 fprintf(stderr, 174 "Could not set local address %s.\n", host); 175 goto err; 176 } 177 local_is_set = 1; 178 break; 179 /* Set the number of sites in this replication group */ 180 case 'n': 181 totalsites = atoi(optarg); 182 if ((ret = dbenv->rep_set_nsites(dbenv, totalsites)) 183 != 0) 184 dbenv->err(dbenv, ret, "set_nsites"); 185 break; 186 /* Set this replica's election priority */ 187 case 'p': 188 dbenv->rep_set_priority(dbenv, atoi(optarg)); 189 break; 190 /* Identify another site in the replication group */ 191 case 'r': 192 host = strtok(optarg, ":"); 193 if ((portstr = strtok(NULL, ":")) == NULL) { 194 fprintf(stderr, "Bad host specification.\n"); 195 goto err; 196 } 197 port = (unsigned short)atoi(portstr); 198 if (dbenv->repmgr_add_remote_site(dbenv, host, port, 199 NULL, 0) != 0) { 200 fprintf(stderr, 201 "Could not add site %s.\n", host); 202 goto err; 203 } 204 break; 205 case '?': 206 default: 207 usage(); 208 } 209 210 /* Error check command line. */ 211 if (home == NULL || !local_is_set || !totalsites) 212 usage(); 213 214 if ((ret = env_init(dbenv, home)) != 0) 215 goto err; 216 217 if ((ret = dbenv->repmgr_start(dbenv, 3, DB_REP_ELECTION)) != 0) 218 goto err; 219 220 /* Sleep to give ourselves time to find a master. */ 221 sleep(5); 222 223 if ((ret = doloop(dbenv)) != 0) { 224 dbenv->err(dbenv, ret, "Application failed"); 225 goto err; 226 } 227 228err: if (dbenv != NULL) 229 (void)dbenv->close(dbenv, 0); 230 231 return (ret); 232} </pre> 233 <p> 234 Having updated our <code class="function">main()</code>, we must also 235 update our <code class="function">create_env()</code> function to 236 register our <code class="function">event_callback</code> callback. 237 Notice that our <code class="function">env_init()</code> function, which is 238 responsible for actually opening our environment handle, is 239 unchanged: 240 </p> 241 <pre class="programlisting">int 242create_env(const char *progname, DB_ENV **dbenvp) 243{ 244 DB_ENV *dbenv; 245 int ret; 246 247 if ((ret = db_env_create(&dbenv, 0)) != 0) { 248 fprintf(stderr, "can't create env handle: %s\n", 249 db_strerror(ret)); 250 return (ret); 251 } 252 253 dbenv->set_errfile(dbenv, stderr); 254 dbenv->set_errpfx(dbenv, progname); 255 <strong class="userinput"><code>(void)dbenv->set_event_notify(dbenv, event_callback);</code></strong> 256 257 *dbenvp = dbenv; 258 return (0); 259} 260 261int 262env_init(DB_ENV *dbenv, const char *home) 263{ 264 u_int32_t flags; 265 int ret; 266 267 (void)dbenv->set_cachesize(dbenv, 0, CACHESIZE, 0); 268 (void)dbenv->set_flags(dbenv, DB_TXN_NOSYNC, 1); 269 270 flags = DB_CREATE | 271 DB_INIT_LOCK | 272 DB_INIT_LOG | 273 DB_INIT_MPOOL | 274 DB_INIT_REP | 275 DB_INIT_TXN | 276 DB_RECOVER | 277 DB_THREAD; 278 if ((ret = dbenv->open(dbenv, home, flags, 0)) != 0) 279 dbenv->err(dbenv, ret, "can't open environment"); 280 return (ret); 281}</pre> 282 <p> 283 That done, we need to implement our 284 <code class="function">event_callback()</code> callback. Note that what we use 285 here is no different from the callback that we described in 286 the previous section. However, for the sake of completeness we 287 provide the implementation here again. 288</p> 289 <pre class="programlisting"> 290 <strong class="userinput"> 291 <code>/* 292 * A callback used to determine whether the local environment is a 293 * replica or a master. This is called by the Replication Manager 294 * when the local replication environment changes state. 295 */ 296void * 297event_callback(DB_ENV *dbenv, u_int32_t which, void *info) 298{ 299 APP_DATA *app = dbenv->app_private; 300 301 info = NULL; /* Currently unused. */ 302 303 switch (which) { 304 case DB_EVENT_REP_MASTER: 305 app->is_master = 1; 306 break; 307 308 case DB_EVENT_REP_CLIENT: 309 app->is_master = 0; 310 break; 311 312 case DB_EVENT_REP_STARTUPDONE: /* fallthrough */ 313 case DB_EVENT_REP_NEWMASTER: 314 /* Ignore. */ 315 break; 316 317 default: 318 dbenv->errx(dbenv, "ignoring event %d", which); 319 } 320}</code> 321 </strong> 322 </pre> 323 <p> 324 That done, we need to update our 325 <code class="function">doloop()</code> 326 <span>function. This is the place where we most 327 heavily modify our application.</span> 328 329 330</p> 331 <p> 332 We begin by introducing <code class="literal">APP_DATA</code> to the 333 function: 334</p> 335 <pre class="programlisting">/* 336 * Provides the main data processing function for our application. 337 * This function provides a command line prompt to which the user 338 * can provide a ticker string and a stock price. Once a value is 339 * entered to the application, the application writes the value to 340 * the database and then displays the entire database. 341 */ 342#define BUFSIZE 1024 343int 344doloop(DB_ENV *dbenv) 345{ 346 DB *dbp; 347 <strong class="userinput"><code>APP_DATA *app_data;</code></strong> 348 DBT key, data; 349 char buf[BUFSIZE], *rbuf; 350 int ret; 351 u_int32_t flags; 352 353 dbp = NULL; 354 ret = 0; 355 memset(&key, 0, sizeof(key)); 356 memset(&data, 0, sizeof(data)); 357 <strong class="userinput"><code>app_data = dbenv->app_private;</code></strong></pre> 358 <p> 359 Next we begin to modify our main loop. 360 To start, upon entering the loop we create the database handle and configure it as 361 normal. But we also have to decide what flags we will use for 362 the open. Again, it depends on whether we are a replica or a 363 master. 364 </p> 365 <pre class="programlisting"> for (;;) { 366 if (dbp == NULL) { 367 if ((ret = db_create(&dbp, dbenv, 0)) != 0) 368 return (ret); 369 370 <strong class="userinput"><code>flags = DB_AUTO_COMMIT; 371 if (app_data->is_master) 372 flags |= DB_CREATE;</code></strong> </pre> 373 <p> 374 When we open the database, we modify our error handling to 375 account for the case where the database does not yet exist. This can 376 happen if our code is running as a replica and the Replication Manager has not 377 yet had a chance to create the databases for us. Recall that replicas never 378 write to their own databases directly, and so they cannot 379 create databases on their own. 380</p> 381 <p> 382 If we detect that the database does not yet exist, we simply 383 close the database handle, sleep for a short period of time 384 and then continue processing. This gives the Replication Manager a chance to 385 create the database so that our replica can continue 386 operations. 387</p> 388 <pre class="programlisting"> if ((ret = dbp->open(dbp, 389 NULL, DATABASE, NULL, DB_BTREE, flags, 0)) != 0) { 390 <strong class="userinput"><code>if (ret == ENOENT) { 391 printf( 392 "No stock database yet available.\n"); 393 if ((ret = dbp->close(dbp, 0)) != 0) { 394 dbenv->err(dbenv, ret, 395 "DB->close"); 396 goto err; 397 } 398 dbp = NULL; 399 sleep(SLEEPTIME); 400 continue; 401 }</code></strong> 402 dbenv->err(dbenv, ret, "DB->open"); 403 goto err; 404 } 405 } </pre> 406 <p> 407 Next we modify our prompt, so that if the local process is running 408 as a replica, we can tell from the shell that the prompt is for a 409 read-only process. 410 </p> 411 <pre class="programlisting"> printf("QUOTESERVER<strong class="userinput"><code>%s</code></strong>> ", 412 <strong class="userinput"><code>app_data->is_master ? "" : " (read-only)");</code></strong> 413 fflush(stdout); </pre> 414 <p> 415 When we collect data from the prompt, there is a case that says 416 if no data is entered then show the entire stocks database. 417 This display is performed by our 418 <code class="function">print_stocks()</code> 419 <span>function</span> 420 421 (which has not 422 required a modification since we first introduced it in 423 <a class="xref" href="simpleprogramlisting.html#printstocks_c" title="Function: print_stocks()"> 424 <span>Function: print_stocks()</span> 425 426 427 </a>). 428 </p> 429 <p> 430 When we call 431 <span><code class="function">print_stocks()</code>, </span> 432 433 we check for a dead replication handle. Dead 434 replication handles happen whenever a replication election 435 results in a previously committed transaction becoming 436 invalid. This is an error scenario caused by a new master having a 437 slightly older version of the data than the original 438 master and so all replicas must modify their database(s) to 439 reflect that of the new master. In this situation, some 440 number of previously committed transactions may have to be 441 unrolled. From the replica's perspective, the database 442 handles should all be closed and then opened again. 443 </p> 444 <pre class="programlisting"> if (fgets(buf, sizeof(buf), stdin) == NULL) 445 break; 446 if (strtok(&buf[0], " \t\n") == NULL) { 447 switch ((ret = print_stocks(dbp))) { 448 case 0: 449 continue; 450 <strong class="userinput"><code>case DB_REP_HANDLE_DEAD: 451 (void)dbp->close(dbp, DB_NOSYNC); 452 dbp = NULL; 453 dbenv->errx(dbenv, "Got a dead replication handle"); 454 continue; </code></strong> 455 default: 456 dbp->err(dbp, ret, "Error traversing data"); 457 goto err; 458 } 459 } 460 rbuf = strtok(NULL, " \t\n"); 461 if (rbuf == NULL || rbuf[0] == '\0') { 462 if (strncmp(buf, "exit", 4) == 0 || 463 strncmp(buf, "quit", 4) == 0) 464 break; 465 dbenv->errx(dbenv, "Format: TICKER VALUE"); 466 continue; 467 }</pre> 468 <p> 469 That done, we need to add a little error checking to our 470 command prompt to make sure the user is not attempting to 471 modify the database at a replica. Remember, replicas must never 472 modify their local databases on their own. This guards against 473 that happening due to user input at the prompt. 474 </p> 475 <pre class="programlisting"> <strong class="userinput"><code>if (!app_data->is_master) { 476 dbenv->errx(dbenv, "Can't update at client"); 477 continue; 478 }</code></strong> 479 key.data = buf; 480 key.size = (u_int32_t)strlen(buf); 481 482 data.data = rbuf; 483 data.size = (u_int32_t)strlen(rbuf); 484 485 if ((ret = dbp->put(dbp, 486 NULL, &key, &data, 0)) != 0) { 487 dbp->err(dbp, ret, "DB->put"); 488 goto err; 489 } 490 } 491 492err: if (dbp != NULL) 493 (void)dbp->close(dbp, DB_NOSYNC); 494 495 return (ret); 496} </pre> 497 <p> 498 With that completed, we are all done updating our application 499 for replication. 500 501 The only remaining 502 <span>function, <code class="function">print_stocks()</code>,</span> 503 504 505 506 is unmodified from when we 507 originally introduced it. For details on that function, see 508 <a class="xref" href="simpleprogramlisting.html#printstocks_c" title="Function: print_stocks()"> 509 <span>Function: print_stocks()</span> 510 511 512 </a>. 513</p> 514 <div class="sect2" lang="en" xml:lang="en"> 515 <div class="titlepage"> 516 <div> 517 <div> 518 <h3 class="title"><a id="runningit"></a>Running It</h3> 519 </div> 520 </div> 521 </div> 522 <p> 523 To run our replicated application, we need to make 524 sure each participating environment has its own unique 525 home directory. We can do this by running 526 each site on a separate networked machine, but that 527 is not strictly necessary; multiple instances of this 528 code can run on the same machine provided the 529 environment home restriction is observed. 530 </p> 531 <p> 532 To run a process, make sure the environment home 533 exists and then start the process using the 534 <code class="literal">-h</code> option to specify that 535 directory. You must also use the <code class="literal">-l</code> 536 option to identify the local host and port that this 537 process will use to listen for replication messages, the 538 <code class="literal">-n</code> option to specify the number of sites 539 in the replication group, and 540 the <code class="literal">-r</code> option to identify the other 541 processes in the replication group. Finally, use the 542 <code class="literal">-p</code> option to specify a priority. 543 The process that you designate to have the highest priority will become 544 the master. 545 </p> 546 <pre class="programlisting">> mkdir env1 547> ./ex_rep_gsg_repmgr -h env1 -n 2 -l localhost:8080 \ 548-r localhost:8081 -p 10 549No stock database yet available. 550No stock database yet available. </pre> 551 <p> 552 Now, start another process. This time, change the environment 553 home to something else, use the <code class="literal">-l</code> flag to at 554 least change the port number the process is listening on, and 555 use the <code class="literal">-r</code> option to identify the host and 556 port of the other replication process: 557</p> 558 <pre class="programlisting">> mkdir env2 559> ./ex_rep_gsg_repmgr -h env2 -n 2 -l localhost:8081 \ 560-r localhost:8080 -p 20</pre> 561 <p> 562 After a short pause, the second process should display the master 563 prompt: 564</p> 565 <pre class="programlisting"> 566QUOTESERVER > </pre> 567 <p> 568 And the first process should 569 display the read-only prompt: 570</p> 571 <pre class="programlisting"> 572QUOTESERVER (read-only)> </pre> 573 <p> 574 Now go to the master process and give it a couple of stocks and stock 575 prices: 576</p> 577 <pre class="programlisting">QUOTESERVER> FAKECO 9.87 578QUOTESERVER> NOINC .23 579QUOTESERVER> </pre> 580 <p> 581 Then, go to the replica and hit <strong class="userinput"><code>return</code></strong> at the prompt to 582 see the new values: 583</p> 584 <pre class="programlisting">QUOTESERVER (read-only)> 585 Symbol Price 586 ====== ===== 587 FAKECO 9.87 588 NOINC .23 589QUOTESERVER (read-only)> </pre> 590 <p> 591 Doing the same at the master results in the same thing: 592</p> 593 <pre class="programlisting">QUOTESERVER> 594 Symbol Price 595 ====== ===== 596 FAKECO 9.87 597 NOINC .23 598QUOTESERVER> </pre> 599 <p> 600 You can change a stock by simply entering the stock value and 601 new price at the master's prompt: 602</p> 603 <pre class="programlisting">QUOTESERVER> FAKECO 10.01 604QUOTESERVER> </pre> 605 <p> 606 Then, go to either the master or the replica to see the updated 607 database. On the master: 608</p> 609 <pre class="programlisting">QUOTESERVER> 610 Symbol Price 611 ====== ===== 612 FAKECO 10.01 613 NOINC .23 614QUOTESERVER> </pre> 615 <p> 616 And on the replica: 617</p> 618 <pre class="programlisting">QUOTESERVER (read-only)> 619 Symbol Price 620 ====== ===== 621 FAKECO 10.01 622 NOINC .23 623QUOTESERVER (read-only)> </pre> 624 <p> 625 Finally, to quit the applications, simply type 626 <code class="literal">quit</code> at both prompts. On the replica: 627</p> 628 <pre class="programlisting">QUOTESERVER (read-only)> quit 629> </pre> 630 <p> 631 And on the master as well: 632</p> 633 <pre class="programlisting">QUOTESERVER> quit 634> </pre> 635 </div> 636 </div> 637 <div class="navfooter"> 638 <hr /> 639 <table width="100%" summary="Navigation footer"> 640 <tr> 641 <td width="40%" align="left"><a accesskey="p" href="processingloop.html">Prev</a> </td> 642 <td width="20%" align="center"> 643 <a accesskey="u" href="fwrkmasterreplica.html">Up</a> 644 </td> 645 <td width="40%" align="right"> <a accesskey="n" href="addfeatures.html">Next</a></td> 646 </tr> 647 <tr> 648 <td width="40%" align="left" valign="top">Processing Loop </td> 649 <td width="20%" align="center"> 650 <a accesskey="h" href="index.html">Home</a> 651 </td> 652 <td width="40%" align="right" valign="top"> Chapter 5. Additional Features</td> 653 </tr> 654 </table> 655 </div> 656 </body> 657</html> 658