1<?xml version="1.0" encoding="UTF-8" standalone="no"?> 2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 3<html xmlns="http://www.w3.org/1999/xhtml"> 4 <head> 5 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> 6 <title>Adding the Replication Framework to 7 8 SimpleTxn 9 </title> 10 <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> 11 <meta name="generator" content="DocBook XSL Stylesheets V1.62.4" /> 12 <link rel="home" href="index.html" title="Getting Started with Replicated Berkeley DB Applications" /> 13 <link rel="up" href="repapp.html" title="Chapter��3.��The DB Replication Framework" /> 14 <link rel="previous" href="repapp.html" title="Chapter��3.��The DB Replication Framework" /> 15 <link rel="next" href="fwrkpermmessage.html" title="Permanent Message Handling" /> 16 </head> 17 <body> 18 <div class="navheader"> 19 <table width="100%" summary="Navigation header"> 20 <tr> 21 <th colspan="3" align="center">Adding the Replication Framework to 22 23 SimpleTxn 24 </th> 25 </tr> 26 <tr> 27 <td width="20%" align="left"><a accesskey="p" href="repapp.html">Prev</a>��</td> 28 <th width="60%" align="center">Chapter��3.��The DB Replication Framework</th> 29 <td width="20%" align="right">��<a accesskey="n" href="fwrkpermmessage.html">Next</a></td> 30 </tr> 31 </table> 32 <hr /> 33 </div> 34 <div class="sect1" lang="en" xml:lang="en"> 35 <div class="titlepage"> 36 <div> 37 <div> 38 <h2 class="title" style="clear: both"><a id="repmgr_init_example_c"></a>Adding the Replication Framework to 39 40 <span>SimpleTxn</span> 41 </h2> 42 </div> 43 </div> 44 <div></div> 45 </div> 46 <p> 47 We now use the methods described above to add partial 48 support to the 49 50 <tt class="literal">SimpleTxn</tt> 51 example that we presented in 52 <a href="txnapp.html">Transactional Application</a>. 53 That is, in this section we will: 54 </p> 55 <div class="itemizedlist"> 56 <ul type="disc"> 57 <li> 58 <p> 59 Enhance our command line options to 60 accept information of interest to a 61 replicated application. 62 </p> 63 </li> 64 <li> 65 <p> 66 Configure our environment handle to use 67 replication and the replication framework. 68 </p> 69 </li> 70 <li> 71 <p> 72 Minimally configure the replication framework. 73 </p> 74 </li> 75 <li> 76 <p> 77 Start replication. 78 </p> 79 </li> 80 </ul> 81 </div> 82 <p> 83 Note that when we are done with this section, we will be 84 only partially ready to run the application. Some critical 85 pieces will be missing; specifically, we will not yet be 86 handling the differences between a master and a 87 replica. (We do that in the next chapter). 88 </p> 89 <p> 90 Also, note that in the following code fragments, additions 91 and changes to the code are marked in <b class="userinput"><tt>bold</tt></b>. 92 </p> 93 <p> 94 To begin, we copy the 95 96 <tt class="literal">SimpleTxn</tt> 97 code to a new file called 98 <tt class="literal">RepMgr.cpp</tt>. 99 100 101 102 <span> 103 Having done that, we must make some significant 104 changes to our 105 <tt class="classname">RepConfigInfo</tt> 106 class because now we will be using it to 107 maintain a lot more information. 108 </span> 109 </p> 110 <p> 111 First, we create a new structure, 112 <tt class="literal">RepHostInfoObj</tt>, which we use to store 113 host and port information for all "other" servers 114 identified to the application via the 115 <tt class="literal">-o</tt> command line option. This structure 116 is chain-able, which makes cleaning up at program shutdown 117 time easier. 118 </p> 119 <pre class="programlisting">#include <db_cxx.h> 120#include <iostream> 121 122<b class="userinput"><tt>// Chain-able struct used to store host information. 123typedef struct RepHostInfoObj{ 124 char* host; 125 int port; 126 RepHostInfoObj* next; // used for chaining multiple "other" hosts. 127} REP_HOST_INFO; </tt></b></pre> 128 <p> 129 Next, we update our <tt class="classname">RepConfigInfo</tt> class 130 definition to manage a lot more information and a new method. 131</p> 132 <pre class="programlisting">class RepConfigInfo { 133public: 134 RepConfigInfo(); 135 virtual ~RepConfigInfo(); 136 137 <b class="userinput"><tt>void addOtherHost(char* host, int port);</tt></b> 138public: 139 <b class="userinput"><tt>u_int32_t start_policy;</tt></b> 140 char* home; 141 <b class="userinput"><tt>bool got_listen_address; 142 REP_HOST_INFO this_host; 143 int totalsites; 144 int priority; 145 // used to store a set of optional other hosts. 146 REP_HOST_INFO *other_hosts;</tt></b> 147}; </pre> 148 <p> 149 Then, we update our constructor to initialize our new variables. 150</p> 151 <pre class="programlisting">RepConfigInfo::RepConfigInfo() 152{ 153 <b class="userinput"><tt>start_policy = DB_REP_ELECTION;</tt></b> 154 home = "TESTDIR"; 155 <b class="userinput"><tt>got_listen_address = false; 156 totalsites = 0; 157 priority = 100; 158 other_hosts = NULL;</tt></b> 159} </pre> 160 <p> 161 Next, we implement our new method, <tt class="methodname">RepConfigInfo::addOtherHost</tt>, 162 which is used to create <tt class="literal">RepHostInfoObj</tt> instances and add them to 163 the chain of "other" hosts. 164</p> 165 <pre class="programlisting"> 166 <b class="userinput"> 167 <tt>RepConfigInfo::addOtherHost(char* host, int port) 168{ 169 REP_HOST_INFO *newinfo; 170 newinfo = (REP_HOST_INFO*)malloc(sizeof(REP_HOST_INFO)); 171 newinfo->host = host; 172 newinfo->port = port; 173 if (other_hosts == NULL) { 174 other_hosts = newinfo; 175 newinfo->next = NULL; 176 } else { 177 newinfo->next = other_hosts; 178 other_hosts = newinfo; 179 } 180} </tt> 181 </b> 182 </pre> 183 <p> 184 Having done that, we update our class destructor to release the <tt class="literal">RepHostInfoObj</tt> 185 chain of objects at class destruction time. 186</p> 187 <pre class="programlisting">RepConfigInfo::~RepConfigInfo() 188{ 189 <b class="userinput"><tt>// release any other_hosts structs. 190 if (other_hosts != NULL) { 191 REP_HOST_INFO *CurItem = other_hosts; 192 while (CurItem->next != NULL) 193 { 194 REP_HOST_INFO *TmpItem = CurItem; 195 free(CurItem); 196 CurItem = TmpItem; 197 } 198 free(CurItem); 199 } 200 other_hosts = NULL;</tt></b> 201} </pre> 202 <p> 203 Having completed our update to the 204 <tt class="classname">RepConfigInfo</tt> 205 206 class, we can now start making 207 changes to the main portion of our program. We begin by changing 208 the program's name. 209</p> 210 <pre class="programlisting">using std::cout; 211using std::cin; 212using std::cerr; 213using std::endl; 214using std::flush; 215 216#define CACHESIZE (10 * 1024 * 1024) 217#define DATABASE "quote.db" 218 219<b class="userinput"><tt>const char *progname = "RepMgr";</tt></b> </pre> 220 <p> 221 Next we update our usage function. The application will continue to 222 accept the <tt class="literal">-h</tt> parameter so that we can identify 223 the environment home directory used by this application. However, 224 we also add the 225</p> 226 <div class="itemizedlist"> 227 <ul type="disc"> 228 <li> 229 <p> 230 <tt class="literal">-m</tt> parameter which allows us to 231 identify the host and port used by this application to 232 listen for replication messages. 233 </p> 234 </li> 235 <li> 236 <p> 237 <tt class="literal">-o</tt> parameter which allows us to 238 specify other replicas. 239 </p> 240 </li> 241 <li> 242 <p> 243 <tt class="literal">-n</tt> parameter which allows us to 244 identify the number of sites in this replication 245 group. 246 </p> 247 </li> 248 <li> 249 <p> 250 <tt class="literal">-p</tt> option, which is used to identify 251 this replica's priority (recall that the priority is 252 used as a tie breaker for elections) 253 </p> 254 </li> 255 </ul> 256 </div> 257 <pre class="programlisting">class RepMgr 258{ 259public: 260 // Constructor. 261 RepMgr(); 262 // Initialization method. Creates and opens our environment handle. 263 int init(RepConfigInfo* config); 264 // The doloop is where all the work is performed. 265 int doloop(); 266 // terminate() provides our shutdown code. 267 int terminate(); 268 269private: 270 // disable copy constructor. 271 RepMgr(const RepMgr &); 272 void operator = (const RepMgr &); 273 274 // internal data members. 275 RepConfigInfo *app_config; 276 DbEnv dbenv; 277 278 // private methods. 279 // print_stocks() is used to display the contents of our database. 280 static int print_stocks(Db *dbp); 281}; 282 283static void usage() 284{ 285 cerr << "usage: " << progname << endl 286 << "[-h home]<b class="userinput"><tt>[-o host:port][-m host:port]</tt></b>" 287 <b class="userinput"><tt><< "[-n nsites][-p priority]" << endl;</tt></b> 288 289 cerr <b class="userinput"><tt><< "\t -m host:port (required; m stands for me)" << endl 290 << "\t -o host:port (optional; o stands for other; any " 291 << "number of these may be specified)" << endl</tt></b> 292 << "\t -h home directory" << endl 293 <b class="userinput"><tt><< "\t -n nsites (optional; number of sites in replication " 294 << "group; defaults to 0" << endl 295 << "\t in which case we try to dynamically compute the " 296 << "number of sites in" << endl 297 << "\t the replication group)" << endl 298 << "\t -p priority (optional: defaults to 100)" << endl;</tt></b> 299 300 exit(EXIT_FAILURE); 301} </pre> 302 <p> 303 Now we can begin working on our <tt class="literal">main()</tt> function. 304 We begin by adding a couple of variables that we will use to 305 collect TCP/IP host and port information. 306 307 308 309</p> 310 <pre class="programlisting">int main(int argc, char **argv) 311{ 312 RepConfigInfo config; 313 char ch<b class="userinput"><tt>, *portstr, *tmphost; 314 int tmpport;</tt></b> 315 int ret; </pre> 316 <p> 317 Now we collect our command line arguments. As we do so, we will 318 configure host and port information as required, and we will 319 configure the application's election priority if necessary. 320</p> 321 <pre class="programlisting"> // Extract the command line parameters 322 while ((ch = getopt(argc, argv, "h:<b class="userinput"><tt>m:n:o:p:</tt></b>")) != EOF) { 323 switch (ch) { 324 case 'h': 325 config.home = optarg; 326 break; 327 <b class="userinput"><tt>case 'm': 328 config.this_host.host = strtok(optarg, ":"); 329 if ((portstr = strtok(NULL, ":")) == NULL) { 330 cerr << "Bad host specification." << endl; 331 usage(); 332 } 333 config.this_host.port = (unsigned short)atoi(portstr); 334 config.got_listen_address = true; 335 break; 336 case 'n': 337 config.totalsites = atoi(optarg); 338 break; 339 case 'o': 340 tmphost = strtok(optarg, ":"); 341 if ((portstr = strtok(NULL, ":")) == NULL) { 342 cerr << "Bad host specification." << endl; 343 usage(); 344 } 345 tmpport = (unsigned short)atoi(portstr); 346 config.addOtherHost(tmphost, tmpport); 347 break; 348 case 'p': 349 config.priority = atoi(optarg); 350 break;</tt></b> 351 case '?': 352 default: 353 usage(); 354 } 355 } 356 357 // Error check command line. 358 if ((!config.got_listen_address) || config.home == NULL) 359 usage(); </pre> 360 <p> 361 Having done that, the remainder of our <tt class="function">main()</tt> 362 function is left unchanged: 363</p> 364 <pre class="programlisting"> RepMgr runner; 365 try { 366 if((ret = runner.init(&config)) != 0) 367 goto err; 368 if((ret = runner.doloop()) != 0) 369 goto err; 370 } catch (DbException dbe) { 371 cerr << "Caught an exception during initialization or" 372 << " processing: " << dbe.what() << endl; 373 } 374err: 375 runner.terminate(); 376 return 0; 377} </pre> 378 <p> 379 Now we need to update our 380 <tt class="methodname">RepMgr::init()</tt> 381 382 method. Our updates are at first related to configuring 383 replication. First, we need to update the method so that we can 384 identify the local site to the environment handle (that is, the site identified by the 385<tt class="literal">-m</tt> command line option): 386</p> 387 <pre class="programlisting">RepMgr::RepMgr() : app_config(0), dbenv(0) 388{ 389} 390 391int RepMgr::init(RepConfigInfo *config) 392{ 393 int ret = 0; 394 395 app_config = config; 396 397 dbenv.set_errfile(stderr); 398 dbenv.set_errpfx(progname); 399 400 <b class="userinput"><tt>if ((ret = dbenv.repmgr_set_local_site(app_config->this_host.host, 401 app_config->this_host.port, 0)) != 0) { 402 cerr << "Could not set listen address to host:port " 403 << app_config->this_host.host << ":" 404 << app_config->this_host.port 405 << "error: " << ret << endl; 406 }</tt></b> </pre> 407 <p> 408 And we also add code to allow us to identify "other" sites to the environment handle (that is, 409the sites that we identify using the <tt class="literal">-o</tt> command line 410option). To do this, we iterate over each of the "other" sites provided to 411us using the <tt class="literal">-o</tt> command line option, and we add each one 412individually in turn: 413</p> 414 <pre class="programlisting"> <b class="userinput"><tt>for ( REP_HOST_INFO *cur = app_config->other_hosts; cur != NULL; 415 cur = cur->next) { 416 if ((ret = dbenv.repmgr_add_remote_site(cur->host, cur->port, 417 NULL, 0)) != 0) { 418 cerr << "could not add site." << endl 419 } 420 } </tt></b> </pre> 421 <p> 422 And then we need code to allow us to identify the total number of sites 423 in this replication group, and to set the environment's priority. 424</p> 425 <pre class="programlisting"> <b class="userinput"><tt>if (app_config->totalsites > 0) { 426 try { 427 if ((ret = dbenv.rep_set_nsites(app_config->totalsites)) != 0) 428 dbenv.err(ret, "set_nsites"); 429 } catch (DbException dbe) { 430 cerr << "rep_set_nsites call failed. Continuing." << endl; 431 } 432 } 433 dbenv.rep_set_priority(app_config->priority); </tt></b> </pre> 434 <p> 435 436 437 <span>We can now open our environment. Note that the flags</span> 438 439 440 441 we use to open the environment are slightly different for a 442 replicated application than they are for a non-replicated 443 application. Namely, replication requires the 444 <span> 445 <tt class="literal">DB_INIT_REP</tt> flag. 446 </span> 447 448 449 </p> 450 <p> 451 Also, because we are using the replication framework, we must prepare 452 our environment for threaded usage. For this reason, we also 453 need the <tt class="literal">DB_THREAD</tt> flag. 454 </p> 455 <pre class="programlisting"> dbenv.set_cachesize(0, CACHESIZE, 0); 456 dbenv.set_flags(DB_TXN_NOSYNC, 1); 457 458 try { 459 dbenv.open(app_config->home, 460 DB_CREATE | 461 DB_RECOVER | 462 <b class="userinput"><tt>DB_THREAD | 463 DB_INIT_REP |</tt></b> 464 DB_INIT_LOCK | 465 DB_INIT_LOG | 466 DB_INIT_MPOOL | 467 DB_INIT_TXN, 468 0); 469 } catch(DbException dbe) { 470 cerr << "Caught an exception during DB environment open." << endl 471 << "Ensure that the home directory is created prior to starting" 472 << " the application." << endl; 473 ret = ENOENT; 474 goto err; 475 }</pre> 476 <p> 477 Finally, we start replication before we exit this method. 478 Immediately after exiting this method, our application will go into 479 the 480 <tt class="methodname">RepMgr::doloop()</tt> 481 482 method, which is where 483 the bulk of our application's work is performed. We update that 484 method in the next chapter. 485</p> 486 <pre class="programlisting"> if ((ret = dbenv.repmgr_start(3, app_config->start_policy)) != 0) 487 goto err; 488 489err: 490 return ret; 491} </pre> 492 <p> 493 This completes our replication updates for the moment. We are not as 494 yet ready to actually run this program; there remains a few 495 critical pieces left to add to it. However, the work that we 496 performed in this section represents a solid foundation for the 497 remainder of our replication work. 498 </p> 499 </div> 500 <div class="navfooter"> 501 <hr /> 502 <table width="100%" summary="Navigation footer"> 503 <tr> 504 <td width="40%" align="left"><a accesskey="p" href="repapp.html">Prev</a>��</td> 505 <td width="20%" align="center"> 506 <a accesskey="u" href="repapp.html">Up</a> 507 </td> 508 <td width="40%" align="right">��<a accesskey="n" href="fwrkpermmessage.html">Next</a></td> 509 </tr> 510 <tr> 511 <td width="40%" align="left" valign="top">Chapter��3.��The DB Replication Framework��</td> 512 <td width="20%" align="center"> 513 <a accesskey="h" href="index.html">Home</a> 514 </td> 515 <td width="40%" align="right" valign="top">��Permanent Message Handling</td> 516 </tr> 517 </table> 518 </div> 519 </body> 520</html> 521