1<?xml version="1.0" encoding="UTF-8" standalone="no"?> 2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 3<html xmlns="http://www.w3.org/1999/xhtml"> 4 <head> 5 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> 6 <title>Adding the Replication Manager to RepMgr</title> 7 <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> 8 <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> 9 <link rel="start" href="index.html" title="Getting Started with Replicated Berkeley DB Applications" /> 10 <link rel="up" href="repapp.html" title="Chapter��3.��The DB Replication Manager" /> 11 <link rel="prev" href="repapp.html" title="Chapter��3.��The DB Replication Manager" /> 12 <link rel="next" href="fwrkpermmessage.html" title="Permanent Message Handling" /> 13 </head> 14 <body> 15 <div class="navheader"> 16 <table width="100%" summary="Navigation header"> 17 <tr> 18 <th colspan="3" align="center">Adding the Replication Manager to 19 20 <span>RepMgr</span> 21 22 </th> 23 </tr> 24 <tr> 25 <td width="20%" align="left"><a accesskey="p" href="repapp.html">Prev</a>��</td> 26 <th width="60%" align="center">Chapter��3.��The DB Replication Manager</th> 27 <td width="20%" align="right">��<a accesskey="n" href="fwrkpermmessage.html">Next</a></td> 28 </tr> 29 </table> 30 <hr /> 31 </div> 32 <div class="sect1" lang="en" xml:lang="en"> 33 <div class="titlepage"> 34 <div> 35 <div> 36 <h2 class="title" style="clear: both"><a id="repmgr_init_example_c"></a>Adding the Replication Manager to 37 38 <span>RepMgr</span> 39 40 </h2> 41 </div> 42 </div> 43 </div> 44 <p> 45 We now use the methods described above to add partial 46 support to the 47 48 <span>RepMgr</span> 49 50 example that we presented in 51 <a class="xref" href="txnapp.html" title="Chapter��2.��Transactional Application">Transactional Application</a>. 52 That is, in this section we will: 53 </p> 54 <div class="itemizedlist"> 55 <ul type="disc"> 56 <li> 57 <p> 58 Enhance our command line options to 59 accept information of interest to a 60 replicated application. 61 </p> 62 </li> 63 <li> 64 <p> 65 Configure our environment handle to use 66 replication and the Replication Manager. 67 </p> 68 </li> 69 <li> 70 <p> 71 Minimally configure the Replication Manager. 72 </p> 73 </li> 74 <li> 75 <p> 76 Start replication. 77 </p> 78 </li> 79 </ul> 80 </div> 81 <p> 82 Note that when we are done with this section, we will be 83 only partially ready to run the application. Some critical 84 pieces will be missing; specifically, we will not yet be 85 handling the differences between a master and a 86 replica. (We do that in the next chapter). 87 </p> 88 <p> 89 Also, note that in the following code fragments, additions 90 and changes to the code are marked in <strong class="userinput"><code>bold</code></strong>. 91 </p> 92 <p> 93 To begin, we copy the 94 95 <span>SimpleTxn</span> 96 code to a new file called 97 <code class="literal">RepMgrGSG.cpp</code>. 98 99 100 101 <span> 102 Having done that, we must make some significant 103 changes to our 104 <code class="classname">RepConfigInfo</code> 105 class because now we will be using it to 106 maintain a lot more information. 107 </span> 108 </p> 109 <p> 110 First, we create a new structure, 111 <code class="literal">RepHostInfoObj</code>, which we use to store 112 host and port information for all "other" servers 113 identified to the application via the 114 <code class="literal">-o</code> command line option. This structure 115 is chain-able, which makes cleaning up at program shutdown 116 time easier. 117 </p> 118 <pre class="programlisting">#include <db_cxx.h> 119#include <iostream> 120 121<strong class="userinput"><code>// Chain-able struct used to store host information. 122typedef struct RepHostInfoObj{ 123 char* host; 124 u_int16_t port; 125 RepHostInfoObj* next; // used for chaining multiple "other" hosts. 126} REP_HOST_INFO; </code></strong></pre> 127 <p> 128 Next, we update our <code class="classname">RepConfigInfo</code> class 129 definition to manage a lot more information and a new method. 130</p> 131 <pre class="programlisting">class RepConfigInfo { 132public: 133 RepConfigInfo(); 134 virtual ~RepConfigInfo(); 135 136 <strong class="userinput"><code>void addOtherHost(char* host, int port);</code></strong> 137public: 138 <strong class="userinput"><code>u_int32_t start_policy;</code></strong> 139 char* home; 140 <strong class="userinput"><code>bool got_listen_address; 141 REP_HOST_INFO this_host; 142 int totalsites; 143 int priority; 144 // used to store a set of optional other hosts. 145 REP_HOST_INFO *other_hosts;</code></strong> 146}; </pre> 147 <p> 148 Then, we update our constructor to initialize our new variables. 149</p> 150 <pre class="programlisting">RepConfigInfo::RepConfigInfo() 151{ 152 <strong class="userinput"><code>start_policy = DB_REP_ELECTION;</code></strong> 153 home = "TESTDIR"; 154 <strong class="userinput"><code>got_listen_address = false; 155 totalsites = 0; 156 priority = 100; 157 other_hosts = NULL;</code></strong> 158} </pre> 159 <p> 160 Next, we implement our new method, <code class="methodname">RepConfigInfo::addOtherHost</code>, 161 which is used to create <code class="literal">RepHostInfoObj</code> instances and add them to 162 the chain of "other" hosts. 163</p> 164 <pre class="programlisting">RepConfigInfo::addOtherHost(char* host, int port) 165{ 166 REP_HOST_INFO *newinfo; 167 newinfo = (REP_HOST_INFO*)malloc(sizeof(REP_HOST_INFO)); 168 newinfo->host = host; 169 newinfo->port = port; 170 if (other_hosts == NULL) { 171 other_hosts = newinfo; 172 newinfo->next = NULL; 173 } else { 174 newinfo->next = other_hosts; 175 other_hosts = newinfo; 176 } 177}</pre> 178 <p> 179 Having done that, we update our class destructor to release the <code class="literal">RepHostInfoObj</code> 180 chain of objects at class destruction time. 181</p> 182 <pre class="programlisting">RepConfigInfo::~RepConfigInfo() 183{ 184 <strong class="userinput"><code>// release any other_hosts structs. 185 if (other_hosts != NULL) { 186 REP_HOST_INFO *CurItem = other_hosts; 187 while (CurItem->next != NULL) 188 { 189 REP_HOST_INFO *TmpItem = CurItem; 190 free(CurItem); 191 CurItem = TmpItem; 192 } 193 free(CurItem); 194 } 195 other_hosts = NULL;</code></strong> 196} </pre> 197 <p> 198 Having completed our update to the 199 <code class="classname">RepConfigInfo</code> 200 201 class, we can now start making 202 changes to the main portion of our program. We begin by changing 203 the program's name. 204</p> 205 <pre class="programlisting">using std::cout; 206using std::cin; 207using std::cerr; 208using std::endl; 209using std::flush; 210 211#define CACHESIZE (10 * 1024 * 1024) 212#define DATABASE "quote.db" 213 214<strong class="userinput"><code>const char *progname = "RepMgrGSG";</code></strong> </pre> 215 <p> 216 Next we update our usage function. The application will continue to 217 accept the <code class="literal">-h</code> parameter so that we can identify 218 the environment home directory used by this application. However, 219 we also add the 220</p> 221 <div class="itemizedlist"> 222 <ul type="disc"> 223 <li> 224 <p> 225 <code class="literal">-l</code> parameter which allows us to 226 identify the host and port used by this application to 227 listen for replication messages. 228 </p> 229 </li> 230 <li> 231 <p> 232 <code class="literal">-r</code> parameter which allows us to 233 specify other replicas. 234 </p> 235 </li> 236 <li> 237 <p> 238 <code class="literal">-n</code> parameter which allows us to 239 identify the number of sites in this replication 240 group. 241 </p> 242 </li> 243 <li> 244 <p> 245 <code class="literal">-p</code> option, which is used to identify 246 this replica's priority (recall that the priority is 247 used as a tie breaker for elections) 248 </p> 249 </li> 250 </ul> 251 </div> 252 <pre class="programlisting">class RepMgrGSG 253{ 254public: 255 // Constructor. 256 RepMgrGSG(); 257 // Initialization method. Creates and opens our environment handle. 258 int init(RepConfigInfo* config); 259 // The doloop is where all the work is performed. 260 int doloop(); 261 // terminate() provides our shutdown code. 262 int terminate(); 263 264private: 265 // disable copy constructor. 266 RepMgrGSG(const RepMgrGSG &); 267 void operator = (const RepMgrGSG &); 268 269 // internal data members. 270 RepConfigInfo *app_config; 271 DbEnv dbenv; 272 273 // private methods. 274 // print_stocks() is used to display the contents of our database. 275 static int print_stocks(Db *dbp); 276}; 277 278static void usage() 279{ 280 cerr << "usage: " << progname << endl 281 << "-h home<strong class="userinput"><code>[-r host:port][-l host:port]</code></strong>" 282 <strong class="userinput"><code><< "[-n nsites][-p priority]" << endl;</code></strong> 283 284 cerr <strong class="userinput"><code><< "\t -l host:port (required; l stands for local)" << endl 285 << "\t -r host:port (optional; r stands for replica; any " 286 << "number of these may be specified)" << endl</code></strong> 287 << "\t -h home directory" << endl 288 <strong class="userinput"><code><< "\t -n nsites (optional; number of sites in replication " 289 << "group; defaults to 0" << endl 290 << "\t in which case we try to dynamically compute the " 291 << "number of sites in" << endl 292 << "\t the replication group)" << endl 293 << "\t -p priority (optional: defaults to 100)" << endl;</code></strong> 294 295 exit(EXIT_FAILURE); 296} </pre> 297 <p> 298 Now we can begin working on our <code class="literal">main()</code> function. 299 We begin by adding a couple of variables that we will use to 300 collect TCP/IP host and port information. 301 302 303 304</p> 305 <pre class="programlisting">int main(int argc, char **argv) 306{ 307 RepConfigInfo config; 308 char ch<strong class="userinput"><code>, *portstr, *tmphost; 309 int tmpport;</code></strong> 310 int ret; </pre> 311 <p> 312 Now we collect our command line arguments. As we do so, we will 313 configure host and port information as required, and we will 314 configure the application's election priority if necessary. 315</p> 316 <pre class="programlisting"> // Extract the command line parameters 317 while ((ch = getopt(argc, argv, "h:<strong class="userinput"><code>l:n:p:r:</code></strong>")) != EOF) { 318 switch (ch) { 319 case 'h': 320 config.home = optarg; 321 break; 322 <strong class="userinput"><code>case 'l': 323 config.this_host.host = strtok(optarg, ":"); 324 if ((portstr = strtok(NULL, ":")) == NULL) { 325 cerr << "Bad host specification." << endl; 326 usage(); 327 } 328 config.this_host.port = (unsigned short)atoi(portstr); 329 config.got_listen_address = true; 330 break; 331 case 'n': 332 config.totalsites = atoi(optarg); 333 break; 334 case 'p': 335 config.priority = atoi(optarg); 336 break; 337 case 'r': 338 tmphost = strtok(optarg, ":"); 339 if ((portstr = strtok(NULL, ":")) == NULL) { 340 cerr << "Bad host specification." << endl; 341 usage(); 342 } 343 tmpport = (unsigned short)atoi(portstr); 344 config.addOtherHost(tmphost, tmpport); 345 break;</code></strong> 346 case '?': 347 default: 348 usage(); 349 } 350 } 351 352 // Error check command line. 353 if ((!config.got_listen_address) || config.home == NULL) 354 usage(); </pre> 355 <p> 356 Having done that, the remainder of our <code class="function">main()</code> 357 function is left unchanged: 358</p> 359 <pre class="programlisting"> RepMgrGSG runner; 360 try { 361 if((ret = runner.init(&config)) != 0) 362 goto err; 363 if((ret = runner.doloop()) != 0) 364 goto err; 365 } catch (DbException dbe) { 366 cerr << "Caught an exception during initialization or" 367 << " processing: " << dbe.what() << endl; 368 } 369err: 370 runner.terminate(); 371 return 0; 372} </pre> 373 <p> 374 Now we need to update our 375 <code class="methodname">RepMgrGSG::init()</code> 376 377 method. Our updates are at first related to configuring 378 replication. First, we need to update the method so that we can 379 identify the local site to the environment handle (that is, the site identified by the 380<code class="literal">-l</code> command line option): 381</p> 382 <pre class="programlisting">RepMgrGSG::RepMgrGSG() : app_config(0), dbenv(0) 383{ 384} 385 386int RepMgrGSG::init(RepConfigInfo *config) 387{ 388 int ret = 0; 389 390 app_config = config; 391 392 dbenv.set_errfile(stderr); 393 dbenv.set_errpfx(progname); 394 395 <strong class="userinput"><code>if ((ret = dbenv.repmgr_set_local_site(app_config->this_host.host, 396 app_config->this_host.port, 0)) != 0) { 397 cerr << "Could not set listen address to host:port " 398 << app_config->this_host.host << ":" 399 << app_config->this_host.port 400 << "error: " << ret << endl; 401 }</code></strong> </pre> 402 <p> 403 And we also add code to allow us to identify "other" sites to the environment handle (that is, 404the sites that we identify using the <code class="literal">-o</code> command line 405option). To do this, we iterate over each of the "other" sites provided to 406us using the <code class="literal">-o</code> command line option, and we add each one 407individually in turn: 408</p> 409 <pre class="programlisting"> <strong class="userinput"><code>for ( REP_HOST_INFO *cur = app_config->other_hosts; cur != NULL; 410 cur = cur->next) { 411 if ((ret = dbenv.repmgr_add_remote_site(cur->host, cur->port, 412 NULL, 0, 0)) != 0) { 413 cerr << "could not add site." << endl 414 } 415 } </code></strong> </pre> 416 <p> 417 And then we need code to allow us to identify the total number of sites 418 in this replication group, and to set the environment's priority. 419</p> 420 <pre class="programlisting"> <strong class="userinput"><code>if (app_config->totalsites > 0) { 421 try { 422 if ((ret = dbenv.rep_set_nsites(app_config->totalsites)) != 0) 423 dbenv.err(ret, "set_nsites"); 424 } catch (DbException dbe) { 425 cerr << "rep_set_nsites call failed. Continuing." << endl; 426 } 427 } 428 dbenv.rep_set_priority(app_config->priority); </code></strong> </pre> 429 <p> 430 431 432 <span>We can now open our environment. Note that the flags</span> 433 434 435 436 we use to open the environment are slightly different for a 437 replicated application than they are for a non-replicated 438 application. Namely, replication requires the 439 <span> 440 <code class="literal">DB_INIT_REP</code> flag. 441 </span> 442 443 444 </p> 445 <p> 446 Also, because we are using the Replication Manager, we must prepare 447 our environment for threaded usage. For this reason, we also 448 need the <code class="literal">DB_THREAD</code> flag. 449 </p> 450 <pre class="programlisting"> dbenv.set_cachesize(0, CACHESIZE, 0); 451 dbenv.set_flags(DB_TXN_NOSYNC, 1); 452 453 try { 454 dbenv.open(app_config->home, 455 DB_CREATE | 456 DB_INIT_LOCK | 457 DB_INIT_LOG | 458 DB_INIT_MPOOL | 459 <strong class="userinput"><code>DB_INIT_REP |</code></strong> 460 DB_INIT_TXN | 461 DB_RECOVER <strong class="userinput"><code>| 462 DB_THREAD;</code></strong> 463 0); 464 } catch(DbException dbe) { 465 cerr << "Caught an exception during DB environment open." << endl 466 << "Ensure that the home directory is created prior to " 467 << "starting the application." << endl; 468 ret = ENOENT; 469 goto err; 470 }</pre> 471 <p> 472 Finally, we start replication before we exit this method. 473 Immediately after exiting this method, our application will go into 474 the 475 <code class="methodname">RepMgrGSG::doloop()</code> 476 477 method, which is where 478 the bulk of our application's work is performed. We update that 479 method in the next chapter. 480</p> 481 <pre class="programlisting"> if ((ret = dbenv.repmgr_start(3, app_config->start_policy)) != 0) 482 goto err; 483 484err: 485 return ret; 486} </pre> 487 <p> 488 This completes our replication updates for the moment. We are not as 489 yet ready to actually run this program; there remains a few 490 critical pieces left to add to it. However, the work that we 491 performed in this section represents a solid foundation for the 492 remainder of our replication work. 493 </p> 494 </div> 495 <div class="navfooter"> 496 <hr /> 497 <table width="100%" summary="Navigation footer"> 498 <tr> 499 <td width="40%" align="left"><a accesskey="p" href="repapp.html">Prev</a>��</td> 500 <td width="20%" align="center"> 501 <a accesskey="u" href="repapp.html">Up</a> 502 </td> 503 <td width="40%" align="right">��<a accesskey="n" href="fwrkpermmessage.html">Next</a></td> 504 </tr> 505 <tr> 506 <td width="40%" align="left" valign="top">Chapter��3.��The DB Replication Manager��</td> 507 <td width="20%" align="center"> 508 <a accesskey="h" href="index.html">Home</a> 509 </td> 510 <td width="40%" align="right" valign="top">��Permanent Message Handling</td> 511 </tr> 512 </table> 513 </div> 514 </body> 515</html> 516