1<?xml version="1.0" encoding="UTF-8" standalone="no"?> 2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 3<html xmlns="http://www.w3.org/1999/xhtml"> 4 <head> 5 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> 6 <title>Chapter��3.��The DB Replication Manager</title> 7 <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> 8 <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> 9 <link rel="start" href="index.html" title="Getting Started with Replicated Berkeley DB Applications" /> 10 <link rel="up" href="index.html" title="Getting Started with Replicated Berkeley DB Applications" /> 11 <link rel="prev" href="simpleprogramlisting.html" title="Program Listing" /> 12 <link rel="next" href="repmgr_init_example_c.html" title="Adding the Replication Manager to RepMgr" /> 13 </head> 14 <body> 15 <div class="navheader"> 16 <table width="100%" summary="Navigation header"> 17 <tr> 18 <th colspan="3" align="center">Chapter��3.��The DB Replication Manager</th> 19 </tr> 20 <tr> 21 <td width="20%" align="left"><a accesskey="p" href="simpleprogramlisting.html">Prev</a>��</td> 22 <th width="60%" align="center">��</th> 23 <td width="20%" align="right">��<a accesskey="n" href="repmgr_init_example_c.html">Next</a></td> 24 </tr> 25 </table> 26 <hr /> 27 </div> 28 <div class="chapter" lang="en" xml:lang="en"> 29 <div class="titlepage"> 30 <div> 31 <div> 32 <h2 class="title"><a id="repapp"></a>Chapter��3.��The DB Replication Manager</h2> 33 </div> 34 </div> 35 </div> 36 <div class="toc"> 37 <p> 38 <b>Table of Contents</b> 39 </p> 40 <dl> 41 <dt> 42 <span class="sect1"> 43 <a href="repapp.html#rep_init_code"> 44 Starting and Stopping Replication 45 </a> 46 </span> 47 </dt> 48 <dd> 49 <dl> 50 <dt> 51 <span class="sect2"> 52 <a href="repapp.html#election_flags">Managing Election Policies</a> 53 </span> 54 </dt> 55 <dt> 56 <span class="sect2"> 57 <a href="repapp.html#thread_count">Selecting the Number of Threads</a> 58 </span> 59 </dt> 60 </dl> 61 </dd> 62 <dt> 63 <span class="sect1"> 64 <a href="repmgr_init_example_c.html">Adding the Replication Manager to 65 66 <span>RepMgr</span> 67 68 </a> 69 </span> 70 </dt> 71 <dt> 72 <span class="sect1"> 73 <a href="fwrkpermmessage.html">Permanent Message Handling</a> 74 </span> 75 </dt> 76 <dd> 77 <dl> 78 <dt> 79 <span class="sect2"> 80 <a href="fwrkpermmessage.html#fmwrkpermpolicy">Identifying Permanent Message Policies</a> 81 </span> 82 </dt> 83 <dt> 84 <span class="sect2"> 85 <a href="fwrkpermmessage.html#fmwrkpermtimeout">Setting the Permanent Message Timeout</a> 86 </span> 87 </dt> 88 <dt> 89 <span class="sect2"> 90 <a href="fwrkpermmessage.html#perm2fmwrkexample">Adding a Permanent Message Policy to 91 92 <span>RepMgrGSG</span> 93 94 </a> 95 </span> 96 </dt> 97 </dl> 98 </dd> 99 <dt> 100 <span class="sect1"> 101 <a href="electiontimes.html">Managing Election Times</a> 102 </span> 103 </dt> 104 <dd> 105 <dl> 106 <dt> 107 <span class="sect2"> 108 <a href="electiontimes.html#electiontimeout">Managing Election Timeouts</a> 109 </span> 110 </dt> 111 <dt> 112 <span class="sect2"> 113 <a href="electiontimes.html#electretrytime">Managing Election Retry Times</a> 114 </span> 115 </dt> 116 </dl> 117 </dd> 118 <dt> 119 <span class="sect1"> 120 <a href="fmwrkconnectretry.html">Managing Connection Retries</a> 121 </span> 122 </dt> 123 <dt> 124 <span class="sect1"> 125 <a href="heartbeats.html">Managing Heartbeats</a> 126 </span> 127 </dt> 128 </dl> 129 </div> 130 <p> 131 The easiest way to add replication to your transactional 132 application is to use the Replication Manager. The Replication Manager provides a comprehensive 133 communications layer that enables replication. For a brief listing 134 of the Replication Manager's feature set, see 135 <a class="xref" href="apioverview.html#repframeworkoverview" title="Replication Manager Overview">Replication Manager Overview</a>. 136 </p> 137 <p> 138 To use the Replication Manager, you make use of special methods off the 139 140 <span><code class="classname">DbEnv</code> class.</span> 141 That is: 142 </p> 143 <div class="orderedlist"> 144 <ol type="1"> 145 <li> 146 <p> 147 Create an environment handle as normal. 148 </p> 149 </li> 150 <li> 151 <p> 152 Configure your environment handle as 153 needed (e.g. set the error file and 154 error prefix values, if desired). 155 </p> 156 </li> 157 <li> 158 <p> 159 Use the Replication Manager replication methods to 160 configure the Replication Manager. Using these 161 methods causes DB to know that you 162 are using the Replication Manager. 163 </p> 164 <p> 165 Configuring the Replication Manager entails setting the replication 166 environment's priority, setting the TCP/IP address 167 that this replication environment will use for 168 incoming replication messages, identifying 169 TCP/IP addresses of other replication 170 environments, setting the number of 171 replication environments in the 172 replication group, and so forth. These actions are 173 discussed throughout the remainder of 174 this chapter. 175 </p> 176 </li> 177 <li> 178 <p> 179 Open your environment handle. When you 180 do this, be sure to specify 181 182 <span><code class="literal">DB_INIT_REP</code> and 183 <code class="literal">DB_THREAD</code> to your 184 open flags. (This is in addition to the 185 flags that you normally use for a 186 single-threaded transactional 187 application). The first of these causes 188 replication to be initialized for the 189 application. The second causes your 190 environment handle to be free-threaded 191 (thread safe). Both flags are required 192 for Replication Manager usage. 193 </span> 194 195 196 </p> 197 </li> 198 <li> 199 <p> 200 Start replication by calling 201 202 <span><code class="methodname">DbEnv::repmgr_start()</code>.</span> 203 204 </p> 205 </li> 206 <li> 207 <p> 208 Open your databases as needed. Masters 209 must open their databases for read 210 and write activity. Replicas can open 211 their databases for read-only activity, but 212 doing so means they must re-open the 213 databases if the replica ever becomes a 214 master. Either way, replicas should never attempt to 215 write to the database(s) directly. 216 </p> 217 </li> 218 </ol> 219 </div> 220 <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"> 221 <h3 class="title">Note</h3> 222 <p> 223 The Replication Manager allows you to only use one 224 environment handle per process. 225 </p> 226 </div> 227 <p> 228 When you are ready to shut down your application: 229 </p> 230 <div class="orderedlist"> 231 <ol type="1"> 232 <li> 233 <p> 234 Close your databases 235 </p> 236 </li> 237 <li> 238 <p> 239 Close your environment. This causes 240 replication to stop as well. 241 </p> 242 </li> 243 </ol> 244 </div> 245 <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"> 246 <h3 class="title">Note</h3> 247 <p> 248 Before you can use the Replication Manager, you may have to 249 enable it in your DB library. This is 250 <span class="emphasis"><em>not</em></span> a requirement for 251 Microsoft Windows systems, or Unix systems that 252 use pthread mutexes by default. Other systems, 253 notably BSD and BSD-derived systems (such as 254 Mac OS X), must enable the Replication Manager when you 255 configure the DB build. 256 </p> 257 <p> 258 You do this by <span class="emphasis"><em>not</em></span> 259 disabling replication and by configuring the 260 library with POSIX threads support. In other 261 words, replication must be turned on in the 262 build (it is by default), and POSIX thread 263 support must be enabled if it is not already by 264 default. To do this, use the 265 <code class="literal">--enable-pthread_api</code> switch 266 on the configure script. 267 </p> 268 <p> 269 For example: 270 </p> 271 <pre class="programlisting">../dist/configure --enable-pthread-api</pre> 272 </div> 273 <div class="sect1" lang="en" xml:lang="en"> 274 <div class="titlepage"> 275 <div> 276 <div> 277 <h2 class="title" style="clear: both"><a id="rep_init_code"></a> 278 Starting and Stopping Replication 279 </h2> 280 </div> 281 </div> 282 </div> 283 <div class="toc"> 284 <dl> 285 <dt> 286 <span class="sect2"> 287 <a href="repapp.html#election_flags">Managing Election Policies</a> 288 </span> 289 </dt> 290 <dt> 291 <span class="sect2"> 292 <a href="repapp.html#thread_count">Selecting the Number of Threads</a> 293 </span> 294 </dt> 295 </dl> 296 </div> 297 <p> 298 As described above, you introduce replication to an 299 application by starting with a transactional 300 application, performing some basic replication 301 configuration, and then starting replication using 302 303 <span><code class="methodname">DbEnv::repmgr_start()</code>.</span> 304 305 </p> 306 <p> 307 You stop replication by closing your environment 308 cleanly in the same way you would for any DB application. 309 </p> 310 <p> 311 For example, the following code fragment initializes, then 312 stops and starts replication. Note that other replication 313 activities are omitted for brevity. 314 </p> 315 <pre class="programlisting">#include <db_cxx.h> 316 317/* Use a 10mb cache */ 318#define CACHESIZE (10 * 1024 * 1024) 319 320... 321 322 DbEnv *dbenv; /* Environment handle. */ 323 const char *progname; /* Program name. */ 324 const char *envHome; /* Environment home directory. */ 325 const char *listen_host; /* A TCP/IP hostname. */ 326 const char *other_host; /* A TCP/IP hostname. */ 327 u_int16 listen_port; /* A TCP/IP port. */ 328 u_int16 other_port; /* A TCP/IP port. */ 329 330 /* Initialize variables */ 331 dbenv = NULL; 332 progname = "example_replication"; 333 envHome = "ENVIRONMENT_HOME"; 334 listen_host = "mymachine.sleepycat.com"; 335 listen_port = 5001; 336 other_host = "anothermachine.sleepycat.com"; 337 other_port = 4555; 338 339 try { 340 /* Create the environment handle */ 341 dbenv = new DbEnv(0); 342 343 /* 344 * Configure the environment handle. Here we configure 345 * asynchronous transactional commits for performance reasons. 346 */ 347 dbenv->set_errfile(stderr); 348 dbenv->set_errpfx(progname); 349 (void)dbenv->set_cachesize(0, CACHESIZE, 0); 350 (void)dbenv->set_flags(DB_TXN_NOSYNC, 1); 351 352 /* 353 * Configure the local address. This is the local hostname and 354 * port that this replication environment will use to receive 355 * incoming replication messages. Note that this can be 356 * performed only once for the replication environment. 357 * It is required. 358 */ 359 dbenv->repmgr_set_local_site(listen_host, listen_port, 0); 360 361 /* 362 * Set this replication environment's priority. This is used 363 * for elections. 364 * 365 * Set this number to a positive integer, or 0 if you do not want 366 * this site to be able to become a master. 367 */ 368 dbenv->rep_set_priority(100); 369 370 /* 371 * Add a site to the list of replication environments known to 372 * this application. 373 */ 374 dbenv->repmgr_add_remote_site(dbenv, other_host, other_port, 375 NULL, 0); 376 377 /* 378 * Identify the number of sites in the replication group. This is 379 * necessary so that elections and permanent message handling 380 * can be performed correctly. 381 */ 382 dbenv->rep_set_nsites(2); 383 384 /* Open the environment handle. Note that we add DB_THREAD and 385 * DB_INIT_REP to the list of flags. These are required. 386 */ 387 dbenv->open(home, DB_CREATE | DB_RECOVER | 388 DB_INIT_LOCK | DB_INIT_LOG | 389 DB_INIT_MPOOL | DB_INIT_TXN | 390 DB_THREAD | DB_INIT_REP, 391 0); 392 393 /* 394 * Start the replication manager such that it uses 3 395 * threads. 396 */ 397 dbenv->repmgr_start(3, DB_REP_ELECTION); 398 399 /* Sleep to give ourselves time to find a master */ 400 sleep(5); 401 402 /* 403 ********************************************************** 404 *** All other application code goes here, including ***** 405 *** database opens ***** 406 ********************************************************** 407 */ 408 409 410 } catch (DbException &de) { 411 /* Error handling goes here */ 412 } 413 414 /* Close out the application here. 415 try { 416 /* 417 * Make sure all your database handles are closed 418 * (omitted from this example). 419 */ 420 421 /* Close the environment */ 422 if (dbenv != NULL) 423 (void)dbenv->close(dbenv, 0); 424 425 } catch (DbException &de) { 426 /* Error handling goes here */ 427 } 428 429 /* All done */ </pre> 430 <div class="sect2" lang="en" xml:lang="en"> 431 <div class="titlepage"> 432 <div> 433 <div> 434 <h3 class="title"><a id="election_flags"></a>Managing Election Policies</h3> 435 </div> 436 </div> 437 </div> 438 <p> 439 Before continuing, it is worth taking a look at the 440 441 <span> 442 startup election flags accepted by 443 444 <span><code class="methodname">DbEnv::repgmr_start()</code>.</span> 445 These flags control how your replication application will 446 behave when it first starts up. 447 </span> 448 449 450 </p> 451 <p> 452 In the previous example, we specified 453 <code class="literal">DB_REP_ELECTION</code> 454 455 when we started replication. This causes the 456 application to try to find a master upon startup. If it 457 cannot, it calls for an election. In the event an 458 election is held, the environment receiving the most number of 459 votes will become the master. 460 </p> 461 <p> 462 There's some important points to make here: 463 </p> 464 <div class="itemizedlist"> 465 <ul type="disc"> 466 <li> 467 <p> 468 This 469 <span>flag</span> 470 471 only requires that other 472 environments in the replication group 473 participate in the vote. There is no 474 requirement that 475 <span class="emphasis"><em>all</em></span> such 476 environments participate. In other 477 words, if an environment 478 starts up, it can call for an 479 election, and select a master, even 480 if all other environment have not yet 481 joined the replication group. 482 </p> 483 </li> 484 <li> 485 <p> 486 It only requires a simple majority of 487 participating environments to elect a master. The number of 488 environments used to calculate the simple 489 majority is based on the value set for 490 491 492 <span><code class="methodname">DbEnv::rep_set_nsites()</code>.</span> 493 494 495 496 This is always true of elections held using the Replication Manager. 497 </p> 498 </li> 499 <li> 500 <p> 501 As always, the environment participating in the election with the most 502 up-to-date log files is selected as 503 master. If an environment with more recent log files 504 has not yet joined the replication 505 group, it may not become the master. 506 </p> 507 </li> 508 </ul> 509 </div> 510 <p> 511 Any one of these points may be enough to cause a 512 less-than-optimum environment to be selected as master. 513 Therefore, to give you a better degree of control over 514 which environment becomes a master at application startup, 515 the Replication Manager offers the following start-up 516 <span>flags:</span> 517 518 </p> 519 <div class="informaltable"> 520 <table border="1" width="80%"> 521 <colgroup> 522 <col /> 523 <col /> 524 </colgroup> 525 <thead> 526 <tr> 527 <th>Flag</th> 528 <th>Description</th> 529 </tr> 530 </thead> 531 <tbody> 532 <tr> 533 <td> 534 <code class="literal">DB_REP_MASTER</code> 535 </td> 536 <td> 537 <p> 538 The application starts up and declares the environment to be a master 539 without calling for an election. It is an error for more 540 than one environment to start up using this flag, or for 541 an environment 542 to use this flag when a master already exists. 543 </p> 544 <p> 545 Note that no replication group should 546 <span class="emphasis"><em>ever</em></span> operate with more than 547 one master. 548 </p> 549 <p> 550 In the event that a environment attempts to become a 551 master when a master already exists, the 552 replication code will resolve the problem by 553 holding an election. Note, however, that there 554 is always a possibility of data loss in the face 555 of duplicate masters, because once a master is 556 selected, the environment that loses the election will 557 have to roll back any transactions committed 558 until it is in sync with the "real" master. 559 </p> 560 561 </td> 562 </tr> 563 <tr> 564 <td> 565 <code class="literal">DB_REP_CLIENT</code> 566 </td> 567 <td> 568 <p> 569 The application starts up and declares 570 the environment to be a replica without calling for 571 an election. Note that the environment 572 can still become a master if a subsequent 573 application starts up, calls for an 574 election, and this environment is elected 575 master. 576 </p> 577 </td> 578 </tr> 579 <tr> 580 <td> 581 <code class="literal">DB_REP_ELECTION</code> 582 </td> 583 <td> 584 <p> 585 As described above, the application starts up, 586 looks for a master, and if one is not found calls 587 for an election. 588 </p> 589 </td> 590 </tr> 591 </tbody> 592 </table> 593 </div> 594 </div> 595 <div class="sect2" lang="en" xml:lang="en"> 596 <div class="titlepage"> 597 <div> 598 <div> 599 <h3 class="title"><a id="thread_count"></a>Selecting the Number of Threads</h3> 600 </div> 601 </div> 602 </div> 603 <p> 604 Under the hood, the Replication Manager is threaded and you can 605 control the number of threads used to process messages received from 606 other replicas. The threads that the Replication Manager uses are: 607 </p> 608 <div class="itemizedlist"> 609 <ul type="disc"> 610 <li> 611 <p> 612 Incoming message thread. This thread 613 receives messages from the site's 614 socket and passes those messages to 615 message processing threads (see below) 616 for handling. 617 </p> 618 </li> 619 <li> 620 <p> 621 Outgoing message thread. Outgoing 622 messages are sent from whatever thread 623 performed a write to the database(s). 624 That is, the thread that called, for 625 example, 626 627 <code class="methodname">Db::put()</code> 628 629 is the thread that writes replication messages 630 about that fact to the socket. 631 </p> 632 <p> 633 Note that if this write activity would 634 cause the thread to be blocked due to 635 some condition on the socket, the Replication Manager 636 will hand the outgoing message to the 637 incoming message thread, and it will 638 then write the message to the socket. 639 This prevents your database write 640 threads from blocking due to abnormal 641 network I/O conditions. 642 </p> 643 </li> 644 <li> 645 <p> 646 Message processing threads are 647 responsible for parsing and then 648 responding to incoming replication 649 messages. Typically, a response will 650 include write activity to your 651 database(s), so these threads can be 652 busy performing disk I/O. 653 </p> 654 </li> 655 </ul> 656 </div> 657 <p> 658 Of these threads, the only ones that you have any 659 configuration control over are the message processing 660 threads. In this case, you can determine how many 661 of these threads you want to run. 662 </p> 663 <p> 664 It is always a bit of an art to decide on a thread count, 665 but the short answer is you probably do not need more 666 than three threads here, and it is likely that one will 667 suffice. That said, the best thing to do is set your 668 thread count to a fairly low number and then increase 669 it if it appears that your application will benefit 670 from the additional threads. 671 </p> 672 </div> 673 </div> 674 </div> 675 <div class="navfooter"> 676 <hr /> 677 <table width="100%" summary="Navigation footer"> 678 <tr> 679 <td width="40%" align="left"><a accesskey="p" href="simpleprogramlisting.html">Prev</a>��</td> 680 <td width="20%" align="center">��</td> 681 <td width="40%" align="right">��<a accesskey="n" href="repmgr_init_example_c.html">Next</a></td> 682 </tr> 683 <tr> 684 <td width="40%" align="left" valign="top">Program Listing��</td> 685 <td width="20%" align="center"> 686 <a accesskey="h" href="index.html">Home</a> 687 </td> 688 <td width="40%" align="right" valign="top">��Adding the Replication Manager to 689 690 <span>RepMgr</span> 691 692 </td> 693 </tr> 694 </table> 695 </div> 696 </body> 697</html> 698