1<?xml version="1.0" encoding="UTF-8" standalone="no"?> 2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 3<html xmlns="http://www.w3.org/1999/xhtml"> 4 <head> 5 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> 6 <title>Chapter 3. The DB Replication Manager</title> 7 <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> 8 <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> 9 <link rel="start" href="index.html" title="Getting Started with Replicated Berkeley DB Applications" /> 10 <link rel="up" href="index.html" title="Getting Started with Replicated Berkeley DB Applications" /> 11 <link rel="prev" href="simpleprogramlisting.html" title="Program Listing" /> 12 <link rel="next" href="repmgr_init_example_c.html" title="Adding the Replication Manager to ex_rep_gsg_simple" /> 13 </head> 14 <body> 15 <div class="navheader"> 16 <table width="100%" summary="Navigation header"> 17 <tr> 18 <th colspan="3" align="center">Chapter 3. The DB Replication Manager</th> 19 </tr> 20 <tr> 21 <td width="20%" align="left"><a accesskey="p" href="simpleprogramlisting.html">Prev</a> </td> 22 <th width="60%" align="center"> </th> 23 <td width="20%" align="right"> <a accesskey="n" href="repmgr_init_example_c.html">Next</a></td> 24 </tr> 25 </table> 26 <hr /> 27 </div> 28 <div class="chapter" lang="en" xml:lang="en"> 29 <div class="titlepage"> 30 <div> 31 <div> 32 <h2 class="title"><a id="repapp"></a>Chapter 3. The DB Replication Manager</h2> 33 </div> 34 </div> 35 </div> 36 <div class="toc"> 37 <p> 38 <b>Table of Contents</b> 39 </p> 40 <dl> 41 <dt> 42 <span class="sect1"> 43 <a href="repapp.html#rep_init_code"> 44 Starting and Stopping Replication 45 </a> 46 </span> 47 </dt> 48 <dd> 49 <dl> 50 <dt> 51 <span class="sect2"> 52 <a href="repapp.html#election_flags">Managing Election Policies</a> 53 </span> 54 </dt> 55 <dt> 56 <span class="sect2"> 57 <a href="repapp.html#thread_count">Selecting the Number of Threads</a> 58 </span> 59 </dt> 60 </dl> 61 </dd> 62 <dt> 63 <span class="sect1"> 64 <a href="repmgr_init_example_c.html">Adding the Replication Manager to 65 <span>ex_rep_gsg_simple</span> 66 67 68 </a> 69 </span> 70 </dt> 71 <dt> 72 <span class="sect1"> 73 <a href="fwrkpermmessage.html">Permanent Message Handling</a> 74 </span> 75 </dt> 76 <dd> 77 <dl> 78 <dt> 79 <span class="sect2"> 80 <a href="fwrkpermmessage.html#fmwrkpermpolicy">Identifying Permanent Message Policies</a> 81 </span> 82 </dt> 83 <dt> 84 <span class="sect2"> 85 <a href="fwrkpermmessage.html#fmwrkpermtimeout">Setting the Permanent Message Timeout</a> 86 </span> 87 </dt> 88 <dt> 89 <span class="sect2"> 90 <a href="fwrkpermmessage.html#perm2fmwrkexample">Adding a Permanent Message Policy to 91 <span>ex_rep_gsg_repmgr</span> 92 93 94 </a> 95 </span> 96 </dt> 97 </dl> 98 </dd> 99 <dt> 100 <span class="sect1"> 101 <a href="electiontimes.html">Managing Election Times</a> 102 </span> 103 </dt> 104 <dd> 105 <dl> 106 <dt> 107 <span class="sect2"> 108 <a href="electiontimes.html#electiontimeout">Managing Election Timeouts</a> 109 </span> 110 </dt> 111 <dt> 112 <span class="sect2"> 113 <a href="electiontimes.html#electretrytime">Managing Election Retry Times</a> 114 </span> 115 </dt> 116 </dl> 117 </dd> 118 <dt> 119 <span class="sect1"> 120 <a href="fmwrkconnectretry.html">Managing Connection Retries</a> 121 </span> 122 </dt> 123 <dt> 124 <span class="sect1"> 125 <a href="heartbeats.html">Managing Heartbeats</a> 126 </span> 127 </dt> 128 </dl> 129 </div> 130 <p> 131 The easiest way to add replication to your transactional 132 application is to use the Replication Manager. The Replication Manager provides a comprehensive 133 communications layer that enables replication. For a brief listing 134 of the Replication Manager's feature set, see 135 <a class="xref" href="apioverview.html#repframeworkoverview" title="Replication Manager Overview">Replication Manager Overview</a>. 136 </p> 137 <p> 138 To use the Replication Manager, you make use of special methods off the 139 <span><code class="classname">DB_ENV</code> class.</span> 140 141 That is: 142 </p> 143 <div class="orderedlist"> 144 <ol type="1"> 145 <li> 146 <p> 147 Create an environment handle as normal. 148 </p> 149 </li> 150 <li> 151 <p> 152 Configure your environment handle as 153 needed (e.g. set the error file and 154 error prefix values, if desired). 155 </p> 156 </li> 157 <li> 158 <p> 159 Use the Replication Manager replication methods to 160 configure the Replication Manager. Using these 161 methods causes DB to know that you 162 are using the Replication Manager. 163 </p> 164 <p> 165 Configuring the Replication Manager entails setting the replication 166 environment's priority, setting the TCP/IP address 167 that this replication environment will use for 168 incoming replication messages, identifying 169 TCP/IP addresses of other replication 170 environments, setting the number of 171 replication environments in the 172 replication group, and so forth. These actions are 173 discussed throughout the remainder of 174 this chapter. 175 </p> 176 </li> 177 <li> 178 <p> 179 Open your environment handle. When you 180 do this, be sure to specify 181 182 <span><code class="literal">DB_INIT_REP</code> and 183 <code class="literal">DB_THREAD</code> to your 184 open flags. (This is in addition to the 185 flags that you normally use for a 186 single-threaded transactional 187 application). The first of these causes 188 replication to be initialized for the 189 application. The second causes your 190 environment handle to be free-threaded 191 (thread safe). Both flags are required 192 for Replication Manager usage. 193 </span> 194 195 196 </p> 197 </li> 198 <li> 199 <p> 200 Start replication by calling 201 <span><code class="methodname">DB_ENV->repmgr_start()</code>.</span> 202 203 204 </p> 205 </li> 206 <li> 207 <p> 208 Open your databases as needed. Masters 209 must open their databases for read 210 and write activity. Replicas can open 211 their databases for read-only activity, but 212 doing so means they must re-open the 213 databases if the replica ever becomes a 214 master. Either way, replicas should never attempt to 215 write to the database(s) directly. 216 </p> 217 </li> 218 </ol> 219 </div> 220 <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"> 221 <h3 class="title">Note</h3> 222 <p> 223 The Replication Manager allows you to only use one 224 environment handle per process. 225 </p> 226 </div> 227 <p> 228 When you are ready to shut down your application: 229 </p> 230 <div class="orderedlist"> 231 <ol type="1"> 232 <li> 233 <p> 234 Close your databases 235 </p> 236 </li> 237 <li> 238 <p> 239 Close your environment. This causes 240 replication to stop as well. 241 </p> 242 </li> 243 </ol> 244 </div> 245 <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"> 246 <h3 class="title">Note</h3> 247 <p> 248 Before you can use the Replication Manager, you may have to 249 enable it in your DB library. This is 250 <span class="emphasis"><em>not</em></span> a requirement for 251 Microsoft Windows systems, or Unix systems that 252 use pthread mutexes by default. Other systems, 253 notably BSD and BSD-derived systems (such as 254 Mac OS X), must enable the Replication Manager when you 255 configure the DB build. 256 </p> 257 <p> 258 You do this by <span class="emphasis"><em>not</em></span> 259 disabling replication and by configuring the 260 library with POSIX threads support. In other 261 words, replication must be turned on in the 262 build (it is by default), and POSIX thread 263 support must be enabled if it is not already by 264 default. To do this, use the 265 <code class="literal">--enable-pthread_api</code> switch 266 on the configure script. 267 </p> 268 <p> 269 For example: 270 </p> 271 <pre class="programlisting">../dist/configure --enable-pthread-api</pre> 272 </div> 273 <div class="sect1" lang="en" xml:lang="en"> 274 <div class="titlepage"> 275 <div> 276 <div> 277 <h2 class="title" style="clear: both"><a id="rep_init_code"></a> 278 Starting and Stopping Replication 279 </h2> 280 </div> 281 </div> 282 </div> 283 <div class="toc"> 284 <dl> 285 <dt> 286 <span class="sect2"> 287 <a href="repapp.html#election_flags">Managing Election Policies</a> 288 </span> 289 </dt> 290 <dt> 291 <span class="sect2"> 292 <a href="repapp.html#thread_count">Selecting the Number of Threads</a> 293 </span> 294 </dt> 295 </dl> 296 </div> 297 <p> 298 As described above, you introduce replication to an 299 application by starting with a transactional 300 application, performing some basic replication 301 configuration, and then starting replication using 302 <span><code class="methodname">DB_ENV->repmgr_start()</code>.</span> 303 304 305 </p> 306 <p> 307 You stop replication by closing your environment 308 cleanly in the same way you would for any DB application. 309 </p> 310 <p> 311 For example, the following code fragment initializes, then 312 stops and starts replication. Note that other replication 313 activities are omitted for brevity. 314 </p> 315 <pre class="programlisting">#include <db.h> 316 317/* Use a 10mb cache */ 318#define CACHESIZE (10 * 1024 * 1024) 319 320... 321 322 DB_ENV *dbenv; /* Environment handle. */ 323 const char *progname; /* Program name. */ 324 const char *envHome; /* Environment home directory. */ 325 const char *listen_host; /* A TCP/IP hostname. */ 326 const char *other_host; /* A TCP/IP hostname. */ 327 int ret; /* Error return code. */ 328 u_int16 listen_port; /* A TCP/IP port. */ 329 u_int16 other_port; /* A TCP/IP port. */ 330 331 /* Initialize variables */ 332 dbenv = NULL; 333 progname = "example_replication"; 334 envHome = "ENVIRONMENT_HOME"; 335 listen_host = "mymachine.sleepycat.com"; 336 listen_port = 5001; 337 other_host = "anothermachine.sleepycat.com"; 338 other_port = 4555; 339 ret = 0; 340 341 /* Create the environment handle */ 342 if ((ret = db_env_create(&dbenv, 0)) != 0 ) { 343 fprintf(stderr, "Error creating environment handle: %s\n", 344 db_strerror(ret)); 345 goto err; 346 } 347 348 /* 349 * Configure the environment handle. Here we configure asynchronous 350 * transactional commits for performance reasons. 351 */ 352 dbenv->set_errfile(dbenv, stderr); 353 dbenv->set_errpfx(dbenv, progname); 354 (void)dbenv->set_cachesize(dbenv, 0, CACHESIZE, 0); 355 (void)dbenv->set_flags(dbenv, DB_TXN_NOSYNC, 1); 356 357 358 /* 359 * Configure the local address. This is the local hostname and port 360 * that this replication environment will use to receive incoming 361 * replication messages. Note that this can be performed only once 362 * for the replication environment. It is required. 363 */ 364 if ((ret = dbenv->repmgr_set_local_site(dbenv, listen_host, 365 listen_port, 0)) != 0) { 366 fprintf(stderr, "Could not set local address (%d).\n", ret); 367 goto err; 368 } 369 370 /* 371 * Set this replicatioin environment's priority. This is used 372 * for elections. 373 * 374 * Set this number to a positive integer, or 0 if you do not want 375 * this site to be able to become a master. 376 */ 377 dbenv->rep_set_priority(dbenv, 100); 378 379 /* 380 * Add a site to the list of replication environments known to this 381 * application. 382 */ 383 if (dbenv->repmgr_add_remote_site(dbenv, other_host, other_port, 384 NULL, 0) != 0) { 385 fprintf(stderr, "Could not add site %s.\n", other_host); 386 goto err; 387 } 388 389 /* 390 * Identify the number of sites in the replication group. This is 391 * necessary so that elections and permanent message handling can be 392 * performed correctly. 393 */ 394 if (dbenv->rep_set_nsites(dbenv, 2) != 0) { 395 fprintf(stderr, "Could not set the number of sites.\n"; 396 goto err; 397 } 398 399 /* Open the environment handle. Note that we add DB_THREAD and 400 * DB_INIT_REP to the list of flags. These are required. 401 */ 402 if ((ret = dbenv->open(dbenv, home, DB_CREATE | DB_RECOVER | 403 DB_INIT_LOCK | DB_INIT_LOG | 404 DB_INIT_MPOOL | DB_INIT_TXN | 405 DB_THREAD | DB_INIT_REP, 406 0)) != 0) { 407 goto err; 408 } 409 410 /* Start the replication manager such that it uses 3 threads. */ 411 if ((ret = dbenv->repmgr_start(dbenv, 3, DB_REP_ELECTION)) != 0) 412 goto err; 413 414 /* Sleep to give ourselves time to find a master */ 415 sleep(5); 416 417 /* 418 ********************************************************** 419 *** All other application code goes here, including ***** 420 *** database opens ***** 421 ********************************************************** 422 */ 423 424err: /* 425 * Make sure all your database handles are closed 426 * (omitted from this example). 427 */ 428 429 /* Close the environment */ 430 if (dbenv != NULL) 431 (void)dbenv->close(dbenv, 0); 432 433 /* All done */ 434 return (ret); </pre> 435 <div class="sect2" lang="en" xml:lang="en"> 436 <div class="titlepage"> 437 <div> 438 <div> 439 <h3 class="title"><a id="election_flags"></a>Managing Election Policies</h3> 440 </div> 441 </div> 442 </div> 443 <p> 444 Before continuing, it is worth taking a look at the 445 446 <span> 447 startup election flags accepted by 448 <span><code class="methodname">DB_ENV->repgmr_start()</code>.</span> 449 450 These flags control how your replication application will 451 behave when it first starts up. 452 </span> 453 454 455 </p> 456 <p> 457 In the previous example, we specified 458 <code class="literal">DB_REP_ELECTION</code> 459 460 when we started replication. This causes the 461 application to try to find a master upon startup. If it 462 cannot, it calls for an election. In the event an 463 election is held, the environment receiving the most number of 464 votes will become the master. 465 </p> 466 <p> 467 There's some important points to make here: 468 </p> 469 <div class="itemizedlist"> 470 <ul type="disc"> 471 <li> 472 <p> 473 This 474 <span>flag</span> 475 476 only requires that other 477 environments in the replication group 478 participate in the vote. There is no 479 requirement that 480 <span class="emphasis"><em>all</em></span> such 481 environments participate. In other 482 words, if an environment 483 starts up, it can call for an 484 election, and select a master, even 485 if all other environment have not yet 486 joined the replication group. 487 </p> 488 </li> 489 <li> 490 <p> 491 It only requires a simple majority of 492 participating environments to elect a master. The number of 493 environments used to calculate the simple 494 majority is based on the value set for 495 496 <span><code class="methodname">DB_ENV->rep_set_nsites()</code>.</span> 497 498 499 500 501 This is always true of elections held using the Replication Manager. 502 </p> 503 </li> 504 <li> 505 <p> 506 As always, the environment participating in the election with the most 507 up-to-date log files is selected as 508 master. If an environment with more recent log files 509 has not yet joined the replication 510 group, it may not become the master. 511 </p> 512 </li> 513 </ul> 514 </div> 515 <p> 516 Any one of these points may be enough to cause a 517 less-than-optimum environment to be selected as master. 518 Therefore, to give you a better degree of control over 519 which environment becomes a master at application startup, 520 the Replication Manager offers the following start-up 521 <span>flags:</span> 522 523 </p> 524 <div class="informaltable"> 525 <table border="1" width="80%"> 526 <colgroup> 527 <col /> 528 <col /> 529 </colgroup> 530 <thead> 531 <tr> 532 <th>Flag</th> 533 <th>Description</th> 534 </tr> 535 </thead> 536 <tbody> 537 <tr> 538 <td> 539 <code class="literal">DB_REP_MASTER</code> 540 </td> 541 <td> 542 <p> 543 The application starts up and declares the environment to be a master 544 without calling for an election. It is an error for more 545 than one environment to start up using this flag, or for 546 an environment 547 to use this flag when a master already exists. 548 </p> 549 <p> 550 Note that no replication group should 551 <span class="emphasis"><em>ever</em></span> operate with more than 552 one master. 553 </p> 554 <p> 555 In the event that a environment attempts to become a 556 master when a master already exists, the 557 replication code will resolve the problem by 558 holding an election. Note, however, that there 559 is always a possibility of data loss in the face 560 of duplicate masters, because once a master is 561 selected, the environment that loses the election will 562 have to roll back any transactions committed 563 until it is in sync with the "real" master. 564 </p> 565 566 </td> 567 </tr> 568 <tr> 569 <td> 570 <code class="literal">DB_REP_CLIENT</code> 571 </td> 572 <td> 573 <p> 574 The application starts up and declares 575 the environment to be a replica without calling for 576 an election. Note that the environment 577 can still become a master if a subsequent 578 application starts up, calls for an 579 election, and this environment is elected 580 master. 581 </p> 582 </td> 583 </tr> 584 <tr> 585 <td> 586 <code class="literal">DB_REP_ELECTION</code> 587 </td> 588 <td> 589 <p> 590 As described above, the application starts up, 591 looks for a master, and if one is not found calls 592 for an election. 593 </p> 594 </td> 595 </tr> 596 </tbody> 597 </table> 598 </div> 599 </div> 600 <div class="sect2" lang="en" xml:lang="en"> 601 <div class="titlepage"> 602 <div> 603 <div> 604 <h3 class="title"><a id="thread_count"></a>Selecting the Number of Threads</h3> 605 </div> 606 </div> 607 </div> 608 <p> 609 Under the hood, the Replication Manager is threaded and you can 610 control the number of threads used to process messages received from 611 other replicas. The threads that the Replication Manager uses are: 612 </p> 613 <div class="itemizedlist"> 614 <ul type="disc"> 615 <li> 616 <p> 617 Incoming message thread. This thread 618 receives messages from the site's 619 socket and passes those messages to 620 message processing threads (see below) 621 for handling. 622 </p> 623 </li> 624 <li> 625 <p> 626 Outgoing message thread. Outgoing 627 messages are sent from whatever thread 628 performed a write to the database(s). 629 That is, the thread that called, for 630 example, 631 <code class="methodname">DB->put()</code> 632 633 634 is the thread that writes replication messages 635 about that fact to the socket. 636 </p> 637 <p> 638 Note that if this write activity would 639 cause the thread to be blocked due to 640 some condition on the socket, the Replication Manager 641 will hand the outgoing message to the 642 incoming message thread, and it will 643 then write the message to the socket. 644 This prevents your database write 645 threads from blocking due to abnormal 646 network I/O conditions. 647 </p> 648 </li> 649 <li> 650 <p> 651 Message processing threads are 652 responsible for parsing and then 653 responding to incoming replication 654 messages. Typically, a response will 655 include write activity to your 656 database(s), so these threads can be 657 busy performing disk I/O. 658 </p> 659 </li> 660 </ul> 661 </div> 662 <p> 663 Of these threads, the only ones that you have any 664 configuration control over are the message processing 665 threads. In this case, you can determine how many 666 of these threads you want to run. 667 </p> 668 <p> 669 It is always a bit of an art to decide on a thread count, 670 but the short answer is you probably do not need more 671 than three threads here, and it is likely that one will 672 suffice. That said, the best thing to do is set your 673 thread count to a fairly low number and then increase 674 it if it appears that your application will benefit 675 from the additional threads. 676 </p> 677 </div> 678 </div> 679 </div> 680 <div class="navfooter"> 681 <hr /> 682 <table width="100%" summary="Navigation footer"> 683 <tr> 684 <td width="40%" align="left"><a accesskey="p" href="simpleprogramlisting.html">Prev</a> </td> 685 <td width="20%" align="center"> </td> 686 <td width="40%" align="right"> <a accesskey="n" href="repmgr_init_example_c.html">Next</a></td> 687 </tr> 688 <tr> 689 <td width="40%" align="left" valign="top">Program Listing </td> 690 <td width="20%" align="center"> 691 <a accesskey="h" href="index.html">Home</a> 692 </td> 693 <td width="40%" align="right" valign="top"> Adding the Replication Manager to 694 <span>ex_rep_gsg_simple</span> 695 696 697 </td> 698 </tr> 699 </table> 700 </div> 701 </body> 702</html> 703