1<?xml version="1.0" encoding="UTF-8" standalone="no"?> 2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 3<html xmlns="http://www.w3.org/1999/xhtml"> 4 <head> 5 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> 6 <title>Database and log file archival</title> 7 <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> 8 <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> 9 <link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" /> 10 <link rel="up" href="transapp.html" title="Chapter��11.�� Berkeley DB Transactional Data Store Applications" /> 11 <link rel="prev" href="transapp_checkpoint.html" title="Checkpoints" /> 12 <link rel="next" href="transapp_logfile.html" title="Log file removal" /> 13 </head> 14 <body> 15 <div class="navheader"> 16 <table width="100%" summary="Navigation header"> 17 <tr> 18 <th colspan="3" align="center">Database and log file archival</th> 19 </tr> 20 <tr> 21 <td width="20%" align="left"><a accesskey="p" href="transapp_checkpoint.html">Prev</a>��</td> 22 <th width="60%" align="center">Chapter��11.�� 23 Berkeley DB Transactional Data Store Applications 24 </th> 25 <td width="20%" align="right">��<a accesskey="n" href="transapp_logfile.html">Next</a></td> 26 </tr> 27 </table> 28 <hr /> 29 </div> 30 <div class="sect1" lang="en" xml:lang="en"> 31 <div class="titlepage"> 32 <div> 33 <div> 34 <h2 class="title" style="clear: both"><a id="transapp_archival"></a>Database and log file archival</h2> 35 </div> 36 </div> 37 </div> 38 <p>The third component of the administrative infrastructure, archival for 39catastrophic recovery, concerns the recoverability of the database in 40the face of catastrophic failure. Recovery after catastrophic failure 41is intended to minimize data loss when physical hardware has been 42destroyed -- for example, loss of a disk that contains databases or log 43files. Although the application may still experience data loss in this 44case, it is possible to minimize it.</p> 45 <p>First, you may want to periodically create snapshots (that is, backups) 46of your databases to make it possible to recover from catastrophic 47failure. These snapshots are either a standard backup, which creates a 48consistent picture of the databases as of a single instant in time; or 49an on-line backup (also known as a <span class="emphasis"><em>hot</em></span> backup), which creates 50a consistent picture of the databases as of an unspecified instant 51during the period of time when the snapshot was made. The advantage of 52a hot backup is that applications may continue to read and write the 53databases while the snapshot is being taken. The disadvantage of a hot 54backup is that more information must be archived, and recovery based on 55a hot backup is to an unspecified time between the start of the backup 56and when the backup is completed.</p> 57 <p>Second, after taking a snapshot, you should periodically archive the 58log files being created in the environment. It is often helpful to 59think of database archival in terms of full and incremental filesystem 60backups. A snapshot is a full backup, whereas the periodic archival of 61the current log files is an incremental backup. For example, it might 62be reasonable to take a full snapshot of a database environment weekly 63or monthly, and archive additional log files daily. Using both the 64snapshot and the log files, a catastrophic crash at any time can be 65recovered to the time of the most recent log archival; a time long after 66the original snapshot.</p> 67 <p>To create a standard backup of your database that can be used to recover 68from catastrophic failure, take the following steps:</p> 69 <div class="orderedlist"> 70 <ol type="1"> 71 <li>Commit or abort all ongoing transactions.</li> 72 <li>Stop writing your databases until the backup has completed. Read-only 73operations are permitted, but no write operations and no filesystem 74operations may be performed (for example, the <a href="../api_reference/C/envremove.html" class="olink">DB_ENV->remove()</a> and 75<a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> methods may not be called).</li> 76 <li>Force an environment checkpoint (see the <a href="../api_reference/C/db_checkpoint.html" class="olink">db_checkpoint utility</a> for more 77information).</li> 78 <li> 79 <p> 80 Run the <a href="../api_reference/C/db_archive.html" class="olink">db_archive utility</a> with option <span class="bold"><strong>-s</strong></span> to identify all the database data 81 files, and copy them to a backup device such as CD-ROM, alternate 82 disk, or tape. 83 </p> 84 <p> 85 If the database files are stored in a separate directory from the 86 other Berkeley DB files, it may be simpler to archive the directory 87 itself instead of the individual files (see <a href="../api_reference/C/envset_data_dir.html" class="olink">DB_ENV->set_data_dir()</a> for 88 additional information). <span class="bold"><strong>Note: if any of the 89 database files did not have an open <a href="../api_reference/C/db.html" class="olink">DB</a> handle during 90 the lifetime of the current log files, the <a href="../api_reference/C/db_archive.html" class="olink">db_archive utility</a> will 91 not list them in its output!</strong></span> This is another reason 92 it may be simpler to use a separate database file directory and 93 archive the entire directory instead of archiving only the files 94 listed by the <a href="../api_reference/C/db_archive.html" class="olink">db_archive utility</a>. 95 </p> 96 </li> 97 <li> 98 Run the <a href="../api_reference/C/db_archive.html" class="olink">db_archive utility</a> with option <span class="bold"><strong>-l</strong></span> to 99 identify all the log files, and copy the last one (that is, the one 100 with the highest number) to a backup device such as CD-ROM, alternate 101 disk, or tape. 102</li> 103 </ol> 104 </div> 105 <p> 106 To create a <span class="emphasis"><em>hot</em></span> backup of your 107 database that can be used to recover from catastrophic failure, take 108 the following steps: 109</p> 110 <div class="orderedlist"> 111 <ol type="1"> 112 <li> 113 Archive your databases, as described in the previous step #4. You 114 do not have to halt ongoing transactions or force a checkpoint. As 115 this is a hot backup, and the databases may be modified during the 116 copy, the utility you use to copy the databases must read database 117 pages atomically (as described by <a class="xref" href="transapp_reclimit.html" title="Berkeley DB recoverability">Berkeley DB recoverability</a>). 118 </li> 119 <li> 120 <p> 121 Archive <span class="bold"><strong>all</strong></span> of the log files. 122 The order of these two operations is required, and the database 123 files must be archived <span class="bold"><strong>before</strong></span> 124 the log files. This means that if the database files and log 125 files are in the same directory, you cannot simply archive the 126 directory; you must make sure that the correct order of 127 archival is maintained. 128 </p> 129 <p> 130 To archive your log files, run the <a href="../api_reference/C/db_archive.html" class="olink">db_archive utility</a> using the 131 <span class="bold"><strong>-l</strong></span> option to identify all the 132 database log files, and copy them to your backup media. If the 133 database log files are stored in a separate directory from the 134 other database files, it may be simpler to archive the 135 directory itself instead of the individual files (see the 136 <a href="../api_reference/C/envset_lg_dir.html" class="olink">DB_ENV->set_lg_dir()</a> method for more information). 137 </p> 138 </li> 139 </ol> 140 </div> 141 <p>To minimize the archival space needed for log files when doing a hot 142backup, run db_archive to identify those log files which are not in use. 143Log files which are not in use do not need to be included when creating 144a hot backup, and you can discard them or move them aside for use with 145previous backups (whichever is appropriate), before beginning the hot 146backup.</p> 147 <p>After completing one of these two sets of steps, the database 148environment can be recovered from catastrophic failure (see 149<a class="xref" href="transapp_recovery.html" title="Recovery procedures">Recovery procedures</a> for more information).</p> 150 <p>For an example of a hot backup implementation in the Berkeley DB distribution, 151see the source code for the <a href="../api_reference/C/db_hotbackup.html" class="olink">db_hotbackup utility</a>.</p> 152 <p>To update either a hot or cold backup so that recovery from catastrophic 153failure is possible to a new point in time, repeat step #2 under the 154hot backup instructions and archive <span class="bold"><strong>all</strong></span> of the log files in the 155database environment. Each time both the database and log files are 156copied to backup media, you may discard all previous database snapshots 157and saved log files. Archiving additional log files does not allow you 158to discard either previous database snapshots or log files. Generally, 159updating a backup must be integrated with the application's log file 160removal procedures.</p> 161 <p>The time to restore from catastrophic failure is a function of the 162number of log records that have been written since the snapshot was 163originally created. Perhaps more importantly, the more separate pieces 164of backup media you use, the more likely it is that you will have a 165problem reading from one of them. For these reasons, it is often best 166to make snapshots on a regular basis.</p> 167 <p> 168 <span class="bold"> 169 <strong>Obviously, the reliability of your archive media will affect the safety 170of your data. For archival safety, ensure that you have multiple copies 171of your database backups, verify that your archival media is error-free 172and readable, and that copies of your backups are stored offsite!</strong> 173 </span> 174 </p> 175 <p>The functionality provided by the <a href="../api_reference/C/db_archive.html" class="olink">db_archive utility</a> is also 176available directly from the Berkeley DB library. The following code fragment 177prints out a list of log and database files that need to be archived:</p> 178 <pre class="programlisting">void 179log_archlist(DB_ENV *dbenv) 180{ 181 int ret; 182 char **begin, **list; 183 184 /* Get the list of database files. */ 185 if ((ret = dbenv->log_archive(dbenv, 186 &list, DB_ARCH_ABS | DB_ARCH_DATA)) != 0) { 187 dbenv->err(dbenv, ret, "DB_ENV->log_archive: DB_ARCH_DATA"); 188 exit (1); 189 } 190 if (list != NULL) { 191 for (begin = list; *list != NULL; ++list) 192 printf("database file: %s\n", *list); 193 free (begin); 194 } 195 196 /* Get the list of log files. */ 197 if ((ret = dbenv->log_archive(dbenv, 198 &list, DB_ARCH_ABS | DB_ARCH_LOG)) != 0) { 199 dbenv->err(dbenv, ret, "DB_ENV->log_archive: DB_ARCH_LOG"); 200 exit (1); 201 } 202 if (list != NULL) { 203 for (begin = list; *list != NULL; ++list) 204 printf("log file: %s\n", *list); 205 free (begin); 206 } 207}</pre> 208 </div> 209 <div class="navfooter"> 210 <hr /> 211 <table width="100%" summary="Navigation footer"> 212 <tr> 213 <td width="40%" align="left"><a accesskey="p" href="transapp_checkpoint.html">Prev</a>��</td> 214 <td width="20%" align="center"> 215 <a accesskey="u" href="transapp.html">Up</a> 216 </td> 217 <td width="40%" align="right">��<a accesskey="n" href="transapp_logfile.html">Next</a></td> 218 </tr> 219 <tr> 220 <td width="40%" align="left" valign="top">Checkpoints��</td> 221 <td width="20%" align="center"> 222 <a accesskey="h" href="index.html">Home</a> 223 </td> 224 <td width="40%" align="right" valign="top">��Log file removal</td> 225 </tr> 226 </table> 227 </div> 228 </body> 229</html> 230