1<?xml version="1.0" encoding="UTF-8" standalone="no"?> 2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 3<html xmlns="http://www.w3.org/1999/xhtml"> 4 <head> 5 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> 6 <title>Queue and Recno access method specific configuration</title> 7 <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> 8 <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> 9 <link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" /> 10 <link rel="up" href="am_conf.html" title="Chapter 2. Access Method Configuration" /> 11 <link rel="prev" href="hash_conf.html" title="Hash access method specific configuration" /> 12 <link rel="next" href="am.html" title="Chapter 3. Access Method Operations" /> 13 </head> 14 <body> 15 <div class="navheader"> 16 <table width="100%" summary="Navigation header"> 17 <tr> 18 <th colspan="3" align="center">Queue and Recno access method specific configuration</th> 19 </tr> 20 <tr> 21 <td width="20%" align="left"><a accesskey="p" href="hash_conf.html">Prev</a> </td> 22 <th width="60%" align="center">Chapter 2. 23 Access Method Configuration 24 </th> 25 <td width="20%" align="right"> <a accesskey="n" href="am.html">Next</a></td> 26 </tr> 27 </table> 28 <hr /> 29 </div> 30 <div class="sect1" lang="en" xml:lang="en"> 31 <div class="titlepage"> 32 <div> 33 <div> 34 <h2 class="title" style="clear: both"><a id="rq_conf"></a>Queue and Recno access method specific configuration</h2> 35 </div> 36 </div> 37 </div> 38 <div class="toc"> 39 <dl> 40 <dt> 41 <span class="sect2"> 42 <a href="rq_conf.html#am_conf_recno">Managing record-based databases</a> 43 </span> 44 </dt> 45 <dt> 46 <span class="sect2"> 47 <a href="rq_conf.html#am_conf_extentsize">Selecting a Queue extent size</a> 48 </span> 49 </dt> 50 <dt> 51 <span class="sect2"> 52 <a href="rq_conf.html#am_conf_re_source">Flat-text backing files</a> 53 </span> 54 </dt> 55 <dt> 56 <span class="sect2"> 57 <a href="rq_conf.html#am_conf_renumber">Logically renumbering records</a> 58 </span> 59 </dt> 60 </dl> 61 </div> 62 <p> 63 There are a series of configuration tasks which you can perform when 64 using the Queue and Recno access methods. They are described in the following sections. 65</p> 66 <div class="sect2" lang="en" xml:lang="en"> 67 <div class="titlepage"> 68 <div> 69 <div> 70 <h3 class="title"><a id="am_conf_recno"></a>Managing record-based databases</h3> 71 </div> 72 </div> 73 </div> 74 <p>When using fixed- or variable-length record-based databases, particularly 75with flat-text backing files, there are several items that the user can 76control. The Recno access method can be used to store either variable- 77or fixed-length data items. By default, the Recno access method stores 78variable-length data items. The Queue access method can only store 79fixed-length data items.</p> 80 <div class="sect3" lang="en" xml:lang="en"> 81 <div class="titlepage"> 82 <div> 83 <div> 84 <h4 class="title"><a id="id1591165"></a>Record Delimiters</h4> 85 </div> 86 </div> 87 </div> 88 <p>When using the Recno access method to store variable-length records, 89records read from any backing source file are separated by a specific 90byte value which marks the end of one record and the beginning of the 91next. This delimiting value is ignored except when reading records from 92a backing source file, that is, records may be stored into the database 93that include the delimiter byte. However, if such records are written 94out to the backing source file and the backing source file is 95subsequently read into a database, the records will be split where 96delimiting bytes were found.</p> 97 <p>For example, UNIX text files can usually be interpreted as a sequence of 98variable-length records separated by ASCII newline characters. This byte 99value (ASCII 0x0a) is the default delimiter. Applications may specify a 100different delimiting byte using the <a href="../api_reference/C/dbset_re_delim.html" class="olink">DB->set_re_delim()</a> method. If no 101backing source file is being used, there is no reason to set the 102delimiting byte value.</p> 103 </div> 104 <div class="sect3" lang="en" xml:lang="en"> 105 <div class="titlepage"> 106 <div> 107 <div> 108 <h4 class="title"><a id="id1590878"></a>Record Length</h4> 109 </div> 110 </div> 111 </div> 112 <p>When using the Recno or Queue access methods to store fixed-length 113records, the record length must be specified. Since the Queue access 114method always uses fixed-length records, the user must always set the 115record length prior to creating the database. Setting the record length 116is what causes the Recno access method to store fixed-length, not 117variable-length, records.</p> 118 <p>The length of the records is specified by calling the 119<a href="../api_reference/C/dbset_re_len.html" class="olink">DB->set_re_len()</a> method. The default length of the records is 0 bytes. 120Any record read from a backing source file or otherwise stored in the 121database that is shorter than the declared length will automatically be 122padded as described for the <a href="../api_reference/C/dbset_re_pad.html" class="olink">DB->set_re_pad()</a> method. Any record stored 123that is longer than the declared length results in an error. For 124further information on backing source files, see 125<a class="xref" href="rq_conf.html#am_conf_re_source" title="Flat-text backing files">Flat-text backing files</a>.</p> 126 </div> 127 <div class="sect3" lang="en" xml:lang="en"> 128 <div class="titlepage"> 129 <div> 130 <div> 131 <h4 class="title"><a id="id1591461"></a>Record Padding Byte Value</h4> 132 </div> 133 </div> 134 </div> 135 <p>When storing fixed-length records in a Queue or Recno database, a pad 136character may be specified by calling the <a href="../api_reference/C/dbset_re_pad.html" class="olink">DB->set_re_pad()</a> method. Any 137record read from the backing source file or otherwise stored in the 138database that is shorter than the expected length will automatically be 139padded with this byte value. If fixed-length records are specified but 140no pad value is specified, a space character (0x20 in the ASCII 141character set) will be used. For further information on backing source 142files, see <a class="xref" href="rq_conf.html#am_conf_re_source" title="Flat-text backing files">Flat-text backing files</a>.</p> 143 </div> 144 </div> 145 <div class="sect2" lang="en" xml:lang="en"> 146 <div class="titlepage"> 147 <div> 148 <div> 149 <h3 class="title"><a id="am_conf_extentsize"></a>Selecting a Queue extent size</h3> 150 </div> 151 </div> 152 </div> 153 <p>In Queue databases, records are allocated sequentially and directly 154mapped to an offset within the file storage for the database. As 155records are deleted from the Queue, pages will become empty and will 156not be reused in normal queue operations. To facilitate the reclamation 157of disk space a Queue may be partitioned into extents. Each extent is 158kept in a separate physical file.</p> 159 <p>Extent files are automatically created as needed and marked for deletion 160when the head of the queue moves off the extent. The extent will not 161be deleted until all processes close the extent. In addition, Berkeley DB 162caches a small number of extents that have been recently used; this may 163delay when an extent will be deleted. The number of extents left open 164depends on queue activity.</p> 165 <p>The extent size specifies the number of pages that make up each extent. 166By default, if no extent size is specified, the Queue resides in a 167single file and disk space is not reclaimed. In choosing an extent size 168there is a tradeoff between the amount of disk space used and the 169overhead of creating and deleting files. If the extent size is too 170small, the system will pay a performance penalty, creating and deleting 171files frequently. In addition, if the active part of the queue spans 172many files, all those files will need to be open at the same time, 173consuming system and process file resources.</p> 174 </div> 175 <div class="sect2" lang="en" xml:lang="en"> 176 <div class="titlepage"> 177 <div> 178 <div> 179 <h3 class="title"><a id="am_conf_re_source"></a>Flat-text backing files</h3> 180 </div> 181 </div> 182 </div> 183 <p>It is possible to back any Recno database (either fixed or variable 184length) with a flat-text source file. This provides fast read (and 185potentially write) access to databases that are normally created and 186stored as flat-text files. The backing source file may be specified by 187calling the <a href="../api_reference/C/dbset_re_source.html" class="olink">DB->set_re_source()</a> method.</p> 188 <p>The backing source file will be read to initialize the database. In the 189case of variable length records, the records are assumed to be separated 190as described for the <a href="../api_reference/C/dbset_re_delim.html" class="olink">DB->set_re_delim()</a> method. For example, standard 191UNIX byte stream files can be interpreted as a sequence of variable 192length records separated by ASCII newline characters. This is the 193default.</p> 194 <p>When cached data would normally be written back to the underlying 195database file (for example, when the <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a> or 196<a href="../api_reference/C/dbsync.html" class="olink">DB->sync()</a> methods are called), the in-memory copy of the database will 197be written back to the backing source file.</p> 198 <p>The backing source file must already exist (but may be zero-length) when 199<a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> is called. By default, the backing source file is read 200lazily, that is, records are not read from the backing source file until 201they are requested by the application. If multiple processes (not 202threads) are accessing a Recno database concurrently and either 203inserting or deleting records, the backing source file must be read in 204its entirety before more than a single process accesses the database, 205and only that process should specify the backing source file as part of 206the <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> call. This can be accomplished by calling the 207<a href="../api_reference/C/dbset_flags.html" class="olink">DB->set_flags()</a> method with the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_SNAPSHOT" class="olink">DB_SNAPSHOT</a> flag.</p> 208 <p>Reading and writing the backing source file cannot be transactionally 209protected because it involves filesystem operations that are not part of 210the Berkeley DB transaction methodology. For this reason, if a temporary 211database is used to hold the records (a NULL was specified as the file 212argument to <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a>), <span class="bold"><strong>it is possible to lose the 213contents of the backing source file if the system crashes at the right 214instant</strong></span>. If a permanent file is used to hold the database (a filename 215was specified as the file argument to <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a>), normal database 216recovery on that file can be used to prevent information loss. It is 217still possible that the contents of the backing source file itself will 218be corrupted or lost if the system crashes.</p> 219 <p>For all of the above reasons, the backing source file is generally used 220to specify databases that are read-only for Berkeley DB applications, and that 221are either generated on the fly by software tools, or modified using a 222different mechanism such as a text editor.</p> 223 </div> 224 <div class="sect2" lang="en" xml:lang="en"> 225 <div class="titlepage"> 226 <div> 227 <div> 228 <h3 class="title"><a id="am_conf_renumber"></a>Logically renumbering records</h3> 229 </div> 230 </div> 231 </div> 232 <p>Records stored in the Queue and Recno access methods are accessed by 233logical record number. In all cases in Btree databases, and optionally 234in Recno databases (see the <a href="../api_reference/C/dbset_flags.html" class="olink">DB->set_flags()</a> method and the 235<a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_RENUMBER" class="olink">DB_RENUMBER</a> flag for more information), record numbers are 236mutable. This means that the record numbers may change as records are 237added to and deleted from the database. The deletion of record number 2384 causes any records numbered 5 and higher to be renumbered downward by 2391; the addition of a new record after record number 4 causes any 240records numbered 5 and higher to be renumbered upward by 1. In all 241cases in Queue databases, and by default in Recno databases, record 242numbers are not mutable, and the addition or deletion of records to the 243database will not cause already-existing record numbers to change. For 244this reason, new records cannot be inserted between already-existing 245records in databases with immutable record numbers.</p> 246 <p>Cursors pointing into a Btree database or a Recno database with mutable 247record numbers maintain a reference to a specific record, rather than 248a record number, that is, the record they reference does not change as 249other records are added or deleted. For example, if a database contains 250three records with the record numbers 1, 2, and 3, and the data items 251"A", "B", and "C", respectively, the deletion of record number 2 ("B") 252will cause the record "C" to be renumbered downward to record number 2. 253A cursor positioned at record number 3 ("C") will be adjusted and 254continue to point to "C" after the deletion. Similarly, a cursor 255previously referring to the now deleted record number 2 will be 256positioned between the new record numbers 1 and 2, and an insertion 257using that cursor will appear between those records. In this manner 258records can be added and deleted to a database without disrupting the 259sequential traversal of the database by a cursor.</p> 260 <p>Only cursors created using a single <a href="../api_reference/C/db.html" class="olink">DB</a> handle can adjust each 261other's position in this way, however. If multiple <a href="../api_reference/C/db.html" class="olink">DB</a> handles 262have a renumbering Recno database open simultaneously (as when multiple 263processes share a single database environment), a record referred to by 264one cursor could change underfoot if a cursor created using another 265<a href="../api_reference/C/db.html" class="olink">DB</a> handle inserts or deletes records into the database. For 266this reason, applications using Recno databases with mutable record 267numbers will usually make all accesses to the database using a single 268<a href="../api_reference/C/db.html" class="olink">DB</a> handle and cursors created from that handle, or will 269otherwise single-thread access to the database, for example, by using 270the Berkeley DB Concurrent Data Store product.</p> 271 <p>In any Queue or Recno databases, creating new records will cause the 272creation of multiple records if the record number being created is more 273than one greater than the largest record currently in the database. For 274example, creating record number 28, when record 25 was previously the 275last record in the database, will implicitly create records 26 and 27 276as well as 28. All first, last, next and previous cursor operations 277will automatically skip over these implicitly created records. So, if 278record number 5 is the only record the application has created, 279implicitly creating records 1 through 4, the <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> method with the 280<a href="../api_reference/C/dbcget.html#dbcget_DB_FIRST" class="olink">DB_FIRST</a> flag will return record number 5, not record number 1. 281Attempts to explicitly retrieve implicitly created records by their 282record number will result in a special error return, 283<a class="link" href="program_errorret.html#program_errorret.DB_KEYEMPTY">DB_KEYEMPTY</a>.</p> 284 <p>In any Berkeley DB database, attempting to retrieve a deleted record, using 285a cursor positioned on the record, results in a special error return, 286<a class="link" href="program_errorret.html#program_errorret.DB_KEYEMPTY">DB_KEYEMPTY</a>. In addition, when using Queue databases or Recno 287databases with immutable record numbers, attempting to retrieve a deleted 288record by its record number will also result in the <a class="link" href="program_errorret.html#program_errorret.DB_KEYEMPTY">DB_KEYEMPTY</a> 289return.</p> 290 </div> 291 </div> 292 <div class="navfooter"> 293 <hr /> 294 <table width="100%" summary="Navigation footer"> 295 <tr> 296 <td width="40%" align="left"><a accesskey="p" href="hash_conf.html">Prev</a> </td> 297 <td width="20%" align="center"> 298 <a accesskey="u" href="am_conf.html">Up</a> 299 </td> 300 <td width="40%" align="right"> <a accesskey="n" href="am.html">Next</a></td> 301 </tr> 302 <tr> 303 <td width="40%" align="left" valign="top">Hash access method specific configuration </td> 304 <td width="20%" align="center"> 305 <a accesskey="h" href="index.html">Home</a> 306 </td> 307 <td width="40%" align="right" valign="top"> Chapter 3. 308 Access Method Operations 309 </td> 310 </tr> 311 </table> 312 </div> 313 </body> 314</html> 315