1<!--"$Id: 2.5.9.html,v 1.4 2007/05/17 18:17:18 bostic Exp $ (Sleepycat) $Date: 2007/05/17 18:17:18 $"--> 2<html> 3<head> 4<title>The Berkeley DB Package: DB 2.5.9 Change Log</title> 5<meta name="description" content="DB: A database programmatic toolkit."> 6<meta name="keywords" content="embedded,database,programmatic,toolkit,b+tree,btree,hash,hashing,transaction,transactions,locking,logging,access method,access methods"> 7</head> 8<body bgcolor=white> 9 10<h3 align=center>Berkeley DB 2.5.9 Change Log</h3> 11 12<h3>Interface Additions in Berkeley DB 2.5.9:</h3> 13 14<ol> 15 16<p><li> 17A new flag is available in this release for the Berkeley DB cursor 18key/data retrieval interface: <b>DB_NEXT_DUP</b>. This flag causes the 19DBcursor->c_get routine to return the next duplicate in a list of 20duplicates, and DB_NOTFOUND if there are no additional duplicates to 21return. 22 23<p><li> 24A new flag is available in this release for the Berkeley DB cursor 25key/data retrieval interfaces: <b>DB_GET_BOTH</b>. This flag causes the 26DB->get and DBcursor->c_get routines to return success only if <b>both</b> 27the specified key and data items match the entry in the database. 28 29<p><li> 30A new flag is available in this release for the Berkeley DB key/data 31retrieval interfaces: <b>DB_RMW</b>. This flag causes the DB->get and 32DBcursor->c_get routines to acquire write locks instead of read locks when 33doing the retrieval. Setting this flag may decrease the likelihood of 34deadlock during a read-modify-write cycle by immediately acquiring the 35write lock during the read part of the cycle, so that another thread of 36control acquiring a read lock for the same item, in its own 37read-modify-write cycle, will not result in deadlock. 38 39<p><li> 40A new flag is available in this release: <b>DB_DUPSORT</b>. This flag 41causes duplicate records to be maintained in sorted order. By default, 42the sort order is the same default lexical sort used by the Btree access 43method. A new field, in the DB_INFO structure passed to the db_open 44routine, is available in this release as well: <b>dup_compare</b>. This 45field is a sort function that is optionally used to sort duplicate data 46items. It is intended to allow applications to maintain duplicates in a 47non-standard sort order. 48 49<p><li> 50A new interface is available in this release, <b>DB->join</b>. This 51interface takes a set of Berkeley DB cursors as arguments, and returns a 52specialized Berkeley DB cursor whose get function performs a database join 53on the records referenced by the set of cursors. 54 55<p><li> 56A new field is available in this release, set in the Berkeley DB structure 57returned by db_open, <b>DB->byteswapped</b>. This field is set if the 58underlying database was not in the native host byte order, and can be used 59by the application to determine if its stored data will require host-order 60cleanups before use. 61 62<p><li> 63The dbmclose() interface has been added to the Berkeley DB dbm/ndbm 64compatibility interface, for application compatibility with the Sun 65Microsystems Solaris and other dbm/ndbm interfaces. 66 67</ol> 68 69<h3>Interface Changes in Berkeley DB 2.5.9:</h3> 70 71<ol> 72 73<p><li> 74Previous Berkeley DB releases have been inconsistent with respect to which 75<b>DBT structure</b> flags may be specified to which Berkeley DB 76interfaces. For example, calling DB->put with the DB_DBT_MALLOC flag 77specified makes no sense, and DB has been inconsistent historically as to 78whether this was treated as an error or simply ignored. 79<p> 80As of this release, inappropriate flags in the DBT structure will simply 81be ignored. This is intended to make it easy to, for example, retrieve 82a key/data pair and then use the data DBT as the key DBT for another 83database without having to reinitialize the flags in the DBT. 84<p> 85Previous Berkeley DB releases also required that threaded applications 86always set the DB_DBT_MALLOC or DB_DBT_USERMEM flags in DBT structures 87when retrieving key/data items. As of this release, specifying 88DB_DBT_MALLOC or DB_DBT_USERMEM is only required when using the non-cursor 89DB interfaces (for example, DB->get). When using cursor interfaces 90(for example, DBcursor->c_get), the flags are not required, as memory 91in which the key/data items are returned is allocated and maintained on 92a per-cursor basis. 93 94<p><li> 95Berkeley DB log files are now named <b>log.XXXXXXXXXX</b> instead of 96<b>log.XXXXX</b>, in order to ensure that applications do not run out of 97log filename space. 98<p><font color=red> 99This change is transparent to applications, but may NOT be transparent to 100local shell scripts and utilities. 101</font> 102 103<p><li> 104Previous Berkeley DB releases returned statistics for the Btree databases 105that were only valid for the lifetime of the handle with which they were 106requested, that is, these statistics as returned for a particular Berkeley 107DB handle would only reflect database operations done by that Berkeley DB 108handle and any cursors associated with it. 109<p> 110The following statistics have been removed from the returned Btree 111statistical information: <b>bt_freed</b>, <b>bt_pfxsaved</b>, 112<b>bt_split</b>, <b>bt_rootsplit</b>, <b>bt_fastsplit</b>, 113<b>bt_added</b>, <b>bt_deleted</b>, <b>bt_get</b>, <b>bt_cache_hit</b>, 114<b>bt_cache_miss</b>. If any of these are sufficiently useful to 115application writers that they should be put back into the system, please 116let us know. 117<p><font color=red> 118This change is NOT transparent to applications. 119</font> 120 121<p><li> 122Previous Berkeley DB releases did not support embedded white space in 123Berkeley DB environment configuration strings. As of this release, 124configuration NAME/VALUE strings are still separated by one or more 125whitespace characters (which are discarded), but the VALUE string may 126contain embedded whitespace characters and is terminated by trailing 127whitespace characters and a newline character, both of which are also 128discarded. In addition, empty lines and lines whose first character is 129a whitespace or hash (<b>#</b>) character, in the Berkeley DB 130configuration, file are discarded. 131<p><font color=red> 132This change is potentially NOT transparent to applications. 133</font> 134 135<p><li> 136The DB_REGION_INIT flag to the db_value_set interface has been enhanced 137to write a byte to each page in the region. This allows applications 138to use DB_REGION_INIT to ensure that there is sufficient disk space for 139the backing region file. 140 141</ol> 142 143<h3>Berkeley DB Environment failures:</h3> 144<blockquote> 145<p> 146There exists a class of errors that Berkeley DB considers fatal to an 147entire Berkeley DB environment. An example of this type of error is a 148log write failure due to the disk being out of free space. The only way 149to recover from these failures is for the application to exit, run 150recovery of the Berkeley DB environment, and re-enter DB. (It is not 151strictly necessary that the application exit, although that is the only 152way to recover system resources, for example, file descriptors and 153memory, currently allocated by Berkeley DB.) 154<p> 155In previous Berkeley DB releases, the only way an application could 156determine that a fatal error had occurred was to monitor Berkeley DB 157function return values, looking for unexpected ones, such as ENOSPC, or 158EPERM (which has historically been returned by Berkeley DB to indicate a 159potential underlying database corruption). 160<p> 161As of this release, we have added a new error return value, DB_RUNRECOVERY. 162 This error can be returned by <b>any</b> Berkeley DB interface. If a 163fatal error occurs, DB_RUNRECOVERY will then be returned from all 164subsequent DB calls made by any threads or processes participating in the 165DB environment. 166<p> 167The EPERM error return no longer has a special meaning in Berkeley DB. 168<p> 169Optionally, applications may also specify a fatal-error callback function 170by setting the <b>db_paniccall</b> field of the DB_ENV structure before 171initializing the environment with db_appinit (DbEnv::appinit). This 172callback function will be called with two arguments: the DB_ENV structure 173associated with the environment and the errno value associated with the 174underlying error that caused the problem. 175<p> 176Applications can handle fatal errors in one of two ways: by checking for 177DB_RUNRECOVERY as part of their normal Berkeley DB error return checking, 178or, in applications that have no cleanup processing of their own, by 179simply exiting the application when the callback function is called. 180<p> 181We would be very interested in any comments that you'd care to make on 182this interface change, in particular, any comments on the sufficiency 183of the interface for your Berkeley DB application. 184<p><font color=red> 185This change is NOT transparent to applications. 186</font> 187</blockquote> 188 189<h3>Documentation Changes:</h3> 190 191<ol> 192 193<p><li> 194The Berkeley DB documentation has been completely reworked. It is no 195longer available in flat text, UNIX roff or PostScript formats, but is 196now <b>only available in HTML format</b>. To use the Berkeley DB 197documentation, point your browser to the Berkeley DB distribution or 198installation directory <b>db-2.5.9/docs/index.html</b>. This release 199also includes the beginnings of the Berkeley DB Reference Guide, as well 200as the manual pages. 201 202</ol> 203 204<h3>B+tree Access Method Bug Fixes:</h3> 205 206<ol> 207 208<p><li> 209Cursor delete operations were not necessarily being undone after deadlock, 210potentially leading to incorrect data. 211 212<p><li> 213Deleted, off-page duplicate items could be recovered incorrectly, 214potentially leading to incorrect data. 215 216<p><li> 217Log records could be written outside of a transaction under some 218circumstances, potentially corrupting the log so that recovery would fail. 219 220<p><li> 221Completely emptying large trees could cause corruption of the database 222root page during the final reverse split. 223 224<p><li> 225Failure during page split could leave cursors referencing incorrect data. 226 227<p><li> 228Retrieving records based on logical record number could return incorrect 229data if logically adjacent records had previously been deleted. 230 231<p><li> 232The Btree access method is more aggressive in this release about 233discarding locks within transactions that are not needed for correctness. 234This change significantly decreases the probability of deadlock for some 235applications. 236 237</ol> 238 239<h3>Hash Access Method Bug Fixes:</h3> 240 241<ol> 242 243<p><li> 244Storing duplicate data items using the DB_CURRENT flag could result in 245incorrect data. 246 247<p><li> 248Cursors and their locks did not always return unchanged on operation 249failure. 250 251<p><li> 252Entering a sufficient number of duplicate data items into the database 253could result in incorrect cursor positioning and/or a corrupted database. 254 255</ol> 256 257<h3>Recno Access Method Bug Fixes:</h3> 258 259<ol> 260 261<p><li> 262The logical record number returned from DB_APPEND calls was stored into 263library memory instead of into the user-specified memory. 264 265<p><li> 266The memory in which the backing source filename was stored could be freed 267multiple times, potentially leading to application core dump. 268 269</ol> 270 271<h3>General Access Method Bug Fixes:</h3> 272 273<ol> 274 275<p><li> 276Incorrectly treated setting the database cachesize in the presence of a 277Berkeley DB environment to be an error, even if the environment didn't 278not initialize a shared memory buffer pool. 279 280<p><li> 281Using the DBcursor->c_get interface with the DB_KEYFIRST or DB_KEYLAST 282flags to insert a new key into the database would fail. 283 284<p><li> 285In previous Berkeley DB releases each cursor operation (when not part of 286a transaction) potentially used a different locker ID, making it possible 287for cursor operations to lock against themselves. In the 2.5.9 release, 288the cursor locker ID is maintained for the life of the cursor, instead. 289 290<p><li> 291The optional user-specified transaction recovery function was not being 292called during Berkeley DB recovery, and when using the db_printlog 293utility. 294 295<p><li> 296Pages of duplicate data items were incorrectly split and logged/recovered, 297potentially leading to database corruption. 298 299<p><li> 300During recovery, deleted database files could cause recovery to fail. 301 302</ol> 303 304<h3>C++ API Changes and Bug Fixes</h3> 305 306<ol> 307 308<p><li> 309A DbEnv::version method has been added to allow access to major, minor 310and patch numbers for the current version. 311 312<p><li> 313The DbEnv class has been cleaned up so that inappropriate get and set 314methods have been removed: 315<p> 316<blockquote> 317DbEnv::get_data_cnt<br> 318DbEnv::get_data_dir<br> 319DbEnv::get_data_next<br> 320DbEnv::get_flags<br> 321DbEnv::get_home<br> 322DbEnv::get_log_dir<br> 323DbEnv::get_tmp_dir<br> 324DbEnv::set_data_cnt<br> 325DbEnv::set_data_dir<br> 326DbEnv::set_data_next<br> 327DbEnv::set_flags<br> 328DbEnv::set_home<br> 329DbEnv::set_log_dir<br> 330DbEnv::set_tmp_dir<br> 331</blockquote> 332<p> 333These methods are unneeded because the constructor with arguments, 334or the appinit() method, can be used to set this information. 335<p> 336<blockquote> 337DbEnv::get_errcall<br> 338DbEnv::get_errfile<br> 339DbEnv::get_error_model<br> 340DbEnv::get_error_stream<br> 341DbEnv::get_errpfx<br> 342DbEnv::get_lg_max<br> 343DbEnv::get_lk_conflicts<br> 344DbEnv::get_lk_detect<br> 345DbEnv::get_lk_max<br> 346DbEnv::get_lk_modes<br> 347DbEnv::get_lorder<br> 348DbEnv::get_mp_mmapsize<br> 349DbEnv::get_mp_size<br> 350DbEnv::get_tx_max<br> 351DbEnv::get_tx_recover<br> 352DbEnv::get_verbose<br> 353</blockquote> 354<p> 355These get methods accessed information that was never set by Berkeley DB. 356 357<p><li> 358Remaining DbEnv::set_* methods may throw an exception if they are called 359after the environment has been initialized (either via appinit or the 360constructor with arguments). 361 362<p><li> 363The DbInfo class has been reworked so that inappropriate get methods 364have been removed. These get methods accessed information that was 365never set by Berkeley DB. 366<p> 367<blockquote> 368DbInfo::get_bt_compare<br> 369DbInfo::get_bt_maxkey<br> 370DbInfo::get_bt_minkey<br> 371DbInfo::get_bt_prefix<br> 372DbInfo::get_cachesize<br> 373DbInfo::get_flags<br> 374DbInfo::get_h_ffactor<br> 375DbInfo::get_h_hash<br> 376DbInfo::get_h_nelem<br> 377DbInfo::get_lorder<br> 378DbInfo::get_malloc<br> 379DbInfo::get_pagesize<br> 380DbInfo::get_re_delim<br> 381DbInfo::get_re_len<br> 382DbInfo::get_re_pad<br> 383DbInfo::get_re_source<br> 384</blockquote> 385<p> 386Methods to get and set underlying lock identifiers in a DbLock have 387been removed, as lock identifiers should be completely opaque to the 388application. 389 390</ol> 391 392<h3>Java API Changes and Bug Fixes</h3> 393 394<ol> 395 396<p><li> 397The DB_SET_RANGE flag did not correctly return data items. 398 399<p><li> 400Db.stat() is now declared to return an Object. The object returned is of 401type DbBtreeStat if the file was created using Db.DB_BTREE. In the 402future, this will return other types, for example, DbHashStat. 403<p> 404The DbBtreeStat, DbLockStat, DbMpoolFStat, DbMpoolStat and DbTxnStat classes 405have been changed to allow direct access to their data members. DbLogStat 406is a new class. 407 408<p><li> 409The DbEnv class has been reworked and all inappropriate get and set 410methods have been removed: 411<p> 412<blockquote> 413DbEnv.get_data_cnt<br> 414DbEnv.get_data_next<br> 415DbEnv.get_flags<br> 416DbEnv.get_home<br> 417DbEnv.get_log_dir<br> 418DbEnv.get_tmp_dir<br> 419DbEnv.set_data_cnt<br> 420DbEnv.set_data_next<br> 421DbEnv.set_flags<br> 422DbEnv.set_home<br> 423DbEnv.set_log_dir<br> 424DbEnv.set_tmp_dir<br> 425</blockquote> 426<p> 427These methods are unneeded because the constructor with arguments, or the 428appinit() method, can be used to set this information. 429 430<p> 431<blockquote> 432DbEnv.get_errcall<br> 433DbEnv.get_errpfx<br> 434DbEnv.get_lg_max<br> 435DbEnv.get_lk_conflicts<br> 436DbEnv.get_lk_detect<br> 437DbEnv.get_lk_max<br> 438DbEnv.get_lk_modes<br> 439DbEnv.get_lorder<br> 440DbEnv.get_mp_mmapsize<br> 441DbEnv.get_mp_size<br> 442DbEnv.get_tx_max<br> 443DbEnv.get_verbose<br> 444</blockquote> 445<p> 446These get methods used to access information that was never set by 447Berkeley DB. 448<p> 449The DbEnv.get_java_version_string method has been removed, and the Java 450part of Berkeley DB no longer maintains its own version information. 451 452<p><li> 453Remaining DbEnv.set_* methods may throw a DbException if they are called 454after the environment has been initialized (either via appinit or the 455constructor with arguments). 456 457<p><li> 458The DbInfo class has been reworked so that inappropriate get methods 459have been removed. These get methods used to access information that 460was never set by Berkeley DB. 461<p> 462<blockquote> 463DbInfo.get_bt_maxkey<br> 464DbInfo.get_bt_minkey<br> 465DbInfo.get_cachesize<br> 466DbInfo.get_flags<br> 467DbInfo.get_h_ffactor<br> 468DbInfo.get_h_hash<br> 469DbInfo.get_h_nelem<br> 470DbInfo.get_lorder<br> 471DbInfo.get_pagesize<br> 472DbInfo.get_re_delim<br> 473DbInfo.get_re_len<br> 474DbInfo.get_re_pad<br> 475DbInfo.get_re_source<br> 476</blockquote> 477 478<p><li> 479Methods to get and set underlying lock identifiers in a DbLock have 480been removed, as lock identifiers should be completely opaque to the 481application. 482 483<p><li> 484The DbRunRecoveryException class has been added as a subclass of 485DbException. A DbRunRecoveryException object will be thrown when a 486fatal error occurs in Berkeley DB, requiring recovery to be performed. 487 488</ol> 489 490<h3>Shared Memory Buffer Pool Subsystem Bug Fixes:</h3> 491 492<ol> 493 494<p><li> 495It was possible for threads opening and closing databases in a fairly full 496buffer cache to free memory that was still in use, resulting in 497application failure. 498 499<p><li> 500If the memp_trickle() interface was unable to find a single buffer to 501flush in the entire buffer list, it would return with the shared memory 502region mutex locked. 503 504<p><li> 505Opening underlying files of certain sizes in the buffer pool would 506incorrectly fail. 507 508</ol> 509 510<h3>Locking Subsystem Bug Fixes:</h3> 511 512<ol> 513 514<p><li> 515If the a locker being forced to wait does not currently hold any locks, 516the deadlock detector is no longer run. 517 518</ol> 519 520<h3>Logging Subsystem Bug Fixes:</h3> 521 522<ol> 523 524<p><li> 525If the first log get call after database recovery used the DB_NEXT flag, 526it would fail. 527 528<p><li> 529If databases were opened multiple times without intervening closes, 530recovery could fail. 531 532<p><li> 533A memory leak in the log_archive interface has been fixed. 534 535</ol> 536 537<h3>Additional Bug Fixes:</h3> 538 539<ol> 540 541<p><li> 542Windows/NT: using shared anonymous memory did not work correctly between 543processes sharing the database. 544 545<p><li> 546Utilities: Signal handling and Berkeley DB region exit was incorrect. 547Among other issues, the db_load utility could exit holding a region mutex. 548 549<p><li> 550Dbm/Ndbm: System error values were not always being correctly returned. 551 552</ol> 553 554<h3>System Porting and Build Procedure Changes:</h3> 555 556<ol> 557 558<p><li> 559A alpha-release port to VMS has been added to the Berkeley DB 560distribution. The port has not yet run the Berkeley DB test 561suite, but there are no known problems. 562 563<p><li> 564Berkeley DB now uses the pstat_getdynamic(2) interface on Hewlett-Packard 565HP/UX systems to detect the presence of multiple processors. 566 567<p><li> 568For performance reasons, the Berkeley DB release now uses the Sun 569Microsystems Solaris pread(2) and pwrite(2) UNIX interfaces, if they 570are available. 571 572<p><li> 573Berkeley DB now compiles with the -D_THREAD_SAFE C preprocessor flag 574and loads with the libc_r.a C library by default on FreeBSD systems. 575 576<p><li> 577For portability reasons, shared memory segments allocated using the UNIX 578shmget(2) function are now allocated as IPC_PRIVATE. (Apparently, marking 579them as IPC_PRIVATE does not affect that they are available to other 580processes.) 581 582<p><li> 583The standard UNIX install for the Berkeley DB library now installs 584Berkeley DB into its own hierarchy instead of into separate local 585directories. By default, the install locations are: 586<p> 587<table border=1> 588<tr> 589 <th>Location</th> 590 <th>Contents</th> 591</tr><tr> 592 <td>/usr/local/BerkeleyDB/bin</td> 593 <td>binaries</td> 594</tr><tr> 595 <td>/usr/local/BerkeleyDB/include</td> 596 <td>include files</td> 597</tr><tr> 598 <td>/usr/local/BerkeleyDB/lib</td> 599 <td>libraries</td> 600</tr><tr> 601 <td>/usr/local/BerkeleyDB/docs</td> 602 <td>HTML documentation</td> 603</tr> 604</table> 605 606<p><li> 607For portability reasons, the standard UNIX Berkeley DB library archive is 608built with the <b>-cr</b> options in this release, instead of the 609<b>-cq</b> options as done previously. 610 611<p><li> 612The standard UNIX Berkeley DB configuration will now automatically detect 613and use gcc if no compiler named cc is found. 614 615<p><li> 616When the C pre-processor DIAGNOSTIC value is #defined, memory is 617overwritten with a 0xdb pattern instead of a 0xff pattern. 618 619</ol> 620 621<h3>Additional Changes: db_dump</h3> 622 623<ol> 624 625<p><li> 626The db_dump utility has a new option, <b>-N</b>. This option allows 627db_dump to be run without acquiring any shared region mutexes. This 628option is intended for debugging use only. 629 630<p><li> 631The db_dump utility now allows a Berkeley DB environment directory to be 632specified (the <b>-h</b> option) at the same time as the "debugging 633output" option (the <b>-d</b> option). 634 635<p><li> 636The db_dump utility now uses the shared memory buffer pool region if a 637Berkeley DB environment directory is specified, which allows users to see 638the current state of the database instead of only the database state that 639has already been flushed to disk. 640 641</ol> 642 643</body> 644</html> 645