1<!--$Id: 4.3.29.html,v 1.34 2007/05/17 18:17:18 bostic Exp $--> 2<html> 3<head> 4<title>The Berkeley DB Package: DB 4.3.29 Change Log</title> 5<meta name="description" content="Berkeley DB: A database programmatic toolkit."> 6<meta name="keywords" content="embedded,database,programmatic,toolkit,b+tree,btree,hash,hashing,transaction,transactions,locking,logging,access method,access methods"> 7</head> 8<body bgcolor=white> 9 10<h3 align=center>Berkeley DB 4.3.29 Change Log</h3> 11 12<h3>Database or Log File On-Disk Format Changes:</h3> 13<ol> 14 15<li>The on-disk log format has changed. 16 17</ol> 18<h3>New Features:</h3> 19<ol> 20 21<li>Add support for light weight, transactionally protected Sequence 22Number generation. [#5739] 23<li>Add support for Degree 2 isolation. [#8689] 24<li>Add election generation information to replication to support Paxos 25compliance. [#9068] 26<li>Add support for 64-bit and ANSI C implementations of the RPCGEN 27utility. [#9548] 28 29</ol> 30<h3>Database Environment Changes:</h3> 31<ol> 32 33<li>Fix a bug where the permissions on system shared memory segments did 34not match the mode specified in the DB_ENV->open() method. [#8921] 35<li>Add a new return error from the DB_ENV->open() method call, 36DB_VERSION_MISMATCH, which is returned in the case of an application 37compiled under one version of Berkeley DB attempting to open an 38environment created under a different version. [#9077] 39<li>Add support for importing databases from a transactional database 40environment into a different environment. [#9324] 41<li>Fix a bug where a core dump could occur if a zero-length database 42environment name was specified. [#9233] 43<li>Increase the number of environment regions to 100. [#9297] 44<li>Remove the DB_ENV->set_verbose() method flag DB_VERB_CHKPOINT. 45[#9405] 46<li>Fix bugs where database environment getters could return incorrect 47information after the database environment was opened, if a different 48thread of control changed the database environment values. Fix bugs 49where database environment getter/setter functions could race with other 50threads of control. [#9724] 51<li>Change the DbEnv.set_lk_detect method to match the DbEnv.open 52semantics. That is, DbEnv.set_lk_detect may be called after the database 53environment is opened, allowing applications to configure automatic 54deadlock detection if it has not yet been configured. [#9724] 55<li>Fix cursor locks for environments opened without DB_THREAD so that 56they use the same locker ID. This eliminates many common cases of 57application self-deadlock, particularly in CDS. [#9742] 58<li>Fix a bug in DB->get_env() in the C API where it could return an 59error when it should only return the DB_ENV handle. C++ and Java are 60unchanged. [#9828] 61<li>Fix a bug where we only need to initialize the cryptographic memory 62region when MPOOL, Log or Transactions have been configured. [#9872] 63<li>Change private database environments at process startup to only 64allocate the heap memory required at any particular time, rather than 65always allocating the maximum amount of heap memory configured for the 66environment. [#9889] 67<li>Add a method to create nonexistent intermediate directories when 68opening database files. [#9898] 69<li>Add support for in-memory logging within database environments. 70[#9927] 71<li>Change Berkeley DB so configuring a database environment for 72automatic log file removal affects all threads in the environment, not 73just the DbEnv handle in which the configuration call is made. [#9947] 74<li>Change the signature of the error callback passed to the 75DB_ENV->set_errcall and DB->set_errcall methods to add a DB_ENV handle, 76to provide database environment context for the callback function. 77[#10025] 78<li>Fix a race condition between DB->close and DB->{remove,rename} on 79filesystems that don't allow file operations on open files (such as 80Windows). [#10180] 81<li>Add a DB_DSYNC_LOG flag to the DbEnv::set_flags method, which 82configures O_DSYNC on POSIX systems and FILE_FLAG_WRITE_THROUGH on Win32 83systems. This offers significantly better performance for some 84applications on certain Solaris/filesystem combinations. [#10205] 85<li>Fix a bug where calling the DB or DBEnv database remove or rename 86methods could cause a transaction checkpoint or cache flush to fail. 87[#10286] 88<li>Change file operations not to flush a file if it hasn't been written. 89[#10537] 90<li>Remove 4GB restriction on region sizes on 64 bit machines. [#10668] 91<li>Simplify the signature of substitute system calls for ftruncate and 92seek. [#10668] 93<li>Change Berkeley DB so that opening an environment without specifying 94a home directory will cause the DB_CONFIG file in the current directory 95to be read, if it exists. [#11424] 96<li>Fix a bug that caused a core dump if DB handles without associated 97database environments were used for database verification. [#11649] 98<li>Fix Windows mutexes shared between processes run as different users. 99[#11985] 100<li>Fix Windows mutexes for some SMP machines. [#12417] 101 102</ol> 103<h3>Concurrent Data Store Changes:</h3> 104<ol> 105 106<li>Fix cursor locks for environments opened without DB_THREAD so that 107they use the same locker ID. This eliminates many common cases of 108application self-deadlock, particularly in CDS. [#9742] 109 110</ol> 111<h3>General Access Method Changes:</h3> 112<ol> 113 114<li>Fix a bug where Berkeley DB log cursors would close and reopen the 115underlying log file each time the log file was read. [#8934] 116<li>Improve performance of DB->open() for existing subdatabases 117maintained within the same database file. [#9156] 118<li>Add a new error, DB_BUFFER_SMALL, to differentiate from ENOMEM. The 119new error indicates that the supplied DBT is too small. ENOMEM is now 120always fatal. [#9314] 121<li>Fix a bug when an update through a secondary index is deadlocked it 122is possible for the deadlock to be ignored, resulting in a partial 123update to the data. [#9492] 124<li>Fix a bug where a record could get inserted into the wrong database 125when a page was deallocated from one subdatabase and reallocated to 126another subdatabase maintained within the same database file. [#9510] 127<li>Enhance file allocation so that if the operating system supports 128decreasing the size of a file and the last page of the file is freed, 129it will be returned to the operating system. [#9620] 130<li>Fix a bug where DB_RUNRECOVERY could be returned if there was no 131more disk space while aborting the allocation of a new page in a 132database. [#9643] 133<li>Fix a bug where the cryptographic code could memcpy too many bytes. 134[#9648] 135<li>Fix a bug with DB->join() cursors that resulted in a memory leak and 136incomplete results. [#9763] 137<li>Disallow cursor delete, followed a cursor put of the current item 138across all access methods. [#9900] 139<li>Fix a bug where recovery of operations on unnamed databases that 140were never closed, could fail. [#10118] 141<li>Fix a bug where DB->truncate of a database with overflow records 142that spanned more than one page would loop. [#10151] 143<li>Improve performance in the database open/close path. [#10266] 144<li>Fix a bug that restricted the number of temporary files that could 145be created to 127. [#10415] 146<li>Fix a bug which could cause a Too many files error when trying to 147create temporary files. Limit the number of temporary file creation 148retries. [#10760] [#10773] 149<li>Fix a memory leak bug with Sequence Numbers. [#11589] 150<li>Fix a bug on Windows platforms that prevents database files from 151growing to over 2GB. [#11839] 152<li>Fix a platform independence bug with sequence numbers. Existing 153sequence numbers will be automatically upgraded upon next access. 154[#12202] 155<li>Fix a race between truncate and read/write operations on Windows 156platforms that could cause corrupt database files. [#12598] 157 158</ol> 159<h3>Btree Access Method Changes:</h3> 160<ol> 161 162<li>Fix a bug where a record could get placed on the wrong page when two 163threads are simultaneously trying to split a four level (or greater) 164Btree. [#9542] 165<li>Fix a bug where calling DB->truncate() on a Btree which has 166duplicate keys that overflow the leaf page would not properly free the 167overflow pages and possibly loop. [#10666] 168 169</ol> 170<h3>Hash Access Method Changes:</h3> 171<ol> 172 173<li>Fix a bug where a delete to a HASH database with off page duplicates 174could fail to have the proper lock when deleting an off page duplicate 175tree. [#9585] 176<li>Fix a bug where a dirty reader using a HASH database would leave a 177lock on the meta page. [#10105] 178<li>Fix a bug where a DB->del() on a HASH database supporting dirty 179reads could fail to upgrade a WWRITE lock to a WRITE lock when deleting 180an off page duplicate. [#10649] 181 182</ol> 183<h3>Queue Access Method Changes:</h3> 184<ol> 185 186<li>Fix a bug where DB_CONSUME_WAIT may loop rather than wait for a new 187record to enter the queue, if the queue gets into a state where there 188are only deleted records between the head and the end of queue. [#9215] 189<li>Fix a bug where a Queue extent file could be closed when it was 190empty, even if a thread was still accessing a page from that file. 191[#9291] 192<li>Fix a bug where DBC->c_put(key, data, DB_CURRENT) where inserting a 193new record after the current record had been deleted was returning 194DB_KEYEMPTY. [#9314] 195<li>Fix a bug where a Queue extent file could be reported as not found 196if a race condition was encountered between removing the file and 197writing out a stale buffer. [#9462] 198<li>Fix a bug where the Queue access method might fail to release a 199record lock when running without transactions. [#9487] 200<li>Add DB_INORDER flag for Queue databases to guarantee FIFO (First In, 201First Out) ordering when using DB_CONSUME or DB_CONSUME_WAIT. [#9689] 202<li>Fix a bug where remove and rename calls could fail with a 203"Permission denied" error. [#9775] 204<li>Fix a bug where aborting a transaction that opened and renamed a 205queue database using extents could leave some of the extent files with 206the wrong name on Windows. [#9781] 207<li>Fix a bug where a db_dump of a queue database could return an error 208at the end of the queue if the head or tail of the queue is the first 209record on a page. [#10215] 210<li>Fix a race condition which would leave a Queue extent file open 211until the database handle was closed, preventing it from being removed. 212[#10591] 213<li>Fix a bug where a deadlock of a put on a database handle with dirty 214readers could generate a lock downgrade error. [#10678] 215<li>Fix a bug which caused DB_SET_RANGE and DB_GET_BOTH_RANGE to not 216return the next record when an exact match was not found. [#10860] 217 218</ol> 219<h3>Recno Access Method Changes</h3> 220<ol> 221 222<li>Fix a bug where the key/data counts returned by the Db->stat method 223for Recno databases did not match the documentation. [#8639] 224<li>Fix a bug where DBC->c_put(key, data, DB_CURRENT) where inserting a 225new record after the current record had been deleted was returning 226DB_KEYEMPTY. [#9314]. 227 228</ol> 229<h3>C++-specific API Changes:</h3> 230<ol> 231 232<li>Change DbException to extend std::exception, making it possible for 233applications to catch all exceptions in one place. [#10022] 234<li>Fix a bug where errors during transaction destructors (commit, 235abort) could cause an invalid memory access. [#10302] 236<li>Fix a bug that could lead to a read through an uninitialized pointer 237when a DbLockNotGrantedException is thrown. [#10470] 238<li>Fix a bug in the C++ DbEnv::rep_elect method API where the arguments 239were swapped, leading to an "Invalid Argument" return when that method 240is called. [#11906] 241 242</ol> 243<h3>Java-specific API Changes:</h3> 244<ol> 245 246<li>Fix a bug where the Java API did not respect non-zero return values 247from secondaryKeyCreate, including DB_DONOTINDEX. [#9474] 248<li>Fix a bug where a self-deadlock occurred with a non-transactional 249class catalog database used in a transactional environment. The bug 250only occurred when the collections API was not used for starting 251transactions. [#9521] 252<li>Improve Javadoc for the Java API. [#9614] 253<li>Improve memory management and performance when large byte arrays are 254being passed to DB methods. [#9801] 255<li>Improve performance of accessing statistics information from the 256Java API. [#9835] 257<li>Allow Java application to run without DB_THREAD so they can be used 258as RPC clients. [#10097] 259<li>Fix a bug where an uninitialized pointer is dereferenced for 260logArchive(Db.DB_ARCH_REMOVE). [#10225] 261<li>Fix a bug in the Collections API where a deadlock exception could 262leave a cursor open. [#10516] 263<li>Fix the replication callback in the Java API so that the parameter 264names match the C API. [#10550] 265<li>Add get methods to the Java statistics classes. [#10807] 266<li>Fix bugs in the Java API handling of null home directories and 267environments opened without a memory pool. [#11424] 268<li>Fix the Java API in the non-crypto package. [#11752] 269<li>Fix a bug that would cause corruption of error prefix strings. 270[#11967] 271<li>Fix handling of LSNs in the Java API. [#12223] 272<li>Dont throw a NullPointerException if the list of files returned by 273log_archive is empty. [#12383] 274 275</ol> 276<h3>Tcl-specific API Changes:</h3> 277 278None. 279 280<h3>RPC-specific Client/Server Changes:</h3> 281<ol> 282 283<li>Add support for 64-bit and ANSI C implementations of the RPCGEN 284utility. [#9548] 285<li>Fix a small memory leak in RPC clients. [#9595] 286<li>Fix a bug in the RPC server to avoid self-deadlock by always setting 287DB_TXN_NOWAIT. [#10181] 288<li>Fix a bug in the RPC server so that if it times out an environment, 289it first closes all the Berkeley DB handles in that environment. 290[#10623] 291 292</ol> 293<h3>Replication Changes:</h3> 294<ol> 295 296<li>Add an Environment ID to distinguish between clients from different 297replication groups. [#7786] 298<li>Add number of votes required and flags parameters to 299DB_ENV->rep_elect() method. [#7812] 300<li>Fix a bug where a client's env_openfiles pass could start with the 301wrong LSN. This could result in very long initial sync-up times for 302clients joining a replication group. [#8635] 303<li>Add election generation information to replication to support Paxos 304compliance. [#9068] 305<li>Add rep019 to test running normal recovery on clients to make sure 306we synch to the correct LSNs. [#9151] 307<li>Remove support for logs-only replication clients. Use of the 308DB_REP_LOGSONLY flag to the DB_ENV->rep_start() method should be 309replaced with the DB_REP_CLIENT flag. [#9331] 310<li>Fix a bug where replication clients fail to lock all the necessary 311pages when applying updates when there are more than one database in the 312transaction. [#9569] 313<li>Fix a bug in replication elections where when elections are called 314by multiple threads the wrong master could get elected. [#9770] 315<li>Fix a bug where the master could get a DB_REP_OUTDATED error. 316Instead send an OUTDATED message to the client. [#9881] 317<li>Add support for automatic initialization of replication clients. 318[#9927] 319<li>Modify replication timestamp so that non-replication client 320applications can get a DB_REP_HANDLE_DEAD. [#9986] 321<li>Add a new DB_REP_STARTUPDONE return value for rep_process_message() 322and st_startup_done to rep_stat() to indicate when a client has finished 323syncing up to a master and is processing live messages. [#10310] 324<li>Add _pp to secondary handles, add RPRINT, fix a deadlock. [#10429] 325<li>Fix a bug where an old client (and no master) that dropped the ALIVE 326message would never update to the current generation. [#10469] 327<li>Fix a bug where a message could get sent to a new client before 328NEWSITE has been returned to the application. Broadcast instead. 329[#10508] 330<li>Fix a crash when verbose replication messages are configured and a 331NULL DB_LSN pointer is passed to rep_process_message. [#10508] 332<li>Add code to respect set_rep_limit in LOG_REQ processing. [#10716] 333<li>Fix a synchronization problem between replication recovery and 334database open. [#10731] 335<li>Change elections to adjust timeout if egen changes while we are 336waiting. [#10686] 337<li>Client perm messages now return ISPERM/NOTPERM instead of 0. 338[#10855] [#10905] 339<li>Fix a race condition during rep_start when a role change occurs. Fix 340memory leaks. [#11030] 341<li>Fix problems with duplicate records. A failure will no longer occur 342if the records are old records (LOG_MORE) and archived. [#11090] 343<li>Fix a bug where the replication temporary database would grow during 344automatic client initialization. [#11090] 345<li>Add throttling to PAGE_REQ. [#11130] 346<li>Remove optimization-causing problems with racing threads in 347rep_verify_match. [#11208] 348<li>Fix memory leaks. [#11239] 349<li>Fix an initialization bug when High Availability configurations are 350combined with private database environments, which can cause 351intermittent failures. [#11795] 352<li>Fix a bug in the C++ DbEnv::rep_elect method API where the arguments 353were swapped, leading to an "Invalid Argument" return when that method 354is called. [#11906] 355 356</ol> 357<h3>XA Resource Manager Changes:</h3> 358 359None. 360 361<h3>Locking Subsystem Changes:</h3> 362<ol> 363 364<li>Fix a bug where a deadlock of an upgrade from a dirty read to a 365write lock during an aborted transaction, may not be detected. [#7143] 366<li>Add support for Degree 2 isolation. [#8689] 367<li>Change the system to return DB_LOCK_DEADLOCK if a transaction 368attempts to get new locks after it has been selected as the deadlock 369victim. [#9111] 370<li>Fix a bug where when configured to support dirty reads, a writer may 371not downgrade a write lock as soon as possible, potentially blocking 372dirty readers. [#9197] 373<li>Change the test-and-set mutex implementation to avoid interlocked 374instructions when we know the instruction is unlikely to succeed. 375[#9204] 376<li>Fix a bug where a thread supporting dirty readers can get blocked 377while trying to get a write lock. It will allocate a new lock rather 378than using an existing WAS_WRITE lock when it becomes unblocked, causing 379the application to hang. [#10093] 380<li>The deadlock detector will now note that a parent transaction should 381be considered in abort if one of its children is. [#10394] 382<li>Remove a deadlock where database closes could deadlock with page 383acquisition. [#10726] 384<li>Fix a bug where a dirty reader could read an overflow page that was 385about to be deleted. [#10979] 386<li>Fix a bug that failed to downgrade existing write locks during a 387btree page split when supporting dirty reads. [#10983] 388<li>Fix a bug that would fail to upgrade a write lock when moving a 389cursor off a previously deleted record. [#11042] 390 391</ol> <h3>Logging Subsystem Changes:</h3> <ol> 392 393<li>Fix a bug where recovery could leave too many files open. [#9452] 394<li>Fix a bug where aborting a transaction with a file open in it could 395result in an unrecoverable log file. [#9636] 396<li>Fix a bug where recovery would not return a fatal error if the 397transaction log was corrupted. [#9841] 398<li>Fix a bug in recovery so that the final checkpoint no longer tries 399to flush the log. This will permit recovery to complete even if there 400is no disk space to grow the log file. [#10204] 401<li>Improve performance of log flushes by pre-allocating log files and 402using fdatasync() in preference to fsync(). [#10228] 403<li>Fix a bug where recovery of a page split after a non-transactional 404update to the next page would fail to update the back pointer. [#10421] 405<li>Fix a bug in log_archive() where __env_rep_enter() was called twice. 406[#10577] 407<li>Fix a bug with in-memory logs that could cause a memory leak in the 408log region. [#11505] 409 410</ol> 411<h3>Memory Pool Subsystem Changes:</h3> 412<ol> 413 414<li>Fix a bug in the MPOOLFILE file_written flag value so that 415checkpoint doesn't repeatedly open, flush and sync files in the cache 416for which there are no active application handles. [#9529] 417 418</ol> 419<h3>Transaction Subsystem Changes:</h3> 420<ol> 421 422<li>Fix a bug where the same transaction ID could get allocated twice 423if you wrapped the transaction ID space twice and then had a very old 424transaction. [#9036] 425<li>Fix a bug where a transaction abort that contained a page allocation 426could loop if the filesystem was full. [#9461] 427<li>Fix implementation of DB->get_transactional() to match 428documentation: there is no possibility of error return, only 0 or 1. 429[#9526] 430<li>Fix a bug where re-setting any of the DB_TXN_NOSYNC, 431DB_TXN_NOT_DURABLE and DB_TXN_WRITE_NOSYNC flags could fail to clear 432previous state, potentially leading to incorrect transactional behavior 433in the application. [#9947] 434<li>Add a feature to configure the maximum number of files that a 435checkpoint will hold open. [#10026] 436<li>An aborting transaction will no longer generate an undetected 437deadlock. [#10394] 438<li>Fix a bug that prevented a child transaction from accessing a 439database handle that was opened by its parent transaction. [#10783] 440<li>Fix a bug where a checkpoint or a delayed write in another process 441could raise an EINVAL error if the database had been opened with the 442DB_TXN_NOT_DURABLE flag. [#10824] 443<li>Fix private transactional environments on 64-bit systems. [#11983] 444 445</ol> 446<h3>Utility Changes:</h3> 447<ol> 448 449<li>Add debugging and performance tuning information to db_stat. Add 450new Berkeley DB handle methods to output debugging and performance 451tuning information to a C library FILE handle (C and C++ APIs only). 452[#9204] 453<li>Fix a bug where db_stat could drop core if DB->open fails and no 454subdatabase was specified. [#9273] 455<li>Add command-line arguments to the db_printlog utility to restrict 456the range of log file records that are displayed. [#9307] 457<li>Fix a bug in the locking statistics where current locks included 458failed lock requests. [#9314] 459<li>Fix a bug where db_archive would remove all log files when 460--enable-diagnostic and DB_NOTDURABLE were both specified. [#9459] 461<li>Fix a bug where db_dump with the -r flag would output extra copies 462of the subdatabase information. [#9808] 463<li>Fix a bug in db_archive that would cause log file corruption if the 464application had configured the environment with DB_PRIVATE. [#9841] 465<li>Add support in db_load for resetting database LSNs and file IDs 466without having to reload the database. [#9916] 467<li>Change the DB->stat() method to take a transaction handle as an 468argument, allowing DB->stat() to be called from within a transaction. 469[#9920] 470<li>Fix a bug in db_printlog where only the first filewould be displayed 471for in-memory logs. [#11505] 472<li>Fix a bug that prevented database salvage from working in Berkeley 473DB 4.3.21. [#11649] 474<li>Fix a bug in db_load which made it impossible to specify more than 475a single option on the command line. [#11676] 476 477</ol> 478<h3>Configuration, Documentation, Portability and Build Changes:</h3> 479<ol> 480 481<li>Add pread and pwrite to the list of system calls applications can 482replace at run-time. [#8954] 483<li>Add support for UTF-8 encoding of filenames on Windows. [#9122] 484<li>Remove C++ dependency on snprintf. Compilers on HPUX 10.20 are 485missing header file support for snprintf(). [#9284] 486<li>Change Berkeley DB to not use the open system call flag O_DIRECT, 487unless DB configured using --enable-o_direct. [#9298] 488<li>Fix several problems with mutex alignment on HP/UX 10.20. [#9404] 489<li>Fix a memory leak when HAVE_MUTEX_SYSTEM_RESOURCES is enabled. 490[#9546] 491<li>Fix a bug in the sec002.tcl test for binary data. [#9626] 492<li>Fix a bug where filesystem blocks were not being zeroed out in the 493On-Time embedded Windows OS. [#9640] 494<li>Fix build problems with the Java API in Visual Studio .NET 2003. 495[#9701] 496<li>Add support for the gcc compiler on the Opteron platform. [#9725] 497<li>Add support for the small_footprint build option for VxWorks. 498[#9820] 499<li>Add support for linking of DLLs with MinGW. [#9957] 500<li>Remove the make target which builds the RPM package from the 501Berkeley DB distribution. [#10233] 502<li>Add a C++/XML example for ex_repquote. [#10380] 503<li>Fix a bug to link with lrt only if detected by configure (Mac OS X 504issue). [#10418] 505<li>Fix a bug and link Java and Tcl shared libraries with lpthread if 506required, for mutexes. [#10418] 507<li>Add support for building Berkeley DB on the HP NonStop OSS (Tandem) 508platform. [10483] 509<li>Change Berkeley DB to ignore EAGAIN from system calls. This fixed 510problems on NFS mounted databases. [#10531] 511<li>Remove a line with bt_compare and bt_prefix from the db_dump 512recovery test suite, which can cause failures on OpenBSD. [#10567] 513<li>Fix a conflict with the lock_init for building Berkeley DB on 514Cygwin. [#10582] 515<li>Add Unicode support for the Berkeley DB Windows API. [#10598] 516<li>Add support for 64-bit builds on Windows. [#10664] 517<li>Libtool version is now 1.5.8. [#10950] 518<li>Remove mt compilation flag for HP-UX 11.0. [#11427] 519<li>Fix a bug to link with lrt on Solaris to support fdatasync. [#11437] 520 521</ol> 522 523</body> 524</html> 525