1<!--$Id: get_bulk.so,v 10.7 2004/09/03 19:47:57 mjc Exp $--> 2<!--Copyright (c) 1997,2008 Oracle. All rights reserved.--> 3<!--See the file LICENSE for redistribution information.--> 4<html> 5<head> 6<title>Berkeley DB Reference Guide: Retrieving records in bulk</title> 7<meta name="description" content="Berkeley DB: An embedded database programmatic toolkit."> 8<meta name="keywords" content="embedded,database,programmatic,toolkit,btree,hash,hashing,transaction,transactions,locking,logging,access method,access methods,Java,C,C++"> 9</head> 10<body bgcolor=white> 11<a name="2"><!--meow--></a> 12<table width="100%"><tr valign=top> 13<td><b><dl><dt>Berkeley DB Reference Guide:<dd>Access Methods</dl></b></td> 14<td align=right><a href="/am_misc/align.html"><img src="/images/prev.gif" alt="Prev"></a><a href="/toc.html"><img src="/images/ref.gif" alt="Ref"></a><a href="/am_misc/partial.html"><img src="/images/next.gif" alt="Next"></a> 15</td></tr></table> 16<p align=center><b>Retrieving records in bulk</b></p> 17<p>When retrieving large numbers of records from the database, the number 18of method calls can often dominate performance. Berkeley DB offers bulk get 19interfaces which can significantly increase performance for some 20applications. To retrieve records in bulk, an application buffer must 21be specified to the <a href="/api_c/db_get.html">DB->get</a> or <a href="/api_c/dbc_get.html">DBcursor->get</a> methods. This is done 22in the C API by setting the <b>data</b> and <b>ulen</b> fields of the 23<b>data</b> <a href="/api_c/dbt_class.html">DBT</a> to reference an application buffer, and the 24<b>flags</b> field of that structure to <a href="/api_c/dbt_class.html#DB_DBT_USERMEM">DB_DBT_USERMEM</a>. In 25the Berkeley DB C++ and Java APIs, the actions are similar, although there 26are API-specific methods to set the <a href="/api_c/dbt_class.html">DBT</a> values. Then, the 27<a href="/api_c/dbc_get.html#DB_MULTIPLE">DB_MULTIPLE</a> or <a href="/api_c/dbc_get.html#DB_MULTIPLE_KEY">DB_MULTIPLE_KEY</a> flags are specified to 28the <a href="/api_c/db_get.html">DB->get</a> or <a href="/api_c/dbc_get.html">DBcursor->get</a> methods, which cause multiple records 29to be returned in the specified buffer.</p> 30<p>The difference between <a href="/api_c/dbc_get.html#DB_MULTIPLE">DB_MULTIPLE</a> and <a href="/api_c/dbc_get.html#DB_MULTIPLE_KEY">DB_MULTIPLE_KEY</a> 31is as follows: <a href="/api_c/dbc_get.html#DB_MULTIPLE">DB_MULTIPLE</a> returns multiple data items for a 32single key. For example, the <a href="/api_c/dbc_get.html#DB_MULTIPLE">DB_MULTIPLE</a> flag would be used to 33retrieve all of the duplicate data items for a single key in a single 34call. The <a href="/api_c/dbc_get.html#DB_MULTIPLE_KEY">DB_MULTIPLE_KEY</a> flag is used to retrieve multiple 35key/data pairs, where each returned key may or may not have duplicate 36data items.</p> 37<p>Once the <a href="/api_c/db_get.html">DB->get</a> or <a href="/api_c/dbc_get.html">DBcursor->get</a> method has returned, the 38application will walk through the buffer handling the returned records. 39This is implemented for the C and C++ APIs using four macros: 40<a href="/api_c/dbt_bulk.html#DB_MULTIPLE_INIT">DB_MULTIPLE_INIT</a>, <a href="/api_c/dbt_bulk.html#DB_MULTIPLE_NEXT">DB_MULTIPLE_NEXT</a>, 41<a href="/api_c/dbt_bulk.html#DB_MULTIPLE_KEY_NEXT">DB_MULTIPLE_KEY_NEXT</a>, and <a href="/api_c/dbt_bulk.html#DB_MULTIPLE_RECNO_NEXT">DB_MULTIPLE_RECNO_NEXT</a>. For 42the Java API, this is implemented as three iterator classes: 43<a href="/java/com/sleepycat/db/MultipleDataEntry.html">MultipleDataEntry</a>, 44<a href="/java/com/sleepycat/db/MultipleKeyDataEntry.html">MultipleKeyDataEntry</a>, and 45<a href="/java/com/sleepycat/db/MultipleRecnoDataEntry.html">MultipleRecnoDataEntry</a>.</p> 46<p>The <a href="/api_c/dbt_bulk.html#DB_MULTIPLE_INIT">DB_MULTIPLE_INIT</a> macro is always called first. It 47initializes a local application variable and the <b>data</b> 48<a href="/api_c/dbt_class.html">DBT</a> for stepping through the set of returned records. Then, 49the application calls one of the remaining three macros: 50<a href="/api_c/dbt_bulk.html#DB_MULTIPLE_NEXT">DB_MULTIPLE_NEXT</a>, <a href="/api_c/dbt_bulk.html#DB_MULTIPLE_KEY_NEXT">DB_MULTIPLE_KEY_NEXT</a>, and 51<a href="/api_c/dbt_bulk.html#DB_MULTIPLE_RECNO_NEXT">DB_MULTIPLE_RECNO_NEXT</a>.</p> 52<p>If the <a href="/api_c/dbc_get.html#DB_MULTIPLE">DB_MULTIPLE</a> flag was specified to the <a href="/api_c/db_get.html">DB->get</a> or 53<a href="/api_c/dbc_get.html">DBcursor->get</a> method, the application will always call the 54<a href="/api_c/dbt_bulk.html#DB_MULTIPLE_NEXT">DB_MULTIPLE_NEXT</a> macro. If the <a href="/api_c/dbc_get.html#DB_MULTIPLE_KEY">DB_MULTIPLE_KEY</a> flag 55was specified to the <a href="/api_c/db_get.html">DB->get</a> or <a href="/api_c/dbc_get.html">DBcursor->get</a> method, and, the 56underlying database is a Btree or Hash database, the application will 57always call the <a href="/api_c/dbt_bulk.html#DB_MULTIPLE_KEY_NEXT">DB_MULTIPLE_KEY_NEXT</a> macro. If the 58<a href="/api_c/dbc_get.html#DB_MULTIPLE_KEY">DB_MULTIPLE_KEY</a> flag was specified to the <a href="/api_c/db_get.html">DB->get</a> or 59<a href="/api_c/dbc_get.html">DBcursor->get</a> method, and, the underlying database is a Queue or Recno 60database, the application will always call the 61<a href="/api_c/dbt_bulk.html#DB_MULTIPLE_RECNO_NEXT">DB_MULTIPLE_RECNO_NEXT</a> macro. The <a href="/api_c/dbt_bulk.html#DB_MULTIPLE_NEXT">DB_MULTIPLE_NEXT</a>, 62<a href="/api_c/dbt_bulk.html#DB_MULTIPLE_KEY_NEXT">DB_MULTIPLE_KEY_NEXT</a>, and <a href="/api_c/dbt_bulk.html#DB_MULTIPLE_RECNO_NEXT">DB_MULTIPLE_RECNO_NEXT</a> macros 63are called repeatedly, until the end of the returned records is reached. 64The end of the returned records is detected by the application's local 65pointer variable being set to NULL.</p> 66<p>The following is an example of a routine that displays the contents of 67a Btree database using the bulk return interfaces.</p> 68<blockquote><pre>int 69rec_display(dbp) 70 DB *dbp; 71{ 72 DBC *dbcp; 73 DBT key, data; 74 size_t retklen, retdlen; 75 char *retkey, *retdata; 76 int ret, t_ret; 77 void *p; 78<p> 79 memset(&key, 0, sizeof(key)); 80 memset(&data, 0, sizeof(data)); 81<p> 82 /* Review the database in 5MB chunks. */ 83#define BUFFER_LENGTH (5 * 1024 * 1024) 84 if ((data.data = malloc(BUFFER_LENGTH)) == NULL) 85 return (errno); 86 data.ulen = BUFFER_LENGTH; 87 data.flags = DB_DBT_USERMEM; 88<p> 89 /* Acquire a cursor for the database. */ 90 if ((ret = dbp->cursor(dbp, NULL, &dbcp, 0)) != 0) { 91 dbp->err(dbp, ret, "DB->cursor"); 92 free(data.data); 93 return (ret); 94 } 95<p> 96 for (;;) { 97 /* 98 * Acquire the next set of key/data pairs. This code does 99 * not handle single key/data pairs that won't fit in a 100 * BUFFER_LENGTH size buffer, instead returning DB_BUFFER_SMALL 101 * to our caller. 102 */ 103 if ((ret = dbcp->c_get(dbcp, 104 &key, &data, DB_MULTIPLE_KEY | DB_NEXT)) != 0) { 105 if (ret != DB_NOTFOUND) 106 dbp->err(dbp, ret, "DBcursor->c_get"); 107 break; 108 } 109<p> 110 for (DB_MULTIPLE_INIT(p, &data);;) { 111 DB_MULTIPLE_KEY_NEXT(p, 112 &data, retkey, retklen, retdata, retdlen); 113 if (p == NULL) 114 break; 115 printf("key: %.*s, data: %.*s\n", 116 (int)retklen, retkey, (int)retdlen, retdata); 117 } 118 } 119<p> 120 if ((t_ret = dbcp->c_close(dbcp)) != 0) { 121 dbp->err(dbp, ret, "DBcursor->close"); 122 if (ret == 0) 123 ret = t_ret; 124 } 125<p> 126 free(data.data); 127<p> 128 return (ret); 129}</pre></blockquote> 130<table width="100%"><tr><td><br></td><td align=right><a href="/am_misc/align.html"><img src="/images/prev.gif" alt="Prev"></a><a href="/toc.html"><img src="/images/ref.gif" alt="Ref"></a><a href="/am_misc/partial.html"><img src="/images/next.gif" alt="Next"></a> 131</td></tr></table> 132<p><font size=1>Copyright (c) 1996,2008 Oracle. All rights reserved.</font> 133</body> 134</html> 135
