1/* 2 * Copyright (c) 2010 Apple Inc. All rights reserved. 3 * 4 * @APPLE_LICENSE_HEADER_START@ 5 * 6 * Redistribution and use in source and binary forms, with or without 7 * modification, are permitted provided that the following conditions 8 * are met: 9 * 10 * 1. Redistributions of source code must retain the above copyright 11 * notice, this list of conditions and the following disclaimer. 12 * 2. Redistributions in binary form must reproduce the above copyright 13 * notice, this list of conditions and the following disclaimer in the 14 * documentation and/or other materials provided with the distribution. 15 * 3. Neither the name of Apple Inc. ("Apple") nor the names of its 16 * contributors may be used to endorse or promote products derived from 17 * this software without specific prior written permission. 18 * 19 * THIS SOFTWARE IS PROVIDED BY APPLE AND ITS CONTRIBUTORS "AS IS" AND ANY 20 * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 21 * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 22 * DISCLAIMED. IN NO EVENT SHALL APPLE OR ITS CONTRIBUTORS BE LIABLE FOR ANY 23 * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES 24 * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 25 * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND 26 * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 27 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF 28 * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 29 * 30 * Portions of this software have been released under the following terms: 31 * 32 * (c) Copyright 1989-1993 OPEN SOFTWARE FOUNDATION, INC. 33 * (c) Copyright 1989-1993 HEWLETT-PACKARD COMPANY 34 * (c) Copyright 1989-1993 DIGITAL EQUIPMENT CORPORATION 35 * 36 * To anyone who acknowledges that this file is provided "AS IS" 37 * without any express or implied warranty: 38 * permission to use, copy, modify, and distribute this file for any 39 * purpose is hereby granted without fee, provided that the above 40 * copyright notices and this notice appears in all source code copies, 41 * and that none of the names of Open Software Foundation, Inc., Hewlett- 42 * Packard Company or Digital Equipment Corporation be used 43 * in advertising or publicity pertaining to distribution of the software 44 * without specific, written prior permission. Neither Open Software 45 * Foundation, Inc., Hewlett-Packard Company nor Digital 46 * Equipment Corporation makes any representations about the suitability 47 * of this software for any purpose. 48 * 49 * Copyright (c) 2007, Novell, Inc. All rights reserved. 50 * Redistribution and use in source and binary forms, with or without 51 * modification, are permitted provided that the following conditions 52 * are met: 53 * 54 * 1. Redistributions of source code must retain the above copyright 55 * notice, this list of conditions and the following disclaimer. 56 * 2. Redistributions in binary form must reproduce the above copyright 57 * notice, this list of conditions and the following disclaimer in the 58 * documentation and/or other materials provided with the distribution. 59 * 3. Neither the name of Novell Inc. nor the names of its contributors 60 * may be used to endorse or promote products derived from this 61 * this software without specific prior written permission. 62 * 63 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY 64 * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 65 * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 66 * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY 67 * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES 68 * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 69 * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND 70 * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 71 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF 72 * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 73 * 74 * @APPLE_LICENSE_HEADER_END@ 75 */ 76 77/* 78** 79** NAME: 80** 81** dsm.idl 82** 83** FACILITY: 84** 85** Global Location Broker (GLB) 86** 87** ABSTRACT: 88** 89** 90** 91** 92*/ 93 94[local] interface dsm { 95 96/* DSM (Data Store Manager) public interface definition 97 98 The Data Store Manager is a heap storage allocation package wherein 99 allocated records are strongly associated with storage in a backing 100 file, such that they can be stably stored upon modification. The basic 101 paradigm is that the client ALLOCATEs a block of some requested size, 102 modifies it in memory, and WRITEs it; successful completion of the 103 WRITE implies that the record has been stably stored in the file. 104 105 DSM uses OS page alignment to define an atomic operation (a write of 106 or within a page is assumed to either succeed or fail, without any 107 intermediate state). Records are laid out in the file such that the 108 DSM header, as well as some reasonable chunk of the start of 109 application data (e.g. the first 64 bytes total of each record) are 110 contiguous in a page, and so can be written atomically. A write that 111 spans a page boundary occurs in two phases (assuming the record being 112 written was previously free and is so marked on disk): the data portion 113 is written and synched first, then the DSM header (specifically the 114 'inuse/free' mark) to commit the write. 115 116 Updates are not atomically supported. Changing the contents of a 117 record requires conceptually adding a new version and deleting the 118 old one. DSM provides an operation to 'detach' a record (mark it free 119 in the file, effectively deleting it if a crash occurs at this point), 120 after which it can be written normally. This is adequate for 121 applications like DRM which can recover from crashes by replaying the 122 last operation on its propagation queue. Another approach would be 123 to allocate a new record and make a copy, setting a 'new' flag in its 124 application header, then freeing the old copy, and finally clearing 125 the 'new' flag in the new copy. Upon crash recovery the application 126 might see two versions of the same datum, one flagged 'new', and can 127 discard the other one (or it may see only the 'new' version). 128 129 The DSM does not currently itself provide mutual exclusion, although 130 it must be used in such a context (the caller is currently assumed 131 to do the mutex). 132 133 Function summary: 134 135 dsm_create - create a new, empty data store in a named file 136 dsm_open - open an existing data store in a named file 137 dsm_close - close an open data store 138 139 dsm_allocate - allocate a block of some specified size 140 dsm_write - write a block to the backing store, stable upon completion 141 dsm_free - free a block 142 143 dsm_read - successively step through the store, returning each 144 active record 145 dsm_marker_reset - reset the marker used by dsm_read to the beginning 146 147 dsm_get_info 148 dsm_set_info - allow access to application-defined per-file info 149 150 dsm_get_stats - get statistics about file size, free space, etc. 151 152 dsm_reclaim - reclaim free space in data store 153 154 dsm_detach - mark a block free on the backing store (allow simple 155 update paradigm) 156 dsm_write_hdr - write only the beginning of a record (a small header can 157 be updated atomically) 158 159(see below for details on calling sequences and use) 160 161*/ 162 163const long dsm_mod = 0x1C0B0000; 164 165const long dsm_err_create_failed = 0x1C0B0001; 166const long dsm_err_file_io_error = 0x1C0B0002; 167const long dsm_err_open_failed = 0x1C0B0003; 168const long dsm_err_version = 0x1C0B0004; 169const long dsm_err_no_memory = 0x1C0B0005; 170const long dsm_err_duplicate_write = 0x1C0B0006; 171const long dsm_err_header_too_long = 0x1C0B0007; 172const long dsm_err_no_more_entries = 0x1C0B0008; 173const long dsm_err_invalid_handle = 0x1C0B0009; 174const long dsm_err_invalid_pointer = 0x1C0B000A; 175const long dsm_err_info_too_long = 0x1C0B000B; 176const long dsm_err_file_busy = 0x1C0B000C; 177const long dsm_err_invalid_marker = 0x1C0B000D; 178 179typedef void * dsm_handle_t; 180typedef unsigned long dsm_marker_t; 181 182typedef struct dsm_stats_t { /* returned from dsm_get_stats */ 183 unsigned long size; /* size in bytes */ 184 unsigned long free_space; /* how much of that is free */ 185 unsigned long largest_free; /* size of largest free block */ 186} dsm_stats_t; 187 188const long dsm_version = 1; 189const long dsm_hdr_size = 48; /* amount you can write_hdr */ 190const long dsm_magic_marker = 0xF000000F; /* value of reset marker */ 191 192/** dsm_create(fname, &dsh, &st) 193 194 Create a new data store, to live in the (new) file named fname. Returns 195 a data store handle to be used in subsequent dsm calls to refer to that 196 data store. 197*/ 198 199void dsm_create( 200 [in, string] char * fname, /* filename */ 201 [out] dsm_handle_t * dsh, /* data store handle */ 202 [out] error_status_t * st /* status */ 203); 204 205/** dsm_open(fname, &dsh, &st) 206 207 Open the preexisting data store in the file named fname. Returns 208 a data store handle to be used in subsequent dsm calls to refer to that 209 data store. 210*/ 211 212void dsm_open( 213 [in, string] char * fname, /* filename */ 214 [out] dsm_handle_t * dsh, /* data store handle */ 215 [out] error_status_t * st /* status */ 216); 217 218/** dsm_close(&dsh, &st) 219 220 Close the open data store identified by dsh. 221*/ 222 223void dsm_close( 224 [in, out] dsm_handle_t * dsh, /* data store handle */ 225 [out] error_status_t * st /* status */ 226); 227 228/** dsm_allocate(dsh, len, &p, &st) 229 230 Allocate a block of storage len bytes long, in memory and in the 231 backing store identified by dsh; effect is similar to p = malloc(len). 232 The caller will fill in the desired contents of the block and then 233 use dsm_write to commit those contents stably -- until this is done 234 the block is strictly volatile. 235*/ 236 237void dsm_allocate( 238 [in] dsm_handle_t dsh, /* data store handle */ 239 [in] unsigned long len, /* size of block to allocate */ 240 [out] void * * p, /* pointer to block */ 241 [out] error_status_t * st /* status */ 242); 243 244/** dsm_write(dsh, p, &st) 245 246 p is a pointer returned by dsm_allocate. dsm_write writes the block's 247 contents to the backing store (in a stable manner); upon return with 248 a good status, the caller is assured that the block is backed up. 249*/ 250void dsm_write( 251 [in] dsm_handle_t dsh, /* data store handle */ 252 [in] void * p, /* pointer to block */ 253 [out] error_status_t * st /* status */ 254); 255 256/** dsm_write_hdr(dsh, p, len, &st) 257 258 Stably writing arbitrary-length blocks is inherently more expensive than 259 certain shorter writes. DSM provides the feature that the first 48 bytes 260 of a record can be written more efficiently than the general case, in 261 support of the common case of maintaining more volatile information about 262 a record in a header. dsm_write_hdr is equivalent to dsm_write, except 263 that only the first len bytes are written (len being <= 48). (It is 264 expected that len will generally be expressed as a sizeof expression). 265*/ 266 267void dsm_write_hdr( 268 [in] dsm_handle_t dsh, /* data store handle */ 269 [in] void * p, /* pointer to block */ 270 [in] unsigned long len, /* length of header */ 271 [out] error_status_t * st /* status */ 272); 273 274/** dsm_free(dsh, p, &st) 275 276 p is a pointer to a block in the data store identified by dsh; dsm_free 277 returns it to the free pool (future references through p are undefined). 278*/ 279void dsm_free( 280 [in] dsm_handle_t dsh, /* data store handle */ 281 [in] void * p, /* pointer to block */ 282 [out] error_status_t * st /* status */ 283); 284 285/** dsm_detach(dsh, p, &st) 286 287 DSM does not support the concept of updating records, that is, 288 dsm_write may not be called with a pointer to a block that has been 289 written by dsm_write in the past. Generally update operations must 290 be considered as a delete and an add, with the calling application 291 providing semantics for ensuring stability if only one succeeds. For 292 instance, DAM uses a sequence of: allocating a new copy flagged "New" 293 in its header, writing that, then freeing the original, and finally 294 using dsm_write_hdr to clear the "New" mark from the replacement. 295 Its database initialization code deals appropriately with finding a 296 record marked "New". However, some applications have inherent 297 mechanisms for recovering from partial success of delete/add pairs 298 (such as an activity log, or propagation list). For these applications 299 the dsm_detach operation is provided. It is similar to dsm_free, 300 except that the in-memory copy of the record is not freed, and becomes 301 a candidate for dsm_write. Thus a sequence of dsm_detach/dsm_write 302 provides an update operation, with the proviso that a failure between 303 the detach and write could result in the record being lost if no 304 preventative action is taken. dsm_detach of an already free/detached 305 record is a no-op. 306*/ 307 308void dsm_detach( 309 [in] dsm_handle_t dsh, /* data store handle */ 310 [in] void * p, /* pointer to block */ 311 [out] error_status_t * st /* status */ 312); 313 314/** dsm_get_info(dsh, &info, len, &st) 315 dsm_set_info(dsh, &info, len, &st) 316 317 DSM provides a mechanism for an application to maintain a small amount 318 (up to 256 bytes) of its own information pertaining to an entire data 319 store, such as version information. dsm_set_info copies len bytes of 320 application information from info into the data store header. 321 dsm_get_info retrieves len bytes from the data store into info. len 322 is presumed to be constant, typically a sizeof expression, and is not 323 stored in the data store. 324*/ 325 326void dsm_get_info( 327 [in] dsm_handle_t dsh, /* data store handle */ 328 [in] void * info, /* information buffer */ 329 [in] unsigned long len, /* length of header */ 330 [out] error_status_t * st /* status */ 331); 332 333void dsm_set_info( 334 [in] dsm_handle_t dsh, /* data store handle */ 335 [in] void * info, /* information buffer */ 336 [in] unsigned long len, /* length of header */ 337 [out] error_status_t * st /* status */ 338); 339 340/** dsm_read(dsh, &marker, &p, &st) 341 dsm_marker_reset(&marker) 342 343 dsm_read is used to successively examine every allocated record in 344 the data store identified by dsh. marker is used by dsm to hold 345 information required to find successive records; it should be 346 initialized by calling dsm_marker_reset to get the first record. 347 dsm_read will return with status_ok for every valid stably-stored 348 record, and with dsm_err_no_more_entries when there are no more valid 349 entries. Note that if the data store is changed between dsm_read 350 calls (e.g. by another task, or if a marker is stored over time) it 351 is not guaranteed that all records will be seen. However, the 352 behavior of dsm_read is well-defined in such circumstances and should 353 see as many records as possible consistent with a linear pass through 354 the data store, without abnormal behavior. 355 356*/ 357 358void dsm_read( 359 [in] dsm_handle_t dsh, /* data store handle */ 360 [in, out] dsm_marker_t * marker, /* marker */ 361 [out] void * * p, /* record */ 362 [out] error_status_t * st /* status */ 363); 364 365void dsm_marker_reset( 366 [in, out] dsm_marker_t * marker /* marker */ 367); 368 369/** dsm_locate(dsh, marker, &p, &st) 370 371 Returns a pointer to a dsm block, given a marker identifying that block, as 372 returned by dsm_read (the marker returned by a dsm_read call corresponds 373 to the record returned by that call), or by dsm_inq_marker. A marker, 374 unlike a pointer, is valid across datastore closes. An application could 375 store a marker in a file before closing a datastore, and later relocate 376 a pointer to the storage (after reopening the datastore) with dsm_locate. 377 Freeing a block clearly invalidates any markers to it. A marker clearly 378 only applies to the datastore it originally pertained to. It is not 379 meaningful to locate a reset marker. 380*/ 381 382void dsm_locate( 383 [in] dsm_handle_t dsh, /* data store handle */ 384 [in] dsm_marker_t marker, /* marker */ 385 [out] void * * p, /* record */ 386 [out] error_status_t * st /* status */ 387); 388 389/** dsm_inq_marker(dsh, p, &marker, &st) 390 391 Returns a marker to the given dsm block. 392*/ 393void dsm_inq_marker( 394 [in] dsm_handle_t dsh, /* data store handle */ 395 [in] void * p, /* record */ 396 [out] dsm_marker_t * marker, /* marker */ 397 [out] error_status_t * st /* status */ 398); 399/** dsm_inq_size(dsh, p, &size, &st) 400 401 Returns (in the "size" parameter) the size in bytes allocated to the 402 given dsm block. This may be larger than the amount requested when 403 the block was allocated. 404*/ 405void dsm_inq_size( 406 [in] dsm_handle_t dsh, /* data store handle */ 407 [in] void * p, /* record */ 408 [out] unsigned32 * size, /* its size */ 409 [out] error_status_t * st /* status */ 410); 411 412/** dsm_get_stats(dsh, &stats, &st) 413 414 Fills in the given DSM statistics record (see definition above) with 415 information about the open data store (such as total size, free space, 416 largest free block). Immediately after opening a data store the largest 417 free block represents the largest record that could be allocated without 418 growing the file. 419*/ 420 421void dsm_get_stats( 422 [in] dsm_handle_t dsh, /* data store handle */ 423 [out] dsm_stats_t * stats, /* statistics */ 424 [out] error_status_t * st /* status */ 425); 426 427/** dsm_reclaim(dsh, name, tempname, oldname, pct, &st): boolean 428 429 DSM datastores grow as required, without limit, but will not shrink; 430 that is, if you allocate (and write) a large amount of data, and 431 subsequently free it, the space occupied on the disk and in memory 432 becomes eligible for reuse by new DSM records, but is not returned 433 to the operating system. dsm_reclaim is used to reclaim this "high 434 water mark" free storage. It makes a new copy of a datastore 435 without the excess storage (DSM stable memory requirements make it 436 impossible to reclaim space "in place"), if more than a specified 437 percentage of the file is free space. 438 439 dsm_reclaim takes a handle to a DSM datastore. If the handle contains 440 NULL, dsm_reclaim opens the data store, does the reclaim (if appropriate), 441 and then closes the data store. If the handle points to an open data 442 store, DSM returns a handle to the original data store (if no reclaim 443 was done) or to the reclaimed data store. In some error cases, DSM 444 returns a NULL handle even though the input handle pointed to an open 445 data store. 446 447 dsm_reclaim also takes 3 filenames: the datastore name, the 'old' name, 448 and the 'new' name; typical examples might be "mydb", "mydb.bak", and 449 "mydb.new". The following sequence is applied: 450 451 if handle is NULL, open mydb 452 check mydb stats 453 if freespace doesn't exceed specified percent 454 if handle is NULL, close mydb 455 return 456 457 create "mydb.new" 458 for every record in mydb, make a copy in mydb.new 459 close both datastores 460 rename "mydb" to "mydb.bak" 461 rename "mydb.new" to "mydb" 462 if handle is non-NULL, open mydb 463 464 dsm_reclaim returns true if the file was reclaimed, false otherwise (in case 465 of error or under free_pct). If the input handle pointed to an open data 466 store then the returned handle points to an open data store (except in certain 467 error cases when it contains NULL). 468 469 NB: Since data is moved around in the datastore during dsm_reclaim, 470 markers and pointers into the DSM datastore are invalid after 471 dsm_reclaim and their use will yield indeterminate results. It 472 is best to invoke dsm_reclaim on a closed data store or immediately 473 after opening the data store. 474 475 Observe that this is really at a higher level than DSM per se; dsm_reclaim 476 is provided as a common utility as most DSM applications will want this 477 sort of functionality. Individual applications may choose to provide 478 variations on this basic algorithm. This particular sequence is safe in 479 the face of crashes. There should be two files in any case: if "mydb" 480 exists, it should be used; if it doesn't then "mydb.bak" contains the old 481 version and "mydb.new" the new (reclaimed) one. (if "mydb.new" and "mydb" 482 both exist, "mydb.new" probably contains an incompletely reclaimed copy, and 483 should be discarded). 484 485*/ 486 487boolean dsm_reclaim( 488 [in, out] dsm_handle_t * dsh, /* data store handle */ 489 [in, string] char * fname, /* datastore filename */ 490 [in, string] char * tempname, /* temp file to build new datastore in */ 491 [in, string] char * oldname, /* what to rename the pre-reclamation store */ 492 [in] unsigned32 free_pct, /* integer %age of free space to tolerate */ 493 [out] error_status_t * st /* status */ 494); 495 496} 497