Lines Matching defs:in
10 ** This file is all you need to compile SQLite. To use SQLite in other
17 ** language. The code for the "sqlite3" command-line shell is also in a
54 ** for those options in which the value is meaningful. */
68 ** This array looks large, but in a typical installation actually uses
833 ** In all cases, the special comment must be enclosed in the usual
849 ** compiler warnings due to subsequent content in this file and other files
852 /************** Include msvc.h in the middle of sqliteInt.h ******************/
897 /************** Continuing where we left off in sqliteInt.h ******************/
902 /************** Include vxworks.h in the middle of sqliteInt.h ***************/
938 /************** Continuing where we left off in sqliteInt.h ******************/
947 ** code in all source files.
953 ** without this option, LFS is enable. But LFS does not exist in the kernel
954 ** in Red Hat 6.0, so the code won't work. Hence, for maximum binary
957 ** The previous paragraph was written in 2005. (This paragraph is written
960 ** lack LFS in which case the SQLITE_DISABLE_LFS macro might still be useful.
979 ** version numbers in clang are for "marketing" only and are inconsistent
997 ** Some C99 functions in "math.h" are only present for MSVC when its version
1049 ** library in use (e.g. for Windows XP).
1059 ** first in QNX. Also, the _USE_32BIT_TIME_T macro must appear first for
1062 /************** Include sqlite3.h in the middle of sqliteInt.h ***************/
1077 ** or constant definition does not appear in this file, then it is
1081 ** Some of the definitions that are in this file are marked as
1085 ** if experience from use "in the wild" suggest such changes are prudent.
1088 ** from comments in this file. This file is the authoritative source
1091 ** The name of this file under configuration management is "sqlite.h.in".
1134 ** These no-op macros are used in front of interfaces to mark those
1138 ** experimental interfaces are subject to change in point releases.
1162 ** ^(The [SQLITE_VERSION] C preprocessor macro in the sqlite3.h header
1163 ** evaluates to a string literal that is the SQLite version in the
1168 ** numbers used in [SQLITE_VERSION].)^
1175 ** SQLite source code has been stored in the
1178 ** a string which identifies a particular check-in of SQLite
1180 ** string contains the date and time of the check-in (UTC) and a SHA1
1182 ** been edited in any way since it was last checked in, then the last
1200 ** programmers might include assert() statements in their application to
1201 ** verify that values returned by these interfaces match the macros in
1214 ** function is provided for use in DLLs since DLL users usually do not have
1380 ** where the order in which destructors are called is arbitrary.
1414 ** in the context of the [database connection] passed in as its 1st
1432 ** occur, then sqlite3_exec() sets the pointer in its 5th parameter to
1440 ** number of columns in the result. ^The 3rd argument to the sqlite3_exec()
1478 ** here in order to indicate success or failure.
1480 ** New error codes may be added in future versions of SQLite.
1487 #define SQLITE_INTERNAL 2 /* Internal logic error in SQLite */
1491 #define SQLITE_LOCKED 6 /* A table in the database is locked */
1497 #define SQLITE_NOTFOUND 12 /* Unknown opcode in sqlite3_file_control() */
1613 ** These bit values are intended for use in the
1615 ** in the 4th parameter to the [sqlite3_vfs.xOpen] method.
1655 ** mean that writes of blocks that are nnn bytes in size and
1661 ** information is written to disk in the same order as calls
1663 ** after reboot following a crash or power loss, the only bytes in a
1739 ** An [sqlite3_file] object represents an open file in the
1771 ** flag may be ORed in to indicate that only the data of the file
1784 ** either in this process or in some other process, is holding a RESERVED,
1792 ** point to a structure that may contain arguments or space in which to
1807 ** other bytes in the file. The xDeviceCharacteristics()
1831 ** mean that writes of blocks that are nnn bytes in size and
1837 ** information is written to disk in the same order as calls
1841 ** in the unread portions of the buffer with zeros. A VFS that
1870 /* Additional methods may be added in future releases */
1896 ** file space based on this hint in order to help writes to the database
1900 ** The [SQLITE_FCNTL_SIZE_LIMIT] opcode is used by in-memory VFS that
1902 ** of the in-memory database. The argument is a pointer to a [sqlite3_int64].
1903 ** If the integer pointed to is negative, then it is filled in with the
1910 ** extends and truncates the database file in chunks of a size specified
1913 ** for the nominated database. Allocating database file space in large
1929 ** No longer in use.
1936 ** [PRAGMA synchronous | PRAGMA synchronous=OFF] it is invoked in place
1956 ** windows [VFS] in order to provide robustness in the presence of
1980 ** in order for the database to be readable. The fourth parameter to
2004 ** all [VFSes] in the VFS stack. The names are of all VFS shims and the
2006 ** [sqlite3_malloc()] and the result is stored in the char* variable
2011 ** pointer in case this file-control is not implemented. This file-control
2016 ** [VFSes] currently in use. ^(The argument X in
2020 ** ^When there are multiple VFS shims in the stack, this opcode finds the
2028 ** pointers to strings (char**) in which the second element of the array
2046 ** it is able to override built-in [PRAGMA] statements.
2051 ** shortly after it is opened in order to provide a custom VFS with access
2055 ** busy-handler, this function should be invoked with the second (void *) in
2073 ** is an advisory maximum number of bytes in the file to memory map. The
2076 ** can be queried by passing in a pointer to a negative number. This
2083 ** The argument is a zero-terminated string. Higher layers in the
2109 ** circumstances in order to fix a problem with priority inversion.
2123 ** the file descriptor is placed in "batch write mode", which
2170 ** connections possibly in other processes. The [sqlite3_total_changes()]
2185 ** in wal mode before the client starts to copy pages from the wal
2190 ** in wal mode after the client has finished copying pages from the wal
2257 ** structure must be typedefed in order to work around compiler warnings
2267 ** in the name of the object stands for "virtual file system". See
2272 ** is incremented. The iVersion value started out as 1 in
2277 ** may increase again in future versions of SQLite.
2279 ** of the sqlite3_vfs object changed in the transition from
2285 ** a pathname in this VFS.
2290 ** in a thread-safe way. The [sqlite3_vfs_find()] interface
2294 ** The pNext field is the only field in the sqlite3_vfs
2320 ** The flags argument to xOpen() includes all bits set in
2325 ** include [SQLITE_OPEN_READONLY]. Other bits in *pOutFlags may be set.
2347 ** file will be doing page-aligned sector reads and writes in a random
2362 ** ^The [SQLITE_OPEN_EXCLUSIVE] flag is always used in conjunction
2374 ** allocate the structure; it should just fill it in. Note that
2386 ** flag is never actually used and is not implemented in the built-in
2390 ** the file given in the second argument is illegal. If SQLITE_OK
2403 ** included in the VFS structure for completeness.
2412 ** Day Number multiplied by 86400000 (the number of milliseconds in
2454 ** The methods above are in version 1 of the sqlite_vfs object
2455 ** definition. Those that follow are added in version 2 or later
2459 ** The methods above are in versions 1 and 2 of the sqlite_vfs object.
2466 ** The methods above are in versions 1 through 3 of the sqlite_vfs object.
2467 ** New fields may be appended in future versions. The iVersion
2482 ** (in other words, if files can be added, removed, and renamed within
2485 ** [temp_store_directory pragma], though this could change in a future
2489 ** currently unused, though it might be used in a future release of
2540 ** These routines are designed to aid in process initialization and
2584 ** default behavior in some future release of SQLite.
2618 ** changes to SQLite in order to tune SQLite to the specific needs of
2639 ** in the first argument.
2654 ** [database connection] (specified in the first argument).
2672 ** This object is used in only one place in the SQLite interface.
2682 ** Note that SQLite comes with several [built-in memory allocators]
2686 ** also used during testing of SQLite in order to specify an alternative
2687 ** memory allocator that simulates memory out-of-memory conditions in
2704 ** Every memory allocation request coming in through [sqlite3_malloc()]
2748 ** New configuration options may be added in future releases of SQLite.
2773 ** are enabled so that SQLite will be safe to use in a multi-threaded
2790 ** same [prepared statement] in different threads at the same time.
2801 ** alternative low-level memory allocation routines to be used in place of
2820 ** but some application might prefer to run slower in exchange for
2838 ** compiled with [SQLITE_DEFAULT_MEMSTATUS]=0 in which case memory
2885 ** the number of bytes in the memory buffer, and the minimum allocation size.
2900 ** in place the mutex routines built into SQLite.)^ ^SQLite makes a copy of
2978 ** argument which is interpreted as a boolean in order to enable or disable
2979 ** the use of covering indices for full table scans in the query optimizer.
3001 ** in three separate circumstances, identified by the value passed as the
3009 ** configuration option can be seen in the "test_sqllog.c" source file in
3037 ** bytes per page required for each page in [SQLITE_CONFIG_PAGECACHE].
3056 ** [Statement journals] are held in memory until their size (in bytes)
3059 ** exclusively in memory.
3071 ** to an ORDER BY clause, all fields required by the caller are present in the
3075 ** is stored in each sorted record and the required column values loaded
3076 ** from the database as records are returned in sorted order. The default
3085 ** [sqlite3_int64] parameter which is the default maximum size for an in-memory
3130 ** New configuration options may be added in future releases of SQLite.
3145 ** may be NULL in which case SQLite will allocate the
3148 ** slots. The size of the buffer in the first argument must be greater than
3154 ** connection is not currently using lookaside memory, or in other words
3158 ** memory is in use leaves the configuration unchanged and returns
3169 ** following this call. The second parameter may be a NULL pointer, in
3180 ** following this call. The second parameter may be a NULL pointer, in
3185 ** this option is off. So, in other words, this option now only disables
3186 ** triggers in the main database schema or in the schemas of ATTACH-ed
3197 ** following this call. The second parameter may be a NULL pointer, in
3202 ** this option is off. So, in other words, this option now only disables
3203 ** views in the main database schema or in the schemas of ATTACH-ed
3217 ** following this call. The second parameter may be a NULL pointer, in
3235 ** be a NULL pointer, in which case the new setting is not reported back.
3241 ** which will become the new schema name in place of "main". ^SQLite
3249 ** <dd> Usually, when a database in wal mode is closed or detached from a
3268 ** the QPSG active, SQLite will always use the same query plan in the field as
3269 ** was used during testing in the lab.
3291 ** [VACUUM] in order to reset a database back to an empty database
3299 ** the database in WAL mode after the reset if it was in WAL mode before
3387 ** integer found at offset 44 into the database header) of 1. This in turn
3439 ** ^Each entry in most SQLite tables (except for [WITHOUT ROWID] tables)
3459 ** part of committing a transaction (e.g. to flush data accumulated in memory
3463 ** tables in this way can avoid this problem by restoring the original
3575 ** ^Changes made as part of [foreign key actions] are included in the
3608 ** called in response to a user action such as pressing "Cancel"
3626 ** ^The sqlite3_interrupt(D) call is in effect until all currently running
3650 ** independent tokens (they are part of the token in which they are
3670 ** UTF-16 string in native byte order.
3704 ** handler could result in a deadlock, it will go ahead and return [SQLITE_BUSY]
3729 ** result in undefined behavior.
3776 ** There are (N+1)*M elements in the array. The first M pointers point
3779 ** in NULL pointers. All other values are in their UTF-8 zero-terminated
3799 ** in an array named azResult. Then azResult holds this content:
3813 ** semicolon-separated SQL statements in the zero-terminated UTF-8
3815 ** pointer given in its 3rd parameter.
3818 ** it must pass the result table pointer to sqlite3_free_table() in order to
3827 ** interface defined here. As a consequence, errors that occur in the
3829 ** reflected in subsequent calls to [sqlite3_errcode()] or
3850 ** See the [built-in printf()] documentation for details.
3880 ** See also: [built-in printf()], [printf() SQL function]
3891 ** internal memory allocation needs. "Core" in the previous sentence
3896 ** of memory at least N bytes in length, where N is the parameter.
3912 ** memory might result in a segmentation fault or other severe error.
3926 ** of at least N bytes in size or NULL if insufficient memory is available.
3939 ** sqlite3_msize(X) returns the size of that memory allocation in bytes.
3974 ** routines, which form the built-in memory allocation subsystem.
3982 ** added by SQLite in its implementation of [sqlite3_malloc()],
4001 ** the built-in random() and randomblob() SQL functions. This interface allows
4024 ** [database connection], supplied in the first argument.
4052 ** Applications must always be prepared to encounter a NULL pointer in any
4058 ** a NULL value in place of the table column that would have
4063 ** extracted from that table (for example in a query like
4077 ** database. An authorizer could then be put in place while the
4084 ** in addition to using an authorizer.
4086 ** ^(Only a single authorizer can be in place on a database connection
4094 ** database connections for the meaning of "modify" in this paragraph.
4099 ** correct authorizer callback remains in place during the [sqlite3_step()].
4103 ** performed during statement evaluation in [sqlite3_step()], unless
4104 ** as stated in the previous paragraph, sqlite3_step() invokes
4117 ** return either [SQLITE_OK] or one of these two constants in order
4202 ** the length of [bound parameter] expansion in the output of sqlite3_trace().
4208 ** time is in units of nanoseconds, however the current implementation
4210 ** digits in the time are meaningless. Future versions of SQLite
4230 ** New tracing constants may be added in future releases.
4234 ** ^The C argument is a copy of the context pointer passed in as the
4293 ** ignored, though this may change in future releases. Callback
4342 ** database connections for the meaning of "modify" in this paragraph.
4353 ** sqlite3_open() and sqlite3_open_v2() and as UTF-16 in the native byte
4355 ** returned in *ppDb, even if an error occurs. The only exception is that
4366 ** created using sqlite3_open16() will be UTF-16 in the native byte order.
4380 ** <dd>The database is opened in read-only mode. If the database does not
4402 ** <dd>The database will be opened as an in-memory database. The database
4417 ** (Mutexes will block any actual concurrency, but in this mode
4418 ** there is no harm in trying.)
4444 ** ^If the filename is ":memory:", then a private, temporary in-memory database
4445 ** is created for the connection. ^This in-memory database will vanish when
4456 ** [[URI filenames in sqlite3_open()]] <h3>URI Filenames</h3>
4461 ** set in the third argument to sqlite3_open_v2(), or if it has
4486 ** SQLite and its built-in [VFSes] interpret the
4502 ** access, just as if the [SQLITE_OPEN_READONLY] flag had been set in the
4508 ** set to "memory" then a pure [in-memory database] that never reads
4511 ** the flags passed in the third parameter to sqlite3_open_v2().
4515 ** SQLITE_OPEN_SHAREDCACHE bit in the flags argument passed to
4518 ** ^If sqlite3_open_v2() is used and the "cache" parameter is present in
4527 ** which if set disables file locking in rollback journal modes. This
4539 ** property on a database file that does in fact change can result
4540 ** in incorrect query results and/or [SQLITE_CORRUPT] errors.
4545 ** ^Specifying an unknown parameter in the query component of a URI is not an
4555 ** Open the file "data.db" in the current directory.
4565 ** C:. Note that the %20 escaping in this example is not strictly
4567 ** in URI filenames.
4569 ** Open file "data.db" in the current directory for read-only access.
4574 ** that uses dot-files in place of posix advisory locking.
4635 ** If F is a suitable filename (as described in the previous paragraph)
4646 ** value of query parameter P is one of "yes", "true", or "on" in any
4649 ** query parameter P is one of "no", "false", or "off" in any case or
4674 ** in addition to the main database file. Prior to version 3.31.0, these
4726 ** This routine is intended for use in custom [VFS] implementations
4732 ** of this routine results in undefined and probably undesirable
4745 ** with N URI parameters key/values pairs in the array P. The result from
4761 ** The P parameter in sqlite3_create_filename(D,J,W,N,P) should be an array
4762 ** of 2*N pointers to strings. Each pair of pointers in this array corresponds
4764 ** pointer if N is zero. None of the 2*N pointers in the P array may be
4829 ** When the serialized [threading mode] is in use, it might be the
4830 ** case that a second error occurs on a separate thread in between
4891 ** (The "_LIMIT_" in the name is changed to "_MAX_".))^
4900 ** Run-time limits are intended for use in applications that manage
4913 ** New run-time limit categories may be added in future releases.
4924 ** Additional information is available at [limits | Limits in SQLite].
4928 ** <dd>The maximum size of any string or BLOB or table row, in bytes.<dd>)^
4931 ** <dd>The maximum length of an SQL statement, in bytes.</dd>)^
4934 ** <dd>The maximum number of columns in a table definition or in the
4935 ** result set of a [SELECT] or the maximum number of columns in an index
4936 ** or in an ORDER BY or GROUP BY clause.</dd>)^
4942 ** <dd>The maximum number of terms in a compound SELECT statement.</dd>)^
4945 ** <dd>The maximum number of instructions in a virtual machine program
4948 ** in a single prepared statement, an SQLITE_NOMEM error is returned.</dd>)^
4963 ** <dd>The maximum index number of any [parameter] in an SQL statement.)^
4993 ** New flags may be added in future releases of SQLite.
5032 ** program using one of these routines. Or, in other words, these routines
5061 ** is the number of bytes in the input string <i>including</i>
5065 ** past the end of the first SQL statement in zSql. These routines only
5066 ** compile the first statement in zSql, so *pzTail is left pointing to
5087 ** behave differently in three ways:
5102 ** in order to find the underlying cause of the problem. With the "v2" prepare
5107 ** ^If the specific value bound to a [parameter | host parameter] in the
5119 ** <p>^sqlite3_prepare_v3() differs from sqlite3_prepare_v2() only in having
5128 int nByte, /* Maximum length of zSql in bytes. */
5135 int nByte, /* Maximum length of zSql in bytes. */
5142 int nByte, /* Maximum length of zSql in bytes. */
5150 int nByte, /* Maximum length of zSql in bytes. */
5157 int nByte, /* Maximum length of zSql in bytes. */
5164 int nByte, /* Maximum length of zSql in bytes. */
5273 ** This interface can be used in combination [sqlite3_next_stmt()]
5275 ** connection that are in need of being reset. This can be used,
5276 ** for example, in diagnostic routines to search for prepared
5286 ** that can be stored in a database table. SQLite uses dynamic typing
5287 ** for the values it stores. ^Values stored in sqlite3_value objects
5303 ** or if SQLite is run in one of reduced mutex modes
5326 ** The context in which an SQL function executes is stored in an
5389 ** found in first character, which is removed, or in the absence of a BOM
5391 ** machine for sqlite3_bind_text16() or the byte order specified in
5398 ** number of bytes in the parameter. To be clear: the value is the
5399 ** number of <u>bytes</u> in the value, not the number of characters.)^
5422 ** information is in static, unmanaged space and does not need to be freed.
5429 ** to specify the encoding of the text in the third parameter. If
5441 ** ^A negative value for the zeroblob results in a zero-length BLOB.
5443 ** ^The sqlite3_bind_pointer(S,I,P,T,D) routine causes the I-th parameter in
5494 ** in a [prepared statement]. SQL parameters are tokens of the
5502 ** there may be gaps in the list.)^
5515 ** the name of the N-th [SQL parameter] in the [prepared statement] P.
5528 ** always in UTF-8 encoding even if the named parameter was
5529 ** originally specified as UTF-16 in [sqlite3_prepare16()],
5546 ** name must be given in UTF-8 even if the original statement
5570 ** ^Return the number of columns in the result set returned by the
5587 ** in the result set of a [SELECT] statement. ^The sqlite3_column_name()
5617 ** table column that is the origin of a particular result column in
5627 ** again in a different encoding.
5686 ** data stored in that column is of the declared type. SQLite is
5748 ** more threads at the same moment in time.
5754 ** [sqlite3_reset()] would result in an [SQLITE_MISUSE] return from
5757 ** calling [sqlite3_reset()] automatically in this circumstance rather
5766 ** [sqlite3_reset()] or [sqlite3_finalize()] in order to find one of the
5779 ** CAPI3REF: Number of columns in a result set
5782 ** ^The sqlite3_data_count(P) interface returns the number of columns in the
5791 ** [SQLITE_ROW], except in the case of the [PRAGMA incremental_vacuum]
5803 ** ^(Every value in SQLite has one of five fundamental datatypes:
5815 ** Note that the SQLITE_TEXT constant was also used in SQLite version 2
5848 ** or a UTF-8 TEXT result in bytes
5851 ** TEXT in bytes
5864 ** ^The number of columns in the result can be determined using
5880 ** each return the value of a result column in a specific data format. If
5881 ** the result column is not initially in the requested format (for example,
5892 ** automatic type conversions have occurred for the value in question.
5903 ** routine returns the number of bytes in that BLOB or string.
5908 ** the number of bytes in that string.
5912 ** routine returns the number of bytes in that BLOB or string.
5917 ** the number of bytes in that string.
5924 ** bytes in the string, not the number of characters.
5935 ** [sqlite3_column_value()] is used in any other way, including calls
5976 ** in the following cases:
5990 ** ^Conversions between UTF-16be and UTF-16le are always done in place and do
5993 ** of conversion are done in place when it is possible, but sometimes they
5994 ** are not possible and in those cases prior pointers are invalidated.
5997 ** in one of the following ways:
6071 ** The application must finalize every [prepared statement] in order to avoid
6074 ** statement after it has been finalized can result in undefined and
6127 ** redefined. ^The length of the name is limited to 255 bytes in a UTF-8
6129 ** length limit is in UTF-8 bytes, not characters nor UTF-16 bytes.
6131 ** will result in [SQLITE_MISUSE] being returned.
6157 ** deterministic. The built-in [random()] SQL function is an example of a
6195 ** must both be non-NULL. xValue and xInverse may either both be NULL, in
6197 ** non-NULL, in which case the new function may be used as either an aggregate
6214 ** the implementation that most closely matches the way in which the
6224 ** ^Built-in functions may be overloaded by new application-defined functions.
6229 ** statement in which the function is running.
6302 ** be deterministic in order to be used in certain contexts such as
6303 ** with the WHERE clause of [partial indexes] or in [generated columns].
6310 ** from top-level SQL, and cannot be used in VIEWs or TRIGGERs nor in
6332 ** disable the use of SQL functions inside views and triggers and in
6335 ** the function is tagged with SQLITE_INNOCUOUS. Most built-in functions
6390 ** <tr><td><b>sqlite3_value_text16</b><td>→<td>UTF-16 TEXT value in
6396 ** or a UTF-8 TEXT in bytes
6399 ** TEXT in bytes
6405 ** <td>→ <td>True if the column is unchanged in an UPDATE
6427 ** in the native byte-order of the host machine. ^The
6452 ** such a conversion is possible without loss of information (in other
6465 ** sqlite3_value_nochange(X) is true will in all other respects appear
6577 ** value of N in any subsequent call to sqlite3_aggregate_context() within
6580 ** N=0 in calls to sqlite3_aggregate_context(C,N) so that no
6591 ** This routine must be called from the same thread in which
6606 ** This routine must be called from the same thread in which
6631 ** of where this might be useful is in a regular-expression matching
6662 ** Note the last bullet in particular. The destructor X in
6677 ** These routines must be called from the same thread in which
6687 ** These are special values for the destructor that is passed in as the
6691 ** SQLITE_TRANSIENT value means that the content will likely change in
6695 ** The typedef is necessary to work around problems in certain
6712 ** functions used to bind values to host parameters in prepared statements.
6722 ** a BLOB containing all zero bytes and N bytes in size.
6747 ** returned by SQLite as a result of an error in a function. ^By default,
6759 ** value given in the 2nd argument.
6762 ** value given in the 2nd argument.
6773 ** application-defined function to be a text string in an encoding
6787 ** in the string at a byte offset that is less than the value of the 3rd
6796 ** assumes that the text or BLOB result is in constant space and does not
6827 ** so that the [sqlite3_value] specified in the parameter may change or
6879 ** of the subtype T are preserved in current versions of SQLite;
6882 ** in future releases of SQLite.
6895 ** and a UTF-16 string in native byte order for sqlite3_create_collation16().
6926 ** application data pointer and with two strings in the encoding specified
6928 ** function callback are the length of the two strings, in bytes. The collating
7001 ** encoded in UTF-8. ^If sqlite3_collation_needed16() is used,
7002 ** the names are passed as UTF-16 in machine native byte order.
7042 ** for at least a number of milliseconds specified in its parameter.
7053 ** in the previous paragraphs.
7062 ** created by SQLite when using a built-in [sqlite3_vfs | VFS]
7063 ** will be placed in that directory.)^ ^If this variable
7072 ** be avoided in new projects.
7074 ** It is not safe to read or modify this variable in more than one
7076 ** if a [database connection] is being used at the same time in a separate
7086 ** that this variable points to is held in memory obtained from
7121 ** SQLite when using a built-in windows [sqlite3_vfs | VFS] will be assumed
7129 ** open can result in a corrupt database.
7131 ** It is not safe to read or modify this variable in more than one
7133 ** if a [database connection] is being used at the same time in a separate
7143 ** that this variable points to is held in memory obtained from
7193 ** zero if the given database connection is or is not in autocommit mode,
7220 ** create the statement in the first place.
7231 ** connection D, or if database N is a temporary or in-memory database, then
7271 ** [transaction state] of schema S in database connection D. ^If S is NULL,
7273 ** is returned. Transaction states are (in order of lowest to highest):
7290 ** constants in order to describe the transaction state of schema S
7291 ** in [database connection] D.
7300 ** in a read transaction. Content has been read from the database file
7301 ** but nothing in the database file has changed. The transaction state
7309 ** in a write transaction. Content has been written to the database file
7327 ** The [database connection] pointer D in a call to
7329 ** connection and in particular must not be a NULL pointer.
7359 ** or rollback hook in the first place.
7362 ** the database connections for the meaning of "modify" in this paragraph.
7389 ** to be invoked whenever a row is updated, inserted or deleted in
7395 ** row is updated, inserted or deleted in a rowid table.
7414 ** The exceptions defined in this paragraph might change in a future
7422 ** database connections for the meaning of "modify" in this paragraph.
7454 ** that was in effect at the time they were opened.)^
7501 ** [sqlite3_release_memory()] interface, this interface is in effect even
7518 ** limit by reducing the number of pages held in the page cache
7533 ** the heap limit prior to the call, or negative in the case of an
7555 ** ^(The heap limits are not enforced in the current implementation
7571 ** changes in future releases of SQLite.
7593 ** information about column C of table T in database D
7595 ** interface returns SQLITE_OK and fills in the non-NULL pointers in
7602 ** does not. If the table name parameter T in a call to
7618 ** NULL, in which case the corresponding element of metadata is omitted.
7676 ** [SQLite extension] library contained in the file zFile. If
7684 ** ^(zProc may be 0, in which case SQLite will try to come up with an
7688 ** characters in the filename from the last "/" to the first following
7694 ** fill *pzErrMsg with error message text stored in memory
7724 ** ^So as not to open security holes in older applications that are
7807 ** to be experimental. The interface might change in incompatible ways.
7830 ** ^A virtual table module is created by filling in a persistent
7866 /* The methods above are in version 1 of the sqlite_module object. Those
7871 /* The methods above are in versions 1 and 2 of the sqlite_module object.
7892 ** stored in aConstraint[].op using one of the
7894 ** ^(The index of the column is stored in
7900 ** and makes other simplifications to the WHERE clause in an attempt to
7905 ** ^Information about the ORDER BY clause is stored in aOrderBy[].
7910 ** zero in the order in which they appear within the CREATE TABLE statement
7922 ** and becomes the argvIndex-th entry in argv. ^(If aConstraintUsage[].omit
7926 ** is left in its default setting of false, the constraint will always be
7927 ** checked separately in byte code. If the omit flag is change to true, then
7928 ** the constraint may or may not be checked in byte code. In other words,
7937 ** ^The orderByConsumed means that output from [xFilter]/[xNext] will occur in
7980 int nConstraint; /* Number of entries in aConstraint */
7987 int nOrderBy; /* Number of terms in the ORDER BY clause */
8002 /* Fields below are only available in SQLite 3.8.2 and later */
8004 /* Fields below are only available in SQLite 3.9.0 and later */
8006 /* Fields below are only available in SQLite 3.10.0 and later */
8024 ** an operator that is part of a constraint term in the wHERE clause of
8145 ** This superclass exists in order to define fields of the cursor that
8170 ** must exist in order to be overloaded.)^
8185 ** to be experimental. The interface might change in incompatible ways.
8202 ** ^The [sqlite3_blob_bytes()] interface returns the size of the BLOB in bytes.
8212 ** in row iRow, column zColumn, table zTable in database zDb;
8213 ** in other words, the same BLOB that would be selected by:
8221 ** the name that appears after the AS keyword in the [ATTACH] statement.
8230 ** in *ppBlob. Otherwise an [error code] is returned and, unless the error
8241 ** <li> ^(Row iRow is not present in the table)^,
8279 ** and the built-in [zeroblob] SQL function may be used to create a
8311 ** it must exist and there must be either a blob or text value stored in
8312 ** the nominated column.)^ ^If the new row is not present in the table, or if
8333 ** the database is in auto-commit mode and there are no other open read-write
8339 ** open blob handle results in undefined behaviour. ^Calling this routine
8351 ** ^Returns the size in bytes of the BLOB accessible via the
8352 ** successfully opened [BLOB handle] in its only argument. ^The
8358 ** been closed by [sqlite3_blob_close()]. Passing any other pointer in
8359 ** to this routine results in undefined and probably undesirable behavior.
8385 ** been closed by [sqlite3_blob_close()]. Passing any other pointer in
8386 ** to this routine results in undefined and probably undesirable behavior.
8427 ** been closed by [sqlite3_blob_close()]. Passing any other pointer in
8428 ** to this routine results in undefined and probably undesirable behavior.
8478 ** implementations are available in the SQLite core:
8487 ** that does no real locking and is appropriate for use in
8529 ** not want to. SQLite will only request a recursive mutex in
8532 ** might return such a mutex in response to SQLITE_MUTEX_FAST.
8551 ** mutex results in undefined behavior.
8649 ** If xMutexInit fails in any way, it is expected to clean up after itself
8677 ** These routines should return true if the mutex in their argument
8706 ** next. Applications that override the built-in mutex logic must be
8722 #define SQLITE_MUTEX_STATIC_VFS1 11 /* For use by built-in VFS */
8735 ** serializes access to the [database connection] given in the argument
8753 ** ^A NULL pointer can be used in place of "main" to refer to the
8853 ** by enclosing in double-quotes) so as not to confuse the parser.
8860 ** of bytes in the keyword into *L. The string that *Z points to is not
8864 ** sqlite3_keyword_name(N,Z,L) result in undefined behavior.
8871 ** a keyword as an identifier as long as such use does not result in a
8886 ** <li> Include a digit somewhere in every identifier name.
8926 ** valid [sqlite3_str] object, though in the event of an out-of-memory
8936 ** D parameter in [sqlite3_str_new(D)] is not NULL, then the maximum
8937 ** length of the string contained in the [sqlite3_str] object will be
8954 ** string in [sqlite3_str] object X is zero bytes long.
8966 ** [sqlite3_str_vappendf(X,F,V)] interfaces uses the [built-in printf]
8973 ** zero-terminated string in its entirety, use the [sqlite3_str_appendall()]
8984 ** inside [sqlite3_str] object X back to zero bytes in length.
8987 ** is recorded in the [sqlite3_str] object and can be recovered by a
9004 ** in sqlite3_str X, then the [sqlite3_str_errcode(X)] method will return
9010 ** ^The [sqlite3_str_length(X)] method returns the current length, in bytes,
9011 ** of the dynamic string under construction in [sqlite3_str] object X.
9016 ** content of the dynamic string under construction in X. The value
9039 ** ^The highest recorded value is returned in *pHighwater. ^If the
9078 ** memory controlled by [SQLITE_CONFIG_PAGECACHE] is not included in
9080 ** sizes as reported by the xSize method in [sqlite3_mem_methods].</dd>)^
9085 ** internal equivalents). Only the value returned in the
9097 ** value returned is in pages, not in bytes.</dd>)^
9107 ** no space was left in the page cache.</dd>)^
9111 ** handed to the [pagecache memory allocator]. Only the value returned in the
9154 ** to grow in future releases of SQLite.
9175 ** New verbs may be added in future releases of SQLite. Existing verbs
9203 ** memory already being in use.
9255 ** wal file in wal mode databases, or the number of pages written to the
9256 ** database file in rollback mode databases. Any pages written as part of
9265 ** been written to disk in the middle of a transaction due to the page
9331 ** <dd>^This is the number of times that SQLite has stepped forward in
9338 ** A non-zero value in this counter may indicate an opportunity to
9343 ** were created automatically in order to help joins run faster.
9344 ** A non-zero value in this counter may indicate an opportunity to
9400 ** The sqlite3_pcache_page object represents a single page in the
9418 ** register an alternative page cache implementation by passing in an
9424 ** the amount of memory consumed by SQLite, the way in which
9431 ** The built-in page cache is recommended for most uses.
9446 ** built-in default page cache is used instead of the application defined
9459 ** in multithreaded applications.
9468 ** first parameter, szPage, is the size in bytes of the pages that must
9478 ** false if it is used for an in-memory database. The cache implementation
9497 ** stored in the cache, both pinned and unpinned.
9500 ** The xFetch() method locates a page in the cache and returns a pointer to
9506 ** for each entry in the page cache.
9512 ** If the requested page is already in the page cache, then the page cache
9514 ** intact. If the requested page is not already in the cache, then the
9519 ** <tr><th> createFlag <th> Behavior when page is not already in cache
9593 ** retained in the header file for backwards compatibility only.
9628 ** for copying in-memory databases to or from persistent files.
9658 ** temporary database, or the name specified after the AS keyword in
9672 ** returned and an error code and error message are stored in the
9703 ** <li> the destination database is an in-memory database and the
9715 ** is called, then [SQLITE_LOCKED] is returned immediately. ^Again, in this
9719 ** there is no point in retrying the call to sqlite3_backup_step(). These
9769 ** in the source database at the conclusion of the most recent
9772 ** sqlite3_backup_step(). If the source database is modified in a way that
9774 ** those changes are not reflected in the output of sqlite3_backup_pagecount()
9793 ** backup is in progress might also also cause a mutex deadlock.
9795 ** If running in [shared cache mode], the application must
9824 ** ^When running in shared-cache mode, a database operation may fail with
9849 ** ^(If sqlite3_unlock_notify() is called in a multi-threaded application,
9883 ** and the second is the number of entries in the array.
9906 ** system in a deadlocked state, then SQLITE_LOCKED is returned and no
9907 ** unlock-notify callback is registered. The system is said to be in
9925 ** sqlite3_unlock_notify() results in the unlock-notify callback being
9931 ** extended error code is set to SQLITE_LOCKED_SHAREDCACHE. Otherwise, in
9947 ** strings in a case-independent fashion, using the same definition of "case
9958 ** ^The definition of [GLOB] pattern matching used in
9959 ** [sqlite3_strglob(P,X)] is the same as for the "X GLOB P" operator in the
9975 ** ^The definition of [LIKE] pattern matching used in
9977 ** operator in the SQL dialect understood by SQLite. ^For "X LIKE P" without
10009 ** will not use dynamically allocated memory. The log message is stored in
10021 ** is invoked each time data is committed to a database in wal mode.
10032 ** is the number of pages currently in the write-ahead log file,
10064 ** more frames in the [write-ahead log] file. ^Passing zero or
10094 ** In brief, sqlite3_wal_checkpoint(D,X) causes the content in the
10114 ** operation on database X of [database connection] D in mode M. Status
10122 ** in the log were checkpointed. ^The [busy-handler callback]
10123 ** is never invoked in the SQLITE_CHECKPOINT_PASSIVE mode.
10131 ** snapshot. ^It then checkpoints all frames in the log file and syncs the
10150 ** ^If pnLog is not NULL, then *pnLog is set to the total number of frames in
10152 ** of an error or because the database is not in [WAL mode]. ^If pnCkpt is not
10153 ** NULL,then *pnCkpt is set to the total number of checkpointed frames in the
10156 ** because the database is not in WAL mode. ^Note that upon successful
10163 ** busy-handler configured, it will not be invoked in this case.
10172 ** checkpoint operation proceeds from that point in the same way as
10174 ** without blocking any further. ^SQLITE_BUSY is returned in this case.
10188 ** ^If database zDb is the name of an attached database that is not in WAL
10205 int *pnLog, /* OUT: Size of WAL log in frames */
10234 ** [database connection] in which the virtual table is being created and
10235 ** which is passed in as the first argument to the [xConnect] or [xCreate]
10281 ** SQLITE_CONSTRAINT, in which case SQLite falls back to OR ABORT
10337 ** same column in the [xUpdate] method.
10423 ** The select-id is the same value as is output in the first column
10458 ** ^Statistics might not be available for all loops in all statements. ^In cases
10489 ** pages in the pager-cache that are not currently in use are written out
10490 ** to disk. A dirty page may be in use if a database cursor created by an
10492 ** file (page 1 is always "in use"). ^The [sqlite3_db_cacheflush(D)]
10499 ** in the usual manner. ^If the required lock still cannot be obtained, then
10546 ** the name given after the AS keyword in the [ATTACH] statement for attached
10567 ** to the preupdate callback results in undefined and probably undesirable
10571 ** in the row that is being inserted, updated, or deleted.
10637 ** database for some specific point in history.
10643 ** as it existed for the point in time when the transaction first started.
10662 ** schema S in database connection D. ^On success, the
10671 ** in this case.
10674 ** <li> The database handle must not be in [autocommit mode].
10689 ** database handle in autocommit mode but fails for some other reason,
10716 ** ^In order to succeed, the database connection must not be in
10738 ** schema S is in [WAL mode]. A database connection might not know
10739 ** that the database file is in [WAL mode] if there has been no prior
10743 ** database connection in order to make it ready to use snapshots.)
10827 ** If P is not a NULL pointer, then the size of the database in bytes
10831 ** copy of the disk file. For an in-memory database or a "TEMP" database,
10872 ** a pointer to contiguous in-memory database that it is currently using,
10874 ** a contiguous in-memory database, then this option causes
10876 ** using a contiguous in-memory database if it has been initialized by a
10886 ** reopen S as an in-memory database based on the serialization contained
10887 ** in P. The serialized database P is N bytes in size. M is the size of
10889 ** the SQLITE_DESERIALIZE_READONLY bit is not set in F, then SQLite is
10890 ** permitted to add content to the in-memory database as long as the total
10893 ** If the SQLITE_DESERIALIZE_FREEONCLOSE bit is set in F, then SQLite will
10900 ** database is currently in a read transaction or is involved in a backup
10904 ** SQLITE_DESERIALIZE_FREEONCLOSE bit is set in argument F, then
10914 sqlite3_int64 szDb, /* Number bytes in the deserialization */
10926 ** in the P argument is held in memory obtained from [sqlite3_malloc64()]
10934 ** Without this flag, the deserialized database cannot increase in size beyond
11048 unsigned int *anQueue; /* Number of pending entries in the queue */
11051 int mxLevel; /* The largest iLevel value in the tree */
11057 /* The following fields are only available in 3.8.11 and later */
11131 ** The session object will be used to create changesets for tables in
11218 ** in the database. If additional tables are added to the database (by
11227 ** It is not an error if the named table does not exist in the database. Nor
11229 ** no changes will be recorded in either of these scenarios.
11232 ** in one or more of their PRIMARY KEY columns.
11248 ** rows a zero-length blob (SQL value X'') is stored in the changeset or
11276 ** in tables that are not attached to the Session object, the filter is called
11297 ** and *pnChangeset to the size of the changeset in bytes before returning
11309 ** is represented in a changeset as a DELETE followed by an INSERT.
11311 ** Changes are not recorded for rows that have NULL values stored in one or
11313 ** no corresponding change is present in the changesets returned by this
11314 ** function. If an existing row with one or more NULL values stored in
11316 ** only an INSERT is appears in the changeset. Similarly, if an existing row
11330 ** are sorted in the same order in which they were attached (or auto-attached)
11331 ** to the sqlite3_session object. The order in which the changes related to
11345 ** updated or deleted in the lifetime of the session.
11371 ** the changeset. Or, if no such row is found in the table, a DELETE
11373 ** primary key in the database, but all fields contain their original
11379 ** will be present in the changeset. Or if a row is deleted and then later a
11389 ** the same session object is disabled, no INSERT record will appear in the
11406 ** argument, this function attaches table zTbl in the same manner as the
11418 ** <li> Has the same set of columns declared in the same order, and
11428 ** used to update the table in database zFrom (call this the "from-table")
11433 ** <li> For each row (primary key) that exists in the to-table but not in
11436 ** <li> For each row (primary key) that exists in the to-table but not in
11439 ** <li> For each row (primary key) that exists in both tables, but features
11440 ** different non-PK values in each, an UPDATE record is added to the
11488 ** in the same way as for changesets.
11490 ** Changes within a patchset are ordered in the same way as for changesets
11492 ** a single table are grouped together, tables appear in the order in which
11510 ** changeset that contains no changes. This can happen when a row in
11521 ** This API returns the total amount of heap memory in bytes currently
11569 int nChangeset, /* Size of changeset blob in bytes */
11574 int nChangeset, /* Size of changeset blob in bytes */
11603 ** does not point to any change in the changeset. Assuming the changeset
11605 ** point to the first change in the changeset. Each subsequent call advances
11606 ** the iterator to point to the next change in the changeset (if any). If
11609 ** Otherwise, if all changes in the changeset have already been visited,
11634 ** *pnCol is set to the number of columns in the table affected by the change; and
11648 ** be trusted in this case.
11653 int *pnCol, /* OUT: Number of columns in table */
11665 ** <li> The number of columns in the table, and
11672 ** nCol is the number of columns in the table. Elements of *pabPK are set to
11677 ** in the table.
11687 int *pnCol /* OUT: Number of entries in output array */
11703 ** of columns in the table affected by the current change. Otherwise,
11734 ** of columns in the table affected by the current change. Otherwise,
11766 ** of columns in the table affected by the current change. Otherwise,
11790 ** violations in the destination database and returns SQLITE_OK.
11814 ** function (for example an [SQLITE_CORRUPT] in [sqlite3changeset_next()] or an
11815 ** [SQLITE_NOMEM] in [sqlite3changeset_new()]) then an error code corresponding
11845 ** This function does not change the order in which changes appear within
11849 ** is stored in *ppOut, the size of the same buffer is stored in *pnOut, and
11892 int nA, /* Number of bytes in buffer pA */
11894 int nB, /* Number of bytes in buffer pB */
11896 int *pnOut, /* OUT: Number of bytes in output changeset */
11916 ** always in the same format as the input.
11939 ** new() and delete(), and in any order.
11951 ** Add all changes within the changeset (or patchset) in buffer pData (size
11960 ** Rows within the changeset and changegroup are identified by the values in
11961 ** their PRIMARY KEY columns. A change in the changeset is considered to
11962 ** apply to the same row as a change already present in the changegroup if
11965 ** Changes to rows that do not already appear in the changegroup are
11979 ** The INSERT change remains in the changegroup. The values in the
11997 ** If one or more of the column values in the row inserted by the
11998 ** new change differ from those in the row deleted by the existing
12013 ** in the changegroup, then the number of columns and the position of the
12036 ** table are grouped together in the output of this function. Tables appear
12037 ** in the same order as for the very first changeset added to the changegroup.
12039 ** changes for tables that do not appear in the first changeset, they are
12040 ** appended onto the end of the output changeset, again in the order in
12052 int *pnData, /* OUT: Size of output buffer in bytes */
12066 ** update the "main" database attached to handle db with the changes found in
12071 ** change in the changeset, the filter callback is invoked with
12083 ** <li> The table has the same name as the name recorded in the
12085 ** <li> The table has at least as many columns as recorded in the
12087 ** <li> The table has primary key columns in the same position as
12088 ** recorded in the changeset.
12094 ** one such warning is issued for each table in the changeset.
12123 ** original row values stored in the changeset. If it does, and the values
12124 ** stored in all non-primary key columns also match the values stored in
12129 ** row value stored in the changeset, the conflict-handler function is
12131 ** database table has more columns than are recorded in the changeset,
12136 ** If no row with matching primary key values is found in the database,
12168 ** original row values stored in the changeset. If it does, and the values
12169 ** stored in all modified non-primary key columns also match the values
12170 ** stored in the changeset the row is updated within the target database.
12174 ** original row value stored in the changeset, the conflict-handler function
12180 ** If no row with matching primary key values is found in the database,
12197 ** All changes made by these functions are enclosed in a savepoint transaction.
12207 ** is set to the size of the buffer in bytes. It is the responsibility of the
12222 int nChangeset, /* Size of changeset in bytes */
12237 int nChangeset, /* Size of changeset in bytes */
12262 ** a single call to apply_v2() or apply_v2_strm() in a [SAVEPOINT]. The
12286 ** PRIMARY KEY fields is present in the database, but one or more other
12290 ** The conflicting row, in this case, is the database row with the matching
12296 ** required PRIMARY KEY fields is not present in the database.
12298 ** There is no conflicting row in this case. The results of invoking the
12304 ** in duplicate primary key values.
12306 ** The conflicting row in this case is the database row with the matching
12311 ** database in a state containing foreign key violations, the conflict
12327 ** There is no conflicting row in this case. The results of invoking the
12347 ** continues to the next change in the changeset.
12377 ** Suppose there is a site hosting a database in state S0. And that
12381 ** applied to the database. The database is then in state
12386 ** do not have to be resolved elsewhere in the network.
12415 ** to reflect the new.* values in the UPDATE.
12420 ** is changed into an INSERT. Any undefined values in the new.* record
12421 ** from the update change are filled in using the old.* values from
12426 ** the old.* values are rebased using the new.* values in the remote
12448 ** combined on a per-field basis, not per-row. This means that in the
12464 ** multiple times, in the same order that the multiple
12503 ** in size. This function allocates and populates a buffer with a copy
12507 ** (*pnOut) to its size in bytes and SQLITE_OK returned. It is the
12546 ** require that the entire changeset be stored in a single buffer in memory.
12549 ** Normally this is convenient. However, if an application running in a
12572 ** argument, pData, points to a buffer (*pnData) bytes in size. Assuming no
12588 ** return them in chunks by way of a callback function instead of via a
12607 ** points to a buffer nData bytes in size containing the chunk of output
12705 ** changes to the sessions module in order to tune it to the specific needs
12722 ** and output data in approximately 1 KiB chunks. This operand may be used
12764 ** Interfaces to extend FTS5. Using the interfaces defined in this file,
12795 int nVal, /* Number of values in apVal[] array */
12813 ** to the total number of tokens in the FTS5 table. Or, if iCol is
12814 ** non-negative but less than the number of columns in the table, return
12815 ** the total number of tokens in column iCol, considering all rows in
12819 ** in the table, SQLITE_RANGE is returned. Or, if an error occurs (e.g.
12824 ** Return the number of columns in the table.
12828 ** to the total number of tokens in the current row. Or, if iCol is
12829 ** non-negative but less than the number of columns in the table, set
12830 ** *pnToken to the number of tokens in column iCol of the current row.
12833 ** in the table, SQLITE_RANGE is returned. Or, if an error occurs (e.g.
12843 ** containing the text in utf-8 encoding, (*pn) is set to the size in bytes
12849 ** Returns the number of phrases in the current query expression.
12852 ** Returns the number of tokens in phrase iPhrase of the query. Phrases
12872 ** to the column in which it occurs and *piOff the token offset of the
12893 ** phrase iPhrase of the current query is included in $p. For each
12948 ** This function is used to retrieve the total number of rows in the table.
12987 ** of iterating through all instances of a phrase in the current row, these
12988 ** APIs are used to iterate through the set of columns in the current row
13119 ** For each token in the input string, the supplied callback xToken() must
13123 ** size of the token in bytes. The 4th and 5th arguments are the byte offsets
13131 ** FTS5 assumes the xToken() callback is invoked for each token in the
13144 ** Custom tokenizers may also support synonyms. Consider a case in which a
13146 ** built-in tokenizers, the FTS5 query 'first + place' will match instances
13150 ** the user specified in the MATCH query text.
13152 ** There are several ways to approach this in FTS5:
13156 ** same token for inputs "first" and "1st". Say that token is in
13173 ** first token in the MATCH query and FTS5 effectively runs a query
13193 ** 'first + place' or '1st + place', as there are entries in the
13213 ** by making multiple calls to xToken(FTS5_TOKEN_COLOCATED) in sequence.
13219 ** so it is efficient in terms of disk space and query speed. However, it
13262 int nToken, /* Size of token in bytes */
13330 /************** Continuing where we left off in sqliteInt.h ******************/
13341 /************** Include sqliteLimit.h in the middle of sqliteInt.h ***********/
13359 ** The maximum length of a TEXT or BLOB in bytes. This also
13360 ** limits the size of a row in a table or index.
13372 ** * Columns in a table
13373 ** * Columns in an index
13374 ** * Columns in a view
13375 ** * Terms in the SET clause of an UPDATE statement
13376 ** * Terms in the result set of a SELECT statement
13377 ** * Terms in the GROUP BY or ORDER BY clauses of a SELECT statement.
13378 ** * Terms in the VALUES clause of an INSERT statement
13381 ** tell you that in a well-normalized database, you usually should
13382 ** not have more than a dozen or so columns in any table. And if
13383 ** that is the case, there is no point in having more than a few
13384 ** dozen values in any of the other situations described above.
13391 ** The maximum length of a single SQL statement in bytes.
13412 ** The maximum number of terms in a compound SELECT statement.
13417 ** any limit on the number of terms in a compount SELECT.
13424 ** The maximum number of opcodes in a VDBE program.
13439 ** The suggested maximum number of in-memory pages to use for
13452 ** The default number of frames to accumulate in the log file before
13453 ** checkpointing the database in WAL mode.
13525 ** Maximum number of pages in one database file.
13536 ** Maximum length (in bytes) of the pattern in a LIKE or GLOB
13555 /************** Continuing where we left off in sqliteInt.h ******************/
13567 ** WAL mode depends on atomic aligned 32-bit loads and stores in a few
13605 ** So we have to define the macros in different ways depending on the
13666 ** logic is partially replicated in ctime.c. If it is updated here, it should
13687 ** default unless SQLite is compiled with SQLITE_DEFAULT_MEMSTATUS=0 in
13695 ** Exactly one of the following macros must be defined in order to
13736 ** We need to define _XOPEN_SOURCE as follows in order to enable
13751 ** assert() statements in the code. So we want the default action
13753 ** is set. Thus NDEBUG becomes an opt-in rather than an opt-out
13771 ** The testcase() macro is used to aid in coverage testing. When
13773 ** testcase() must be evaluated both true and false in order to
13775 ** to help ensure adequate test coverage in places where simple
13820 ** are included in a few cases in order to enhance the resilience
13906 ** Is the sqlite3ErrName() function needed in the build? Currently,
13927 ** to fit in 32-bits. This macro is used inside of various testcase()
13936 ** in theory, be used by the compiler to generate better code, but
13942 /************** Include hash.h in the middle of sqliteInt.h ******************/
13956 ** used in SQLite.
13976 ** There are Hash.htsize buckets. Each bucket points to a spot in
13978 ** element pointed to plus the next _ht.count-1 elements in the list.
13983 ** in the table, it is faster to do a linear search than to manage
13987 unsigned int htsize; /* Number of buckets in the hash table */
13988 unsigned int count; /* Number of entries in this table */
13996 /* Each element in the hash table is an instance of the following
14003 HashElem *next, *prev; /* Next and previous elements in the table */
14035 ** Number of entries in a hash table
14042 /************** Continuing where we left off in sqliteInt.h ******************/
14043 /************** Include parse.h in the middle of sqliteInt.h *****************/
14229 /************** Continuing where we left off in sqliteInt.h ******************/
14240 ** should not be used in production. This is a performance measurement
14299 ** Provide a default value for SQLITE_TEMP_STORE in case it is not specified
14443 ** that can be stored in a u32 without loss of data. The value
14445 ** have to specify the value in the less intuitive manner shown:
14450 ** The datatype used to store estimates of the number of rows in a
14487 ** Set the SQLITE_PTRSIZE macro to the number of bytes in a pointer
14607 ** Default maximum size of memory used by memory-mapped I/O in the VFS
14707 ** A convenience macro that returns the number of elements in
14766 ** For example those parameters only used in assert() statements. In these
14836 ** tables in a join to 32 instead of 64. But it also reduces the size
14846 ** The number of bits in a Bitmask. "BMS" means "BitMask Size".
14851 ** A bit in a Bitmask
14859 ** in the SQL statement (such as $abc, @pqr, or :xyz) and the integer
14871 /************** Include pager.h in the middle of sqliteInt.h *****************/
14902 ** The type used to represent a page number. The first page in a file
14918 ** Page number PAGER_MJ_PGNO is never used in an SQLite database (it is
14920 ** used in the journal to signify that the remainder of the journal file
14922 ** roll back. See comments for function writeSuperJournal() in pager.c
14930 ** NOTE: These values must match the corresponding BTREE_ values in btree.h.
14946 ** are exposed in the API via the "PRAGMA journal_mode" command and
15117 /************** Continuing where we left off in sqliteInt.h ******************/
15118 /************** Include btree.h in the middle of sqliteInt.h *****************/
15132 ** subsystem. See comments in the source code for a detailed description
15176 ** NOTE: These values must match the corresponding PAGER_ values in
15180 #define BTREE_MEMORY 2 /* This is an in-memory DB */
15217 /* in vdbe.c and pager.c See https://sqlite.org/lang_savepoint.html */
15236 ** is stored in the leaves. (BTREE_INTKEY is used for SQL tables.) With
15257 ** to constants so that the offset of the corresponding field in an
15266 ** The BTREE_DATA_VERSION value is not really a value stored in the header.
15291 ** TK_REGISTER nodes that refer to values stored in the array of registers
15293 ** then the value of the node is the value in Mem[pExpr.iTable]. Any
15294 ** TK_COLUMN node in the expression tree refers to the Expr.iColumn-th
15316 ** to be filled with content that is already in sorted order.
15324 #define BTREE_BULKLOAD 0x00000001 /* Used to full index in sorted order */
15342 ** alternative storage engines that might be substituted in place of this
15343 ** b-tree system. For alternative storage engines in which a delete of
15387 ** entry in either an index or table btree.
15396 ** the key and passed in the nKey field. The pKey field is zero.
15409 ** nMem not used entries in aMem
15423 sqlite3_value *aMem; /* First of nMem value in the unpacked pKey */
15530 /************** Continuing where we left off in sqliteInt.h ******************/
15531 /************** Include vdbe.h in the middle of sqliteInt.h ******************/
15556 ** in the source file sqliteVdbe.c are allowed to see the insides
15562 ** The names of the following types declared in vdbeInt.h are required
15609 ** with flags in the upper 8 bits */
15620 int nOp; /* Elements in aOp[] */
15701 /************** Include opcodes.h in the middle of vdbe.h ********************/
15750 #define OP_RowSetTest 45 /* jump, synopsis: if r[P3] in rowset(P1) goto P2 */
15820 #define OP_OpenPseudo 115 /* synopsis: P3 columns in r[P2] */
15885 /* Properties such as "out2" or "jump" that are specified in
15886 ** comments following the "case" for each opcode in the vdbe.c
15929 /************** Continuing where we left off in vdbe.h ***********************/
16061 ** comments in VDBE programs that show key decision points in the code
16083 ** numbers in the sqlite3.c source file. VDBE branch coverage testing
16102 ** // in distingishing equal and not-equal.
16106 ** -DSQLITE_DEBUG then an ALWAYS() will fail in the vdbeTakeBranch()
16107 ** routine in vdbe.c, alerting the developer to the missed tag.
16117 ** See the VdbeBranchTaken() macro and vdbeTakeBranch() function in the
16159 /************** Continuing where we left off in sqliteInt.h ******************/
16160 /************** Include pcache.h in the middle of sqliteInt.h ****************/
16183 ** Every page in the cache is controlled by an instance of the following
16205 PgHdr *pDirtyNext; /* Next element in list of dirty pages */
16206 PgHdr *pDirtyPrev; /* Previous element in list of dirty pages */
16247 /* Return the size in bytes of a PCache object. Used to preallocate
16272 /* Get a list of all dirty pages in the cache, sorted by page number */
16292 /* Return the total number of pages stored in the cache */
16296 /* Iterate through all dirty pages currently stored in the cache. This
16321 ** The spill-size is the minimum number of pages in cache before the cache
16354 /************** Continuing where we left off in sqliteInt.h ******************/
16355 /************** Include os.h in the middle of sqliteInt.h ********************/
16383 /************** Include os_setup.h in the middle of os.h *********************/
16444 /************** Continuing where we left off in os.h *************************/
16462 ** alphanumeric characters, and no file extension. They are stored in the
16464 ** If sqlite is being embedded in another program, you may wish to change the
16470 ** Mcafee started using SQLite in their anti-virus product and it
16471 ** started putting files with the "sqlite" name in the c:/temp folder.
16520 ** An EXCLUSIVE_LOCK is obtained by locking all bytes in the range.
16529 ** for probabilistic locking in Win95/98/ME. Hence, the locking scheme
16535 ** SHARED_SIZE is the number of bytes available in the pool from which
16547 ** Locking in windows is manditory. For this reason, we cannot store
16548 ** actual data in the bytes used for locking. The pager never allocates
16549 ** the pages involved in locking therefore. SHARED_SIZE is selected so
16556 ** Changing the value of PENDING_BYTE results in a subtly incompatible
16632 /************** Continuing where we left off in sqliteInt.h ******************/
16633 /************** Include mutex.h in the middle of sqliteInt.h *****************/
16649 ** to all source files. We break it out in an effort to keep the code
16708 /************** Continuing where we left off in sqliteInt.h ******************/
16744 ** in the sqlite.aDb[] array. aDb[0] is the main database file and
16768 ** A thread must be holding a mutex on the corresponding Btree in order
16784 int cache_size; /* Number of pages to use in the cache */
16788 ** These macros can be used to test, set, or clear bits in the
16825 ** lookaside malloc subsystem. Each available memory allocation in
16831 ** be stored in lookaside because in shared cache mode the schema information
16839 ** in a performance-critical path. sz should be set by to szTrue whenever
16846 ** outstanding at any point in the past - by subtracting the number of
16863 u16 sz; /* Size of each buffer in bytes */
16880 LookasideSlot *pNext; /* Next buffer in the list of free buffers */
16887 /* Size of the smaller allocations in two-size lookside */
16895 ** A hash table for built-in function definitions. (Application-defined
16910 ** Information held in the "sqlite3" database connection object and used
16916 int nAuthPW; /* Size of the zAuthPW in bytes */
16948 ** in the style of sqlite3_trace()
16973 int nDb; /* Number of backends currently in use */
17072 VTable *pDisconnect; /* Disconnect these in next sqlite3_prepare() */
17080 int busyTimeout; /* Busy handler timeout, in msec */
17085 int *pnBytesFreed; /* If not NULL, increment this in DbFree() */
17088 ** mutex, not by sqlite3.mutex. They are used by code in notify.c.
17101 sqlite3 *pNextBlocked; /* Next in list of all blocked connections */
17116 ** upper 32 bits are included in the argument. Necessary because some
17130 #define SQLITE_LegacyFileFmt 0x00000002 /* Create new databases in format 1 */
17137 ** vtabs in the schema definition */
17141 #define SQLITE_ReadUncommit 0x00000400 /* READ UNCOMMITTED in shared-cache */
17160 #define SQLITE_DqsDDL 0x20000000 /* dbl-quoted strings allowed in DDL*/
17161 #define SQLITE_DqsDML 0x40000000 /* dbl-quoted strings allowed in DML*/
17181 #define DBFLAG_PreferBuiltin 0x0002 /* Preference to built-in funcs */
17182 #define DBFLAG_Vacuum 0x0004 /* Currently in a VACUUM */
17201 #define SQLITE_OmitNoopJoin 0x00000100 /* Omit unused tables in joins */
17234 #define SQLITE_MAGIC_BUSY 0xf03b7906 /* Database currently in use */
17240 ** structure. For global built-in functions (ex: substr(), max(), count())
17241 ** a pointer to this structure is held in the sqlite3BuiltinFunctions object.
17243 ** structure is held in the db->aHash hash table.
17288 ** are assert() statements in the code to verify this.
17297 ** SQLITE_FUNC_ENCMASK depends on SQLITE_UTF* macros in the API
17304 #define SQLITE_FUNC_LENGTH 0x0040 /* Built-in length() function */
17305 #define SQLITE_FUNC_TYPEOF 0x0080 /* Built-in typeof() function */
17306 #define SQLITE_FUNC_COUNT 0x0100 /* Built-in count(*) aggregate */
17308 #define SQLITE_FUNC_UNLIKELY 0x0400 /* Built-in unlikely() function */
17313 #define SQLITE_FUNC_TEST 0x4000 /* Built-in testing functions */
17314 #define SQLITE_FUNC_OFFSET 0x8000 /* Built-in sqlite_offset() function */
17315 #define SQLITE_FUNC_WINDOW 0x00010000 /* Built-in window-only function */
17317 #define SQLITE_FUNC_DIRECT 0x00080000 /* Not for use in TRIGGERs or VIEWs */
17320 #define SQLITE_FUNC_INLINE 0x00400000 /* Functions implemented in-line */
17322 /* Identifier numbers for each in-line function */
17350 ** zName is the name of a function that is implemented by in-line
17356 ** zName is the name of a test-only function implemented by in-line
17380 ** are interpreted in the same way as the first 4 parameters to
17386 ** are interpreted in the same way as the first 4 parameters to
17440 ** All current savepoints are stored in a linked list starting at
17441 ** sqlite3.pSavepoint. The first element in the list is the most recently
17463 ** instance of the following structure, stored in the sqlite3.aModule
17476 ** Information about each column of an SQL table is held in an instance
17477 ** of the Column structure, in the Table.aCol[] array.
17481 ** "table column index" This is the index of the column in the
17483 ** the column in the original CREATE TABLE stmt.
17485 ** "storage column index" This is the index of the column in the
17498 u8 szEst; /* Estimated size of value in this column. sizeof(INT)==1 */
17511 #define COLFLAG_HIDDEN 0x0002 /* A hidden column in a virtual table */
17542 #define SQLITE_SO_ASC 0 /* Sort in ascending order */
17543 #define SQLITE_SO_DESC 1 /* Sort in ascending order */
17586 #define SQLITE_STOREP2 0x20 /* Store result in reg[P2] rather than jump */
17591 ** An object of this type is created for each virtual table present in
17599 ** database connections, even when the rest of the in-memory database
17608 ** All VTable objects that correspond to a single table in a shared
17609 ** database schema are initially stored in a linked-list pointed to by
17614 ** sqlite3_vtab* handle in the compiled query.
17616 ** When an in-memory Table object is deleted (for example when the
17629 ** sqlite3DbMalloc(), using the connection handle stored in VTable.db as
17640 VTable *pNext; /* Next in linked list (see above) */
17650 ** The schema for each SQL table and view is represented in memory
17658 FKey *pFKey; /* Linked list of all foreign keys in this table */
17661 /* ... also used as column name list in a VIEW */
17666 i16 nCol; /* Number of columns in this table */
17668 LogEst nRowLogEst; /* Estimated rows in table - from sqlite_stat1 table */
17669 LogEst szTabRow; /* Estimated size of each table row in bytes */
17673 u8 keyConf; /* What to do in case of uniqueness conflict on iPKey */
17675 int addColOffset; /* Offset in CREATE TABLE stmt to add a new column */
17682 Trigger *pTrigger; /* List of triggers stored in pSchema */
17692 ** the TF_OOOHidden attribute would apply in this case. Such tables require
17760 ** key. The "to" table is the table that is named in the REFERENCES clause.
17781 ** is held in Schema.fkeyHash with a hash key of Z.
17785 FKey *pNextFrom; /* Next FKey with the same in pFrom. Next parent of pFrom */
17789 int nCol; /* Number of columns in this key */
17794 struct sColMap { /* Mapping of columns in pFrom to columns in zTo */
17795 int iFrom; /* Index of column in pFrom */
17796 char *zCol; /* Name of column in zTo. If NULL use PRIMARY KEY */
17803 ** causes the operation in process to fail and for the current transaction
17804 ** to be rolled back. ABORT processing means the operation in process
17807 ** the operation in progress stops and returns an error code. But prior
17837 #define OE_Update 6 /* Process as a DO UPDATE in an upsert */
17857 u16 nKeyField; /* Number of key columns in the index */
17865 ** Allowed bit values for entries in the KeyInfo.aSortFlags[] array.
17887 ** or greater than a key in the btree, respectively. These are normally
17889 ** is in DESC order.
17893 ** multiple entries in the b-tree with the same key (when only looking
17902 ** eqSeen field will indicate whether or not an exact match exists in the
17908 u16 nField; /* Number of entries in apMem[] */
17918 ** Each SQL index is represented in memory by an
17929 ** three columns in the table. In the Index structure describing
17932 ** first column to be indexed (c3) has an index of 2 in Ex1.aCol[].
17933 ** The second column to be indexed (c1) has an index of 0 in
17943 ** While parsing a CREATE TABLE or CREATE INDEX statement in order to
17964 LogEst szIdxRow; /* Estimated average row size in bytes */
17966 u16 nColumn; /* Number of columns stored in the index */
17979 int nSample; /* Number of elements in aSample[] */
17981 tRowcnt *aAvgEq; /* Average nEq values for keys not in aSample */
17984 tRowcnt nRowEst0; /* Non-logarithmic number of rows in the index */
17986 Bitmask colNotIdxed; /* 0 for unindexed columns in pTab */
18010 ** Each sample stored in the sqlite_stat4 table is represented in memory
18016 int n; /* Size of record in bytes */
18035 ** in the middle of the Parse.zSql text. But it might also point to a
18040 unsigned int n; /* Number of characters in this token */
18048 ** pointer to this structure. The Expr.iAgg field is the index in
18063 int nSortingColumn; /* Number of columns in the sorting index */
18066 struct AggInfo_col { /* For each column used in source tables */
18072 i16 iSorterColumn; /* Column number in the sorting index */
18074 int nColumn; /* Number of used entries in aCol[] */
18084 int nFunc; /* Number of entries in aFunc[] */
18092 ** it uses less memory in the Expr object, which is a big memory user
18093 ** in systems with lots of prepared statements. And few applications
18105 ** Each node of an expression in the parse tree is an instance
18111 ** to represent the greater-than-or-equal-to operator in the expression
18126 ** the form "<lhs> IN (SELECT ...)". If the EP_xIsSelect bit is set in the
18130 ** An expression of the form ID or ID.ID refers to a column in a table.
18134 ** expression is used as a result in an aggregate SELECT, then the
18135 ** value is also stored in the Expr.iAgg column in the aggregate so that
18139 ** character '?' in the original SQL) then the Expr.iTable holds the index
18154 ** Expr objects can use a lot of memory space in database schema. To
18157 ** two or more Expr objects will be stored in a single memory allocation,
18162 ** the child Expr objects in the Expr.pLeft and Expr.pRight subtrees
18164 ** the subtrees in Expr.x.pList or Expr.x.pSelect are always separately
18183 /* If the EP_TokenOnly flag is set in the Expr.flags mask, then no
18185 ** access them will result in a segfault or malfunction.
18195 /* If the EP_Reduced flag is set in the Expr.flags mask, then no
18197 ** access them will result in a segfault or malfunction.
18213 i16 iAgg; /* Which entry in pAggInfo->aCol[] or ->aFunc[] */
18228 ** The following are the meanings of bits in the Expr.flags field.
18234 #define EP_FromJoin 0x000001 /* Originates in ON/USING clause of outer join */
18240 #define EP_DblQuoted 0x000040 /* token.z was originally in "..." */
18244 #define EP_IntValue 0x000400 /* Integer value contained in u.iValue */
18261 #define EP_Static 0x8000000 /* Held in memory not obtained from malloc() */
18274 ** These macros can be used to test, set, or clear bits in the
18306 ** struct, an Expr struct with the EP_Reduced flag set in Expr.flags
18333 ** name. An expr/name combination can be used in several ways, such
18334 ** as the list of "expr AS ID" fields following a "SELECT" or in the
18335 ** list of "ID = expr" items in an UPDATE. A list of expressions can
18336 ** also be used as the argument to a function, in which case the a.zName
18355 struct ExprList_item { /* For each expression in the list */
18366 u16 iOrderByCol; /* For ORDER BY, column number in result set */
18369 int iConstExprReg; /* Register in which Expr value is cached */
18371 } a[1]; /* One slot for each expression in the list */
18383 ** such as the list "a,b,c" in the following statements:
18390 ** column names after a table name in an INSERT statement. In the statement
18399 int idx; /* Index in some Table.aCol[] of a column named zName */
18405 ** The SrcItem object represents a single term in the FROM clause of a query.
18414 Select *pSelect; /* A SELECT statement used in place of a table name */
18425 unsigned isRecursive :1; /* True for recursive reference in WITH */
18445 ** Each table or subquery in the FROM clause is a separate element of
18451 ** such a table must be a simple name: ID. But in SQLite, the table can
18463 int nSrc; /* Number of tables or subqueries in the FROM clause */
18464 u32 nAlloc; /* Number of entries allocated in a[] below */
18503 #define WHERE_USE_LIMIT 0x4000 /* Use the LIMIT in cost estimates */
18514 ** A NameContext defines a context in which to resolve table and column
18530 ** NameContext in the parent query. Thus the process of scanning the
18575 #define NC_IsDDL 0x10000 /* Resolving names in a CREATE statement */
18581 ** clause in an upsert.
18598 Upsert *pNextUpsert; /* Next ON CONFLICT clause in the list */
18623 ** These addresses must be stored so that we can go back and fill in
18625 ** the number of columns in P2 can be computed at the same time
18645 Select *pPrior; /* Prior select in a compound select statement */
18646 Select *pNext; /* Next select to the left in a compound */
18681 #define SF_IncludeHidden 0x0020000 /* Include hidden columns in output */
18692 ** The results of a SELECT can be distributed in several ways, as defined
18696 ** SRT_Union Store results as a key in a temporary index
18701 ** SRT_Exists Store a 1 in memory cell pDest->iSDParm if the result
18709 ** opcode) for each row in the result set.
18713 ** in register pDest->iSDParm then abandon the rest
18717 ** row of result as the key in table pDest->iSDParm.
18729 ** of the co-routine is stored in register pDest->iSDParm
18730 ** and the result row is stored in pDest->nDest registers
18733 ** SRT_Table Store results in temporary table pDest->iSDParm.
18739 ** SRT_DistFifo Store results in a temporary table pDest->iSDParm.
18744 ** SRT_Queue Store results in priority queue pDest->iSDParm (really
18748 ** SRT_DistQueue Store results in priority queue pDest->iSDParm only if
18752 ** SRT_Upfrom Store results in the temporary table already opened by
18754 ** table is an intkey table - in this case the first
18757 ** table. (pDest->iSDParm) is the number of key columns in
18758 ** each index record in this case.
18760 #define SRT_Union 1 /* Store result as keys in an index */
18771 #define SRT_Queue 7 /* Store result in an queue */
18778 #define SRT_Mem 10 /* Store result in a memory cell */
18779 #define SRT_Set 11 /* Store results as keys in an index */
18804 ** information in case inserts are done within triggers. Triggers do not
18809 AutoincInfo *pNext; /* Next info block in a list of them all */
18811 int iDb; /* Index in sqlite3.aDb[] of database holding pTab */
18818 ** statement. All such objects are stored in the linked list headed at
18824 ** TriggerPrg.orconf, is stored in the TriggerPrg.pProgram variable.
18835 TriggerPrg *pNext; /* Next entry in Parse.pTriggerPrg list */
18873 ** the parser and down into all the parser action routine in order to
18895 u8 nTempReg; /* Number of temporary registers in aTempReg[] */
18906 int iRangeReg; /* First register in temporary register block */
18914 int nLabelAlloc; /* Number of slots in aLabel */
18925 int nTableLock; /* Number of locks in aTableLock */
18958 ** first field in the recursive region.
18962 ynVar nVar; /* Number of '?' variables seen in the SQL so far */
19038 ** Bitfield flags for P5 value in various opcodes.
19050 /* Also used in P2 (not P5) of OP_Delete */
19056 #define OPFLAG_USESEEKRESULT 0x10 /* Try to avoid a seek in BtreeInsert() */
19066 #define OPFLAG_AUXDELETE 0x04 /* OP_Delete: index in a DELETE op */
19071 * Each trigger present in the database schema is stored as an instance of
19074 * Pointers to instances of struct Trigger are stored in two ways.
19114 * Instances of struct TriggerStep are stored in a singly linked list (linked
19161 TriggerStep *pNext; /* Next in the link-list */
19162 TriggerStep *pLast; /* Last element in link-list. Valid for 1st elem only */
19174 int nRetCol; /* Number of in pReturnEL after expansion */
19180 ** do not necessarily know how big the string will be in the end.
19185 u32 nAlloc; /* Amount of space allocated in zText */
19230 u8 bExtraSchemaChecks; /* Verify type,name,tbl_name in schema */
19245 int szPage; /* Size of each page in pPage[] */
19246 int nPage; /* Number of pages in pPage[] */
19253 int inProgress; /* True while initialization in progress */
19280 u32 szSorterRef; /* Min size in bytes to use sorter-refs */
19343 const char *zDb; /* Make sure all objects are contained in this database */
19396 int nCte; /* Number of CTEs in the WITH clause */
19398 Cte a[1]; /* For each CTE in the WITH clause.... */
19417 LogEst nRowEst; /* Estimated number of rows in the table */
19429 u8 bLine[100]; /* Draw vertical in column i if bLine[i] is true */
19434 ** This object is used in various ways, most (but not all) related to window
19438 ** the Expr.y.pWin field for each window function in an expression tree.
19439 ** This object holds the information contained in the OVER clause,
19442 ** (2) All window functions in a single SELECT form a linked-list
19450 ** of this object is stored in Expr.y.pWin with eFrmType set to
19454 ** to be accessible in two different ways. Use case (3) are separate objects.
19468 Window **ppThis; /* Pointer to this object in Select.pWin list */
19479 int nBufferCol; /* Number of columns in buffer table */
19570 ** needed by FTS3 when FTS3 is included in the amalgamation.
20123 ** be defined locally, but now we use the varint routines in the util.c
20456 /* Declarations for functions in fkey.c. All of these are replaced by
20494 ** The interface to the code in fault.c used for identifying "benign"
20584 ** sqlite3MemdebugHasType() returns true if any of the bits in its second
20588 ** sqlite3MemdebugNoType() returns true if none of the bits in its second
20707 ** The following 256 byte lookup table is used to support SQLites built-in
20729 ** Bit 0x40 is set if the character is non-alphanumeric and can be used in an
20803 ** page size in bytes.
20810 ** threshold (in bytes). 0 means that statement journals are created and
20812 ** before 3.12.0). -1 means always keep the entire statement journal in
20813 ** memory. (The statement journal is also always held entirely in memory
20823 ** number of bytes in each lookaside slot (should be a multiple of 8)
20843 /* The default maximum size of an in-memory database created using
20919 ** The following performance counter can be used in place of
20927 ** 1-gibabyte boundary) in a compatible database. SQLite never uses
20933 ** a different position in the file. This allows code that has to
20939 ** 0x40000000 results in an incompatible database file format!
20940 ** Changing the pending byte during operation will result in undefined
20957 ** from the comments following the "case OP_xxxx:" statements in
20985 /************** Include vdbeInt.h in the middle of status.c ******************/
21039 /* Opaque type used by code in vdbesort.c */
21055 ** - In the main database or in an ephemeral database
21059 ** * A one-row "pseudotable" stored in a single register
21064 i8 iDb; /* Index of cursor database in db->aDb[] (or -1) */
21104 i16 nField; /* Number of fields in the header */
21109 u32 payloadSize; /* Total number of bytes in the record */
21110 u32 szRow; /* Byte available in aRow */
21116 ** static element declared in the structure. nField total array slots for
21131 ** values stored in the Vdbe struct. When the sub-program is finished,
21137 ** cell in the parent (calling) frame. When the memory cell is deleted or
21140 ** list is deleted when the VM is reset in VdbeHalt(). The reason for doing
21145 ** The currently executing frame is stored in Vdbe.pFrame. Vdbe.pFrame is
21163 int nCursor; /* Number of entries in apCsr */
21164 int pc; /* Program Counter in parent (calling) frame */
21166 int nMem; /* Number of entries in aMem */
21189 double r; /* Real value used when MEM_Real is set in flags */
21190 i64 i; /* Integer value used when MEM_Int is set in flags */
21198 int n; /* Number of characters in string value, excluding '\0' */
21203 u32 uTemp; /* Transient storage for serial_type in OP_MakeRecord */
21219 ** representations of the value stored in the Mem struct.
21226 ** Usually this is encoded in the same unicode encoding as the main
21249 #define MEM_Term 0x0200 /* String in Mem.z is zero terminated */
21290 ** implementation calling sqlite3_set_auxdata() is stored in an instance
21292 ** are stored in a linked list headed at Vdbe.pAuxData. All are destroyed
21300 AuxData *pNextAux; /* Next element in list */
21308 ** There is a typedef for this structure in sqlite.h. So all routines,
21355 DblquoteStr *pNextStr; /* Next string literal in the list */
21370 ynVar nVar; /* Number of entries in aVar[] */
21373 int nCursor; /* Number of slots in apCsr[] */
21392 int nOp; /* Number of instructions in the program */
21405 u16 nResColumn; /* Number of columns in one row of the result set */
21406 u8 errorAction; /* Recovery action to do in case of an error */
21428 int nFrame; /* Number of frames in pFrame list */
21434 int nScan; /* Entries in aScan[] */
21613 /************** Continuing where we left off in status.c *********************/
21616 ** Variables in which to record status information.
22007 ** There is only one exported symbol in this file - the function
22013 ** in Greenwich on November 24, 4714 B.C. according to the Gregorian
22030 ** in the following text:
22048 ** defined in "os_win.c".
22063 int tz; /* Timezone offset in minutes */
22066 char rawS; /* Raw numeric value stored in s */
22098 ** the 2-digit day which is the last integer in the set.
22151 ** of change in p->tz and return 0. If a parser error occurs,
22344 ** If the value is a valid unix timestamp, put it in p->s and set p->rawS.
22502 ** library function localtime_r() is used to assist in the calculation of
22537 ** Compute the difference (in milliseconds) between localtime and UTC
22538 ** (a.k.a. GMT) for the time value p where p is in UTC. If no error occurs,
22542 ** is undefined in this case.
22644 ** is in a system call (i.e. localtime()), then an error message is written
22651 int n, /* Length of zMod in bytes */
22907 ** Return the julian day number of the date specified in the arguments
23221 ** functions. This should be the only routine in this file with
23393 ** by the SQLite core, so there is no point in simulating OOMs for them.
23579 ** error in sqlite3_os_init() by the upper layers can be tested.
23816 ** This routine is the only routine in this file with external linkage.
23818 ** Populate the low-level memory allocation function pointers in
23819 ** sqlite3GlobalConfig.m with pointers to the routines in this file.
23856 ** routines specified in the sqlite3_mem_methods object. The content of
23860 ** default configuration is to use memory allocation routines in this
24110 ** This routine is the only routine in this file with external linkage.
24112 ** Populate the low-level memory allocation function pointers in
24113 ** sqlite3GlobalConfig.m with pointers to the routines in this file.
24148 ** information to each allocation in order to help detect and fix memory
24152 ** routines specified in the sqlite3_mem_methods object.
24212 ** when this module is combined with other in the amalgamation.
24228 ** The number of levels of backtrace to save in new allocations.
24234 ** Title text to insert in front of each block
24470 ** allocation into a new place in memory. In this way, if the
24493 ** Populate the low-level memory allocation function pointers in
24494 ** sqlite3GlobalConfig.m with pointers to the routines in this file.
24523 ** Return TRUE if the mask of type in eType matches the type of the
24545 ** Return TRUE if the mask of type in eType matches no bits of the type of the
24687 ** in the build only if SQLITE_ENABLE_MEMSYS3 is defined.
24701 ** Maximum size (in Mem3Blocks) of a "small" chunk.
24718 ** size of the allocation in blocks if the allocation is free.
24723 ** the previous chunk in blocks if the previous chunk is on the
24728 ** We often identify a chunk by its index in mem3.aPool[]. When
24736 ** Pointers to the head of the list are stored in mem3.aiSmall[]
24747 u32 prevSize; /* Size of previous chunk in Mem3Block elements */
24748 u32 size4x; /* 4x the size of current chunk in Mem3Block elements */
24751 u32 next; /* Index in mem3.aPool[] of next free chunk */
24752 u32 prev; /* Index in mem3.aPool[] of previous free chunk */
24761 ** when this module is combined with other in the amalgamation.
24766 ** (in Mem3Blocks) pointed to by aPool less 2.
24788 ** occur off of this chunk. szKeyBlk is the size (in Mem3Blocks)
24790 ** The key chunk is not in either the aiHash[] or aiSmall[].
24882 ** will already be held (obtained by code in malloc.c) if
24964 ** or same size hash. In other words, *pRoot is an entry in either
25011 ** Return a block of memory of at least nBytes in size.
25032 ** Look for an entry of the correct size in either the small
25033 ** chunk table or in the large chunk hash table. This is
25136 ** Return the size of an outstanding allocation, in bytes. The
25221 /* Store a pointer to the memory block in global structure mem3. */
25326 ** This routine is the only routine in this file with external
25329 ** Populate the low-level memory allocation function pointers in
25330 ** sqlite3GlobalConfig.m with pointers to the routines in this file. The
25377 ** in the build only if SQLITE_ENABLE_MEMSYS5 is defined.
25388 ** This algorithm is described in: J. M. Robson. "Bounds for Some Functions
25418 ** verified in memsys5Init().
25443 ** when this module is combined with other in the amalgamation.
25449 int szAtom; /* Smallest possible allocation in bytes */
25450 int nBlock; /* Number of szAtom sized blocks in zPool */
25551 ** Return the size of an outstanding allocation, in bytes.
25564 ** Return a block of memory of at least nBytes in size.
25571 ** threads can be in this routine at the same time.
25599 ** two in order to create a new free block of size iLogsize.
25647 /* Set iBlock to the index of the block pointed to by pOld in
25807 int nMinLog; /* Log base 2 of minimum allocation size in bytes */
25824 /* boundaries on sqlite3GlobalConfig.mnReq are enforced in sqlite3_config() */
25911 ** This routine is the only routine in this file with external
26310 ** This implementation in this file does not provide any mutual
26311 ** exclusion and is thus suitable for use only in applications
26312 ** that use SQLite in a single thread. The routines defined
26531 ** The code in this file is only used if we are compiling threadsafe
26602 ** and also for the implementation of xShmBarrier in the VFS in cases
26648 ** not want to. But SQLite will only request a recursive mutex in
26651 ** might return such a mutex in response to SQLITE_MUTEX_FAST.
26791 /* Use the built-in recursive mutexes if they are available.
26837 /* Use the built-in recursive mutexes if they are available.
26931 /************** Include os_common.h in the middle of mutex_w32.c *************/
26956 ** At least two bugs have slipped in because we changed the MEMORY_DEBUG
26974 /************** Include hwtime.h in the middle of os_common.h ****************/
27053 ** and analysis configurations, not in any deliverable, so this
27063 /************** Continuing where we left off in os_common.h ******************/
27129 /************** Continuing where we left off in mutex_w32.c ******************/
27134 /************** Include os_win.h in the middle of mutex_w32.c ****************/
27226 /************** Continuing where we left off in mutex_w32.c ******************/
27230 ** The code in this file is only used if we are compiling multithreaded
27283 ** and also for the xShmBarrier method of the VFS in cases when SQLite is
27341 /* Another thread is (in the process of) initializing the static
27394 ** not want to. But SQLite will only request a recursive mutex in
27397 ** might return such a mutex in response to SQLITE_MUTEX_FAST.
27624 ** cache database pages that are not currently in use.
27780 ** Return true if the heap is currently under memory pressure - in other
27839 ** implementation of malloc_good_size(), which must be called in debug
27999 ** Add the size of memory allocation "p" to the count in
28069 /* The 0x7ffff00 limit term is explained in comments on sqlite3Malloc() */
28143 ** the mallocFailed flag in the connection pointer.
28169 ** If the allocation fails, set the mallocFailed flag in
28176 ** This is an important assumption. There are many places in the
28250 ** resize fails, set the mallocFailed flag in the connection object.
28296 ** and set the mallocFailed flag in the database connection.
28308 ** Make a copy of a string in memory obtained from sqliteMalloc(). These
28311 ** called via macros that record the current file and line number in the
28342 ** SQL statement. Make a copy of this phrase in space obtained form
28354 ** Free any prior content in *pz and replace it with a copy of zNew.
28436 ** The "printf" code that follows dates from the 1980's. It is in
28461 /* The rest are extensions, not normally found in printf() */
28463 #define etSQLESCAPE2 10 /* Strings with '\'' doubled and enclosed in '',
28481 ** Each builtin conversion character (ex: the 'd' in "%d") is described
28557 ** 16 (the number of significant digits in a 64-bit float) '0' is
28603 ** of the output buffer in pAccum, then cause an SQLITE_TOOBIG error.
28646 int c; /* Next character in the format string */
29044 /* The sign in front of the number */
29099 /* The converted number is in buf[] and zero terminated. Output it.
29100 ** Note that the number is in the usual order, not reversed as with
29211 /* Set length to the number of bytes needed in order to display
29226 /* Adjust width to account for extra bytes in UTF-8 characters */
29232 case etSQLESCAPE2: /* %Q: Escape ' and enclose in '...' */
29316 ** the output. Both length and width are in bytes, not characters,
29318 ** indicating that width and precision should be expressed in characters,
29500 /* Return the current length of p in bytes */
29532 ** zBase: An initial buffer. May be NULL in which case the initial buffer
29534 ** n: Size of zBase in bytes. If total space requirements never exceed
29637 ** are not able to use a "," as the decimal point in place of "." as
29673 ** We house it in a separate routine from sqlite3_log() to avoid using
29764 ** The interfaces in this file is only available when compiling
29772 ** is not the last item in the tree.
30312 ** up in the treeview output as "SOFT-COLLATE". Explicit COLLATE
30313 ** operators that appear in the original SQL always have the
30314 ** EP_Collate bit set and appear in treeview output as just "COLLATE" */
30396 ** X is stored in pExpr->pLeft.
30397 ** Y is stored in pExpr->pList->a[0].pExpr.
30398 ** Z is stored in pExpr->pList->a[1].pExpr.
30412 ** to a column in the new.* or old.* pseudo-tables available to
30548 ** Random numbers are used by some of the database backends in order
30607 ** Nothing in this file or anywhere else in SQLite does any kind of
30692 ** or sqlite3ThreadJoin() call. This is, in fact, what happens in
30693 ** single threaded systems. Nothing in SQLite requires multiple threads.
30723 void *(*xTask)(void*), /* Routine to run in a separate thread */
30731 /* This routine is never used in single-threaded mode */
30818 void *(*xTask)(void*), /* Routine to run in a separate thread */
30884 ** work of the thread in the main thread, when either the thread is created
30898 void *(*xTask)(void*), /* Routine to run in a separate thread */
31073 ** * Bytes in the range of 0x80 through 0xbf which occur as the first
31130 sqlite3_int64 len; /* Maximum length of output string in bytes */
31177 /* Set len to the maximum number of bytes required in the output buffer. */
31189 ** character. Two bytes are required in the output buffer for the
31317 ** UTF-16 string stored in *pMem. If one is present, it is removed and
31357 ** return the number of unicode characters in pZ up to (but not including)
31359 ** number of unicode characters in the first nByte of pZ (or up to
31380 ** Hence it is only available in debug builds.
31389 ** The translation is done in-place and aborted if the output
31410 ** Convert a UTF-16 string in the native encoding into a UTF-8 string.
31434 ** Return the number of bytes in the first nChar unicode characters
31435 ** in pZ. nChar must be non-negative.
31457 ** characters in each encoding are inverses of each other.
31521 ** or to bypass normal error detection during testing in order to let
31557 ** Compute a string length that is limited to what can be stored in
31614 ** to do based on the SQLite error code in rc.
31629 ** error string in the style of the printf functions: The following
31639 ** encoded in UTF-8.
31711 ** the quote characters. The conversion is done in-place. If the
31771 ** the contents of two buffers containing UTF-8 strings in a
31829 ** Compute 10 to the E-th power. Examples: E==1 results in 10.
31830 ** E==2 results in 100. E==50 results in 1.0e50.
31871 ** The string z[] is length bytes in length (bytes, not characters) and
31883 ** Valid numbers are in one of these formats:
31915 *pResult = 0.0; /* Default return value, in case of an error */
32094 ** Render an signed 64-bit integer as text. Store the result in zOut[].
32096 ** The caller must ensure that zOut[] is at least 21 bytes in size.
32124 ** in the values of the last digit if the only difference is in the
32155 ** 0 Successful transformation. Fits in a 64-bit signed integer.
32160 ** length is the number of bytes in the string (bytes, not characters).
32230 /* Less than 19 digits, so we know that it fits in 64 bits */
32256 ** Transform a UTF-8 integer literal, in either decimal or hexadecimal,
32262 ** 0 Successful transformation. Fits in a 64-bit signed integer.
32288 ** If zNum represents an integer that will fit in 32-bits, then set
32449 ** inline in order to work around bugs in the RVT compiler.
32461 ** Return the number of bytes read. The value is stored in *v.
32548 /* we can skip this cause it was (effectively) done above in calc'ing s */
32615 ** Return the number of bytes read. The value is stored in *v.
32617 ** If the varint stored in p[0] is larger than can fit in a 32-bit unsigned
32668 /* A 32-bit varint is used to store size information in btrees.
32726 ** file. In that case we are not in any hurry. Use the (relatively
32847 ** misuse of the interface such as passing in db pointers that are
32891 ** the other 64-bit signed integer at *pA and store the result in *pA.
32892 ** Return 0 on success. Or if the operation would have resulted in an
32965 ** filename in zBaseFilename is a URI with the "8_3_names=1" parameter and
32966 ** if filename in z[] has a suffix (a.k.a. "extension") that is longer than
33082 ** possible to this routine is 310, resulting in a maximum x of 31 */
33129 int nName, /* Bytes of text in zName */
33134 int i; /* Index in pIn[] where zName is stored */
33159 ** Return a pointer to the name of a variable in the given VList that
33160 ** has the value iVal. Or return a NULL if there is no such variable in
33176 ** Return the number of the variable named zName, if it is in VList.
33206 ** used in SQLite.
33270 HashElem *pHead; /* First element already in pEntry */
33335 /* This function (for internal use only) locates an element in an
33424 ** The key is not copied in this instance. If a malloc fails, then
33523 /* 45 */ "RowSetTest" OpHelp("if r[P3] in rowset(P1) goto P2"),
33593 /* 115 */ "OpenPseudo" OpHelp("P3 columns in r[P2]"),
33679 ** There are actually several different VFS implementations in this file.
33680 ** The differences are in the way that file locking is done. The default
33688 ** in the correct division and should be clearly labeled.
33850 ** kernel modules in VxWorks. */
33918 ** transaction counter in bytes 24-27 of database files are updated
33926 unsigned char inNormalWrite; /* True if in a normal write operation */
33932 ** it is larger than the struct CrashFile defined in test6.c.
33963 /************** Include os_common.h in the middle of os_unix.c ***************/
33988 ** At least two bugs have slipped in because we changed the MEMORY_DEBUG
34006 /************** Include hwtime.h in the middle of os_common.h ****************/
34085 ** and analysis configurations, not in any deliverable, so this
34095 /************** Continuing where we left off in os_common.h ******************/
34161 /************** Continuing where we left off in os_unix.c ********************/
34224 ** Different Unix systems declare open() in different ways. Same use
34415 ** On some systems, calls to fchown() will trigger a message in a security
34511 ** Do not accept any file descriptor less than this value, in order to avoid
34737 * introspection, in which it actually means what it says */
34757 ** unique file ID in VxWorks. Each instance of this structure contains
34767 struct vxworksFileId *pNext; /* Next in a list of them all */
34788 ** Changes are made in-place. Return the new name length.
34790 ** The original filename is in z[0..n-1]. Return the number of
34791 ** characters in the simplified name.
34911 ** to synchronize access for threads in separate processes, but not
34917 ** st_dev and st_ino fields of the stat structure that fstat() fills in)
34925 ** canonical filename and implemented in the previous division.)
34964 ** in thread B. But there is no way to know at compile-time which
34973 ** per database file in the same process and (2) database connections
35007 ** (1) Only the pLockMutex mutex must be held in order to read or write
35046 ** Must hold unixBigLock in order to read or write this variable.
35071 ** It is invoked after an error occurs in an OS function and errno has been
35138 ** We assume that close() almost always works, since it is only in a
35156 ** Set the pFile->lastErrno. Do this in a subroutine as that provides
35164 ** Close all file descriptors accumuated in the unixInodeInfo->pUnused list.
35223 unixFile *pFile, /* Unix file with file desc used in the key */
35250 ** around this problem (we consider it a bug in OS X, not SQLite)
35254 ** in the header of every SQLite database. In this way, if there
35379 /* Check if a thread in this process holds such a lock */
35458 ** in order to coordinate access between separate database connections
35459 ** within this process, but all of that is handled in memory and the
35505 ** are inserted in between. The locking might fail on one of the later
35521 ** lock transitions in terms of the POSIX advisory shared and exclusive
35524 ** slightly in order to be compatible with Windows95 systems simultaneously
35525 ** accessing the same database file, in case that is ever required.
35527 ** Symbols defined in os.h indentify the 'pending byte' and the 'reserved
35680 /* We are trying for an exclusive lock but another thread in this
35766 ** around a bug in BSD NFS lockd (also seen on MacOSX 10.3+) that fails to
35804 ** the lock in 2 blocks, so that part of the range will be covered by a
35865 ** SQLITE_BUSY would confuse the upper layer (in practice it causes
35887 ** OS call only when all threads in this same process have released
36033 ** corruption if this locking mode is used in situations where multiple
36078 ** Dotfile locking works by creating a subdirectory in the same directory as
36085 ** The file suffix added to the data base filename in order to create the
36096 ** In dotfile locking, either a lock exists or it does not. So in this
36124 ** are inserted in between. The locking might fail on one of the later
36249 ** flock() locking is like dot-file locking in that the various
36289 /* Check if a thread in this process holds such a lock */
36341 ** are inserted in between. The locking might fail on one of the later
36353 ** lock states in the sqlite3_file structure, but all locks SHARED or
36454 ** Semaphore locking is like dot-lock and flock in that it really only
36476 /* Check if a thread in this process holds such a lock */
36491 /* someone else has the lock when we are in NO_LOCK */
36515 ** are inserted in between. The locking might fail on one of the later
36527 ** lock states in the sqlite3_file structure, but all locks SHARED or
36680 OSTRACE(("AFPSETLOCK [%s] for %d%s in range %llx:%llx\n",
36725 /* Check if a thread in this process holds such a lock */
36736 /* if we succeeded in taking the reserved lock, unlock it to restore
36765 ** are inserted in between. The locking might fail on one of the later
36892 /* We are trying for an exclusive lock but another thread in this
37030 ** OS call only when all threads in this same process have released
37130 ** methods were defined in divisions above (one locking method per
37142 ** in any form by default, we will not attempt to define _XOPEN_SOURCE.
37210 ** file), the bytes in the locking range should never be read or written. */
37277 int nBuf, /* Size of buffer pBuf in bytes */
37313 ** Seek to the offset in id->offset then read cnt bytes into pBuf.
37340 ** file), the bytes in the locking range should never be read or written. */
37549 ** The ability to override this routine was added in support of the
37694 ** Determine the current size of a file in bytes
37709 ** writes a single byte into that file in order to work around a bug
37710 ** in the OS-X msdos filesystem. In order to avoid problems with upper
37722 ** Handler for proxy-locking file-control verbs. Defined below in the
37730 ** file-control operation. Enlarge the database to nBytes in size
37757 ** single byte to the last byte in each block that falls entirely
38043 ** Return the sector size in bytes of the underlying block device for
38048 ** if two files are created in the same file-system directory (i.e.
38066 ** very rare. And asserting PSOW makes a large reduction in the amount
38082 ** This function should not be called directly by other code in this file.
38109 ** every open file that does not use shared memory (in other words, most
38126 ** in this structure.
38264 ** to use 64KB pages - in this case each mapping must cover at least two
38326 ** DMS byte and truncate the *-shm file to zero bytes in size. Then
38373 ** The file used to implement shared-memory is in the same directory
38379 ** Another approach to is to use files in /dev/shm or /dev/tmp or an
38380 ** some other tmpfs mount. But if a file in a different directory
38385 ** in database corruption. Nevertheless, this tmpfs file usage
38388 ** option results in an incompatible build of SQLite; builds of SQLite
38392 ** "unsupported" and may go away in a future SQLite release.
38395 ** file are currently open, in this process or in other processes, then
38410 int nShmFilename; /* Size of the SHM filename in bytes */
38531 ** bytes in size.
38536 ** region has not been allocated (by any client, including one running in a
38610 ** last page in order to extend the file. But writing to all new
38684 ** is to be used in an assert(). e.g.
38717 ** different here than in posix. In xShmLock(), one can go from unlocked
38747 ** in the following list than the lock being obtained are already held:
38754 ** occur later in the above list than the lock being obtained may be
38828 ** also update the in-memory values. */
38961 i64 nOrig = pFd->mmapSizeActual; /* Size of pOrig region in bytes */
39023 ** in this case. */
39075 ** If such a pointer can be obtained, store it in *pp and return SQLITE_OK.
39078 ** value of *pp is undefined in this case.
39180 ** to the METHOD object in the previous bullet.
39287 ** The proxy locking method is a "super-method" in the sense that it
39291 ** proxy locking is located much further down in the file. But we need
39457 /* No locking occurs in temporary files */
39489 /* Cache zFilename in the locking context (AFP and dotlock override) for
39504 /* If an error occurred in findInodeInfo(), close the file descriptor
39506 ** in two scenarios:
39519 ** implicit assumption here is that if fstat() fails, things are in
39530 /* AFP locking uses the file path so it needs to be included in
39557 /* Dotfile locking uses the file path so it needs to be included in
39576 ** included in the semLockingContext
39617 ** Return the name of a directory in which to put temporary files.
39650 ** Create a temporary file name in zBuf. zBuf must be allocated
39682 ** Implementation in the proxy-lock division, but used by unixOpen()
39697 ** Refer to comments in the unixClose() function and the lengthy comment
39719 ** For this reason, if an error occurs in the stat() call here, it is
39806 int nDb; /* Number of valid bytes in zDb */
39823 ** a '-' character. However in 8+3 filename mode, or if a corrupt
39851 ** Previously, the SQLite OS layer used three functions in place of this
39873 sqlite3_file *pFile, /* The file descriptor to be filled in */
39908 ** a temporary file. Use this buffer to store the file name in.
40036 ** in that case we do not want to attempt the chown().
40114 /* Use unixClose to clean up the resources added in fillInUnixFile
40217 ** If the last component of the pathname in z[0]..z[j-1] is something
40301 ** is stored as a nul-terminated string in the buffer pointed to by
40305 ** (in this case, MAX_PATHNAME bytes). The full-path is written to
40311 int nOut, /* Size of output buffer in bytes */
40466 ** uninitialized space in zBuf - but valgrind errors tend to worry
40469 ** in the random seed.
40534 SQLITE_API int sqlite3_current_time = 0; /* Fake system time in seconds since 1970. */
40538 ** Find the current time (in Universal Coordinated Time). Write into *piNow
40541 ** epoch of noon in Greenwich on November 24, 4714 B.C according to the
40575 ** Find the current time (in Universal Coordinated Time). Write the
40612 ** Proxy locking is a "uber-locking-method" in this sense: It uses the
40616 ** until late in the file (here) after all of the other I/O methods have
40622 ** The default locking schemes in SQLite use byte-range locks on the
40630 ** address in the shared range is taken for a SHARED lock, the entire
40638 ** slowdown in read performance on AFP because the AFP client disables
40671 ** host ID in it, the proxy path in the conch file will be used, otherwise
40700 ** proxy path against the values stored in the conch. The conch file is
40701 ** stored in the same directory as the database file and the file name
40766 ** and local proxy files in it
40825 ** Creates the lock file and any missing directories in lockPath
40861 ** Create a new VFS file descriptor (stored in memory obtained from
40862 ** sqlite3_malloc) and open the file named "path" in the file descriptor.
40957 /* Not always defined in the headers as it ought to be */
41138 ** lockPath means that the lockPath in the conch file will be used if the
41188 /* if the host id matches and the lock path already exists in the conch
41192 do { /* in case we need to try again for an :auto: named lock file */
41200 ** use the local lock file path that's already in there
41244 /* We are trying for an exclusive lock but another thread in this
41356 } while (1); /* in case we need to retry the :auto: lock file -
41385 ** Store the conch filename in memory obtained from sqlite3_malloc64().
41397 char *conchPath; /* buffer in which to construct conch name */
41462 ** is a string buffer at least MAXPATHLEN+1 characters in size.
41470 /* afp style keeps a reference to the db path in the filePath field
41491 ** Takes an already filled in unix file and alters it so all file locking
41493 ** are preserved in the locking context so that they can be restored and
41683 ** are inserted in between. The locking might fail on one of the later
41794 ** should be the only routines in this file that are visible from other
41849 ** All default VFSes for unix are contained in the following array.
41887 /* Register all VFSes defined in the aVfs[] array */
41898 ** Some operating systems might need to do some cleanup in this routine,
41931 /************** Include os_common.h in the middle of os_win.c ****************/
41956 ** At least two bugs have slipped in because we changed the MEMORY_DEBUG
41974 /************** Include hwtime.h in the middle of os_common.h ****************/
42053 ** and analysis configurations, not in any deliverable, so this
42063 /************** Continuing where we left off in os_common.h ******************/
42129 /************** Continuing where we left off in os_win.c *********************/
42138 ** available in Windows platforms based on the NT kernel.
42193 ** target system. GetVersionEx was first deprecated in Win8.1.
42206 ** When compiling for Windows 10, always assume "mincore.lib" is in use.
42217 ** This constant should already be defined (in the "WinDef.h" SDK file).
42224 ** Maximum pathname length (in chars) for Win32. This should normally be
42232 ** This constant should already be defined (in the "WinNT.h" SDK file).
42239 ** Maximum pathname length (in chars) for WinNT. This should normally be
42247 ** Maximum pathname length (in bytes) for Win32. The MAX_PATH macro is in
42256 ** Maximum pathname length (in bytes) for WinNT. This should normally be
42265 ** Maximum error message length (in chars) for WinRT.
42296 ** mode or memory mapped files (e.g. these APIs are available in the Windows
42297 ** CE SDK; however, they are not present in the header file)?
42442 * data will almost certainly result in an immediate access violation.
42450 * This is the maximum possible initial size of the Win32-specific heap, in
42459 * in bytes. This value may be zero.
42466 * Calculate the maximum legal cache size, in pages, based on the maximum
42477 * This is cache size used in the calculation of the initial size of the
42489 * Make sure that the calculated cache size, in pages, cannot cause the
42491 * of memory that can be specified in the call to HeapCreate.
42515 * The extra flags to use in calls to the Win32 heap APIs. This value may be
43357 ** largest committed free block in the heap, in bytes.
43429 ** The Win32 native heap cannot be modified because it may be in use.
43647 ** Return the size of an outstanding allocation, in bytes.
43753 ** Populate the low-level memory allocation function pointers in
43754 ** sqlite3GlobalConfig.m with pointers to the routines in this file. The
44078 ** the provided arguments. The type argument must be 1 in order to set the
44079 ** data directory or 2 in order to set the temporary directory. The zValue
44092 ** is zero if the error message fits in the buffer, or non-zero
44175 ** This routine is invoked after an error occurs in an OS function.
44229 ** non-zero if the error code is transient in nature and the operation
44236 ** may be used to include additional error codes in the set that should
44237 ** result in the failing I/O operation being retried by the caller. If
44296 ** corresponding section in "date.c" cannot use it.
44349 ** Create the mutex and shared memory used for locking in the file
44405 /* If we succeeded in making the shared memory handle, map it. */
44451 /* The following blocks should probably assert in debug mode, but they
44452 are to cleanup in case any locks remained open */
45021 ** This can result in incremental vacuum not truncating the file,
45029 ** but involves adding a few branches in the common write code path which
45032 ** can maybe revisit this decision in the future.
45182 ** Determine the current size of a file in bytes
45328 ** are inserted in between. The locking might fail on one of the later
45465 /* Update the state of the lock has held in the file descriptor then
45568 ** corruption if this locking mode is used in situations where multiple
45745 ** Return the sector size in bytes of the underlying block device for
45750 ** if two files are created in the same file-system directory (i.e.
45824 ** in this structure.
45845 winShmNode *pNext; /* Next in list of all winShmNode objects */
46019 ** file are currently open, in this process or in other processes, then
46027 int nName; /* Size of zName in bytes */
46294 ** bytes in size.
46299 ** region has not been allocated (by any client, including one running in a
46602 ** If such a pointer can be obtained, store it in *pp and return SQLITE_OK.
46605 ** value of *pp is undefined in this case.
46785 ** operating system wants filenames in. Space to hold the result
46826 ** The pointer returned in pzBuf must be freed via sqlite3_free().
47023 ** name in the following format:
47027 ** If not, return SQLITE_ERROR. The number 17 is used here in order to
47112 void *zConverted; /* Filename in OS encoding */
47113 const char *zUtf8Name = zName; /* Filename in UTF-8 encoding */
47117 ** a temporary file. Use this buffer to store the file name in.
47410 ** will open a journal file shortly after it is created in order to do
47631 ** a legal UNC name, a volume relative path, or an absolute path name in the
47660 ** bytes in size.
47665 int nFull, /* Size of output buffer in bytes */
47916 int na; /* Size of a[] in bytes */
48006 SQLITE_API int sqlite3_current_time = 0; /* Fake system time in seconds since 1970. */
48010 ** Find the current time (in Universal Coordinated Time). Write into *piNow
48013 ** epoch of noon in Greenwich on November 24, 4714 B.C according to the
48028 /* 2^32 - to avoid use of LL and warnings in gcc */
48058 ** Find the current time (in Universal Coordinated Time). Write the
48084 ** is zero if the error message fits in the buffer, or non-zero
48087 ** in the output buffer.
48278 ** This file implements an in-memory VFS. A database is held as a contiguous
48375 memdbLock, /* xUnlock - same as xLock in this case */
48395 ** in which case we own the pData pointer and need to free it.
48551 ** Return the sector-size in bytes for an memdb-file.
48616 ** files, none of which exist in memdb. So this routine is never used */
48645 ** to the pathname in zPath. zOut is guaranteed to point to a buffer
48675 ** Return a pointer to the symbol zSymbol in the dynamic library pHandle.
48706 ** Return the current time as a Julian Day number in *pTimeOut.
48818 sqlite3_int64 szDb, /* Number bytes in the deserialization */
48894 ** it in, to be safe, but it is marked as NO_TEST since there
48923 ** or all of the pages in a database can get journalled. In those cases,
48936 ** Bitvec object is the number of pages in the database file at the
48942 /* Size of the Bitvec structure in bytes. */
48955 /* Size, in bits, of the bitmap element. */
48957 /* Number of elements in a bitmap array. */
48959 /* Number of bits in the bitmap array. */
48962 /* Number of u32 values in hash table. */
48964 /* Maximum number of entries in hash table before
48969 ** (an arbitrary prime)in the hash function provided
49103 /* in hash, if not, try to find a spot for it */
49109 /* we didn't find it in the hash. h points to the first */
49226 ** 5 N S X Set N bits from S increment X in array only, not in bitvec
49231 ** Opcode 5 is used to deliberately induce a fault in order to
49335 ** entry in the cache holds a single page of the database file. The
49344 ** PgHdr.pDirtyNext and pDirtyPrev. The list is maintained in LRU order
49346 ** PCache.pDirty points to the first (newest) element in the list and
49353 ** to either the oldest page in the pDirty/pDirtyTail list that has a
49359 PgHdr *pDirty, *pDirtyTail; /* List of dirty pages in LRU order */
49360 PgHdr *pSynced; /* Last synced page in dirty page list */
49364 int szPage; /* Size of every page in this cache */
49380 ** is displayed for many operations, resulting in a lot of output.
49445 ** If NEED_SYNC had been cleared in step 2, then it would not be reset
49446 ** in step 3, and page might be written into the database without first
49452 ** in that sector are marked NEED_SYNC even if they are still CLEAN, just
49453 ** in case they are later modified, since all pages in the same sector
49499 /* If there are now no dirty pages in the cache, set eCreate to 2.
49543 ** being used for an in-memory database, this function is a no-op.
49579 ** built-in default page cache is used instead of the application defined
49594 ** Return the size in bytes of a PCache object.
49600 ** has already been allocated and is passed in as the p pointer.
49608 ** structure in the pager.
49660 ** such an object is already in cache, or if a new one is created.
49661 ** This routine returns a NULL pointer if the object was not in cache
49668 ** is not already in the cache. If createFlag is 1, then a new page
49735 ** If the LRU page in the dirty list that has a clear PGHDR_NEED_SYNC
49774 ** requires extra stack manipulation that can be avoided in the common
49800 ** must be called after sqlite3PcacheFetch() in order to get a usable
49901 ** Make every page in the cache clean.
49979 if( ALWAYS(pPage1) ){ /* Page 1 is always available in cache, because
50006 ** Merge two lists of pages connected by pDirty and in pgno order.
50036 ** Sort the list of pages in accending order by pgno. Pages are
50040 ** Since there cannot be more than 2^31 distinct pages in a database,
50042 ** One extra bucket is added to catch overflow in case something
50064 /* To get here, there need to be 2^(N_SORT_BUCKET) elements in
50079 ** Return a list of all dirty pages in the cache, sorted by page number.
50107 ** Return the total number of pages in the cache.
50162 ** in the page-cache hierarchy.
50167 ** Return the number of dirty pages currently in the cache, as a percentage
50180 ** Return true if there are one or more dirty pages in the cache. Else false.
50189 ** For all dirty pages currently in the cache, invoke the specified
50247 ** fragmentation" in academic literature) of allocating a few bytes more
50296 ** in memory.
50299 ** but causes a 2-byte gap in the structure for most architectures (since
50301 ** in memory directly after the associated page data, if the database is
50306 ** in the structure.
50313 PgHdr1 *pNext; /* Next in hash table chain */
50315 PgHdr1 *pLruNext; /* Next in LRU list of unpinned pages */
50316 PgHdr1 *pLruPrev; /* Previous in LRU list of unpinned pages */
50322 ** that the page is in active use and must not be deallocated.
50332 ** This page cache implementation works in one of two modes:
50342 ** and is therefore often faster. Mode 2 requires a mutex in order to be
50359 ** open database file (including each in-memory database and each
50388 unsigned int nRecyclable; /* Number of pages in the LRU list */
50389 unsigned int nPage; /* Total number of pages in apHash */
50390 unsigned int nHash; /* Number of slots in apHash[] */
50397 ** Free slots in the allocator used to divide up the global page cache
50434 ** All code in this file should access the global structure above via the
50529 ** such buffer exists or there is no space left in it, this function falls
50533 ** in pcache1 need to be protected via mutex.
50552 /* Memory is not available in the SQLITE_CONFIG_PAGECACHE pool. Get
50723 ** allocating a new page cache entry in order to avoid stressing
50804 ** (PCache1.apHash structure) that it is currently stored in.
50863 ** cache, then there is no point in scanning the entire hash table.
50912 ** PGroup in pcache1.grp is used for all page caches (mode-2).
50916 ** * Use a unified cache in single-threaded applications that have
50953 ** Note that the static mutex allocated in xInit does
51068 ** in the header of the pcache1Fetch() procedure.
51153 ** For a non-purgeable cache (a cache used as the storage for an in-memory
51164 ** 2. If createFlag==0 and the page is not already in the cache, NULL is
51167 ** 3. If createFlag is 1, and the page is not already in the cache, then
51214 /* Step 2: If the page was found in the hash table, then return it.
51215 ** If the page was not in the hash table and createFlag is 0, abort.
51216 ** Otherwise (page not in hash and createFlag!=0) continue with
51349 ** Discard all unpinned pages in the cache with a page number equal to
51426 ** held by the pager system. Memory in use by any SQLite pager allocated
51499 ** are inserted into the RowSet in an arbitrary order. Inserts
51504 ** elements of the RowSet in sorted order. Once this extraction
51517 ** TEST checks to see if an element is already in the RowSet. SMALLEST
51521 ** allocated in chunks so most INSERTs do no allocation. There is an
51527 ** in the batch number. In other words, if an INSERT occurs between
51538 ** batch number is O(NlogN) where N is the number of elements in the RowSet.
51544 ** be possible, but the feature was not used, so it was removed in order
51562 ** Each entry in a RowSet is an instance of the following object.
51566 ** in the list, pLeft points to the tree, and v is unused. The
51576 ** RowSetEntry objects are allocated in large chunks (instances of the
51587 ** A RowSet in an instance of the following structure.
51589 ** A typedef of this structure if found in sqliteInt.h.
51722 ** assumed to each already be in sorted order.
51857 ** as deep as it needs to be in order to contain the entire list.
51969 /* Test to see if the iRowid value appears anywhere in the forest.
52011 /************** Include wal.h in the middle of pager.c ***********************/
52026 ** the implementation of each function in log.c for further details.
52078 ** at an instant in time. sqlite3WalOpenSnapshot gets a read lock and
52101 ** position in the WAL */
52104 /* Move the write position of the WAL back to iFrame. Called in
52121 int *pnLog, /* OUT: Number of frames in WAL */
52122 int *pnCkpt /* OUT: Number of backfilled frames in WAL */
52126 ** number of frames in the WAL at the point of the last commit since
52153 ** stored in each frame (i.e. the db page-size when the WAL was created).
52170 /************** Continuing where we left off in pager.c **********************/
52193 ** (c) The page number is greater than the largest page that existed in
52206 ** both the content in the database when the rollback journal was written
52207 ** and the content in the database at the beginning of the current
52211 ** in length and are aligned on a page boundary.
52214 ** an integer multiple of the page size in length or are taken from the
52225 ** all queries. Note in particular the content of freelist leaf
52240 ** (9) Whenever the database file is modified, at least one bit in the range
52245 ** (10) The pattern of bits in bytes 24 through 39 shall not repeat in less
52283 ** pager may be in any one of the seven states shown in the following
52321 ** The pager starts up in this state. Nothing is guaranteed in this
52331 ** In this state all the requirements for reading the database in
52333 ** was) in exclusive-locking mode, a user-level read transaction is
52334 ** open. The database size is known in this state.
52339 ** running in locking_mode=exclusive (including temp databases) remains in
52351 ** there is no hot-journal in the file-system.
52363 ** to in this state. If the transaction is committed or rolled back while
52364 ** in WRITER_LOCKED state, all that is required is to unlock the database
52372 ** * If the connection is open in rollback-mode, a RESERVED or greater
52374 ** * If the connection is open in WAL-mode, a WAL write transaction
52416 ** by finalizing the journal file. Once in WRITER_FINISHED state, it is
52430 ** SQLITE_IOERR_NOMEM) occurs at a point in the code that makes it
52431 ** difficult to be sure that the in-memory pager state (cache contents,
52434 ** Temporary pager files may enter the ERROR state, but in-memory pagers
52438 ** the contents of the page-cache may be left in an inconsistent state.
52450 ** page-cache and any other in-memory state at the same time. Everything
52458 ** 1. An error occurs while attempting a rollback. This happens in
52462 ** following a commit in function sqlite3PagerCommitPhaseTwo().
52465 ** database file in function pagerStress() in order to free up
52475 ** automatically attempt a rollback, as it assumes that an error in a
52476 ** read-only statement cannot leave the pager in an internally inconsistent
52482 ** * The pager is not an in-memory pager.
52487 ** * A pager is never in WRITER_DBMOD or WRITER_FINISHED state if the
52488 ** connection is open in WAL mode. A WAL connection is always in one
52491 ** * Normally, a connection open in exclusive mode is never in PAGER_OPEN
52528 ** in the file-system that needs to be rolled back (as part of an OPEN->SHARED
52537 ** doesn't know it because of a previous error in xUnlock). If this happens
52539 ** transaction in another process, causing SQLite to read from the database
52543 ** database in the ERROR state, Pager.eLock is set to UNKNOWN_LOCK. It
52551 ** Pager.eLock may only be set to UNKNOWN_LOCK when the pager is in
52567 ** savepoint and statement transaction in the system. All such structures
52568 ** are stored in the Pager.aSavepoint[] array, which is allocated and
52580 i64 iOffset; /* Starting offset in main journal */
52582 Bitvec *pInSavepoint; /* Set of pages in this savepoint */
52583 Pgno nOrig; /* Original number of pages in file */
52584 Pgno iSubRec; /* Index of first record in sub-journal */
52612 ** For a temporary or in-memory database (neither of which require any
52635 ** This mechanism means that when running in exclusive mode, a connection
52646 ** the way in which the journal file is finalized after the transaction is
52647 ** committed or rolled back when running in "journal_mode=PERSIST" mode.
52652 ** running in "journal_mode=truncate" mode.
52668 ** to the file-system in order to free up memory).
52672 ** The SPILLFLAG_ROLLBACK case is done in a very obscure case that
52675 ** while it is being traversed by code in pager_playback(). The SPILLFLAG_OFF
52681 ** is larger than the database page-size in order to prevent a journal sync
52682 ** from happening in between the journalling of two pages on the same sector.
52687 ** is opened as an in-memory journal file. If false, then in-memory
52688 ** sub-journals are only used for in-memory pager files.
52695 ** Variable dbSize is set to the number of pages in the database file.
52696 ** It is valid in PAGER_READER and higher states (all states except for
52702 ** is not an integer multiple of the page-size, the value stored in
52704 ** Except, any file that is greater than 0 bytes in size is considered
52709 ** dbSize are modified in the cache, dbSize is updated accordingly.
52713 ** Variables dbOrigSize and dbFileSize are valid in states
52720 ** the file on disk in pages. It is set to a copy of dbSize when the
52749 ** The Pager.errCode variable is only ever used in PAGER_ERROR state. It
52750 ** is set to zero in all other states. In PAGER_ERROR state, Pager.errCode
52758 ** and contains the flags used to sync the checkpoint operations in the
52759 ** lower two bits, and sync flags used for transaction commits in the WAL
52760 ** file in bits 0x04 and 0x08. In other words, to get the correct sync flags
52763 ** that with synchronous=NORMAL in WAL mode, transaction commit is not synced
52777 u8 noLock; /* Do not lock (except in WAL mode) */
52783 ** routine operation. Class members not in this block are either fixed
52795 u8 subjInMemory; /* True to use in-memory sub-journals */
52798 Pgno dbSize; /* Number of pages in the database */
52800 Pgno dbFileSize; /* Number of pages in the database file */
52806 Bitvec *pInJournal; /* One bit for each page in the database file */
52810 i64 journalOff; /* Current write offset in the journal file */
52814 int nSavepoint; /* Number of elements in aSavepoint[] */
52825 u16 nExtra; /* Add this many bytes to each in-memory page */
52829 int pageSize; /* Number of bytes in a page */
52862 ** testing purposes only. These variables do not exist in
52882 ** written, semi-random garbage data might appear in the journal
52885 ** sanity checking data is an attempt to discover the garbage in the
52891 ** This cksum is initialized to a 32-bit random value that appears in the
52894 ** data that was once in other files that have now been deleted. If the
52904 ** The size of the of each page record in the journal is given by
52916 ** The macro MEMDB is true if we are dealing with an in-memory database.
52957 ** * there are no dirty pages in the cache, and
52958 ** * the desired page is not currently in the wal file.
52991 ** This function runs many asserts to try to find inconsistencies in
53020 /* Check that MEMDB implies noSync. And an in-memory journal. Since
53021 ** this means an in-memory pager performs no IO at all, it cannot encounter
53023 ** a journal file. (although the in-memory journal implementation may
53025 ** is therefore not possible for an in-memory pager to enter the ERROR
53114 ** in ERROR state. Otherwise the pager should have already dropped
53128 ** Return a pointer to a human readable string in a static buffer
53131 ** to "print *pPager" in gdb:
53135 ** This routine has external linkage in order to suppress compiler warnings
53137 ** not appear in normal builds.
53210 ** * The bit corresponding to the page-number is not set in
53232 ** Return true if the page is already in the journal file.
53241 ** that is read in *pRes. Return SQLITE_OK if everything worked, or an
53256 ** Write a 32-bit integer into a string buffer in big-endian byte order.
53337 ** returned in this case.
53431 ** name in the journal is longer than nSuper bytes (including a
53433 ** were present in the journal.
53448 u32 len; /* Length in bytes of super-journal name */
53449 i64 szJ; /* Total size in bytes of journal file pJrnl */
53489 ** following the value in pPager->journalOff, assuming a sector
53521 ** set to 0, then truncate the journal file to zero bytes in size. Otherwise,
53523 ** if the pager is not in no-sync mode, sync the journal file immediately
53528 ** journal file in bytes is larger than this value, then truncate the
53577 ** - 4 bytes: Number of records in journal, or -1 no-sync mode is on.
53614 ** if in full-sync mode), the zero is overwritten with the true number
53624 ** * When the pager is in no-sync mode. Corruption can follow a
53625 ** power failure in this case anyway.
53662 ** record is written to the following sector (leaving a gap in the file
53663 ** that will be implicitly filled in by the OS).
53671 ** The loop is required here in case the sector-size is larger than the
53673 ** bytes in size, more than one call to sqlite3OsWrite() may be required
53688 ** (JOURNAL_HDR_SZ bytes) is read from the current location in the journal
53689 ** file. The current location in the journal file is given by
53695 ** database before the transaction began, in pages. Also, pPager->cksumInit
53697 ** in this case.
53706 i64 journalSize, /* Size of the open journal file in bytes */
53726 /* Read in the first 8 bytes of the journal header. If they do not match
53772 ** are within range. To be 'in range', both values need to be a power
53780 /* If the either the page-size or sector-size in the journal-header is
53812 ** thing written to a journal file. If the pager is in full-sync mode, the
53817 ** + N bytes: super-journal filename in utf-8.
53818 ** + 4 bytes: N (length of super-journal name in bytes, no nul-terminator).
53822 ** The super-journal page checksum is the sum of the bytes in thesuper-journal
53831 i64 iHdrOff; /* Offset of header in journal file */
53847 /* Calculate the length in bytes and the checksum of zSuper */
53852 /* If in full-sync mode, advance to the next disk sector before writing
53853 ** the super-journal name. This is in case the previous page written to
53875 /* If the pager is in peristent-journal mode, then the physical
53882 ** Easiest thing to do in this scenario is to truncate the journal
53894 ** Discard the entire contents of the in-memory page-cache.
53910 ** Free all structures in the Pager.aSavepoint[] array and set both
53912 ** if it is open and the pager is not in exclusive mode.
53929 ** Set the bit number pgno in the PagerSavepoint.pInSavepoint
53949 ** This function is a no-op if the pager is in exclusive mode and not
53950 ** in the ERROR state. Otherwise, it switches the pager to PAGER_OPEN
53953 ** If the pager is not in exclusive-access mode, the database file is
53958 ** If the pager is in ERROR state when this function is called, the
53960 ** the OPEN state. Regardless of whether the pager is in exclusive-mode
53961 ** or not, any journal file left in the file-system will be treated
54001 /* If the pager is in the ERROR state and the call to unlock the database
54013 ** code is cleared and the cache reset in the block below.
54021 ** it can safely move back to PAGER_OPEN state. This happens in both
54052 ** is stored in Pager.errCode. While the pager remains in the ERROR state,
54093 ** when the number of dirty pages in memory exceeds 25% of the total
54110 ** This routine is never called in PAGER_ERROR state. If it is called
54111 ** in PAGER_NONE or PAGER_SHARED state and the lock held is less
54120 ** depends on whether or not the pager is running in exclusive mode and
54125 ** in-memory journal.
54128 ** Journal file is truncated to zero bytes in size.
54132 ** the first journal header in the file, and hence the entire journal
54138 ** If the pager is running in exclusive mode, this method of finalizing
54140 ** DELETE and the pager is in exclusive mode, the method described under
54144 ** If running in non-exclusive rollback mode, the lock on the file is
54151 ** tries to unlock the database file if not in exclusive mode. If the
54214 ** the database file, it will do so using an in-memory journal.
54253 /* Drop the WAL write-lock, if any. Also, if the connection was in
54260 /* This branch is taken when committing a transaction in rollback-journal
54292 ** call to pager_unlock() will discard all in-memory pages, unlock
54294 ** means that there is a hot-journal left in the file-system, the next
54328 ** Changing the formula used to compute this checksum results in an
54351 ** value is increased to the start of the next page in the journal.
54371 ** corrupted, SQLITE_DONE is returned. Data is considered corrupted in
54392 PgHdr *pPg; /* An existing page in the cache */
54393 Pgno pgno; /* The page number of a page in journal */
54411 ** is in state OPEN and holds an EXCLUSIVE lock. Hot-journal rollback
54462 /* If the pager is in CACHEMOD state, then there must be a copy of this
54463 ** page in the pager cache. In this case just update the pager cache,
54464 ** not the database file. The page is left marked dirty in this case.
54466 ** An exception to the above rule: If the database is in no-sync mode
54468 ** not be in the pager cache. Later: if a malloc() or IO error occurs
54469 ** during a Movepage() call, then the page may not be in the cache
54470 ** either. So the condition described in the above paragraph is not
54473 ** If in WRITER_DBMOD, WRITER_FINISHED or OPEN state, then we update the
54475 ** not dirty. Since this code is only executed in PAGER_OPEN state for
54477 ** if the pager is in OPEN state.
54485 ** journal. Otherwise, a power loss might leave modified data in the
54486 ** database file without an entry in the rollback journal that can
54490 ** in the main journal either because the page is not in cache or else
54524 ** is if the data was just read from an in-memory sub-journal. In that
54537 ** the database and the page is not in-memory, there is a potential
54543 ** obscure. When running in synchronous mode, this can only happen
54547 ** The solution is to add an in-memory page to the cache containing
54562 /* No page should ever be explicitly rolled back that is in use, except
54563 ** for page 1 which is held in use in order to keep the lock on the
54565 ** of an internal error resulting in an automatic call to
54659 ** sufficient space (in zSuperPtr) to hold the names of super-journal
54731 ** file in the file-system. This only happens when committing a transaction,
54734 ** If the main database file is not open, or the pager is not in either
54813 ** might change if a crash occurs while writing to a single byte in
54827 ** may not have been opened yet, in which case the OsSectorSize()
54837 ** the state it was in before we started making changes.
54843 ** in the journal. If this value is 0xffffffff, then compute the
54850 ** is this many bytes in size.
54859 ** Each entry in the journal is an instance of the 8th item.
54862 ** valid page entries in the journal. In most cases, you can compute the
54868 ** that reason, we always use the nRec value in the header.
54873 ** in this case. But for things like temporary table (which will be
54894 i64 szJ; /* Size of the journal file in bytes */
54895 u32 nRec; /* Number of Records in the journal */
54897 Pgno mxPg = 0; /* Size of the original file in pages */
54905 /* Figure out how many records are in the journal. Abort early if
54943 ** not enough bytes left in the journal file for a complete header, or
54956 ** working in no-sync mode. This means that the rest of the journal
54966 ** process and if this is the final header in the journal, then it means
54974 ** when doing a ROLLBACK and the nRec==0 chunk is the last chunk in
55014 ** case, the database should have never been written in the
55036 /* Following a rollback, the database file should be back in its original
55048 ** modification may just have been reverted. If this happens in exclusive
55051 ** problems for other processes at some point in the future. So, just
55052 ** in case this has happened, clear the changeCountDone flag now.
55057 /* Leave 4 bytes of space before the super-journal filename in memory.
55058 ** This is because it may end up being passed to sqlite3OsOpen(), in
55059 ** which case it requires 4 0x00 bytes in memory immediately before
55140 ** zero or the size of the database in page. Bytes 32..35 and 35..39
55165 ** Update the value of the change-counter at offsets 24 and 92 in
55179 /* Also store the SQLite version number in bytes 96..99 and in
55193 ** If page iPg is present in the cache, and has no outstanding references,
55238 /* For all pages in the cache that are currently dirty or have already
55273 int nList; /* Number of pages in pList */
55279 /* Verify that the page list is in accending order */
55287 /* If a WAL transaction is being committed, there is no point in writing
55329 ** makes a snapshot of the database at the current point in time and preserves
55330 ** that snapshot for use by the reader in spite of concurrently changes by
55341 ** transaction in locking_mode=EXCLUSIVE. So call it now. If we
55342 ** are in locking_mode=NORMAL and EndRead() was previously called,
55360 ** in pages (assuming the page size currently stored in Pager.pageSize).
55363 ** in pages is stored in *pnPage. Otherwise, an error code (perhaps
55381 /* If the number of pages in the database is not available from the
55387 i64 n = 0; /* Size of db file in bytes */
55395 /* If the current number of pages in the file is greater than the
55414 ** in WAL mode. If the database is empty or if no *-wal file exists and
55465 ** performed in the order specified:
55481 ** corresponding bit is set in a bitvec structure (variable pDone in the
55483 ** rolled back the first time it is encountered in either journal.
55486 ** journal file. There is no need for a bitvec in this case.
55521 ** journal. The actual file might be larger than this in
55530 ** There might be records in the main journal that have a page number
55560 ** test is related to ticket #2565. See the discussion in the
55576 ** previously rolled back out of the main journal (and are hence in pDone)
55602 ** Change the maximum number of in-memory pages that are allowed
55610 ** Change the maximum number of in-memory pages that are allowed
55649 ** Adjust settings of the pager to those specified in the pgFlags parameter.
55651 ** The "level" in pgFlags & PAGER_SYNCHRONOUS_MASK sets the robustness
55663 ** in a state which would cause damage to the database
55668 ** of the journal header - being written in between the two
55683 ** an ordinary commit in NORMAL mode with WAL. FULL means that the WAL
55684 ** file is synced following each commit operation, in addition to the
55815 ** is passed in *pPageSize.
55817 ** If the pager is in the error state when this function is called, it
55828 ** * the database is either not an in-memory database or it is
55829 ** an in-memory database that currently consists of zero pages.
55839 ** conditions above is not true, the pager was in error state when this
55850 ** At one point this function returned an error if the pager was in
55929 ** less than the total number of valid pages in the database. But this
55937 ** errors in places where we do not care about errors.
55963 ** opened on a file less than N bytes in size, the output buffer is
55995 ** the pager. It returns the total number of pages in the database.
55997 ** However, if the file is between 1 and <page-size> bytes in size, then
56042 ** following is true for all dirty pages currently in the page-cache:
56045 ** current database image, in pages, OR
56056 ** content of the page. However, since this content is not present in either
56075 ** Truncate the in-memory database file image to nPage pages. This
56092 ** if one or more savepoints are open, present in the savepoint
56193 ** Free all PgHdr objects stored in the Pager.pMmapFreelist list.
56232 ** If a transaction was in progress when this routine is called, that
56236 ** result in a coredump.
56240 ** a hot journal may be left in the filesystem but no error is returned
56319 ** disk and can be restored in the event of a hot-journal rollback.
56325 ** * If the journal file is an in-memory journal file, no action need
56331 ** been written following it. If the pager is operating in full-sync
56337 ** Or, in pseudo-code:
56339 ** if( NOT <in-memory journal> ){
56348 ** page currently held in memory before returning SQLITE_OK. If an IO
56371 ** that wrote to this database was operating in persistent-journal
56373 ** than Pager.journalOff bytes. If the next thing in the journal
56409 /* Write the nRec value into the journal file header. If in
56415 ** SAFE_APPEND property. Because in this case it is not possible
56452 /* Unless the pager is in noSync mode, the journal file was just
56463 ** The argument is the first in a linked list of dirty pages connected
56465 ** in-memory pages in the list to the database file. The argument may
56479 ** the pages are written out to the database file in list order. Writing
56487 ** in Pager.dbFileVers[] is updated to match the new value stored in
56497 /* This function is only called for rollback pagers in WRITER_DBMOD state. */
56528 /* If there are dirty pages in the page cache with page numbers greater
56549 ** the value now stored in the database file. If writing this
56603 ** If successful, set the bit corresponding to pPg->pgno in the bitvecs
56608 ** SQLITE_NOMEM if a malloc fails while setting a bit in a savepoint
56658 ** (cast as a void*). The pager is always 'purgeable' (not an in-memory
56690 ** Spilling is also prohibited when in an error state since that could
56693 ** while in the error state, hence it is impossible for this routine to
56694 ** be called in the error state. Nevertheless, we include a NEVER()
56770 ** in *ppPager. The pager should eventually be freed by passing it
56777 ** all information is held in cache. It is never written to disk.
56778 ** This can be used to implement an in-memory database.
56804 int nExtra, /* Extra bytes append to each in-memory page */
56812 int tempFile = 0; /* True for temp files (incl. in-memory files) */
56813 int memDb = 0; /* True if this is an in-memory file */
56822 int nPathname = 0; /* Number of bytes in zPathname */
56834 /* Set the output variable to NULL in case an error occurs. */
56849 /* Compute and store the full pathname in an allocated buffer pointed
56883 ** bytes in length. This means the database cannot be opened,
56897 ** file name. The layout in memory is as follows:
56916 ** misuse of SQLite and a bug in the 3rd-party software, but the 3rd-party
56917 ** software is in widespread use, so we try to avoid changing the filename
56967 /* Fill in the Pager.zFilename and pPager.zQueryParam fields */
56980 /* Fill in Pager.zJournal */
56994 /* Fill in Pager.zWal */
57024 ** choose a default page size in case we have to create the
57069 ** This branch is also run for an in-memory database. An in-memory
57071 ** disk and uses an in-memory rollback journal.
57078 pPager->eLock = EXCLUSIVE_LOCK; /* Pretend we are in EXCLUSIVE mode */
57183 ** PAGER_SHARED state. It tests if there is a hot journal present in
57188 ** * The journal file exists in the file system, and
57190 ** * The database file itself is greater than 0 bytes in size, and
57238 ** in fact there is none. This results in a false-positive which will
57243 Pgno nPage; /* Number of pages in database file */
57248 /* If the database is zero pages in size, that means that either (1) the
57285 /* If we cannot open the rollback journal file in order to see if
57287 ** it might be due to the race condition described above and in
57313 ** 1) If the pager is currently in PAGER_OPEN state (no lock held
57322 ** 2) If the pager is running in exclusive-mode, and there are currently
57323 ** no outstanding references to any pages, and is in the error state,
57337 ** be OPEN or READER. READER is only possible if the pager is or was in
57380 ** this point in the code and fail to obtain its own EXCLUSIVE lock
57383 ** Unless the pager is in locking_mode=exclusive mode, the lock is
57393 ** in exclusive-access mode the file descriptor will be kept open
57395 ** is usually required to finalize the journal in journal_mode=persist
57399 ** other connection managed to get in and roll it back before
57401 ** may mean that the pager was in the error-state when this
57451 ** to ERROR state in the state diagram at the top of this file,
57455 ** to be in ERROR state when there are zero outstanding page
57472 ** occurring on the very first access to a file, in order to save a
57479 ** a codec is in use.
57511 /* If there is a WAL file in the file-system, open this database in WAL
57545 ** Except, in locking_mode=EXCLUSIVE when there is nothing to in
57565 ** getPageError() -- Used if the pager is in an error state
57568 ** If the requested page is already in the cache, it is returned.
57576 ** already in the cache when this function is called, then the extra
57581 ** requested page is not already stored in the cache, then no
57586 ** the contents of the page. This occurs in two scenarios:
57596 ** to pgno in Pager.pInJournal (bitvec of pages already written to the
57599 ** point in the future, using a call to sqlite3PagerWrite(), its contents
57606 ** to find a page in the in-memory cache first. If the page is not already
57607 ** in memory, this routine goes to disk to read it in whereas Lookup()
57674 /* Failure to set the bits in the InJournal bit-vectors is benign.
57678 ** a bit in a bit vector.
57728 ** temporary or in-memory database. */
57737 ** test in the previous statement, and avoid testing pgno==0 in the
57810 ** Acquire a page if it is already in the in-memory cache. Do
57812 ** or 0 if the page is not in cache.
57816 ** in the page if the page is not already in cache. This routine
57817 ** returns NULL if the page is not in cache or if a disk I/O error
57879 ** If the journal file is already open (as it may be in exclusive mode),
57898 /* If already in the error state, this function is a no-op. But on
57899 ** the other hand, this routine is never called if we are already in
57972 ** within this transaction will be opened as an in-memory file. This
57974 ** running in exclusive mode) or if the transaction does not require a
57976 ** sub-journal is implemented in-memory if pPager is an in-memory database,
58024 ** This is because in those states the code to roll back savepoint
58026 ** file as well as into the page cache. Which would be incorrect in
58065 ** page in the block above, set the need-sync flag for the page.
58066 ** Otherwise, when the transaction is rolled back, the logic in
58068 ** in the database file. And if an IO error occurs while doing so,
58101 ** one of the journals, the corresponding bit is set in the
58111 ** It is never called in the ERROR state.
58128 ** an error might occur and the pager would end up in WRITER_LOCKED state
58129 ** with pages marked as dirty in the cache.
58141 /* If a rollback journal is in use, them make sure the page that is about
58142 ** to change is in the rollback journal, or if the page is a new page off
58172 /* If the statement journal is open and the page is not in it,
58190 ** a sector need to be journalled in case of a power loss in the middle of
58193 ** Usually, the sector size is less than or equal to the page size, in which
58194 ** case pages can be individually written. This routine only runs in the
58199 Pgno nPageCount; /* Total number of pages in database file */
58310 ** Return TRUE if the page given in the argument was previously passed
58337 ** to be written out to disk so that it may be read back in if the
58383 ** atomic-write optimization is enabled in this build, then isDirect
58389 ** 'isDirect' below, as well as the block enclosed in the
58410 ** operating in direct-mode, make page 1 writable. When not in
58411 ** direct mode, page 1 is always held in cache and hence the PagerGet()
58422 /* If running in direct mode, write the contents of page 1 to the file. */
58451 ** Sync the database file to disk. This is a no-op for in-memory databases
58470 ** This function may only be called while a write-transaction is active in
58471 ** rollback. If the connection is in WAL mode, this call is a no-op.
58476 ** successful, or the connection is in WAL mode, SQLITE_OK is returned.
58520 ** journal file in this case.
58551 /* If this is an in-memory db, or no pages have been written to, or this
58553 ** backup in progress needs to be restarted. */
58599 ** * Exactly one page has been modified and store in the journal file.
58603 ** counter in 'indirect-mode'. If the optimization is compiled in but
58606 ** pager_incr_changecounter() to update the change-counter in indirect
58611 ** in 'direct' mode. In this case the journal file will never be
58626 ** following call will modify the in-memory representation of page 1
58664 ** journal requires a sync here. However, in locking_mode=exclusive
58711 ** last page in the db image moved to the free-list. In this case the
58740 ** synced to disk. The journal file still exists in the file-system
58756 ** But if (due to a coding error elsewhere in the system) it does get
58768 ** this transaction, the pager is running in exclusive-mode and is
58775 ** header. Since the pager is in exclusive mode, there is no need
58798 ** If the pager is already in PAGER_ERROR state when this function is called,
58799 ** it returns Pager.errCode immediately. No work is performed in this case.
58801 ** Otherwise, in rollback mode, this function performs two functions:
58804 ** in-memory cache pages to the state they were in when the transaction
58808 ** rollback at any point in the future.
58822 /* PagerRollback() is a no-op if called in READER or OPEN state. If
58823 ** the pager is already in the ERROR state, the rollback is not
58866 ** if the database is (in theory) writable.
58953 ** Return true if this is an in-memory or temp-file backed pager.
58980 ** if the allocation fails. Otherwise, zero the new portion in case a
58981 ** malloc failure occurs while populating it in the for(...) loop below.
59072 ** operation. Store this value in nNew. Then free resources associated
59082 ** the sub-journal to zero bytes in size. */
59086 /* Only truncate if it is an in-memory sub-journal. */
59108 ** back journal_mode=off, put the pager in the error state. This way,
59128 ** Except, if the pager is in-memory only, then return an empty string if
59132 ** shared cache, it uses nullIfMemDb==0 so that in-memory databases can
59133 ** participate in shared-cache.
59180 ** Move the page pPg to location pgno in the file.
59184 ** in cache. If the page previously located at pgno is not already
59185 ** in the rollback journal, it is not put there by by this routine.
59188 ** meta-data associated with pPg (i.e. data stored in the nExtra bytes
59216 /* In order to be able to rollback, an in-memory database must journal
59230 ** <journal page X, then modify it in memory>
59254 ** be written to, store pPg->pgno in local variable needSyncPgno.
59282 /* Do not discard pages from an in-memory database since we might
59294 /* For an in-memory database, make sure the original page continues
59295 ** to exist, in case the transaction needs to roll back. Use pPgOld
59306 ** Currently, no such page exists in the page-cache and the
59312 ** to a malloc() or IO failure), clear the bit in the pInJournal[]
59313 ** array. Otherwise, if the page is loaded and written again in
59315 ** it is synced into the journal file. This way, it may end up in
59400 ** * An in-memory database can only have its journal_mode set to _OFF
59424 /* Do allow the journalmode of an in-memory database to be set to
59441 ** mode except WAL, unless the pager is in locking_mode=exclusive mode,
59460 ** while it is in use by some other client.
59503 ** Return TRUE if the pager is in a state where it is OK to change the
59530 ** in backup.c maintains the content of this variable. This module
59540 ** Unless this is an in-memory or temporary database, clear the pager cache.
59561 int *pnLog, /* OUT: Final number of frames in log */
59609 ** Call sqlite3WalOpen() to open the WAL handle. If the pager is in
59612 ** in. Otherwise, use the normal shared-memory.
59620 /* If the pager is already in exclusive-mode, the WAL module will use
59649 ** file (not a temp file or an in-memory database), and the WAL file
59653 ** not modified in either case.
59655 ** If the pager is open on a temp-file (or in-memory database), or if
59703 /* If the log file is not already open, but does exist in the file-system,
59738 ** If pager pPager is a wal-mode database not in exclusive locking mode,
59844 ** the pager is in WAL mode and the WAL file currently contains one or more
59845 ** frames, return the size in bytes of the page images stored within the
59871 ** This file contains the implementation of a write-ahead log (WAL) used in
59882 ** transferred back into the database file in an operation called a
59892 ** The WAL header is 32 bytes in size and consists of the following eight
59910 ** 4: For commit records, the size of the database image in pages
59920 ** (1) The salt-1 and salt-2 values in the frame-header match
59921 ** salt values in the wal-header
59923 ** (2) The checksum values in the final 8 bytes of the frame-header
59929 ** magic number in the first 4 bytes of the WAL is 0x377f0683 and it
59931 ** The checksum values are always stored in the frame header in a
59943 ** in reverse order (the largest fibonacci weight occurs on the first element
59954 ** value is randomized. This prevents old and new frames in the WAL from
59969 ** valid frame in the WAL. The reader uses this recorded "mxFrame" value
59973 ** of the database from a single point in time. This technique allows
59977 ** The reader algorithm in the previous paragraphs works correctly, but
60003 ** as big endian, the wal-index can store multi-byte values in the native
60008 ** last frame in the wal before frame M for page P in the WAL, or return
60009 ** NULL if there are no frames for page P in the WAL prior to M.
60015 ** in the mxFrame field.
60021 ** first index block are the same size as all other index blocks in the
60028 ** for the first index block) 32-bit page numbers. The first entry in the
60030 ** first frame in the WAL file. The first entry in the second index block
60031 ** in the WAL file corresponds to the (HASHTABLE_NPAGE_ONE+1)th frame in
60034 ** The last index block in a wal-index usually contains less than the full
60048 ** HASHTABLE_NSLOT = 2*HASHTABLE_NPAGE, and there is one entry in the
60049 ** hash table for each page number in the mapping section, so the hash
60052 ** 1-based index of an entry in the mapping section of the same
60053 ** index block. Let K be the 1-based index of the largest entry in
60059 ** To look for page P in the hash table, first compute a hash iKey on
60072 ** no hash slot such that aHash[i]==p) then page P is not in the
60079 ** average, only two or three slots in each index block need to be
60080 ** examined in order to either find the last entry for page P, or to
60081 ** establish that no such entry exists in the block. Each index block
60084 ** comparisons (on average) suffice to either locate a frame in the
60085 ** WAL or to establish that the frame does not exist in the WAL. This
60088 ** Note that entries are added in order of increasing K. Hence, one
60093 ** the correct result. There may be entries in the hash table with
60095 ** slots in the hash table and so the first reader will get an answer as
60097 ** in the first place - which is what reader one wants. Meanwhile, the
60124 ** values in the wal-header are correct and (b) the version field is not
60144 ** WalCkptInfo.aLock[] array in the wal-index header. In other words, all
60146 ** should be 120) is the location in the shm file for the first locking
60166 ** The actual header in the wal-index consists of two copies of this
60174 ** added in 3.7.1 when support for 64K pages was added.
60181 u8 bigEndCksum; /* True if checksums in WAL are big-endian */
60182 u16 szPage; /* Database page size in bytes. 1==64K */
60183 u32 mxFrame; /* Index of last valid frame in the WAL */
60184 u32 nPage; /* Size of database in pages */
60185 u32 aFrameCksum[2]; /* Checksum of last frame in log */
60191 ** A copy of the following object occurs in the wal-index immediately
60195 ** nBackfill is the number of frames in the WAL that have been written
60213 ** There is one entry in aReadMark[] for each reader lock. If a reader
60214 ** holds read-lock K, then the value in aReadMark[K] is no greater than
60229 ** in use (that is, every aReadMark[j] for which there is a corresponding
60234 ** in the WAL has been backfilled into the database) then new readers
60242 ** (in other words, if there are no WAL_READ_LOCK(i) where i>0) then
60246 ** We assume that 32-bit loads are atomic and so no locks are needed in
60267 /* Size of header before each frame in wal */
60274 ** significant bit also set (WAL_MAGIC | 0x00000001) is stored in 32-bit
60275 ** big-endian format in the first 4 bytes of a WAL file.
60285 ** Return the offset of frame iFrame in the write-ahead log file,
60305 volatile u32 **apWiData; /* Pointer to wal-index content in memory */
60309 u8 exclusiveMode; /* Non-zero if connection is in exclusive mode */
60310 u8 writeLock; /* True if in a write transaction */
60321 u32 nCkpt; /* Checkpoint sequence counter in the wal-header */
60355 ** all frames in the WAL in database page order. Where two or more frames
60357 ** frame most recently written to the WAL (in other words, the frame with
60370 int nSegment; /* Number of entries in aSegment[] */
60372 int iNext; /* Next slot in aIndex[] not yet returned */
60375 int nEntry; /* Nr. of entries in aPgno[] and aIndex[] */
60377 } aSegment[1]; /* One for every 32KB page in the wal-index */
60381 ** Define the parameters of the hash tables in the wal-index file. There
60382 ** is a hash-table following every HASHTABLE_NPAGE page numbers in the
60393 ** The block of page numbers associated with the first hash-table in a
60477 ** Return a pointer to the WalCkptInfo structure in the wal-index.
60485 ** Return a pointer to the WalIndexHdr structure in the wal-index.
60505 ** Generate or extend an 8 byte checksum based on the data in
60516 int nByte, /* Bytes of content in a[]. Must be a multiple of 8. */
60576 ** Write the header information in pWal->hdr into the wal-index.
60600 ** 4: For commit records, the size of the database image in pages
60634 ** Check to see if the frame with header in aFrame[] and content
60635 ** in aData[] is valid. If it is a valid frame, fill *piPage and
60650 /* A frame is only valid if the salt values in the frame-header
60651 ** match the salt values in the wal-header.
60666 ** and the frame-data matches the checksum in the last 8
60754 ** the hash to the next value in the event of a collision.
60767 ** of a page hash table in the wal-index. This becomes the return value
60783 ** in the wal-index file. Set pLoc->iZero to one less than the frame
60785 ** slot in the hash table is set to N, it refers to frame number
60786 ** (pLoc->iZero+N) in the log.
60833 ** Return the page number associated with frame iFrame in this WAL.
60858 int nByte; /* Number of bytes to zero in aPgno[] */
60876 if( NEVER(rc) ) return; /* Defense-in-depth, in case (1) above is wrong */
60889 /* Zero the entries in the aPgno array that correspond to frames with
60896 /* Verify that the every entry in the mapping region is still reachable
60914 ** Set an entry in the wal-index that will map database page number
60943 /* If the entry in aPgno[] is already set, then the previous writer
60944 ** must have exited unexpectedly in the middle of a transaction (after
60963 /* Verify that the number of entries in the hash table exactly equals
60964 ** the number of entries in the mapping region.
60968 int nEntry = 0; /* Number of entries in the hash table */
60973 /* Verify that the every entry in the mapping region is reachable
61013 /* Obtain an exclusive lock on all byte in the locking range not already
61042 int szFrame; /* Number of bytes in buffer aFrame[] */
61049 u32 iLastFrame; /* Last frame in wal, based on nSize alone */
61051 /* Read in the WAL header. */
61261 int bNoShm, /* True to run in heap-memory mode */
61274 ** in os_unix.c and os_win.c agree with the WALINDEX_LOCK_OFFSET value.
61336 ** Find the smallest page number out of all pages held in the WAL that
61342 ** Return 0 on success. If there are no pages in the WAL with a page
61386 ** of indices such that the aRight[] contains every index that appears in
61399 const u32 *aContent, /* Pages in wal - keys for the sort */
61401 int nLeft, /* IN: Elements in array *paLeft */
61403 int *pnRight, /* IN/OUT: Elements in *paRight */
61406 int iLeft = 0; /* Current index in aLeft */
61407 int iRight = 0; /* Current index in aRight */
61408 int iOut = 0; /* Current index in output buffer */
61439 ** Sort the elements in list aList using aContent[] as the sort key.
61443 ** The aList[] entries are indices into aContent[]. The values in
61456 const u32 *aContent, /* Pages in wal */
61459 int *pnList /* IN/OUT: Number of elements in aList[] */
61462 int nList; /* Number of elements in aList */
61467 int nMerge = 0; /* Number of elements in list aMerge */
61524 ** pages in the WAL following frame nBackfill in ascending order. Frames
61538 u32 iLast; /* Last frame in log */
61545 ** it only runs if there is actually content in the log (mxFrame>0).
61578 int nEntry; /* Number of entries in this segment */
61721 ** Return the page-size in bytes used by the database.
61740 ** The value of parameter salt1 is used as the aSalt[1] value in the
61762 ** in response to an sqlite3_wal_checkpoint() request or the equivalent.
61768 ** All I/O barrier operations (a.k.a fsyncs) occur in this routine when
61769 ** SQLite is in WAL-mode in synchronous=NORMAL. That means that if
61775 ** in the WAL and can be recovered following a power-loss or hard reset.
61779 ** it safe to delete the WAL since the new content will persist in the
61788 ** checkpoint is running (in any other thread or process) at the same
61817 ** in the SQLITE_CHECKPOINT_PASSIVE mode. */
61820 /* Compute in mxSafeFrame the index of the last frame of the WAL that is
61822 ** overwrite database pages that are in use by active readers and thus
61872 ** database plus the amount of data in the wal file, plus the
61953 ** wal-index header in shared memory, as all subsequent reader or
61956 ** as it would leave the system in a state where the contents of
61974 ** If the WAL file is currently larger than nMax bytes in size, truncate
62029 ** fsyned (rc==SQLITE_OK) and if we are not in persistent-wal
62034 ** completed and fsynced (rc==SQLITE_OK) and we are in persistent
62062 ** The wal-index is in shared memory. Another thread or process might
62064 ** read it, which might result in inconsistency. A dirty read is detected
62078 WalIndexHdr volatile *aHdr; /* Header in shared memory */
62084 ** same area of shared memory on a different CPU in a SMP,
62090 ** When reading, read [0] first then [1]. Writes are in the reverse order.
62097 ** reliably in that environment.
62138 ** Set *pChanged to 1 if the wal-index header value in pWal->hdr is
62156 assert( rc!=SQLITE_READONLY ); /* READONLY changed to OK in walIndexPage */
62174 /* page0 can be NULL if the SHM is zero bytes in size and pWal->writeLock
62243 ** Open a transaction in a connection where the shared-memory is read-only
62256 ** constructed in pWal->apWiData[] using heap memory instead of shared
62270 i64 szWal; /* Size of wal file on disk in bytes */
62274 int szFrame; /* Number of bytes in buffer aFrame[] */
62301 ** shared memory to be used in its place.
62324 ** Assume the in-memory WAL-index substitute is correct and load it
62329 /* Make sure some writer hasn't come in and changed the WAL file out
62355 ** Return WAL_RETRY which will cause the in-memory WAL-index to be
62445 ** in the range 0 <= pWal->readLock < WAL_NREADER. If pWal->readLock==(-1)
62460 ** update values of the aReadMark[] array in the header, but if it does
62465 volatile WalCkptInfo *pInfo; /* Checkpoint information in wal-index */
62495 int nDelay = 1; /* Pause time in microseconds */
62510 /* If there is not a recovery running in another thread or process
62522 ** modules may return SQLITE_BUSY due to a race condition in the
62568 ** it finished. Leaving a corrupt image in the database file.
62627 ** value in the aReadMark[] array or the contents of the wal-index
62634 ** that occur later in the log than pWal->hdr.mxFrame may have been
62641 ** since it was read, set Wal.minFrame to the first frame in the wal
62647 ** nBackfill and checking that the wal-header in shared-memory still
62648 ** matches the one cached in pWal->hdr, it is guaranteed that the
62650 ** header newer than that cached in pWal->hdr. If it were, that could
62654 ** page later in the wal file. But if version B happens to like past
62702 i64 szDb; /* Size of db file in bytes */
62714 u32 pgno; /* Page number in db file */
62755 ** instant in time. The current thread will continue to use this snapshot.
62783 ** snapshot X, where X is later in the wal file than pSnapshot, but
62847 ** have omitted to checkpoint a frame earlier than minFrame in
62882 ** contains the page. Otherwise, if pgno is not in the wal file, set *piRead
62894 u32 iLast = pWal->hdr.mxFrame; /* Last page in WAL for this reader */
62903 ** in this case as an optimization. Likewise, if pWal->readLock==0,
62916 ** This code might run concurrently to the code in walIndexAppend()
62926 ** For the reasons above, the if(...) condition featured in the inner
62989 ** (which is nOut bytes in size). Return SQLITE_OK if successful, or an
62995 int nOut, /* Size of buffer pOut in bytes */
63010 ** Return the size of the database in pages (or zero, if unknown).
63029 ** returns SQLITE_BUSY in that case and no write transaction is started.
63110 ** was in before the client began writing to the database.
63119 ** is passed as the second argument is (a) in the cache and
63141 ** point in the event of a savepoint rollback (via WalSavepointUndo()).
63153 ** the values in the aWalData[] array. aWalData must point to an array
63206 /* If all readers are using WAL_READ_LOCK(0) (in other words if no
63253 ** In other words, if iSyncPoint is in between iOffset and iOffset+iAmt,
63290 u8 aFrame[WAL_FRAME_HDRSIZE]; /* Buffer to assemble frame-header in */
63312 u8 aFrame[WAL_FRAME_HDRSIZE]; /* Buffer to assemble frame-headers in */
63359 int szPage, /* Database page-size in bytes */
63368 PgHdr *pLast = 0; /* Last frame in list */
63371 i64 iOffset; /* Next byte to write in WAL file */
63408 u8 aWalHdr[WAL_HDRSIZE]; /* Buffer to assemble wal-header in */
63435 ** an out-of-order write following a WAL restart could result in
63533 /* If this frame set completes the first transaction in the WAL and
63549 ** be in use by existing readers is being overwritten.
63604 int *pnLog, /* OUT: Number of frames in WAL */
63605 int *pnCkpt /* OUT: Number of backfilled frames in WAL */
63616 ** in the SQLITE_CHECKPOINT_PASSIVE mode. */
63633 ** it will not be invoked in this case.
63716 ** number of frames in the WAL at the point of the last commit since
63735 ** on the pWal->readLock byte. If the WAL is already in locking_mode=NORMAL
63744 ** WAL is already in exclusive-locking mode - meaning that this
63760 ** happen if the connection is actually in exclusive mode (as no xShmLock
63761 ** locks are taken in this case). Nor should the pager attempt to
63775 /* Already in locking_mode=NORMAL */
63802 ** in the object.
63843 /* aSalt[0] is a copy of the value stored in the wal file header. It
63925 ** This code really belongs in btree.c. But btree.c is getting too
63929 /************** Include btreeInt.h in the middle of btmutex.c ****************/
63963 ** disk where M is the number of entries in the tree.
63989 ** 16 2 Page size in bytes. (1 means 65536)
63999 ** 36 4 Number of freelist pages in the file
64021 ** space in a page that can be consumed by a single cell for standard
64032 ** except that it applies to leaf nodes in a LEAFDATA tree. The maximum
64034 ** not specified in the header.
64069 ** which is stored in the key size entry of the cell header rather than in
64074 ** offsets from the beginning of the page to the cell content in the cell
64075 ** content area. The cell pointers occur in sorted order. The system strives
64083 ** freeblocks. Each freeblock is at least 4 bytes in size. The byte offset
64084 ** to the first freeblock is given in the header. Freeblocks occur in
64085 ** increasing order. Because a freeblock must be at least 4 bytes in size,
64086 ** any group of 3 or fewer unused bytes in the cell content area cannot
64088 ** a fragment. The total number of bytes in all fragments is recorded.
64089 ** in the page header at offset 7.
64093 ** 2 Bytes in this freeblock
64095 ** Cells are of variable length. Cells are stored in the cell content area at
64096 ** the end of the page. Pointers to the cells are in the cell pointer array
64098 ** contiguous or in order, but cell pointers are contiguous and in order.
64106 ** allows a 64-bit integer to be encoded in 9 bytes.
64117 ** bytes of key and data in a btree cell.
64136 ** Freelist pages come in two subtypes: trunk pages and leaf pages. The
64137 ** file header points to the first in a linked list of trunk page. Each trunk
64156 ** plus 2 bytes for the index to the cell in the page header). Such
64168 ** SQLite database in order to identify the file as a real database.
64193 ** page that has been loaded into memory. The information in this object
64201 ** stored in MemPage.pBt->mutex.
64215 u8 nOverflow; /* Number of overflow cell bodies in aCell[] */
64218 u16 cellOffset; /* Index in aData of first cell pointer */
64246 BtLock *pNext; /* Next in BtShared.pLock list */
64268 ** All fields in this structure are accessed under sqlite3.mutex.
64270 ** in the referenced BtShared that point back to this Btree since those
64322 ** A single database file can be in use at the same time by two
64329 ** Fields in this structure are accessed under the BtShared.mutex
64349 ** while in the 'pending-lock' state, no connection may start a new
64369 u16 maxLocal; /* Maximum local payload in non-LEAFDATA tables */
64370 u16 minLocal; /* Minimum local payload in non-LEAFDATA tables */
64371 u16 maxLeaf; /* Maximum local payload in a LEAFDATA table */
64372 u16 minLeaf; /* Minimum local payload in a LEAFDATA table */
64376 u32 nPage; /* Number of pages in the database */
64406 ** about a cell. The parseCellPtr() function fills in this structure
64432 ** The entry is identified by its MemPage and the index in
64439 ** Fields in this structure are accessed under the BtShared.mutex
64473 i8 iPage; /* Index of current page in apPage */
64476 u16 aiIdx[BTCURSOR_MAX_DEPTH-1]; /* Current index in apPage[i] */
64512 ** in variables BtCursor.pKey and BtCursor.nKey. When a cursor is in
64519 ** cursor. The error has left the cache in an inconsistent state.
64521 ** should return the error code stored in BtCursor.skipNext
64538 ** page number to look up in the pointer map.
64555 ** each child page in the database file. The parent page is the page that
64556 ** contains a pointer to the child. Every page in the database contains
64563 ** position in the file to another as part of autovacuum. When a page
64564 ** is moved, the pointer in its parent must be updated to point to the
64568 ** used in this case.
64571 ** is not used in this case.
64573 ** PTRMAP_OVERFLOW1: The database page is the first page in a list of
64577 ** PTRMAP_OVERFLOW2: The database page is the second or later page in a list of
64579 ** page in the overflow page list.
64582 ** identifies the parent page in the btree.
64614 ** in order to keep track of some global state information.
64616 ** The aRef[] array is allocated so that there is 1 bit for each page in
64617 ** the database. As the integrity-check proceeds, for each page used in
64626 u8 *aPgRef; /* 1 bit per page in the db (see above) */
64627 Pgno nPage; /* Number of pages in the database */
64632 Pgno v1; /* Value for first %u substitution in zPfx */
64633 int v2; /* Value for second %d substitution in zPfx */
64650 ** cell addresses in a btree header.
64663 /************** Continuing where we left off in btmutex.c ********************/
64705 ** But we keep a reference count in Btree.wantToLock so the behavior
64708 ** To avoid deadlocks, multiple Btrees are locked in the same order
64718 ** connected by pNext and pPrev should be in sorted order by
64748 ** in the common case.
64765 ** the other BtShared locks that we used to hold in ascending
64826 ** Enter the mutexes in accending order by BtShared pointer address
64828 ** two or more btrees in common both try to lock all their btrees
64913 ** in single threaded applications that use shared cache. Except for
64914 ** these two routines, all mutex operations are no-ops in that case and
64915 ** are null #defines in btree.h.
64918 ** the ones below, are no-ops and are null #defines in btree.h.
65024 ** in shared cache. This variable has file scope during normal builds,
65057 ** manipulate entries in the BtShared.pLock linked list used to store
65124 ** When writing to an index that resides in a sharable database, the
65177 ** useful in that case. */
65274 /* The condition (pIter->eLock!=eLock) in the following if(...)
65280 ** may hold a WRITE_LOCK on any table in this file (since there can
65327 ** by a connection in read-uncommitted mode is on the sqlite_schema
65328 ** table, and that lock is obtained in BtreeBeginTrans(). */
65415 ** be zero already. So this next line is harmless in that case.
65454 ** database connetion. This is important in shared-cache mode. If the database
65554 ** set in the bitvec. Whenever a leaf page is extracted from the free-list,
65556 ** set in BtShared.pHasContent. The contents of the bitvec are cleared
65612 ** function saves the current cursor key in variables pCur->nKey and
65617 ** (the rowid) is stored in pCur->nKey and pCur->pKey is left set to
65619 ** set to point to a malloced buffer pCur->nKey bytes in size containing
65658 ** Save the current cursor position in the variables BtCursor.nKey
65696 ** the location in the btree is remembered in such a way that it can be
65699 ** table, for example in BtreeDelete() or BtreeInsert().
65703 ** routine enforces that rule. This routine only needs to be called in
65711 ** need to be saved. It calls out to saveCursorsOnList() in the (unusual)
65712 ** event that cursors are in need to being saved.
65800 ** Restore the cursor to the position it was in (or as close to as possible)
65957 int offset; /* Offset in pointer map page */
66016 int offset; /* Offset of entry in pointer map */
66074 CellInfo *pInfo /* Fill in this structure */
66080 ** in between minLocal and maxLocal.
66082 ** Warning: changing the way overflow payload is distributed in any
66083 ** way will result in an incompatible file format.
66124 ** Parse a cell content block and fill in the CellInfo structure.
66137 CellInfo *pInfo /* Fill in this structure */
66154 CellInfo *pInfo /* Fill in this structure */
66222 CellInfo *pInfo /* Fill in this structure */
66259 CellInfo *pInfo /* Fill in this structure */
66268 ** Compute the total number of bytes that a Cell needs in the cell
66386 ** present in the page after this routine returns.
66390 ** unused bytes are contained in the unallocated space region, and all
66425 ** offsets to each pointer in the cell-pointer array than it is to
66473 /* These conditions have already been verified in btreeInitPage()
66516 ** Search the free-list on page pPg for space to store a cell nByte bytes in
66542 ** in bytes, including the 4-byte header. */
66549 ** number of bytes in fragments may not exceed 60. */
66571 /* The next slot in the chain is not past the end of the current slot */
66591 ** allocation. This routine might need to defragment in order to bring
66594 ** allocation is being made in order to insert a new cell, so we will
66618 ** However, that integer is too large to be stored in a 2-byte unsigned
66619 ** integer, so a value of 0 is used in its place. */
66667 /* Allocate memory from the gap in between the cell pointer array
66697 u8 nFrag = 0; /* Reduction in fragmentation */
66711 /* The list of freeblocks must be in ascending order. Find the
66749 ** pointer in the page header) then check to see if iStart should be
66851 ** in the pPage->nFree field.
66908 /* Freeblock not in ascending order */
67085 ** to fetch the content. Just fill in the content with zeros for now.
67086 ** If in the future we call sqlite3PagerWrite() on this page, that
67093 MemPage **ppPage, /* Return the page in this parameter */
67109 ** already in the pager cache return NULL. Initialize the MemPage.pBt and
67123 ** Return the size of the database file in pages. If there is any kind of
67138 ** call. Do additional sanity checking on the page in this case.
67234 ** * If the page is already in use for some other purpose, immediately
67241 MemPage **ppPage, /* Return the page in this parameter */
67280 ** the call for every page that comes in for re-initing. */
67301 ** be exclusively in memory, or it might use a disk-based memory cache.
67305 ** If zFilename is ":memory:" then an in-memory database is created
67311 ** If the database is already opened in the same database connection
67312 ** and we are in shared cache mode, then the open will fail with an
67314 ** objects in the same database connection since doing so will lead
67335 /* Set the variable isMemdb to true for an in-memory database, or
67349 assert( (flags&0xff)==flags ); /* flags fit in 8 bits */
67494 /* If the magic name ":memory:" will create an in-memory database, then
67547 ** The list is kept in ascending order by pBt address.
67660 ** a cell is less than 4 bytes in size, it is rounded up to 4 bytes
67754 ** Change the "soft" limit on the number of pages in the cache.
67756 ** pages in the cache exceeds this soft limit. But the size of the
67758 ** dirty pages or pages still in active use.
67770 ** Change the "spill" limit on the number of pages in the cache.
67772 ** the pager might attempt to "spill" pages to the journal early in
67805 ** Change the way data is synced to disk in order to increase or decrease
67887 ** This is useful in one special case in the backup API code where it is
67943 ** freelist leaf pages are not written back to the database. Thus in-page
68054 u32 nPage; /* Number of pages in the database */
68055 u32 nPageFile = 0; /* Number of pages in the database file */
68056 u32 nPageHeader; /* Number of pages in the database according to hdr */
68082 ** with the following 16 bytes (in hex): 53 51 4c 69 74 65 20 66 6f 72 6d
68104 ** in WAL mode. If the log is not already open, open it now. Then
68107 ** required as the version of page 1 currently in the page1 buffer
68108 ** may not be the latest version - there may be a newer one in the log
68152 /* EVIDENCE-OF: R-59310-51205 The "reserved space" size in the 1-byte
68229 ** in assert() expressions, so it is only compiled if NDEBUG is not
68251 ** If there are no outstanding cursors and we are not in the middle
68256 ** If there is a transaction in progress, this routine is a no-op.
68353 ** returned when there is already a read-lock in order to avoid a deadlock.
68371 /* If the btree is already in a write-transaction, or it
68372 ** is already in a read-transaction and a read-transaction
68545 int nCell; /* Number of cells in page pPage */
68586 ** overflow page in the list.
68592 /* The pointer is always the first 4 bytes of the page in this case. */
68641 ** Move the open database page pDbPage to location iFreePage in the
68728 ** SQLITE_OK. If there is no work to do (and therefore no point in
68733 ** that the last page of the file currently in use is no longer in use.
68836 ** nOrig pages in size containing nFree free pages. Return the expected
68837 ** size of the database in pages following an auto-vacuum operation.
68905 ** pages are in use.
68916 Pgno nFin; /* Number of pages in database after autovacuuming */
69050 ** Commit the transaction currently in progress.
69057 ** routine has to do is delete or truncate or zero the header in the
69067 ** functions return code. So, even if an error occurs in the pager layer,
69185 ** Rollback the transaction in progress.
69190 ** a tripped cursor will result in an error.
69330 ** are also met. These are the conditions that must be met in order
69336 ** but which are not in the READ_UNCOMMITTED state may not have
69339 ** the read cursors in the other database connection.
69349 ** this implementation. But in a hypothetical alternative storage engine
69350 ** in which index entries are automatically deleted when corresponding table
69405 /* Now that no other errors can occur, finish filling in the BtCursor
69455 ** Return the size of a BtCursor object in bytes.
69518 ** Make sure the BtCursor* given in the argument has a valid
69520 ** btreeParseCell() to fill it in.
69522 ** BtCursor.info is a cache of the information in the current cell.
69555 ** that is currently pointing to a row in a (non-empty) table.
69643 ** Given the page number of an overflow page in the database (parameter
69644 ** ovfl), this function finds the page number of the next page in the
69650 ** The page number of the next overflow page in the linked list is
69651 ** written to *pPgnoNext. If page ovfl is the last page in its linked
69675 /* Try to find the next page in the overflow list using the
69676 ** autovacuum pointer-map pages. Guess that the next page in
69772 ** the cursor is moved to a different row. Additionally, in auto-vacuum
69776 ** * A commit in auto_vacuum="full" mode,
69839 ** in the overflow chain. The page number of the first overflow page is
69840 ** stored in aOverflow[0], etc. A value of 0 in the aOverflow[] array
69882 ** number for the next page in the overflow chain. The page
69909 ** 3) there are no dirty pages in the page-cache
69911 ** 5) the page is not in the WAL file
69974 ** to a valid row in the table. For sqlite3BtreePayloadChecked(), the
69991 ** in the CURSOR_VALID state. It is only used by the sqlite3_blob_read()
70075 ** in the common case where no overflow pages are used.
70114 ** cell in page pParent. Or, if iIdx is equal to the total number of
70115 ** cells in pParent, that page number iChild is the right-child of
70120 ** in a corrupt database */
70233 ** in such a way that page pRoot is linked into a second b-tree table
70266 ** in ascending order.
70291 ** key in ascending order.
70312 /* Move the cursor to the first entry in the table. Return SQLITE_OK
70334 /* Move the cursor to the last entry in the table. Return SQLITE_OK
70348 ** to the last entry in the b-tree. */
70406 ** exists an entry in the table that exactly matches pIdxKey.
70493 u8 *pCell; /* Pointer to current cell in pPage */
70545 int nCell; /* Size of the pCell cell in bytes */
70578 ** bytes of padding is allocated at the end of the buffer in
70660 ** past the last entry in the table or sqlite3BtreePrev() moves past
70664 /* TODO: What if the cursor is in CURSOR_REQUIRESEEK but all table entries
70672 ** Return an estimate for the number of rows in the table that pCur is
70697 ** Advance the cursor to the next entry in the database.
70710 ** If bit 0x01 of the F argument in sqlite3BtreeNext(C,F) is 1, then the
70743 ** in a corrupt database file where the table being DELETE-ed from
70744 ** has pages in common with the table being queried. See TH3
70753 ** only happen if the database is corrupt in such a way as to link the
70788 UNUSED_PARAMETER( flags ); /* Used in COMDB2 but not native SQLite */
70807 ** Step the cursor to the back to the previous entry in the database.
70878 UNUSED_PARAMETER( flags ); /* Used in COMDB2 but not native SQLite */
70900 ** an error. *ppPage is set to NULL in the event of an error.
70903 ** locate a page close to the page number "nearby". This can be used in an
70904 ** attempt to keep related pages close to each other in the database file,
70905 ** which in turn can make database access faster.
70982 ** is the page number of the next freelist trunk page in the list or
71051 ** page in this case.
71163 ** that is required in the event of a rollback. In this case, do
71275 /* If the database supports auto-vacuum, write an entry in the pointer-map
71285 ** trunk page in the free-list is full, then this page will become a
71287 ** first trunk page in the current free-list. This block tests if it
71315 ** coded. But due to a coding error in versions of SQLite prior to
71320 ** for now. At some point in the future (once everyone has upgraded
71325 ** avoid using the last six entries in the freelist trunk page array in
71344 ** the page being freed as a leaf page of the first trunk in the free-list.
71346 ** first trunk in the free-list is full. Either way, the page being freed
71347 ** will become the new first trunk page in the free-list.
71377 ** size information about the cell in pInfo.
71433 ** caller is iterating through or using in some other way, this
71453 ** allocated and filled in as necessary. The calling procedure
71459 ** be constructed in this temporary area then copied into pPage->aData
71486 /* Fill in the header. */
71502 /* Fill in the payload */
71536 ** nPayload Total payload size in bytes
71605 ** then the optimistic overflow chain processing in clearCell()
71650 ** "sz" must be the number of bytes in the cell.
71701 ** in pPage->apOvfl[] and make it point to the cell content (either
71702 ** in pTemp or the original pCell) and also record its index.
71703 ** Allocating a new entry in pPage->aCell[] implies that
71712 int sz, /* Bytes of content in pCell */
71717 int idx = 0; /* Where to write new cell content in data[] */
71720 u8 *pIns; /* The point in pPage->aCellIdx[] where no cell inserted */
71747 /* When multiple overflows occur, they are always sequential and in
71752 assert( j==0 || pPage->aiOvfl[j-1]<(u16)i ); /* Overflows in sorted order */
71772 /* In a corrupt database where an entry in the cell index section of
71774 ** as many as 4 bytes in front of the start of the aData buffer for
71802 ** in a balancing operation. NN is the number of neighbors on either side
71803 ** of the page that participate in the balancing operation. NB is the
71808 ** (to 2 or 3) gives a modest improvement in SELECT and DELETE performance
71809 ** in exchange for a larger degradation in INSERT and UPDATE performance.
71818 #define NB 3 /* (NN*2+1): Total pages involved in the balance */
71824 ** The cells in this array are the divider cell or cells from the pParent
71849 ** The order of cells is in the array is for an index btree is:
71851 ** 1. All cells from Child-1 in order
71853 ** 3. All cells from Child-2 in order
71855 ** 5. All cells from Child-3 in order
71858 ** content exists only in leaves and there are no divider cells.
71862 ** respectively. The ixNx[] array holds the number of cells contained in
71865 ** ixNx[0] = Number of cells in Child-1.
71866 ** ixNx[1] = Number of cells in Child-1 plus 1 for first divider.
71867 ** ixNx[2] = Number of cells in Child-1 and Child-2 + 1 for 1st divider.
71868 ** ixNx[3] = Number of cells in Child-1 and Child-2 + both divider cells
71874 ** ixNx[0] = Number of cells in Child-1.
71875 ** ixNx[1] = Number of cells in Child-1 and Child-2.
71885 int nCell; /* Number of cells in apCell[] */
71888 u16 *szCell; /* Local size of all cells in apCell[] */
71929 ** szCell[] array contains the size in bytes of each cell. This function
71933 ** Some of the cells in apCell[] may currently be stored in pPg. This
71942 int iFirst, /* First cell in pCArray to use */
71956 int k; /* Current slot in pCArray->apEnd[] */
72009 ** This function attempts to add the cells stored in the array to page pPg.
72014 ** Argument pCellptr points to the first entry in the cell-pointer array
72017 ** cell in the array. It is the responsibility of the caller to ensure
72028 ** area must be extended to before this point in order to accomodate all
72029 ** cells in apCell[], then the cells do not fit and non-zero is returned.
72044 int k; /* Current slot in pCArray->apEnd[] */
72090 ** This function adds the space associated with each cell in the array
72142 ** pCArray contains pointers to and sizes of all cells in the page being
72261 ** tree, in other words, when the new entry will become the largest
72262 ** entry in the tree.
72265 ** a new page to the right-hand side and put the one new entry in
72271 ** pPage is the leaf page which is the right-most page in the tree.
72279 ** least 13 bytes in size.
72348 ** record-length (a variable length integer at most 32-bits in size)
72423 ** parent page stored in the pointer map is page pTo. If pFrom contained
72427 ** If pFrom is currently carrying any overflow cells (entries in the
72482 ** page are used in the balancing, though both siblings might come from one
72486 ** participate in the balancing.
72489 ** one or two in an effort to keep pages nearly full but not over full.
72492 ** might not actually be stored in MemPage.aData[]. This can happen
72503 ** in a corrupted state. So if this routine fails, the database should
72520 int iParentIdx, /* Index of "the page" in pParent */
72527 int nNew = 0; /* Number of pages in apNew[] */
72528 int nOld; /* Number of pages in apOld[] */
72530 int nxDiv; /* Next divider slot in pParent->aCell[] */
72534 int usableSpace; /* Bytes in pPage beyond the header */
72541 u8 *pRight; /* Location in parent of right-sibling pointer */
72542 u8 *apDiv[NB-1]; /* Divider cells in pParent */
72543 int cntNew[NB+2]; /* Index in b.paCell[] of cell after i-th page */
72544 int cntOld[NB+2]; /* Old index in b.apCell[] */
72547 Pgno pgno; /* Temp var to store a page number in */
72574 /* Find the sibling pages to balance. Also locate the cells in pParent
72582 ** overflow cells in the parent page, since if any existed they will
72640 ** But not if we are in secure-delete mode. In secure-delete mode,
72660 /* Make nMaxCells a multiple of 4 in order to preserve 8-byte
72691 ** into aSpace1[]. In this way, all cells in b.apCell[] are without
72692 ** child pointers. If siblings are not leaves, then all cell in
72693 ** b.apCell[] include child pointers. Either way, all cells in b.apCell[]
72719 /* Load b.apCell[] with pointers to all cells in pOld. If pOld
72720 ** contains overflow cells, include them in the b.apCell[] array
72721 ** in the correct spot.
72727 ** adjacent and are inserted in order. There is an assert() tagged
72728 ** with "NOTE 1" in the overflow cell insertion loop to prove this
72731 ** This must be done in advance. Once the balance starts, the cell
72800 ** Store this number in "k". Also compute szNew[] which is the total
72802 ** in b.apCell[] of the cell that divides page i from page i+1.
72809 ** cntNew[i]: Index in b.apCell[] and b.szCell[] for the first cell to
72892 int r; /* Index of right-most cell in left sibling */
72969 ** Reassign page numbers so that the new pages are in ascending order.
72970 ** This helps to keep entries in the disk file in order so that a scan
72971 ** of the table is closer to a linear scan through the file. That in turn
72988 ** we do the detection here in order to avoid populating the pager
73033 ** originally in the same field of the right-most old sibling page. */
73045 ** entry associated with the first page in the overflow chain, and
73114 ** then there is no divider cell in b.apCell[]. Instead, the divider
73150 /* Now update the actual sibling pages. The order in which they are updated
73164 ** The iPg value in the following loop starts at nNew-1 goes down
73220 ** sub-algorithm in some documentation.
73379 ** The page that pCur currently points to has just been modified in
73446 ** happens, the overflow cell is stored in the aBalanceQuickSpace[]
73452 ** of the aBalanceQuickSpace[] might sneak in.
73467 ** are stored in the pSpace buffer allocated immediately below.
73555 ** contained in pX.
73618 ** key is an arbitrary byte sequence stored in pX.pKey,nKey. The
73880 ** is advantageous to leave the cursor pointing to the last entry in
73882 ** entry in the table, and the next row inserted has an integer key
73928 ** for the destination database. The size of the cell, in bytes, is left
73929 ** in BtShared.nPreformatSize. The caller completes the insertion by
74032 ** But if that bit is set, then the cursor is left in a state such that
74053 int bSkipnext = 0; /* Leaf cursor in SKIPNEXT state */
74078 ** key and leaving the cursor in CURSOR_REQUIRESEEK state before
74082 ** will be left in CURSOR_SKIPNEXT state pointing to the entry immediately
74099 ** the cursor to the largest entry in the tree that is smaller than
74134 ** is currently pointing to the largest entry in the sub-tree headed
74178 ** tree that we can be sure that any problem in the internal node has
74222 ** following values of flags are currently in use. Other values for
74294 /* Save the positions of any open cursors. This is required in
74345 /* When the new root page was allocated, page 1 was made writable in
74440 ** Delete all information from a single table in the database. iTable is
74450 ** entries in the table.
74481 ** Erase all information in a table and add the root of the table to
74489 ** root page in the database file, then the last root page
74490 ** in the database file is moved into the slot formerly occupied by
74495 ** page number that used to be the last root page in the file before
74497 ** The last root page is recorded in meta[3] and the value of
74532 ** number in the database, put the root page on the free list.
74541 ** number in the database. So move the page that does into the
74565 /* Set the new 'max-root-page' value in the database header. This
74599 ** is the number of free pages currently in the database. Meta[1]
74609 ** from the pager. The BTREE_DATA_VERSION value is not actually stored in the
74629 /* If auto-vacuum is disabled in this build and this is an auto-vacuum
74670 ** number of entries in the b-tree and write the result to *pnEntry.
74677 i64 nEntry = 0; /* Value to return in *pnEntry */
74687 ** page in the B-Tree structure (not including overflow pages).
74690 int iIdx; /* Index of child node in parent */
74704 ** the next page in the tree that has not yet been visited. The
74706 ** of the page, or to the number of cells in the page if the next page
74709 ** If all pages in the tree have been visited, return SQLITE_OK to the
74780 ** Return non-zero if the bit in the IntegrityCk.aPgRef[] array that
74789 ** Set the bit in the IntegrityCk.aPgRef[] array that corresponds to page iPg.
74803 ** Also check that the page number is in bounds.
74821 ** Check that the entry in the pointer-map for page iChild maps to
74857 Pgno iPage, /* Page number for first page in the list */
74858 u32 N /* Expected number of pages in the list */
74900 ** page in this overflow list, check that the pointer-map entry for
74930 ** root node aHeap[1] is always the minimum value currently in the heap.
74934 ** removes the root element from the heap (the minimum value in the heap)
74982 ** 2. Make sure integer cell keys are in order.
75033 /* Clear MemPage.isInit to make sure the corruption detection code in
75079 /* For leaf pages, the coverage check will occur in the same loop
75185 ** big-endian integer which is the offset in the b-tree page of the next
75186 ** freeblock in the chain, or zero if the freeblock is the last on the
75189 /* EVIDENCE-OF: R-06866-39125 Freeblocks are always connected in order of
75196 ** freeblocks, and counting the number of untracked bytes in nFrag.
75203 ** The loop below pulls entries from the min-heap in order and compares
75221 /* EVIDENCE-OF: R-43263-13491 The total number of bytes in all fragments
75222 ** is stored in the fifth field of the b-tree page header.
75247 ** a table. nRoot is the number of entries in aRoot.
75252 ** Write the number of error seen in *pnErr. Except for some memory
75253 ** allocation errors, an error message held in memory obtained from
75257 ** If the first entry in aRoot[] is 0, that indicates that the list of
75260 ** zero is skipped, of course. But in addition, the freelist checks
75270 int nRoot, /* Number of entries in aRoot[] */
75372 /* Make sure every page in the file is referenced
75416 ** an empty string if the database is in-memory or a TEMP database.
75581 ** required in case any of them are holding references to an xFetch
75621 ** "write version" (single byte at byte offset 19) fields in the database
75719 u32 iDestSchema; /* Original schema cookie in destination */
75747 ** structure, in that order.
75750 ** invoked by the pager layer to report various state changes in
75764 ** Non-sharable Btrees (in-memory databases for example), do not have
75770 ** in connection handle pDb. If such a database cannot be found, return
75818 ** message in database handle db.
75822 sqlite3ErrorWithMsg(db, SQLITE_ERROR, "destination database is in use");
75830 ** connection handle pSrcDb to zDestDb in pDestDb. If successful, return
75834 ** stored in database handle pDestDb.
75852 ** handle is not locked in this routine, but it is locked in
75856 ** database connection while a backup is in progress may cause
75942 /* Catch the case where the destination is an in-memory database and the
76036 int nSrcPage = -1; /* Size of source db in pages */
76039 /* If the source pager is currently in a write-transaction, return
76060 ** is especially important on ZipVFS systems, as in that case it is
76075 /* Do not allow backup if the destination database is in WAL mode
76085 ** source pager for the number of pages in the database.
76111 /* Update the schema version field in the destination database. This
76112 ** is to make sure that the schema-version really does change in
76134 /* Set nDestTruncate to the final number of pages in the destination
76141 ** sqlite3PagerTruncateImage() here so that any pages in the
76166 ** pending-byte page in the source database may need to be
76184 ** database has been stored in the journal for pDestPager and the
76186 ** the database file in any way, knowing that if a power failure
76335 ** Return the total number of pages in the source database as of the most
76444 /* 0x7FFFFFFF is the hard limit for the number of pages in a database
76483 ** stores a single value in the VDBE. Mem is an opaque structure visible
76511 ** That saves a few cycles in inner loops. */
76552 ** (1) Memory in Mem.zMalloc and managed by the Mem object
76698 ** in pMem->z is discarded.
76753 ** Any prior string or blob content in the pMem object may be discarded.
76778 ** to be a double-zero byte at an even byte boundary in order to
76794 ** Change pMem so that its MEM_Str or MEM_Blob value is stored in
76819 ** blob stored in dynamically allocated space.
76933 ** the results in memory cell pMem.
76957 ** invoking the external callback in Mem.xDel, then this routine
76981 ** by p->xDel and memory in p->zMalloc.
76983 ** This is a helper routine invoked by sqlite3VdbeMemRelease() in
76984 ** the unusual case where there really is memory in p that needs
77136 ** The second and third terms in the following conditional enforces
77222 ** Cast the datatype of the value in pMem according to the affinity
77223 ** "aff". Casting is different from applying affinity in that a cast
77225 ** affinity even if that results in loss of data. This routine is
77280 ** Delete any previous value and set the value stored in *pMem to NULL.
77328 ** Delete any previous value and set the value stored in *pMem to val,
77344 ** Set the value stored in *pMem should already be a NULL.
77363 ** Delete any previous value and set the value stored in *pMem to val,
77439 sqlite3DebugPrintf("Invalidate R[%d] due to change in R[%d]\n",
77506 ** Transfer the contents of pFrom to pTo. Any existing value in pTo is
77540 int n, /* Bytes in string, or negative */
77574 ** also sets a flag in local variable "flags" to indicate the memory
77643 ** to read from the disk) then the pMem is left in an inconsistent state.
77649 Mem *pMem /* OUT: Return data in this Mem structure. */
77671 Mem *pMem /* OUT: Return data in this Mem structure. */
77734 ** external API. It works in a similar way to sqlite3_value_text(),
77735 ** except the data returned is in the encoding specified by the second
77939 ** Extract a value from the supplied expression in the manner described
77946 ** in all cases.
77988 /* Handle negative integers in a single step. This is needed in the
78121 ** A value is extracted in the following cases:
78176 ** in the sqlite_stat4 table.
78250 ** is undefined in this case.
78262 ** Extract the iCol-th column from the nRec-byte record in pRec. Write
78272 int nRec, /* Size of buffer pRec in bytes */
78277 int nHdr; /* Size of the header in the record */
78352 ** The sqlite3ValueBytes() routine returns the number of bytes in the
78434 ** Change the error string stored in Vdbe.zErrMsg
78590 ** Add a new instruction to the list of instructions current in the
78674 ** One register is initialized for each characgter in zTypes[]. For each
78675 ** "s" character in zTypes[], the register is a string if the argument is
78677 ** in zTypes[], the register is initialized to an integer.
78726 ** means in the WHERE clause of a partial index. NC_GenCol means called
78793 ** Set a debugger breakpoint on the following routine in order to
78980 ** in a Vdbe main program and each of the sub-programs (triggers) it may
78998 int nSub; /* Number of entries in apSub */
79047 ** Check if the program stored in the VM associated with pParse may
79065 ** part of an assert statement in the compiler. Similar to:
79095 ** OP_Clear. So this routine may end up returning true in the case
79113 ** true for this case to prevent the assert() in the callers frame
79123 ** Increment the nWrite counter in the VDBE if the cursor is not an
79139 ** Assert if an Abort at this point in time might result in a corrupt
79155 ** and store that value in *pMaxFuncArgs.
79178 /* Only JUMP opcodes and the short list of special opcodes in the switch
79276 ** Verify that at least N opcode slots are available in p without
79292 ** by code in pragma.c to ensure that the implementation of certain
79293 ** pragmas comports with the flags specified in the mkpragmatab.tcl
79307 ** verify that the VDBE program can safely call Abort in the current
79322 ** Before returning, *pnOp is set to the number of entries in the returned
79324 ** the number of entries in the Vdbe.apArg[] array required to execute the
79607 ** registers, except any identified by mask, are no longer in use.
79715 ** P4 must not be P4_INT32. Use sqlite3VdbeChangeP4() in either of
79752 ** in a production build.
79837 ** The Synopsis: field in comments in the vdbe.c source file gets converted
79936 ** that can be displayed in the P4 column of EXPLAIN output.
80132 ** The prepared statements need to know in advance the complete set of
80134 ** is maintained in p->btreeMask. The p->lockMask value is the subset of
80163 ** statement p will ever use. Let N be the number of bits in p->btreeMask
80331 ** Locate the next opcode to be displayed in EXPLAIN or EXPLAIN
80351 int iPc; /* Rowid. Copy of value in *piPc */
80355 ** nRow is the sum of the number of rows in the main program, plus
80356 ** the sum of the number of rows in all trigger subprograms encountered
80381 /* The rowid is small enough that we are still in the
80400 ** kept in p->aMem[9].z to hold the new program - assuming this subprogram
80447 ** allocated by the OP_Program opcode in sqlite3VdbeExec().
80464 ** Give a listing of the program in the virtual machine.
80472 ** are shown in a different format. p->explain==2 is used to implement
80634 ** memory on success. If insufficient memory is available in the
80666 ** Rewind the VDBE back to the beginning in preparation for
80730 int nArg; /* Number of arguments in subprograms */
80793 /* Memory for registers, parameters, cursor, etc, is allocated in one or two
80797 ** pass will fill in the remainder using a fresh memory allocation.
80877 ** Close all cursors in the current frame.
80893 ** Copy the values stored in the VdbeFrame structure to its Vdbe. This
80921 ** Also release any dynamic memory held by the VM in the Vdbe.aMem memory
80923 ** pointers to VdbeFrame objects, which may in turn contain pointers to
81025 ** virtual module tables written in this transaction. This has to
81167 /* Write the name of each database file in the transaction into the new
81204 /* Sync all the db files involved in the transaction. The same call
81205 ** sets the super-journal pointer in each individual journal. If
81211 ** in case the super-journal file name was written into the journal
81265 ** matches the number of vdbe's in the list sqlite3.pVdbe that are
81386 ** has made changes and is in autocommit mode, then commit those
81391 ** call this on a VM that is in the SQLITE_MAGIC_HALT state.
81412 ** Then the internal cache might have been left in an inconsistent
81450 ** pagerStress() in pager.c), the rollback is required to restore
81596 ** in p->rc. This routine sets that result back to SQLITE_OK.
81689 ** called), set the database error in this case as well.
81779 ** * the corresponding bit in argument mask is clear (where the first
81967 ** encapsulate the code that serializes values for storage in SQLite
81996 ** The 8 and 9 types were added in 3.3.0, file format 4. Prior versions
82002 ** Return the serial-type for the value stored in pMem.
82007 ** opcode in the byte-code engine. But by moving this routine in-line, we
82120 ** float in the wrong order. And that error has been propagated
82145 static u64 floatSwap(u64 in){
82152 u.r = in;
82164 ** Write the serialized data blob for the value stored in pMem into
82168 ** nBuf is the amount of space left in buf[]. The caller is responsible
82173 ** of bytes in the zero-filled tail is included in the return value only
82174 ** if those bytes were zeroed in buf[].
82223 ** and store the result in pMem. Return the number of bytes read.
82227 ** routine so that in most cases the overhead of moving the stack pointer
82314 /* Work around a sign-extension bug in the HP compiler for HP/UX */
82331 /* These use local variables, so do them in a separate routine
82332 ** to avoid having to move the frame pointer in the common case */
82344 /* EVIDENCE-OF: R-14606-31564 Value is a BLOB that is (N-12)/2 bytes in
82346 ** EVIDENCE-OF: R-28401-00140 Value is a string in the text encoding and
82347 ** (N-13)/2 bytes in length. */
82387 ** Given the nKey-byte encoding of a record in pKey[], populate the
82399 u32 idx; /* Offset in aKey[] to read from */
82435 ** This function compares two index or table record keys in the same way
82439 ** in assert() statements to ensure that the optimized code in
82452 u32 szHdr1; /* Number of bytes in header */
82468 ** But in fact, mem1.u.i will never actually be used uninitialized, and doing
82485 /* Read the serial types for the next element in each key. */
82492 ** sqlite3VdbeSerialTypeLen() in the common case.
82546 ** Count the number of fields (a.k.a. columns) in the record given by
82584 ** *pMem2, respectively. Similar in spirit to "rc = (*pMem1) - (*pMem2);".
82593 /* The strings are already in the correct encoding. Call the
82676 if( x>r ) return +1; /*NO_TEST*/ /* work around bugs in gcov */
82677 return 0; /*NO_TEST*/ /* work around bugs in gcov */
82848 ** fields that appear in both keys are equal, then pPKey2->default_rc is
82863 u32 szHdr1; /* Size of record header in bytes */
82864 u32 idx1; /* Offset of first type in header */
82872 ** two elements in the keys are equal. Fix the various stack variables so
83064 ** size-of-header varint at the start of (pKey1/nKey1) fits in a single
83149 ** fields. Return pPKey2->default_rc in this case. */
83162 ** at the start of (pKey1/nKey1) fits in a single byte.
83231 ** fits in a single byte (i.e. is 127 or less). varintRecordCompareInt()
83236 ** limit the size of the header to 64 bytes in cases where the first field
83273 ** Read the rowid (the last field in the record) and store it in *rowid.
83289 ** Any corruption is detected in sqlite3BtreeParseCellPtr(), though, so
83296 /* Read in the complete content of the index entry */
83349 ** the key string in pUnpacked. Write into *pRes a number
83400 ** Set a flag in the vdbe to update the change counter when it is finalised
83474 ** in a better query plan.
83511 zMsg = sqlite3_mprintf("non-deterministic use of %s() in %s",
83523 ** in memory obtained from sqlite3_malloc) into a Vdbe.zErrMsg (text stored
83524 ** in memory obtained from sqlite3DbMalloc).
83541 ** with the memory cells in the p->aMem[] array. Also free the UnpackedRecord
83545 ** the vdbeUnpackRecord() function found in vdbeapi.c.
83650 ** execution environment changes in a way that would alter the program
83772 ** Set all the parameters in the compiled SQL statement to NULL.
83858 /* EVIDENCE-OF: R-12793-43283 Every value in SQLite has one of five
84001 int n, /* Bytes in string, or negative */
84257 ** for version 3.6.23 or earlier do in fact depend on SQLITE_MISUSE
84287 ** error has occurred, then return the error code in p->rc to the
84288 ** caller. Set the error code in the database handle to the same value.
84349 ** error has occurred, then return the error code in p->rc to the
84350 ** caller. Set the error code in the database handle to the same value.
84622 ** Return the number of columns in the result set for the statement pStmt.
84701 ** column value (i.e. a value returned by evaluating an SQL expression in the
84735 ** in the result set.
84944 ** Routines used to attach values to wildcards in a compiled SQL statement.
84947 ** Unbind the value bound to variable i in virtual machine p. This is the
84954 ** The error code stored in database p->db is overwritten with the return
84955 ** value in any case.
84981 /* If the bit corresponding to this variable in Vdbe.expmask is set, then
84985 ** parameter in the WHERE clause might influence the choice of query plan
85288 ** in the argument belongs. This is the same database handle that was
85290 ** the statement in the first place.
85313 ** Return true if the prepared statement is in need of being reset.
85426 ** record in nKey/pKey. Return a pointer to the new UnpackedRecord structure
85509 ** the number of columns in the row being updated, deleted or inserted.
85706 ** bytes in this text up to but excluding the first character in
85708 ** the total number of bytes in the text.
85730 ** This function returns a pointer to a nul-terminated string in memory
85745 ** ALGORITHM: Scan the input string looking for host parameters in any of
85750 ** parameter index is known, locate the value in p->aVar[]. Then render
85751 ** the value as a literal in place of the host parameter name.
85819 int nOut; /* Number of bytes of the string text to include in output */
85852 int nOut; /* Number of bytes of the blob to include in output */
85890 ** The code in this file implements the function that runs the
85893 ** Various scripts scan this source file in order to generate HTML
85895 ** of the code in this file is, therefore, important. See other comments
85896 ** in this file for details. If in doubt, do not deviate from existing
85907 ** is changed while the copy is still in use, the string or blob might
85930 ** each instruction in the VDBE. When it reaches zero, the u1.isInterrupted
85931 ** field of the sqlite3 structure is set in order to simulate an interrupt.
85934 ** in an ordinary build.
86034 ** may not be taken, depending on the SQLITE_JUMPIFNULL flags in p5.
86040 ** code line number is in the lower 24 bits of iSrcLine and the upper
86048 ** interested in equal or not-equal. In other words, I==0 and I==2
86068 ** a branch really does go in one of those directions, assert right
86080 ** are filled in automatically so that the coverage
86121 int nField, /* Number of fields in the table or index */
86131 ** purposes in a vdbe program. The different uses might require
86173 ** The string in pRec is known to look like an integer and to have a
86175 ** integer value if the string is in range to be an integer. Otherwise,
86456 ** Show the values of all registers in the virtual machine. Used for
86479 /************** Include hwtime.h in the middle of vdbe.c *********************/
86558 ** and analysis configurations, not in any deliverable, so this
86568 /************** Continuing where we left off in vdbe.c ***********************/
86576 ** the number of non-transaction savepoints currently in the
86784 ** separate instruction in the virtual machine. If we follow the usual
86788 ** big comment (similar to this one) will mark the point in the code where
86800 ** Other keywords in the comment that follows each case are used to
86807 ** comment lines are used in the generation of the opcode.html documentation
86813 ** Do not deviate from the formatting style currently in use.
86894 /* Most jump operations do a goto to this spot in order to update
86903 ** Jump to the next instruction after the address in register P1. After
86939 ** The instruction at the address in register P1 is a Yield.
86960 ** Swap the program counter with the value in register P1. This
86986 ** Check the value in register P3. If it is NULL then Halt using
86988 ** value in register P3 is not NULL, then this routine is a no-op.
87078 sqlite3_log(pOp->p1, "abort at %d in [%s]: %s", pcx, p->zSql, p->zErrMsg);
87175 ** The string value P4 of length P1 (bytes) is stored in register P2.
87207 ** NULL into register P3 and every register in between P2 and P3. If P3
87256 ** blob in register P2.
87272 ** If the parameter is named, then its name appears in P4.
87296 ** Move the P3 values in register P1..P1+P3-1 over into
87397 ** Transfer the integer value held in register P1 into register P2.
87413 ** Output value in register P1 as the chance count for a DML statement,
87415 ** foreign key error in the statement, trigger the error now.
87462 /* The registers in the result will not be used again when the
87487 ** Add the text in register P1 onto the end of the text in
87488 ** register P2 and store the result in register P3.
87489 ** If either the P1 or P2 text are NULL then store NULL in P3.
87558 ** Add the value in register P1 to the value in register P2
87559 ** and store the result in register P3.
87566 ** Multiply the value in register P1 by the value in register P2
87567 ** and store the result in register P3.
87573 ** Subtract the value in register P1 from the value in register P2
87574 ** and store the result in register P3.
87580 ** Divide the value in register P1 by the value in register P2
87581 ** and store the result in register P3 (P3=P2/P1). If the value in
87589 ** register P1 and store the result in register P3.
87590 ** If the value in register P1 is zero the result is NULL.
87681 ** be returned. This is used by the built-in min(), max() and nullif()
87690 ** publicly. Only built-in functions have access to this feature.
87703 ** Take the bit-wise AND of the values in register P1 and P2 and
87704 ** store the result in register P3.
87710 ** Take the bit-wise OR of the values in register P1 and P2 and
87711 ** store the result in register P3.
87717 ** Shift the integer value in register P2 to the left by the
87718 ** number of bits specified by the integer in register P1.
87719 ** Store the result in register P3.
87725 ** Shift the integer value in register P2 to the right by the
87726 ** number of bits specified by the integer in register P1.
87727 ** Store the result in register P3.
87756 /* If shifting by a negative amount, shift in the other direction */
87785 ** Add the constant P2 to the value in register P1.
87800 ** Force the value in register P1 to be an integer. If the value
87801 ** in P1 is not an integer and cannot be converted into an integer
87850 ** Force the value in register P1 to be the type defined by P2.
87884 ** Compare the values in register P1 and P3. If reg(P3)==reg(P1) then
87885 ** jump to address P2. Or if the SQLITE_STOREP2 flag is set in P5, then
87886 ** store the result of comparison in register P2.
87899 ** are text, then the appropriate collating function specified in
87906 ** If SQLITE_NULLEQ is set in P5 then the result of comparison is always either
87920 ** the operands in registers P1 and P3 are not equal. See the Eq opcode for
87930 ** Compare the values in register P1 and P3. If reg(P3)<reg(P1) then
87931 ** jump to address P2. Or if the SQLITE_STOREP2 flag is set in P5 store
87949 ** are text, then the appropriate collating function specified in
88075 ** the answer to this operator in res2, depending on what the comparison
88077 ** that the 6 comparison operators are consecutive integers in this
88104 ** is only used in contexts where either:
88136 ** SQLITE_STOREP2 bit set in the P5 field.
88165 ** Set the permutation used by the OP_Compare operator in the next
88166 ** instruction. The permutation is stored in the P4 operand.
88169 ** the OPFLAG_PERMUTE bit set in P5. Typically the OP_Permutation should
88172 ** The first integer in the P4 integer array is the length of the array
88186 ** Compare two vectors of registers in reg(P1)..reg(P1+P3-1) (call this
88187 ** vector "A") and in reg(P2)..reg(P2+P3-1) ("B"). Save the result of
88192 ** OPFLAG_PERMUTE bit is clear, then register are compared in sequential
88266 ** in the most recent OP_Compare instruction the P1 vector was less than
88283 ** Take the logical AND of the values in registers P1 and P2 and
88293 ** Take the logical OR of the values in register P1 and P2 and
88294 ** store the answer in register P3.
88330 ** Interpret the value in register P1 as a boolean value. Store that
88331 ** boolean (a 0 or 1) in register P2. Or if the value in register P1 is
88332 ** NULL, then the P3 is stored in register P2. Invert the answer if P4
88356 ** Interpret the value in register P1 as a boolean value. Store the
88357 ** boolean complement in register P2. If the value in register P1 is
88358 ** NULL, then a NULL is stored in P2.
88376 ** a NULL then store a NULL in P2.
88401 ** For subprograms, there is a bitmask in the VdbeFrame that determines
88429 ** Jump to P2 if the value in register P1 is true. The value
88431 ** in P1 is NULL then take the jump if and only if P3 is non-zero.
88443 ** Jump to P2 if the value in register P1 is False. The value
88445 ** in P1 is NULL then take the jump if and only if P3 is non-zero.
88458 ** Jump to P2 if the value in register P1 is NULL.
88472 ** Jump to P2 if the value in register P1 is not NULL.
88505 ** Store in register r[P3] the byte offset into the database file that is the
88538 ** values in the record, extract a NULL.
88540 ** The value extracted is stored in register P3.
88643 ** (ex: pC->nHdrParsed<=p2) in the next section, we achieve a
88663 ** parsed and valid information is in aOffset[] and pC->aType[].
88666 /* If there is more header available for parsing in the record, try
88680 /* Fill in pC->aType[i] and aOffset[i] values through the p2-th field. */
88826 ** memory cell in the range.
88842 ** that will fit in 6 bytes, then change the type to MEM_IntReal
88870 ** use as a data record in a database table or as a key
88871 ** in an index. The OP_Column opcode can decode the record later.
88878 ** macros defined in sqliteInt.h.
88899 int nVarint; /* Number of bytes in a varint */
88903 int nField; /* Number of fields in the record */
88976 ** be used for that term in the generated record:
89006 ** computing an unchanging column value in an UPDATE statement.
89095 ** which determines the total number of bytes in the header. The varint
89096 ** value is the size of the header in bytes including the size varint
89149 /* EVIDENCE-OF: R-64536-51728 The values for each column in the record
89164 ** Store the number of entries (an integer value) in the table or index
89165 ** opened by cursor P1 in register P2.
89224 sqlite3VdbeError(p, "cannot open savepoint - SQL statements in progress");
89283 "SQL statements in progress");
89349 ** constraint violations present in the database to the value stored
89405 "SQL statements in progress");
89456 ** connection is currently not in autocommit mode, or if there are other
89467 ** cookie in P3 differs from the schema cookie in the database header or
89468 ** if the schema generation counter in P4 differs from the current
89537 ** stored with the in-memory representation of the schema, do
89540 ** If virtual-tables are in use, this is not just an optimization.
89541 ** Often, v-tables store their data in other SQLite tables, which
89603 ** from the database schema version, resulting in a schema reset.
89623 /* Record changes in the file format */
89640 ** P2 in a database file. The database file is determined by P3.
89690 ** page is P2 (or whose root page is held in register P2 if the
89691 ** OPFLAG_P2ISREG bit is set in P5 - see below).
89707 ** and subsequently delete entries in an index btree. This is a
89716 ** in read/write mode.
89872 ** P2 is the number of columns in the ephemeral table.
89875 ** that defines the format of keys in the index.
89878 ** in btree.h. These flags control aspects of the operation of
89892 ** indices in joins.
90014 ** Synopsis: P3 columns in r[P2]
90019 ** MEM_Blob content contained in register P2.
90026 ** P3 is the number of fields in the records that will be stored by
90066 ** (P4_INT64) in which the first 63 bits are one for each of the
90084 ** use the value in register P3 as the key. If cursor P1 refers
90085 ** to an SQL index, then P3 is the first in an array of P4 registers
90101 ** This opcode leaves the cursor configured to move in forward order,
90111 ** use the value in register P3 as a key. If cursor P1 refers
90112 ** to an SQL index, then P3 is the first in an array of P4 registers
90119 ** This opcode leaves the cursor configured to move in forward order,
90129 ** use the value in register P3 as a key. If cursor P1 refers
90130 ** to an SQL index, then P3 is the first in an array of P4 registers
90137 ** This opcode leaves the cursor configured to move in reverse order,
90147 ** use the value in register P3 as a key. If cursor P1 refers
90148 ** to an SQL index, then P3 is the first in an array of P4 registers
90155 ** This opcode leaves the cursor configured to move in reverse order,
90178 int nField; /* Number of columns or fields in the key */
90180 int eqOnly; /* Only interested in == results */
90207 /* The input value in P3 might be of any type: integer, real, string,
90398 ** is earlier in the btree than the target row, then fall through
90406 ** row does not exist in the btree) then jump to SeekOP.P2.
90510 ** The seekHit integer represents the maximum of terms in an index for which
90512 ** than the total number of equality terms in an index lookup, then the
90554 ** is a prefix of any entry in P1 then a jump is made to P2 and
90557 ** This operation leaves the cursor in a state where it can be
90558 ** advanced in the forward direction. The Next instruction will work,
90571 ** is not the prefix of any entry in P1 then a jump is made to P2. If P1
90576 ** This operation leaves the cursor in a state where it cannot be
90577 ** advanced in either direction. In other words, the Next and Prev
90594 ** A value of N in the seekHit flag of cursor P1 means that there exists
90595 ** a key P3:N that will match some record in the index. We want to know
90596 ** if it is possible for a record P3:P4 to match some record in the
90601 ** This opcode is used in IN clause processing for a multi-column key.
90622 ** record are not-NULL then a check is done to determine if any row in the
90630 ** This operation leaves the cursor in a state where it cannot be
90631 ** advanced in either direction. In other words, the Next and Prev
90746 ** This opcode leaves the cursor in a state where it cannot be advanced
90747 ** in either direction. In other words, the Next and Prev opcodes will
90763 ** P3 register to contain a non-integer value, in which case the jump is
90769 ** This opcode leaves the cursor in a state where it cannot be advanced
90770 ** in either direction. In other words, the Next and Prev opcodes will
90858 ** The record number is not previously used as a key in the database
90862 ** If P3>0 then P3 is a register in the root frame of this VDBE that holds
90890 ** thing) is obtained in a two-step algorithm.
90898 ** it already exists in the table. If it does not exist, we have
90967 assert( pOp->p3==0 ); /* We cannot be in random rowid mode if this is
90996 ** entry is overwritten. The data is the value MEM_Blob stored in register
90997 ** number P2. The key is stored in register P3. The key must
91073 /* Prevent post-update hook from running in cases when it should not */
91116 ** tables, register P3 contains the rowid to use with the new record in
91144 ** record in the table. If it is left pointing at the next record, then
91145 ** the next Next instruction will be a no-op. As a result, in this case
91148 ** left in an undefined state.
91164 ** have been positioned using OP_NotFound prior to invoking this opcode in
91167 ** P4 is not NULL, and the OPFLAG_NCHANGE flag is set in P2.
91169 ** If the OPFLAG_ISUPDATE flag is set in P2, then P3 contains the address
91289 ** record blob in register P3 against a prefix of the entry that
91293 ** If either P3 or the sorter contains a NULL in one of their significant
91351 ** it is found in the database file.
91363 ** position in order that they can write to the same table. If P3==0
91368 ** in OP_Result and any OP_Result will invalidate the P2 register content.
91392 ** would fail. Should this ever change (because of changes in the code
91415 ** Store in register P2 an integer which is the key of the table entry that
91500 ** will refer to the last entry in the database table or index.
91505 ** This opcode leaves the cursor configured to move in reverse order,
91546 ** Estimate the number of rows in the table P1. Jump to P2 if that
91606 ** will refer to the first entry in the database table or index.
91611 ** This opcode leaves the cursor configured to move in forward order,
91649 ** Advance cursor P1 so that it points to the next key/data pair in its
91670 ** number P5-1 in the prepared statement is incremented.
91676 ** Back up cursor P1 so that it points to the previous key/data pair in its
91698 ** number P5-1 in the prepared statement is incremented.
91762 ** If P4 is not zero, then it is the number of values in the unpacked
91898 ** one entry for each column in the P3 table. If array entry a(i)
91901 ** from P1. This information is stored in P3 and used to redirect
91908 ** Write into register P2 an integer which is the last entry in the record at
92108 ** Delete an entire database table or index whose root page in the database
92111 ** The table being destroyed is in the main database file if P3==0. If
92112 ** P3==1 then the table to be clear is in the auxiliary database file
92116 ** might be moved into the newly deleted root page in order to keep all
92119 ** is stored in register P2. If no page movement was required (because the
92120 ** table being dropped was already the last one in the database) then a
92121 ** zero is stored in register P2. If AUTOVACUUM is disabled then a zero
92122 ** is stored in register P2.
92126 ** updating existing cursors when a root page is moved in an AUTOVACUUM
92128 ** db in order to avoid introducing an incompatibility between autovacuum
92169 ** in the database file is given by P1. But, unlike Destroy, do not
92172 ** The table being clear is in the main database file if P2==0. If
92173 ** P2==1 then the table to be clear is in the auxiliary database file
92178 ** count is incremented by the number of rows in the table being cleared.
92179 ** If P3 is greater than zero, then the value stored in register P3 is
92180 ** also incremented by the number of rows in the table being cleared.
92234 ** Allocate a new b-tree in the main database file if P1==0 or in the
92235 ** TEMP database file if P1==1 or in an attached database if
92238 ** The root page number of the new b-tree is stored in register P2.
92261 ** Run the SQL statement or statements specified in the P4 string.
92367 ** Remove the internal (in-memory) data structures that describe
92368 ** the table named P4 in database P1. This is called after a table
92369 ** is dropped from disk (using the Destroy opcode) in order to keep
92381 ** Remove the internal (in-memory) data structures that describe
92382 ** the index named P4 in database P1. This is called after an index
92384 ** in order to keep the internal representation of the
92395 ** Remove the internal (in-memory) data structures that describe
92396 ** the trigger named P4 in database P1. This is called after a trigger
92397 ** is dropped from disk (using the Destroy opcode) in order to keep
92411 ** Do an analysis of the currently open database. Store in
92413 ** If no problems are found, store a NULL in register P1.
92420 ** The root page numbers of all tables in the database are integers
92421 ** stored in P4_INTARRAY argument.
92468 ** held in register P1.
92487 ** Extract the smallest value from the RowSet object in P1
92513 ** Synopsis: if r[P3] in rowset(P1) goto P2
92517 ** the value held in P3, jump to register P2. Otherwise, insert the
92518 ** integer in P3 into the RowSet and continue on to the
92522 ** are inserted in distinct phases, which each set contains no duplicates.
92544 /* If there is anything other than a rowset object in memory cell P1,
92572 ** cell in an array of values used as arguments to the sub-program. P2
92575 ** of a memory cell in this (the parent) VM that is used to allocate the
92587 Mem *pEnd; /* Last memory cell in new array */
92588 VdbeFrame *pFrame; /* New vdbe frame to execute in */
92625 ** program stored in SubProgram.aOp. As well as these, one memory
92716 ** This opcode is only ever present in sub-programs called via the
92717 ** OP_Program instruction. Copy a value currently stored in a memory
92718 ** cell of the calling (parent) frame to cell P2 in the current frames
92722 ** The address of the cell in the parent frame is determined by adding
92786 ** P1 is a register in the root frame of this VM (the root frame is
92789 ** its current value and the value in register P2.
92818 ** value in P1 and jump to P2.
92840 ** of the LIMIT and OFFSET and stores that value in r[P2]. The r[P2]
92842 ** visited in order to complete the query.
92879 ** initially greater than zero, then decrement the value in register P1.
92897 ** Register P1 must hold an integer. Decrement the value in P1
92944 ** the FuncDef stored in P4 is converted into an sqlite3_context and
93001 /* If this function is inside of a trigger, the register array in aMem[]
93054 ** for an aggregate and store the result in P1.
93066 ** Invoke the xValue() function and store the result in register P3.
93108 ** Checkpoint database P1. This is a no-op if P1 is not currently in
93111 ** SQLITE_BUSY or not, respectively. Write the number of pages in the
93113 ** in the WAL that have been checkpointed after the checkpoint
93187 ** in temporary storage or if the VFS does not support shared memory
93355 ** P1 is the index of the database in sqlite3.aDb[] of the database
93407 ** P2 is a register that holds the name of a virtual table in database
93436 ** P4 is the name of a virtual table in database P1. Call the xDestroy method
93454 ** table and stores that cursor in P1.
93504 ** by P1. The integer query plan parameter to xFilter is stored in register
93561 ** Store in register P3 the value of the P2-th column of
93621 ** Advance virtual table P1 to the next row in its result set and
93665 ** in register P1 is passed as the zName argument to the xRename method.
93702 ** invocation. The value in register (P3+P2-1) corresponds to the
93710 ** rowid for itself. The subsequent elements in the array are
93711 ** the values of columns in the new row.
93721 ** apply in the case of a constraint failure on an insert or update.
93782 ** Write the current number of pages in database P1 to memory cell P2.
93795 ** Try to set the maximum page count for database P1 to the value in P3.
93799 ** Store the maximum page count after the change in register P2.
93822 ** from register P2 and successors. The number of arguments is in
93825 ** in register P3. Register P3 must not be one of the function inputs.
93841 ** from register P2 and successors. The number of arguments is in
93844 ** in register P3. Register P3 must not be one of the function inputs.
93853 ** This opcode works exactly like OP_Function. The only difference is in
93854 ** its name. This opcode is used in places where the function must be
93855 ** purely non-deterministic. Some built-in date/time functions can be
93857 ** When those function are used in a non-deterministic way, they will check
93871 /* If this function is inside of a trigger, the register array in aMem[]
93930 ** the UTF-8 string contained in P4 is emitted on the trace callback.
94017 ** satisfy the Expr in P4. TK_REGISTER terms in the P4 expression refer
94018 ** to values currently held in registers. TK_COLUMN terms in the P4
94019 ** expression refer to columns in the b-tree to which cursor P1 is pointing.
94040 ** might cause database corruption. This opcode only appears in debugging
94056 ** Release registers from service. Any content that was in the
94066 ** generate an assertion fault in sqlite3VdbeMemAboutToChange().
94073 ** However, there are places in the code generator will release registers
94078 ** This opcode is only available in testing and debugging builds. It is
94110 ** the same as a no-op. This opcodesnever appears in a real VM program.
94262 int nByte; /* Size of open blob, in bytes */
94263 int iOffset; /* Byte offset of blob in cursor data */
94280 ** contain a value of type TEXT or BLOB in the column nominated when the
94295 /* Set the value of register r[1] in the SQL statement to integer iRow.
94365 int iCol; /* Index of zColumn in row-record */
94546 ** we can invoke OP_Column to fill in the vdbe cursors type
94631 ** already been invalidated. Return SQLITE_ABORT in this case.
94650 ** SQLITE_UPDATE where the PK columns do not change is handled in the
94725 ** already been invalidated. Return SQLITE_ABORT in this case.
94759 ** This file contains code for the VdbeSorter object, used in concert with
94774 ** object. The row is a binary blob in the
94777 ** in the case of a SELECT w/ ORDER BY, or
94779 ** in the case of a CREATE INDEX.
94795 ** Used to enforce uniqueness in a
94805 ** The interfaces above must be called in a particular order. Write() can
94806 ** only occur in between Init()/Reset() and Rewind(). Next(), Rowkey(), and
94807 ** Compare() can only occur in between Rewind() and Close()/Reset(). i.e.
94819 ** unsorted in main memory. Assuming the amount of memory used never exceeds
94821 ** an in-memory merge sort. In this case, no temporary files are required
94825 ** If the amount of space used to store records in main memory exceeds the
94826 ** threshold, then the set of records currently in memory are sorted and
94827 ** written to a temporary file in "Packed Memory Array" (PMA) format.
94836 ** that returned by "PRAGMA main.cache_size", in bytes.
94838 ** If the sorter is running in single-threaded mode, then all PMAs generated
94839 ** are appended to a single temporary file. Or, if the sorter is running in
94847 ** The sorter is running in multi-threaded mode if (a) the library was built
94852 ** When Rewind() is called, any data remaining in memory is flushed to a
94853 ** final PMA. So at this point the data is stored in some number of sorted
94856 ** If there are fewer than SORTER_MAX_MERGE_COUNT PMAs in total and the
94857 ** sorter is running in single-threaded mode, then these PMAs are merged
94859 ** MergeEngine object, described in further detail below, performs this
94862 ** Or, if running in multi-threaded mode, then a background thread is
94871 ** If there are more than SORTER_MAX_MERGE_COUNT PMAs in total when
94878 ** If running in multi-threaded mode and there are more than
94890 ** messages to stderr that may be helpful in understanding the performance
94891 ** characteristics of the sorter in multi-threaded mode.
94898 ** Hard-coded maximum amount of data to accumulate in memory before flushing
94911 typedef struct SortSubtask SortSubtask; /* A sub-task in the sort process */
94918 ** stored in the file.
94922 i64 iEof; /* Bytes of data stored in pFd */
94926 ** An in-memory list of objects to be sorted.
94930 ** are stored in the aMemory[] bulk memory, one right after the other, and
94936 int szPMA; /* Size of pList as PMA in bytes */
94942 ** combined into one big PMA in order to be able to step through the sorted
94943 ** records in order.
94949 ** actually N elements in size, where N is the smallest power of 2 greater
94953 ** The aTree[] array is also N elements in size. The value of N is stored in
94985 ** be advanced to the next key in its segment. Say the next key is
95011 ** This object represents a single thread of control in a sort operation.
95021 ** each thread requries its own UnpackedRecord object to unpack records in
95048 int nPMA; /* Number of PMAs currently in file */
95062 ** largest record in the sorter.
95065 int mnPmaSize; /* Minimum PMA size, in bytes */
95066 int mxPmaSize; /* Maximum PMA size, in bytes. 0==no limit */
95074 SorterList list; /* List of in-memory records */
95075 int iMemory; /* Offset of free space in list.aMemory */
95076 int nMemory; /* Size of list.aMemory allocation in bytes */
95090 ** PMA, in sorted order. The next key to be read is cached in nKey/aKey.
95101 int nKey; /* Number of bytes in key */
95106 int nBuffer; /* Size of read buffer in bytes */
95121 ** and aFile[1]. Neither file is allowed to grow to more than mxSz bytes in
95157 ** size. But I/O is more efficient if it occurs in page-sized blocks where
95162 int eFWErr; /* Non-zero if in an error state */
95164 int nBuffer; /* Size of write buffer in bytes */
95167 i64 iWriteOff; /* Offset of start of buffer in file */
95173 ** held in memory and prior to being written out as part of a PMA.
95176 ** by this module. If using a separate allocation for each in-memory record
95184 ** has finished passing records to the sorter, or when the in-memory buffer
95190 int nVal; /* Size of the record in bytes */
95192 SorterRecord *pNext; /* Pointer to next record in list */
95230 ** The buffer returned in *ppOut is only valid until the
95239 int nAvail; /* Bytes of data available in buffer */
95251 ** than p->nBuffer bytes remaining in the PMA, read all remaining data. */
95273 /* The requested data is available in the in-memory buffer. In this
95279 /* The requested data is not all available in the in-memory buffer.
95295 /* Copy as much data as is available in the buffer into the start of
95358 ** *pp is undefined in this case.
95381 i64 iOff /* Offset in pFile */
95421 ** Advance PmaReader pReadr to the next key in its PMA. Return SQLITE_OK if
95426 u64 nRec = 0; /* Size of record in bytes */
95463 ** Initialize PmaReader pReadr to scan through the PMA stored in file pFile
95465 ** leaves the PmaReader pointing to the first key in the PMA (or EOF if the
95474 i64 iStart, /* Start offset in pFile */
95487 u64 nByte = 0; /* Size of PMA in bytes */
95664 ** statement. In this case, keys are always delivered to the sorter in
95670 ** The sorter can guarantee a stable sort when running in single-threaded
95671 ** mode, but not in multi-threaded mode.
95677 int nField, /* Number of key fields in each record */
95684 int szKeyInfo; /* Size of pCsr->pKeyInfo in bytes */
95685 int sz; /* Size of pSorter in bytes */
95740 i64 mxCache; /* Cache size in bytes*/
95747 ** KiB in size. */
95881 void *(*xTask)(void*), /* Routine to run in a separate thread */
96024 ** is guaranteed to be nByte bytes or smaller in size. This function
96025 ** attempts to extend the file to nByte bytes in size and to ensure that
96280 ** Write the current contents of in-memory linked-list pList to a level-0
96281 ** PMA in the temp file belonging to sub-task pTask. Return SQLITE_OK if
96287 ** in the PMA (not including the varint itself).
96289 ** * One or more records packed end-to-end in order of ascending keys.
96291 ** key). The varint is the number of bytes in the blob of data.
96376 ** advanced (iPrev) and the one next to it in the array. */
96381 /* Compare pReadr1 and pReadr2. Store the result in variable iRes. */
96395 ** case there is no cache of pReadr2 in pTask->pUnpacked, so set
96456 /* Select a sub-task to sort and flush the current list of in-memory
96457 ** records to disk. If the sorter is running in multi-threaded mode,
96542 ** already in memory and (b) the new value will not fit in memory.
96547 ** * The total memory allocated for the in-memory list is greater
96550 ** * The total memory allocated for the in-memory list is greater
96621 ** of the data stored in aFile[1] is the same as that used by regular PMAs,
96694 ** aFile[0] and aFile[1] in place. If the contents of pMerger have not
96778 int iOut /* Store the result in pMerger->aTree[iOut] */
96825 ** Only INCRINIT_NORMAL is valid in single-threaded builds (when
96843 ** MergeEngine object in the usual fashion.
96871 /* eMode is always INCRINIT_NORMAL in single-threaded mode */
96881 /* PmaReaders should be normally initialized in order, as if they are
96883 ** However, in the INCRINIT_ROOT case, if PmaReader aReadr[nTask-1] is
96884 ** in use it will block the vdbePmaReaderNext() call while it uses
96908 ** in the sub-tree headed by pReadr are also initialized. Data is then
96910 ** the first key in its range.
96913 ** to be a multi-threaded PmaReader and this function is being called in a
96914 ** background thread. In this case all PmaReaders in the sub-tree are
96921 ** in the INCRINIT_TASK case is that vdbePmaReaderNext() assumes that it has
96930 ** the current PmaReader set to point to the first key in its range.
96940 /* eMode is always INCRINIT_NORMAL in single-threaded mode */
96976 ** then this function is already running in background thread
96979 ** If this is the INCRINIT_ROOT object, then it is running in the
96998 ** The main routine for vdbePmaReaderIncrMergeInit() operations run in
97055 i64 *piOffset, /* IN/OUT: Readr offset in pTask->file */
97240 ** all records stored in the sorter.
97288 ** in single-threaded mode. This is important, as the
97324 ** in sorted order.
97336 ** from the in-memory list. */
97347 /* Write the current in-memory list to a PMA. When the VdbeSorterWrite()
97372 ** Advance to the next element in the sorter. Return value:
97418 int *pnKey /* OUT: Size of current key in bytes */
97461 ** Compare the key in memory cell pVal with the key that the sorter cursor
97470 ** key in pVal is smaller than, equal to or larger than the current sorter
97473 ** This routine forms the core of the OP_SorterCompare opcode, which in
97953 ** This file contains code use to implement an in-memory rollback journal.
97954 ** The in-memory rollback journal is used to journal transactions for
97957 ** Update: The in-memory journal is also used to temporarily cache
97960 ** entirely in memory, thus reducing the number of file I/O calls, and
97963 ** in the common case, they are usually small and no file I/O needs to
97976 ** The zChunk array is always at least 8 bytes in size - usually much more.
97977 ** Its actual size is stored in the MemJournal.nChunkSize variable.
97980 FileChunk *pNext; /* Next chunk in the journal */
98013 FileChunk *pFirst; /* Head of in-memory chunk-list */
98023 ** Read data from the in-memory journal file. This is the implementation
98105 /* No error has occurred. Free the in-memory buffers. */
98111 ** the original before returning. This way, SQLite uses the in-memory
98144 /* If the contents of this write should be stored in memory */
98146 /* An in-memory journal file should only ever be appended to. Random
98148 ** the in-memory journal is being used by a connection using the
98196 ** Truncate the in-memory file.
98236 ** syncing an in-memory journal is a no-op.
98244 ** Query the size of the file in bytes.
98283 ** all content is always stored in main-memory. Finally, if nSpill is a
98284 ** positive value, then the journal file is initially created in-memory
98286 ** flushed to disk either when it grows larger than nSpill bytes in size,
98300 ** case none of the code in this module is executed as a result of calls
98323 ** Open an in-memory journal file.
98333 ** in-memory-only journal file (i.e. is one that was opened with a +ve
98345 ** paths are complex, so it seems prudent to leave the test in as
98346 ** a NEVER(), in case our analysis is subtly flawed. */
98361 ** Return true if this "journal file" is currently stored in heap memory,
98483 ** Call sqlite3WalkExpr() for every expression in list p or until
98515 ** symbols (e.g. a table that does not exist) in a window definition. */
98525 ** Walk the parse trees associated with all subqueries in the
98528 ** and on any subqueries further down in the tree. Return
98553 ** Call sqlite3WalkExpr() for every expression in Select statement p.
98554 ** Invoke sqlite3WalkSelect() for subqueries in the FROM clause and
98610 ** subquery in the parser tree.
98619 ** subquery in the parser tree.
98647 ** Magic table number to mean the EXCLUDED table in an UPSERT statement.
98660 ** See also the sqlite3WindowExtraAggFuncDepth() routine in window.c
98678 ** result set in pEList.
98698 int iCol, /* A column in the result set. 0..pEList->nExpr-1 */
98745 ** Return TRUE if the name zCol occurs anywhere in the USING clause.
98762 ** result sets in ExprList.a[].zSpan, in the form "DATABASE.TABLE.COLUMN".
98837 ** that name in the set of source tables in pSrcList and make the pExpr
98841 ** pExpr->iDb Set the index in db->aDb[] of the database X
98860 ** in pParse and return WRC_Abort. Return WRC_Prune on success.
98877 NameContext *pTopNC = pNC; /* First namecontext in the list */
98884 assert( zCol ); /* The Z in X.Y.Z cannot be NULL */
98891 /* Translate the schema name in zDb into a pointer to the corresponding
98893 ** resulting in an appropriate error message toward the end of this routine
98967 ** is for the right-hand table of a NATURAL JOIN or is in a
98998 ** a reference in the RETURNING clause to a table being modified.
99113 ** we are resolving names in the WHERE clause of the following command:
99118 ** forms the result set entry ("a+b" in the example) and return immediately.
99119 ** Note that the expression in the result set should have already been
99122 ** The ability to use an output result-set column in the WHERE, GROUP BY,
99123 ** or HAVING clauses, or as part of a larger expression in the ORDER BY
99180 ** If X and Y are NULL (in other words if only the column name Z is
99181 ** supplied) and the value of Z is enclosed in double-quotes, then
99187 ** fields are not changed in any context.
99198 ** This hack was added in the early days of SQLite in a misguided attempt
99200 ** I now sorely regret putting in this hack. The effect of this hack is
99246 /* If a column from a table in pSrcList is referenced, then record
99247 ** this fact in the pSrcList.a[].colUsed bitmask. Column 0 causes
99300 ** from datasource iSrc in SrcList pSrc.
99358 sqlite3ErrorMsg(pParse, "%s prohibited in %s", zMsg, zIn);
99383 ** node in the expression tree. Return 0 to continue the search down
99411 ** column in the FROM clause. This is used by the LIMIT and ORDER BY
99437 ** to the state they where in before the "column" LHS expression was
99474 ** be one call to lookupName(). Then the compiler will in-line
99521 int nId; /* Number of characters in function name */
99590 ** in an index or generated column. Curiously, they can be used
99591 ** in a CHECK constraint. SQLServer, MySQL, and PostgreSQL all
99596 assert( (NC_SelfRef & 0xff)==NC_SelfRef ); /* Must fit in 8 bits */
99822 ** a SELECT statement. pE is a term in an ORDER BY or GROUP BY clause.
99826 ** elements in pEList, corresponding to the matching entry. If there is
99855 ** pE is a pointer to an expression which is a single term in the
99863 ** Attempt to match pE against result set columns in the left-most
99867 ** same integer value that would be used in the SQL statement to indicate
99887 /* Resolve all names in the ORDER BY term expression
99903 ** in the result set. Return an 1-based index of the matching
99931 ** Analyze the ORDER BY clause in a compound SELECT statement. Modify
99933 ** and N where N is the number of columns in the compound SELECT.
99959 sqlite3ErrorMsg(pParse, "too many terms in ORDER BY clause");
99991 ** expression, resolving any symbols in it, and then comparing
99997 ** resolve the symbols in the actual expression, not a duplicate.
99999 ** as is instead of transforming it to an integer as in the usual
100000 ** case. This allows the code in alter.c to modify column
100046 "column in the result set", i+1);
100054 ** Check every term in the ORDER BY or GROUP BY clause pOrderBy of
100076 sqlite3ErrorMsg(pParse, "too many terms in %s BY clause", zType);
100124 ** pOrderBy is an ORDER BY or GROUP BY clause in SELECT statement pSelect.
100130 ** number of columns in the result set of the SELECT) then the expression
100131 ** in the resolution is a copy of the I-th result-set expression. If
100134 ** result-set expression. Otherwise, the expression is resolved in
100138 ** an appropriate error message might be left in pParse. (OOM errors
100151 int nResult; /* Number of terms in the result set */
100191 ** to an identical expression in the result set, remove all Window
100202 ** Resolve names in the SELECT statement p and all of its descendants.
100230 ** this routine in the correct order.
100245 /* Resolve the expressions in the LIMIT and OFFSET clauses. These
100269 /* Recursively resolve names in all subqueries
100283 ** expressions in the sub-select were resolved, the sub-select
100302 /* Resolve names in the result set. */
100306 /* If there are no aggregate functions in the result-set, and no GROUP BY
100307 ** expression, do not allow aggregates in any of the other expressions.
100326 ** other expressions in the SELECT statement. This is so that
100327 ** expressions in the WHERE clause (etc.) can refer to expressions by
100328 ** aliases in the result set.
100339 /* Resolve names in table-valued-function arguments */
100349 /* The ORDER BY and GROUP BY clauses may not refer to terms in
100397 sqlite3ErrorMsg(pParse, "aggregate functions are not allowed in "
100418 ** number of expressions in the select list. */
100452 ** Y: The name of a table in a FROM clause. Or in a trigger
100455 ** Z: The name of a column in table Y.
100461 ** Expr.iColumn The column index in X.Y. -1 for the rowid.
100467 ** size of an AS clause in the result-set of a SELECT. The Z expression
100470 ** tree. For example, in:
100485 ** An error message is left in pParse if anything is amiss. The number
100489 NameContext *pNC, /* Namespace to resolve expressions in. */
100523 ** Resolve all names for all expression in an expression list. This is
100528 NameContext *pNC, /* Namespace to resolve expressions in. */
100571 ** Resolve all names in all expressions of a SELECT and in all
100573 ** subqueries in expressions, and subqueries used as FROM clause
100599 ** Resolve names in expressions that can only reference a single table
100606 ** (3) Expressions in indexes on expressions NC_IdxExpr
100614 ** Any errors cause an error message to be set in pParse.
100665 ** for generating VDBE code that evaluates expressions in SQLite.
100689 ** i.e. the WHERE clause expressions in the following statements all
100741 ** If a memory allocation error occurs, that fact is recorded in pParse->db
100834 ** a TK_COLUMN but was previously evaluated and cached in a register */
100959 ** the comparison in pExpr.
101023 ** is reversed in the sqlite3BinaryCompareCollSeq() call so that the
101067 ** A vector is defined as any expression that results in two or more
101079 ** return the number of expressions in the vector. Or, if the expression
101080 ** is a sub-select, return the number of columns in the sub-select. For
101156 ** iColumn: Index of a column in pVector
101158 ** pLeft->iTable: First in an array of register holding result, or 0
101184 ** it. Return the register in which the result is stored (or, if the
101185 ** sub-select returns more than one column, the first in an array
101186 ** of registers in which the result is stored).
101208 ** case parameter regSelect should be the first in an array of registers
101222 int regSelect, /* First in array of registers */
101333 ** expression depth allowed. If it is not, leave an error message in
101385 ** Set the Expr.nHeight variable in the structure passed as an
101410 ** leave an error in pParse.
101460 ** stored in u.zToken. Instead, the integer values is written
101672 /* Functions prohibited in triggers and views if:
101686 ** in the original SQL statement.
101852 ** to store a copy of an expression or expression tree. They differ in
101911 ** This function returns the space in bytes required to store the copy
101932 ** space to duplicate all Expr nodes in the tree formed by Expr.pLeft
102016 /* Fill in the pNew->x.pSelect or pNew->x.pList member. */
102024 /* Fill in pNew->pLeft and pNew->pRight. */
102133 ** part of the in-memory representation of the database schema.
102313 ** Reason: This routine assumes that the number of slots in pList->a[]
102386 ** wildcards ("*") in the result set of the SELECT must be expanded before
102414 /* Store the SELECT statement in pRight so it will be deleted when
102419 /* Remember the size of the LHS in iTable so that we can check that
102526 ** leave an error message in pParse.
102537 sqlite3ErrorMsg(pParse, "too many columns in %s", zObject);
102560 ** Return the bitwise-OR of all Expr.flags fields in the given
102577 ** always "fails". By "fail" in this case, we mean set
102589 ** Check the input string to see if it is "true" or "false" (in any case).
102678 ** expressions in a CREATE TABLE statement. The Walker.eCode value is 5
102712 /* Convert "true" or "false" in a DEFAULT clause into the
102747 /* A bound parameter in a CREATE statement that originates from
102788 ** (2) the expression does originate in the ON or USING clause
102843 ** in pGroupBy that sort with the BINARY collation sequence.
102851 ** A=B in every other collating sequence. The requirement that the
102872 ** in a CREATE TABLE statement. Return non-zero if the expression is
102879 ** (such as ? or $abc) in the expression are converted into NULL. When
102881 ** allowed in a CREATE TABLE statement, but some legacy versions of SQLite
102916 ** to fit in a 32-bit integer, return 1 and put the value of the integer
102917 ** in *pValue. If the expression is not an integer or if it is too big
102918 ** to fit in a signed 32-bit integer, return 0 and leave *pValue unchanged.
102924 /* If an expression is an integer literal that fits in a signed 32-bit
102960 ** (returning TRUE when in fact the expression can never be NULL) might
102963 ** will likely result in an incorrect answer. So when in doubt, return
102992 ** unchanged by OP_Affinity with the affinity given in the second
102996 ** can be omitted. When in doubt return FALSE. A false negative
102997 ** is harmless. A false positive, however, can result in the wrong
103071 if( pSrc->nSrc!=1 ) return 0; /* Single term in FROM clause */
103131 ** be used either to test for membership in the RHS set or to iterate through
103189 ** If a register is allocated and its location stored in *prRhsHasNull, then
103190 ** the value in that register will be NULL if the b-tree contains one or more
103226 ** to NOT NULL constraints in the schema). If no NULL values are possible,
103277 ** comparison is the same as the affinity of each column in table
103470 ** Expression pExpr is a vector that has been used in a context where
103494 ** in the RHS of an IN operator. The IN operator can be in either of two
103502 ** table is computed, the cursor number is also stored in pExpr->iTable,
103506 ** If the LHS expression ("x" in the examples) is a column value, or
103607 testcase( pKeyInfo==0 ); /* Caused by OOM in sqlite3KeyInfoAlloc() */
103629 ** store it in the temporary table. If <expr> is a column, then use
103649 /* Loop through each expression in <exprlist>. */
103697 ** the result is stored in a contiguous array of registers and the
103756 ** preexisting limit is discarded in place of the new LIMIT 1.
103842 ** match the number of columns in the vector on the LHS. If the RHS is
103846 ** The result is false if the LHS is definitely not in the RHS. The
103847 ** result is NULL if the presence of the LHS in the RHS cannot be
103852 ** is contained in the RHS then jump to destIfNull. If the LHS is contained
103855 ** See the separate in-operator.md documentation file in the canonical
103861 int destIfFalse, /* Jump here if LHS is not contained in the RHS */
103875 int destStep2; /* Where to jump when NULLs seen in step 2 */
103878 int destNotNull; /* Jump here if a comparison is not true in step 6 */
103919 ** vector, then it is stored in an array of nVector registers starting
103923 ** so that the fields are in the same order as an existing index. The
103950 ** This is step (1) in the in-operator.md optimized algorithm.
104093 /* Jumps here in order to return true. */
104174 int regOut /* Store the index column value in this register */
104192 ** and store the result in register regOut
104271 ** table pTab and store the column value in register iReg.
104273 ** There must be an open cursor to pTab in iTable when this routine
104317 ** the result in continguous temporary registers. Return the index of
104362 ** in-line rather than by using the usual callbacks.
104368 int target /* Store function result in this register */
104375 assert( nFarg>0 ); /* All in-line functions have at least one argument */
104378 /* Attempt a direct implementation of the built-in COALESCE() and
104471 ** expression. Attempt to store the results in register "target".
104475 ** be stored in target. The result might be stored in some other
104483 int inReg = target; /* Results stored in register inReg */
104555 /* Other columns in the same row for CHECK constraints or
104558 ** 0-(pParse->iSelfTab). The rowid (if any) is in a register
104598 ** in the index refer to the table to which the index belongs */
104912 ** see if it is a column in a virtual table. This is done because
104916 ** "glob(B,A). We want to use the A in "A glob B" to test
104917 ** for function overloading. But we use the B term in "glob(B,A)".
105002 ** X is stored in pExpr->pLeft.
105003 ** Y is stored in pExpr->pList->a[0].pExpr.
105004 ** Z is stored in pExpr->pList->a[1].pExpr.
105019 ** to a column in the new.* or old.* pseudo-tables available to
105029 ** set to (n+1), where n is the number of columns in each pseudo-table.
105030 ** For a reference to any other column in the new.* pseudo-table, p1
105110 ** X (if it exists) is in pExpr->pLeft.
105111 ** Y is in the last element of pExpr->x.pList if pExpr->x.pList->nExpr is
105112 ** odd. The Y is also optional. If the number of elements in x.pList
105114 ** Ei is in pExpr->pList->a[i*2] and Ri is pExpr->pList->a[i*2+1].
105153 ** The value in regFree1 might get SCopy-ed into the file result.
105225 ** the end of the prepared statement in the initialization section.
105227 ** If regDest>=0 then the result is always stored in that register and the
105238 int regDest /* Store the value in this register */
105289 ** code to fill the register in the initialization section of the
105290 ** VDBE program, in order to factor it out of the evaluation loop.
105317 ** results in register target. The results are guaranteed to appear
105318 ** in register target.
105353 ** results in register target. The results are guaranteed to appear
105354 ** in register target. If the expression is constant, then this routine
105379 ** The SQLITE_ECEL_REF flag means that expressions in the list with
105381 ** in registers at srcReg, and so the value can be copied from there.
105454 ** NULL: Store the boolean result in reg[dest]
105467 Expr exprAnd; /* The AND operator in x>=y AND x<=z */
105497 ** for clarity, but we are out of bits in the Expr.flags field so we
105528 ** operation. Special comments in vdbe.c and the mkopcodeh.awk script in
105529 ** the make process cause these values to align. Assert()s in the code
105898 ** to compare equal to an equivalent element in pA with Expr.iTable==iTab.
105910 ** just might result in some slightly slower code. But returning
105913 ** If pParse is not NULL then TK_VARIABLE terms in pA with bindings in
105914 ** pParse->pReprepare can be matched against literals in pB. The
105916 ** If pParse is NULL (the normal case) then any TK_VARIABLE term in
105994 ** to compare equal to an equivalent element in pA with Expr.iTable==iTab.
106123 ** If pParse is not NULL, then the values of bound variables in pE1 are
106124 ** compared against literal values in pE2 and pParse->pVdbe->expmask is
106128 ** When in doubt, return false. Returning true might give a performance
106226 /* The y.pTab=0 assignment in wherecode.c always happens after the
106253 ** in an incorrect answer.
106298 ** Check to see if there are references to columns in table
106317 ** that are not found in the index pIdx.
106342 ** to count references to table columns in the arguments of an
106343 ** aggregate function, in order to implement the
106347 SrcList *pSrc; /* One particular FROM clause in a nested query */
106348 int iSrcInner; /* Smallest cursor number in this context */
106349 int nThis; /* Number of references to columns in pSrcList */
106350 int nOther; /* Number of references to columns in other FROM clauses */
106356 ** SrcCount.iSrcInner to the cursor number of the leftmost object in
106389 ** TK_COLUMN nodes with smaller Expr.iTable values are in an
106401 ** references columns but not columns of tables found in pSrcList.
106435 ** will not generate any code in the preamble.
106530 /* Check to see if the column is in one of the tables in the FROM
106539 ** that is in the FROM clause of the aggregate query.
106541 ** Make an entry for the column in pAggInfo->aCol[] if there
106580 /* There is now an entry for pExpr in pAggInfo->aCol[] (either
106600 ** function that is already in the pAggInfo structure
106609 /* pExpr is original. Make a new entry in pAggInfo->aFunc[]
106666 ** Call sqlite3ExprAnalyzeAggregates() for every expression in an
106738 ** the sub/co-routine does not use registers in common with the code that
106788 ** The code in this file only exists if we are not omitting the
106797 ** in pParse->zErr (system tables may not be altered) and returns non-zero.
106879 int nTabName; /* Number of UTF-8 characters in zTabName */
106901 ** in database iDb. If so, this is an error.
106958 /* figure out how many UTF-8 characters are in zName */
106962 /* Rewrite all CREATE TABLE, INDEX, TRIGGER or VIEW statements in
106991 /* If the sqlite_sequence table exists in this database, then update
107161 /* substr() operations on characters, but addColOffset is in bytes. So we
107196 ** This function is called by the parser after the table-name in
107245 /* Put a copy of the Table struct in Parse.pNewTable for the
107357 /* Make sure the old name really is a column name in the table to be
107413 ** * The name of a table column in Column.zName
107421 ** to look up the actual token value that created some element in
107437 int nList; /* Number of tokens in pList */
107447 ** 1. Checks that pointer pPtr does not already appear in the
107450 ** 2. Dereferences each pointer in the rename-token list.
107465 ** following a valid object, it may not be used in comparison operations.
107489 ** in pParse->pRename.
107492 ** with tail recursion in tokenExpr() routine, for a small performance
107558 ** Unmap all tokens in the IdList object passed as the second argument.
107637 ** Free the list of RenameToken objects given in the second argument
107693 ** For every TK_COLUMN node in the expression tree, search to see
107697 ** constructed in RenameCtx object at pWalker->u.pRename.
107718 ** RenameToken in the RenameCtx and remove that RenameToken from the
107721 ** return all column name tokens in the order that they are encountered
107722 ** in the SQL statement.
107742 ** sub-routine is currently stored in pParse->zErrMsg. This function
107743 ** adds context to the error message and then stores it in pCtx.
107756 zErr = sqlite3_mprintf("error in %s %s%s%s: %s",
107765 ** For each name in the the expression-list pEList (i.e. each
107766 ** pEList->a[i].zName) that matches the string in zOld, extract the
107791 ** For each name in the the id-list pIdList (i.e. each pIdList->a[i].zName)
107792 ** that matches the string in zOld, extract the corresponding rename-token
107835 ** occurs and the parse does not result in a new table, index or
107853 /* Ensure that all mappings in the Parse.pRename list really do map to
107875 ** Or, if an error occurs (i.e. an OOM condition), an error is left in
107894 ** identifier zNew. If the corresponding identifier in the original
107910 ** corresponding to all tokens in the input SQL that must be replaced
107919 int iOff; /* Offset of token to replace in zOut */
107955 ** Resolve all symbols in the trigger at pParse->pNewTrigger, assuming
107958 ** message in the Parse object.
107980 /* Resolve symbols in WHEN clause */
108056 /* Find tokens to edit in WHEN clause */
108059 /* Find tokens to edit in trigger steps */
108116 ** Do a column rename operation on the CREATE statement given in zSql.
108253 /* Find tokens to edit in UPDATE OF clause */
108258 /* Find tokens to edit in various expressions and selects */
108407 /* If this is the table being altered, fix any table refs in CHECK
108476 ** resolution problems in a CREATE TRIGGER|TABLE|VIEW|INDEX statement.
108478 ** reloaded, this function is called on each SQL statement in the schema
108493 ** * the table that the trigger is attached to is in database zDb.
108629 int iDb; /* Index of db containing pTab in aDb[] */
108632 int iCol; /* Index of column zCol in pTab->aCol[] */
108753 ** Register built-in functions used to help implement ALTER TABLE
108792 ** Additional tables might be added in future releases of SQLite.
108801 ** available when compiled with SQLITE_ENABLE_STAT4 and in SQLite
108811 ** name in the idx column. The tbl column is the name of the table to
108813 ** a string consisting of a list of integers. The first integer in this
108814 ** list is the number of rows in the index. (This is the same as the
108815 ** number of rows in the table, except for partial indices.) The second
108816 ** integer is the average number of rows in the index that have the same
108817 ** value in the first column of the index. The third integer is the average
108818 ** number of rows in the index that have the same value for the first two
108819 ** columns. The N-th integer (for N>1) is the average number of rows in
108821 ** a K-column index, there will be K+1 integers in the stat column. If
108824 ** The list of integers in the stat column can optionally be followed
108832 ** rows in the table identified by sqlite_stat1.tbl.
108841 ** the index belongs. There are usually 10 rows in the sqlite_stat2
108845 ** inclusive are samples of the left-most key value in the index taken at
108847 ** (10 in the standard build) and let C be the number of rows in the index.
108869 ** to aid the query planner in choosing good indices based on the values
108870 ** that indexed columns are compared against in the WHERE clauses of
108879 ** of entries in the index whose left-most column exactly matches
108880 ** the left-most column of the sample. The second integer in nEq
108881 ** is the approximate number of entries in the index where the
108885 ** integer in nLt contains the number of entries in the index where the
108887 ** The K-th integer in the nLt entry is the number of index entries
108890 ** number of distinct entries in the index that are less than the
108904 ** of a blob encoding of the complete index key as is found in
108907 ** integer in the equivalent columns in sqlite_stat4.
108929 ** or it may be a NULL pointer. If it is not NULL, then all entries in
108935 int iDb, /* The database we are looking in */
108980 ** of the new table in register pParse->regRoot. This is important
109039 i64 iRowid; /* Rowid in main table of the key */
109053 int nCol; /* Number of columns in index + pk/rowid */
109062 int iMin; /* Index in a[] of entry with minimum score */
109064 int nMaxEqZero; /* Max leading 0 in anEq[] for any a[] entry */
109148 ** N: The number of columns in the index including the rowid/pk (note 1)
109149 ** K: The number of columns in the index excluding the rowid/pk.
109150 ** C: Estimated number of rows in the index
109155 ** total number of columns in the table.
109158 ** WITHOUT ROWID tables, N=K+P where P is the number of columns in the
109162 ** This routine allocates the StatAccum object in heap memory. The return
109173 int nCol; /* Number of columns in index being sampled */
109332 ** values in the anEq[] array of any sample in StatAccum.a[]. In
109342 /* This sample is being added because the prefix that ends in column
109343 ** iCol occurs many times in the table. However, if we have already
109382 ** for the last sample in the p->a[] array. Otherwise, the samples would
109392 /* Zero the first nEqZero entries in the anEq[] array. */
109601 /* Return the value to store in the "stat" column of the sqlite_stat1
109605 ** the index. The first integer in the list is the total number of
109606 ** entries in the index. There is one additional integer in the list
109730 int i; /* Index of column in the table */
109828 int nCol; /* Number of columns in pIdx. "N" */
109896 ** (1) the number of columns in the index including the rowid
109898 ** (2) the number of columns in the key without the rowid/pk
109899 ** (3) estimated number of rows in the index,
110144 ** Generate code that will do an analysis of a single table in
110146 ** in pTab that should be analyzed.
110175 ** Form 1 causes all indices in all attached databases to be analyzed.
110190 ** and code in pParse and return NULL. */
110244 int nOut, /* Number of slots in aOut[] */
110312 ** Entries for which argv[1]==NULL simply record the number of rows in
110401 ** stored in pIdx->aSample[].
110422 tRowcnt nRow; /* Number of rows in index */
110424 i64 nDist100; /* Number of distinct values in index */
110437 ** occur in the stat4 table for this index. Set sumEq to the sum of
110496 Index *pPrevIdx = 0; /* Previous index in the loop */
110497 IndexSample *pSample; /* A slot in pIdx->aSample[] */
110509 int nIdxCol = 1; /* Number of columns in stat4 records */
110564 int nCol = 1; /* Number of columns in index */
110583 ** This is in case the sample record is corrupted. In that case, the
110629 ** If the sqlite_stat1 table is not present in the database, SQLITE_ERROR
110635 ** sqlite_stat4 table is not present in the database, SQLITE_ERROR is
110636 ** returned. However, in this case, data is read from the sqlite_stat1
110682 /* Set appropriate defaults on all indexes not in the sqlite_stat1 table */
110846 zErrDyn = sqlite3MPrintf(db, "database %s is already in use", zName);
110851 /* Allocate the new entry in the db->aDb[] array and initialize the schema
111182 "%s %T cannot reference objects in database %s",
111236 ** was left unspecified in the original SQL statement. The pFix structure
111240 ** view in one database does not refer to objects in a different database.
111241 ** (Exception: indices, triggers, and views in the TEMP database are
111243 ** to an object in a different database, an error message is added to
111328 ** All of the code in this file may be omitted by defining a single
111405 ** table zTab in database zDb. This function assumes that an authorization
111410 ** is treated as SQLITE_DENY. In this case an error is left in pParse.
111441 ** is in pTabList or else it is the NEW or OLD table of a trigger.
111456 int iSrc; /* Index in pTabList->a[] of table being read */
111458 int iCol; /* Index of column in table */
111502 ** is returned, then the error count and error message in pParse are
111592 ** when syntax rules are reduced. The routines in this file handle the
111621 ** The table to be locked has root page iTab and is found in database iDb.
111856 ** Run the parser and code generator recursively in order to generate
111908 ** Locate the in-memory structure that describes a particular database
111969 /* Attached databases are in order of attachment */
111988 ** Locate the in-memory structure that describes a particular database
111991 ** error message in pParse->zErrMsg.
111994 ** routine leaves an error message in pParse->zErrMsg where
111998 Parse *pParse, /* context in which to report errors */
112007 ** and code in pParse and return NULL. */
112017 /* If zName is the not the name of a table in the schema created using
112074 ** Locate the in-memory structure that describes
112120 ** For the index called zIdxName which is found in the database iDb,
112151 ** Look through the list of open database files in db->aDb[] and if
112266 ** contains lookaside memory. (Table objects in the schema do not use
112275 /* Record the number of outstanding lookaside allocations in schema Tables
112281 ** that no lookaside memory is used in this case either. */
112374 ** Open the sqlite_schema table stored in database number iDb for
112389 ** function returns the index of the named database in db->aDb[], or
112409 ** index of the named database in db->aDb[], or -1 if the named db
112439 Token *pName1, /* The "xxx" in the name "xxx.yyy" or "xxx" */
112440 Token *pName2, /* The "yyy" in the name "xxx.yyy" */
112485 ** "sqlite_" (in upper, lower or mixed case). This portion of the namespace
112538 ** for the column iCol in the table (as defined by the CREATE TABLE statement)
112539 ** find the (first) offset of that column in index pIdx. Or return -1
112540 ** if column iCol is not used in index pIdx.
112554 ** as it appears in the record on disk. The true column number
112555 ** is the index (0,1,2,...) of the column in the CREATE TABLE statement.
112577 ** as it appears in the record on disk. Or, if the input column is
112579 ** the number of non-virtual columns in the table plus N.
112581 ** The true column number is the index (0,1,2,...) of the column in
112585 ** in storage. But the value sometimes is cached in registers that
112588 ** values for use by OP_Param opcodes in triggers. Hence, if the
112602 ** So, in other words, this routine shifts all the virtual columns to
112629 ** Begin constructing a new table representation in memory. This is
112630 ** the first of several action routines that get called in response
112633 ** flag is true if the table should be stored in the auxiliary database
112634 ** file instead of in the main database file. This is normally the case
112635 ** when the "TEMP" or "TEMPORARY" keyword occurs in between
112638 ** The new table record is initialized and put in pParse->pNewTable.
112657 int iDb; /* Database number to create the table in */
112709 ** index or table name in the same database. Issue an error message if
112756 ** then record a pointer to this table in the main database structure
112767 ** the schema table. Note in particular that we must go ahead
112788 /* If the file format and encoding in the database have not been set,
112803 /* This just creates a place-holder record in the sqlite_schema table.
112805 ** by the real entry in code generated at sqlite3EndTable().
112807 ** The rowid for the new entry is left in register pParse->regRowid.
112808 ** The root page number of the new table is left in reg pParse->regRoot.
112892 sqlite3ErrorMsg(pParse, "cannot use RETURNING in a trigger");
112930 ** in a CREATE TABLE statement. sqlite3StartTable() gets called
113001 ** This routine is called by the parser while in the middle of
113033 ** substrings in the following table. If one of the substrings is
113050 ** If none of the substrings in the above table are found,
113127 ** This routine is called by the parser while in the middle of
113176 ** Historical versions of SQLite accepted strings as column names in
113177 ** indexes and PRIMARY KEY constraints and in UNIQUE constraints. Example:
113184 ** the expression given in its argument from a TK_STRING into a TK_ID
113380 /* generated column in an CREATE TABLE IF NOT EXISTS that already exists */
113411 sqlite3ErrorMsg(pParse, "error in generated column \"%s\"",
113437 ** 1 chance in 2^32. So we're safe enough.
113471 ** to the specified offset in the buffer and updates *pIdx to refer
113629 ** This is used to determine if the column number x appears in any of the
113651 ** This routine differs from hasColumn() in that both the column and the
113684 ** not considered to be covered by the index, even if they are in the
113685 ** index, because we do not trust the logic in whereIndexExprTrans() to be
113688 ** the actual table at hand in order to recompute the virtual column, if
113723 ** (4) Set the Index.tnum of the PRIMARY KEY Index object in the
113822 /* Update the in-memory representation of all UNIQUE indices by converting
113903 ** Return true if zName is a shadow table name in the current database
113910 char *zTail; /* Pointer to the last "_" in zName */
113958 ** An entry for the table is made in the schema table on disk, unless
113962 ** recently changed, so the entry for this table already exists in
113973 Token *pEnd, /* The ')' before options in the CREATE TABLE */
113979 int iDb; /* Database in which the table lives */
114033 /* Resolve names in all CHECK constraint expressions.
114059 /* If there are errors in resolving the expression, change the
114063 ** illegal in a schema and will lead to errors or heap corruption
114086 ** in the schema table of the database.
114120 ** new table is in register pParse->regRoot.
114122 ** Once the SELECT has been coded by sqlite3Select(), it is in a
114187 /* A slot for the record has already been allocated in the
114227 /* Add the table to the in-memory representation of the database.
114256 ** The parser calls this routine in order to create a new VIEW
114278 sqlite3ErrorMsg(pParse, "parameters are not allowed in views");
114335 ** The Table structure pTable is really a VIEW. Fill in the names of
114336 ** the columns of the view in the pTable structure. Return the number
114337 ** of errors. If an error is seen leave an error message in pParse->zErrMsg.
114379 ** in the following:
114393 ** "*" elements in the results set of the view and will assign cursors
114421 ** The names of the columns in the table are taken from
114422 ** arglist which is stored in pTable->pCheck. The pCheck field
114468 ** Clear the column names from every VIEW in database idx.
114491 ** root-page of a table or index in database iDb has changed from iFrom
114503 ** in order to be certain that we got the right one.
114543 /* OP_Destroy stores an in integer r1. If this integer
114548 ** The "#NNN" in the SQL is a special constant that means whatever value
114549 ** is in register NNN. See grammar rules associated with the TK_REGISTER
114563 ** in case a root-page belonging to another table is moved by the btree layer
114569 ** table and index root-pages in order, starting with the numerically
114578 ** and root page 5 happened to be the largest root-page number in the
114612 ** Remove entries from the sqlite_statN tables (for N in (1,2,3))
114669 ** at the btree level, in case the sqlite_sequence table needs to
114670 ** move as a result of the drop (can happen in auto-vacuum mode).
114680 /* Drop all entries in the schema table that refer to the
114684 ** created in the temp database that refers to a table in another
114708 ** Return TRUE if shadow tables should be read-only in the current
114846 ** in the current table point to the foreign key. If pFromCol==0 then
114849 ** of tables in the parent pTo table. flags contains all
114851 ** in the ON DELETE, ON UPDATE and ON INSERT clauses.
114854 ** under construction in the pParse->pNewTable field.
114861 ExprList *pFromCol, /* Columns in this table that point to other table */
114863 ExprList *pToCol, /* Columns in the other table */
114890 "number of columns in foreign key does not match the number of "
114891 "columns in the referenced table");
114931 "unknown column \"%s\" in foreign key definition",
115002 ** content of an index in response to a REINDEX command.
115014 int iSorter; /* Cursor opened by OpenSorter (if in use) */
115092 ** with DESC primary keys, since those indexes have there keys in
115114 ** pointer to this extra space in *ppExtra.
115118 i16 nCol, /* Total number of columns in the index */
115146 ** an explicit "NULLS FIRST" or "NULLS LAST" clause, leave an error in
115193 int nName; /* Number of characters in zName */
115196 int sortOrderMask; /* 1 to honor DESC in index. 0 to ignore. */
115346 ** "sqlite3_butoindex..." in order to make the names distinct.
115402 assert( pList->nExpr + nExtraCol <= 32767 /* Fits in i16 */ );
115436 ** a table column, store that column in aiColumn[]. For general expressions,
115437 ** populate pIndex->aColExpr and store XN_EXPR (-2) in aiColumn[].
115459 sqlite3ErrorMsg(pParse, "expressions prohibited in PRIMARY KEY and "
115567 ** the constraint occur in different orders, then the constraints are
115568 ** considered distinct and both result in separate indices.
115590 ** constraint specified somewhere in the CREATE TABLE statement.
115618 ** in-memory database structures.
115646 ** the index in the sqlite_schema table and populate the index with
115652 ** or UNIQUE index in a CREATE TABLE statement. Since the table
115667 ** doing so, code a Noop instruction and store its address in
115668 ** Index.tnum. This is required in case this index is actually a
115691 /* Add an entry in sqlite_schema for this index
115770 ** aiRowEst[0] is supposed to contain the number of elements in the index.
115772 ** number of rows in the table that match any particular value of the
115782 ** are based on typical values found in actual indices.
115795 /* Set the first entry (number of rows in the index) to the estimated
115796 ** number of rows in the table, or half the number of rows in the table
115801 ** estimated number of rows in the table be less than 1000 (LogEst 99).
115893 ** pArray is a pointer to an array of objects. Each object in the
115894 ** array is szEntry bytes in size. This routine uses sqlite3DbRealloc()
115898 ** the array (in entries - so the allocation is ((*pnEntry) * szEntry) bytes
115899 ** in total).
115904 ** returned. *pIdx is set to the index of the new array entry in this case.
115912 int szEntry, /* Size of each object in the array */
115913 int *pnEntry, /* Number of objects currently in use */
115978 ** Return the index in pList of the identifier named zId. Return -1
115994 ** than 64 tables in a join. So any value larger than 64 here
116020 ** in pParse.
116026 int iStart /* Index in pSrc->a[] of first new slot */
116078 ** need be. A new entry is created in the SrcList even if pTable is NULL.
116112 Parse *pParse, /* Parsing context, in which errors are reported */
116154 ** Assign VdbeCursor index numbers to all tables in a SrcList
116197 ** are the name of the table and database named in the FROM clause term.
116214 Select *pSubquery, /* A subquery used in place of a table name */
116317 ** When building up a FROM clause in the parser, the join operator
116328 ** in p->a[0] and p->a[1], respectively. The parser initially stores the
116420 ** the number of errors. Leave any error messages in the pParse structure.
116512 ** inserting multiple rows in a table, or inserting a row and index entries.)
116532 ** makes it more difficult to prove that the code is correct (in
116664 ** Recompute all indices of all tables in all databases where the
116673 HashElem *k; /* For looping over tables in pDb */
116674 Table *pTab; /* A table in the database */
116695 ** Form 1 causes all indices in all attached databases to be rebuilt.
116696 ** Form 2 rebuilds all indices in all databases that use the named
116705 Table *pTab; /* A table in the database */
116712 ** and code in pParse and return NULL. */
116870 ** not, store an error in the Parse structure. */
116936 ** in the encoding enc of name zName, length nName.
116962 ** collation function in the best encoding but there may be other versions
117014 ** Each pointer stored in the sqlite3.aCollSeq hash table contains an
117019 ** the collation sequence name. A pointer to this string is stored in
117044 /* If a malloc() failure occurred in sqlite3HashInsert(), it will
117108 ** requested collation sequence is not available in the desired encoding.
117113 ** The return value is either the collation sequence to be used in database
117155 ** in the database native encoding, the collation factory is invoked to
117157 ** and the sequence is available in another text encoding, then that is
117186 ** matches the request for a function with nArg arguments in a system
117339 /* If no match is found, search the built-in functions.
117341 ** If the DBFLAG_PreferBuiltin flag is set, then search the built-in
117343 ** priority to built-in functions.
117348 ** new function. But the FuncDefs for built-in functions are read-only.
117467 ** in order to generate code for DELETE FROM statements.
117472 ** While a SrcList can in general represent multiple tables and subqueries
117473 ** (as in the FROM clause of a SELECT statement) in this case it contains
117474 ** the name of a single table, as one might find in an INSERT, DELETE,
117475 ** or UPDATE statement. Look up that table in the symbol table and
117479 ** The following fields are initialized appropriate in pSrc:
117512 ** 3) The table is a shadow table, the database connection is in
117552 ** Evaluate a view and store its result in an ephemeral table. The
117554 ** set of rows in the view that are to be added to the ephemeral table.
117712 NameContext sNC; /* Name context to resolve expressions in */
117721 i16 nPk = 1; /* Number of columns in the PRIMARY KEY */
117723 i16 nKey; /* Number of memory cells in the row key */
117730 ** subqueries in the WHERE clause */
117746 ** put in an SrcList structure because some of the subroutines we
117835 /* Resolve the column names in the WHERE clause.
117949 ** one, so just keep it in its register(s) and fall through to the
117999 /* Set up a loop over the rowids/primary-keys that were found in the
118089 ** they may interfere with compilation of other functions in this file
118090 ** (or in another file, if this file becomes part of the amalgamation). */
118107 ** in the case of a rowid table, or the PRIMARY KEY index in the case
118113 ** 3. The primary key for the row to be deleted must be stored in a
118115 ** that a search record formed from OP_MakeRecord is contained in the
118153 int iOld = 0; /* First register in OLD.* array */
118174 ** use for the old.* references in the triggers. */
118176 u32 mask; /* Mask of OLD.* columns in use */
118230 ** a view (in which case the only effect of the DELETE statement is to
118257 ** handle rows (possibly in other tables) that refer via a foreign key
118326 ** Generate code that will assemble an index key and stores it in register
118332 ** Return a register number which is the first in a block of
118337 ** If *piPartIdxLabel is not NULL, fill it in with a label and jump
118350 ** a column in common, then the register corresponding to that column already
118397 ** might be stored in the table as an integer (using a compact
118436 ** functions of SQLite. (Some function, and in particular the date and
118611 ** in haystack and returns the number of previous characters plus 1,
118615 ** the number of bytes in haystack prior to the first occurrence of needle,
118616 ** or 0 if needle never occurs in haystack.
118760 ** is essentially a back-out of the bug-fix in check-in [5fc125d362df4b8]
118830 /* If Y==0 and X will fit in a 64-bit int,
118922 ** routines in the function table. The noopFunc macro provides this.
118943 ** in a way that is testable, mask the sign bit off of negative
118944 ** values, resulting in a positive value. Then take the
119039 ** character is exactly one byte in size. Also, provde the Utf8Read()
119040 ** macro for fast reading of the next character in the common case where
119071 ** SQLITE_NOWILDCARDMATCH: No match in spite of having * or % wildcards.
119082 ** [^...] Matches one character not in the enclosed list.
119085 ** in the list by making it the first character after '[' or '^'. A
119088 ** it the last character in the list.
119101 ** This routine is usually quick, but can be N**2 in the worst case.
119117 /* Skip over multiple "*" characters in the pattern. If there
119134 ** recursive search in this case, but it is an unusual case. */
119146 ** pattern string past the "*". Search in the input string for the
119151 ** c but in the other case and search the input string for either
119260 ** the build-in LIKE operator. The first argument to the function is the
119295 ** of deep recursion and N*N behavior in patternCompare().
119456 ** "NULL". Otherwise, the argument is enclosed in single quotes with
119742 int nIn; /* Number of bytes in input */
119745 unsigned char *aLen = 0; /* Length of each character in zCharSet */
119746 unsigned char **azChar = 0; /* Individual characters in zCharSet */
119747 int nChar; /* Number of characters in zCharSet */
119818 ** The "unknown" function is automatically substituted in place of
119823 ** involving application-defined functions to be examined in a generic
119945 ** 0.0 in that case. In addition, TOTAL always returns a float where
120240 ** of the built-in functions above are part of the global function set.
120252 ** Re-register the built-in LIKE functions. The caseSensitive
120280 ** operator if c is a string literal that is exactly one byte in length.
120281 ** That one byte is stored in aWc[3]. aWc[3] is set to zero if there is
120308 ** the first three statements in the compareInfo structure. The
120383 ** in our own actual functions.
120531 ** All of the FuncDef structures in the aBuiltinFunc[] array above
120540 ** defined in this file.
120695 #if 0 /* Enable to print out how the built-in functions are hashed */
120736 ** Foreign keys in SQLite come in two flavours: deferred and immediate.
120771 ** row in the parent table. Decrement the counter for each row
120778 ** deleted row in the child table. If such a row is not found,
120783 ** in the parent table. For each found increment the counter.
120857 ** A foreign key constraint requires that the key columns in the parent
120869 ** N is the number of columns in the parent key. The first element of the
120871 ** constraint to the parent table column stored in the left-most column
120889 ** consists of a different number of columns to the child key in
120897 Parse *pParse, /* Parse context to store any error in */
120901 int **paiCol /* OUT: Map of index columns in pFKey */
120905 int nCol = pFKey->nCol; /* Number of columns in parent key */
120959 ** map to an explicit list of columns in table pParent. Check if this
120964 i16 iCol = pIdx->aiColumn[i]; /* Index of column in parent tbl */
120972 ** unusable. Bail out early in this case. */
121013 ** row in the parent table that corresponds to the row being inserted into
121016 ** found in the parent table:
121028 ** These operations are identified in the comment at the top of this file
121035 Index *pIdx, /* Unique index on parent key columns in pTab */
121057 ** Check if any of the key columns in the child table row are NULL. If
121059 ** search for a matching row in the parent table. */
121239 ** the number of FK violations in the db) or +1 when deleting one (as this
121242 ** The code generated by this function scans through the rows in the child
121260 ** These operations are identified in the comment at the top of this file
121303 i16 iCol; /* Index of column in child table */
121304 const char *zCol; /* Name of column in child table */
121354 /* Resolve the references in the WHERE clause. */
121360 /* Create VDBE to loop through the entries in pSrc that match the WHERE
121426 ** constraint violations in the database,
121487 ** actually updated, the corresponding element in the aChange[] array
121514 ** actually updated, the corresponding element in the aChange[] array
121570 ** first register in an array of (pTab->nCol+1) registers containing the
121573 ** zero in this case.
121611 Index *pIdx = 0; /* Index on key columns in pTo */
121626 ** on the parent key columns in the parent table. If either of these
121627 ** schema items cannot be located, set an error in pParse and return
121640 ** before actually dropping it in order to check FK constraints.
121685 ** in the parent table. */
121726 ** an immediate foreign key violation. So do nothing in this case. */
121755 ** So do not set the "may-abort" flag in this case.
121784 ** row contained in table pTab.
121814 ** row contained in table pTab. If the operation is a DELETE, then
121816 ** to an array of size N, where N is the number of columns in table pTab.
121818 ** entry in the aChange[] array is set to -1. If the column is modified,
121847 ** table in question is either the child or parent table for any
121888 ** For example, if pFKey is the foreign key and pTab is table "p" in
121923 int nFrom; /* Length in bytes of zFrom */
121939 Token tFromCol; /* Name of column in child table */
121940 Token tToCol; /* Name of column in parent table */
121941 int iFromCol; /* Idx of column in child table */
122038 sizeof(TriggerStep) + /* Single step in trigger program */
122150 /* EV: R-30323-21917 Each foreign key constraint in SQLite is
122181 ** to handle INSERT statements in SQLite.
122197 int iDb, /* The database index in sqlite3.aDb[] */
122223 ** pIdx. A column affinity string has one character for each column in
122235 ** rowid that appears as the last column in every index.
122334 ** Return non-zero if the table pTab in database iDb or any of its indices
122335 ** have been opened at any point in the VDBE program. This is used to see if
122374 ** referenced columns in a CHECK constraint or generated column expression.
122450 ** Companion code in the TK_COLUMN case of sqlite3ExprCodeTarget() will
122489 ** which is in database iDb. Return the register number for the register
122504 ** (3) The rowid in sqlite_sequence of pTab
122505 ** (4) The original value of the max ROWID in pTab, or NULL if none
122550 pToplevel->nMem +=2; /* Rowid in sqlite_sequence + orig max val */
122619 ** larger than the maximum rowid in the memId memory cell, then the
122703 ** The IDLIST appears in the pColumn parameter. pColumn is NULL if IDLIST
122727 ** in other words if the SELECT pulls all columns from a single table
122737 ** transfer all records in <table2> over to <table>
122753 ** loop over the rows in the SELECT
122774 ** loop over the tables in the SELECT
122804 int nColumn; /* Number of columns in the data */
122812 int addrCont = 0; /* Top of insert loop. Label "C" in templates 3 and 4 */
122815 u8 useTempTable = 0; /* Store SELECT results in intermediate table */
122818 u8 bIdListInOrder; /* True if IDLIST is in table order */
122820 int iRegStore; /* Register in which to store next column */
122923 /* If this is an AUTOINCREMENT table, look up the sequence number in the
122924 ** sqlite_sequence table and store it in memory cell regAutoinc.
122944 ** is named in the IDLIST, then record in the ipkColumn variable
122946 ** the index of the primary key as it appears in IDLIST, not as
122947 ** is appears in the original table. (The index of the INTEGER
122948 ** PRIMARY KEY in the original table is pTab->iPKey.) After this
122953 ** bIdListInOrder is true if the columns in IDLIST are in storage
123029 ** temp table in the case of row triggers.
123084 ** column index in the original table definition.
123102 /* Make sure the number of columns in the source data matches the number
123219 ** are written in storage order into registers starting with regData.
123220 ** Only ordinary columns are computed in this loop. The rowid
123233 ** using the rowid. So put a NULL in the IPK slot of the record to avoid
123242 /* Virtual columns do not participate in OP_MakeRecord. So back up
123243 ** iRegStore by one slot to compensate for the iRegStore++ in the
123257 /* Hidden columns that are not explicitly named in the INSERT
123266 /* A column not named in the insert column list gets its
123328 ** computing the ROWID in case one of the generated columns
123399 ** computing the ROWID in case one of the generated columns
123428 ** parent table in a foreign key constraint. It is safe to set the
123429 ** flag in the second case as if any REPLACE constraint is hit, an
123466 ** OP_ReleaseReg will be included in the loop. */
123507 ** they may interfere with compilation of other functions in this file
123508 ** (or in another file, if this file becomes part of the amalgamation). */
123520 ** Meanings of bits in of pWalker->eCode for
123553 ** the new row in the UPDATE statement.
123584 ** the indexes of a table in the order provided in the Table->pIndex list.
123586 ** the indexes in a different order. The following data structures accomplish
123590 ** of a table in either Index.pNext order, or in some other order established
123614 int ix; /* Which entry in the original Table.pIndex list is this index*/
123650 ** The regNewData parameter is the first register in a range that contains
123652 ** pTab->nCol+1 registers in this range. The first register (the one
123653 ** that regNewData points to) will contain the new rowid, or NULL in the
123654 ** case of a WITHOUT ROWID table. The second register in the range will
123677 ** indices where aRegIdx[i]==0. The order of indices in aRegIdx[] is
123682 ** main table, if pTab is a rowid table, and stores that record in the
123683 ** register identified by aRegIdx[nIdx] - in other words in the first
123694 ** for the first index in the pTab->pIndex list. Cursors for other indices
123718 ** any IGNORE The attempt in insert or update the current
123730 ** CHECK REPLACE Illegal. The results in an exception.
123743 int regNewData, /* First register in a range holding values to insert */
123761 int nPkField; /* Number of fields in PRIMARY KEY. 1 for ROWID tables */
123786 ** normal rowid tables. nPkField is the number of key fields in the
123788 ** number of fields in the true primary key of the table. */
123946 /* UNIQUE and PRIMARY KEY constraints should be handled in the following
123956 ** could happen in any order, but they are grouped up front for
123964 ** Constraint checking code is generated in this order:
123972 ** in the list. See sqlite3CreateIndex() for where that happens.
123994 ** checked first and in order. */
124050 ** in a second pass. The addrRecheck and lblRecheckOk variables are
124052 ** in the generate bytecode.
124079 ** exist in the table.
124084 /* Figure out what action to take in case of a rowid collision */
124092 /* figure out whether or not upsert applies in this case */
124132 /* Check to see if the new rowid already exists in the table. Skip
124236 int addrConflictCk; /* First opcode in the conflict check logic */
124264 ** the insert or update. Store that record in the aRegIdx[ix] register
124304 /* Find out what action to take in case there is a uniqueness conflict */
124333 ** must be explicitly deleted in order to ensure any pre-update hook
124370 ** store it in registers regR..regR+nPk-1 */
124437 int nConflictCk; /* Number of opcodes in conflict check logic */
124564 ** to be the number of columns in table pTab that must not be NULL-trimmed.
124691 ** the table itself, but the PRIMARY KEY index in the case of a WITHOUT
124692 ** ROWID table) is returned in *piDataCur. The first index cursor is
124693 ** returned in *piIdxCur. The number of indices is returned.
124700 ** For a WITHOUT ROWID table, *piDataCur will be somewhere in the range
124728 ** can detect if they are used by mistake in the caller. */
124776 ** for index pDest in an insert transfer optimization. The rules
124803 return 0; /* Different expressions in the index */
124828 ** performance. Raw index records are transferred in the same way.
124832 ** embedded in the code for details.
124855 Table *pSrc; /* The table in the FROM clause of SELECT */
124928 ** correct syntactic form to participate in this optimization. Now
124952 return 0; /* Number of columns must be the same in tab1 and tab2 */
124980 ** We could, in theory, disable this (except for internal use by the
125029 return 0; /* pDestIdx has no corresponding index in pSrc */
125034 ** bypassed during testing, in order to exercise other corruption tests
125171 ** index will be populated by inserting keys in strictly sorted
125180 ** a VACUUM command. In that case keys may not be written in strictly
125238 ** Main file for the SQLite library. The routines in this file
125239 ** implement the programmer interface to the library. Routines in
125251 ** If the SQL is a query, then for each row in the query result
125387 #define SQLITE_CORE 1 /* Disable the API redefinition in sqlite3ext.h */
125389 /************** Include sqlite3ext.h in the middle of loadext.c **************/
125418 ** interfaces in the middle of this structure, then older different
125734 ** is also defined in the file "loadext.c".
126056 /************** Continuing where we left off in loadext.c ********************/
126162 ** in order to preserve backwards compatibility.
126531 ** Attempt to load an SQLite extension library contained in the file
126532 ** zFile. The entry point is zProc. zProc may be 0 in which case a
126615 ** character in the filename after the last "/" upto the first ".",
126649 "no entry point [%s] in shared library [%s]", zEntry, zFile);
126698 ** Call this routine when the database connection is closing in order
126712 ** default so as not to open security holes in older applications.
126736 u32 nExt; /* Number of entries in aExt[] */
126852 ** If anything goes wrong, set an error in the database connection.
126921 ** object. This ensures that the aPragmaName[] table is arranged in
126923 ** Do not edit pragma.h directly. Edit and rerun the script in at
126925 /************** Include pragma.h in the middle of pragma.c *******************/
127052 /* Definitions of all built-in pragmas */
127057 u8 iPragCName; /* Start of column names in pragCName[] */
127577 /************** Continuing where we left off in pragma.c *********************/
127656 ** backed temporary databases, 2 for the Red-Black tree in memory database
127804 ** defined in pager.h. This function returns the associated lowercase
127827 ** Locate a pragma in the aPragmaName[] array.
127847 ** Create zero or more entries in the output for the SQL functions
127853 int isBuiltin, /* True if this is a built-in function */
127899 ** string held in register 3. Decrement the result count in register 1
127950 ** index of the database this pragma is being applied to in db.aDb[]. */
127986 ** file control is an array of pointers to strings (char**) in which the
128014 /* Locate the pragma in the lookup table */
128040 ** pages in the page cache. The second form sets both the current
128042 ** stored in the database file.
128091 ** database page size in bytes. The second form sets the
128148 ** maximum number of pages in the database file. The
128158 ** Return the number of pages in the specified database.
128251 /* Do not allow journal-mode "OFF" in defensive since the database
128309 ** incr-vacuum flags. This is required in case this connection
128316 ** "incremental", write the value of meta[6] in the database
128375 ** number of pages in the cache. If N is negative, then the
128402 ** number of pages in the cache. If N is negative, then the
128442 ** The parameter N is measured in bytes.
128651 ** in auto-commit mode. */
128941 Index *pIdx; /* Index in the parent table */
128944 HashElem *k; /* Loop counter: Next table in schema */
129008 ** row cannot cause an FK violation. Jump directly to addrOk in
129128 HashElem *x; /* For looping over tables in the schema */
129129 Hash *pTbls; /* Set of all tables in the schema */
129131 int cnt = 0; /* Number of entries in aRoot[] */
129142 ** for all tables and indices in the database.
129181 sqlite3MPrintf(db, "*** in database %s ***\n", db->aDb[i].zDbSName),
129202 /* reg[7] counts the number of entries in the table.
129203 ** reg[8+i] counts the number of entries in the i-th index
129229 zErr = sqlite3MPrintf(db, "NULL value in %s.%s", pTab->zName,
129251 zErr = sqlite3MPrintf(db, "CHECK constraint failed in %s",
129299 sqlite3VdbeLoadString(v, 3, "non-unique entry in index ");
129310 sqlite3VdbeLoadString(v, 2, "wrong # of entries in index ");
129370 ** In the second form this pragma sets the text encoding to be used in
129438 ** stored in the database header.
129495 ** Return the names of all compile-time options used in this build,
129540 ** after accumulating N frames in the log. Or query for the current value
129572 ** Attempt to optimize the database. All schemas are optimized in the first
129573 ** two forms, and only the specified schema is optimized in the latter two.
129577 ** this pragma will perform new optimizations in future releases.
129589 ** information from the current session in the
129614 ** the number of rows in the table has increased by 25 times or more
129617 ** The rules for when tables are analyzed are likely to change in
129625 Table *pTab; /* A table in the schema */
129645 /* If table pTab has not been used in a way that would benefit from
129762 ** in each index that it looks at. Return the new limit.
130176 "error in %s %s after %s: %s", azObj[0], azObj[1],
130210 int nBytes, /* Length of zSql in bytes. */
130356 /* Construct the in-memory representation schema tables (sqlite_schema or
130419 ** Note: The #defined SQLITE_UTF* symbols in sqliteInt.h correspond to
130485 /* Ticket #2804: When we open a database in the newer file format,
130567 ** bit is set in the flags field of the Db structure.
130619 ** Check schema cookies in all databases. If any cookie is out
130649 ** value stored as part of the in-memory schema representation,
130667 ** which database file in db->aDb[] the schema refers to.
130675 /* If pSchema is NULL, then return -32768. This happens when code in
130700 ** Free all memory allocations in the pParse object
130730 ** common cleanups, we save a single NULL-pointer comparison in
130735 ** pParse->earlyCleanup flag is set in that case. Calling code show verify
130743 ** except if an early cleanup occurs, in which case it returns NULL. So
130777 int nBytes, /* Length of zSql in bytes. */
130806 ** some other database connection is holding a write-lock, which in
130812 ** back and different changes are made in their place, then when this
130817 ** of the sqlite3BtreeEnterAll() in sqlite3LockAndPrepare()) so it
130917 int nBytes, /* Length of zSql in bytes. */
130993 ** version, the original SQL text is not saved in the prepared statement
131002 int nBytes, /* Length of zSql in bytes. */
131014 int nBytes, /* Length of zSql in bytes. */
131023 ** Proof in that the 5th parameter to sqlite3LockAndPrepare is 0 */
131032 int nBytes, /* Length of zSql in bytes. */
131039 ** sqlite3_prepare_v2() only in having the extra prepFlags parameter,
131060 int nBytes, /* Length of zSql in bytes. */
131067 ** tricky bit is figuring out the pointer to return in *pzTail.
131109 ** version, the original SQL text is not saved in the prepared statement
131118 int nBytes, /* Length of zSql in bytes. */
131130 int nBytes, /* Length of zSql in bytes. */
131142 int nBytes, /* Length of zSql in bytes. */
131171 ** to handle SELECT statements in SQLite.
131202 ** When the sorter-reference optimization is used, there is one entry in the
131218 u8 nDefer; /* Number of valid entries in aDefer[] */
131281 ExprList *pEList, /* which columns to include in the result */
131342 ** Return a pointer to the right-most SELECT statement in a compound.
131352 ** in terms of the following bit values:
131364 ** a join type, but put an error in the pParse structure.
131373 u8 i; /* Beginning of keyword text in zKeyText[] */
131374 u8 nChar; /* Length of the keyword in characters */
131424 ** Return the index of a column in a table. Return -1 if the column
131425 ** is not contained in the table.
131438 ** Search the first N tables in pSrc, from left to right, looking for a
131448 int N, /* Number of tables in pSrc->a[] to search */
131454 int i; /* For looping over tables in pSrc */
131480 ** where tab1 is the iSrc'th table in SrcList pSrc and tab2 is the
131486 SrcList *pSrc, /* List of tables in FROM clause */
131487 int iLeft, /* Index of first table to join in pSrc */
131488 int iColLeft, /* Index of column in first table */
131489 int iRight, /* Index of second table in pSrc */
131490 int iColRight, /* Index of column in second table */
131519 ** And set the Expr.iRightJoinTable to iTable for every term in the
131524 ** join restriction specified in the ON or USING clause and not a part
131527 ** originated in the ON or USING clause.
131531 ** explicitly mentioned in the expression. That information is needed
131540 ** after the t1 loop and rows with t1.x!=5 will never appear in
131591 ** The terms of a FROM clause are contained in the Select.pSrc structure.
131592 ** The left most table is the first entry in Select.pSrc. The right-most
131593 ** table is the last entry. The join operator is held in the entry to
131601 SrcList *pSrc; /* All tables in the FROM clause */
131617 ** every column that the two tables have in common.
131626 char *zName; /* Name of column in the right table */
131628 int iLeftCol; /* Matching column in the left table */
131639 /* Disallow both ON and USING clauses in the same join
131643 "clauses in the same join");
131657 ** in the USING clause. Example: If the two tables to be joined are
131660 ** Report an error if any column mentioned in the USING clause is
131661 ** not contained in both tables to be joined.
131666 char *zName; /* Name of the term in the USING clause */
131677 "not present in both tables", zName);
131694 int regResult; /* Store results in array of registers here */
131725 ** Return the register in which the result is stored.
131745 ** Generate code that will push the record in registers regData
131754 int nData, /* Number of elements in the regData data array */
131760 int nBase = nExpr + bSeq + nData; /* Fields in sorter record */
131774 ** (2) All output columns are included in the sort record. In that
131848 ** in an array of registers. They need to be composed into a record
131851 ** the largest record currently in the sorter. If (b) is true and there
131852 ** are already LIMIT+OFFSET items in the sorter, delete the largest
131854 ** than LIMIT+OFFSET items in the sorter.
131904 ** seen combinations of the N values. A new entry is made in iTab
131940 ** storing values read from that column in the sorter records, the PK of
131944 ** can be more efficient than storing them directly in the sorter records.
131946 ** The ExprList_item.bSorterRef flag is set for each expression in pEList
131951 ** expressions for all extra PK values that should be stored in the
132015 ** are evaluated in order to get the data for this row. If srcTab is
132038 /* Usually, regResult is the first cell in an array of memory cells
132070 ** there are columns in the table on the left. The error will be caught
132072 ** to avoid other spurious errors in the meantime. */
132097 /* For each expression in p->pEList that is a copy of an expression in
132118 ** composite primary keys in the SortCtx.aDefer[] array. */
132127 ** from the sorter by the optimizations in this branch */
132272 ** in the index, do not write it to the output. If not, add the
132306 ** aggregates do. Don't record that empty row in the output table. */
132328 ** ORDER BY in this case since the order of entries in the set
132329 ** does not matter. But there might be a LIMIT clause, in which
132345 /* If any row exist in the result set, record that fact and abort.
132354 ** store the results in the appropriate memory cell or array of
132388 ** pDest->pOrderBy (in pSO). pDest->iSDParm (in iParm) is the cursor for an
132449 ** there is a sorter, in which case the sorter has already limited
132512 ** the collating sequence for each expression in that expression list.
132580 ** in sqlite3Select() to assign values to structure member variables that
132595 ** then the results were placed in a sorter. After the loop is terminated
132618 int nKey; /* Number of key columns in sorter record */
132800 ** result in *pEstWidth.
132805 ** is considered a column can be complex in the presence of subqueries. The
132806 ** result-set expression in all of the following SELECT statements is
132848 ** extracted from in NameContext.pSrcList. This table may be real
132853 int iCol = pExpr->iColumn; /* Index of column in pTab */
132875 ** when columnType() is called on the expression "t1.col" in the
132888 /* The "table" is actually a sub-select or a view in the FROM clause
132936 ** origin info for the single column in the result set of the SELECT
132965 ** in the result set.
132989 ** column specific strings, in case the schema is reset before this
133022 ** originally appears in the SELECT statement. In
133115 ** store NULL in *paCol and 0 in *pnCol and return SQLITE_NOMEM.
133136 int nCol; /* Number of columns in the result set */
133138 int nName; /* Size of name in zName[] */
133227 ** This routine requires that all identifiers in the SELECT
133312 ** If an error occurs, return NULL and leave a message in pParse.
133330 ** that appear in the original SQL statement after the LIMIT and OFFSET
133472 ** There is exactly one reference to the recursive-table in the FROM clause
133478 ** extracted row (now in the iCurrent table) becomes the content of the
133488 ** If the query has an ORDER BY, then entries in the Queue table are kept in
133505 int nCol = p->pEList->nExpr; /* Number of columns in the recursive table */
133525 sqlite3ErrorMsg(pParse, "cannot use window functions in recursive queries");
133553 ** the Distinct table must be exactly one greater than Queue in order
133601 /* Store the results of the setup-query in Queue. */
133609 /* Find the next row in the Queue and output that row */
133612 /* Transfer the next row in Queue over to Current */
133621 /* Output the single row in Current */
133632 /* Execute the recursive SELECT taking the single row in Current as
133633 ** the value for the recursive-table. Store the results in the Queue.
133673 ** The sqlite3CodeSubselect will have added the LIMIT 1 clause in tht case.
133725 ** in which case this routine will be called recursively.
133742 ** The arrows in the diagram above represent the Select.pPrior pointer.
133744 ** pPrior will be the t2 query. p->op will be TK_UNION in this case.
133762 ** the last (right-most) SELECT in the series may have an ORDER BY or LIMIT.
133792 /* Make sure all SELECTs in the statement have the same number of elements
133793 ** in their result sets.
133929 /* Convert the data in the temporary table into whatever form
134048 int nCol; /* Number of columns in result set */
134108 ** The data to be output is contained in pIn->iSdst. There are
134115 ** If regPrev>0 then it is the first register in a vector that
134121 ** If the LIMIT found in p->iLimit is reached, jump immediately to
134192 ** store the results in the appropriate memory cell and break out
134204 /* The results are stored in a sequence of registers
134221 ** For SRT_Output, results are stored in a sequence of registers.
134256 ** co-routines. Then run the co-routines in parallel and merge the results
134266 ** appears only in B.)
134299 ** Duplicate removal in the UNION, EXCEPT, and INTERSECT cases is handled
134369 int nOrderBy; /* Number of terms in the ORDER BY clause */
134540 ** are exhausted and only data in select B remains.
134554 ** are exhausted and only data in select A remains.
134612 /* Jump to the this point in order to terminate the query.
134636 ** All references to columns in table iTable are to be replaced by corresponding
134637 ** expressions in pEList.
134653 ** a column in table number iTable with a copy of the iColumn-th
134654 ** entry in pEList. (But leave references to the ROWID column
134658 ** whose result set is defined by pEList appears as entry in the
134666 Expr *pExpr /* Expr in which substitution occurs */
134746 ExprList *pList /* List to scan and in which to make substitutes */
134756 Select *p, /* SELECT statement in which to make substitutions */
134783 ** pSelect is a SELECT statement and pSrcItem is one item in the FROM
134815 ** Assign new cursor numbers to each of the items in pSrc. For each
134816 ** new cursor number assigned, set an entry in the aCsrMap[] array
134822 ** existing cursor numbers in pSrc.
134829 int *aCsrMap, /* Array to store cursor mappings in */
134863 ** Assign a new cursor number to each cursor in the FROM clause (Select.pSrc)
134865 ** cursor in the FROM clause of any FROM clause sub-selects, recursively.
134866 ** Except, do not assign a new cursor number to the iExcept'th element in
134876 ** * the array entries for all cursor numbers that do *not* appear in
134907 ** subquery first and store the results in a temporary table, then
135017 ** appear as unmodified result columns in the outer query. But we
135018 ** have other optimizations in mind to deal with that case.
135028 ** recursive queries in multiSelect().
135031 ** The subquery may not be an aggregate that uses the built-in min() or
135037 ** function in the select list or ORDER BY clause, flattening
135054 int iFrom, /* Index in p->pSrc->a[] of the inner subquery */
135060 Select *pSub1; /* Pointer to the rightmost select in sub-query */
135151 ** every reference to any result column from subquery in a join, even
135236 ** select statements in the compound sub-query.
135317 /* The following loop runs once for each term in a compound-subquery
135324 ** the cursor number for the original outer query FROM element in
135328 ** elements we are now copying in.
135336 nSubSrc = pSubSrc->nSrc; /* Number of terms in subquery FROM clause */
135344 ** query. If the subquery has more than one element in its FROM clause,
135352 ** The outer query has 3 slots in its FROM clause. One slot of the
135355 ** The middle slot is expanded to two slots in order to make space
135356 ** for the two elements in the FROM clause of the subquery.
135377 ** references to the iParent in the outer query.
135385 ** We look at every expression in the outer query and every place we see
135392 ** do not necessarily correspond to columns in SELECT statement pParent,
135530 ** Find all terms of COLUMN=VALUE or VALUE=COLUMN in pExpr where VALUE
135560 ** columns named in pWalker->u.pConst, then overwrite it with its
135619 ** "b=a" to "b=123" but rather the "a" in "b=a" is tagged with EP_FixedCol
135627 Select *p /* The query in which to propagate constants */
135664 ** of constants and copies of expressions that appear in the PARTITION
135717 ** (5) The WHERE clause expression originates in the ON or USING clause
135740 ** copies of expressions found in the PARTITION BY clause of
135746 ** be materialized. (This restriction is implemented in the calling
135777 ** sure no other terms are marked SF_Recursive in case something changes
135778 ** in the future.
135820 /* Restriction 6c has prevented push-down in this case */
135839 ** The pFunc is the only aggregate function in the query. Check to see
135928 ** SQLITE_ERROR and leave an error in pParse. Otherwise, populate
135970 ** there are COLLATE terms in the ORDER BY.
135992 ** by this function is not required in this case. */
136034 ** arguments. If it does, leave an error message in pParse and return
136125 /* There are no WITH clauses in the stack. No match is possible */
136147 ** recursive reference to CTE pCte. Leave an error in pParse and return
136260 pCte->zCteErr = "recursive reference in a subquery: %s";
136297 ** sub-query in the FROM clause of a SELECT statement. This function
136331 ** (2) Fill in the pTabList->a[].pTab fields in the SrcList that
136332 ** defines FROM clause. When views appear in the FROM clause,
136342 ** (4) Scan the list of columns in the result set (pEList) looking
136344 ** If found, expand each "*" to be every column in every table
136345 ** and TABLE.* to be every column in TABLE.
136375 /* Make sure cursor numbers have been assigned to all entries in
136380 /* Look up every table named in the FROM clause of the select. If
136392 /* A sub-query in the FROM clause of a SELECT */
136405 /* An ordinary table or view name in the FROM clause */
136465 /* For every "*" that occurs in the column list, insert the names of
136466 ** all columns in all tables. And for every TABLE.* insert the names
136467 ** of all columns in TABLE. The parser inserted a special expression
136468 ** with the TK_ASTERISK operator for each "*" that it found in the column
136470 ** expressions and expand each one to the list of all columns in
136488 ** in the result set and expand them one by one.
136574 /* In a join with a USING clause, omit columns in the
136630 sqlite3ErrorMsg(pParse, "too many columns in result set");
136655 ** Expanding a SELECT statement is the first step in processing a
136708 /* A sub-query in the FROM clause of a SELECT */
136723 ** the Table structures of all FROM-clause subqueries in a
136747 ** * Wildcards "*" and "TABLE.*" in result sets are expanded.
136748 ** * Identifiers in expression are matched to tables.
136772 ** routine generates code that stores NULLs in all of those memory
136815 ** in the AggInfo structure.
136835 ** in pAggInfo, then only populate the pAggInfo->nAccumulator accumulator
137022 SrcList *pTabList, /* Search for self-joins in this FROM clause */
137040 ** names in the same FROM clause. */
137095 if( p->pSrc->nSrc!=1 ) return 0; /* One table in FROM */
137150 ** Generate code for the SELECT statement given in the p argument.
137153 ** See comments in sqliteInt.h for further information.
137156 ** encountered, then an appropriate error message is left in
137159 ** This routine does NOT free the Select structure passed in. The
137245 "target object/alias may not appear in FROM clause: %s",
137276 ** reduction of join operators) in the FROM clause up into the main query
137290 ** of the LEFT JOIN used in the WHERE clause.
137305 /* Catch mismatch in the declared columns of a view and the number of
137306 ** columns in the SELECT on the RHS */
137318 ** flattening in that case.
137382 ** as the equivalent optimization will be handled by query planner in
137409 /* For each term in the FROM clause, do two things:
137431 ** have a column named by the empty string, in which case there is no way to
137443 /* Generate code for all sub-queries in the FROM clause
137471 /* Make copies of constant WHERE-clause terms in the outer query down
137539 /* This view has already been materialized by a prior entry in
137644 ** being unused if the data can be extracted in pre-sorted order.
137779 int groupBySort; /* Rows come from source in GROUP BY order */
137790 struct ExprList_item *pItem; /* For looping over expression in a list */
137804 ** in the correct order. It also may not - the GROUP BY might use a
137811 /* The GROUP BY processing doesn't care whether rows are delivered in
137813 ** So set the ASC/DESC flags in the GROUP BY to match those in the
137814 ** ORDER BY to maximize the chances of rows being delivered in an
137832 /* Convert TK_COLUMN nodes into TK_AGG_COLUMN and make entries in
137833 ** sAggInfo for all TK_AGG_FUNCTION nodes in expressions of the
137952 /* Begin a loop that will extract all source rows in GROUP BY order.
137953 ** This might involve two separate loops with an OP_Sort in between, or
137955 ** in the right order to begin with.
137965 /* The optimizer is able to deliver rows in group by order so
137971 /* Rows are coming out in undetermined order. We have to push
137973 ** then loop over the sorting index in order to get the output
137974 ** in sorted order
138023 ** will naturally deliver rows in the order required by the ORDER BY
138036 /* Evaluate the current GROUP BY terms and store in b0, b1, b2...
138039 ** from the previous row currently stored in a0, a1, a2...
138060 ** Changes in the GROUP BY are detected by the previous code
138063 ** This code copies current group by terms in b0,b1,b2,...
138065 ** and resets the aggregate accumulator registers in preparation
138082 VdbeComment((v, "indicate data in accumulator"));
138108 ** increments the iAbortFlag memory location before returning in
138168 ** (2013-10-03) Do not count the entries in a partial index.
138284 /* The SELECT has been coded. If there is an error in the Parse structure,
138337 ** These routines are in a separate files so that they will not be linked
138352 u32 nRow; /* Number of rows in the result */
138353 u32 nColumn; /* Number of columns in the result */
138354 u32 nData; /* Slots used in azResult[]. (nRow+1)*nColumn */
138359 ** This routine is called once for each row in the result table. Its job
138360 ** is to fill in the TabResult structure appropriately, allocating new
138365 int need; /* Slots needed in p->azResult[] */
138369 /* Make sure there is enough space in p->azResult to hold everything
138432 ** The result that is written to ***pazResult is held in memory obtained
138441 int *pnRow, /* Write the number of rows in the result here */
138562 ** All of the triggers on pTab that are in the same database as pTab
138564 ** triggers on pTab in the TEMP schema. This routine prepends all
138570 ** pTab as well as the triggers lised in pTab->pTrigger.
138612 ** in pParse->pNewTrigger. After the trigger actions have been parsed, the
138632 int iDb; /* The database to store the trigger in */
138649 /* Figure out the db that the trigger will be created in */
138673 ** then set iDb to 1 to create the trigger in the temporary database.
138809 ** drop so the trigger cannot be dropped. This results in an
138821 ** in order to complete the process of building the trigger.
138867 /* Make an entry in the sqlite_schema table */
138922 ** The parser calls this routine when it finds a SELECT statement in
138984 IdList *pColumn, /* List of columns in pTableName to insert into */
139226 ** in pEList is of the format <id>=<expr>. If any of the entries
139227 ** in pEList have an <id> which matches an identifier in pIdList,
139246 ** least one of the columns in pChanges is being modified.
139252 ExprList *pChanges, /* Columns that change in an UPDATE statement */
139317 ** forming the SrcList. This prevents a trigger in one database from
139318 ** referring to a target in another database. An exception is when the
139319 ** trigger is in TEMP in which case it can refer to any other database it
139355 Expr *pTerm /* A term in the RETURNING clause */
139422 ** that invoke a subprogram in the bytecode, the code for RETURNING
139423 ** is generated in-line.
139429 int regIn /* The first in an array of registers */
139561 ** program. It is not used in production code, only for debugging.
139723 /* It may be that this trigger has already been coded (or is in the
139726 ** in the Parse.pTriggerPrg list. Search for such an entry. */
139743 ** function are the same as those described in the header function for
139759 /* Code the OP_Program opcode in the parent VDBE. P4 of the OP_Program
139788 ** The reg argument is the address of the first in an array of registers
139790 ** in the trigger program. If N is the number of columns in table pTab
139807 ** Similarly, for ON INSERT triggers the values stored in the OLD.* registers
139825 int reg, /* The first in an array of registers (see above) */
139838 ** always defined. The trigger must be in the same schema as the table
139864 ** Triggers may access values stored in the old.* or new.* pseudo-table.
139873 ** are more than 32 columns in the table, and at least one of the columns
139884 ** included in the returned mask if the TRIGGER_BEFORE bit is set in the
139886 ** included in the returned mask if the TRIGGER_AFTER bit is set in tr_tm.
139948 ExprList *pChanges, /* The columns to change in the UPDATE statement */
139950 int *aXRef, /* Mapping from columns of pTab to entries in pChanges */
139961 ** The default value of a column is specified by a DEFAULT clause in the
139964 ** command. If the latter, then the row-records in the table btree on disk
140011 ** performance degradation, but false-negatives can result in a corrupt
140041 ** performance degradation, but false-negatives can result in a corrupt
140062 ** table in the source-list (pSrc->a[0]).
140090 ** the <other-columns> in the query above are is determined by the type
140095 ** of the table, in order. The results of the query are written to ephemeral
140105 ** rowid value in <other-columns> is used as the integer key, and the
140204 SrcList *pTabList, /* The table in which we should change things */
140226 int *aXRef = 0; /* aXRef[i] is the index in pChanges->a[] of the
140230 u8 chngPk; /* PRIMARY KEY changed in a WITHOUT ROWID table */
140231 u8 chngRowid; /* Rowid changed in a normal table */
140234 int iRowidExpr = -1; /* Index of "rowid=" (or IPK) assignment in pChanges */
140236 NameContext sNC; /* The name-context to resolve expressions in */
140251 int nKey = 0; /* Number of elements in regKey for WITHOUT ROWID */
140264 int regNew = 0; /* Content of the NEW.* table in triggers */
140265 int regOld = 0; /* Content of OLD.* table in triggers */
140299 ** in the change-list. Otherwise, set it to 0. There cannot be a FROM
140325 ** allocate enough space, just in case.
140367 /* Resolve the column names in all the expressions of the
140369 ** for each column to be updated in the pChanges array. For each
140440 ** obvious when looking at aXRef[] in a symbolic debugger.
140470 /* There is one entry in the aRegIdx[] array for each index on the table
140471 ** being updated. Fill in aRegIdx[] with a register number that will hold
140501 ** indexes in case they are needed to delete records. */
140512 ** reallocated. aRegIdx[nAllIdx] is the register in which the main
140547 /* Resolve the column names in all the expressions in the
140670 ** leave it in register regOldRowid. */
140680 ** ONEPASS_OFF mode, serialize the array into a record and store it in
140681 ** the ephemeral table. Or, in ONEPASS_SINGLE or MULTI mode, change
140683 ** is not required) and leave the PK fields in the array of registers. */
140848 ** a new.* reference in a trigger program.
140879 ** is deleted or renamed by a BEFORE trigger - is left undefined in the
140894 ** in case this has happened. Only unmodified columns are reloaded.
140992 ** handle rows (possibly in other tables) that refer via a foreign key
141053 ** they may interfere with compilation of other functions in this file
141054 ** (or in another file, if this file becomes part of the amalgamation). */
141075 ** (C) The content of every column in the row.
141081 ** stores the same values (A, B and C above) in a register array and
141088 ExprList *pChanges, /* The columns to change in the UPDATE statement */
141090 int *aXRef, /* Mapping from columns of pTab to entries in pChanges */
141101 int regArg; /* First register in VUpdate arg array */
141102 int regRec; /* Register in which to assemble record */
141109 /* Allocate nArg registers in which to gather the arguments for VUpdate. Then
141110 ** create and open the ephemeral table in which the records created from
141310 Upsert *pNext /* Next ON CONFLICT clause in the list */
141334 ** symbols in the conflict-target.
141359 /* Resolve all symbolic names in the conflict-target clause, which
141581 ** Most of the code in this file may be omitted by defining the
141645 ** in PostgreSQL. The VACUUM command works as follows:
141660 ** Two writes per page are required in step (3) because the original
141668 ** And a power loss in between deleting the original and renaming the
141737 sqlite3SetString(pzErrMsg, db,"cannot VACUUM - SQL statements in progress");
141838 ** in the temporary database.
141856 /* Loop through the tables in the main database. For each, do
141895 /* This array determines which meta meta values are preserved in the
141914 /* GetMeta() and UpdateMeta() cannot fail in this context because
142014 sqlite3 *db, /* Database in which module is registered */
142062 sqlite3 *db, /* Database in which module is registered */
142083 sqlite3 *db, /* Database in which module is registered */
142098 sqlite3 *db, /* Database in which module is registered */
142196 ** Table p is a virtual table. This function moves all elements in the
142200 ** connection db is left in the p->pVTable list.
142211 ** database connection that may have an entry in the p->pVTable list.
142237 ** list in p->pVTab. It also decrements the VTable ref count. This is
142261 ** Disconnect all the virtual table objects in the sqlite3.pDisconnect list.
142273 ** associated with the database the virtual table is stored in is held
142274 ** or, if the virtual table is stored in a non-sharable database, then
142308 ** in the list are moved to the sqlite3.pDisconnect list of the associated
142387 assert( iDb>=0 ); /* The database the table is being created in */
142396 ** in pParse->zArg[] and appends it to the list of arguments on the
142397 ** virtual table currently under construction in pParse->pTable.
142422 ** first time (in other words if the virtual table is actually being
142425 ** in the sqlite_schema table.
142442 /* A slot for the record has already been allocated in the
142447 ** entry in the sqlite_schema table tht was created for this vtab
142474 /* If we are rereading the sqlite_schema table create the in-memory
142476 ** the first time the virtual table is used in an SQL statement. This
142496 ** of an argument to the module name in a CREATE VIRTUAL TABLE statement.
142506 ** in an argument to the module name in a CREATE VIRTUAL TABLE statement.
142653 ** and an error left in pParse.
142723 ** of the virtual table named zTab in database iDb.
142850 ** of the virtual table named zTab in database iDb. This occurs
142890 ** of each of the virtual tables in the sqlite3.aVTrans array. The method
142892 ** the offset of the method to call in the sqlite3_module structure.
142918 ** Invoke the xSync method of all virtual tables in the sqlite3.aVTrans
142922 ** If an error message is available, leave it in p->zErrMsg.
142943 ** Invoke the xRollback method of all virtual tables in the
142952 ** Invoke the xCommit method of all virtual tables in the
142966 ** in the sqlite3.aVTrans array.
142975 ** virtual module tables in this case, so return SQLITE_LOCKED.
142988 /* If pVtab is already in the aVTrans array, return early */
143066 ** If pExpr is a column in a virtual table, then let the virtual
143090 /* Check to see the left operand is a column in a virtual table */
143106 ** with an all lower-case function name. Continue in this tradition to
143140 ** Make sure virtual table pTab is contained in the pParse->apVirtualLock[]
143142 ** array if it is missing. If pTab is already in the array, this routine
143172 ** statement in order to come into existance. Eponymous virtual table
143219 ** sqlite3DeleteTable() routine will know that it is not stored in
143228 ** Return the ON CONFLICT resolution mode in effect for the virtual
143229 ** table update operation currently in progress.
143311 ** This file was split off from where.c on 2015-06-06 in order to reduce the
143317 /************** Include whereInt.h in the middle of wherecode.c **************/
143332 ** planner logic in "where.c". These definitions are broken out into
143356 ** loop in WHERE clause.
143364 ** each term in the FROM clause (which is to say, for each of the
143385 u8 iFrom; /* Which entry in the FROM clause */
143390 int nIn; /* Number of entries in aInLoop[] */
143395 int nPrefix; /* Number of prior entires in the key */
143398 } in; /* Used when pWLoop->wsFlags&WHERE_IN_ABLE */
143428 u8 iTab; /* Position in FROM clause of table for this loop */
143450 u16 nLTerm; /* Number of entries in aLTerm[] */
143456 WhereLoop *pNextLoop; /* Next WhereLoop object in the WhereClause */
143461 ** subquery on one operand of an OR operator in the WHERE clause.
143484 ** Think of each WhereLoop object as a node in a graph with arcs
143495 ** number of nodes in the FROM clause. The best (lowest cost) WherePath
143499 Bitmask maskLoop; /* Bitmask of all WhereLoop objects in this path */
143538 ** If a term in the WHERE clause does not match either of the two previous
143541 ** but no other fields in the WhereTerm object are meaningful.
143545 ** cursor number into bits and the translated bit is stored in the prereq
143546 ** fields. The translation is used in order to maximize the number of
143547 ** bits that will fit in a Bitmask. The VDBE cursor numbers might be
143551 ** beginning with 0 in order to make the best possible use of the available
143552 ** bits in the Bitmask. So, in the example above, the cursor numbers
143555 ** The number of terms in a join is limited by the number of bits
143556 ** in prereqRight and prereqAll. The default is 64 bits, hence SQLite
143568 int leftCursor; /* Cursor number of X in "X <op> <expr>" */
143571 int leftColumn; /* Column number of X in "X <op> <expr>" */
143572 int iField; /* Field in (?,?,?) IN (SELECT...) vector */
143606 ** terms in the WHERE clause that are useful to the query planner.
143614 unsigned char nEquiv; /* Number of entries in aEquiv[] */
143615 unsigned char iEquiv; /* Next unused slot in aEquiv[] */
143618 int aiCur[11]; /* Cursors in the equivalence class */
143619 i16 aiColumn[11]; /* Corresponding column number in the eq-class */
143640 int nSlot; /* Number of entries in a[] */
143655 Bitmask indexable; /* Bitmask of all indexable tables in the clause */
143668 ** between VDBE cursor numbers and bits of the bitmasks in WhereTerm.
143670 ** The VDBE cursor numbers are small integers contained in
143673 ** contain gaps in the numbering sequence. But we want to make maximum
143674 ** use of the bits in our bitmasks. This structure provides a mapping
143715 int nRecValid; /* Number of valid fields currently in pRec */
143731 ** pathological queries can spend excess time in the sqlite3WhereBegin()
143737 ** clause is processed, so that every table in a join is guaranteed to be
143750 ** in an expression tree to cause that node to point to a column
143774 SrcList *pTabList; /* List of tables in the join */
143800 WhereLevel a[1]; /* Information about each nest loop in WHERE */
143898 ** These are definitions of bits in the WhereLoop.wsFlags field.
143899 ** The particular combination of bits in each WhereLoop help to
143928 /************** Continuing where we left off in wherecode.c ******************/
143983 ** rows scanned by the strategy in the form of an SQL expression.
144023 ** is added to the output to describe the table scan strategy in pLevel.
144162 ** Disable a term in the WHERE clause. Except, do not disable the term
144163 ** if it controls a LEFT OUTER JOIN and it did not originate in the ON
144166 ** Consider the term t2.z='ok' in the following queries:
144172 ** The t2.z='ok' is disabled in the in (2) because it originates
144173 ** in the ON clause. The term is disabled in (3) because it is not part
144176 ** Disabling a term causes that term to not be tested in the inner loop
144178 ** by indices, we disable them to prevent redundant tests in the inner
144181 ** as we can without disabling too much. If we disabled in (1), we'd get
144192 ** Only the parent term was in the original WHERE clause. The child1
144230 ** in zAff are SQLITE_AFF_BLOB or SQLITE_AFF_NONE, then no code gets generated.
144270 ** * the affinity change in zAff is guaranteed not to change the value.
144274 int n, /* Number of vector elements in comparison */
144292 ** LHS. But not all terms in the vector are indexable and the terms might
144293 ** not be in the correct order for indexing.
144297 ** the vector contains only index terms and those terms are in the correct
144309 ** Since only columns e and c can be used with the index, in that order,
144390 ** The current value for the constraint is left in a register, the index
144391 ** of which is returned. An attempt is made store the result in iTarget but
144393 ** constraint is a TK_EQ or TK_IS, then the current value might be left in
144396 ** For a constraint of the form X=expr, the expression is evaluated in
144406 int iTarget /* Attempt to leave results in this register */
144477 if( pLevel->u.in.nIn==0 ){
144484 i = pLevel->u.in.nIn;
144485 pLevel->u.in.nIn += nEq;
144486 pLevel->u.in.aInLoop =
144487 sqlite3DbReallocOrFree(pParse->db, pLevel->u.in.aInLoop,
144488 sizeof(pLevel->u.in.aInLoop[0])*pLevel->u.in.nIn);
144489 pIn = pLevel->u.in.aInLoop;
144491 int iMap = 0; /* Index in aiMap[] */
144527 pLevel->u.in.nIn = 0;
144542 ** The index has as many as three equality constraints, but in this
144546 ** in consecutive registers and the index of the first register is returned.
144555 ** an inequality constraint (such as the "c>=5 AND c<10" in the example) that
144559 ** the index of the first memory cell in that range. The code that
144568 ** sqlite3DbMalloc(). Except, entries in the copy of the string associated
144578 ** a key to search the index. Hence the first byte in the returned affinity
144579 ** string in this example would be set to SQLITE_AFF_BLOB.
144753 ** should be included in the cursor-hint for a table that is on the rhs
144816 /* An aggregate function in the WHERE clause of a query means this must
144818 ** the parent context. Do not walk the function arguments in this case.
144821 ** expression, as the result of the expression must be stored in a
144874 ** Say there is a single row in t2 that matches (t1.a=t2.b), but its
144900 /* All terms in pWLoop->aLTerm[] except pEndRange are used to initialize
144911 /* For an index scan, make sure referenced columns are actually in
144939 ** rowid stored in register iRowid.
144951 ** does not appear in the index at all, the array entry is set to 0.
144995 ** result in register iReg.
145024 ** mapping from an expression on table columns into a column in an index
145096 ** in pExpr and change that subexpression into a reference to the appropriate
145099 ** 2019-10-24: Updated to also translate references to a VIRTUAL column in
145107 WhereInfo *pWInfo /* Transform expressions in this WHERE clause */
145141 ** that are contained in the index. Pulling the generated column
145188 ** Generate code for the start of the iLevel-th loop in the WHERE clause
145202 int bRev; /* True if we need to scan in reverse order */
145211 int iRowidReg = 0; /* Rowid is stored in this register, if not zero */
145247 ** there are no IN operators in the constraints, the "addrNxt" label
145318 iIn = pLevel->u.in.nIn;
145334 ** encoding of the value in the register, so it *must* be reloaded. */
145335 assert( pLevel->u.in.aInLoop!=0 || db->mallocFailed );
145337 assert( iIn>=0 && iIn<pLevel->u.in.nIn );
145338 pOp = sqlite3VdbeGetOp(v, pLevel->u.in.aInLoop[iIn].addrInTop);
145365 /* These registers need to be preserved in case there is an IN operator
145535 ** constraints but an index is selected anyway, in order
145586 /* Like optimization range constraints always occur in pairs */
145596 assert( pRangeStart->wtFlags & TERM_LIKEOPT ); /* occur in pairs */
145660 ** and store the values of those terms in an array of registers
145732 ** of entries in the tree, so basing the number of steps to try
145733 ** on the estimated number of rows in the btree seems like a good
145865 ** all the expressions in pWInfo and try to transform matching expressions
145867 ** to virtual columns in the table into references to (stored) columns
145937 ** Null 1 # Zero the rowset in reg 1
145952 ** Null 1 # Zero the rowset in reg 1
145989 /* Set up a new SrcList in pOrTab containing the table being scanned
145990 ** by this loop in the a[0] slot and all notReady tables in a[1..] slots.
145991 ** This becomes the SrcList in the recursive call to sqlite3WhereBegin().
146013 ** capable of holding primary keys in the case of a WITHOUT ROWID.
146017 ** is required in a few obscure LEFT JOIN cases where control jumps
146039 ** That way, terms in y that are factored into the disjunction will
146043 ** the "interesting" terms of z - terms that did not originate in the
146048 ** is not contained in the ON clause of a LEFT JOIN.
146108 ** row will be skipped in subsequent sub-WHERE clauses.
146131 ** the row has already been included in the result set and
146137 ** is zero, assume that the key cannot already be present in
146176 ** If the call to sqlite3WhereBegin() above resulted in a scan that
146228 /* Tables marked isRecursive have only a single row that is stored in
146421 ** analyzing Expr objects in the WHERE clause.
146448 ** The index in pWC->a[] of the new WhereTerm is returned on success.
146450 ** allocation error. The memory allocation failure will be recorded in
146574 ** that virtual tables cannot participate in the LIKE optimization.) The
146582 int *pisComplete, /* True if the only wildcard is % in the last character */
146588 u8 c; /* One character in z[] */
146660 ** Getting this right has been a persistent source of bugs in the
146742 ** "expr" expression (even though in forms (6) and (8) the column is on the
146746 ** of virtual table in forms (5) or (7) then return 2.
146776 /* Built-in operators MATCH, GLOB, LIKE, and REGEXP attach to a
146778 ** the left-hand side operand in their in-fix form.
146804 ** with function names in an arbitrary case.
146852 ** If the pBase expression originated in the ON or USING clause of
146888 ** two subterms are in disjunction - they are OR-ed together.
146916 int idxNew; /* Index in pWC of the next virtual term */
146948 ** subterms. So in:
146953 ** This routine analyzes terms such as the middle term in the above example.
146974 ** a single table T (as shown in example B above) then create a new virtual
146992 ** The added conjunct can sometimes be helpful in query planning.
147017 ** always prefer case 1, so in that case we pretend that case 3 is not
147026 ** to "bitmap indices" in other database engines.
147052 ** stored in a WhereClause structure containing within the WhereOrInfo
147157 ** IN operator because one or more terms in the OR clause contain
147158 ** something other than == on a column in the single table. The 1-bit
147178 ** other of the == operator in every subterm. That table and column
147179 ** will be recorded in iCursor and iColumn. There might not be any
147197 /* This term must be of the form t1.a==t2.b where t2 is in the
147222 ** table and column is common to every term in the OR clause */
147295 ** 3. Not originating in the ON clause of an OUTER JOIN
147300 ** for the LHS anyplace else in the WHERE clause where the LHS column occurs.
147324 ** a bitmask indicating which tables are used in that expression
147354 ** in any index. Return TRUE (1) if pExpr is an indexed term and return
147446 ** If there is no sub-select in pExpr, then return true if pExpr
147448 ** or is not (bUses==0) in pSrc.
147459 ** nodes reference tables not found in pSrc
147462 ** that references a table not in pSrc.
147482 SrcList *pSrc; /* The tables in an EXISTS(SELECT ... FROM <here> ...) */
147490 ** Iterate through all AND connected nodes in the expression tree
147688 ** "pExpr" field filled in. The job of this routine is to analyze the
147954 ** lower-case (upper-case is less than lower-case in ASCII) so that
148127 ** This routine identifies subexpressions in the WHERE clause where
148129 ** operator specified in the op parameter. The WhereClause structure
148136 ** The original WHERE clause in pExpr is unaltered. All this routine
148139 ** In the previous sentence and in the diagram, "slot[]" refers to
148198 ** a bitmask indicating which tables are used in that expression
148245 ** Call exprAnalyze on all terms in a WHERE clause.
148267 ** a HIDDEN column in the table.
148377 ** to emit rows in increasing order, and if the last row emitted by the
148380 ** will not fit in the sorter either) and continue with the second inner
148383 ** When a row does not fit in the sorter (because the sorter already
148393 ** inner-most loop, in the sense that a correct answer will result.
148437 ** Return the VDBE address or label to jump to in order to continue
148446 ** Return the VDBE address or label to jump to in order to break
148540 ** iCursor is not in the set.
148556 ** There is one cursor per table in the FROM clause. The number of
148557 ** tables in the FROM clause is limited by a test early in the
148732 ** Search for a term in the WHERE clause that is of the form "X <op> <expr>"
148741 ** The term returned might by Y=<expr> if there is another constraint in
148743 ** identified by the WO_EQUIV bit in the pTerm->eOperator field. The
148745 ** slots in aiCur[]/aiColumn[] so that means we can look for X plus up to 10
148749 ** If there are multiple terms in the WHERE clause of the form "X <op> <expr>"
148750 ** then try for the one with no dependencies on <expr> - in other words where
148752 ** the form "X <op> Y" where Y is a column in another table if no terms of
148787 ** If such an expression is found, its index in pList->a[] is returned. If
148840 ** A DISTINCT list is redundant if any subset of the columns in the
148854 /* If there is more than one table or sub-select in the FROM clause of
148876 ** 2. All of the columns in the index are either part of the pDistinct
148910 ** Convert OP_Column opcodes to OP_Copy in previously generated code.
148918 ** iAutoidxCur cursor, in order to generate unique rowids for the
148925 int iRegister, /* The first column is in this register */
149006 Bitmask notReady /* Tables in outer loops of the join */
149043 int nKeyCol; /* Number of columns in the constructed index */
149054 int mxBitCol; /* Maximum column in pSrc->colUsed */
149278 /* If the ORDER BY clause contains only columns in the current
149349 /* The direct assignment in the previous line is possible only because
149386 ** comes in as the 3rd argument to this function.
149391 ** the current configuration of "unusable" flags in sqlite3_index_info can
149392 ** not result in a valid plan.
149423 ** Estimate the location of a particular key among all keys in an
149424 ** index. Store the results in aStat as follows:
149432 ** based on the contents of aSample[] and the number of fields in record
149443 int iCol; /* Index of required stats in anEq[] etc. */
149449 int nField; /* Number of fields in pRec */
149461 ** is simply the aSample[] array. If the samples in aSample[] contain more
149465 ** samples in aSample[] (truncated to N fields), the search also has to
149467 ** in aSample is:
149490 ** For each sample in the aSample[] array, N samples are present in the
149496 ** equal to the previous sample in the array. For example, in the above,
149498 ** appears that it should be 1 field in size. However, that would make it
149507 int iSamp; /* Index in aSample[] of test sample */
149508 int n; /* Number of fields in test sample */
149554 ** all samples in the aSample[] array, pRec must be smaller than the
149563 ** in the aSample[] array. Otherwise, if (iCol>0) then pRec must
149588 ** is larger than all samples in the array. */
149671 ** extracted from pLower and pUpper with the corresponding column in each
149742 ** using the method described in the header comment for this function. */
149775 ** If either of the upper or lower bound is not present, then NULL is passed in
149778 ** The value in (pBuilder->pNew->u.btree.nEq) is the number of the index
149795 ** rows in the index. Assuming no error occurs, *pnOut is adjusted (reduced)
149827 /* Variable iLower will be set to the estimate of the number of rows in
149831 ** columns of the index, and $L is the value in pLower.
149835 ** range is $P. Due to a quirk in the way whereKeyStats() works, even
149861 ** have been requested when testing key $P in whereEqualScanEst(). */
149916 ** the estimated selectivity more in line with what it would be
149946 ** range query (e.g. col > ?) is assumed to match 1/4 of the rows in the
149969 ** an equality constraint x=VALUE and where that VALUE occurs in
149982 ** in the pParse structure.
149987 Expr *pExpr, /* Expression for VALUE in the x=VALUE constraint */
150046 ** in the pParse structure.
150279 sqlite3DbFree(db, pLevel->u.in.aInLoop);
150375 ** Search the list of WhereLoops in *ppPrev looking for one that can be
150463 ** information is gathered in the pBuilder->pOrSet object. This special
150601 ** value corresponds to -1 in LogEst notation, so this means decrement
150607 ** of rows in the table. In other words, assume that x==EXPR will filter
150610 ** on the "x" column and so in that case only cap the output row estimate
150616 LogEst nRow /* Number of rows in the entire table */
150668 ** in the vector can be optimized using column nEq of the index. This
150680 ** then this function would be invoked with nEq=1. The value returned in
150776 LogEst rSize; /* Number of rows in the table */
150831 ** be used by the right table of a LEFT JOIN. Only constraints in the
150870 ** first such term in use, and sets nIn back to 0 if it is not. */
150881 ** N = the total number of rows in the table
150883 ** M = the number of rows in the table that match terms to the
150884 ** to the left in the same index. If the IN operator is on
150893 ** Our estimates for M, K, and N might be inaccurate, so we build in
150898 ** tables (less than 2 rows) as it is pointless in that case.
150943 ** always used in pairs. */
151010 ** to be true for half or more of the rows in the table.
151047 /* Set rCostIdx to the cost of visiting selected rows in index. Add
151050 ** visiting the rows in the main table. */
151092 ** number of repeats in the left-most terms is at least 18.
151096 ** contains fewer than 2^17 rows we assume otherwise in other parts of
151117 /* TUNING: Because uncertainties in the estimates for skip-scan queries,
151133 ** Return True if it is possible that pIndex might be useful in
151134 ** implementing the ORDER BY clause in pBuilder.
151137 ** if there is no way for pIndex to be useful in implementing that
151172 ** in the current query. Return true if it can be and false if not.
151226 ** WHERE clause includes "x IN (....)" terms used in place of "x=?". Or when
151232 ** log(nRow) factor is omitted from a non-covering index scan in order to
151233 ** bias the scoring in favor of using an index, since the worst-case
151252 LogEst rSize; /* number of rows in the table */
151253 LogEst rLogSize; /* Logarithm of the number of rows in the table */
151271 /* There is no INDEXED BY clause. Create a fake Index object in local
151273 ** fake index the first in a chain of Index objects with all of the real
151321 ** estimated to be X*N*log2(N) where N is the number of rows in
151336 /* TUNING: Each index lookup yields 20 rows in the table. This
151393 ** to use an index in that case, since the constraint might be for
151394 ** the scarcer of the two values, and in that case an index lookup is
151437 /* If this is a non-covering index scan, add in the cost of
151501 ** * It is not one of the operators specified in the mExclude mask passed
151502 ** as the fourth argument (which in practice is either WO_IN or 0).
151505 ** virtual table in question. These are added to the plans prerequisites
151565 ** Make no entries in the loop table.
151690 ** If there are no LEFT or CROSS JOIN joins in the query, both mPrereq and
151692 ** entries that occur before the virtual table in the FROM clause and are
151704 ** All the tables in mPrereq must be scanned before the current virtual
151706 ** mPrereq may be specified as "usable" in all calls to xBestIndex.
151707 ** Conversely, all tables in mUnusable must be scanned after the current
151722 int nConstraint; /* Number of constraints in p */
151755 ** and does not use an IN(...) operator, then there is no point in making
151779 ** in the set of terms that apply to the current virtual table. */
151803 /* If the calls to xBestIndex() in the above loop did not find a plan
151970 /* Loop over the tables in the join, from left to right */
152021 ** parameters) to see if it outputs rows in the requested ORDER BY
152031 ** and DISTINCT do not require rows to appear in any particular order as long
152033 ** the pOrderBy terms can be matched in any order. With ORDER BY, the
152034 ** pOrderBy terms must be matched in strict left-to-right order.
152041 u16 nLoop, /* Number of entries in pPath->aLoop[] */
152043 Bitmask *pRevMask /* OUT: Mask of WhereLoops to run in reverse order */
152052 u16 nKeyCol; /* Number of key columns in pIndex */
152053 u16 nColumn; /* Total number of ordered columns in the index */
152054 u16 nOrderBy; /* Number terms in the ORDER BY clause */
152055 int iLoop; /* Index of WhereLoop in pPath being processed */
152076 ** Every one-row WhereLoop will have the WHERE_ONEROW bit set in wsFlags.
152079 ** that WhereLoop that are in the ORDER BY clause are different for every
152081 ** order-distinct. A WhereLoop that has no columns in the ORDER BY clause
152088 ** rowid appears in the ORDER BY clause, the corresponding WhereLoop is
152124 /* Mark off any ORDER BY term X that is a column in the table of
152125 ** the current loop for which there is term in the WHERE
152139 /* IN terms are only valid for sorting in the ORDER BY LIMIT
152193 ** terms imply that the index is not UNIQUE NOT NULL in which case
152227 /* Get the column number in the table (iColumn) and sort order
152282 /* Make sure the sort order is compatible in an ORDER BY clause.
152351 ** If the WHERE_GROUPBY flag is set in the mask passed to sqlite3WhereBegin(),
152356 ** Normally, in this case it is not possible for the caller to determine
152357 ** whether or not the rows are really being delivered in sorted order, or
152358 ** just in some other order that provides the required grouping. However,
152361 ** true if the rows really will be sorted in the specified order, or false
152393 ** nOrderby columns and that the first nSorted columns are already in
152443 ** will be nRowEst (in the 10*log2 representation). Or, ignore sorting
152451 int nLoop; /* Number of terms in the join */
152460 int nTo, nFrom; /* Number of valid entries in aTo[] and aFrom[] */
152509 ** the ORDER BY clause are already in order, where X is the array
152526 /* If nLoop is zero, then there are no FROM terms in the query. Since
152527 ** in this case the query may return a maximum of one row, the results
152528 ** are already in the requested order. Set isOrdered to nOrderBy to
152555 ** index is useful in the outer loop of a correlated subquery. */
152581 ** where the rows emerge in the correct order without any sorting
152591 rUnsorted -= 2; /* TUNING: Slight bias in favor of no-sort plans */
152619 ** paths currently in the best-so-far buffer. So discard
152949 ** Display all WhereLoops in pWInfo
152973 ** in order to complete the WHERE clause processing.
152977 ** The basic idea is to do a nested loop, one loop for each table in
152979 ** same as a SELECT with only a single table in the FROM clause.) For
152986 ** foreach row1 in t1 do \ Code generated
152987 ** foreach row2 in t2 do |-- by sqlite3WhereBegin()
152988 ** foreach row3 in t3 do /
152994 ** Note that the loops might not be nested in the order in which they
152995 ** appear in the FROM clause if a different order is better able to make
152996 ** use of indices. Note also that when the IN operator appears in
152997 ** the WHERE clause, it might result in additional nested loops for
153006 ** in pTabList pointing at their appropriate entries. The [...] code
153012 ** the tables have indices and there are terms in the WHERE clause that
153018 ** make it to the "..." in the middle of the loop. After each "foreach",
153019 ** terms of the WHERE clause that use only terms in that loop and outer
153028 ** foreach row1 in t1 do
153030 ** foreach row2 in t2 do
153044 ** if the WHERE_GROUPBY flag is set in wctrlFlags) of a SELECT statement
153052 ** the first cursor in an array of cursors for all indices. iIdxCur should
153062 u16 wctrlFlags, /* The WHERE_* flags defined in sqliteInt.h */
153067 int nTabList; /* Number of elements in pTabList */
153073 WhereLevel *pLevel; /* A single level in pWInfo->a[] */
153104 /* The number of tables in the FROM clause is limited by the number of
153105 ** bits in a Bitmask
153109 sqlite3ErrorMsg(pParse, "at most %d tables in a join", BMS);
153113 /* This function normally generates a nested loop for all tables in
153115 ** only generate code for the first table in pTabList and assume that
153175 /* Assign a bit from the bitmask to every term in the FROM clause.
153184 ** Note that bitmasks are created for all pTabList->nSrc tables in
153210 /* Special case: WHERE terms that do not refer to any tables in the join
153216 ** preserves SQLite's legacy behaviour in the following two cases:
153271 /* If one or more WhereTerm.truthProb values were used in estimating
153457 /* Open all tables in the pTabList and any indices selected for
153497 ** it is advantageous to reduce the "column count" field in
153713 if( pLoop->wsFlags & WHERE_IN_ABLE && pLevel->u.in.nIn>0 ){
153717 for(j=pLevel->u.in.nIn, pIn=&pLevel->u.in.aInLoop[j-1]; j>0; j--, pIn--){
153806 ** the co-routine into OP_Copy of result contained in a register.
153844 ** Calls to the code generator in between sqlite3WhereBegin and
153866 printf("TRANSLATE opcodes in range %d..%d\n", k, last-1);
153957 ** Any SELECT statement that contains one or more window functions in
153960 ** in order to support window function processing. For example, with the
153977 ** (in this case max()) to process rows sorted in order of (c, d), which
153986 ** select-list and ORDER BY expressions in the parent query are
153990 ** If there is more than one window function in the SELECT that uses
153998 ** is transformed in the same way as the example above. However:
154012 ** so that both min() and max() may process rows in the order defined by
154017 ** When processing the rewritten SELECT statement, code in select.c calls
154028 ** in the associated Window.regResult registers.
154030 ** * The required terminal values are stored in the current row of
154035 ** in a temp table before returning any rows. In other cases it does not.
154037 ** select.c is the same in either case.
154041 ** This implementation features the following built-in window functions:
154055 ** These are the same built-in window functions supported by Postgres.
154058 ** be implemented using an API, built-in window functions are much more
154060 ** may only be implemented by caching the entire partition in memory.
154061 ** As such, some built-in window functions use the same API as aggregate
154072 ** As well as some of the built-in window functions, aggregate window
154079 ** Implementation of built-in window function row_number(). Assumes that the
154110 ** Implementation of built-in window function dense_rank(). Assumes that
154139 ** Implementation of built-in window function nth_value(). This
154140 ** implementation is used in "slow mode" only - when the EXCLUDE clause
154229 ** Implementation of built-in window function rank(). Assumes that
154260 ** Implementation of built-in window function percent_rank(). Assumes that
154305 ** Implementation of built-in window function cume_dist(). Assumes that
154348 i64 nTotal; /* Total rows in partition */
154478 ** Static names for the built-in window function names. These static
154499 ** for built-in window functions that never call those interfaces.
154545 ** Register those built-in window functions that are not also aggregates.
154587 ** * If the OVER clause refered to a named window (as in "max(x) OVER win"),
154590 ** in pParse.
154592 ** * If the function is a built-in window function that requires the
154683 ** expression (*ppExpr) in place.
154781 ** Iterate through each expression in expression-list pEList. For each:
154790 ** pWin->iEphCsr, where N is the number of elements in (*ppSub) after
154797 ExprList *pEList, /* Rewrite expressions in this list */
154824 ** Append a copy of each expression in expression-list pAppend to
154859 ** When rewriting a query, if the new subquery in the FROM clause
154864 ** See also the incrAggDepth() routine in resolve.c
154879 ** are invoked in the correct order as described under "SELECT REWRITING"
154974 ** keep everything legal in this case.
154986 ("New window-function subquery in FROM clause of (%u/%p)\n",
154997 /* Might actually be some other kind of error, but in that case
155073 ** variable values in the expression tree.
155114 ** starting boundary type may not occur earlier in the following list than
155182 ** stored in the linked list starting at pWin->pNextWin. This function
155184 ** leaves an error in pParse.
155240 ** in if either (a) there are no other windows already linked to this
155263 ** or not. Identical window objects can be processed in a single scan.
155290 ** This is called by code in select.c before it calls sqlite3WhereBegin()
155374 ** evaluated and the result left in register reg. This function generates VM
155438 int reg; /* First in array of peer values */
155450 ** in an array of accumulator registers - one for each window function
155451 ** in the WindowCodeArg.pMWin list.
155455 ** that it processes in a temporary table. If it is not zero, this
155456 ** variable indicates when rows may be removed from the temp table (in
155465 ** callbacks have been invoked in it.
155469 ** callbacks have been invoked in it.
155476 ** The windows functions implmentation caches the input rows in a temp
155492 ** Depending on the window-frame in question, all three cursors may not
155498 Window *pMWin; /* First in list of functions being processed */
155502 int regArg; /* First in array of accumulator registers */
155534 ** xInverse (if bInverse is non-zero) for each window function in the
155535 ** linked list starting at pMWin. Or, for built-in window functions
155540 ** the first register in an array of registers guaranteed to be large
155549 ** number of rows in the current partition.
155569 /* All OVER clauses in the same window function aggregate step must
155661 ** (bFin==1) for each window function in the linked list starting at
155662 ** pMWin. Or, for built-in window-functions that do not use the standard
155699 ** Generate code to calculate the current values of all window functions in the
155701 ** results in the Window.regResult registers, ready to return the upper
155798 ** Invoke the sub-routine at regGosub (generated by code in select.c) to
155802 ** for per-row processing is only generated for the following built-in window
155882 ** in the linked list passed as the second argument to NULL. And perform
155883 ** any equivalent initialization required by any built-in window functions
155884 ** in the list.
155915 ** Return true if the current frame should be cached in the ephemeral table,
155935 ** regOld and regNew are each the first register in an array of size
155947 int regNew, /* First in array of new values */
155948 int regOld, /* First in array of old values */
155970 ** the ORDER BY term in the window, and that argument op is OP_Ge, it generates
155976 ** operator in the above pseudo-code is replaced with ">" or "<=", respectively.
155978 ** If the sort-order for the ORDER BY term in the window is DESC, then the
156029 ** This block adds (or subtracts for DESC) the numeric value in regVal
156052 ** reg2 are NULL are handled separately in the following block. The code
156357 ** // rows. The first row in the partition, which all three cursors
156396 ** cases because they change the order in which the three cursors (csrStart,
156525 ** The new "continue" statement in the above jumps to the next iteration
156531 ** 1. The else branch in the main loop is only taken if the row just
156555 ** deal in groups instead of rows. As for ROWS and GROUPS, there are three
156678 int addrEmpty; /* Address of OP_Rewind in flush: */
156680 int regRecord; /* regNew array in record form */
156681 int regRowid; /* Rowid for regRecord in eph table */
156703 /* Fill in the context object */
156748 ** samve values in record form, and the rowid used to insert said record
156780 ** a record in register regRecord. */
156789 ** If so, it does an OP_Gosub to an address to be filled in later. The
156790 ** address of the OP_Gosub is stored in local variable addrGosubFlush. */
157017 ** Disable all error recovery processing in the parser push-down
157099 ** all elements in the list. And make sure list length does not exceed
157125 sqlite3ErrorMsg(pParse, "too many terms in compound SELECT");
157216 # error too many tokens in the grammar
157421 ** that indicate what to do in response to the next
157434 ** which is sqlite3ParserTOKENTYPE. The entry in the union
157447 ** YYNRULE the number of rules in the grammar
157519 ** Applications can choose to define yytestcase() in the %include section
157520 ** to a macro that can assist in verifying code coverage. For production
157548 ** slots in the yy_action[] table.
157564 ** a reduce action) then the yy_reduce_ofst[] array is used in place of
157567 ** The following are the tables generated in this section:
157570 ** yy_lookahead[] A table containing the lookahead for each entry in
158181 ** appears in the grammar, then ID becomes a fallback token for X, Y,
158186 ** This feature can be used, for example, to cause some keywords in a language
158187 ** to revert to identifiers if they keyword does not apply in the context where
158387 ** the information used by the action routines in the grammar.
158395 YYACTIONTYPE stateno; /* The state-number, or reduce action in SHIFTREDUCE */
158403 /* The state of the parser is completely contained in an instance of
158421 yyStackEntry *yystackEnd; /* Last entry in the stack */
159223 ** putting an appropriate #define in the %include section of the input
159268 ** A pointer to a parser. This pointer is used in subsequent calls
159447 ** is defined in a %include section of the input grammar) then it is
159474 ** is in state X and has a lookahead token Y. In a well-tested
159652 YYACTIONTYPE yyNewState, /* The new state to shift in */
159653 YYCODETYPE yyMajor, /* The major token to shift in */
159654 sqlite3ParserTOKENTYPE yyMinor /* The minor token to shift in */
160504 ** only called from one place, optimizing compilers will in-line it, which
161253 /* When doing a nested parse, one can include terms in an expression
161255 ** in the virtual machine. #N is the N-th register. */
162078 ** user wants (and specified in the grammar) and is available for
162119 fprintf(yyTraceFILE,"%sInput '%s' in state %d\n",
162352 #define CC_KYWD 2 /* Alphabetics or '_'. Usable in a keyword */
162377 #define CC_ID 27 /* unicode characters usable in IDs */
162463 ** mkkeywordhash.c, located in the tool subdirectory of the distribution.
162468 /************** Include keywordhash.h in the middle of tokenize.c ************/
162472 ** The code in this file has been automatically generated by
162476 ** The code in this file implements a function that determines whether
162484 /* zKWText[] encodes 1007 bytes of keyword text in 667 bytes */
162566 /* aKWLen[i] is the length (in bytes) of the i-th keyword */
162954 /************** Continuing where we left off in tokenize.c *******************/
162958 ** If X is a character that can be used in an identifier then
162962 ** allowed in an identifier. For 7-bit characters,
162968 ** Ticket #1066. the SQL standard does not allow '$' in the
162970 ** SQLite will allow '$' in identifiers for compatibility.
163000 ** Return the id of the next token in string (*pz). Before returning, set
163027 ** the ambiguity in some constructions. e.g.
163076 ** Return the length (in bytes) of the token that begins at z[0].
163077 ** Store the token type in *tokenType before returning.
163313 /* This token started out using characters that can appear in keywords,
163361 ** passed in. An SQLITE_ status code is returned. If an error occurs
163437 ** with tokens TK_SEMI and 0, in that order. */
163492 sqlite3_log(pParse->rc, "%s in \"%s\"",
163515 ** structure built up in pParse->pNewTable. The calling code (see vtab.c)
163544 ** the normalization in space obtained from sqlite3DbMalloc(). Or return
163557 int iStartIN; /* Start of RHS of IN operator in z[] */
163689 ** This is defined in tokenize.c. We just have to import the definition.
163718 ** Return TRUE if the given SQL string ends in a semicolon.
163729 ** returns 1 if it ends in the START state and 0 if it ends
163730 ** in any other state.
163732 ** (2) NORMAL We are in the middle of statement which ends with a single
163742 ** (5) TRIGGER We are in the middle of a trigger definition that must be
163745 ** (6) SEMI We've seen the first semicolon in the ";END;" that occurs at
163771 u8 state = 0; /* Current state, using numbers defined in header comment */
163846 case '[': { /* Microsoft-style identifiers in [...] */
163971 ** Main file for the SQLite library. The routines in this file
163972 ** implement the programmer interface to the library. Routines in
163979 /************** Include fts3.h in the middle of main.c ***********************/
164009 /************** Continuing where we left off in main.c ***********************/
164012 /************** Include rtree.h in the middle of main.c **********************/
164046 /************** Continuing where we left off in main.c ***********************/
164049 /************** Include sqliteicu.h in the middle of main.c ******************/
164079 /************** Continuing where we left off in main.c ***********************/
164115 ** built-in extensions.
164325 ** the pInitMutex mutex. Return an error in either case. */
164340 ** methods. The sqlite3_pcache_methods.xInit() all is embedded in the
164426 ** while any part of SQLite is otherwise in use in any thread. This
164462 ** this function resulted in the heap subsystem actually being shutdown.
164490 ** the SQLite library is in use. */
164496 /* Mutex configuration options are only available in a threadsafe
164545 ** low-level memory allocation routines to be used in place of the memory
164584 ** in SQLITE_CONFIG_PAGECACHE. */
164629 ** number of bytes in the memory buffer, and the minimum allocation size.
164705 ** boolean in order to enable or disable the use of covering indices for
164706 ** full table scans in the query optimizer. */
164797 ** The sz parameter is the number of bytes in each lookaside slot.
164938 ** Flush any dirty pages in the pager-cache for any attached database
164992 u32 mask; /* Mask of the bit in sqlite3.flags to set/clear */
165053 /* EVIDENCE-OF: R-65033-28449 The built-in BINARY collation compares
165081 ** Return true if CollSeq is the default built-in BINARY.
165089 ** Another built-in collating sequence: NOCASE.
165093 ** extends only to the 26 characters used in the English language.
165140 ** Return the number of changes in the most recent call to sqlite3_exec().
165266 ** tables in the db->aVTrans[] array. The following sqlite3VtabRollback()
165398 /* Tell the code in notify.c that the connection no longer holds any
165463 ** any write cursors are invalidated ("tripped" - as in "tripping a circuit
165466 ** but are "saved" in case the table pages are moved around.
165476 ** This is important in case the transaction being rolled back has
165478 ** here, then another shared-cache connection might sneak in between
165480 ** corruption reports in some cases. */
165515 ** specified in the argument.
165626 ** Return a static string that describes the kind of error specified in the
165693 ** an integer number of milliseconds passed in as the first
165731 ** must be done in increments of whole seconds */
165852 ** that if a malloc() fails in sqlite3_create_function(), an error code
166100 ** fails with an error message stating that the function is used in the
166115 "unable to use function %s in the requested context", zName);
166129 ** A global function must exist in order for name resolution to work
166341 ** Invoke sqlite3_wal_checkpoint if the number of frames in the log file
166363 ** more frames in the log file. Passing zero or a negative value as the
166423 int *pnLog, /* OUT: Size of WAL log in frames */
166436 /* Initialize the output variables to -1 in case an error occurs. */
166492 ** not currently open in WAL mode.
166670 ** Return a string that describes the kind of error specified in the
166756 ** initializer must be kept in sync with the SQLITE_LIMIT_*
166757 ** #defines in sqlite3.h.
166834 ** macro called SQLITE_MAX_NAME. (The "_LIMIT_" in the name is changed to
166875 ** the default flags to open the database handle with. The value stored in
166882 ** stored in *pzFile is a database name acceptable to sqlite3_uri_parameter()
166883 ** and is in the same format as names created using sqlite3_create_filename().
166885 ** the value returned in *pzFile to avoid a memory leak.
166981 ** case we ignore all text in the remainder of the path, name or
166993 /* If ENABLE_URI_00_ERROR is defined, "%00" in a URI is an error. */
166994 *pzErrMsg = sqlite3_mprintf("unexpected %%00 in uri");
167188 ** dealt with in the previous code block. Besides these, the only
167320 ** EVIDENCE-OF: R-52786-44878 SQLite defines three built-in collating
167334 ** Only allow sensible combinations of bits in the flags argument.
167337 ** assert() statements in deeper layers. Sensible combinations
167394 /* Register all built-in functions, but do not attempt to read the
167403 /* Load compiled-in extensions */
167498 char const *zFilename8; /* zFilename encoded in UTF-8 instead of UTF-16 */
167650 ** Test to see whether or not the database connection is in autocommit
167670 ** 1. Serve as a convenient place to set a breakpoint in a debugger
167726 ** See comment in sqlite3.h (sqlite.h.in) for details.
167765 /* Locate the table in question */
167794 ** to the caller in local variables zDataType, zCollSeq, notnull, primarykey
167852 /* This function works in milliseconds, but the underlying OsSleep()
167958 ** schema cookie in the main database for db, or
168041 ** Set the PENDING byte to the value in the argument, if X>0.
168045 ** IMPORTANT: Changing the PENDING byte from 0x40000000 results in
168047 ** while any database connection is open results in undefined and
168136 ** is obtained in every case.
168157 ** the database connection given in the argument.
168170 ** testing causes certain assert() statements in the code to be activated
168188 ** in production.
168197 ** too big to test in a reasonable amount of time, so this control is
168246 ** run a single CREATE TABLE statement to construct the imposter table in
168271 ** state/lookahead combinations in the parser state machine
168289 ** inject an intreal() value at arbitrary places in an SQL statement,
168319 ** op==0 Store the current sqlite3SelectTrace in *ptr
168321 ** op==3 Store the current sqlite3WhereTrace in *ptr
168343 ** consecutively in memory, in that order. The database filename is prefixed
168347 ** This only works if the filename passed in was obtained from the Pager.
168358 ** character after then zero terminator on the new text in p[].
168704 /* Since nOpt is normally in single digits, a linear search is
168779 ** 1) Each entry in the list has a non-NULL value for either
168782 ** 2) All entries in the list that share a common value for
168785 ** 3) If the argument db is not NULL, then none of the entries in the
168864 ** (call it pOther) in the same process was busy using the same shared
168944 ** This function loops through each entry in the blocked connections
168960 int nArg = 0; /* Number of entries in aArg[] */
168969 /* This loop runs once for each entry in the blocked-connections list. */
169084 ** The code in this file is only compiled if:
169087 ** (in which case SQLITE_CORE is not defined), or
169090 ** SQLite (in which case SQLITE_ENABLE_FTS3 is defined).
169093 /* The full-text index is stored in a series of b+tree (-like)
169095 ** structures are like b+trees in layout, but are constructed from the
169096 ** bottom up in optimal fashion and are not updatable. Since trees
169103 ** varint. We encode variable-length integers in little-endian order
169115 ** This is similar in concept to how sqlite encodes "varints" but
169117 ** are are limited to 9 bytes in length whereas FTS3 varints are
169118 ** little-endian and can be up to 10 bytes in length (in theory).
169154 ** Here, array { X } means zero or more occurrences of X, adjacent in
169155 ** memory. A "position" is an index of a token in the token stream
169157 ** in the same logical place as the position element, and act as sentinals
169166 ** The 123 value is the first docid. For column zero in this document
169204 ** Here, array { X } means zero or more occurrences of X, adjacent in
169207 ** Leaf nodes are broken into blocks which are stored contiguously in
169208 ** the %_segments table in sorted order. This means that when the end
169209 ** of a node is reached, the next term is in the node with the next
169214 ** larger than STANDALONE_MIN (default 1024) is placed in a standalone
169250 ** means zero or more occurrences of X, adjacent in memory.
169272 ** The segment directory in table %_segdir stores meta-information for
169284 ** The meta-information in the segment directory is:
169299 ** merged in batches. Each increase in level represents exponentially
169310 ** A segment merge traverses all segments at a given level in
169312 ** leaf nodes are written in to the %_segments table in order, this
169320 ** a tiny bit slower (perhaps due to more overhead in merge-time
169358 /************** Include fts3Int.h in the middle of fts3.c ********************/
169404 /************** Include fts3_tokenizer.h in the middle of fts3Int.h **********/
169443 ** implementation. The xCreate() function in turn returns an
169465 ** Create a new tokenizer. The values in the argv[] array are the
169519 ** of this buffer in bytes. The input text that generated the token is
169520 ** identified by the byte offsets returned in *piStartOffset and
169522 ** byte of the token in the input buffer. *piEndOffset should be set
169523 ** to the index of the first byte just past the end of the token in
169537 int *piStartOffset, /* OUT: Byte offset of token in input buffer */
169538 int *piEndOffset, /* OUT: Byte offset of end of token in input buffer */
169569 /************** Continuing where we left off in fts3Int.h ********************/
169570 /************** Include fts3_hash.h in the middle of fts3Int.h ***************/
169584 ** used in SQLite. We've modified it slightly to serve as a standalone
169606 int count; /* Number of entries in this table */
169608 int htsize; /* Number of buckets in the hash table */
169615 /* Each element in the hash table is an instance of the following
169622 Fts3HashElem *next, *prev; /* Next and previous elements in the table */
169632 ** is respected in comparisons.
169679 ** Number of entries in a hash table
169686 /************** Continuing where we left off in fts3Int.h ********************/
169708 ** This is the maximum amount of data (in bytes) to store in the
169710 ** populated as documents are inserted/updated/deleted in a transaction
169718 ** Macro to return the number of elements in an array. SQLite has a
169720 ** a collision when building an amalgamation with built-in FTS3.
169742 ** in the document set and zero or more prefix indexes. All indexes are stored
169743 ** as one or more b+-trees in the %_segments and %_segdir tables.
169746 ** value stored in the "%_segdir.level" column. Given this value L, the index
169879 int nColumn; /* number of named columns in virtual table */
169902 u8 bDescIdx; /* True if doclists are in reverse order */
169912 ** The current language id is stored in variable iPrevLangid.
169915 ** there is an entry in the aIndex[] array. Index 0 is an index of all the
169916 ** terms that appear in the document set. Each subsequent index in aIndex[]
169951 /* Number of segments in a level */
169974 sqlite3_stmt *pStmt; /* Prepared statement in use by the cursor */
169977 int nPhrase; /* Number of matchable phrases in query */
169983 u8 bDesc; /* True to sort in descending order */
169985 int nRowAvg; /* Average size of database rows, in pages */
169986 sqlite3_int64 nDoc; /* Documents in table */
169989 int isMatchinfoNeeded; /* True when aMatchinfo[] needs filling in */
170001 ** of the column to be searched. For example, in
170028 int nAll; /* Size of a[] in bytes */
170038 ** A "phrase" is a sequence of one or more tokens that must match in
170040 ** For a sequence of tokens contained in double-quotes (i.e. "one two three")
170041 ** nToken will be the number of tokens in the string.
170045 int n; /* Number of bytes in buffer z */
170050 ** parsed (by code in fts3_expr.c). Below this point the variables are
170070 int nToken; /* Number of tokens in the phrase */
170072 Fts3PhraseToken aToken[1]; /* One entry for each token in the phrase */
170080 ** of this phrase query in FTS3 doclist format. As usual, the initial
170081 ** "Length" field found in doclists stored on disk is omitted from this
170086 ** where nCol is the number of columns in the queried FTS table. The array
170111 int iPhrase; /* Index of this phrase in matchinfo() results */
170117 ** four values is in order of precedence when parsing expressions. For
170199 char *aBuffer; /* Buffer to merge doclists in */
170200 int nBuffer; /* Allocated size of aBuffer[] in bytes */
170211 int nTerm; /* Size of zTerm in bytes */
170213 int nDoclist; /* Size of aDoclist[] in bytes */
170295 /************** Continuing where we left off in fts3.c ***********************/
170330 ** assert() conditions in the fts3 code are activated - conditions that are
170347 q[-1] &= 0x7f; /* turn off high bit in final byte */
170384 ** The value is stored in *v.
170394 ** The value is stored in *v.
170455 ** the quote characters. The conversion is done in-place. If the
170520 ** interested in. So, unless the doclist is corrupt, the 0x80 bit is
170578 sqlite3 *db, /* Database in which to run SQL */
170633 ** Otherwise, if an error occurs, an SQLite error code is stored in *pRc
170750 ** Store the current database page-size in bytes in p->nPgsz.
170753 ** Otherwise, if an error occurs, an SQLite error code is stored in *pRc
170839 ** Return a copy of input string zInput enclosed in double-quotes (") and
170869 ** could be used in a SELECT statement such as the following:
170873 ** to return the docid, followed by each column of text data in order
170929 ** of columns in the %_content table (one for the docid plus one for each
170971 ** Decode this value and store it in *pnOut, returning the number of bytes
171025 ** the allocated array. *pnIndex is set to the number of elements in the
171032 const char *zParam, /* ABC in prefix=ABC parameter to parse */
171037 int nIndex = 1; /* Number of entries in array */
171091 ** the name of the corresponding column in table xxx. The array
171179 int argc, /* Number of elements in argv array */
171191 int nCol = 0; /* Number of columns in the FTS table */
171243 ** + If there is a tokenizer specification included in the arguments,
171452 /* Fill in the zName and zDb fields of the vtab structure. */
171461 /* Fill in the azColumn array */
171476 /* Fill in the abNotindexed array */
171500 sqlite3Fts3ErrMsg(pzErr, "missing %s parameter in fts4 constructor", zMiss);
171506 /* If this is an xCreate call, create the underlying tables in the
171520 /* Figure out the page-size for the database. This is required in order to
171557 ** work is done in function fts3InitVtab().
171562 int argc, /* Number of elements in argv array */
171572 int argc, /* Number of elements in argv array */
171594 ** Set the SQLITE_INDEX_SCAN_UNIQUE flag in pIdxInfo->flags. Unless this
171608 ** are three possible strategies, in order of preference:
171642 ** function MATCH in the requested context" error. To discourage
171667 ** it would lead to an "unable to use function MATCH in the requested
171721 /* Regardless of the strategy selected, FTS can deliver rows in rowid (or
171860 ** table is missing a row that is present in the full-text index.
171879 ** passed in zTerm/nTerm.
171892 int nTerm, /* Size of term zTerm in bytes */
171914 ** not make this guarantee explicitly, but in practice there are always
172003 ** left-most leaf node in the tree that may contain the specified term.
172019 int nTerm, /* Size of term zTerm in bytes */
172026 int iHeight; /* Height of this node in tree */
172036 int nBlob = 0; /* Size of zBlob in bytes */
172086 ** a single document record of a doclist. So, in other words, this
172087 ** routine advances *ppPoslist so that it points to the next docid in
172106 ** last byte in the position-list.
172171 ** called, *pp may point to the start of the next varint in the position-list
172173 ** (in which case **pp will be a terminator bytes POS_END (0) or
172183 ** in the position-list. Because positions are delta-encoded, the value
172184 ** of the previous position is needed in order to compute the value of
172223 ** into *pp contains all positions of both *pp1 and *pp2 in sorted
172226 ** that there is enough space in *pp to hold the complete output.
172238 int iCol1; /* The current column index in pp1 */
172239 int iCol2; /* The current column index in pp2 */
172321 ** each position in *pp2 for which there exists one or more positions in
172330 int nToken, /* Maximum difference in token positions */
172409 ** column-number in the position list.
172444 ** expression and *pp2 to the right. As usual, the indexes in the position
172445 ** lists are the offsets of the last token in each phrase (tokens "1" and "2"
172446 ** in the example above).
172449 ** entries that are not sufficiently NEAR entries in *pp1 removed.
172454 int nRight, /* Maximum difference in token positions */
172455 int nLeft, /* Maximum difference in token positions */
172493 int anOutput[16]; /* Size each output buffer in bytes */
172573 ** sorted in either ascending or descending order.
172580 ** positions contained in either argument doclist). If the docids in the
172581 ** input doclists are sorted in ascending order, parameter bDescDoclist
172582 ** should be false. If they are sorted in ascending order, it should be
172587 ** *pnOut is set to the number of bytes in the output doclist.
172590 ** are undefined in this case.
172614 ** are delta encoded. If they are in ascending order (bDescDoclist==0),
172615 ** then the first docid in each list is simply encoded as a varint. For
172617 ** current and previous docid (a positive number - since the list is in
172621 ** same number of bytes as it is in whichever of the input lists it is
172623 ** consumes either the same or less bytes as it did in the input (since
172624 ** the difference between it and the previous value in the output must
172632 ** be larger in the output than it was in the input (since the delta value
172639 ** A symetric argument may be made if the doclists are in descending
172685 ** doclist for which there is a position in the left-hand input doclist
172688 ** If the docids in the input doclists are sorted in ascending order,
172689 ** parameter bDescDoclist should be false. If they are sorted in ascending
172758 ** Argument pList points to a position list nList bytes in size. This
172760 ** a token in position 0 (of any column). If so, it writes argument iDelta
172768 int nList, /* Size of pList in bytes */
172809 ** Merge all doclists in the TermSelect.aaOutput[] array into a single
172810 ** doclist stored in TermSelect.aaOutput[0]. If successful, delete all
172814 ** the responsibility of the caller to free any doclists left in the
172822 /* Loop through the doclists in the aaOutput[] array. Merge them all
172875 int nDoclist /* Size of aDoclist in bytes */
172884 ** doclists are stored in order=ASC order, this padding would not be
172886 ** than or equal to the size of [doclistA] in that case). But this is
172888 ** may be smaller than (-1), as in the first example the -1 may be stored
172889 ** as a single-byte delta, whereas in the second it must be stored as a
172892 ** Similar padding is added in the fts3DoclistOrMerge() function.
172975 int nTerm, /* Size of zTerm in bytes */
173049 int nTerm, /* Size of zTerm in bytes */
173079 int nTerm, /* Number of bytes in zTerm */
173103 int nTerm, /* Size of zTerm in bytes */
173215 ** This function counts the total number of docids in the doclist stored
173216 ** in buffer aList[], size nList bytes.
173239 ** Advance the cursor to the next row in the %_content table that
173242 ** simply the next row in the %_content table. For a docid lookup,
173294 ** in the %_content table.
173305 int nVal, /* Number of elements in apVal */
173463 /* The column value supplied by SQLite must be in range. */
173525 ** in place to remove the entries that have already been merged. This
173553 int mxLevel = 0; /* Maximum relative level value in db */
173658 ** is the first in the doclist. In this case do not skip forward 2 bytes.
173713 int nToken = 15; /* Default number of tokens in snippet */
173769 ** function merges all segments in the database to a single segment.
173882 ** PendingTermsFlush() in in case that changes.
174019 ** The fts3 built-in tokenizers - "simple", "porter" and "icu"- are
174020 ** implemented in files fts3_tokenizer1.c, fts3_porter.c and fts3_icu.c
174022 ** declared in these files used to retrieve the respective implementations.
174080 /* Load the built-in tokenizers into the hash table */
174139 ** Allocate an Fts3MultiSegReader for each token in the expression headed
174147 ** If the allocated Fts3MultiSegReader just seeks to a single entry in a
174150 ** and merged incrementally. Otherwise, it has to be merged into an in-memory
174156 int *pnToken, /* OUT: Total number of tokens in phrase. */
174157 int *pnOr, /* OUT: Total number of OR nodes in expr. */
174187 ** It is merged into the main doclist stored in p->doclist.aAll/nAll.
174200 int nList /* Number of bytes in pList */
174291 ** means that the phrase does not appear in the current row, doclist.pList
174299 int nPoslist = 0; /* Number of bytes in aPoslist */
174398 ** This function is called for each Fts3Phrase in a full-text query
174416 ** scanned in forward order, and the phrase consists of
174459 ** doclists in reverse and by the fts3_write.c module to iterate through
174462 ** The doclist may be sorted in ascending (parameter bDescIdx==0) or
174469 int nDoclist, /* Length of aDoclist in bytes */
174525 int nDoclist, /* Length of aDoclist in bytes */
174556 ** Advance the iterator pDL to the next entry in pDL->aAll/nAll. Set *pbEof
174591 ** edited in place by fts3EvalNearTrim(), then pIter may not actually
174616 ** multi-token phrase. Advance it to the next matching document in the
174665 ** Advance it to the next matching documnent in the database and populate
174707 /* Advance the iterator for each token in the phrase once. */
174808 ** each phrase in the expression (subject to deferred token processing).
174817 Fts3Expr *pExpr, /* Expression to initialize phrases in */
174842 ** of the xFilter() method). There is one element in the array for each
174843 ** token in the FTS expression.
174845 ** Tokens are divided into AND/NEAR clusters. All tokens in a cluster belong
174848 ** separately. The root of a tokens AND/NEAR cluster is stored in
174854 int iToken; /* Position of token in phrase */
174912 ** Determine the average document (row) size in pages. If successful,
174916 ** The average document size in pages is calculated by first calculating
174917 ** determining the average size in bytes, B. If B is less than the amount
174918 ** of data that will fit on a single leaf page of an intkey table in
174920 ** the number of overflow pages consumed by a record B bytes in size.
174930 ** varints, where nCol is the number of columns in the FTS3 table.
174931 ** The first varint is the number of documents currently stored in
174933 ** data stored in all rows of each column of the table, from left
174974 ** This function is called once for each AND/NEAR cluster in the
174987 int nTC /* Number of entries in aTC[] */
174994 int nToken = 0; /* Total number of tokens in cluster */
175008 /* Count the tokens in this AND/NEAR cluster. If none of the doclists
175010 ** only 1 token, exit early. No tokens to defer in this case. */
175019 /* Obtain the average docsize (in pages). */
175024 /* Iterate through all tokens in this AND/NEAR cluster, in ascending order
175029 ** a. The cheapest token in the entire query (i.e. the one visited by the
175036 ** contains. Set variable "nMinEst" to the smallest number of documents in
175041 ** Then, for each token, defer it if loading the doclist would result in
175072 ** overflowing the 32-bit integer it is stored in. */
175076 /* Either this is the cheapest token in the entire query, or it is
175106 ** the full-text query currently stored in pCsr->pExpr. To iterate through
175122 /* Allocate a MultiSegReader for each token in the expression. */
175125 /* Determine which, if any, tokens in the expression should be deferred. */
175182 ** Parameter nNear is passed the NEAR distance of the expression (5 in
175184 ** the position list, and *pnToken is the number of phrase tokens in the
175189 ** All positions in the pPhrase position list that are not sufficiently
175190 ** close to a position in the *paPoslist position list are removed. If this
175194 ** associated with pPhrase. And *pnToken is set to the number of tokens in
175198 int nNear, /* NEAR distance. As in "NEAR/nNear". */
175201 int *pnToken, /* IN/OUT: Tokens in phrase of *paPoslist */
175235 ** point to the next matching row in the database. Expressions iterate through
175236 ** matching rows in docid order. Ascending order if Fts3Cursor.bDesc is zero,
175240 ** successful, the following variables in pExpr are set:
175249 ** FTs3Expr.pPhrase->doclist.nList (length of pList in bytes)
175256 ** entirely of deferred tokens, it is assumed to match every row in
175264 ** where "*" may match any single token. The position list in this case
175406 ** expression, using the data currently stored in the position lists
175407 ** (Fts3Expr->pPhrase.doclist.pList/nList) for each phrase in the expression.
175410 ** phrase in the NEAR expression is edited in place to contain only those
175424 ** which is represented in tree form as:
175437 ** no exceptions to this - it's the way the parser in fts3_expr.c works.
175515 ** all phrases involved in the NEAR. This is because the snippet(),
175594 ** * Up until this point, "NEAR" operators in the expression have been
175639 ** Advance to the next document that matches the FTS expression in
175719 ** After allocating the Fts3Expr.aMI[] array for each phrase in the
175722 ** the values in Fts3Expr.aMI[] according to the position-list currently
175723 ** found in Fts3Expr.pPhrase->doclist.pList for each of the phrase
175833 /* Caution: pRoot may iterate through docids in ascending or descending
175854 ** 1. The total number of occurrences of the phrase in each column of
175857 ** 2. For each column, the number of rows in the table for which the
175869 ** values are set to the number of documents in the table. In other
175870 ** words we assume that very common tokens occur exactly once in each
175878 ** that meet the NEAR constraint are included in the counts.
175914 ** a position-list indicating the occurrences of the phrase in column iCol
175918 ** occurrence of the phrase in the column, stored using the normal (delta+2)
175961 ** pCsr->iPrevId may lie earlier in the doclist buffer. Or, if the
175974 ** into memory in this case. */
176165 int argc, /* Number of elements in argv array */
176180 /* The user should invoke this in one of two forms:
176267 /* This vtab delivers always results in "ORDER BY term ASC" order. */
176423 ** integer encountered in state 1 is not 0 or 1, then we need to
176470 int nVal, /* Number of elements in apVal */
176479 int iEq = -1; /* Index of term=? value in apVal */
176480 int iGe = -1; /* Index of term>=? value in apVal */
176481 int iLe = -1; /* Index of term<=? value in apVal */
176482 int iLangid = -1; /* Index of languageid=? value in apVal */
176704 ** operator in a similar format to that used by the lemon parser
176747 ** FTSQUERY_PHRASE with a unary "-" attached to it. i.e. "mysql" in the
176757 int nCol; /* Number of entries in azCol[] */
176825 ** and other information (column names etc.) in pParse. Create an Fts3Expr
176925 ** is included in the buffer. This function attempts to tokenize the entire
176955 ** The first pass, in the block below, uses a tokenizer cursor to iterate
176956 ** through the tokens in the expression. This pass uses fts3ReallocOrFree()
176957 ** to assemble data in two dynamic buffers:
176965 ** The second pass, in the block that begins "if( rc==SQLITE_DONE )" below,
176966 ** appends buffer zTemp to buffer p, and fills in the Fts3Expr and Fts3Phrase
177061 unsigned char parenOnly; /* Only valid in paren mode */
177140 ** a quote character embedded in a string.
177204 ** group more tightly). For example, in the C language, the == operator
177208 ** is defined), the order of the operators in precedence from highest to
177237 ** of pNew and the existing nodes of the tree. This may result in the head
177238 ** of the tree changing, in which case *ppHead is set to the new root node.
177262 ** Parse the fts3 query expression found in buffer z, length n. This function
177279 Fts3Expr *pNotBranch = 0; /* Only used in legacy parse mode */
177319 ** an expression contained in parenthesis is required. If a
177351 ** Return an error in either case.
177437 ** an equivalent but more balanced form. The tree is modified in place.
177450 int eType = pRoot->eType; /* Type of node in this tree */
177470 /* Set $p to point to the left-most leaf in the tree of eType nodes. */
177476 /* This loop runs once for each leaf in the tree of eType nodes. */
177517 /* Set $p to point to the next leaf in the tree of eType nodes */
177618 ** in this case.
177625 int nCol, /* Number of entries in azCol[] */
177674 ** of each column in the target fts3 table, in order from left to right.
177688 int nCol, /* Number of entries in azCol[] */
177967 ** This is the implementation of generic hash-tables used in SQLite.
177973 ** The code in this file is only compiled if:
177976 ** (in which case SQLITE_CORE is not defined), or
177979 ** SQLite (in which case SQLITE_ENABLE_FTS3 is defined).
178086 ** The C syntax in this function definition may be unfamilar to some
178107 ** For help in interpreted the obscure C code in the function definition,
178126 Fts3HashElem *pHead; /* First element already in pEntry */
178171 /* This function (for internal use only) locates an element in an
178274 ** The key is not copied in this instance. If a malloc fails, then
178283 int nKey, /* Number of bytes in the key */
178358 ** The code in this file is only compiled if:
178361 ** (in which case SQLITE_CORE is not defined), or
178364 ** SQLite (in which case SQLITE_ENABLE_FTS3 is defined).
178390 int iOffset; /* current position in zInput */
178427 ** used to incrementally tokenize this string is returned in
178478 ** isConsonant() and isVowel() determine if their first character in
178484 ** in which case it is a vowel.
178486 ** In these routine, the letters are in reverse order. So the 'y' rule
178526 ** In this routine z[] is in reverse order. So we are really looking
178573 ** Return TRUE if the word ends in a double consonant.
178588 ** first three letters and the first one cannot be in [wxy].
178603 ** The input word *pz and zFrom are both in reverse order. zTo
178604 ** is in normal order.
178660 ** Stem the input word zIn[0..nIn-1]. Store the output in zOut.
178664 ** Any upper-case characters in the US-ASCII character set ([A-Z])
178675 ** in [a-zA-Z] then no stemming is attempted and this routine just
178699 /* The use of a character not in [a-zA-Z] means that we fallback
178723 /* Do nothing. The work was all in the test */
178731 /* Do nothing. The work was all in the test */
178905 /* z[] is now the stemmed word in reverse order. Flip it back
178938 int *pnBytes, /* OUT: Number of bytes in token */
178995 ** tokenizer in *ppModule
179024 ** The code in this file is only compiled if:
179027 ** (in which case SQLITE_CORE is not defined), or
179030 ** SQLite (in which case SQLITE_ENABLE_FTS3 is defined).
179063 ** the string <key-name> must already exist in the has table. Otherwise,
179244 ** designed to be used in concert with the Tcl testing framework. This
179254 ** list. For each token in the <input-string>, three elements are
179258 ** using the built-in "simple" tokenizer:
179323 zErr = "error in xCreate()";
179328 zErr = "error in xOpen()";
179341 zErr = "error in xClose()";
179345 zErr = "error in xDestroy()";
179413 ** This function is used for testing only, it is not included in the
179417 ** can be used as designed by the C-code in the queryTokenizer and
179419 ** in the README.tokenizer file as an example, so it is important to
179467 ** Set up SQL objects in database db used to access the contents of
179475 ** fts3TokenizerFunc() in this file for details) and, if ENABLE_TABLE is
179546 ** The code in this file is only compiled if:
179549 ** (in which case SQLITE_CORE is not defined), or
179552 ** SQLite (in which case SQLITE_ENABLE_FTS3 is defined).
179573 int iOffset; /* current position in pInput */
179602 ** track such information in the database, then we'd only want this
179639 ** used to incrementally tokenize this string is returned in
179689 int *pnBytes, /* OUT: Number of bytes in token */
179755 ** tokenizer in *ppModule
179796 ** statement and returns one row for each token in the result. With
179834 int nToken; /* Size of zToken in bytes */
179871 ** in this case. Or, if an error occurs, an SQLite error code is returned.
179872 ** The final value of *pazDequote is undefined in this case.
179875 int argc, /* Number of elements in argv[] */
179927 int argc, /* Number of elements in argv array */
180103 int nVal, /* Number of elements in apVal */
180242 ** code in fts3.c.
180258 ** of it. i.e. if a full-text index node is 900 bytes in size, then a buffer
180275 ** bytes and larger. Nodes are loaded in chunks of FTS3_NODE_CHUNKSIZE bytes.
180295 ** The values that may be meaningfully bound to the :1 parameter in
180305 ** debugging FTS only, it should not usually be turned on in production
180340 int iCol; /* Column token must occur in */
180341 Fts3DeferredToken *pNext; /* Next in list of deferred tokens */
180348 ** this structure are only manipulated by code in this file, opaque handles
180349 ** of type Fts3SegReader* are also used by code in fts3.c to iterate through
180369 sqlite3_int64 iEndBlock; /* Rowid of final block in segment (or 0) */
180384 int nTerm; /* Number of bytes in current term */
180388 int nDoclist; /* Size of doclist in current entry */
180402 ** An instance of this structure is used to create a segment b-tree in the
180412 sqlite3_int64 iFirst; /* First slot in %_segments written */
180413 sqlite3_int64 iFree; /* Next free slot in %_segments */
180415 int nTerm; /* Number of bytes in zTerm */
180419 int nData; /* Bytes of data in aData */
180437 ** the tree is assembled in memory and written out only once all leaves have
180448 int nTerm; /* Number of bytes in zTerm */
180532 /* Return segments in order from oldest to newest.*/
180558 ** of the oldest level in the db that contains at least ? segments. Or,
180559 ** if no level in the FTS index contains more than ? segments, the statement
180565 /* Estimate the upper limit on the number of leaf nodes in a new segment
180604 ** Return the largest relative level in the FTS index or indexes. */
180607 /* Return segments in order from oldest to newest.*/
180712 ** Similar to fts3SqlStmt(). Except, after binding the parameters in
180775 ** level within the index) are encoded in 64-bit integer values stored
180776 ** in the %_segdir table on disk. This function is used to convert three
180793 int iIndex, /* Index in p->aIndex[] */
180807 ** all rows in the %_segdir table, from oldest to newest. If successful,
180910 ** Zero is always returned in this case. Otherwise, if no OOM error occurs,
181008 ** pending-terms hash-table. The docid used is that currently stored in
181139 ** operation. It adds entries for each term in the new record to the
181206 ** defined columns in the FTS3 table, plus one for the docid field.
181285 ** The first element in the apVal[] array is assumed to contain the docid
181337 ** This function allocates a new level iLevel index in the segdir table.
181379 ** full, merge all segments in level iLevel into a single iLevel+1
181403 ** with the contents of the blob stored in the "block" column of the
181405 ** to the size of the blob in bytes before returning.
181412 ** This function may leave an open sqlite3_blob* handle in the
181425 char **paBlob, /* OUT: Blob data in malloc'd buffer */
181533 ** Move the iterator passed as the first argument to the next term in the
181544 int nPrefix; /* Number of bytes in term prefix */
181545 int nSuffix; /* Number of bytes in term suffix */
181653 ** Set the SegReader to point to the first docid in the doclist associated
181679 ** Advance the SegReader to point to the next docid in the doclist
181684 ** in the doclist entry (i.e. immediately past the docid varint).
181692 int *pnOffsetList /* OUT: Length of *ppOffsetList in bytes */
181702 ** Pending-terms doclists are always built up in ascending order, so
181749 /* List may have been edited in place by fts3EvalNearTrim() */
181752 /* If there are no more entries in the doclist, set pOffsetList to
181866 /* The entire segment is stored in the root node. */
181903 ** a subset of the terms stored in the Fts3Table.pendingTerms array.
181906 ** through each term in the pending-terms table. Or, if isPrefixIter is
181909 ** "firebird", then the iterator visits the following 'terms' (in the order
181962 ** objects in term order using qsort(). This uses the same comparison
181970 /* The query is a simple term lookup that matches at most one term in
182041 ** version, it is assumed that each SegReader points to an entry in
182044 ** 1) EOF (end of doclist in this case) is greater than not EOF.
182086 int nTerm /* Size of term zTerm in bytes */
182104 ** the final (nSegment-nSuspect) members are already in sorted order
182106 ** the array around until all entries are in sorted order.
182145 int n /* Size of buffer z in bytes */
182160 ** Find the largest relative level number in the table. If successful, set
182190 sqlite3_int64 nLeafData, /* Bytes of leaf data in segment */
182192 int nRoot /* Number of bytes in buffer zRoot */
182218 ** zNext, in bytes. For example,
182226 int nPrev, /* Size of buffer zPrev in bytes */
182228 int nNext /* Size of buffer zNext in bytes */
182245 int nTerm /* Size of term in bytes */
182255 int nData = pTree->nData; /* Current size of node in bytes */
182287 /* There is no prefix-length field for first term in a node */
182318 ** If this is the first node in the tree, the term is added to it.
182385 int iHeight, /* Height of this node in tree */
182387 sqlite3_int64 iFree, /* Block id of next free slot in %_segments */
182390 int *pnRoot /* OUT: Size of root node in bytes */
182447 ** and return it via the input/output variable *ppWriter in this case.
182456 int nTerm, /* Size of term in bytes */
182458 int nDoclist /* Size of doclist in bytes */
182460 int nPrefix; /* Size of term prefix in bytes */
182461 int nSuffix; /* Size of term suffix in bytes */
182476 /* Allocate a buffer in which to accumulate data */
182481 /* Find the next free blockid in the %_segments table */
182521 ** to the database (still available in pWriter->zTerm), and
182648 ** The first value in the apVal[] array is assumed to contain an integer.
182677 ** Set *pnMax to the largest segment level in the database for the index
182680 ** Segment levels are stored in the 'level' column of the %_segdir table.
182744 ** Delete all entries in the %_segments table associated with the segment
182772 ** each of the SegReader objects in the array passed as the third
182831 ** If there are no entries in the input position list for column iCol, then
182841 int *pnList /* IN/OUT: Size of buffer *ppList in bytes */
182876 ** Cache data in the Fts3MultiSegReader.aBuffer[] buffer (overwriting any
182905 int *pnPoslist /* OUT: Size of position list in bytes */
182972 int nTerm /* Length of zTerm in bytes */
183014 int nTerm /* Number of bytes in zTerm */
183055 ** have been made. Calling this function puts the MultiSegReader in such
183061 ** then the entire doclist for the term is available in
183123 /* Advance the first pCsr->nAdvance entries in the apSegment[] array
183124 ** forward. Then sort the list in order of current term again.
183186 sqlite3_int64 iPrev = 0; /* Previous docid stored in doclist */
183188 /* The current term of the first nMerge entries in the array
183347 /* Loop through all entries in the %_segdir table corresponding to
183348 ** segments in this index on levels greater than iAbsLevel. If there is
183350 ** such segments are smaller than nLimit bytes in size, they will be
183384 /* Loop through all %_segdir entries for segments in this index with
183387 ** oldest segment in the range, 1 for the next oldest, and so on.
183390 ** setting the "idx" fields as appropriate to keep them in the same
183424 ** Merge all level iLevel segments in the database into a single
183427 ** currently present in the database.
183430 ** segment in the database, SQLITE_DONE is returned immediately.
183437 int iIndex, /* Index in p->aIndex[] to merge */
183465 /* This call is to merge all segments in the database to a single
183467 ** greatest segment level currently present in the database for this
183604 ** Insert the sizes (in tokens) for each column of the document
183611 u32 *aSz /* Sizes of each column, in tokens */
183614 int nBlob; /* Number of bytes in the BLOB */
183639 ** where N is the number of user defined columns in the fts3 table plus
183643 ** Varint 0: Total number of rows in the table.
183645 ** Varint 1..nCol: For each column, the total number of tokens stored in
183648 ** Varint 1+nCol: The total size, in bytes, of all text values in all
183657 int nChng /* Change in the number of documents */
183848 ** the oldest nSeg segments (idx=0 through idx=(nSeg-1)) in absolute
183902 ** to build up nodes or other blobs of data in.
183908 int n; /* Number of valid bytes of data in a[] */
183930 int iIdx; /* Index of *output* segment in iAbsLevel+1 */
183955 int nDoclist; /* Size of doclist in bytes */
183961 ** bytes in size, extend (realloc) it to be so.
184038 ** Initialize a node-reader object to read the node in buffer aNode/nNode.
184067 ** The block id of the leaf node just written to disk may be found in
184101 ** bytes in size, write the key here. */
184150 ** stored in blob *pNode. The node need not contain any terms, but the
184160 ** If the size of the value in blob pPrev is zero, then this is the first
184176 int nTerm, /* Size of zTerm in bytes */
184178 int nDoclist /* Size of aDoclist in bytes */
184182 int nPrefix; /* Size of term prefix in bytes */
184183 int nSuffix; /* Size of term suffix in bytes */
184233 int nSpace; /* Total space in bytes required on leaf */
184257 ** to the database (still available in pLeaf->key), and
184314 int iRoot; /* Index of root in pWriter->aNodeWriter */
184318 /* Set iRoot to the index in pWriter->aNodeWriter[] of the output segment
184335 ** the node would be stored as a blob in the "root" column of the %_segdir
184336 ** table. However, this is not permitted in this case. The problem is that
184337 ** space has already been reserved in the %_segments table, and so the
184346 ** Todo: Better might be to defer allocating space in the %_segments
184391 ** Compare the term in buffer zLhs (size in bytes nLhs) with that in
184392 ** zRhs (size in bytes nRhs) using memcmp. If one term is a prefix of
184413 ** Query to see if the entry in the %_segments table with blockid iEnd is
184418 ** error code. The final value of *pbRes is undefined in this case.
184452 ** greater than the largest key currently stored in the potential
184460 int nKey, /* Number of bytes in nKey */
184472 int nRoot = 0; /* Size of aRoot[] in bytes */
184497 /* Check for the zero-length marker in the %_segments table */
184604 ** *piIdx is undefined in this case.
184629 ** In the %_segdir table, a segment is defined by the values in three
184639 ** of input segments, multiplied by two. This value is stored in stack
184676 /* Calculate the first block to use in the output segment */
184688 /* Insert the marker in the %_segments table to make sure nobody tries
184737 ** Update the 'idx' values of the remaining segments in the level so that
184746 int nIdx = 0; /* Valid entries in aIdx[] */
184811 ** This function creates a new node image in blob object *pNew by copying
184817 int nNode, /* Size of aNode in bytes */
184820 int nTerm, /* Size of zTerm in bytes */
184821 sqlite3_int64 *piBlock /* OUT: Block number in next layer down */
184865 ** Remove all terms smaller than zTerm/nTerm from segment iIdx in absolute
184867 ** table, and modifying existing entries in both the %_segments and %_segdir
184878 int nTerm /* Number of bytes in buffer zTerm */
184956 ** or modified in place so that it no longer contains those entries that
184957 ** have been duplicated in the output segment.
184974 ** somewhere in the pCsr->apSegment[] array. */
184990 ** segment to the upper level. The segment is modified in place
185008 ** Store an incr-merge hint in the database.
185028 ** exists, is stored in the rowid==1 row of the %_stat table.
185063 ** Otherwise, append an entry to the hint stored in blob *pHint. Each entry
185072 i64 iAbsLevel, /* First varint to store in hint */
185073 int nInput, /* Second varint to store in hint */
185088 ** If no error occurs, return SQLITE_OK. If the hint blob in *pHint does
185117 ** values for the _segdir.idx field) in the highest level that contains
185118 ** at least nMin segments. Multiple merges might occur in an attempt to
185144 int iIdx = 0; /* Largest idx in level (iAbsLevel+1) */
185164 ** last entry in it specifies a relative level smaller than or equal
185175 /* Based on the scan in the block above, it is known that there
185179 ** value of nHintSeg to avoid a large memory allocation in case the
185193 ** nMin segments and no hint in the %_stat table. No work to do.
185194 ** Exit early in this case. */
185206 ** segments available in level iAbsLevel. In this case, no work is
185387 int nTerm, /* Size of zTerm in bytes */
185407 ** Return a checksum of all entries in the FTS index that correspond to
185413 ** return value is undefined in this case.
185491 ** code. The final value of *pbOk is undefined in this case.
185543 int nToken = 0; /* Number of bytes in token */
185545 int iPos = 0; /* Position of token in zText */
185584 ** prefix in the document set, a 64-bit checksum is calculated (by code
185585 ** in fts3ChecksumEntry()) based on the following:
185590 ** + The language-id of the row it appears in,
185591 ** + The docid of the row it appears in,
185592 ** + The column it appears in, and
185595 ** The checksums for all entries in the index are XORed together to create
185598 ** The integrity-check code calculates the same checksum in two ways:
185679 ** Free all entries in the pCsr->pDeffered list. Entries are added to
185694 ** Generate deferred-doclists for all tokens in the pCsr->pDeferred list
185723 int nToken = 0; /* Number of bytes in token */
185725 int iPos = 0; /* Position of token in zText */
185787 int iCol /* Column that token must appear in (or -1) */
185809 ** present in the FTS3 table. If it is, delete it and adjust the contents
185819 int bFound = 0; /* True if *pRowid really is in the table */
185829 ** data in the pendingTerms hash table. */
185871 int nChng = 0; /* Net change in number of documents */
185901 /* Allocate space to hold the change in document sizes */
185933 /* The new rowid is not NULL (in this case the rowid will be
185945 ** call to fts3InsertData() (either just below or further on in this
186001 ** Flush any data in the pending-terms hash table to disk. If successful,
186002 ** merge all segments in the database (including the new segment, if
186046 ** Characters that may appear in the second argument to matchinfo().
186086 int nSnippet; /* Requested snippet length (in tokens) */
186087 int nPhrase; /* Number of phrases in query */
186093 int nToken; /* Number of tokens in phrase */
186095 int iHead; /* Next value in position list */
186097 int iTail; /* Next value in trailing position list */
186103 int iPos; /* Index of first token in snippet */
186115 int nCol; /* Number of columns in table */
186116 int nPhrase; /* Number of matchable phrases in query */
186117 sqlite3_int64 nDoc; /* Number of docs in database */
186124 ** (nElem * sizeof(u32)) bytes in size. See the MatchinfoBuffer code below
186144 int n; /* Length of z in bytes (excl. nul-term) */
186145 int nAlloc; /* Allocated size of buffer z in bytes */
186258 ** the list. *piPos contains the value of the previous entry in the list.
186294 ** Iterate through all phrase nodes in an FTS3 query, except those that
186332 ** Load the doclists for each phrase in the query associated with FTS3 cursor
186336 ** phrases in the expression (all phrases except those directly or
186338 ** pnToken is not NULL, then it is set to the number of tokens in all
186343 int *pnPhrase, /* OUT: Number of phrases in query */
186344 int *pnToken /* OUT: Number of tokens in query */
186451 u64 mHighlight = 0; /* Mask of tokens to highlight in snippet */
186529 ** (a) +1 point for each occurrence of a matchable phrase in the snippet.
186531 ** (b) +1000 points for the first occurrence of each matchable phrase in
186534 ** The selected snippet parameters are stored in structure *pFragment before
186535 ** returning. The score of the selected snippet is stored in *piScore
186548 int nList; /* Number of phrases in expression */
186556 /* Iterate through the phrases in the expression to count them. The same
186575 ** the set of phrases in the expression to populate the aPhrase[] array.
186592 /* Loop through all candidate snippets. Store the best snippet in
186593 ** *pFragment. Store its associated 'score' in iBestScore.
186627 int nAppend /* Size of zAppend in bytes (or -1) */
186664 ** This function "shifts" the beginning of the snippet forward in the
186678 int iLangid, /* Language id to use in tokenizing */
186681 int nDoc, /* Size of buffer zDoc in bytes */
186697 /* Ideally, the start of the snippet should be pushed forward in the
186713 ** or more tokens in zDoc/nDoc.
186745 int isLast, /* True for final fragment in snippet */
186746 int nSnippet, /* Number of tokens in extracted snippet */
186755 int nDoc; /* Size of zDoc in bytes */
186784 int iBegin = 0; /* Offset in zDoc of start of token */
186785 int iFin = 0; /* Offset in zDoc of end of token */
186789 ** in the FTS code the variable that the third argument to xNext points to
186857 ** This function is used to count the entries in a column-list (a
186860 ** beginning of the first varint in the column-list (the varint that
186861 ** contains the position of the first matching term in the column data).
186863 ** the last varint in the column-list (either the 0x00 signifying the end
186865 ** the next column in the position-list).
186867 ** The number of elements in the column-list is returned.
186964 ** Y values are set to nDoc, where nDoc is the number of documents in the
187103 ** hits for a single phrase on a single row in order to calculate the
187157 ** array before returning. SQLITE_OK is returned in this case.
187171 ** contains one element for each matchable phrase in the query.
187186 int nLive = 0; /* Number of iterators in aIter not at EOF */
187250 ** pInfo->aMatchinfo[] buffer is left in is undefined.
187284 sqlite3_int64 nDoc; /* Number of rows in table */
187405 size_t nMatchinfo = 0; /* Number of u32 elements in match-info */
187408 /* Determine the number of phrases in the query */
187412 /* Determine the number of integers in the buffer returned by this call. */
187466 int nToken /* Approximate number of tokens in snippet */
187474 ** the data in the current row. The first iteration of the for(...) loop
187475 ** below attempts to locate a single fragment of text nToken tokens in
187476 ** size that contains at least one instance of all phrases in the query
187477 ** expression that appear in the current row. If such a fragment of text
187481 int nSnippet = 0; /* Number of fragments in this snippet */
187483 int nFToken = -1; /* Number of tokens in each fragment */
187522 /* Find the best snippet of nFToken tokens in column iRead. */
187536 /* If all query phrases seen by fts3BestSnippet() are present in at least
187584 int nTerm; /* Number of tokens in phrase */
187587 int iPos = 0; /* First position in position-list */
187618 int nToken; /* Number of tokens in query */
187631 /* Count the number of terms in the query */
187665 /* Retreive the text stored in column iCol. If an SQL NULL is stored
187666 ** in column iCol, jump immediately to the next iteration of the loop.
187869 int nInput; /* Size of aInput[] in bytes */
187892 ** all characters in string zIn/nIn to be separators (if bAlnum==0) or
187895 ** For each codepoint in the zIn/nIn string, this function checks if the
187900 ** codepoints in the aiException[] array.
187903 ** identifies as a diacritic) occurs in the zIn/nIn string it is ignored.
187911 int nIn /* Length of z in bytes */
187932 int nNew; /* Number of valid entries in array aNew[] */
188044 ** used to incrementally tokenize this string is returned in
188050 int nInput, /* Size of string aInput in bytes */
188207 /* Each unsigned integer in the following array corresponds to a contiguous
188211 ** The most significant 22 bits in each 32-bit value contain the first
188212 ** codepoint in the range. The least significant 10 bits are used to store
188332 ** in the ASCII range with a diacritic added, return the codepoint
188425 /* Each entry in the following array defines a rule for folding a range
188429 ** If the least significant bit in flags is clear, then the rule applies
188432 ** every second codepoint in the range, starting with codepoint C.
188434 ** The 7 most significant bits in flags are an index into the aiOff[]
188581 ** a JSONB type in the future which stores a binary encoding of JSON in
188582 ** a BLOB, but there is no support for JSONB in the current implementation.
188631 ** the library isspace() function, resulting in a 7% overall performance
188655 /* Unsigned integer types. These are already defined in the sqliteInt.h,
188675 u64 nAlloc; /* Bytes of storage available in zBuf[] */
188723 u32 iKey; /* Key for ARRAY objects in json_tree() */
188740 int nJson; /* Length of the zJson string in bytes */
188747 ** This limit is needed to avoid a stack overflow in the recursive
188857 /* Append the N-byte string in zIn to the end of the JsonString string
188858 ** under construction. Enclose the string in "..." and escape
188943 /* Make the JSON in p the result of the SQL function.
188965 ** by which the JsonNode counter should increment in order to go to the
189314 ** instance to the JsonParse. Return the index in pParse->aNode[] of the
189422 /* Control characters are not allowed in strings */
189543 ** to fill in all the descendants of node i.
189570 ** Compute the parentage of all nodes in a completed parse.
189585 ** Magic number used for the JSON parse cache in sqlite3_get_auxdata()
189591 ** Obtain a complete parse of the JSON found in the first argument
189687 const char **pzErr /* Make *pzErr point to any syntax error in zPath */
189915 ** Mark all NULL entries in the Object passed in as JNODE_REMOVE.
190018 ** array that contains all values given in arguments. Or if any argument
190045 ** Return the number of elements in the top-level JSON array.
190130 u32 iTarget, /* Node of the TARGET in pParse */
190227 ** object that contains all name/value given in arguments. Or if any name
190270 ** JSON or PATH arguments result in an error.
190447 ** Return a JSON array composed of all values in the aggregate.
190549 ** Return a JSON object composed of all names and values in the aggregate.
190615 u32 i; /* Index in sParse.aNode[] of current row */
190645 ** the last two columns in the table. Should this ever changes, be
190913 ** columns in the table */
191096 ** The following routines are the only publically visible identifiers in this
191097 ** file. Call the following routines in order to register the various SQL
191216 ** The data structure for a single virtual r-tree table is stored in three
191218 ** in the table name is replaced with the user-supplied name of the r-tree
191225 ** The data for each node of the r-tree structure is stored in the %_node
191227 ** an entry in the %_parent table associating the node with its parent.
191228 ** And for each row of data in the table, there is an entry in the %_rowid
191234 ** empty. The nodeno of the root node is always 1. All other nodes in the
191243 ** stored in the node.
191312 ** the number of rows in the virtual table to calculate the costs of
191327 int iNodeSize; /* Size in bytes of each node in the node table */
191333 u8 nAux; /* # of auxiliary columns in %_rowid */
191342 i64 nRowEst; /* Estimated number of rows in this table */
191375 RtreeNode *aHash[HASHSIZE]; /* Hash table of in-memory nodes. */
191413 ** represents is the iCell-th entry in the parent node.
191447 ** Number of entries in the cursor RtreeNode cache. The first entry is
191462 int nConstraint; /* Number of entries in aConstraint */
191465 int nPoint; /* Number of slots used in aPoint[] */
191528 ** are used in the cursor for contraints such as x=NULL (RTREE_FALSE) or
191542 RtreeNode *pNext; /* Next node in this hash collision chain */
191545 /* Return the number of cells in a node */
191561 ** operators in order to constrain a search.
191568 ** This object is deleted automatically by the destructor mechanism in
191579 ** An instance of this structure (in the form of a BLOB) is returned by
191612 /* The testcase() macro should already be defined in the amalgamation. If
191793 ** in the Rtree.aHash table.
191889 /* Check if the requested node is already in the hash table. If so,
192031 ** If there is not enough free space in pNode, return SQLITE_FULL.
192038 int nCell; /* Current number of cells in pNode */
192122 ** Return coordinate iCoord from cell iCell in node pNode.
192341 ** Store the results in "r".
192470 ** in a coordinate pair. But make pCellData point to the lower bound.
192506 ** The op is given by p->op. The xN is p->iCoord-th coordinate in
192536 ** One of the cells in node pNode is guaranteed to have a 64-bit
192560 ** in its parent. If pNode is the root node, return -1.
192593 ** Interchange two search points in a cursor.
192952 ** return SQLITE_OK. If there is no such record in the table, set
193026 /* Reset the cursor to the same state as rtreeOpen() leaves it in. */
193130 ** table scan strategies to choose from (in order from most to
193142 ** the constraint in sqlite3_index_info.aConstraintUsage[] with
193145 ** The first of each pair of bytes in idxStr identifies the constraint
193249 ** Return the N-dimensional volumn of the cell stored in *p.
193279 ** of the objects size in each dimension.
193292 ** Store the union of cells p1 and p2 in p1.
193372 ** ChooseSubTree in r*tree terminology.
193431 ** the node pNode. This function updates the bounding box cells in
193488 ** (nIdx-1) in no particular order. This function sorts the values
193489 ** in aIdx according to the indexed values in aDistance. For
193560 ** (nIdx-1) in no particular order. This function sorts the values
193561 ** in aIdx according to dimension iDim of the cells in aCell. The
193934 /* Remove the entry in the parent cell. */
193963 /* Remove the node from the in-memory hash table and link it into
194011 ** the in-memory node image, so it cannot fail.
194017 ** cell in the parent node so that it tightly contains the updated
194114 /* Find a node to store this cell in. pNode->iNode currently contains
194183 /* Find a node to store this cell in. pNode->iNode currently contains
194218 int iCell; /* Index of iDelete cell in pLeaf */
194236 /* Delete the cell in question from the leaf node. */
194249 /* Delete the corresponding entry in the <rtree>_rowid table. */
194262 ** in this scenario).
194336 ** Parameter iCol is the index of the leftmost column involved in the
194451 /* If a rowid value was supplied, check if it is already present in
194595 ** of the number of rows in the virtual table. If possible, this is based
194838 ** would fit in a single node, use a smaller node-size.
194871 *pzErr = sqlite3_mprintf("undersize RTree blobs in \"%q_node\"",
195030 ** entry for each cell in the r-tree node. Each entry is itself a
195110 int nLeaf; /* Number of leaf cells in table */
195111 int nNonLeaf; /* Number of non-leaf cells in table */
195114 int nErr; /* Number of lines in zReport */
195134 ** NULL is returned and an error code left in RtreeCheck.rc.
195163 ** appends it to the report being accumuated in pCheck.
195187 ** in the RtreeCheck object indicated by the first argument. NULL is
195188 ** returned in this case.
195193 ** is set to the size of the buffer in bytes.
195196 ** in the RtreeCheck object. The final value of *pnNode is undefined in
195275 "Found (%lld -> %lld) in %s table, expected (%lld -> %lld)",
195297 i64 iNode, /* Node id to use in error messages */
195298 int iCell, /* Cell number to use in error messages */
195339 ** report accumulated in the RtreeCheck object.
195399 ** "_parent". This function checks that the number of entries in the
195401 ** an error message to the report in the RtreeCheck object indicated
195414 rtreeCheckAppendMsg(pCheck, "Wrong number of entries in %%%s table"
195463 /* Find number of dimensions in the rtree table. */
195509 ** 1. For each cell in the r-tree structure (%_node table), that:
195516 ** c) for leaf nodes, that there is an entry in the %_rowid
195520 ** d) for cells on non-leaf nodes, that there is an entry in the
195524 ** 2. That there are the same number of entries in the %_rowid table
195525 ** as there are leaf cells in the r-tree structure, and that there
195526 ** is a leaf cell that corresponds to each entry in the %_rowid table.
195528 ** 3. That there are the same number of entries in the %_parent table
195529 ** as there are non-leaf cells in the r-tree structure, and that
195530 ** there is a non-leaf cell that corresponds to each entry in the
195565 /************** Include geopoly.c in the middle of rtree.c *******************/
195595 #ifndef JSON_NULL /* The following stuff repeats things found in json1 */
195665 ** standard in which the final vertex is a repeat of the first.)
195694 ** We have to use these macros, rather than just say p->a[i] in order
195707 int nVertex; /* Number of vertexes in a[] */
195863 ** in the binary format or JSON text. Compute a GeoPoly object and
195865 ** polygon, put an error message in sqlite3_context and return NULL.
196047 ** This routine can also be used to detect polygons that rotate in
196090 ** then reverse the order of the vertexes in polygon X.
196096 ** contain polygons that wind in the wrong direction.
196183 ** (1) if aCoord!=0 store the bounding box in aCoord, returning NULL
196188 ** the bounding box in aCoord and return a pointer to that GeoPoly.
196441 GeoEvent *pNext; /* Next event in the sorted list */
196449 GeoSegment *pNext; /* Next segment in a list sorted by y */
196592 ** Sort a list of GeoSegments in order of increasing Y and in the event of
196919 /* Reset the cursor to the same state as rtreeOpen() leaves it in. */
197025 ** table scan strategies to choose from (in order from most to
197156 int coordChange = 0; /* Change in coordinates */
197188 /* If a rowid value was supplied, check if it is already present in
197277 ** for use in xBestIndex.
197379 /************** Continuing where we left off in rtree.c **********************/
197629 ** Maximum length (in bytes) of the pattern in a LIKE or GLOB
197709 ** MATCH_ALL. For each MATCH_ONE, skip one character in the
197757 ** the build-in LIKE operator. The first argument to the function is the
197778 ** of deep recursion and N*N behavior in patternCompare().
197921 int nInput; /* Size of utf-16 input string in bytes */
197922 int nOut; /* Size of output buffer in bytes */
198154 int nChar; /* Number of UChar elements in pInput */
198156 int *aOffset; /* Offsets of each character in utf-8 input */
198168 int argc, /* Number of entries in argv[] */
198206 ** used to incrementally tokenize this string is returned in
198212 int nInput, /* Length of zInput in bytes */
198295 int *pnBytes, /* OUT: Number of bytes in token */
198397 ** SQLite database. The tables it expects to find are described in
198398 ** sqlite3rbu.h. Essentially, for each table xyz in the target database
198400 ** created in the RBU database and populated with one row for each row to
198403 ** The update proceeds in three stages:
198409 ** "<database>-oal", they go on using the original database in
198417 ** updates are done in sorted key order.
198425 ** detect the new *-wal file and read the database in wal mode. At
198429 ** 3) The new *-wal file is checkpointed. This proceeds in the same way
198433 ** the checkpoint progress is saved in the RBU database and the
198434 ** checkpoint can be resumed by another RBU client at some point in
198446 ** pass unnoticed, or result in SQLITE_CONSTRAINT errors (due to UNIQUE
198456 ** collected in the rbu_xyz table. And for both UPDATEs and DELETEs all
198470 /************** Include sqlite3rbu.h in the middle of sqlite3rbu.c ***********/
198494 ** The problem is that in order to update a b-tree, the leaf page (at least)
198499 ** Additionally, because the index updates are likely to be applied in
198500 ** random order, access to pages within the database is also likely to be in
198511 ** Additionally, this extension allows the work involved in writing the
198521 ** "RBU" stands for "Resumable Bulk Update". As in a large database update
198536 ** non-NULL PRIMARY KEY values. Rows with NULL values stored in PRIMARY
198554 ** containing an RBU update is an "RBU database". For each table in the
198559 ** the same type as the corresponding column in the target database.
198569 ** The order of the columns in the data_% table does not matter.
198577 ** tables within the RBU database are always processed in order sorted by
198579 ** of the RBU tables the user can therefore control the order in which they
198598 ** target table must be present in the input table. For virtual tables,
198599 ** hidden columns are optional - they are updated by RBU if present in
198618 ** do so results in an SQLITE_MISMATCH error.
198623 ** real primary key values of the row to delete should be stored in the
198624 ** corresponding columns of the data_% table. The values stored in the
198631 ** stored in the corresponding columns of the data_% table row, as should
198632 ** the new values of all columns being update. The text value in the
198634 ** there are columns in the target database table, and must consist entirely
198635 ** of 'x' and '.' characters (or in some special cases 'd' - see below). For
198649 ** target table with the value stored in the corresponding data_% column, the
198650 ** user-defined SQL function "rbu_delta()" is invoked and the result stored in
198652 ** original value currently stored in the target table column and the
198653 ** value specified in the data_xxx table.
198663 ** Finally, if an 'f' character appears in place of a 'd' or 's' in an
198665 ** to be a "fossil delta" - a patch to be applied to a blob value in the
198683 ** However, RBU is more efficient if reading the rows in from each data_xxx
198684 ** table in "rowid" order is roughly the same as reading them sorted by
198710 ** update application is saved in the RBU database for later
198726 ** An RBU update may not be applied to a database in WAL mode. Attempting
198764 ** Or, if zState is non-NULL, it must be a path to a database file in which
198778 ** SQLite's built-in VFSs, including the multiplexor VFS. However it does
198790 ** An RBU vacuum is similar to SQLite's built-in VACUUM command, except
198793 ** The second argument to this function identifies a database in which
198799 ** returned SQLITE_DONE, the vacuum state is stored in the state database.
198806 ** state database is not already present in the file-system, it is created
198833 ** in bytes by the second parameter. If it is positive, the limit is updated.
198838 ** In all cases the returned value is the current limit in bytes (zero
198847 ** Return the current amount of temp file space, in bytes, currently used by
198860 ** Accessing the underlying database handles may be useful in the
198867 ** * If the data_xxx tables in the RBU source database are virtual
198945 ** * Stage 1, in which changes are accumulated in an oal/wal file, and
198946 ** * Stage 2, in which the contents of the wal file are copied into the
198966 ** There must be one row in the table for each source (data_xxx) table within
198983 ** defined in this file. Return values should be interpreted as follows:
199031 ** VFS uses the RBU VFS, which in turn uses any other VFS layers in use
199053 ** that does not include the RBU layer results in an error.
199056 ** non-RBU users. There is no harm in an application accessing the
199079 /************** Continuing where we left off in sqlite3rbu.c *****************/
199135 ** cases when another client appends frames to the *-wal file in the
199140 ** Valid if STAGE==1. The current change-counter cookie value in the
199144 ** Valid if STAGE==1. The size in bytes of the *-oal file.
199187 ** These values must match the values defined in wal.c for the equivalent
199198 ** A structure to store values read from the rbu_state table in memory.
199225 ** An iterator of this type is used to iterate through all objects in
199227 ** iterator visits, in order:
199235 ** it points to an array of flags nTblCol elements in size. The flag is
199240 ** this array set set to 1. This is because in that case, the module has
199258 int bCleanup; /* True in "cleanup" state */
199268 int nCol; /* Number of columns in current object */
199309 #define RBU_UPDATE 6 /* Update a row in a main table b-tree */
199336 ** for all source tables in the RBU database, where nRow is the number
199337 ** of rows in the source table and nIndex the number of indexes on the
199344 ** for an UPDATE operation does not exist in the target table, then
199358 ** stored in the rbu_state table if the RBU update is suspended.
199382 u32 iMaxFrame; /* Largest iWalFrame value in aFrame[] */
199384 int nFrame; /* Entries in aFrame[] array */
199390 i64 szTemp; /* Current size of all temp files in use */
199393 /* Used in RBU vacuum mode only */
199394 int nRbu; /* Number of RBU VFS in the stack */
199427 i64 sz; /* Size of file in bytes (temp only) */
199434 int nShm; /* Number of entries in apShm[] array */
199465 ** in *pz and is decremented once for each character in the integer.
199537 ** binary files) so we also have to pass in the length of the delta in
199540 ** This function returns the size of the output file in bytes (excluding
199700 ** Prepare the SQL statement in buffer zSql against database handle db.
199841 ** left in the RBU handle passed as the first argument. A copy of the
199906 ** the name of a table in the RBU database. The second, if it is present, is 1
199957 ** left in the RBU handle passed as the first argument. A copy of the
199988 ** an error code is stored in the RBU handle passed as the first argument.
200016 ** If an error occurs, an error code and error message is stored in the
200042 ** error code in the rbu handle passed as the first argument. Or, if an
200065 ** error code in the RBU handle passed as the first argument.
200084 ** returns a copy of the string in memory obtained from sqlite3_malloc().
200148 ** if( no entry exists in sqlite_schema ){
200153 ** if( the index that is the pk exists in sqlite_schema ){
200300 ** an error does occur, an error code and error message are also left in
200362 /* Check that all non-HIDDEN columns in the destination table are also
200363 ** present in the input table. Populate the abTblPk[], azTblType[] and
200411 ** column names currently stored in the pIter->azTblCol[] array.
200430 ** in order, for the current table. Before each column name, add the text
200528 ** it, then composes and returns an expression that can be used in a WHERE
200803 ** Return an expression that can be used in a WHERE clause to match the
200849 ** is something wrong with the rbu_control value in the rbu_control value
200850 ** stored in the (p->nCol+1)'th column. Set the error code and error message
200871 ** string, an error code is left in the rbu handle passed as the first
200925 ** string, an error code is left in the rbu handle passed as the first
201074 ** in the sqlite3rbu object indicated by the first argument and returns
201083 ** An imposter table is required in all cases except RBU_PK_VTAB. Only
201141 ** The number of bound variables is equal to the number of columns in
201354 /* Create the SELECT statement to read keys in sorted order */
201559 ** is not an error. Output variable *ppStmt is set to NULL in this case.
201576 ** to the number of statements currently in the cache and pUp to the
201577 ** last object in the list. */
201665 ** If an error occurs, leave an error code and message in the rbu handle
201743 ** error occurs, leave an error code and message in the RBU handle.
201892 ** this call returns SQLITE_NOTFOUND, then the RBU vfs is not in use.
201909 ** filename in zBaseFilename is a URI with the "8_3_names=1" parameter and
201910 ** if filename in z[] has a suffix (a.k.a. "extension") that is longer than
201942 ** The checksum is store in the first page of xShmMap memory as an 8-byte
201969 ** following recovery is not the same as the checksum saved in the RbuState
201971 ** other client appends a transaction to the wal file in the middle of
201988 ** special behaviour in the rbu VFS:
201996 ** would be read/written are recorded in the sqlite3rbu.aFrame[]
202045 ** directory in which the target database and the wal file reside, in
202046 ** case it has not been synced since the rename() call in
202055 ** the rbu object is in capture mode. Record the frame number of the frame
202056 ** being read in the aFrame[] array.
202087 ** file while the rbu handle is in capture mode. Record the page number
202088 ** of the page being written in the aFrame[] array.
202152 ** The RBU handle is currently in RBU_STAGE_OAL state, with a SHARED lock
202154 ** then reopens the database file (this time in vanilla, non-oal, WAL mode).
202155 ** If an error occurs, leave an error code and error message in the rbu
202177 ** in WAL mode). So no other connection may be writing the db.
202247 ** message are left in the RBU handle and zero returned.
202385 ** and message is left in the RBU handle and a copy of the error code
202466 ** statement reads is page 1, which is guaranteed to be in the cache.
202481 ** value stored in the RBU_STATE_STAGE column is eStage. All other values
202563 ** RBU vacuum operation, create the schema in the target db.
202714 ** this by checkpointing frames until the next page in the aFrame[]
202754 ** an rbu handle in OAL stage. If the rbu update has not started (i.e.
202759 ** If an error occurs, an error code and error message are left in the
202791 ** If there is a "*-oal" file in the file-system corresponding to the
202792 ** target database in the file-system, delete it. If an error occurs,
202793 ** leave an error code and error message in the rbu handle.
202808 ** specified a URI with a vfs=? option in place of a target database
202840 ** name of a table expected to appear in the target database. It returns
202885 ** There should be one row in the table for each data_xxx table in the
202890 ** for all rows in the rbu_count table, where nIndex is the number of
202971 ** RBU vacuum operation leaving an old wal file in the directory.
203018 ** transaction is committed in rollback mode) currently stored on
203144 ** If the error code currently stored in the RBU handle is SQLITE_CONSTRAINT,
203336 ** of a standard VFS in the following ways:
203339 ** written, the value of the change-counter cookie is stored in
203341 ** database header field is stored in rbu_file.iWriteVer. This ensures
203355 ** associated with an RBU target database currently in RBU_STAGE_OAL
203360 ** target database may not be in wal mode already.
203366 ** the RBU target in RBU_STAGE_OAL stage, instead of opening the *-wal
203369 ** 3b. The *-shm pages returned by xShmMap() for a target db file in
203370 ** RBU_STAGE_OAL mode are actually stored in heap memory. This is to
203372 ** are no-ops on target database files in RBU_STAGE_OAL mode. This is
203373 ** because assert() statements in some VFS implementations fail if
203376 ** 3c. If an EXCLUSIVE lock is attempted on a target database file in any
203526 ** Write an unsigned 32-bit value in big-endian format to the supplied
203584 rbuPutU32(&aBuf[28], 1); /* size of db file in pages */
203749 /* First try to find another RBU vfs lower down in the vfs stack. If
203750 ** one is found, this vfs will operate in pass-through mode. The lower
203755 /* Now search for a zipvfs instance lower down in the VFS stack. If
203792 ** Return the sector-size in bytes for an rbuVfs-file.
203826 ** is supplying heap memory to the upper layer in place of *-shm
203859 /* If not in RBU_STAGE_OAL, allow this call to pass through. Or, if this
203860 ** rbu is in the RBU_STAGE_OAL state, use heap memory for *-shm space
203869 ** recovery, the connection must request *-shm pages in order
203986 ** pair of '\0' bytes in case the VFS attempts to extract a URI
204067 ** database connection exists, and the RBU update is in RBU_STAGE_OAL,
204072 ** in wal mode, even if the first page of the database file has
204099 ** to the pathname in zPath. zOut is guaranteed to point to a buffer
204132 ** Return a pointer to the symbol zSymbol in the dynamic library pHandle.
204171 ** Return the current time as a Julian Day number in *pTimeOut.
204314 ** information from an SQLite database in order to implement the
204343 ** from. For example, the three overflow pages in a chain linked from
204347 ** '/1c2/000+000000' // First page in overflow chain
204348 ** '/1c2/000+000001' // Second page in overflow chain
204349 ** '/1c2/000+000002' // Third page in overflow chain
204352 ** the overflow pages associated with a cell will appear earlier in the
204367 " pgoffset INTEGER," /* 8 Offset of page in file (NULL for agg) */
204374 /* Forward reference to data structured used in this module */
204384 int nOvfl; /* Entries in aOvfl[] */
204415 StatPage aPage[32]; /* Pages in path to current page */
204416 int iPage; /* Current entry in aPage[] */
204423 int nPage; /* Number of pages in current btree */
204495 ** Compute the best query strategy and return the result in idxNum.
204499 ** 0x01 There is a schema=? term in the WHERE clause
204500 ** 0x02 There is a name=? term in the WHERE clause
204501 ** 0x04 There is an aggregate=? term in the WHERE clause
204518 /* Force DBSTAT table should always be the right-most table in a join */
204552 /* Records are always returned in ascending order of (name, path).
204815 ** entry will be the next page, but in aggregated mode (pCsr->isAgg!=0),
204981 ** arguments in argv[0]. See statBestIndex() for a description of the
204982 ** meaning of the bits in idxNum.
205171 ** uncommitted changes and changes recorded in the WAL file are correctly
205202 int szPage; /* Size of each page in bytes */
205287 ** constraint (in which case we use the "main" schema) or else the
205346 ** Move a dbpagevfs cursor to the next entry in the file.
205520 /* Since we do not know in advance which database files will be
205647 int bNoDiscard; /* If true, do not discard in InputBuffer() */
205648 int iCurrent; /* Offset in aData[] of current change */
205649 int iNext; /* Offset in aData[] of next change */
205651 int nData; /* Number of bytes in aData */
205663 SessionInput in; /* Input buffer or stream */
205671 int nCol; /* Number of columns in zTab */
205681 ** stored in a linked list starting at sqlite3_session.pTable.
205684 ** been modified in any way since the session object was attached to the
205694 int nCol; /* Number of columns in table zName */
205698 int nEntry; /* Total number of entries in hash table */
205707 ** used in SQLite database files. This format is used as part of the
205722 ** and so on in sqlite3.h. For undefined and NULL values, the field consists
205727 ** A varint containing the number of bytes in the value (encoded using
205732 ** A varint containing the number of bytes in the value, followed by
205741 ** Varint values are encoded in the same way as varints in the SQLite
205748 ** but may occur in any order (i.e. deletes, updates and inserts are all
205754 ** Varint: Number of columns in the table.
205765 ** The "old.*" and "new.*" records, if present, are N field records in the
205767 ** columns in the table. The i'th field of each record is associated with
205768 ** the i'th column of the table, counting from left to right in the order
205769 ** in which columns were declared in the CREATE TABLE statement.
205796 ** Varint: Number of columns in the table.
205807 ** As in the changeset format, each field of the single record that is part
205821 ** The records associated with INSERT changes are in the same format as for
205836 ** Each group of entries begins with a table header in changeset format:
205839 ** Varint: Number of columns in the table.
205847 ** record: (in the record format defined above).
205858 ** of the new.* record with the PK fields filled in based on the original
205864 ** this structure stored in a SessionTable.aChange[] hash table.
205869 int nRecord; /* Number of bytes in buffer aRecord[] */
205943 int nByte; /* Size of serialized value in bytes */
206010 ** Allocate and return a pointer to a buffer nByte bytes in size. If
206090 int *pbNullPK /* OUT: True if there are NULL values in PK */
206160 ** Based on the primary key values stored in change aRecord, calculate a
206172 int nBucket /* Assume this many buckets in hash table */
206184 ** module does not record changes for rows with NULL values stored in
206214 ** have the same values stored in the primary key columns), or false
206293 ** as to point to the next value in the record.
206297 ** pointer is returned and *pnVal is set to the number of bytes in the
206299 ** set to the number of bytes in the value at *paOne. If *paOne points
206309 int *pnVal /* OUT: Bytes in returned value */
206413 ** as the change stored in argument pChange. If so, it returns true. Otherwise
206450 /* A SessionChange object never has a NULL value in a PK column */
206491 ** It is possible that a non-fatal OOM error occurs in this function. In
206493 ** Growing the hash table in this case is a performance optimization only,
206539 ** zThis, in schema zDb.
206542 ** of columns in the database table and variable *pzTab is set to point to a
206650 ** free any allocation made. An error code will be returned in this case.
206673 ** If an error occurs, an error code is stored in sqlite3_session.rc and
206677 ** is set to NULL in this case.
206703 ** Versions of the four methods in object SessionHook for use with the
206766 /* Check the number of columns in this xPreUpdate call matches the
206767 ** number of columns in the table. */
206852 ** required values and encodings have already been cached in memory.
206853 ** It is not possible for an OOM to occur in this block. */
207094 const char *zDb1, /* Pick rows in this db only */
207095 const char *zDb2, /* But not in this one */
207211 int nCol; /* Columns in zFrom.zTbl */
207281 int nDb = sqlite3Strlen30(zDb); /* Length of zDb in bytes */
207283 /* Zero the output value in case an error occurs. */
207397 int nName; /* Number of bytes in string zName */
207415 ** simply added to the start of it in order to ensure that tables
207416 ** appear in the correct order when a changeset or patchset is
207433 ** Ensure that there is room in the buffer to append nByte bytes of data.
207528 ** called. Otherwise, append a string to the buffer. All bytes in the string
207566 ** called. Otherwise, append the string zStr enclosed in quotes (") and
207595 ** in column iCol of the row that SQL statement pStmt currently points
207649 ** values that were stored in the row when the session began (the old.*
207668 SessionBuffer buf2 = {0,0,0}; /* Buffer to accumulate new.* record in */
207763 int nCol, /* Number of columns in table */
207810 ** zTab in database zDb based on its primary key. i.e.
207818 int nCol, /* Number of columns in table */
207864 ** Bind the PRIMARY KEY values from the change passed in argument pChange
207873 int nCol, /* Number of columns in table */
207961 ** stored in output variables *pnChangeset and *ppChangeset. Or, if an error
207975 SessionBuffer buf = {0,0,0}; /* Buffer in which to accumlate changeset */
207980 /* Zero the output variables in case an error occurs. If this session
207981 ** object is already in the error state (sqlite3_session.rc != SQLITE_OK),
207997 int nCol; /* Number of columns in table */
208182 ** Return the amount of heap memory in use.
208195 int nChangeset, /* Size of buffer pChangeset in bytes */
208205 /* Zero the output variable in case an error occurs. */
208213 pRet->in.aData = (u8 *)pChangeset;
208214 pRet->in.nData = nChangeset;
208215 pRet->in.xInput = xInput;
208216 pRet->in.pIn = pIn;
208217 pRet->in.bEof = (xInput ? 0 : 1);
208231 int nChangeset, /* Size of buffer pChangeset in bytes */
208238 int nChangeset, /* Size of buffer pChangeset in bytes */
208284 ** Ensure that there are at least nByte bytes available in the buffer. Or,
208285 ** if there are not nByte bytes remaining in the input, that all available
208286 ** data is in the buffer.
208320 int nCol /* Number of values in record */
208340 ** first argument to a copy of the string or blob held in the aData[]
208347 int nData, /* Size of buffer aData[] in bytes */
208362 ** Deserialize a single record from a buffer in memory. See "RECORD FORMAT"
208370 ** (in other words, it is a patchset DELETE record).
208374 ** from the corresponding position in the record. If that value is not
208375 ** included in the record (i.e. because the record is part of an UPDATE change
208383 ** The apOut[] array may have been partially populated in this case.
208387 int nCol, /* Number of values in record */
208452 ** + number of columns in table (varint)
208456 ** This function ensures that all of the above is present in the input
208469 /* The hard upper limit for the number of columns in an SQLite
208501 ** the record in bytes. Otherwise, an SQLite error code is returned. The
208502 ** final value of *pnByte is undefined in this case.
208506 int nCol, /* Number of columns in record */
208507 int *pnByte /* OUT: Size of record in bytes */
208535 ** + number of columns in table (varint)
208553 rc = sessionChangesetBufferTblhdr(&p->in, &nCopy);
208557 nVarint = sessionVarintGet(&p->in.aData[p->in.iNext], &p->nCol);
208560 p->in.iNext += nVarint;
208572 memcpy(&p->tblhdr.aBuf[iPK], &p->in.aData[p->in.iNext], nCopy);
208573 p->in.iNext += nCopy;
208610 /* If the iterator is in the error-state, return immediately. */
208625 p->rc = sessionInputBuffer(&p->in, 2);
208629 if( p->in.iNext>=p->in.nData ){
208633 sessionDiscardData(&p->in);
208634 p->in.iCurrent = p->in.iNext;
208636 op = p->in.aData[p->in.iNext++];
208641 if( (p->rc = sessionInputBuffer(&p->in, 2)) ) return p->rc;
208642 p->in.iCurrent = p->in.iNext;
208643 if( p->in.iNext>=p->in.nData ) return SQLITE_DONE;
208644 op = p->in.aData[p->in.iNext++];
208648 /* The first record in the changeset is not a table header. Must be a
208650 assert( p->in.iNext==1 || p->zTab );
208655 p->bIndirect = p->in.aData[p->in.iNext++];
208670 p->rc = sessionChangesetBufferRecord(&p->in, nVal, pnRec);
208672 *paRec = &p->in.aData[p->in.iNext];
208673 p->in.iNext += *pnRec;
208681 p->rc = sessionReadRecord(&p->in, p->nCol, abPK, apOld, 0);
208687 p->rc = sessionReadRecord(&p->in, p->nCol, 0, apNew, pbEmpty);
208693 ** modified fields are present in the new.* record. The old.* record
208722 ** record is written to *paRec before returning and the number of bytes in
208726 ** successfully advanced to the next change in the changeset, an SQLite
208728 ** changes in the changeset.
208747 ** change in the changeset. This function may return SQLITE_ROW, SQLITE_DONE
208765 int *pnCol, /* OUT: Number of columns in table */
208777 ** Return information regarding the PRIMARY KEY and number of columns in
208785 int *pnCol /* OUT: Number of entries in output array */
208798 ** iVal'th value in the old.* record. Or, if that particular value is not
208799 ** included in the record (because the change is an UPDATE and the field
208826 ** iVal'th value in the new.* record. Or, if that particular value is not
208827 ** included in the record (because the change is an UPDATE and the field
208886 ** violations in the destination database and returns SQLITE_OK.
208917 sqlite3_free(p->in.buf.aBuf);
208927 int *pnInverted, /* OUT: Number of bytes in output changeset */
208932 int nCol = 0; /* Number of cols in current table */
208940 /* Zero the output variables in case an error occurs. */
208959 ** * Number of columns in said table (a varint),
209083 int nChangeset, /* Number of bytes in input */
209085 int *pnInverted, /* OUT: Number of bytes in output changeset */
209136 u8 *abPK; /* Boolean array - true if column is in PK */
209144 u8 bRebaseStarted; /* If table header is already in rebase */
209470 ** See comments in the body of this function for details.
209479 ** of SQLite. If a malloc fails in an sqlite3_value_xxx() function, either
209483 /* This condition occurs when an earlier OOM in a call to
209499 ** to an array nCol elements in size. In this case only those values for
209518 ** in the code below. */
209526 /* The value in the changeset was "undefined". This indicates a
209561 int nCol; /* Number of columns in table */
209583 ** It adds a conflict resolution record to the buffer in
209700 u8 *aBlob = &pIter->in.aData[pIter->in.iCurrent];
209701 int nBlob = pIter->in.iNext - pIter->in.iCurrent;
209790 ** * all columns of the table are PK columns (in this case there is
209857 ** to be done using a SELECT, as there is no PRIMARY KEY in the
209892 ** retried in some manner.
209909 ** a row with the correct PK is present in the db, but one or more other
209949 ** Retry the changes accumulated in the pApply->constraints buffer.
210032 pIter->in.bNoDiscard = 1;
210207 int nChangeset, /* Size of changeset in bytes */
210240 int nChangeset, /* Size of changeset in bytes */
210316 SessionTable *pList; /* List of tables in current patch */
210322 ** allocated and a pointer to it stored in *ppNew.
210332 int nRec, /* Number of bytes in aRec */
210503 ** Add all changes in the changeset traversed by the iterator passed as
210560 ** tables within the output of sqlite3changegroup_output() are in
210579 ** Code below may link it back in.
210621 ** this case (*pnOut) is set to the size of the output buffer in bytes. It
210642 ** hash tables attached to the SessionTable objects in list p->pList.
210694 ** Add the changeset currently stored in buffer pData, size nData bytes,
210765 int nLeft, /* Number of bytes in lhs input */
210767 int nRight /* Number of bytes in rhs input */,
210769 int *pnOut, /* OUT: Number of bytes in output changeset */
210828 ** fields in size. This function appends an nCol sessions module
210830 ** each field that is undefined in a1[], swap in the field from a2[].
210834 int nCol, /* Number of columns in each record */
210866 ** record) is in aChange/nChange. The rebased change is appended to buffer
210872 ** in the rebase buffer is set to "replaced" (type 0xFF). If this
210877 ** corresponding field in the rebase buffer is not "undefined" (0x00)
210879 ** in the rebase buffer.
210942 ** the buffer in bytes. It is the responsibility of the caller to eventually
210954 int *pnOut, /* OUT: Number of bytes in output changeset */
211210 ** Interfaces to extend FTS5. Using the interfaces defined in this file,
211242 int nVal, /* Number of values in apVal[] array */
211260 ** to the total number of tokens in the FTS5 table. Or, if iCol is
211261 ** non-negative but less than the number of columns in the table, return
211262 ** the total number of tokens in column iCol, considering all rows in
211266 ** in the table, SQLITE_RANGE is returned. Or, if an error occurs (e.g.
211271 ** Return the number of columns in the table.
211275 ** to the total number of tokens in the current row. Or, if iCol is
211276 ** non-negative but less than the number of columns in the table, set
211277 ** *pnToken to the number of tokens in column iCol of the current row.
211280 ** in the table, SQLITE_RANGE is returned. Or, if an error occurs (e.g.
211290 ** containing the text in utf-8 encoding, (*pn) is set to the size in bytes
211296 ** Returns the number of phrases in the current query expression.
211299 ** Returns the number of tokens in phrase iPhrase of the query. Phrases
211319 ** to the column in which it occurs and *piOff the token offset of the
211340 ** phrase iPhrase of the current query is included in $p. For each
211395 ** This function is used to retrieve the total number of rows in the table.
211434 ** of iterating through all instances of a phrase in the current row, these
211435 ** APIs are used to iterate through the set of columns in the current row
211566 ** For each token in the input string, the supplied callback xToken() must
211570 ** size of the token in bytes. The 4th and 5th arguments are the byte offsets
211578 ** FTS5 assumes the xToken() callback is invoked for each token in the
211591 ** Custom tokenizers may also support synonyms. Consider a case in which a
211593 ** built-in tokenizers, the FTS5 query 'first + place' will match instances
211597 ** the user specified in the MATCH query text.
211599 ** There are several ways to approach this in FTS5:
211603 ** same token for inputs "first" and "1st". Say that token is in
211620 ** first token in the MATCH query and FTS5 effectively runs a query
211640 ** 'first + place' or '1st + place', as there are entries in the
211660 ** by making multiple calls to xToken(FTS5_TOKEN_COLOCATED) in sequence.
211666 ** so it is efficient in terms of disk space and query speed. However, it
211709 int nToken, /* Size of token in bytes */
211833 ** directive in fts5_index.c will cause the build to fail.
211838 ** Maximum segments permitted in a single index
211889 ** Each entry in the aiCol[] array is a column that may be matched.
211901 ** Interface to code in fts5_config.c. fts5_config.c contains contains code
211928 ** This exists in order to allow the fts5_index.c module to return a
211950 int *aPrefix; /* Sizes in bytes of nPrefix prefix indexes */
211964 int pgsz; /* Approximate page size used in %_data */
211968 int nHashSize; /* Bytes of memory for in-memory hash */
212021 ** End of interface to code in fts5_config.c.
212025 ** Interface to code in fts5_buffer.c.
212071 int n; /* Size of buffer at a[] in bytes */
212072 int i; /* Current offset in a[] */
212107 /* Bucket of terms object used by the integrity-check in offsets=0 mode. */
212114 ** End of interface to code in fts5_buffer.c.
212118 ** Interface to code in fts5_index.c. fts5_index.c contains contains code
212119 ** to access the data stored in the %_data table.
212138 #define FTS5INDEX_QUERY_DESC 0x0002 /* Docs in descending rowid order */
212167 ** Argument p points to a buffer containing utf-8 text that is n bytes in
212168 ** size. Return the number of bytes in the nChar character prefix of the
212169 ** buffer, or 0 if there are less than nChar characters in total.
212218 ** For an insert, it must be called once for each token in the new document.
212220 ** unique token in the document with an iCol value less than zero. The iPos
212225 int iCol, /* Column token appears in (-ve -> delete) */
212241 ** Flush any data stored in the in-memory hash tables to the database.
212247 ** Discard any data stored in the in-memory hash tables. Do not write it
212249 ** table may have changed on disk. So any in-memory caches of %_data
212287 ** End of interface to code in fts5_index.c.
212291 ** Interface to code in fts5_varint.c.
212311 ** End of interface to code in fts5_varint.c.
212316 ** Interface to code in fts5_main.c.
212342 ** End of interface to code in fts5.c.
212346 ** Interface to code in fts5_hash.c.
212359 int iCol, /* Column token appears in (-ve -> delete) */
212375 int *pnDoclist /* OUT: Size of doclist in bytes */
212387 int *pnDoclist /* OUT: size of doclist in bytes */
212392 ** End of interface to code in fts5_hash.c.
212396 ** Interface to code in fts5_storage.c. fts5_storage.c contains contains
212397 ** code to access the data stored in the %_content and %_docsize tables.
212440 ** End of interface to code in fts5_storage.c.
212445 ** Interface to code in fts5_expr.c.
212456 int n; /* Size of buffer p in bytes */
212513 ** C code in this module. The interfaces below this point are called by
212514 ** the parser code in fts5parse.y. */
212564 ** End of interface to code in fts5_expr.c.
212570 ** Interface to code in fts5_aux.c.
212575 ** End of interface to code in fts5_aux.c.
212579 ** Interface to code in fts5_tokenizer.c.
212588 ** End of interface to code in fts5_tokenizer.c.
212592 ** Interface to code in fts5_vocab.c.
212598 ** End of interface to code in fts5_vocab.c.
212603 ** Interface to automatically generated code in fts5_unicode2.c.
212612 ** End of interface to code in fts5_unicode2.c.
212651 ** interstitial "-" characters) contained in this template is changed into
212665 ** Disable all error recovery processing in the parser push-down
212723 ** that indicate what to do in response to the next
212736 ** which is sqlite3Fts5ParserFTS5TOKENTYPE. The entry in the union
212749 ** fts5YYNRULE the number of rules in the grammar
212808 ** Applications can choose to define fts5yytestcase() in the %include section
212809 ** to a macro that can assist in verifying code coverage. For production
212837 ** slots in the fts5yy_action[] table.
212853 ** a reduce action) then the fts5yy_reduce_ofst[] array is used in place of
212856 ** The following are the tables generated in this section:
212859 ** fts5yy_lookahead[] A table containing the lookahead for each entry in
212926 ** appears in the grammar, then ID becomes a fallback token for X, Y,
212931 ** This feature can be used, for example, to cause some keywords in a language
212932 ** to revert to identifiers if they keyword does not apply in the context where
212949 ** the information used by the action routines in the grammar.
212957 fts5YYACTIONTYPE stateno; /* The state-number, or reduce action in SHIFTREDUCE */
212965 /* The state of the parser is completely contained in an instance of
212983 fts5yyStackEntry *fts5yystackEnd; /* Last entry in the stack */
213126 ** putting an appropriate #define in the %include section of the input
213171 ** A pointer to a parser. This pointer is used in subsequent calls
213284 ** is defined in a %include section of the input grammar) then it is
213311 ** is in state X and has a lookahead token Y. In a well-tested
213489 fts5YYACTIONTYPE fts5yyNewState, /* The new state to shift in */
213490 fts5YYCODETYPE fts5yyMajor, /* The major token to shift in */
213491 sqlite3Fts5ParserFTS5TOKENTYPE fts5yyMinor /* The minor token to shift in */
213601 ** only called from one place, optimizing compilers will in-line it, which
213871 ** user wants (and specified in the grammar) and is available for
213912 fprintf(fts5yyTraceFILE,"%sInput '%s' in state %d\n",
214131 ** Object used to iterate through all "coalesced phrase instances" in
214132 ** a single column of the current row. If the phrase instances in the
214134 ** through them. Or, if they do overlap (share one or more tokens in
214158 int iStart; /* First token in coalesced phrase instance */
214159 int iEnd; /* Last token in coalesced phrase instance */
214195 ** iterate through coalesced phrase instances in column iCol.
214226 int iPos; /* Current token offset in zIn[] */
214232 int nIn; /* Size of input text in bytes */
214265 int nToken, /* Size of token in bytes */
214319 int nVal, /* Number of values in apVal[] array */
214368 int nFirst; /* Number of entries in aFirst[] */
214369 int *aFirst; /* Array of first token in each sentence */
214394 ** function. Its job is to identify tokens that are the first in a sentence.
214401 int nToken, /* Size of token in bytes */
214433 int nDocsize, /* Size of column in tokens */
214475 ** Return the value in pVal interpreted as utf-8 text. Except, if pVal
214477 ** bytes in length instead of a NULL pointer.
214491 int nVal, /* Number of values in apVal[] array */
214501 int nPhrase; /* Number of phrases in query */
214506 int nColSize = 0; /* Total size of iBestCol in tokens */
214647 int nPhrase; /* Number of phrases in query */
214648 double avgdl; /* Average number of tokens in each row */
214654 ** Callback used by fts5Bm25GetData() to count the number of rows in the
214683 int nPhrase; /* Number of phrases in query */
214684 sqlite3_int64 nRow = 0; /* Number of rows in table */
214685 sqlite3_int64 nToken = 0; /* Number of tokens in table */
214708 /* Calculate an IDF for each phrase in the query */
214718 ** where "N" is the total number of documents in the set and nHit
214724 ** (1e-6) - roughly the same as a term that appears in just over
214750 int nVal, /* Number of values in apVal[] array */
214760 double D = 0.0; /* Total number of tokens in row */
214763 /* Calculate the phrase frequency (symbol "f(qi,D)" in the documentation)
214764 ** for each phrase in the query for the current row. */
214780 /* Figure out the total size of the current row in tokens. */
214867 ** pBuf. If an OOM error occurs, set the error code in p.
214887 ** the error code in p. If an error has already occurred when this function
214907 ** though this byte is not included in the pBuf->n count.
214925 ** included in the pBuf->n count.
214981 ** the error code in p. If an error has already occurred when this function
215052 ** Append position iPos to the position list being accumulated in buffer
215098 ** is non-negative, then it is the length of the string in bytes. Otherwise,
215269 ** string. Return a pointer to the first character following *pIn in
215282 ** string. Return a pointer to the first character following *pIn in
215366 ** the string and, if found, dequotes the string in place and adds a new
215371 ** found, -1 is returned. If -1 is returned, the buffer is left in an
215408 ** the quote characters. The conversion is done in-place. If the
215462 ** may be left in *pzErr. It is the responsibility of the caller to
215555 *pzErr = sqlite3_mprintf("parse error in tokenize directive");
215636 ** Return a pointer to the character immediately following the last in
215746 ** returned, *ppOut is set to NULL and an error message may be left in
215808 *pzErr = sqlite3_mprintf("parse error in \"%s\"", zOrig);
215831 /* If no zContent option was specified, fill in the default values. */
215922 ** The callback is invoked once for each token in the input text. The
215923 ** arguments passed to it are, in order:
215927 ** int nToken // Size of token in bytes
215930 ** int iPos // Position of token in input (first token is 0)
215956 ** Argument pIn points to the first character in what is expected to be
215985 ** + Zero or more SQL literals in a comma separated list
216213 ** All token types in the generated fts5parse.h file are greater than 0.
216238 int bDesc; /* Iterate in descending rowid order */
216239 int nPhrase; /* Number of phrases in expression */
216284 u8 bFirst; /* True if token must be first in column */
216287 Fts5ExprTerm *pSynonym; /* Pointer to first in list of synonyms */
216291 ** A phrase. One or more terms that must appear in a contiguous sequence
216297 int nTerm; /* Number of entries in aTerm[] */
216308 int nPhrase; /* Number of entries in aPhrase[] array */
216720 ** All individual term iterators in pPhrase are guaranteed to be valid and
216753 /* Initialize a term iterator for each term in the phrase */
216817 int n; /* Size of buffer a[] in bytes */
216818 int i; /* Current offset in position list */
216859 ** If in/out parameter (*pRc) is set to other than SQLITE_OK when this
216862 ** The return value is undefined in both these cases.
217047 /* Check that each phrase in the nearset matches the current row.
217074 ** Initialize all term iterators in the pNear object. If any term is found
217191 ** Nodes that point to values that come later in the iteration order are
217209 ** All individual term iterators in pNear are guaranteed to be valid when
217239 ** iterator skips through rowids in the default ascending order, this means
217283 ** Advance the first term iterator in the first phrase of pNear. Set output
217624 /* Initialize all term iterators in the NEAR object. */
217663 ** Begin iterating through the set of documents in index pIdx matched by
217665 ** parameter is passed a non-zero value, iteration is in descending rowid
217666 ** order. Or, if it is zero, in ascending order.
217668 ** If iterating in ascending rowid order (bDesc==0), the first document
217670 ** to parameter iFirst. Or, if iterating in ascending order (bDesc==1),
217685 /* If not at EOF but the current rowid occurs earlier than iFirst in
217851 int nToken, /* Size of token in bytes */
218100 ** Token pTok has appeared in a MATCH expression where the NEAR operator
218102 ** in the pParse object.
218144 ** If an OOM error occurs, store an error code in pParse and return NULL.
218145 ** The old colset object (if any) is not freed in this case.
218152 int nCol = p ? p->nCol : 0; /* Num. columns already in colset object */
218175 /* Check that the array is in order and contains no duplicate entries. */
218266 ** Remove from colset pColset any columns that are not also in colset pMerge.
218269 int iIn = 0; /* Next input in pColset */
218270 int iMerge = 0; /* Next input in pMerge */
218271 int iOut = 0; /* Next output slot in pColset */
218456 ** OOM error), leave an error code in pParse and return NULL.
219008 ** Return the number of phrases in expression pExpr.
219015 ** Return the number of terms in the iPhrase'th phrase in pExpr.
219090 int nToken, /* Size of token in bytes */
219269 ** This file contains the implementation of an in-memory hash table used
219278 int nEntry; /* Number of entries currently in hash */
219285 ** Each entry in the hash table is represented by an object of the
219287 ** its current data are stored in a single memory allocation. The
219288 ** key immediately follows the object in memory. The position list
219289 ** data immediately follows the key data in memory.
219291 ** The data that follows the key is in a similar, but not identical format
219292 ** to the doclist data stored in the database. It is:
219308 Fts5HashEntry *pScanNext; /* Next entry in sorted order */
219313 int nKey; /* Length of key in bytes */
219455 int nSz = (nData - p->iSzPoslist - 1); /* Size in bytes */
219481 ** Add an entry to the in-memory hash table. The key is the concatenation
219491 int iCol, /* Column token appears in (-ve -> delete) */
219638 ** each sorted in key order. This function merges the two lists into a
219683 ** in sorted order. The hash table is cleared before returning. It is
219739 int *pnDoclist /* OUT: Size of doclist in bytes */
219792 int *pnDoclist /* OUT: size of doclist in bytes */
219821 ** Low level access to the FTS index stored in the database file. The
219822 ** routines in this file file implement all read and write access to the
219824 ** the interface defined in fts5Int.h.
219837 ** * all segment b-tree leaf data is stored in fixed size page records
219839 ** taken to ensure it is possible to iterate in either direction through
219840 ** the entries in a doclist, or to seek to a specific entry within a
219848 ** * extra fields in the "structure record" record the state of ongoing
219855 #define FTS5_WORK_UNIT 64 /* Number of leaf pages in unit of work */
219879 ** recorded in a single record within the %_data table. The record consists
219894 ** + number of input segments in ongoing merge.
219895 ** + total number of segments in level.
219904 ** The first value is the number of rows in the index. Then, for each column
219905 ** from left to right, the total number of tokens in the column for all
219920 ** varint: number of bytes in common with previous term
219937 ** varint: size of poslist in bytes multiplied by 2, not including
219956 ** unsigned integer fields in big-endian format. They are:
219962 ** footer is 0 bytes in size, then this field is the same as the
219963 ** size of the leaf page in bytes.
219968 ** other words, the first varint in the footer is the byte offset of
219974 ** the format is changed in two ways:
219981 ** + the first term on each page is stored in the same way as the
220019 ** Rowids for the averages and structure records in the %_data table.
220026 ** and nodes. All nodes and leaves are stored in the %_data table with large
220078 int nn; /* Size of record in bytes */
220088 int nWorkUnit; /* Leaf pages in a "unit" of work */
220092 ** in-memory hash tables before they are flushed to disk.
220094 Fts5Hash *pHash; /* Hash table for in-memory data */
220128 ** using an Fts5Structure record in memory. Which uses instances of the
220133 int pgnoFirst; /* First leaf page number in segment */
220134 int pgnoLast; /* Last leaf page number in segment */
220137 int nMerge; /* Number of segments in incr-merge */
220144 int nSegment; /* Total segments in this structure */
220145 int nLevel; /* Number of levels in this index */
220169 u8 bFirstRowidInDoclist; /* True if next rowid is first in doclist */
220170 u8 bFirstRowidInPage; /* True if next rowid is first in page */
220172 u8 bFirstTermInPage; /* True if next term will be first in leaf */
220192 ** pair in the segment.
220219 ** has been exhausted. Do not proceed to the next term in the segment.
220223 ** it is set, iterate through rowid in descending order instead of the
220248 ** is the offset of the first rowid in the current doclist. */
220252 int iPgidxOff; /* Next offset in pgidx */
220256 int iRowidOffset; /* Current entry in aRowidOffset[] */
220265 int nPos; /* Number of bytes in current position list */
220293 ** visiting each term/rowid pair in the merged data.
220302 ** even number, is stored in aFirst[(nSeg+N)/2]. The "result" of the
220303 ** comparison in this context is the index of the iterator that currently
220307 ** aFirst[1] contains the index in aSeg[] of the iterator that points to
220325 int bRev; /* True to iterate in reverse order */
220373 ** Allocate and return a buffer at least nByte bytes in size.
220375 ** If an OOM error is encountered, return NULL and set the error code in
220436 ** If an error occurs, NULL is returned and an error left in the
220471 ** table, missing row, non-blob/text in block column - indicate
220633 ** Deserialize and return the structure record currently stored in serialized
220646 int nData, /* Size of buffer pData in bytes */
220839 ** If an error occurs, NULL is returned and an error code left in the
220890 ** Return the total number of segments in index structure pStruct. This
220922 ** If an error occurs, leave an error code in the Fts5Index object. If an
221018 ** as a result. Segments are promoted in two scenarios:
221142 ** points to the first rowid in the doclist-index.
221202 ** before the buffer in memory. */
221367 ** read. Before returning, set *pnSz to the number of bytes in the position
221383 ** in the following variables:
221442 ** the first term in the segment).
221497 ** Initialize the iterator object pIter to iterate through the entries in
221541 ** The iterator is in an unusual state when this function is called: the
221549 ** is in its regular state - Fts5SegIter.iLeafOffset points to the first
221901 ** Iterator pIter currently points to the first rowid in a doclist. This
221902 ** function sets the iterator up so that iterates in reverse order through
221936 /* The last rowid in the doclist may not be on the current page. Search
222021 ** iterator is left pointing to the smallest term in the segment that
222040 int iPgidx; /* Current offset in pgidx */
222055 /* Figure out how many new bytes are in this term */
222166 ** pSeg. If there is no such term in the index, the iterator is set to EOF.
222189 ** contain term (pTerm/nTerm), if it is present in the segment. */
222248 ** in-memory hash table. If there is no such term in the hash-table, the
222327 ** in *pRes is the correct result of comparing the current positions of the
222413 ** in the pIter->aSeg[] array that is (a) not at EOF, and (b) pointing
222415 ** segment-iterator in the pSeg->aSeg[] array.
222578 int iMinset /* Minimum entry in aFirst[] to set */
222654 ** If an error occurs, an error code is left in Fts5Index.rc. It is not
223177 ** The new object will be used to iterate through data in structure pStruct.
223178 ** If iLevel is -ve, then all data in all segments is merged. Or, if iLevel
223182 ** The iterator initially points to the first term/rowid entry in the
223195 int nSeg = 0; /* Number of segment-iters in use */
223252 ** to the first entry in its segment. In this case initialize the
223376 ** returned in this case.
223386 ** array is 63 elements, or 252 bytes, in size. */
223433 ** Discard all data currently cached in the hash-tables.
223444 ** Return the size of the prefix, in bytes, that buffer
223481 ** Grow the pWriter->aDlidx[] array to at least nLvl elements in size.
223506 ** If the current doclist-index accumulating in pWriter->aDlidx[] is large
223527 ** The doclist-index for that term is currently stored in-memory within the
223543 /* The following was already done in fts5WriteInit(): */
223560 ** If an error occurs, an error code is left in Fts5Index.rc. If an error
223751 /* This is the first term on a leaf that is not the leftmost leaf in
223759 ** Usually, the previous term is available in pPage->term. The exception
223760 ** is if this is the first term written in an incremental-merge step.
223809 ** rowid-pointer in the page-header. Also append a value to the dlidx
223810 ** buffer, in case a doclist-index is required. */
223866 int *pnLeaf /* OUT: Number of leaf pages in b-tree */
223909 /* Grow the two buffers to pgsz + padding bytes in size. */
224073 /* Read input from all segments in the input level */
224277 ** in a 32-bit integer. Return the size of the largest prefix of this
224278 ** list nMax bytes or less in size.
224295 ** Flush the contents of in-memory hash table iHash to a new level-0
224305 int pgnoLast = 0; /* Last leaf page number in segment */
224317 Fts5Buffer *pBuf; /* Buffer in which to assemble leaf page */
224318 Fts5Buffer *pPgidx; /* Buffer in which to assemble pgidx */
224339 int nDoclist; /* Size of doclist in bytes */
224392 ** it in one go. */
224453 ** Flush any data stored in the in-memory hash tables to the database.
224509 ** segment in the data structure. */
224702 int nBuf, /* Number of entries in apBuf[] */
224748 PrefixMerger *pNext; /* Next in docid/poslist order */
224781 ** Array aBuf[] contains nBuf doclists. These are all merged in with the
224782 ** doclist in buffer p1.
224787 int nBuf, /* Number of buffers in array aBuf[] */
224788 Fts5Buffer *aBuf /* Other lists to merge in */
224801 /* Initialize a doclist-iterator for each input buffer. Arrange them in
224802 ** a linked-list starting at pHead in ascending order of rowid. Avoid
224818 ** first rowid in one input is a large negative number, and the first in
224860 /* See the earlier comment in this function for an explanation of why
224932 int nToken, /* Size of buffer pToken in bytes */
225096 ** Discard any data stored in the in-memory hash tables. Do not write it
225098 ** table may have changed on disk. So any in-memory caches of %_data
225191 ** Argument p points to a buffer containing utf-8 text that is n bytes in
225192 ** size. Return the number of bytes in the nChar character prefix of the
225193 ** buffer, or 0 if there are less than nChar characters in total.
225219 ** pIn is a UTF-8 encoded string, nIn bytes in size. Return the number of
225220 ** unicode characters in the string.
225239 ** For an insert, it must be called once for each token in the new document.
225241 ** unique token in the document with an iCol value less than zero. The iPos
225246 int iCol, /* Column token appears in (-ve -> delete) */
225302 ** satisfied by scanning multiple terms in the main index.
225307 ** for internal sanity checking by the integrity-check in debug
225335 /* Scan multiple terms in the main index */
225396 ** in ascending or descending rowid order.
225431 ** the number of user defined columns in the FTS table.
225535 ** FTS functionality, or even the integrity-check, in any way.
225539 ** iSegid/iLeaf is iterated in forwards or reverse order.
225581 int n, /* Size of index key in bytes */
225650 ** FTS functionality, or even the integrity-check, in any way.
225666 int nTerm = pPrev->n-1; /* Size of zTerm in bytes */
225692 ** character stored in the main fts index, which will cause the
225832 /* If the leaf in question has already been trimmed from the segment,
225840 ** to or larger than the split-key in zIdxTerm. Also check that if there
225848 int nTerm; /* Size of term on leaf in bytes */
225930 /* Page iter.iLeaf must now be the rightmost leaf-page in the segment */
225978 ** based on all expected entries in the FTS index (including prefix index
225984 ** while doing a linear scan of each individual index in turn.
225994 int n; /* Size of term in bytes */
226202 ** Buffer (pData/nData) contains a doclist in the format used by detail=none
226209 ** pBuf is undefined in this case.
226261 /* Make a copy of the second argument (a blob) in aBlob[]. The aBlob[]
226346 int szLeaf; /* Offset of pgidx in a[] */
226540 ** assert() conditions in the fts5 code are activated - conditions that are
226565 ** in the xSync() callback. Current versions of SQLite ignore any errors
226575 ** in-memory. It is flushed to disk whenever xSync(), xRelease() or
226581 ** is not required, but it is used in the assert() statements executed by
226598 Fts5Auxiliary *pAux; /* First in list of all aux. functions */
226599 Fts5TokenizerModule *pTok; /* First in list of all tokenizer modules */
226601 Fts5Cursor *pCsr; /* First in list of all open cursors */
226643 int nTerm; /* Size of phrase in terms */
226651 ** There is one entry in the aIdx[] array for each phrase in the query,
226659 int nIdx; /* Number of entries in aIdx[] */
226673 ** cursor iterates in ascending order of rowids, iFirstRowid is the lower
226675 ** WHERE clause in the user's query might have been:
226679 ** If the cursor iterates in descending order of rowid, iFirstRowid
226685 Fts5Cursor *pNext; /* Next cursor in Fts5Cursor.pCsr list */
226706 sqlite3_stmt *pRankArgStmt; /* Origin of objects in apRankArg[] */
226710 Fts5Auxdata *pAuxdata; /* First in linked list of saved aux-data */
226759 Fts5Auxdata *pNext; /* Next object in linked list */
226876 int argc, /* Number of elements in argv array */
226936 ** work is done in function fts5InitVtab().
226941 int argc, /* Number of elements in argv array */
226951 int argc, /* Number of elements in argv array */
226970 ** Set the SQLITE_INDEX_SCAN_UNIQUE flag in pIdxInfo->flags. Unless this
227043 ** a) If an unusable MATCH operator is present in the WHERE clause, the
227182 /* Calculate the estimated cost based on the flags set in idxFlags. */
227336 /* nBlob==0 in detail=none mode. */
227374 ** (in the cursors sort order - ASC or DESC) than the current rowid.
227407 ** Advance the cursor to the next row in the table that matches the
227522 ** If SQLite a built-in statement cache, this wouldn't be a problem. */
227572 int n; /* Number of bytes in text at z */
227685 "parse error in rank function: %s", z
227725 int nVal, /* Number of elements in apVal */
227968 ** be left in sqlite3_vtab.zErrMsg.
228028 ** The commands implemented by this function are documented in the "Special
228545 int nUnused, /* Size of token in bytes */
228915 ** the current position-list for each phrase in the query associated with
228919 ** the number of phrases in the query. Following the varints are the
228920 ** concatenated position lists for each phrase, in order.
229005 ** useful in that it may be passed as the first argument to an FTS5
229124 sqlite3_int64 nName; /* Size of zName in bytes, including \0 */
229250 if( pzErr ) *pzErr = sqlite3_mprintf("error in tokenizer constructor");
229463 i64 nTotalRow; /* Total number of rows in FTS table */
229803 int szCol; /* Size of column value in tokens */
229813 int nToken, /* Size of token in bytes */
229828 ** If a row with rowid iDel is present in the %_content table, add the
229938 ** variables in the "averages" record on disk.
230004 ** Delete all entries in the FTS5 index.
230258 int nToken, /* Size of token in bytes */
230345 ** %_content table. This block stores the checksum in ctx.cksum. */
230495 int nCol = p->pConfig->nCol; /* Number of user columns in table */
230553 ** Flush any data currently held in-memory to disk.
230838 int nFold; /* Size of aFold[] in bytes */
231148 /* Any tokens larger than this (in bytes) are passed through without
231991 ** Register all built-in tokenizers with FTS5.
232043 ** in the ASCII range with a diacritic added, return the codepoint
232136 /* Each entry in the following array defines a rule for folding a range
232140 ** If the least significant bit in flags is clear, then the rule applies
232143 ** every second codepoint in the range, starting with codepoint C.
232145 ** The 7 most significant bits in flags are an index into the aiOff[]
232862 /* A 32-bit varint is used to store size information in btrees.
232886 ** inline in order to work around bugs in the RVT compiler.
232897 ** Return the number of bytes read. The value is stored in *v.
232972 /* we can skip these cause they were (effectively) done above in calc'ing s */
232993 /* we can skip this cause it was (effectively) done above in calc'ing s */
233165 ** instances of term $term in column $col (in any row of the fts5 table).
233170 ** One row for each term in the database. The value of $doc is set to
233173 ** $term in the database.
233178 ** One row for each term instance in the database.
233206 int nLeTerm; /* Size of zLeTerm in bytes */
233304 ** or, for tables in the TEMP schema only.
233313 int argc, /* Number of elements in argv array */
233370 ** work is done in function fts5VocabInitVtab().
233375 int argc, /* Number of elements in argv array */
233385 int argc, /* Number of elements in argv array */
233448 /* This virtual table always delivers results in ascending order of
233452 ** are already in sorted order. */
233610 ** Advance the cursor to the next row in the table.
233742 int nUnused, /* Number of elements in apVal */