1<!doctype html public "-//W3C//DTD HTML 4.01 Transitional//EN" 2 "http://www.w3.org/TR/html4/loose.dtd"> 3 4<html> 5 6<head> 7 8<title>Postfix Lookup Table Overview</title> 9 10<meta http-equiv="Content-Type" content="text/html; charset=us-ascii"> 11 12</head> 13 14<body> 15 16<h1><img src="postfix-logo.jpg" width="203" height="98" ALT="">Postfix 17Lookup Table Overview</h1> 18 19<hr> 20 21<h2>Overview </h2> 22 23This document covers the following topics: 24 25<ul> 26 27<li><a href="#intro">The Postfix lookup table model</a> 28 29<li><a href="#lists">Postfix lists versus tables </a> 30 31<li><a href="#preparing">Preparing Postfix for LDAP or SQL lookups</a> 32 33<li><a href="#detect">Maintaining Postfix lookup table files</a> 34 35<li><a href="#safe_db">Updating Berkeley DB files safely</a> 36 37<li><a href="#types">Postfix lookup table types</a> 38 39</ul> 40 41<h2><a name="intro">The Postfix lookup table model</a></h2> 42 43<p> Postfix uses lookup tables to store and look up information 44for access control, address rewriting and even for content filtering. 45All Postfix lookup tables are specified as "type:table", where 46"type" is one of the database types described under "<a 47href="#types">Postfix lookup table types</a>" at the end of this 48document, and where "table" is the lookup table name. The Postfix 49documentation uses the terms "database" and "lookup table" for the 50same thing. </p> 51 52<p> Examples of lookup tables that appear often in the Postfix 53documentation: </p> 54 55<blockquote> 56<pre> 57/etc/postfix/main.cf: 58 alias_maps = hash:/etc/postfix/aliases (local aliasing) 59 header_checks = regexp:/etc/postfix/header_checks (content filtering) 60 transport_maps = hash:/etc/postfix/transport (routing table) 61 virtual_alias_maps = hash:/etc/postfix/virtual (address rewriting) 62</pre> 63</blockquote> 64 65<p> All Postfix lookup tables store information as (key, value) 66pairs. This interface may seem simplistic at first, but it turns 67out to be very powerful. The (key, value) query interface completely 68hides the complexities of LDAP or SQL from Postfix. This is a good 69example of connecting complex systems with simple interfaces. </p> 70 71<p> Benefits of the Postfix (key, value) query interface:</p> 72 73<ul> 74 75<li> You can implement Postfix lookup tables first with local 76Berkeley DB files and then switch to LDAP or MySQL without any 77impact on the Postfix configuration itself, as described under "<a 78href="#preparing">Preparing Postfix for LDAP or SQL lookups</a>" 79below. 80 81<li> You can use Berkeley DB files with fixed lookup strings for 82simple address rewriting operations and you can use regular expression 83tables for the more complicated work. In other words, you don't 84have to put everything into the same table. 85 86</ul> 87 88<h2><a name="lists">Postfix lists versus tables </a></h2> 89 90<p> Most Postfix lookup tables are used to look up information. 91Examples are address rewriting (the lookup string is the old address, 92and the result is the new address) or access control (the lookup 93string is the client, sender or recipient, and the result is an 94action such as "reject"). </p> 95 96<p> With some tables, however, Postfix needs to know only if the 97lookup key exists. Any non-empty lookup result value may be used 98here: the lookup result is not used. Examples 99are the local_recipient_maps that determine what local recipients 100Postfix accepts in mail from the network, the mydestination parameter 101that specifies what domains Postfix delivers locally, or the 102mynetworks parameter that specifies the IP addresses of trusted 103clients or client networks. Technically, these are lists, not 104tables. Despite the difference, Postfix lists are described here 105because they use the same underlying infrastructure as Postfix 106lookup tables. </p> 107 108<h2><a name="preparing">Preparing Postfix for LDAP or SQL lookups</a> 109</h2> 110 111<p> LDAP and SQL are complex systems. Trying to set up both Postfix 112and LDAP or SQL at the same time is definitely not a good idea. 113You can save yourself a lot of time by implementing Postfix first 114with local files such as Berkeley DB. Local files have few surprises, 115and are easy to debug with the postmap(1) command: </p> 116 117<blockquote> 118<pre> 119% <b>postmap -q info@example.com hash:/etc/postfix/virtual </b> 120</pre> 121</blockquote> 122 123<p> Once you have local files working properly you can follow the 124instructions in ldap_table(5), mysql_table(5), pgsql_table(5) 125or sqlite_table(5) 126and replace local file lookups with LDAP or SQL lookups. When you 127do this, you should use the postmap(1) command again, to verify 128that database lookups still produce the exact same results as local 129file lookup: </p> 130 131<blockquote> 132<pre> 133% <b>postmap -q info@example.com ldap:/etc/postfix/virtual.cf </b> 134</pre> 135</blockquote> 136 137<p> Be sure to exercise all the partial address or parent domain 138queries that are documented under "table search order" in the 139relevant manual page: access(5), canonical(5), virtual(5), 140transport(5), or under the relevant configuration parameter: 141mynetworks, relay_domains, parent_domain_matches_subdomains. </p> 142 143<h2><a name="detect">Maintaining Postfix lookup table files</a></h2> 144 145<p> When you make changes to a database while the mail system is 146running, it would be desirable if Postfix avoids reading information 147while that information is being changed. It would also be nice if 148you can change a database without having to execute "postfix reload", 149in order to force Postfix to use the new information. Each time 150you do "postfix reload" Postfix loses a lot of performance. 151</p> 152 153<ul> 154 155<li> <p> If you change a network database such as LDAP, NIS or 156SQL, there is no need to execute "postfix reload". The LDAP, NIS 157or SQL server takes care of read/write access conflicts and gives 158the new data to Postfix once that data is available. </p> 159 160<li> <p> If you change a regexp:, pcre:, cidr: or texthash: file 161then Postfix 162may not pick up the file changes immediately. This is because a 163Postfix process reads the entire file into memory once and never 164examines the file again. </p> 165 166<ul> 167 168<li> <p> If the file is used by a short-running process such as 169smtpd(8), cleanup(8) or local(8), there is no need to execute 170"postfix reload" after making a change. </p> 171 172<li> <p> If the file is being used by a long-running process such 173as trivial-rewrite(8) on a busy server it may be necessary to 174execute "postfix reload". </p> 175 176</ul> 177 178<li> <p> If you change a local file based database such as DBM or 179Berkeley DB, there is no need to execute "postfix reload". Postfix 180uses file locking to avoid read/write access conflicts, and whenever 181a Postfix daemon process notices that a file has changed it will 182terminate before handling the next client request, so that a new 183process can initialize with the new database. </p> 184 185</ul> 186 187<h2><a name="safe_db">Updating Berkeley DB files safely</a></h2> 188 189<p> Postfix uses file locking to avoid access conflicts while 190updating Berkeley DB or other local database files. This used to 191be safe, but as Berkeley DB has evolved to use more aggressive 192caching, file locking may no longer be sufficient. </p> 193 194<p> Furthermore, file locking would not prevent problems when the 195update fails because the disk is full or something else causes a 196database update to fail. In particular, commands such as postmap(1) 197or postalias(1) overwrite existing files. If the overwrite 198fails in the middle then you have no usable database, and Postfix 199will stop working. This is not an issue with the CDB database type 200available with Postfix 2.2 and later: <a href="CDB_README.html">CDB</a> 201creates a new file, and renames the file upon successful completion. 202</p> 203 204<p> With Berkeley DB and other "one file" databases, it is 205possible to add some extra robustness by using "mv" to REPLACE an 206existing database file instead of overwriting it: </p> 207 208<blockquote> 209<pre> 210# <b>postmap access.in && mv access.in.db access.db</b> 211</pre> 212</blockquote> 213 214<p> This converts the input file "access.in" into the output file 215"access.in.db", and replaces the file "access.db" only when the 216postmap(1) command was successful. Of course typing such commands 217becomes boring quickly, and this is why people use "make" instead, 218as shown below. User input is shown in bold font. </p> 219 220<blockquote> 221<pre> 222# <b>cat Makefile</b> 223all: aliases.db access.db virtual.db ...etcetera... 224 225# Note 1: commands are specified after a TAB character. 226# Note 2: use postalias(1) for local aliases, postmap(1) for the rest. 227aliases.db: aliases.in 228 postalias aliases.in 229 mv aliases.in.db aliases.db 230 231access.db: access.in 232 postmap access.in 233 mv access.in.db access.db 234 235virtual.db: virtual.in 236 postmap virtual.in 237 mv virtual.in.db virtual.db 238 239...etcetera... 240# <b>vi access.in</b> 241...editing session not shown... 242# <b>make</b> 243postmap access.in 244mv access.in.db access.db 245# 246</pre> 247</blockquote> 248 249<p> The "make" command updates only the files that have changed. 250In case of error, the "make" command will stop and will not invoke 251the "mv" command, so that Postfix will keep using the existing 252database file as if nothing happened. </p> 253 254<h2><a name="types">Postfix lookup table types</a> </h2> 255 256<p> To find out what database types your Postfix system supports, 257use the "<b>postconf -m</b>" command. Here is a list of database types 258that are often supported: </p> 259 260<blockquote> 261 262<dl> 263 264<dt> <b>btree</b> </dt> 265 266<dd> A sorted, balanced tree structure. This is available only on 267systems with support for Berkeley DB databases. Database files are 268created with the postmap(1) or postalias(1) command. The lookup 269table name as used in "btree:table" is the database file name 270without the ".db" suffix. </dd> 271 272<dt> <b>cdb</b> </dt> 273 274<dd> A read-optimized structure with no support for incremental updates. 275Database files are created with the postmap(1) or postalias(1) command. 276The lookup table name as used in "cdb:table" is the database file name 277without the ".cdb" suffix. This feature is available with Postfix 2.2 278and later. </dd> 279 280<dt> <b>cidr</b> </dt> 281 282<dd> A table that associates values with Classless Inter-Domain 283Routing (CIDR) patterns. The table format is described in cidr_table(5). 284</dd> 285 286<dt> <b>dbm</b> </dt> 287 288<dd> An indexed file type based on hashing. This is available only 289on systems with support for DBM databases. Public database files 290are created with the postmap(1) or postalias(1) command, and private 291databases are maintained by Postfix daemons. The lookup table name 292as used in "dbm:table" is the database file name without the ".dir" 293or ".pag" suffix. </dd> 294 295<dt> <b>environ</b> </dt> 296 297<dd> The UNIX process environment array. The lookup key is the 298variable name. The lookup table name in "environ:table" is ignored. 299</dd> 300 301<dt> <b>fail</b> </dt> 302 303<dd> A table that reliably fails all requests. The lookup table 304name is used for logging only. This table exists to simplify Postfix 305error tests. </dd> 306 307<dt> <b>hash</b> </dt> 308 309<dd> An indexed file type based on hashing. This is available only 310on systems with support for Berkeley DB databases. Public database 311files are created with the postmap(1) or postalias(1) command, and 312private databases are maintained by Postfix daemons. The database 313name as used in "hash:table" is the database file name without the 314".db" suffix. </dd> 315 316<dt> <b>internal</b> </dt> 317 318<dd> A non-shared, in-memory hash table. Its content are lost when 319a process terminates. </dd> 320 321<dt> <b>lmdb</b> </dt> 322 323<dd> OpenLDAP LMDB database. This is available only on systems 324with support for LMDB databases. Public database files are created 325with the postmap(1) or postalias(1) command, and private databases 326are maintained by Postfix daemons. The database name as used in 327"lmdb:table" is the database file name without the ".lmdb" suffix. 328See lmdb_table(5) for details. </dd> 329 330<dt> <b>ldap</b> (read-only) </dt> 331 332<dd> LDAP database client. Configuration details are given in the 333ldap_table(5). </dd> 334 335<dt> <b>memcache</b> </dt> 336 337<dd> Memcache database client. Configuration details are given in 338memcache_table(5). </dd> 339 340<dt> <b>mysql</b> (read-only) </dt> 341 342<dd> MySQL database client. Configuration details are given in 343mysql_table(5). </dd> 344 345<dt> <b>netinfo</b> (read-only) </dt> 346 347<dd> Netinfo database client. </dd> 348 349<dt> <b>nis</b> (read-only) </dt> 350 351<dd> NIS database client. </dd> 352 353<dt> <b>nisplus</b> (read-only) </dt> 354 355<dd> NIS+ database client. Configuration details are given in 356nisplus_table(5). </dd> 357 358<dt> <b>pcre</b> (read-only) </dt> 359 360<dd> A lookup table based on Perl Compatible Regular Expressions. 361The file format is described in pcre_table(5). The lookup table 362name as used in "pcre:table" is the name of the regular expression 363file. </dd> 364 365<dt> <b>pgsql</b> (read-only) </dt> 366 367<dd> PostgreSQL database client. Configuration details are given 368in pgsql_table(5). </dd> 369 370<dt> <b>proxy</b> </dt> 371 372<dd> Postfix proxymap(8) client for shared access to Postfix 373databases. The lookup table name syntax is "proxy:type:table". 374</dd> 375 376<dt> <b>regexp</b> (read-only) </dt> 377 378<dd> A lookup table based on regular expressions. The file format 379is described in regexp_table(5). The lookup table name as used in 380"regexp:table" is the name of the regular expression file. </dd> 381 382<dt> <b>sdbm</b> </dt> 383 384<dd> An indexed file type based on hashing. This is available only 385on systems with support for SDBM databases. Public database files 386are created with the postmap(1) or postalias(1) command, and private 387databases are maintained by Postfix daemons. The lookup table name 388as used in "sdbm:table" is the database file name without the ".dir" 389or ".pag" suffix. </dd> 390 391<dt> <b>socketmap</b> (read-only) </dt> 392 393<dd> Sendmail-style socketmap client. The name of the table is 394either <b>inet</b>:<i>host</i>:<i>port</i>:<i>name</i> for a TCP/IP 395server, or <b>unix</b>:<i>pathname</i>:<i>name</i> for a UNIX-domain 396server. See socketmap_table(5) for details. </dd> 397 398<dt> <b>sqlite</b> (read-only) </dt> 399 400<dd> SQLite database. Configuration details are given in sqlite_table(5). 401</dd> 402 403<dt> <b>static</b> (read-only) </dt> 404 405<dd> A table that always returns its name as the lookup result. 406For example, "static:foobar" always returns the string "foobar" as 407lookup result. </dd> 408 409<dt> <b>tcp</b> </dt> 410 411<dd> TCP/IP client. The protocol is described in tcp_table(5). The 412lookup table name is "tcp:host:port" where "host" specifies a 413symbolic hostname or a numeric IP address, and "port" specifies a 414symbolic service name or a numeric port number. </dd> 415 416<dt> <b>texthash</b> (read-only) </dt> 417 418<dd> A table that produces similar results as hash: files, except 419that you don't have to run the postmap(1) command before you can 420use the file, and that texthash: does not detect changes after the 421file is read. The lookup table name is "texthash:filename", where 422the file name is taken literally; no suffix is appended. </dd> 423 424<dt> <b>unix</b> (read-only) </dt> 425 426<dd> A limited view of the UNIX authentication database. The following 427tables are implemented: 428 429<dl> 430 431<dt> <b>unix:passwd.byname</b> </dt> 432 433<dd>The table is the UNIX password database. The key is a login 434name. The result is a password file entry in passwd(5) format. 435</dd> 436 437<dt> <b>unix:group.byname</b> </dt> 438 439<dd> The table is the UNIX group database. The key is a group name. 440The result is a group file entry in group(5) format. </dd> 441 442</dl> </dd> 443 444</dl> 445 446</blockquote> 447 448<p> Other lookup table types may be available depending on how 449Postfix was built. With some Postfix distributions the list is 450dynamically extensible as support for lookup tables is dynamically 451linked into Postfix. </p> 452 453</body> 454 455</html> 456