1<!doctype html public "-//W3C//DTD HTML 4.01 Transitional//EN" 2 "http://www.w3.org/TR/html4/loose.dtd"> 3 4<html> 5 6<head> 7 8<title>Postfix Before-Queue Content Filter </title> 9 10<meta http-equiv="Content-Type" content="text/html; charset=us-ascii"> 11 12</head> 13 14<body> 15 16<h1><img src="postfix-logo.jpg" width="203" height="98" ALT="">Postfix Before-Queue Content Filter </h1> 17 18<hr> 19 20<h2>WARNING </h2> 21 22<p> The before-queue content filtering feature described in this 23document limits the amount of mail that a site can handle. See the 24"<a href="#pros_cons">Pros and Cons</a>" section below for details. 25</p> 26 27<h2>The Postfix before-queue content filter feature</h2> 28 29<p> As of version 2.1, the Postfix SMTP server can forward all 30incoming mail to a content filtering proxy server that inspects all 31mail BEFORE it is stored in the Postfix mail queue. It is roughly 32equivalent in capabilities to the approach described in MILTER_README, 33except that the latter uses a dedicated protocol instead of SMTP. 34 35<p> The before-queue content filter is meant to be used as follows: </p> 36 37<blockquote> 38 39<table> 40 41<tr> 42 43 <td bgcolor="#f0f0ff" align="center" valign="middle" 44 width="10%"> Internet </td> 45 46 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 47 48 <td bgcolor="#f0f0ff" align="center" valign="middle" 49 width="10%"> <a href="smtpd.8.html">Postfix SMTP server</a> 50 </td> 51 52 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 53 54 <td bgcolor="#f0f0ff" align="center" valign="middle" 55 width="10%"> <b>Before</b> <b>queue</b> <b>filter</b> </td> 56 57 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 58 59 <td bgcolor="#f0f0ff" align="center" valign="middle" 60 width="10%"> <a href="smtpd.8.html">Postfix SMTP server</a> 61 </td> 62 63 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 64 65 <td bgcolor="#f0f0ff" align="center" valign="middle" 66 width="10%"> <a href="cleanup.8.html">Postfix cleanup 67 server</a> </td> 68 69 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 70 71 <td bgcolor="#f0f0ff" align="center" valign="middle" 72 width="10%"> Postfix queue </td> 73 74 <td align="center" valign="middle" width="5%"> <tt> -< </tt> </td> 75 76 <td bgcolor="#f0f0ff" align="center" valign="middle" 77 width="10%"> <a href="smtp.8.html">smtp</a><br> <a 78 href="local.8.html">local</a><br> <a 79 href="virtual.8.html">virtual</a> </td> 80 81</tr> 82 83</table> 84 85</blockquote> 86 87<p> The before-queue content filter is not to be confused with the 88approach described in the FILTER_README document, where mail is 89filtered AFTER it is stored in the Postfix mail queue. </p> 90 91<p> This document describes the following topics: </p> 92 93<ul> 94 95<li><a href="#principles">Principles of operation</a> 96 97<li><a href="#pros_cons">Pros and cons of before-queue content filtering</a> 98 99<li><a href="#config">Configuring the Postfix SMTP pass-through 100proxy feature</a> 101 102<li><a href="#parameters">Configuration parameters</a> 103 104<li><a href="#protocol">How Postfix talks to the before-queue content 105filter</a> 106 107</ul> 108 109<h2><a name="principles">Principles of operation</a></h2> 110 111<p> As shown in the diagram above, the before-queue filter sits 112between two Postfix SMTP server processes. </p> 113 114<ul> 115 116<li> <p> The before-filter Postfix SMTP server accepts connections from the 117Internet and does the usual relay access control, SASL authentication, 118TLS negotiation, 119RBL lookups, rejecting non-existent sender or recipient addresses, 120etc. </p> 121 122<li> <p> The before-queue filter receives unfiltered mail content from 123Postfix and does one of the following: </p> 124 125<ol> 126 127 <li> <p> Re-inject the mail back into Postfix via SMTP, perhaps 128 after changing its content and/or destination. </p> 129 130 <li> <p> Discard or quarantine the mail. </p> 131 132 <li> <p> Reject the mail by sending a suitable SMTP status code 133 back to Postfix. Postfix passes the status back to the remote 134 SMTP client. This way, Postfix does not have to send a bounce 135 message. </p> 136 137</ol> 138 139<li> <p>The after-filter Postfix SMTP server receives mail from the 140content filter. From then on Postfix processes the mail as usual. </p> 141 142</ul> 143 144<p> The before-queue content filter described here works just like 145the after-queue content filter described in the FILTER_README 146document. In many cases you can use the same software, within the 147limitations as discussed in the "<a href="#pros_cons">Pros and 148Cons</a>" section below. </p> 149 150<h2><a name="pros_cons">Pros and cons of before-queue content 151filtering</a></h2> 152 153<ul> 154 155<li> <p> Pro: Postfix can reject mail before the incoming SMTP mail 156transfer completes, so that Postfix does not have to send rejected 157mail back to the sender (which is usually forged anyway). Mail 158that is not accepted remains the responsibility of the remote SMTP 159client. </p> 160 161<li> <p> Con: The remote SMTP client expects an SMTP reply within 162a deadline. As the system load increases, fewer and fewer CPU 163cycles remain available to answer within the deadline, and eventually 164you either have to stop accepting mail or you have to stop filtering 165mail. It is for this reason that the before-queue content filter 166limits the amount of mail that a site can handle. </p> 167 168<li> <p> Con: Content filtering software can use lots of memory 169resources. You have to reduce the number of simultaneous content 170filter processes so that a burst of mail will not drive your system 171into the ground. </p> 172 173<ul> 174 175<li> <p> With Postfix versions 2.7 and later, SMTP clients will 176experience an increase in the delay between the time the client 177sends "end-of-message" and the time the Postfix SMTP server replies 178(here, the number of before-filter SMTP server processes can be 179larger than the number of filter processes). </p> 180 181<li> <p> With Postfix versions before 2.7, SMTP clients will 182experience an increase in the delay before they can receive service 183(here, the number of before-filter SMTP server processes is always 184equal to the number of filter processes). </p> 185 186</ul> 187 188</ul> 189 190<h2><a name="config">Configuring the Postfix SMTP pass-through 191proxy feature</a></h2> 192 193<p> In the following example, the before-filter Postfix SMTP server 194gives mail to a content filter that listens on localhost port 10025. 195The after-filter Postfix SMTP server receives mail from the content 196filter via localhost port 10026. From then on mail is processed as 197usual. </p> 198 199<p> The content filter itself is not described here. You can use 200any filter that is SMTP enabled. For non-SMTP capable content 201filtering software, Bennett Todd's SMTP proxy implements a nice 202PERL/SMTP content filtering framework. See: 203http://bent.latency.net/smtpprox/. </p> 204 205<blockquote> 206 207<table border="0"> 208 209<tr> 210 211 <td bgcolor="#f0f0ff" align="center" valign="middle" 212 width="10%"> Internet </td> 213 214 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 215 216 <td bgcolor="#f0f0ff" align="center" valign="middle" 217 width="10%"> <a href="smtpd.8.html">Postfix SMTP server on 218 port 25</a> </td> 219 220 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 221 222 <td bgcolor="#f0f0ff" align="center" valign="middle" 223 width="10%"> filter on localhost port 10025 </td> 224 225 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 226 227 <td bgcolor="#f0f0ff" align="center" valign="middle" 228 width="10%"> <a href="smtpd.8.html">Postfix SMTP server on 229 localhost port 10026</a> </td> 230 231 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 232 233 <td bgcolor="#f0f0ff" align="center" valign="middle" 234 width="10%"> <a href="cleanup.8.html">Postfix cleanup 235 server</a> </td> 236 237 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 238 239 <td bgcolor="#f0f0ff" align="center" valign="middle" 240 width="10%"> Postfix incoming queue </td> 241 242</tr> 243 244</table> 245 246</blockquote> 247 248<p> This is configured by editing the master.cf file: </p> 249 250<blockquote> 251<pre> 252/etc/postfix/master.cf: 253 # ============================================================= 254 # service type private unpriv chroot wakeup maxproc command 255 # (yes) (yes) (yes) (never) (100) 256 # ============================================================= 257 # 258 # Before-filter SMTP server. Receive mail from the network and 259 # pass it to the content filter on localhost port 10025. 260 # 261 smtp inet n - n - 20 smtpd 262 -o smtpd_proxy_filter=127.0.0.1:10025 263 -o smtpd_client_connection_count_limit=10 264 # Postfix 2.7 and later performance feature. 265 # -o smtpd_proxy_options=speed_adjust 266 # 267 # After-filter SMTP server. Receive mail from the content filter 268 # on localhost port 10026. 269 # 270 127.0.0.1:10026 inet n - n - - smtpd 271 -o smtpd_authorized_xforward_hosts=127.0.0.0/8 272 -o smtpd_client_restrictions= 273 -o smtpd_helo_restrictions= 274 -o smtpd_sender_restrictions= 275 -o smtpd_recipient_restrictions=permit_mynetworks,reject 276 -o smtpd_data_restrictions= 277 -o mynetworks=127.0.0.0/8 278 -o receive_override_options=no_unknown_recipient_checks 279</pre> 280</blockquote> 281 282<p> Note: do not specify spaces around the "=" or "," characters. </p> 283 284<p> The before-filter SMTP server entry is a modified version of the 285default Postfix SMTP server entry that is normally configured at 286the top of the master.cf file: </p> 287 288<ul> 289 290 <li> <p> The number of SMTP sessions is reduced from the default 291 100 to only 20. This prevents a burst of mail from running your 292 system into the ground with too many content filter processes. </p> 293 294 <li> <p> The "-o smtpd_client_connection_count_limit=10" prevents 295 one SMTP client from using up all 20 SMTP server processes. 296 This limit is not necessary if you receive all mail from a 297 trusted relay host. </p> 298 299 <p> Note: this setting is available in Postfix version 2.2 and 300 later. Earlier Postfix versions will ignore it. </p> 301 302 <li> <p> The "-o smtpd_proxy_filter=127.0.0.1:10025" tells the 303 before-filter SMTP server that it should give incoming mail to 304 the content filter that listens on localhost TCP port 10025. 305 306 <li> <p> The "-o smtpd_proxy_options=speed_adjust" tells the 307 before-filter SMTP server that it should receive an entire email 308 message before it connects to a content filter. This reduces 309 the number of simultaneous filter processes. </p> 310 311 <p> NOTE 1: When this option is turned on, a content filter must 312 not <i>selectively</i> reject recipients of a multi-recipient 313 message. Rejecting all recipients is OK, as is accepting all 314 recipients. </p> 315 316 <p> NOTE 2: This feature increases the minimum amount of free 317 queue space by $message_size_limit. The extra space is needed 318 to save the message to a temporary file. </p> 319 320 <li> <p> Postfix ≥ 2.3 supports both TCP and UNIX-domain filters. 321 The above filter could be specified as "inet:127.0.0.1:10025". 322 To specify a UNIX-domain filter, specify "unix:<i>pathname</i>". 323 A relative pathname is interpreted relative to the Postfix queue 324 directory. </p> 325 326</ul> 327 328<p> The after-filter SMTP server is a new master.cf entry: </p> 329 330<ul> 331 332 <li> <p> The "127.0.0.1:10026" makes the after-filter SMTP 333 server listen 334 on the localhost address only, without exposing it to the 335 network. NEVER expose the after-filter SMTP server to the 336 Internet :-) </p> 337 338 <li> <p> The "-o smtpd_authorized_xforward_hosts=127.0.0.0/8" 339 allows the after-filter SMTP server to receive remote SMTP 340 client information from the before-filter SMTP server, so that 341 the after-filter Postfix daemons log the remote SMTP client 342 information instead of logging localhost[127.0.0.1]. </p> 343 344 <li> <p> The other after-filter SMTP server settings avoid 345 duplication of work that is already done in the "before filter" 346 SMTP server. </p> 347 348</ul> 349 350<p> By default, the filter has 100 seconds to do its work. If it 351takes longer then Postfix gives up and reports an error to the 352remote SMTP client. You can increase this time limit (see configuration 353parameter section below) but doing so is pointless because you 354can't control when the remote SMTP client times out. </p> 355 356<h2><a name="parameters">Configuration parameters</a></h2> 357 358<p> Parameters that control proxying: </p> 359 360<ul> 361 362<li> <p> smtpd_proxy_filter (syntax: host:port): The host and TCP 363port of the before-queue content filter. When no host or host: 364is specified here, localhost is assumed. </p> 365 366<li> <p> smtpd_proxy_timeout (default: 100s): Timeout for connecting 367to the before-queue content filter and for sending and receiving 368commands and data. All proxy errors are logged to the maillog 369file. For privacy reasons, all the remote SMTP client sees is "451 370Error: queue file write error". It would not be right to disclose 371internal details to strangers. </p> 372 373<li> <p> smtpd_proxy_ehlo (default: $myhostname): The hostname to 374use when sending an EHLO command to the before-queue content filter. 375</p> 376 377</ul> 378 379<h2><a name="protocol">How Postfix talks to the before-queue content 380filter</a></h2> 381 382<p> The before-filter Postfix SMTP server connects to the content 383filter, delivers one message, and disconnects. While sending mail 384into the content filter, Postfix speaks ESMTP but uses no command 385pipelining. Postfix generates its own EHLO, XFORWARD (for logging 386the remote client IP address instead of localhost[127.0.0.1]), DATA 387and QUIT commands, and forwards unmodified copies of all the MAIL 388FROM and RCPT TO commands that the before-filter Postfix SMTP server 389didn't reject itself. 390Postfix sends no other SMTP commands. </p> 391 392<p> The content filter should accept the same MAIL FROM and RCPT 393TO command syntax as the before-filter Postfix SMTP server, and 394should forward the commands without modification to the after-filter 395SMTP server. If the content filter or after-filter SMTP server 396does not support all the ESMTP features that the before-filter 397Postfix SMTP server supports, then the missing features must be 398turned off in the before-filter Postfix SMTP server with the 399smtpd_discard_ehlo_keywords parameter. </p> 400 401<p> When the filter rejects content, it should send a negative SMTP 402response back to the before-filter Postfix SMTP server, and it 403should abort the connection with the after-filter Postfix SMTP 404server without completing the SMTP conversation with the after-filter 405Postfix SMTP server. </p> 406 407</body> 408 409</html> 410
