1111823Sgshapiro# Copyright (c) 2001-2003 Sendmail, Inc. and its suppliers. 290792Sgshapiro# All rights reserved. 390792Sgshapiro# 490792Sgshapiro# By using this file, you agree to the terms and conditions set 590792Sgshapiro# forth in the LICENSE file which can be found at the top level of 690792Sgshapiro# the sendmail distribution. 790792Sgshapiro# 8168515Sgshapiro# $Id: TUNING,v 1.21 2006/09/25 16:45:05 ca Exp $ 990792Sgshapiro# 1090792Sgshapiro 1190792Sgshapiro******************************************** 1290792Sgshapiro** This is a DRAFT, comments are welcome! ** 1390792Sgshapiro******************************************** 1490792Sgshapiro 1590792Sgshapiro 1690792SgshapiroIf the default configuration of sendmail does not achieve the 1790792Sgshapirorequired performance, there are several configuration options that 1890792Sgshapirocan be changed to accomplish higher performance. However, before 1990792Sgshapirothose options are changed it is necessary to understand why the 2090792Sgshapiroperformance is not as good as desired. This may also involve hardware 2190792Sgshapiroand software (OS) configurations which are not extensively explored 2290792Sgshapiroin this document. We assume that your system is not limited by 2390792Sgshapironetwork bandwidth because optimizing for this situation is beyond 2490792Sgshapirothe scope of this guide. In almost all other cases performance will 2590792Sgshapirobe limited by disk I/O. 2690792Sgshapiro 2790792Sgshapiro 2890792SgshapiroThis text assumes that all options which are mentioned here are 2990792Sgshapirofamiliar to the reader, they are explained in the Sendmail Installation 3090792Sgshapiroand Operations Guide; doc/op/op.txt. 3190792Sgshapiro 3290792SgshapiroThere are basically three different scenarios which are treated 3390792Sgshapiroin the following: 3490792Sgshapiro* Mailing Lists and Large Aliases (1-n Mailing) 3590792Sgshapiro* 1-1 Mass Mailing 3690792Sgshapiro* High Volume Mail 3790792Sgshapiro 3890792SgshapiroDepending on your requirements, these may need different options 3990792Sgshapiroto optimize sendmail for the particular purpose. It is also possible 4090792Sgshapiroto configure sendmail to achieve good performance in all cases, but 4190792Sgshapiroit will not be optimal for any specific purpose. For example, it 42168515Sgshapirois non-trivial to combine low latency (fast delivery of incoming 4390792Sgshapiromail) with high overall throughput. 4490792Sgshapiro 4590792SgshapiroBefore we explore the different scenarios, a basic discussion about 4690792Sgshapirodisk I/O, delivery modes, and queue control is required. 4790792Sgshapiro 4890792Sgshapiro 4990792Sgshapiro* Disk I/O 5090792Sgshapiro----------------------------------------------- 5190792Sgshapiro 52168515SgshapiroIn general mail will be written to disk before a delivery attempt 5390792Sgshapirois made. This is required for reliability and should only be changed 5490792Sgshapiroin a few specific cases that are mentioned later on. To achieve 5590792Sgshapirobetter disk I/O performance the queue directories can be spread 5690792Sgshapiroover several disks to distribute the load. This is some basic tuning 5790792Sgshapirothat should be done in all cases where the I/O speed of a single 5890792Sgshapirodisk is exceeded, which is true for almost every high-volume 5990792Sgshapirosituation except if a special disk subsystem with large (NV)RAM 6090792Sgshapirobuffer is used. 6190792Sgshapiro 6290792SgshapiroDepending on your OS there might be ways to speed up I/O, e.g., 6390792Sgshapirousing softupdates or turning on the noatime mount option. If this 6490792Sgshapirois done make sure the filesystem is still reliable, i.e., if fsync() 6590792Sgshapiroreturns without an error, the file has really been committed to 6690792Sgshapirodisk. 6790792Sgshapiro 6890792Sgshapiro 6990792Sgshapiro* Queueing Strategies and DeliveryMode 7090792Sgshapiro----------------------------------------------- 7190792Sgshapiro 7290792SgshapiroThere are basically three delivery modes: 7390792Sgshapiro 7490792Sgshapirobackground: incoming mail will be immediately delivered by a new process 7590792Sgshapirointeractive: incoming mail will be immediately delivered by the same process 7690792Sgshapiroqueue: incoming mail will be queued and delivered by a queue runner later on 7790792Sgshapiro 7890792SgshapiroThe first offers the lowest latency without the disadvantage of the 79168515Sgshapirosecond, which keeps the connection from the sender open until the 8090792Sgshapirodelivery to the next hop succeeded or failed. However, it does not 8190792Sgshapiroallow for a good control over the number of delivery processes other 8290792Sgshapirothan limiting the total number of direct children of the daemon 8390792Sgshapiroprocesses (MaxChildren) or by load control options (RefuseLA, 8490792SgshapiroDelayLA). Moreover, it can't make as good use as 'queue' mode can 8590792Sgshapirofor connection caching. 8690792Sgshapiro 8790792SgshapiroInteractive DeliveryMode should only be used in rare cases, e.g., 8890792Sgshapiroif the delivery time to the next hop is a known quantity or if the 8990792Sgshapirosender is under local control and it does not matter if it has to 9090792Sgshapirowait for delivery. 9190792Sgshapiro 9290792SgshapiroQueueing up e-mail before delivery is done by a queue runner allows 9390792Sgshapirothe best load control but does not achieve as low latency as the 9490792Sgshapiroother two modes. However, this mode is probably also best for 9590792Sgshapiroconcurrent delivery since the number of queue runners can be specified 9690792Sgshapiroon a queue group basis. Persistent queue runners (-qp) can be used 9790792Sgshapiroto minimize the overhead for creating processes because they just 98168515Sgshapirosleep for the specified interval (which should be short) instead of 9990792Sgshapiroexiting after a queue run. 10090792Sgshapiro 10190792Sgshapiro 10290792Sgshapiro* Queue Groups 10390792Sgshapiro----------------------------------------------- 10490792Sgshapiro 10590792SgshapiroIn most situations disk I/O is a bottleneck which can be mitigated 10690792Sgshapiroby spreading the load over several disks. This can easily be achieved 10790792Sgshapirowith different queue directories. sendmail 8.12 introduces queue 10890792Sgshapirogroups which are collections of queue directories with similar 10990792Sgshapiroproperties, i.e., number of processes to run the queues in the 11090792Sgshapirogroup, maximum number of recipients within an e-mail (envelope), 11190792Sgshapiroetc. Queue groups allow control over the behaviour of different 11290792Sgshapiroqueues. Depending on the setup, it is usually possible to have 11390792Sgshapiroseveral queue runners delivering mails concurrently which should 11490792Sgshapiroincrease throughput. The number of queue runners can be controlled 11590792Sgshapiroper queue group (Runner=) and overall (MaxQueueChildren). 11690792Sgshapiro 11790792Sgshapiro 11890792Sgshapiro* DNS Lookups 11990792Sgshapiro----------------------------------------------- 12090792Sgshapiro 12190792Sgshapirosendmail performs by default host name canonifications by using 12290792Sgshapirohost name lookups. This process is meant to replace unqualified 12390792Sgshapirohost name with qualified host names, and CNAMEs with the non-aliased 12490792Sgshapironame. However, these lookups can take a while for large address 12590792Sgshapirolists, e.g., mailing lists. If you can assure by other means that 12690792Sgshapirohost names are canonical, you should use 12790792Sgshapiro 12890792Sgshapiro FEATURE(`nocanonify', `canonify_hosts') 12990792Sgshapiro 13090792Sgshapiroin your .mc file. For further information on this feature and 13190792Sgshapiroadditional options see cf/README. If sendmail is invoked directly 13290792Sgshapiroto send e-mail then either the -G option should be used or 13390792Sgshapiro 13490792Sgshapiro define(`confDIRECT_SUBMISSION_MODIFIERS', `C') 13590792Sgshapiro 13690792Sgshapiroshould be added to the .mc file. 13790792Sgshapiro 13890792Sgshapiro 13990792Sgshapiro* Mailing Lists and Large Aliases (1-n Mailing) 14090792Sgshapiro----------------------------------------------- 14190792Sgshapiro 142168515SgshapiroBefore 8.12 sendmail would deliver an e-mail sequentially to all its 14390792Sgshapirorecipients. For mailing lists or large aliases the overall delivery 14494334Sgshapirotime can be substantial, especially if some of the recipients are 14594334Sgshapirolocated at hosts that are slow to accept e-mail. Some mailing list 14694334Sgshapirosoftware therefore "split" up e-mails into smaller pieces with 14794334Sgshapirofewer recipients. sendmail 8.12 can do this itself, either across 14894334Sgshapiroqueue groups or within a queue directory. The latter is controlled 14994334Sgshapiroby the 'r=' field of a queue group declaration. 15090792Sgshapiro 151168515SgshapiroLet's assume a simple example: a mailing list where most of the 152168515Sgshapirorecipients are at three domains: the local one (local.domain) and 153168515Sgshapirotwo remotes (one.domain, two.domain) and the rest is splittered 15490792Sgshapiroover several other domains. For this case it is useful to specify 15590792Sgshapirothree queue groups: 15690792Sgshapiro 15790792SgshapiroQUEUE_GROUP(`local', `P=/var/spool/mqueue/local, F=f, R=2, I=1m')dnl 15890792SgshapiroQUEUE_GROUP(`one', `P=/var/spool/mqueue/one, F=f, r=50, R=3')dnl 15990792SgshapiroQUEUE_GROUP(`two', `P=/var/spool/mqueue/two, F=f, r=30, R=4')dnl 16090792SgshapiroQUEUE_GROUP(`remote', `P=/var/spool/mqueue/remote, F=f, r=5, R=8, I=2m')dnl 16190792Sgshapirodefine(`ESMTP_MAILER_QGRP', `remote')dnl 16290792Sgshapirodefine(`confDELIVERY_MODE', `q')dnl 16390792Sgshapirodefine(`confMAX_QUEUE_CHILDREN', `50')dnl 16490792Sgshapirodefine(`confMIN_QUEUE_AGE', `27m')dnl 16590792Sgshapiro 16690792Sgshapiroand specify the queuegroup ruleset as follows: 16790792Sgshapiro 16890792SgshapiroLOCAL_RULESETS 16990792SgshapiroSqueuegroup 17090792SgshapiroR$* @ local.domain $# local 17190792SgshapiroR$* @ $* one.domain $# one 17290792SgshapiroR$* @ $* two.domain $# two 17390792SgshapiroR$* @ $* $# remote 17490792SgshapiroR$* $# mqueue 17590792Sgshapiro 17690792SgshapiroNow it is necessary to control the number of queue runners, which 17790792Sgshapirois done by MaxQueueChildren. Starting the daemon with the option 17890792Sgshapiro-q5m assures that the first delivery attempt for each e-mail is 17990792Sgshapirodone within 5 minutes, however, there are also individual queue 18090792Sgshapirointervals for the queue groups as specified above. MinQueueAge 18190792Sgshapirois set to 27 minutes to avoid that entries are run too often. 18290792Sgshapiro 18390792SgshapiroNotice: if envelope splitting happens due to alias expansion, and 18490792SgshapiroDeliveryMode is not 'i'nteractive, then only one envelope is sent 18590792Sgshapiroimmediately. The rest (after splitting) are queued up and queue 18690792Sgshapirorunners must come along and take care of them. Hence it is essential 18790792Sgshapirothat the queue interval is very short. 18890792Sgshapiro 18990792Sgshapiro 19090792Sgshapiro* 1-1 Mass Mailing 19190792Sgshapiro----------------------------------------------- 19290792Sgshapiro 19390792SgshapiroIn this case some program generates e-mails which are sent to 19490792Sgshapiroindividual recipients (or at most very few per e-mail). A simple 19590792Sgshapiroway to achieve high throughput is to set the delivery mode to 19690792Sgshapiro'interactive', turn off the SuperSafe option and make sure that the 19790792Sgshapiroprogram that generates the mails can deal with mail losses if the 19890792Sgshapiroserver loses power. In no other case should SuperSafe be set to 19990792Sgshapiro'false'. If these conditions are met, sendmail does not need to 20090792Sgshapirocommit mails to disk but can buffer them in memory which will greatly 20190792Sgshapiroenhance performance, especially compared to normal disk subsystems, e.g., 20290792Sgshapironon solid-state disks. 20390792Sgshapiro 20490792Sgshapiro 20590792Sgshapiro* High Volume Mail 20690792Sgshapiro----------------------------------------------- 20790792Sgshapiro 20890792SgshapiroFor high volume mail it is necessary to be able to control the load 20990792Sgshapiroon the system. Therefore the 'queue' delivery mode should be used, 21090792Sgshapiroand all options related to number of processes and the load should 21190792Sgshapirobe set to reasonable values. It is important not to accept mail 212168515Sgshapirofaster than it can be delivered; otherwise the system will be 21390792Sgshapirooverwhelmed. Hence RefuseLA should be lower than QueueLA, the number 21490792Sgshapiroof daemon children should probably be lower than the number of queue 215168515Sgshapirorunners (MaxChildren vs. MaxQueueChildren). DelayLA is a new option 21690792Sgshapiroin 8.12 which allows delaying connections instead of rejecting them. 21790792SgshapiroThis may result in a smoother load distribution depending on how 21890792Sgshapirothe mails are submitted to sendmail. 21990792Sgshapiro 22090792Sgshapiro 22190792Sgshapiro* Miscellaneous 22290792Sgshapiro----------------------------------------------- 22390792Sgshapiro 22490792SgshapiroOther options that are interesting to tweak performance are 22590792Sgshapiro(in no particular order): 22690792Sgshapiro 22790792SgshapiroSuperSafe: if interactive DeliveryMode is used, then this can 22890792Sgshapirobe set to the new value "interactive" in 8.12 to save some disk 22990792Sgshapirosynchronizations which are not really necessary in that mode. 23090792Sgshapiro 231