TUNING revision 90792
190792Sgshapiro# Copyright (c) 2001 Sendmail, Inc. and its suppliers. 290792Sgshapiro# All rights reserved. 390792Sgshapiro# 490792Sgshapiro# By using this file, you agree to the terms and conditions set 590792Sgshapiro# forth in the LICENSE file which can be found at the top level of 690792Sgshapiro# the sendmail distribution. 790792Sgshapiro# 890792Sgshapiro# $Id: TUNING,v 1.16 2001/08/19 21:03:38 gshapiro Exp $ 990792Sgshapiro# 1090792Sgshapiro 1190792Sgshapiro******************************************** 1290792Sgshapiro** This is a DRAFT, comments are welcome! ** 1390792Sgshapiro******************************************** 1490792Sgshapiro 1590792Sgshapiro 1690792SgshapiroIf the default configuration of sendmail does not achieve the 1790792Sgshapirorequired performance, there are several configuration options that 1890792Sgshapirocan be changed to accomplish higher performance. However, before 1990792Sgshapirothose options are changed it is necessary to understand why the 2090792Sgshapiroperformance is not as good as desired. This may also involve hardware 2190792Sgshapiroand software (OS) configurations which are not extensively explored 2290792Sgshapiroin this document. We assume that your system is not limited by 2390792Sgshapironetwork bandwidth because optimizing for this situation is beyond 2490792Sgshapirothe scope of this guide. In almost all other cases performance will 2590792Sgshapirobe limited by disk I/O. 2690792Sgshapiro 2790792Sgshapiro 2890792SgshapiroThis text assumes that all options which are mentioned here are 2990792Sgshapirofamiliar to the reader, they are explained in the Sendmail Installation 3090792Sgshapiroand Operations Guide; doc/op/op.txt. 3190792Sgshapiro 3290792SgshapiroThere are basically three different scenarios which are treated 3390792Sgshapiroin the following: 3490792Sgshapiro* Mailing Lists and Large Aliases (1-n Mailing) 3590792Sgshapiro* 1-1 Mass Mailing 3690792Sgshapiro* High Volume Mail 3790792Sgshapiro 3890792SgshapiroDepending on your requirements, these may need different options 3990792Sgshapiroto optimize sendmail for the particular purpose. It is also possible 4090792Sgshapiroto configure sendmail to achieve good performance in all cases, but 4190792Sgshapiroit will not be optimal for any specific purpose. For example, it 4290792Sgshapirois non-trivival to combine low latency (fast delivery of incoming 4390792Sgshapiromail) with high overall throughput. 4490792Sgshapiro 4590792SgshapiroBefore we explore the different scenarios, a basic discussion about 4690792Sgshapirodisk I/O, delivery modes, and queue control is required. 4790792Sgshapiro 4890792Sgshapiro 4990792Sgshapiro* Disk I/O 5090792Sgshapiro----------------------------------------------- 5190792Sgshapiro 5290792SgshapiroIn general mail will be written to disk up before a delivery attempt 5390792Sgshapirois made. This is required for reliability and should only be changed 5490792Sgshapiroin a few specific cases that are mentioned later on. To achieve 5590792Sgshapirobetter disk I/O performance the queue directories can be spread 5690792Sgshapiroover several disks to distribute the load. This is some basic tuning 5790792Sgshapirothat should be done in all cases where the I/O speed of a single 5890792Sgshapirodisk is exceeded, which is true for almost every high-volume 5990792Sgshapirosituation except if a special disk subsystem with large (NV)RAM 6090792Sgshapirobuffer is used. 6190792Sgshapiro 6290792SgshapiroDepending on your OS there might be ways to speed up I/O, e.g., 6390792Sgshapirousing softupdates or turning on the noatime mount option. If this 6490792Sgshapirois done make sure the filesystem is still reliable, i.e., if fsync() 6590792Sgshapiroreturns without an error, the file has really been committed to 6690792Sgshapirodisk. 6790792Sgshapiro 6890792Sgshapiro 6990792Sgshapiro* Queueing Strategies and DeliveryMode 7090792Sgshapiro----------------------------------------------- 7190792Sgshapiro 7290792SgshapiroThere are basically three delivery modes: 7390792Sgshapiro 7490792Sgshapirobackground: incoming mail will be immediately delivered by a new process 7590792Sgshapirointeractive: incoming mail will be immediately delivered by the same process 7690792Sgshapiroqueue: incoming mail will be queued and delivered by a queue runner later on 7790792Sgshapiro 7890792SgshapiroThe first offers the lowest latency without the disadvantage of the 7990792Sgshapirosecond, which keep the connection from the sender open until the 8090792Sgshapirodelivery to the next hop succeeded or failed. However, it does not 8190792Sgshapiroallow for a good control over the number of delivery processes other 8290792Sgshapirothan limiting the total number of direct children of the daemon 8390792Sgshapiroprocesses (MaxChildren) or by load control options (RefuseLA, 8490792SgshapiroDelayLA). Moreover, it can't make as good use as 'queue' mode can 8590792Sgshapirofor connection caching. 8690792Sgshapiro 8790792SgshapiroInteractive DeliveryMode should only be used in rare cases, e.g., 8890792Sgshapiroif the delivery time to the next hop is a known quantity or if the 8990792Sgshapirosender is under local control and it does not matter if it has to 9090792Sgshapirowait for delivery. 9190792Sgshapiro 9290792SgshapiroQueueing up e-mail before delivery is done by a queue runner allows 9390792Sgshapirothe best load control but does not achieve as low latency as the 9490792Sgshapiroother two modes. However, this mode is probably also best for 9590792Sgshapiroconcurrent delivery since the number of queue runners can be specified 9690792Sgshapiroon a queue group basis. Persistent queue runners (-qp) can be used 9790792Sgshapiroto minimize the overhead for creating processes because they just 9890792Sgshapirosleep for the specified interval (which shold be short) instead of 9990792Sgshapiroexiting after a queue run. 10090792Sgshapiro 10190792Sgshapiro 10290792Sgshapiro* Queue Groups 10390792Sgshapiro----------------------------------------------- 10490792Sgshapiro 10590792SgshapiroIn most situations disk I/O is a bottleneck which can be mitigated 10690792Sgshapiroby spreading the load over several disks. This can easily be achieved 10790792Sgshapirowith different queue directories. sendmail 8.12 introduces queue 10890792Sgshapirogroups which are collections of queue directories with similar 10990792Sgshapiroproperties, i.e., number of processes to run the queues in the 11090792Sgshapirogroup, maximum number of recipients within an e-mail (envelope), 11190792Sgshapiroetc. Queue groups allow control over the behaviour of different 11290792Sgshapiroqueues. Depending on the setup, it is usually possible to have 11390792Sgshapiroseveral queue runners delivering mails concurrently which should 11490792Sgshapiroincrease throughput. The number of queue runners can be controlled 11590792Sgshapiroper queue group (Runner=) and overall (MaxQueueChildren). 11690792Sgshapiro 11790792Sgshapiro 11890792Sgshapiro* DNS Lookups 11990792Sgshapiro----------------------------------------------- 12090792Sgshapiro 12190792Sgshapirosendmail performs by default host name canonifications by using 12290792Sgshapirohost name lookups. This process is meant to replace unqualified 12390792Sgshapirohost name with qualified host names, and CNAMEs with the non-aliased 12490792Sgshapironame. However, these lookups can take a while for large address 12590792Sgshapirolists, e.g., mailing lists. If you can assure by other means that 12690792Sgshapirohost names are canonical, you should use 12790792Sgshapiro 12890792Sgshapiro FEATURE(`nocanonify', `canonify_hosts') 12990792Sgshapiro 13090792Sgshapiroin your .mc file. For further information on this feature and 13190792Sgshapiroadditional options see cf/README. If sendmail is invoked directly 13290792Sgshapiroto send e-mail then either the -G option should be used or 13390792Sgshapiro 13490792Sgshapiro define(`confDIRECT_SUBMISSION_MODIFIERS', `C') 13590792Sgshapiro 13690792Sgshapiroshould be added to the .mc file. 13790792Sgshapiro 13890792Sgshapiro 13990792Sgshapiro* Mailing Lists and Large Aliases (1-n Mailing) 14090792Sgshapiro----------------------------------------------- 14190792Sgshapiro 14290792SgshapiroBefore 8.12 sendmail delivers an e-mail sequentially to all its 14390792Sgshapirorecipients. For mailing lists or large aliases the overall delivery 14490792Sgshapirotime can be substantial, especially if some of the recipients are located 14590792Sgshapiroat hosts that are slow to accept e-mail. Some mailing list software 14690792Sgshapirotherefore "split" up e-mails into smaller pieces with fewer recipients. 14790792Sgshapirosendmail 8.12 can do this itself, either across queue groups or 14890792Sgshapirowithin a queue directory. For the former the option SplitAcrossQueueGroups 14990792Sgshapirooption must be set, the latter is controlled by the 'r=' field of 15090792Sgshapiroa queue group declaration. 15190792Sgshapiro 15290792SgshapiroLet's assume a simple example: a mailing lists where most of 15390792Sgshapirothe recipients are at three domains: the local one (local.domain) 15490792Sgshapiroand two remotes (one.domain, two.domain) and the rest is splittered 15590792Sgshapiroover several other domains. For this case it is useful to specify 15690792Sgshapirothree queue groups: 15790792Sgshapiro 15890792SgshapiroQUEUE_GROUP(`local', `P=/var/spool/mqueue/local, F=f, R=2, I=1m')dnl 15990792SgshapiroQUEUE_GROUP(`one', `P=/var/spool/mqueue/one, F=f, r=50, R=3')dnl 16090792SgshapiroQUEUE_GROUP(`two', `P=/var/spool/mqueue/two, F=f, r=30, R=4')dnl 16190792SgshapiroQUEUE_GROUP(`remote', `P=/var/spool/mqueue/remote, F=f, r=5, R=8, I=2m')dnl 16290792Sgshapirodefine(`ESMTP_MAILER_QGRP', `remote')dnl 16390792Sgshapirodefine(`confSPLIT_ACROSS_QUEUEGROUPS', `True')dnl 16490792Sgshapirodefine(`confDELIVERY_MODE', `q')dnl 16590792Sgshapirodefine(`confMAX_QUEUE_CHILDREN', `50')dnl 16690792Sgshapirodefine(`confMIN_QUEUE_AGE', `27m')dnl 16790792Sgshapiro 16890792Sgshapiroand specify the queuegroup ruleset as follows: 16990792Sgshapiro 17090792SgshapiroLOCAL_RULESETS 17190792SgshapiroSqueuegroup 17290792SgshapiroR$* @ local.domain $# local 17390792SgshapiroR$* @ $* one.domain $# one 17490792SgshapiroR$* @ $* two.domain $# two 17590792SgshapiroR$* @ $* $# remote 17690792SgshapiroR$* $# mqueue 17790792Sgshapiro 17890792SgshapiroNow it is necessary to control the number of queue runners, which 17990792Sgshapirois done by MaxQueueChildren. Starting the daemon with the option 18090792Sgshapiro-q5m assures that the first delivery attempt for each e-mail is 18190792Sgshapirodone within 5 minutes, however, there are also individual queue 18290792Sgshapirointervals for the queue groups as specified above. MinQueueAge 18390792Sgshapirois set to 27 minutes to avoid that entries are run too often. 18490792Sgshapiro 18590792SgshapiroNotice: if envelope splitting happens due to alias expansion, and 18690792SgshapiroDeliveryMode is not 'i'nteractive, then only one envelope is sent 18790792Sgshapiroimmediately. The rest (after splitting) are queued up and queue 18890792Sgshapirorunners must come along and take care of them. Hence it is essential 18990792Sgshapirothat the queue interval is very short. 19090792Sgshapiro 19190792Sgshapiro 19290792Sgshapiro* 1-1 Mass Mailing 19390792Sgshapiro----------------------------------------------- 19490792Sgshapiro 19590792SgshapiroIn this case some program generates e-mails which are sent to 19690792Sgshapiroindividual recipients (or at most very few per e-mail). A simple 19790792Sgshapiroway to achieve high throughput is to set the delivery mode to 19890792Sgshapiro'interactive', turn off the SuperSafe option and make sure that the 19990792Sgshapiroprogram that generates the mails can deal with mail losses if the 20090792Sgshapiroserver loses power. In no other case should SuperSafe be set to 20190792Sgshapiro'false'. If these conditions are met, sendmail does not need to 20290792Sgshapirocommit mails to disk but can buffer them in memory which will greatly 20390792Sgshapiroenhance performance, especially compared to normal disk subsystems, e.g., 20490792Sgshapironon solid-state disks. 20590792Sgshapiro 20690792Sgshapiro 20790792Sgshapiro* High Volume Mail 20890792Sgshapiro----------------------------------------------- 20990792Sgshapiro 21090792SgshapiroFor high volume mail it is necessary to be able to control the load 21190792Sgshapiroon the system. Therefore the 'queue' delivery mode should be used, 21290792Sgshapiroand all options related to number of processes and the load should 21390792Sgshapirobe set to reasonable values. It is important not to accept mail 21490792Sgshapirofaster than it can be delivered otherwise the system will be 21590792Sgshapirooverwhelmed. Hence RefuseLA should be lower than QueueLA, the number 21690792Sgshapiroof daemon children should probably be lower than the number of queue 21790792Sgshapirorunnners (MaxChildren vs. MaxQueueChildren). DelayLA is a new option 21890792Sgshapiroin 8.12 which allows delaying connections instead of rejecting them. 21990792SgshapiroThis may result in a smoother load distribution depending on how 22090792Sgshapirothe mails are submitted to sendmail. 22190792Sgshapiro 22290792Sgshapiro 22390792Sgshapiro* Miscellaneous 22490792Sgshapiro----------------------------------------------- 22590792Sgshapiro 22690792SgshapiroOther options that are interesting to tweak performance are 22790792Sgshapiro(in no particular order): 22890792Sgshapiro 22990792SgshapiroSuperSafe: if interactive DeliveryMode is used, then this can 23090792Sgshapirobe set to the new value "interactive" in 8.12 to save some disk 23190792Sgshapirosynchronizations which are not really necessary in that mode. 23290792Sgshapiro 233