README.TEMPLATING revision 43240
1189251Ssam 2189251Ssam TEMPLATING machine configurations 3189251Ssam 4189251Ssam Matthew Dillon 5189251Ssam dillon@backplane.com 6189251Ssam 7189251Ssam This document describes a general mechanism by which you can template 8189251Ssam / and /usr. That is, to keep a 'master template' of / and /usr on a 9189251Ssam separate machine which is then used to update the rest of your machines. 10189251Ssam 11189251Ssam Generally speaking, you can't simply mirror /. You might be able to 12189251Ssam get away with mirroring /usr. There are two main problems involved with 13189251Ssam templating: 14189251Ssam 15189251Ssam (1) Avoiding overwriting run-time generated files 16189251Ssam 17189251Ssam By default, the system maintains a number of files in the root 18214734Srpaulo partition. For example, sendmail will dbm /etc/aliases into 19214734Srpaulo /etc/aliases.db. vipw or chpass or other password related routines 20189251Ssam will regenerate the password dbm's /etc/spwd.db, /etc/pwd.db, and 21189251Ssam passwd. /etc/namedb/s might contain generated secondaries. And 22189251Ssam so forth. 23189251Ssam 24189251Ssam The templating mechanism must avoid copying over such files. 25189251Ssam 26189251Ssam (2) Customizing machines. 27189251Ssam 28189251Ssam Customizing machines is actually considerably simpler. You create 29189251Ssam a configuration hierarchy and convert the configuration files that 30189251Ssam have to be customized into softlinks that run through a special 31189251Ssam softlink in the configuration directory. This will work for every 32189251Ssam configuration file except possibly /etc/master.passwd 33189251Ssam 34189251Ssam For example, /etc/resolv.conf would be turned into a softlink to 35189251Ssam /conf/ME/resolv.conf, and /conf/ME itself would be a softlink to 36189251Ssam /conf/<HOSTNAME>. The actual resolv.conf configuration file 37189251Ssam would reside in /conf/<HOSTNAME>. 38189251Ssam 39189251Ssam If you have a lot of hosts, some configuration files may be commonly 40189251Ssam classified. For example, all your shell machines might have the 41189251Ssam same /etc/resolv.conf. The solution is to make 42189251Ssam /conf/<HOSTNAME>/resolv.conf a softlink to a common directory, say 43189251Ssam /conf/HT.SHELL/resolv.conf. It may sound a little messy, but this 44189251Ssam sort of categorization actually makes the sysadmins job much, much 45189251Ssam easier. 46189251Ssam 47189251Ssam The /conf/ directory hierarchy is stored on the template and 48189251Ssam distributed to all the machines along with the rest of the root 49189251Ssam partition. 50189251Ssam 51189251Ssam This type of customization is taken from my direct experience 52189251Ssam instituting such a system at BEST. At the time, BEST had over 45 53189251Ssam machines managed from a single template. 54189251Ssam 55189251Ssam RUN-TIME GENERATED OR MODIFIED FILES IN / or /USR 56189251Ssam 57189251Ssam /etc/aliases.db 58189251Ssam /etc/master.passwd 59189251Ssam /etc/spwd.db 60189251Ssam /etc/pwd.db 61189251Ssam /etc/passwd 62189251Ssam /etc/namedb/s 63189251Ssam /root/.history 64189251Ssam /root/.ssh/identity 65189251Ssam /root/.ssh/identity.pub 66189251Ssam /root/.ssh/random_seed 67189251Ssam /root/.ssh/known_hosts 68189251Ssam /conf/ME 69189251Ssam /kernel* ( note 2 ) 70189251Ssam /dev ( note 3 ) 71189251Ssam /var ( note 4 ) 72189251Ssam /home ( note 4 ) 73189251Ssam /lost+found 74189251Ssam 75189251Ssam /usr/lost+found 76189251Ssam /usr/home ( note 4 ) 77189251Ssam /usr/crash ( note 5 ) 78189251Ssam /usr/obj ( note 5 ) 79189251Ssam /usr/ports ( note 5 ) 80189251Ssam /usr/src ( note 5 ) 81189251Ssam /usr/local/crack ( note 5 ) 82189251Ssam /usr/X11R6/lib/X11/xdm/xdm-errors ( note 6 ) 83189251Ssam /usr/X11R6/lib/X11/xdm/xdm-pid ( note 6 ) 84189251Ssam /usr/local/etc/ssh_host_key ( note 6 ) 85189251Ssam /usr/local/etc/ssh_host_key.pub ( note 6 ) 86189251Ssam /usr/local/etc/ssh_random_seed ( note 6 ) 87189251Ssam 88189251Ssam /conf/ME ( note 7 ) 89189251Ssam 90189251Ssam note 2: You typically want to update kernels manually and *NOT* 91189251Ssam template them as a safety measure. This also allows you to run 92189251Ssam different kernels on different machines or. 93189251Ssam 94189251Ssam note 3: /dev must be updated manually. Some devices, such as tty's and 95189251Ssam pty's, use the access and/or modify time and/or user/group 96189251Ssam operationally and regenerating the devices on the fly would be 97189251Ssam bad. 98189251Ssam 99189251Ssam note 4: /var and /home are usually separately mounted partitions and 100189251Ssam thus would not fall under the template, but as a safety measure 101189251Ssam the template copier refuse to copy directories named 'home'. 102189251Ssam 103189251Ssam note 5: These are directories that are as often created directly on 104189251Ssam /usr as they are separately-mounted partitions. You typically 105189251Ssam do not want to template such directories. 106189251Ssam 107189251Ssam note 6: Note that you can solve the problem of xdm and sshd creating 108189251Ssam files in /usr. With xdm, edit /usr/X11R6/lib/xdm/xdm-config 109189251Ssam and change the errorLogFile and pidFile config lines. 110189251Ssam 111189251Ssam With sshd, add 'HostKey' and 'RandomSeed' directives to specify 112189251Ssam /var/db for the location of the host key and run-time sshd 113189251Ssam random seed: 114189251Ssam 115189251Ssam HostKey /var/db/ssh_host_key 116189251Ssam RandomSeed /var/db/ssh_random_seed 117189251Ssam 118189251Ssam note 7: In this example, /conf/ME is the machine customizer and must 119189251Ssam be pointed to the /conf/<full-host-name>/ directory, which is 120189251Ssam different for each machine. Thus, the /conf/ME softlink 121189251Ssam should never be overwritten by the templating copy. 122189251Ssam 123189251Ssam 124189251Ssam TYPICAL CUSTOMIZED CONFIGRATION SOFTLINKS 125189251Ssam 126189251Ssam The following files typically need to be turned into softlinks 127189251Ssam to /conf/ME/<filename>: 128189251Ssam 129189251Ssam /etc/ccd.conf -> /conf/ME/ccd.conf 130189251Ssam /etc/ipfw.conf ... 131189251Ssam /etc/fstab 132189251Ssam /etc/motd 133189251Ssam /etc/resolv.conf 134189251Ssam /etc/aliases 135189251Ssam /etc/sendmail.cw 136189251Ssam /etc/organization 137189251Ssam /etc/named.conf 138189251Ssam /etc/rc.conf.local 139189251Ssam /etc/printcap 140189251Ssam /etc/inetd.conf 141189251Ssam /etc/login.conf 142189251Ssam /etc/gettytab 143189251Ssam /etc/ntp.conf 144189251Ssam /etc/exports 145189251Ssam /root/.k5login -> /conf/ME/root/.k5login 146189251Ssam 147189251Ssam And, of course, /conf/ME is usually a softlink to the appropriate 148189251Ssam /conf/<full-host-name>/. Depending on your system configuration, 149189251Ssam there may be other files not listed above that you have to worry about. 150189251Ssam 151189251Ssam DELETION OF FILES 152189251Ssam 153189251Ssam Any file found on the template destination that does not exist in the 154189251Ssam source and is not listed as an exception by the source should be deleted. 155189251Ssam However, deletion can be dangerous and cpdup will ask for confirmation 156189251Ssam by default. Once you know you aren't going to blow things up, you can 157189251Ssam turn this feature off and update your systems automatically from cron. 158189251Ssam 159189251Ssam By formalizing the delete operation, you can be 100% sure that it is 160189251Ssam possible to recreate / and /usr on any machine with only the original 161189251Ssam template and a backup of the ( relatively few ) explicitly-excepted 162189251Ssam files. The most common mistake a sysop makes is to make a change to a 163189251Ssam file in / or /usr on a target machine instead of the template machine. 164189251Ssam If the target machine is updated once a night from cron, the sysop 165189251Ssam quickly learns not to do this ( because his changes get overwritten 166189251Ssam overnight ). With a manual update, these sorts of mistakes can propogate 167189251Ssam for weeks or months before they are caught. 168189251Ssam 169189251Ssam TEMPLATE COPYING AND SAFETY 170189251Ssam THE CPDUP PROGRAM 171189251Ssam 172189251Ssam The 'cpdup' program is a program which efficiently duplicates a directory 173189251Ssam tree. The program copies source to destination, duplicating devices, 174189251Ssam softlinks, hardlinks, files, modification times, uid, gid, flags, perms, 175189251Ssam and so forth. The program incorporates several major features: 176189251Ssam 177189251Ssam * The program refuses, absolutely, to cross partition boundries. 178189251Ssam i.e. if you were copying the template /usr from an NFS mount to 179189251Ssam your /usr, and you had a mount point called /usr/home, the 180189251Ssam template copying program would *NOT* descend into /usr/home on 181189251Ssam the destination. 182189251Ssam 183189251Ssam This is a safety. 184189251Ssam 185189251Ssam * The program accesses a file called .cpignore in each directory 186189251Ssam it descending into on the source to obtain a list of exceptions 187189251Ssam for that directory -- that is, files not to copy or mess with. 188189251Ssam 189189251Ssam This is a templating function. 190189251Ssam 191189251Ssam * The program refuses to delete a directory on the destination 192189251Ssam being replaced by a softlink or file on the source. 193189251Ssam 194189251Ssam This is a safety mechanism 195189251Ssam 196189251Ssam * The program is capable of maintaing MD5 check cache files and 197189251Ssam doing an MD5 check between source and destination during the 198189251Ssam scan. 199189251Ssam 200189251Ssam * The program is capable of deleting files/directories on the 201189251Ssam destination that do not exist on the source, but asks for 202189251Ssam confirmation by default. 203189251Ssam 204189251Ssam This is a templating and a safety mechanism. 205189251Ssam 206189251Ssam * The program uses a copy-to-tmp-and-rename methodology allowing 207189251Ssam it to be used to update live filesystems. 208189251Ssam 209189251Ssam This is a templating mechanism. 210189251Ssam 211189251Ssam * The program, by default, tries to determine if a copy is required 212189251Ssam by checking modify times, file size, perms, and other stat 213189251Ssam elements. If the elements match, it does not bother to copy 214189251Ssam ( unless an MD5 check is being made, in which case it must read 215189251Ssam the destination file ). 216189251Ssam 217189251Ssam You typically run cpdup on the target machine. The target machine 218189251Ssam temporarily mounts the template machine's / and /usr via NFS, read-only, 219189251Ssam and runs cpdup to update / and /usr. If you use this methodology note 220189251Ssam that THERE ARE SECURITY CONSIDERATIONS! See 'SECURITY CONSIDERATIONS WITH 221189251Ssam NFS' below. 222189251Ssam 223189251Ssam Whatever script you use that does the NFS mounts should ensure that the 224189251Ssam mount succeeded before continuing with the cpdup. 225189251Ssam 226189251Ssam You should create .cpignore files in the appropriate directories on the 227189251Ssam template machine's / and /usr partitions so as not to overwrite active 228189251Ssam files on the target. The most critical .cpignore files should be 229189251Ssam protected with 'chflags schg .cpignore'. Specifically, the ones in / 230189251Ssam and /etc, but possibly others as well. For example, the .cpignore 231189251Ssam hierarchy for protect /root is: 232189251Ssam 233189251Ssam # /root/.cpignore contains 234189251Ssam .history 235189251Ssam 236189251Ssam # /root/.ssh/.cpignore contains 237189251Ssam random_seed 238189251Ssam known_hosts 239189251Ssam authorized_keys 240189251Ssam identity 241189251Ssam identity.pub 242189251Ssam 243189251Ssam WHEN INITIALLY CONVERTING A TARGET MACHINE TO USE TEMPLATING, ALWAYS 244189251Ssam MAKE A FULL BACKUP OF THE TARGET MACHINE FIRST! You may accidently delete 245189251Ssam files on the target during the conversion due to forgetting to enter 246189251Ssam items into appropriate .cpignore files on the source. 247189251Ssam 248189251Ssam SECURITY CONSIDERATIONS WITH NFS ROOT EXPORT FROM TEMPLATE MACHINE 249189251Ssam SECURITY CONSIDERATIONS WITH NFS USR EXPORT FROM TEMPLATE MACHINE 250189251Ssam 251189251Ssam There are some serious security considerations that must be taken into 252189251Ssam account when exporting / and /usr on the template machine. 253189251Ssam 254189251Ssam * only export read-only 255189251Ssam 256189251Ssam * the password file ( aka vipw ) may not contain any crypted passwords 257189251Ssam at all. You MUST use ssh or kerberos to access the template machine. 258189251Ssam 259189251Ssam You can get away with giving only root a crypted password, but only 260189251Ssam if you disallow network root logins and only allow direct root 261189251Ssam logins on the console. 262189251Ssam 263189251Ssam * The machine's private ssh_host_key usually resides in /usr/local/etc. 264189251Ssam You must move this key to /var/db. You can softlink link so no 265189251Ssam modification of sshd_config is required. 266189251Ssam 267189251Ssam * The machine's private ~root/.ssh/identity file is also exposed by 268189251Ssam the NFS export, you should move this file to /var/db as well and 269189251Ssam put a softlink in ~root/.ssh. 270189251Ssam 271189251Ssam * DON'T EXPORT /var ! Either that, or don't put the private keys 272189251Ssam in /var/db ... put them somewhere else. 273189251Ssam 274189251Ssam * You may want to redirect the location of the random_seed file, which 275189251Ssam can be done by editing ~root/.ssh/sshd_config and 276189251Ssam /usr/local/etc/sshd_config so it is not exposed either. 277189251Ssam 278189251Ssam -Matt 279189251Ssam Matthew Dillon 280189251Ssam dillon@backplane.com 281189251Ssam 282189251Ssam