README.BOOTP
1IMPORTANT NOTE:
2
3As of Feb. 11, 2002 (and indeed, for quite some time before that),
4the /etc/rc.diskless{1,2} scripts support a slightly different
5diskless boot process than the one documented in the rest of
6this file (which is 3 years old).
7
8I am not deleting the information below because it contains some
9useful background information on diskless operation, but for the
10actual details you should look at /etc/rc.diskless1, /etc/rc.diskless2,
11and the /usr/share/examples/diskless/clone_root script which can
12be useful to set up clients and server for diskless boot.
13
14--- $FreeBSD$ ---
15------------------------------------------------------------------------
16
17 BOOTP configuration mechanism
18
19 Matthew Dillon
20 dillon@backplane.com
21
22 BOOTP kernels automatically configure the machine's IP address, netmask,
23 optional NFS based swap, and NFS based root mount. The NFS server will
24 typically export a shared read-only /, /usr, and /var to any number of
25 workstations. The shared read-only root is typically either the server's
26 own root or, if you are more security conscious, a contrived root.
27
28 The key issue with starting up a BOOTP kernel is that you typically want
29 to export read-only NFS partitions from the server, yet still be able to
30 customize each workstation ( or not ).
31
32 /etc/rc.diskless1 is responsible for doing core mounts and for retargeting
33 /conf/ME ( part of the read-only root NFS mount ) to /conf/$IP_OF_CLIENT.
34 /etc/rc.conf.local and /etc/rc.local, along with other machine-specific
35 configuration files, are typically softlinks to /conf/ME/<filename>.
36
37 In the BOOTP workstation /conf/$IP/rc.conf.local, you must typically
38 turn *OFF* most of the system option defaults in /etc/rc.conf as well
39 as do additional custom configuration of your environment
40
41 The /usr/src/share/examples/diskless directory contains a typical
42 X session / sshd based workstation configuration. The directories
43 involved are HT.DISKLESS/ and 192.157.86.12/.
44
45 Essentially, the $IP/ directory ( which rc.diskless looks for in
46 /conf/$IP/ ) contains all the junk. The HT.DISKLESS directory exists
47 to hold common elements of your custom configuration so you do not have
48 to repeat those elements for each workstation. The example /conf
49 structure included here shows how to create a working sshd setup ( so
50 you can sshd into the diskless workstation ), retarget xdm's pid and error
51 files to R+W directories if /usr is mounted read-only, and retarget
52 syslogd and other programs. This example is not designed to run out of
53 the box and some modifications are required.
54
55 >> NOTE << HT.DISKLESS/ttys contains the typical configuration required
56 to bring X up at boot time. Essentially, it runs xdm in the foreground
57 with the appropriate arguments rather then a getty on ttyv0. You must
58 run xdm on ttyv0 in order to prevent xdm racing with getty on a virtual
59 terminal. Such a race can cause your keyboard to be directed away from
60 the X session, essentially making the session unusable.
61
62 Typically you should start with a clean slate by tar-copying this example
63 directory to /conf and then hack on it in /conf rather then in
64 /usr/share/examples/diskless.
65
66 BOOTP CLIENT SETUP
67
68 Here is a typical kernel configuration. If you have only one ethernet
69 interface you do not need to wire BOOTP to a specific interface name.
70 BOOTP requires NFS and NFS_ROOT, and our boot scripts require MFS. If
71 your /tmp is *not* a softlink to /var/tmp, the scripts also require NULLFS
72
73# BootP
74#
75options BOOTP # Use BOOTP to obtain IP address/hostname
76options BOOTP_NFSROOT # NFS mount root filesystem using BOOTP info
77options "BOOTP_NFSV3" # Use NFS v3 to NFS mount rootoptions
78options BOOTP_COMPAT # Workaround for broken bootp daemons.
79#options "BOOTP_WIRED_TO=de0"
80
81options MFS # Memory File System
82options NFS # Network Filesystem
83options NFS_ROOT # Nfs can be root
84options NULLFS # nullfs to map /var/tmp to /tmp
85
86 BOOTP SERVER SETUP
87
88 The BOOTP server must be running on the same logical LAN as the the
89 BOOTP client(s). You need to setup two things:
90
91 (1) You need to NFS-export /, /usr, and /var.
92
93 (2) You need to run a BOOTP server. DHCPD can do this.
94
95
96 NFS Export:
97
98 Here is an example "/etc/exports" file.
99
100/ -ro -maproot=root: -network 192.157.86.0 -mask 255.255.255.192
101/usr -ro -maproot=root: -network 192.157.86.0 -mask 255.255.255.192
102/var -ro -maproot=root: -network 192.157.86.0 -mask 255.255.255.192
103
104 In order to be an NFS server, the server must run portmap, mountd,
105 nfsd, and rpc.statd. The standard NFS server options in /etc/rc.conf
106 will work ( you should put your overrides in /etc/rc.conf.local on the
107 server and not edit the distribution /etc/rc.conf, though ).
108
109 BOOTP Server:
110
111 This configuration file "/etc/dhcpd.conf" example is for
112 the '/usr/ports/net/isc-dhcp' dhcpd port.
113
114 subnet 192.157.86.0 netmask 255.255.255.192 {
115 # range if you want to run the core dhcpd service of
116 # dynamic IP assignment, but it is not used with BOOTP
117 # workstations
118 range 192.157.86.32 192.157.86.62;
119
120 # misc configuration.
121 #
122 option routers 192.157.86.2;
123 option domain-name-servers 192.157.86.2;
124
125 server-name "apollo.fubar.com";
126 option subnet-mask 255.255.255.192;
127 option domain-name-servers 192.157.86.2;
128 option domain-name "fubar.com";
129 option broadcast-address 192.157.86.63;
130 option routers 192.157.86.2;
131 }
132
133 host test1 {
134 hardware ethernet 00:a0:c9:d3:38:25;
135 fixed-address 192.157.86.11;
136 option root-path "192.157.86.2:/";
137 option option-128 "192.157.86.2:/images/swap";
138 }
139
140 host test2 {
141 # hardware ethernet 00:e0:29:1d:16:09;
142 hardware ethernet 00:10:5a:a8:94:0e;
143 fixed-address 192.157.86.12;
144 option root-path "192.157.86.2:/";
145 option option-128 "192.157.86.2:/images/swap";
146 }
147
148 SWAP. This example includes options to automatically BOOTP configure
149 NFS swap on each workstation. In order to use this capabilities you
150 need to NFS-export a swap directory READ+WRITE to the workstations.
151
152 You must then create a swap directory for each workstation you wish to
153 assign swap to. In this example I created a dummy user 'lander' and
154 did an NFS export of /images/swap enforcing a UID of 'lander' for
155 all accesses.
156
157 apollo:/usr/ports/net# ls -la /images/swap
158 total 491786
159 drwxr-xr-x 2 root wheel 512 Dec 28 07:00 .
160 drwxr-xr-x 8 root wheel 512 Jan 20 10:54 ..
161 -rw-r--r-- 1 lander wheel 33554432 Dec 23 14:35 swap.192.157.86.11
162 -rw-r--r-- 1 lander wheel 335544320 Jan 24 16:55 swap.192.157.86.12
163 -rw-r--r-- 1 lander wheel 134217728 Jan 21 17:19 swap.192.157.86.6
164
165 A swap file is best created with dd:
166
167 # create a 32MB swap file for a BOOTP workstation
168 dd if=/dev/zero of=swap.IPADDRESS bs=1m count=32
169
170 It is generally a good idea to give your workstations some swap space,
171 but not a requirement if they have a lot of memory.
172
173
README.TEMPLATING
1IMPORTANT NOTE:
2
3As of Feb. 11, 2002 (and indeed, for quite some time before that),
4the /etc/rc.diskless{1,2} scripts support a slightly different
5diskless boot process than the one documented in the rest of
6this file (which is 3 years old).
7
8I am not deleting the information below because it contains some
9useful background information on diskless operation, but for the
10actual details you should look at /etc/rc.diskless1, /etc/rc.diskless2,
11and the /usr/share/examples/diskless/clone_root script which can
12be useful to set up clients and server for diskless boot.
13
14--- $FreeBSD$ ---
15------------------------------------------------------------------------
16
17 TEMPLATING machine configurations
18
19 Matthew Dillon
20 dillon@backplane.com
21
22 This document describes a general mechanism by which you can template
23 / and /usr. That is, to keep a 'master template' of / and /usr on a
24 separate machine which is then used to update the rest of your machines.
25
26 Generally speaking, you can't simply mirror /. You might be able to
27 get away with mirroring /usr. There are two main problems involved with
28 templating:
29
30 (1) Avoiding overwriting run-time generated files
31
32 By default, the system maintains a number of files in the root
33 partition. For example, sendmail will dbm /etc/aliases into
34 /etc/aliases.db. vipw or chpass or other password related routines
35 will regenerate the password dbm's /etc/spwd.db, /etc/pwd.db, and
36 passwd. /etc/namedb/s might contain generated secondaries. And
37 so forth.
38
39 The templating mechanism must avoid copying over such files.
40
41 (2) Customizing machines.
42
43 Customizing machines is actually considerably simpler. You create
44 a configuration hierarchy and convert the configuration files that
45 have to be customized into softlinks that run through a special
46 softlink in the configuration directory. This will work for every
47 configuration file except possibly /etc/master.passwd
48
49 For example, /etc/resolv.conf would be turned into a softlink to
50 /conf/ME/resolv.conf, and /conf/ME itself would be a softlink to
51 /conf/<HOSTNAME>. The actual resolv.conf configuration file
52 would reside in /conf/<HOSTNAME>.
53
54 If you have a lot of hosts, some configuration files may be commonly
55 classified. For example, all your shell machines might have the
56 same /etc/resolv.conf. The solution is to make
57 /conf/<HOSTNAME>/resolv.conf a softlink to a common directory, say
58 /conf/HT.SHELL/resolv.conf. It may sound a little messy, but this
59 sort of categorization actually makes the sysadmins job much, much
60 easier.
61
62 The /conf/ directory hierarchy is stored on the template and
63 distributed to all the machines along with the rest of the root
64 partition.
65
66 This type of customization is taken from my direct experience
67 instituting such a system at BEST. At the time, BEST had over 45
68 machines managed from a single template.
69
70 RUN-TIME GENERATED OR MODIFIED FILES IN / or /USR
71
72 /etc/aliases.db
73 /etc/master.passwd
74 /etc/spwd.db
75 /etc/pwd.db
76 /etc/passwd
77 /etc/namedb/s
78 /root/.history
79 /root/.ssh/identity
80 /root/.ssh/identity.pub
81 /root/.ssh/random_seed
82 /root/.ssh/known_hosts
83 /conf/ME
84 /kernel* ( note 2 )
85 /dev ( note 3 )
86 /var ( note 4 )
87 /home ( note 4 )
88 /lost+found
89
90 /usr/lost+found
91 /usr/home ( note 4 )
92 /usr/crash ( note 5 )
93 /usr/obj ( note 5 )
94 /usr/ports ( note 5 )
95 /usr/src ( note 5 )
96 /usr/local/crack ( note 5 )
97 /usr/local/lib/X11/xdm/xdm-errors ( note 6 )
98 /usr/local/lib/X11/xdm/xdm-pid ( note 6 )
99 /usr/local/etc/ssh_host_key ( note 6 )
100 /usr/local/etc/ssh_host_key.pub ( note 6 )
101 /usr/local/etc/ssh_random_seed ( note 6 )
102
103 /conf/ME ( note 7 )
104
105 note 2: You typically want to update kernels manually and *NOT*
106 template them as a safety measure. This also allows you to run
107 different kernels on different machines or.
108
109 note 3: /dev must be updated manually. Some devices, such as tty's and
110 pty's, use the access and/or modify time and/or user/group
111 operationally and regenerating the devices on the fly would be
112 bad.
113
114 note 4: /var and /home are usually separately mounted partitions and
115 thus would not fall under the template, but as a safety measure
116 the template copier refuse to copy directories named 'home'.
117
118 note 5: These are directories that are as often created directly on
119 /usr as they are separately-mounted partitions. You typically
120 do not want to template such directories.
121
122 note 6: Note that you can solve the problem of xdm and sshd creating
123 files in /usr. With xdm, edit /usr/local/lib/xdm/xdm-config
124 and change the errorLogFile and pidFile config lines.
125
126 With sshd, add 'HostKey' and 'RandomSeed' directives to specify
127 /var/db for the location of the host key and run-time sshd
128 random seed:
129
130 HostKey /var/db/ssh_host_key
131 RandomSeed /var/db/ssh_random_seed
132
133 note 7: In this example, /conf/ME is the machine customizer and must
134 be pointed to the /conf/<full-host-name>/ directory, which is
135 different for each machine. Thus, the /conf/ME softlink
136 should never be overwritten by the templating copy.
137
138
139 TYPICAL CUSTOMIZED CONFIGURATION SOFTLINKS
140
141 The following files typically need to be turned into softlinks
142 to /conf/ME/<filename>:
143
144 /etc/ccd.conf -> /conf/ME/ccd.conf
145 /etc/ipfw.conf ...
146 /etc/fstab
147 /etc/motd
148 /etc/resolv.conf
149 /etc/aliases
150 /etc/sendmail.cw
151 /etc/organization
152 /etc/named.conf
153 /etc/rc.conf.local
154 /etc/printcap
155 /etc/inetd.conf
156 /etc/login.conf
157 /etc/gettytab
158 /etc/ntp.conf
159 /etc/exports
160 /root/.k5login -> /conf/ME/root/.k5login
161
162 And, of course, /conf/ME is usually a softlink to the appropriate
163 /conf/<full-host-name>/. Depending on your system configuration,
164 there may be other files not listed above that you have to worry about.
165
166 In many cases, /conf/ME/filename is itself a softlink to
167 "../HT.xxxx/filename", where HT.xxxx is something like HT.STD ... this
168 added complexity actually makes it easier to manage multiple
169 classifications of machines.
170
171 DELETION OF FILES
172
173 Any file found on the template destination that does not exist in the
174 source and is not listed as an exception by the source should be deleted.
175 However, deletion can be dangerous and cpdup will ask for confirmation
176 by default. Once you know you aren't going to blow things up, you can
177 turn this feature off and update your systems automatically from cron.
178
179 By formalizing the delete operation, you can be 100% sure that it is
180 possible to recreate / and /usr on any machine with only the original
181 template and a backup of the ( relatively few ) explicitly-excepted
182 files. The most common mistake a sysop makes is to make a change to a
183 file in / or /usr on a target machine instead of the template machine.
184 If the target machine is updated once a night from cron, the sysop
185 quickly learns not to do this ( because his changes get overwritten
186 overnight ). With a manual update, these sorts of mistakes can propagate
187 for weeks or months before they are caught.
188
189 TEMPLATE COPYING AND SAFETY
190 THE CPDUP PROGRAM
191
192 The 'cpdup' program is a program which efficiently duplicates a directory
193 tree. The program copies source to destination, duplicating devices,
194 softlinks, hardlinks, files, modification times, uid, gid, flags, perms,
195 and so forth. The program incorporates several major features:
196
197 * The program refuses, absolutely, to cross partition boundaries.
198 i.e. if you were copying the template /usr from an NFS mount to
199 your /usr, and you had a mount point called /usr/home, the
200 template copying program would *NOT* descend into /usr/home on
201 the destination.
202
203 This is a safety.
204
205 * The program accesses a file called .cpignore in each directory
206 it descends into on the source to obtain a list of exceptions
207 for that directory -- that is, files not to copy or mess with.
208
209 This is a templating function.
210
211 * The program refuses to delete a directory on the destination
212 being replaced by a softlink or file on the source.
213
214 This is a safety mechanism
215
216 * The program is capable of maintaining MD5 check cache files and
217 doing an MD5 check between source and destination during the
218 scan.
219
220 * The program is capable of deleting files/directories on the
221 destination that do not exist on the source, but asks for
222 confirmation by default.
223
224 This is a templating and a safety mechanism.
225
226 * The program uses a copy-to-tmp-and-rename methodology allowing
227 it to be used to update live filesystems.
228
229 This is a templating mechanism.
230
231 * The program, by default, tries to determine if a copy is required
232 by checking modify times, file size, perms, and other stat
233 elements. If the elements match, it does not bother to copy
234 ( unless an MD5 check is being made, in which case it must read
235 the destination file ).
236
237 You typically run cpdup on the target machine. The target machine
238 temporarily mounts the template machine's / and /usr via NFS, read-only,
239 and runs cpdup to update / and /usr. If you use this methodology note
240 that THERE ARE SECURITY CONSIDERATIONS! See 'SECURITY CONSIDERATIONS WITH
241 NFS' below.
242
243 Whatever script you use that does the NFS mounts should ensure that the
244 mount succeeded before continuing with the cpdup.
245
246 You should create .cpignore files in the appropriate directories on the
247 template machine's / and /usr partitions so as not to overwrite active
248 files on the target. The most critical .cpignore files should be
249 protected with 'chflags schg .cpignore'. Specifically, the ones in /
250 and /etc, but possibly others as well. For example, the .cpignore
251 hierarchy for protect /root is:
252
253 # /root/.cpignore contains
254 .history
255
256 # /root/.ssh/.cpignore contains
257 random_seed
258 known_hosts
259 authorized_keys
260 identity
261 identity.pub
262
263 WHEN INITIALLY CONVERTING A TARGET MACHINE TO USE TEMPLATING, ALWAYS
264 MAKE A FULL BACKUP OF THE TARGET MACHINE FIRST! You may accidentally
265 delete files on the target during the conversion due to forgetting to
266 enter items into appropriate .cpignore files on the source.
267
268 SECURITY CONSIDERATIONS WITH NFS ROOT EXPORT FROM TEMPLATE MACHINE
269 SECURITY CONSIDERATIONS WITH NFS USR EXPORT FROM TEMPLATE MACHINE
270
271 There are some serious security considerations that must be taken into
272 account when exporting / and /usr on the template machine.
273
274 * only export read-only
275
276 * the password file ( aka vipw ) may not contain any crypted passwords
277 at all. You MUST use ssh or kerberos to access the template machine.
278
279 You can get away with giving only root a crypted password, but only
280 if you disallow network root logins and only allow direct root
281 logins on the console.
282
283 * The machine's private ssh_host_key usually resides in /usr/local/etc.
284 You must move this key to /var/db. You can softlink link so no
285 modification of sshd_config is required.
286
287 * The machine's private ~root/.ssh/identity file is also exposed by
288 the NFS export, you should move this file to /var/db as well and
289 put a softlink in ~root/.ssh.
290
291 * DON'T EXPORT /var ! Either that, or don't put the private keys
292 in /var/db ... put them somewhere else.
293
294 * You may want to redirect the location of the random_seed file, which
295 can be done by editing ~root/.ssh/sshd_config and
296 /usr/local/etc/sshd_config so it is not exposed either.
297
298 -Matt
299 Matthew Dillon
300 dillon@backplane.com
301
302