1.\" Copyright (c) 2010 The FreeBSD Foundation
2.\" Copyright (c) 2010-2011 Pawel Jakub Dawidek <pawel@dawidek.net>
3.\" All rights reserved.
4.\"
5.\" This software was developed by Pawel Jakub Dawidek under sponsorship from
6.\" the FreeBSD Foundation.
7.\"
8.\" Redistribution and use in source and binary forms, with or without
9.\" modification, are permitted provided that the following conditions
10.\" are met:
11.\" 1. Redistributions of source code must retain the above copyright
12.\" notice, this list of conditions and the following disclaimer.
13.\" 2. Redistributions in binary form must reproduce the above copyright
14.\" notice, this list of conditions and the following disclaimer in the
15.\" documentation and/or other materials provided with the distribution.
16.\"
17.\" THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND
18.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
19.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
20.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE
21.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
22.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
23.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
24.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
25.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
26.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
27.\" SUCH DAMAGE.
28.\"
29.\" $FreeBSD: head/sbin/hastd/hast.conf.5 222121 2011-05-20 11:21:39Z pjd $
30.\"
31.Dd May 20, 2011
32.Dt HAST.CONF 5
33.Os
34.Sh NAME
35.Nm hast.conf
36.Nd configuration file for the
37.Xr hastd 8
38daemon and the
39.Xr hastctl 8
40utility.
41.Sh DESCRIPTION
42The
43.Nm
44file is used by both
45.Xr hastd 8
46daemon
47and
48.Xr hastctl 8
49control utility.
50Configuration file is designed in a way that exactly the same file can be
51(and should be) used on both HAST nodes.
52Every line starting with # is treated as comment and ignored.
53.Sh CONFIGURATION FILE SYNTAX
54General syntax of the
55.Nm
56file is following:
57.Bd -literal -offset indent
58# Global section
59control <addr>
60listen <addr>
61replication <mode>
62checksum <algorithm>
63compression <algorithm>
64timeout <seconds>
65exec <path>
66
67on <node> {
68 # Node section
69 control <addr>
70 listen <addr>
71}
72
73on <node> {
74 # Node section
75 control <addr>
76 listen <addr>
77}
78
79resource <name> {
80 # Resource section
81 replication <mode>
82 checksum <algorithm>
83 compression <algorithm>
84 name <name>
85 local <path>
86 timeout <seconds>
87 exec <path>
88
89 on <node> {
90 # Resource-node section
91 name <name>
92 # Required
93 local <path>
94 # Required
95 remote <addr>
96 source <addr>
97 }
98 on <node> {
99 # Resource-node section
100 name <name>
101 # Required
102 local <path>
103 # Required
104 remote <addr>
105 source <addr>
106 }
107}
108.Ed
109.Pp
110Most of the various available configuration parameters are optional.
111If parameter is not defined in the particular section, it will be
112inherited from the parent section.
113For example, if the
114.Ic listen
115parameter is not defined in the node section, it will be inherited from
116the global section.
117In case the global section does not define the
118.Ic listen
119parameter at all, the default value will be used.
120.Sh CONFIGURATION FILE DESCRIPTION
121The
122.Aq node
123argument can be replaced either by a full hostname as obtained by
124.Xr gethostname 3 ,
125only first part of the hostname, or by node's UUID as found in the
126.Va kern.hostuuid
127.Xr sysctl 8
128variable.
129.Pp
130The following statements are available:
131.Bl -tag -width ".Ic xxxx"
132.It Ic control Aq addr
133.Pp
134Address for communication with
135.Xr hastctl 8 .
136Each of the following examples defines the same control address:
137.Bd -literal -offset indent
138uds:///var/run/hastctl
139unix:///var/run/hastctl
140/var/run/hastctl
141.Ed
142.Pp
143The default value is
144.Pa uds:///var/run/hastctl .
145.It Ic listen Aq addr
146.Pp
147Address to listen on in form of:
148.Bd -literal -offset indent
149protocol://protocol-specific-address
150.Ed
151.Pp
152Each of the following examples defines the same listen address:
153.Bd -literal -offset indent
1540.0.0.0
1550.0.0.0:8457
156tcp://0.0.0.0
157tcp://0.0.0.0:8457
158tcp4://0.0.0.0
159tcp4://0.0.0.0:8457
160.Ed
161.Pp
162Multiple listen addresses can be specified.
163By default
164.Nm hastd
165listens on
166.Pa tcp4://0.0.0.0:8457
167and
168.Pa tcp6://[::]:8457
169if kernel supports IPv4 and IPv6 respectively.
170.It Ic replication Aq mode
171.Pp
172Replication mode should be one of the following:
173.Bl -tag -width ".Ic xxxx"
174.It Ic memsync
175.Pp
176Report the write operation as completed when local write completes and
177when the remote node acknowledges the data receipt, but before it
178actually stores the data.
179The data on remote node will be stored directly after sending
180acknowledgement.
181This mode is intended to reduce latency, but still provides a very good
182reliability.
183The only situation where some small amount of data could be lost is when
184the data is stored on primary node and sent to the secondary.
185Secondary node then acknowledges data receipt and primary reports
186success to an application.
187However, it may happen that the secondary goes down before the received
188data is really stored locally.
189Before secondary node returns, primary node dies entirely.
190When the secondary node comes back to life it becomes the new primary.
191Unfortunately some small amount of data which was confirmed to be stored
192to the application was lost.
193The risk of such a situation is very small.
194The
195.Ic memsync
196replication mode is currently not implemented.
197.It Ic fullsync
198.Pp
199Mark the write operation as completed when local as well as remote
200write completes.
201This is the safest and the slowest replication mode.
202The
203.Ic fullsync
204replication mode is the default.
205.It Ic async
206.Pp
207The write operation is reported as complete right after the local write
208completes.
209This is the fastest and the most dangerous replication mode.
210This mode should be used when replicating to a distant node where
211latency is too high for other modes.
212The
213.Ic async
214replication mode is currently not implemented.
215.El
216.It Ic checksum Aq algorithm
217.Pp
218Checksum algorithm should be one of the following:
219.Bl -tag -width ".Ic sha256"
220.It Ic none
221No checksum will be calculated for the data being send over the network.
222This is the default setting.
223.It Ic crc32
224CRC32 checksum will be calculated.
225.It Ic sha256
226SHA256 checksum will be calculated.
227.El
228.It Ic compression Aq algorithm
229.Pp
230Compression algorithm should be one of the following:
231.Bl -tag -width ".Ic none"
232.It Ic none
233Data send over the network will not be compressed.
234.It Ic hole
235Only blocks that contain all zeros will be compressed.
236This is very useful for initial synchronization where potentially many blocks
237are still all zeros.
238There should be no measurable performance overhead when this algorithm is being
239used.
240This is the default setting.
241.It Ic lzf
242The LZF algorithm by Marc Alexander Lehmann will be used to compress the data
243send over the network.
244LZF is very fast, general purpose compression algorithm.
245.El
246.It Ic timeout Aq seconds
247.Pp
248Connection timeout in seconds.
249The default value is
250.Va 20 .
251.It Ic exec Aq path
252.Pp
253Execute the given program on various HAST events.
254Below is the list of currently implemented events and arguments the given
255program is executed with:
256.Bl -tag -width ".Ic xxxx"
257.It Ic "<path> role <resource> <oldrole> <newrole>"
258.Pp
259Executed on both primary and secondary nodes when resource role is changed.
260.Pp
261.It Ic "<path> connect <resource>"
262.Pp
263Executed on both primary and secondary nodes when connection for the given
264resource between the nodes is established.
265.Pp
266.It Ic "<path> disconnect <resource>"
267.Pp
268Executed on both primary and secondary nodes when connection for the given
269resource between the nodes is lost.
270.Pp
271.It Ic "<path> syncstart <resource>"
272.Pp
273Executed on primary node when synchronization process of secondary node is
274started.
275.Pp
276.It Ic "<path> syncdone <resource>"
277.Pp
278Executed on primary node when synchronization process of secondary node is
279completed successfully.
280.Pp
281.It Ic "<path> syncintr <resource>"
282.Pp
283Executed on primary node when synchronization process of secondary node is
284interrupted, most likely due to secondary node outage or connection failure
285between the nodes.
286.Pp
287.It Ic "<path> split-brain <resource>"
288.Pp
289Executed on both primary and secondary nodes when split-brain condition is
290detected.
291.Pp
292.El
293The
294.Aq path
295argument should contain full path to executable program.
296If the given program exits with code different than
297.Va 0 ,
298.Nm hastd
299will log it as an error.
300.Pp
301The
302.Aq resource
303argument is resource name from the configuration file.
304.Pp
305The
306.Aq oldrole
307argument is previous resource role (before the change).
308It can be one of:
309.Ar init ,
310.Ar secondary ,
311.Ar primary .
312.Pp
313The
314.Aq newrole
315argument is current resource role (after the change).
316It can be one of:
317.Ar init ,
318.Ar secondary ,
319.Ar primary .
320.Pp
321.It Ic name Aq name
322.Pp
323GEOM provider name that will appear as
324.Pa /dev/hast/<name> .
325If name is not defined, resource name will be used as provider name.
326.It Ic local Aq path
327.Pp
328Path to the local component which will be used as backend provider for
329the resource.
330This can be either GEOM provider or regular file.
331.It Ic remote Aq addr
332.Pp
333Address of the remote
334.Nm hastd
335daemon.
336Format is the same as for the
337.Ic listen
338statement.
339When operating as a primary node this address will be used to connect to
340the secondary node.
341When operating as a secondary node only connections from this address
342will be accepted.
343.Pp
344A special value of
345.Va none
346can be used when the remote address is not yet known (eg. the other node is not
347set up yet).
348.It Ic source Aq addr
349.Pp
350Local address to bind to before connecting to the remote
351.Nm hastd
352daemon.
353Format is the same as for the
354.Ic listen
355statement.
356.El
357.Sh FILES
358.Bl -tag -width ".Pa /var/run/hastctl" -compact
359.It Pa /etc/hast.conf
360The default
361.Nm
362configuration file.
363.It Pa /var/run/hastctl
364Control socket used by the
365.Xr hastctl 8
366control utility to communicate with the
367.Xr hastd 8
368daemon.
369.El
370.Sh EXAMPLES
371The example configuration file can look as follows:
372.Bd -literal -offset indent
373listen tcp://0.0.0.0
374
375on hasta {
376 listen tcp://2001:db8::1/64
377}
378on hastb {
379 listen tcp://2001:db8::2/64
380}
381
382resource shared {
383 local /dev/da0
384
385 on hasta {
386 remote tcp://10.0.0.2
387 }
388 on hastb {
389 remote tcp://10.0.0.1
390 }
391}
392resource tank {
393 on hasta {
394 local /dev/mirror/tanka
395 source tcp://10.0.0.1
396 remote tcp://10.0.0.2
397 }
398 on hastb {
399 local /dev/mirror/tankb
400 source tcp://10.0.0.2
401 remote tcp://10.0.0.1
402 }
403}
404.Ed
405.Sh SEE ALSO
406.Xr gethostname 3 ,
407.Xr geom 4 ,
408.Xr hastctl 8 ,
409.Xr hastd 8 .
410.Sh AUTHORS
411The
412.Nm
413was written by
414.An Pawel Jakub Dawidek Aq pjd@FreeBSD.org
415under sponsorship of the FreeBSD Foundation.
2.\" Copyright (c) 2010-2011 Pawel Jakub Dawidek <pawel@dawidek.net>
3.\" All rights reserved.
4.\"
5.\" This software was developed by Pawel Jakub Dawidek under sponsorship from
6.\" the FreeBSD Foundation.
7.\"
8.\" Redistribution and use in source and binary forms, with or without
9.\" modification, are permitted provided that the following conditions
10.\" are met:
11.\" 1. Redistributions of source code must retain the above copyright
12.\" notice, this list of conditions and the following disclaimer.
13.\" 2. Redistributions in binary form must reproduce the above copyright
14.\" notice, this list of conditions and the following disclaimer in the
15.\" documentation and/or other materials provided with the distribution.
16.\"
17.\" THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND
18.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
19.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
20.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE
21.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
22.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
23.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
24.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
25.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
26.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
27.\" SUCH DAMAGE.
28.\"
29.\" $FreeBSD: head/sbin/hastd/hast.conf.5 222121 2011-05-20 11:21:39Z pjd $
30.\"
31.Dd May 20, 2011
32.Dt HAST.CONF 5
33.Os
34.Sh NAME
35.Nm hast.conf
36.Nd configuration file for the
37.Xr hastd 8
38daemon and the
39.Xr hastctl 8
40utility.
41.Sh DESCRIPTION
42The
43.Nm
44file is used by both
45.Xr hastd 8
46daemon
47and
48.Xr hastctl 8
49control utility.
50Configuration file is designed in a way that exactly the same file can be
51(and should be) used on both HAST nodes.
52Every line starting with # is treated as comment and ignored.
53.Sh CONFIGURATION FILE SYNTAX
54General syntax of the
55.Nm
56file is following:
57.Bd -literal -offset indent
58# Global section
59control <addr>
60listen <addr>
61replication <mode>
62checksum <algorithm>
63compression <algorithm>
64timeout <seconds>
65exec <path>
66
67on <node> {
68 # Node section
69 control <addr>
70 listen <addr>
71}
72
73on <node> {
74 # Node section
75 control <addr>
76 listen <addr>
77}
78
79resource <name> {
80 # Resource section
81 replication <mode>
82 checksum <algorithm>
83 compression <algorithm>
84 name <name>
85 local <path>
86 timeout <seconds>
87 exec <path>
88
89 on <node> {
90 # Resource-node section
91 name <name>
92 # Required
93 local <path>
94 # Required
95 remote <addr>
96 source <addr>
97 }
98 on <node> {
99 # Resource-node section
100 name <name>
101 # Required
102 local <path>
103 # Required
104 remote <addr>
105 source <addr>
106 }
107}
108.Ed
109.Pp
110Most of the various available configuration parameters are optional.
111If parameter is not defined in the particular section, it will be
112inherited from the parent section.
113For example, if the
114.Ic listen
115parameter is not defined in the node section, it will be inherited from
116the global section.
117In case the global section does not define the
118.Ic listen
119parameter at all, the default value will be used.
120.Sh CONFIGURATION FILE DESCRIPTION
121The
122.Aq node
123argument can be replaced either by a full hostname as obtained by
124.Xr gethostname 3 ,
125only first part of the hostname, or by node's UUID as found in the
126.Va kern.hostuuid
127.Xr sysctl 8
128variable.
129.Pp
130The following statements are available:
131.Bl -tag -width ".Ic xxxx"
132.It Ic control Aq addr
133.Pp
134Address for communication with
135.Xr hastctl 8 .
136Each of the following examples defines the same control address:
137.Bd -literal -offset indent
138uds:///var/run/hastctl
139unix:///var/run/hastctl
140/var/run/hastctl
141.Ed
142.Pp
143The default value is
144.Pa uds:///var/run/hastctl .
145.It Ic listen Aq addr
146.Pp
147Address to listen on in form of:
148.Bd -literal -offset indent
149protocol://protocol-specific-address
150.Ed
151.Pp
152Each of the following examples defines the same listen address:
153.Bd -literal -offset indent
1540.0.0.0
1550.0.0.0:8457
156tcp://0.0.0.0
157tcp://0.0.0.0:8457
158tcp4://0.0.0.0
159tcp4://0.0.0.0:8457
160.Ed
161.Pp
162Multiple listen addresses can be specified.
163By default
164.Nm hastd
165listens on
166.Pa tcp4://0.0.0.0:8457
167and
168.Pa tcp6://[::]:8457
169if kernel supports IPv4 and IPv6 respectively.
170.It Ic replication Aq mode
171.Pp
172Replication mode should be one of the following:
173.Bl -tag -width ".Ic xxxx"
174.It Ic memsync
175.Pp
176Report the write operation as completed when local write completes and
177when the remote node acknowledges the data receipt, but before it
178actually stores the data.
179The data on remote node will be stored directly after sending
180acknowledgement.
181This mode is intended to reduce latency, but still provides a very good
182reliability.
183The only situation where some small amount of data could be lost is when
184the data is stored on primary node and sent to the secondary.
185Secondary node then acknowledges data receipt and primary reports
186success to an application.
187However, it may happen that the secondary goes down before the received
188data is really stored locally.
189Before secondary node returns, primary node dies entirely.
190When the secondary node comes back to life it becomes the new primary.
191Unfortunately some small amount of data which was confirmed to be stored
192to the application was lost.
193The risk of such a situation is very small.
194The
195.Ic memsync
196replication mode is currently not implemented.
197.It Ic fullsync
198.Pp
199Mark the write operation as completed when local as well as remote
200write completes.
201This is the safest and the slowest replication mode.
202The
203.Ic fullsync
204replication mode is the default.
205.It Ic async
206.Pp
207The write operation is reported as complete right after the local write
208completes.
209This is the fastest and the most dangerous replication mode.
210This mode should be used when replicating to a distant node where
211latency is too high for other modes.
212The
213.Ic async
214replication mode is currently not implemented.
215.El
216.It Ic checksum Aq algorithm
217.Pp
218Checksum algorithm should be one of the following:
219.Bl -tag -width ".Ic sha256"
220.It Ic none
221No checksum will be calculated for the data being send over the network.
222This is the default setting.
223.It Ic crc32
224CRC32 checksum will be calculated.
225.It Ic sha256
226SHA256 checksum will be calculated.
227.El
228.It Ic compression Aq algorithm
229.Pp
230Compression algorithm should be one of the following:
231.Bl -tag -width ".Ic none"
232.It Ic none
233Data send over the network will not be compressed.
234.It Ic hole
235Only blocks that contain all zeros will be compressed.
236This is very useful for initial synchronization where potentially many blocks
237are still all zeros.
238There should be no measurable performance overhead when this algorithm is being
239used.
240This is the default setting.
241.It Ic lzf
242The LZF algorithm by Marc Alexander Lehmann will be used to compress the data
243send over the network.
244LZF is very fast, general purpose compression algorithm.
245.El
246.It Ic timeout Aq seconds
247.Pp
248Connection timeout in seconds.
249The default value is
250.Va 20 .
251.It Ic exec Aq path
252.Pp
253Execute the given program on various HAST events.
254Below is the list of currently implemented events and arguments the given
255program is executed with:
256.Bl -tag -width ".Ic xxxx"
257.It Ic "<path> role <resource> <oldrole> <newrole>"
258.Pp
259Executed on both primary and secondary nodes when resource role is changed.
260.Pp
261.It Ic "<path> connect <resource>"
262.Pp
263Executed on both primary and secondary nodes when connection for the given
264resource between the nodes is established.
265.Pp
266.It Ic "<path> disconnect <resource>"
267.Pp
268Executed on both primary and secondary nodes when connection for the given
269resource between the nodes is lost.
270.Pp
271.It Ic "<path> syncstart <resource>"
272.Pp
273Executed on primary node when synchronization process of secondary node is
274started.
275.Pp
276.It Ic "<path> syncdone <resource>"
277.Pp
278Executed on primary node when synchronization process of secondary node is
279completed successfully.
280.Pp
281.It Ic "<path> syncintr <resource>"
282.Pp
283Executed on primary node when synchronization process of secondary node is
284interrupted, most likely due to secondary node outage or connection failure
285between the nodes.
286.Pp
287.It Ic "<path> split-brain <resource>"
288.Pp
289Executed on both primary and secondary nodes when split-brain condition is
290detected.
291.Pp
292.El
293The
294.Aq path
295argument should contain full path to executable program.
296If the given program exits with code different than
297.Va 0 ,
298.Nm hastd
299will log it as an error.
300.Pp
301The
302.Aq resource
303argument is resource name from the configuration file.
304.Pp
305The
306.Aq oldrole
307argument is previous resource role (before the change).
308It can be one of:
309.Ar init ,
310.Ar secondary ,
311.Ar primary .
312.Pp
313The
314.Aq newrole
315argument is current resource role (after the change).
316It can be one of:
317.Ar init ,
318.Ar secondary ,
319.Ar primary .
320.Pp
321.It Ic name Aq name
322.Pp
323GEOM provider name that will appear as
324.Pa /dev/hast/<name> .
325If name is not defined, resource name will be used as provider name.
326.It Ic local Aq path
327.Pp
328Path to the local component which will be used as backend provider for
329the resource.
330This can be either GEOM provider or regular file.
331.It Ic remote Aq addr
332.Pp
333Address of the remote
334.Nm hastd
335daemon.
336Format is the same as for the
337.Ic listen
338statement.
339When operating as a primary node this address will be used to connect to
340the secondary node.
341When operating as a secondary node only connections from this address
342will be accepted.
343.Pp
344A special value of
345.Va none
346can be used when the remote address is not yet known (eg. the other node is not
347set up yet).
348.It Ic source Aq addr
349.Pp
350Local address to bind to before connecting to the remote
351.Nm hastd
352daemon.
353Format is the same as for the
354.Ic listen
355statement.
356.El
357.Sh FILES
358.Bl -tag -width ".Pa /var/run/hastctl" -compact
359.It Pa /etc/hast.conf
360The default
361.Nm
362configuration file.
363.It Pa /var/run/hastctl
364Control socket used by the
365.Xr hastctl 8
366control utility to communicate with the
367.Xr hastd 8
368daemon.
369.El
370.Sh EXAMPLES
371The example configuration file can look as follows:
372.Bd -literal -offset indent
373listen tcp://0.0.0.0
374
375on hasta {
376 listen tcp://2001:db8::1/64
377}
378on hastb {
379 listen tcp://2001:db8::2/64
380}
381
382resource shared {
383 local /dev/da0
384
385 on hasta {
386 remote tcp://10.0.0.2
387 }
388 on hastb {
389 remote tcp://10.0.0.1
390 }
391}
392resource tank {
393 on hasta {
394 local /dev/mirror/tanka
395 source tcp://10.0.0.1
396 remote tcp://10.0.0.2
397 }
398 on hastb {
399 local /dev/mirror/tankb
400 source tcp://10.0.0.2
401 remote tcp://10.0.0.1
402 }
403}
404.Ed
405.Sh SEE ALSO
406.Xr gethostname 3 ,
407.Xr geom 4 ,
408.Xr hastctl 8 ,
409.Xr hastd 8 .
410.Sh AUTHORS
411The
412.Nm
413was written by
414.An Pawel Jakub Dawidek Aq pjd@FreeBSD.org
415under sponsorship of the FreeBSD Foundation.