| carp.4 (224883) | carp.4 (228571) |
|---|---|
| 1.\" $OpenBSD: carp.4,v 1.16 2004/12/07 23:41:35 jmc Exp $ 2.\" 3.\" Copyright (c) 2003, Ryan McBride. All rights reserved. | 1.\" $OpenBSD: carp.4,v 1.16 2004/12/07 23:41:35 jmc Exp $ 2.\" 3.\" Copyright (c) 2003, Ryan McBride. All rights reserved. |
| 4.\" Copyright (c) 2011, Gleb Smirnoff <glebius@FreeBSD.org> |
|
| 4.\" 5.\" Redistribution and use in source and binary forms, with or without 6.\" modification, are permitted provided that the following conditions 7.\" are met: 8.\" 1. Redistributions of source code must retain the above copyright 9.\" notice, this list of conditions and the following disclaimer. 10.\" 2. Redistributions in binary form must reproduce the above copyright 11.\" notice, this list of conditions and the following disclaimer in the --- 6 unchanged lines hidden (view full) --- 18.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 19.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 20.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 21.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 22.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 23.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 24.\" SUCH DAMAGE. 25.\" | 5.\" 6.\" Redistribution and use in source and binary forms, with or without 7.\" modification, are permitted provided that the following conditions 8.\" are met: 9.\" 1. Redistributions of source code must retain the above copyright 10.\" notice, this list of conditions and the following disclaimer. 11.\" 2. Redistributions in binary form must reproduce the above copyright 12.\" notice, this list of conditions and the following disclaimer in the --- 6 unchanged lines hidden (view full) --- 19.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 20.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 21.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 22.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 23.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 24.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 25.\" SUCH DAMAGE. 26.\" |
| 26.\" $FreeBSD: head/share/man/man4/carp.4 224883 2011-08-15 13:33:28Z glebius $ | 27.\" $FreeBSD: head/share/man/man4/carp.4 228571 2011-12-16 12:16:56Z glebius $ |
| 27.\" | 28.\" |
| 28.Dd August 15, 2011 | 29.Dd December 16, 2011 |
| 29.Dt CARP 4 30.Os 31.Sh NAME 32.Nm carp 33.Nd Common Address Redundancy Protocol 34.Sh SYNOPSIS 35.Cd "device carp" 36.Sh DESCRIPTION | 30.Dt CARP 4 31.Os 32.Sh NAME 33.Nm carp 34.Nd Common Address Redundancy Protocol 35.Sh SYNOPSIS 36.Cd "device carp" 37.Sh DESCRIPTION |
| 37The 38.Nm 39interface is a pseudo-device that implements and controls the 40CARP protocol. 41CARP allows multiple hosts on the same local network to share a set of IP addresses. | 38The CARP allows multiple hosts on the same local network to share a set of 39IPv4 and/or IPv6 addresses. |
| 42Its primary purpose is to ensure that these | 40Its primary purpose is to ensure that these |
| 43addresses are always available, but in some configurations 44.Nm 45can also provide load balancing functionality. | 41addresses are always available. |
| 46.Pp | 42.Pp |
| 47A 48.Nm 49interface can be created at runtime using the 50.Nm ifconfig Li carp Ns Ar N Cm create 51command or by configuring 52it via 53.Va cloned_interfaces 54in the 55.Pa /etc/rc.conf 56file. 57.Pp | |
| 58To use 59.Nm , | 43To use 44.Nm , |
| 60the administrator needs to configure at minimum a common virtual host ID (VHID) 61and virtual host IP address on each machine which is to take part in the virtual 62group. 63Additional parameters can also be set on a per-interface basis: | 45the administrator needs to configure at minimum a common virtual host ID 46(vhid) and attach at least one IP address to this vhid on each machine which 47is to take part in the virtual group. 48Additional parameters can also be set on a per-vhid basis: |
| 64.Cm advbase 65and 66.Cm advskew , 67which are used to control how frequently the host sends advertisements when it 68is the master for a virtual host, and 69.Cm pass 70which is used to authenticate 71.Nm --- 16 unchanged lines hidden (view full) --- 88.Cm advskew 89are put inside CARP advertisements. 90These configurations can be done using 91.Xr ifconfig 8 , 92or through the 93.Dv SIOCSVH 94.Xr ioctl 2 . 95.Pp | 49.Cm advbase 50and 51.Cm advskew , 52which are used to control how frequently the host sends advertisements when it 53is the master for a virtual host, and 54.Cm pass 55which is used to authenticate 56.Nm --- 16 unchanged lines hidden (view full) --- 73.Cm advskew 74are put inside CARP advertisements. 75These configurations can be done using 76.Xr ifconfig 8 , 77or through the 78.Dv SIOCSVH 79.Xr ioctl 2 . 80.Pp |
| 81CARP virtual hosts can be configured on multicast capable interfaces: Ethernet, 82layer 2 VLAN, FDDI and Token Ring. 83An arbitrary number of virtual host IDs can be configured on an interface. 84An arbitrary number of IPv4 or IPv6 addresses can be attached to a particular 85vhid. 86It is important that all hosts participating in a vhid have the same list 87of prefixes configured on the vhid, since all prefixes are included in the 88cryptographic checksum supplied in each advertisement. 89Multiple vhids running on one interface participate in master/backup 90elections independently. 91.Pp |
|
| 96Additionally, there are a number of global parameters which can be set using 97.Xr sysctl 8 : | 92Additionally, there are a number of global parameters which can be set using 93.Xr sysctl 8 : |
| 98.Bl -tag -width ".Va net.inet.carp.arpbalance" | 94.Bl -tag -width ".Va net.inet.carp.preempt" |
| 99.It Va net.inet.carp.allow 100Accept incoming 101.Nm 102packets. 103Enabled by default. 104.It Va net.inet.carp.preempt 105Allow virtual hosts to preempt each other. 106It is also used to failover --- 13 unchanged lines hidden (view full) --- 120Value of 0 disables any logging. 121Value of 1 enables logging state changes of 122.Nm 123interfaces. 124Values above 1 enable logging of bad 125.Nm 126packets. 127Default value is 1. | 95.It Va net.inet.carp.allow 96Accept incoming 97.Nm 98packets. 99Enabled by default. 100.It Va net.inet.carp.preempt 101Allow virtual hosts to preempt each other. 102It is also used to failover --- 13 unchanged lines hidden (view full) --- 116Value of 0 disables any logging. 117Value of 1 enables logging state changes of 118.Nm 119interfaces. 120Values above 1 enable logging of bad 121.Nm 122packets. 123Default value is 1. |
| 128.It Va net.inet.carp.arpbalance 129Balance local traffic using ARP (see below). 130Disabled by default. | |
| 131.It Va net.inet.carp.suppress_preempt 132A read only value showing the status of preemption suppression. 133Preemption can be suppressed if link on an interface is down 134or when 135.Xr pfsync 4 136interface is not synchronized. 137Value of 0 means that preemption is not suppressed, since no 138problems are detected. 139Every problem increments suppression counter. 140.El | 124.It Va net.inet.carp.suppress_preempt 125A read only value showing the status of preemption suppression. 126Preemption can be suppressed if link on an interface is down 127or when 128.Xr pfsync 4 129interface is not synchronized. 130Value of 0 means that preemption is not suppressed, since no 131problems are detected. 132Every problem increments suppression counter. 133.El |
| 141.Sh ARP level load balancing 142The 143.Nm 144has limited abilities for load balancing the incoming connections 145between hosts in Ethernet network. 146For load balancing operation, one needs several CARP interfaces that 147are configured to the same IP address, but to a different VHIDs. 148Once an ARP request is received, the CARP protocol will use a hashing 149function against the source IP address in the ARP request to determine 150which VHID should this request belong to. 151If the corresponding CARP interface is in master state, the ARP request 152will be replied, otherwise it will be ignored. 153See the 154.Sx EXAMPLES 155section for a practical example of load balancing. 156.Pp 157The ARP load balancing has some limitations. 158First, ARP balancing only works on the local network segment. 159It cannot balance traffic that crosses a router, because the 160router itself will always be balanced to the same virtual host. 161Second, ARP load balancing can lead to asymmetric routing 162of incoming and outgoing traffic, and thus combining it with 163.Xr pfsync 4 164is dangerous, because this creates a race condition between 165balanced routers and a host they are serving. 166Imagine an incoming packet creating state on the first router, being 167forwarded to its destination, and destination replying faster 168than the state information is packed and synced with the second router. 169If the reply would be load balanced to second router, it will be 170dropped due to no state. | 134.\".Sh ARP level load balancing 135.\"The 136.\".Nm 137.\"has limited abilities for load balancing the incoming connections 138.\"between hosts in Ethernet network. 139.\"For load balancing operation, one needs several CARP interfaces that 140.\"are configured to the same IP address, but to a different vhids. 141.\"Once an ARP request is received, the CARP protocol will use a hashing 142.\"function against the source IP address in the ARP request to determine 143.\"which vhid should this request belong to. 144.\"If the corresponding CARP interface is in master state, the ARP request 145.\"will be replied, otherwise it will be ignored. 146.\"See the 147.\".Sx EXAMPLES 148.\"section for a practical example of load balancing. 149.\".Pp 150.\"The ARP load balancing has some limitations. 151.\"First, ARP balancing only works on the local network segment. 152.\"It cannot balance traffic that crosses a router, because the 153.\"router itself will always be balanced to the same virtual host. 154.\"Second, ARP load balancing can lead to asymmetric routing 155.\"of incoming and outgoing traffic, and thus combining it with 156.\".Xr pfsync 4 157.\"is dangerous, because this creates a race condition between 158.\"balanced routers and a host they are serving. 159.\"Imagine an incoming packet creating state on the first router, being 160.\"forwarded to its destination, and destination replying faster 161.\"than the state information is packed and synced with the second router. 162.\"If the reply would be load balanced to second router, it will be 163.\"dropped due to no state. |
| 171.Sh STATE CHANGE NOTIFICATIONS 172Sometimes it is useful to get notified about 173.Nm 174status change events. 175This can be accomplished by using 176.Xr devd 8 177hooks. | 164.Sh STATE CHANGE NOTIFICATIONS 165Sometimes it is useful to get notified about 166.Nm 167status change events. 168This can be accomplished by using 169.Xr devd 8 170hooks. |
| 178Master/slave events are signalled as 179.Nm 180interface 181.Dv LINK_UP 182or 183.Dv LINK_DOWN 184event. | 171Master/slave events are signalled under system 172.Dv CARP . 173Subsystem specifies vhid and name of interface, where event occured. 174Type of the message displays new state of vhid. |
| 185Please see 186.Xr devd.conf 5 187and 188.Sx EXAMPLES 189section for more information. 190.Sh EXAMPLES 191For firewalls and routers with multiple interfaces, it is desirable to 192failover all of the 193.Nm 194interfaces together, when one of the physical interfaces goes down. 195This is achieved by the preempt option. 196Enable it on both host A and B: 197.Pp 198.Dl sysctl net.inet.carp.preempt=1 199.Pp | 175Please see 176.Xr devd.conf 5 177and 178.Sx EXAMPLES 179section for more information. 180.Sh EXAMPLES 181For firewalls and routers with multiple interfaces, it is desirable to 182failover all of the 183.Nm 184interfaces together, when one of the physical interfaces goes down. 185This is achieved by the preempt option. 186Enable it on both host A and B: 187.Pp 188.Dl sysctl net.inet.carp.preempt=1 189.Pp |
| 200Assume that host A is the preferred master and 192.168.1.x/24 is 201configured on one physical interface and 192.168.2.y/24 on another. | 190Assume that host A is the preferred master and we are running the 191192.168.1.0/24 prefix on em0 and 192.168.2.0/24 on em1. |
| 202This is the setup for host A: 203.Bd -literal -offset indent | 192This is the setup for host A: 193.Bd -literal -offset indent |
| 204ifconfig carp0 create 205ifconfig carp0 vhid 1 pass mekmitasdigoat 192.168.1.1/24 206ifconfig carp1 create 207ifconfig carp1 vhid 2 pass mekmitasdigoat 192.168.2.1/24 | 194ifconfig em0 vhid 1 pass mekmitasdigoat 192.168.1.1/24 195ifconfig em1 vhid 2 pass mekmitasdigoat 192.168.2.1/24 |
| 208.Ed 209.Pp 210The setup for host B is identical, but it has a higher 211.Cm advskew : 212.Bd -literal -offset indent | 196.Ed 197.Pp 198The setup for host B is identical, but it has a higher 199.Cm advskew : 200.Bd -literal -offset indent |
| 213ifconfig carp0 create 214ifconfig carp0 vhid 1 advskew 100 pass mekmitasdigoat 192.168.1.1/24 215ifconfig carp1 create 216ifconfig carp1 vhid 2 advskew 100 pass mekmitasdigoat 192.168.2.1/24 | 201ifconfig em0 vhid 1 advskew 100 pass mekmitasdigoat 192.168.1.1/24 202ifconfig em1 vhid 2 advskew 100 pass mekmitasdigoat 192.168.2.1/24 |
| 217.Ed 218.Pp 219Because of the preempt option, when one of the physical interfaces of 220host A fails, 221.Cm advskew 222is adjusted to 240 on all its 223.Nm 224interfaces. 225This will cause host B to preempt on both interfaces instead of 226just the failed one. | 203.Ed 204.Pp 205Because of the preempt option, when one of the physical interfaces of 206host A fails, 207.Cm advskew 208is adjusted to 240 on all its 209.Nm 210interfaces. 211This will cause host B to preempt on both interfaces instead of 212just the failed one. |
| 213.\".Pp 214.\"In order to set up an ARP balanced virtual host, it is necessary to configure 215.\"one virtual host for each physical host which would respond to ARP requests 216.\"and thus handle the traffic. 217.\"In the following example, two virtual hosts are configured on two hosts to 218.\"provide balancing and failover for the IP address 192.168.1.10. 219.\".Pp 220.\"First the 221.\".Nm 222.\"interfaces on host A are configured. 223.\"The 224.\".Cm advskew 225.\"of 100 on the second virtual host means that its advertisements will be sent 226.\"out slightly less frequently. 227.\".Bd -literal -offset indent 228.\"ifconfig carp0 create 229.\"ifconfig carp0 vhid 1 pass mekmitasdigoat 192.168.1.10/24 230.\"ifconfig carp1 create 231.\"ifconfig carp1 vhid 2 advskew 100 pass mekmitasdigoat 192.168.1.10/24 232.\".Ed 233.\".Pp 234.\"The configuration for host B is identical, except the 235.\".Cm advskew 236.\"is on virtual host 1 rather than virtual host 2. 237.\".Bd -literal -offset indent 238.\"ifconfig carp0 create 239.\"ifconfig carp0 vhid 1 advskew 100 pass mekmitasdigoat 192.168.1.10/24 240.\"ifconfig carp1 create 241.\"ifconfig carp1 vhid 2 pass mekmitasdigoat 192.168.1.10/24 242.\".Ed 243.\".Pp 244.\"Finally, the ARP balancing feature must be enabled on both hosts: 245.\".Pp 246.\".Dl sysctl net.inet.carp.arpbalance=1 247.\".Pp 248.\"When the hosts receive an ARP request for 192.168.1.10, the source IP address 249.\"of the request is used to compute which virtual host should answer the request. 250.\"The host which is master of the selected virtual host will reply to the 251.\"request, the other(s) will ignore it. 252.\".Pp 253.\"This way, locally connected systems will receive different ARP replies and 254.\"subsequent IP traffic will be balanced among the hosts. 255.\"If one of the hosts fails, the other will take over the virtual MAC address, 256.\"and begin answering ARP requests on its behalf. |
|
| 227.Pp | 257.Pp |
| 228In order to set up an ARP balanced virtual host, it is necessary to configure 229one virtual host for each physical host which would respond to ARP requests 230and thus handle the traffic. 231In the following example, two virtual hosts are configured on two hosts to 232provide balancing and failover for the IP address 192.168.1.10. 233.Pp 234First the 235.Nm 236interfaces on host A are configured. 237The 238.Cm advskew 239of 100 on the second virtual host means that its advertisements will be sent 240out slightly less frequently. 241.Bd -literal -offset indent 242ifconfig carp0 create 243ifconfig carp0 vhid 1 pass mekmitasdigoat 192.168.1.10/24 244ifconfig carp1 create 245ifconfig carp1 vhid 2 advskew 100 pass mekmitasdigoat 192.168.1.10/24 246.Ed 247.Pp 248The configuration for host B is identical, except the 249.Cm advskew 250is on virtual host 1 rather than virtual host 2. 251.Bd -literal -offset indent 252ifconfig carp0 create 253ifconfig carp0 vhid 1 advskew 100 pass mekmitasdigoat 192.168.1.10/24 254ifconfig carp1 create 255ifconfig carp1 vhid 2 pass mekmitasdigoat 192.168.1.10/24 256.Ed 257.Pp 258Finally, the ARP balancing feature must be enabled on both hosts: 259.Pp 260.Dl sysctl net.inet.carp.arpbalance=1 261.Pp 262When the hosts receive an ARP request for 192.168.1.10, the source IP address 263of the request is used to compute which virtual host should answer the request. 264The host which is master of the selected virtual host will reply to the 265request, the other(s) will ignore it. 266.Pp 267This way, locally connected systems will receive different ARP replies and 268subsequent IP traffic will be balanced among the hosts. 269If one of the hosts fails, the other will take over the virtual MAC address, 270and begin answering ARP requests on its behalf. 271.Pp | |
| 272Processing of 273.Nm | 258Processing of 259.Nm |
| 274status change events can be set up by using the following devd.conf rules: | 260status change events can be set up by using the following devd.conf rule: |
| 275.Bd -literal -offset indent 276notify 0 { | 261.Bd -literal -offset indent 262notify 0 { |
| 277 match "system" "IFNET"; 278 match "type" "LINK_UP"; 279 match "subsystem" "carp*"; 280 action "/root/carpcontrol.sh $type $subsystem"; | 263 match "system" "CARP"; 264 match "subsystem" "[0-9]+@"; 265 match "type" "(MASTER|BACKUP)"; 266 action "/root/carpcontrol.sh $subsystem $type"; |
| 281}; | 267}; |
| 282 283notify 0 { 284 match "system" "IFNET"; 285 match "type" "LINK_DOWN"; 286 match "subsystem" "carp*"; 287 action "/root/carpcontrol.sh $type $subsystem"; 288}; | |
| 289.Ed 290.Sh SEE ALSO 291.Xr inet 4 , 292.Xr pfsync 4 , 293.Xr rc.conf 5 , 294.Xr devd.conf 5 , 295.Xr ifconfig 8 , 296.Xr sysctl 8 297.Sh HISTORY 298The 299.Nm 300device first appeared in 301.Ox 3.5 . 302The 303.Nm 304device was imported into 305.Fx 5.4 . | 268.Ed 269.Sh SEE ALSO 270.Xr inet 4 , 271.Xr pfsync 4 , 272.Xr rc.conf 5 , 273.Xr devd.conf 5 , 274.Xr ifconfig 8 , 275.Xr sysctl 8 276.Sh HISTORY 277The 278.Nm 279device first appeared in 280.Ox 3.5 . 281The 282.Nm 283device was imported into 284.Fx 5.4 . |
| 285In 286.Fx 10 287the 288.Nm 289was significantly rewritten, and is no longer a pseudo-interface. |
|