KeepAliveD, Rackspace Cloud, Unicast - оба LB переходят в состояние MASTER...?
Я установил два балансировщика нагрузки HAProxy в паре Active->Passive.
KeepAliveD будет использоваться для аварийного переключения между серверами MASTER и BACKUP.
Как и большинство облаков, Multicast не поддерживается, поэтому я не могу использовать виртуальный IP. Вместо этого я пытаюсь использовать Unicast, который я видел в Интернете, как решение.
Моя проблема в том, что экземпляр BACKUP KeepAliveD сразу переходит в состояние MASTER. Он может пропинговать сервер MASTER, но он не способен понять, что это действительно так.
Я бы классифицировал себя как системный администратор n00b, поэтому, пожалуйста, прости меня. По этой причине я надеюсь, что есть некоторые очевидные ошибки, которые я делаю, которые можно легко исправить...
__[HAProxy Active, KeepAliveD MASTER, 10.179.66.95]
/
----|
|
\__[HAProxy Passive, KeepAliveD BACKUP, 10.179.74.172]
Конфиги следующим образом...
KeepAliveD версия на обоих
1.2.9 (поддержка Unicast была добавлена в 1.2.8 и исправлена в 1.2.9).
http://www.keepalived.org/changelog.html
На обоих серверах в /etc/sysctl.conf
# Nonlocal bind for use with KeepAliveD. Allows this instance to take on a non-local IP for failover.
net.ipv4.ip_nonlocal_bind=1
KeepAliveD MASTER в /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
me@me.com
}
notification_email_from me@me.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LB_MASTER_ACTIVE
}
# Define the script used to check if haproxy is still working
vrrp_script chk_haproxy {
script "killall -0 haproxy" # verify the pid existance
interval 2 # check every 2 seconds
weight 2 # add 2 points of prio if OK
}
# Virtual interface.
vrrp_instance VI_1 {
state MASTER
interface eth1
virtual_router_id 51
priority 101
smtp_alert # Activate e-mail notifications.
#advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
# IP of myself and my peer for unicast based failover.
vrrp_unicast_bind 10.179.66.95 # My IP.
vrrp_unicast_peer 10.179.74.172 # The other's IP.
# Check if HAProxy is running or not.
track_script {
chk_haproxy
}
}
KeepAliveD BACKUP в /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
me@me.com
}
notification_email_from me@me.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LB_BACKUP_PASSIVE
}
# Define the script used to check if haproxy is still working
vrrp_script chk_haproxy {
script "killall -0 haproxy"
interval 2
weight 2
}
# Virtual interface.
vrrp_instance VI_1 {
state BACKUP
interface eth1
virtual_router_id 51
priority 100 # MASTER is priority 101.
smtp_alert # Activate e-mail notifications.
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
# IP of myself and my peer for unicast based failover.
vrrp_unicast_bind 10.179.74.172 # My IP.
vrrp_unicast_peer 10.179.66.95 # The other's IP.
# Check if HAProxy is running or not.
track_script {
chk_haproxy
}
}
Сообщения регистрируются на KeepAliveD MASTER при запуске KeepAliveD, tail -f /var/log/messages
Nov 28 10:54:02 mysql-read-lb-1 Keepalived[30158]: Starting Keepalived v1.2.7 (02/21,2013)
Nov 28 10:54:02 mysql-read-lb-1 Keepalived[30159]: Starting Healthcheck child process, pid=30161
Nov 28 10:54:02 mysql-read-lb-1 Keepalived[30159]: Starting VRRP child process, pid=30162
Nov 28 10:54:02 mysql-read-lb-1 Keepalived_healthcheckers[30161]: Interface queue is empty
Nov 28 10:54:02 mysql-read-lb-1 Keepalived_vrrp[30162]: Interface queue is empty
Nov 28 10:54:02 mysql-read-lb-1 Keepalived_healthcheckers[30161]: No such interface, eth1
Nov 28 10:54:02 mysql-read-lb-1 Keepalived_healthcheckers[30161]: No such interface, eth2
Nov 28 10:54:02 mysql-read-lb-1 Keepalived_healthcheckers[30161]: Netlink reflector reports IP 10.179.66.95 added
Nov 28 10:54:02 mysql-read-lb-1 Keepalived_healthcheckers[30161]: Netlink reflector reports IP 192.168.3.1 added
Nov 28 10:54:02 mysql-read-lb-1 Keepalived_healthcheckers[30161]: Netlink reflector reports IP fe80::be76:4eff:fe08:9227 added
Nov 28 10:54:02 mysql-read-lb-1 Keepalived_healthcheckers[30161]: Netlink reflector reports IP fe80::be76:4eff:fe08:8b4d added
Nov 28 10:54:02 mysql-read-lb-1 Keepalived_healthcheckers[30161]: Registering Kernel netlink reflector
Nov 28 10:54:02 mysql-read-lb-1 Keepalived_healthcheckers[30161]: Registering Kernel netlink command channel
Nov 28 10:54:02 mysql-read-lb-1 Keepalived_healthcheckers[30161]: Opening file '/etc/keepalived/keepalived.conf'.
Nov 28 10:54:02 mysql-read-lb-1 Keepalived_vrrp[30162]: No such interface, eth1
Nov 28 10:54:02 mysql-read-lb-1 Keepalived_vrrp[30162]: No such interface, eth2
Nov 28 10:54:02 mysql-read-lb-1 Keepalived_vrrp[30162]: Netlink reflector reports IP 10.179.66.95 added
Nov 28 10:54:02 mysql-read-lb-1 Keepalived_vrrp[30162]: Netlink reflector reports IP 192.168.3.1 added
Nov 28 10:54:02 mysql-read-lb-1 Keepalived_vrrp[30162]: Netlink reflector reports IP fe80::be76:4eff:fe08:9227 added
Nov 28 10:54:02 mysql-read-lb-1 Keepalived_vrrp[30162]: Netlink reflector reports IP fe80::be76:4eff:fe08:8b4d added
Nov 28 10:54:02 mysql-read-lb-1 Keepalived_vrrp[30162]: Registering Kernel netlink reflector
Nov 28 10:54:02 mysql-read-lb-1 Keepalived_vrrp[30162]: Registering Kernel netlink command channel
Nov 28 10:54:02 mysql-read-lb-1 Keepalived_vrrp[30162]: Registering gratuitous ARP shared channel
Nov 28 10:54:02 mysql-read-lb-1 Keepalived_vrrp[30162]: Opening file '/etc/keepalived/keepalived.conf'.
Nov 28 10:54:02 mysql-read-lb-1 Keepalived_healthcheckers[30161]: Configuration is using : 7559 Bytes
Nov 28 10:54:02 mysql-read-lb-1 Keepalived_vrrp[30162]: Truncating auth_pass to 8 characters
Nov 28 10:54:02 mysql-read-lb-1 Keepalived_vrrp[30162]: Configuration is using : 64400 Bytes
Nov 28 10:54:02 mysql-read-lb-1 Keepalived_vrrp[30162]: Using LinkWatch kernel netlink reflector...
Nov 28 10:54:02 mysql-read-lb-1 Keepalived_vrrp[30162]: VRRP sockpool: [ifindex(3), proto(112), fd(10,11)]
Nov 28 10:54:02 mysql-read-lb-1 Keepalived_healthcheckers[30161]: Using LinkWatch kernel netlink reflector...
Nov 28 10:54:03 mysql-read-lb-1 Keepalived_vrrp[30162]: VRRP_Instance(VI_1) Transition to MASTER STATE
Nov 28 10:54:04 mysql-read-lb-1 Keepalived_vrrp[30162]: VRRP_Instance(VI_1) Entering MASTER STATE
Nov 28 10:54:04 mysql-read-lb-1 Keepalived_vrrp[30162]: Remote SMTP server [127.0.0.1]:25 connected.
Nov 28 10:54:04 mysql-read-lb-1 Keepalived_vrrp[30162]: SMTP alert successfully sent.
Сообщения входят в KeepAliveD BACKUP при запуске KeepAliveD, tail -f /var/log/messages
Вы заметите, что он сразу же переходит в состояние MASTER, но должен оставаться в РЕЗЕРВНОМ...
Nov 28 10:57:35 load-balancer-1-passive Keepalived[25048]: Starting Keepalived v1.2.7 (02/21,2013)
Nov 28 10:57:35 load-balancer-1-passive Keepalived[25049]: Starting Healthcheck child process, pid=25050
Nov 28 10:57:35 load-balancer-1-passive Keepalived[25049]: Starting VRRP child process, pid=25052
Nov 28 10:57:35 load-balancer-1-passive Keepalived_healthcheckers[25050]: Interface queue is empty
Nov 28 10:57:35 load-balancer-1-passive Keepalived_healthcheckers[25050]: No such interface, eth1
Nov 28 10:57:35 load-balancer-1-passive Keepalived_healthcheckers[25050]: No such interface, eth2
Nov 28 10:57:35 load-balancer-1-passive Keepalived_healthcheckers[25050]: Netlink reflector reports IP 10.179.74.172 added
Nov 28 10:57:35 load-balancer-1-passive Keepalived_healthcheckers[25050]: Netlink reflector reports IP 192.168.3.2 added
Nov 28 10:57:35 load-balancer-1-passive Keepalived_healthcheckers[25050]: Netlink reflector reports IP fe80::be76:4eff:fe08:93fc added
Nov 28 10:57:35 load-balancer-1-passive Keepalived_healthcheckers[25050]: Netlink reflector reports IP fe80::be76:4eff:fe08:940c added
Nov 28 10:57:35 load-balancer-1-passive Keepalived_healthcheckers[25050]: Registering Kernel netlink reflector
Nov 28 10:57:35 load-balancer-1-passive Keepalived_healthcheckers[25050]: Registering Kernel netlink command channel
Nov 28 10:57:35 load-balancer-1-passive Keepalived_healthcheckers[25050]: Opening file '/etc/keepalived/keepalived.conf'.
Nov 28 10:57:35 load-balancer-1-passive Keepalived_vrrp[25052]: Interface queue is empty
Nov 28 10:57:35 load-balancer-1-passive Keepalived_vrrp[25052]: No such interface, eth1
Nov 28 10:57:35 load-balancer-1-passive Keepalived_vrrp[25052]: No such interface, eth2
Nov 28 10:57:35 load-balancer-1-passive Keepalived_vrrp[25052]: Netlink reflector reports IP 10.179.74.172 added
Nov 28 10:57:35 load-balancer-1-passive Keepalived_healthcheckers[25050]: Configuration is using : 7595 Bytes
Nov 28 10:57:35 load-balancer-1-passive Keepalived_vrrp[25052]: Netlink reflector reports IP 192.168.3.2 added
Nov 28 10:57:35 load-balancer-1-passive Keepalived_vrrp[25052]: Netlink reflector reports IP fe80::be76:4eff:fe08:93fc added
Nov 28 10:57:35 load-balancer-1-passive Keepalived_vrrp[25052]: Netlink reflector reports IP fe80::be76:4eff:fe08:940c added
Nov 28 10:57:35 load-balancer-1-passive Keepalived_vrrp[25052]: Registering Kernel netlink reflector
Nov 28 10:57:35 load-balancer-1-passive Keepalived_vrrp[25052]: Registering Kernel netlink command channel
Nov 28 10:57:35 load-balancer-1-passive Keepalived_vrrp[25052]: Registering gratuitous ARP shared channel
Nov 28 10:57:35 load-balancer-1-passive Keepalived_vrrp[25052]: Opening file '/etc/keepalived/keepalived.conf'.
Nov 28 10:57:35 load-balancer-1-passive Keepalived_vrrp[25052]: Truncating auth_pass to 8 characters
Nov 28 10:57:35 load-balancer-1-passive Keepalived_vrrp[25052]: Configuration is using : 64436 Bytes
Nov 28 10:57:35 load-balancer-1-passive Keepalived_vrrp[25052]: Using LinkWatch kernel netlink reflector...
Nov 28 10:57:35 load-balancer-1-passive Keepalived_vrrp[25052]: VRRP_Instance(VI_1) Entering BACKUP STATE
Nov 28 10:57:35 load-balancer-1-passive Keepalived_vrrp[25052]: VRRP sockpool: [ifindex(3), proto(112), fd(10,11)]
Nov 28 10:57:35 load-balancer-1-passive Keepalived_vrrp[25052]: Remote SMTP server [127.0.0.1]:25 connected.
Nov 28 10:57:35 load-balancer-1-passive Keepalived_healthcheckers[25050]: Using LinkWatch kernel netlink reflector...
Nov 28 10:57:35 load-balancer-1-passive Keepalived_vrrp[25052]: VRRP_Script(chk_haproxy) succeeded
Nov 28 10:57:35 load-balancer-1-passive Keepalived_vrrp[25052]: SMTP alert successfully sent.
Nov 28 10:57:38 load-balancer-1-passive Keepalived_vrrp[25052]: VRRP_Instance(VI_1) Transition to MASTER STATE
Nov 28 10:57:39 load-balancer-1-passive Keepalived_vrrp[25052]: VRRP_Instance(VI_1) Entering MASTER STATE
Nov 28 10:57:39 load-balancer-1-passive Keepalived_vrrp[25052]: Remote SMTP server [127.0.0.1]:25 connected.
Nov 28 10:57:39 load-balancer-1-passive Keepalived_vrrp[25052]: SMTP alert successfully sent.
Интерфейсы сервера KeepAliveD MASTER, "ip a":
[root@load-balancer-1-active keepalived]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast state DOWN qlen 1000
link/ether bc:76:4e:08:92:38 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether bc:76:4e:08:92:27 brd ff:ff:ff:ff:ff:ff
inet 10.179.66.95/18 brd 10.179.127.255 scope global eth1
inet6 fe80::be76:4eff:fe08:9227/64 scope link
valid_lft forever preferred_lft forever
Интерфейсы сервера KeepAliveD BACKUP, "ip a":
[root@load-balancer-1-passive ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast state DOWN qlen 1000
link/ether bc:76:4e:08:4f:b4 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether bc:76:4e:08:93:fc brd ff:ff:ff:ff:ff:ff
inet 10.179.74.172/18 brd 10.179.127.255 scope global eth1
inet6 fe80::be76:4eff:fe08:93fc/64 scope link
valid_lft forever preferred_lft forever
Нюхать на МАСТЕР
Как предлагает "emy " ниже, прослушивание пакетов, чтобы увидеть, проходит ли проверка работоспособности, как предложено здесь: http://www.cyberciti.biz/faq/linux-unix-verify-keepalived-working-or-not/
[root@mysql-read-lb-1 ~]# tcpdump -vvv -n -i eth1 host 10.179.74.172
tcpdump: listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
Ничего такого:(
Нюхать на BACKUP
[root@load-balancer-1-passive ~]# tcpdump -vvv -n -i eth1 host 10.179.66.95
tcpdump: listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
Ничего такого:(
1 ответ
Некоторые догадываются:
Вы уверены, что трафик vrrp проходит? Не могли бы вы понюхать (например, ngrep, tcpdump) на порту 112, если пакеты получены? (Вы должны видеть одну каждую секунду.) Смотрите эту ссылку.
Если нет, это может быть проблема с брандмауэром.