keepalived+LVS实现网站负载均衡和HA

　　如上图所示，102和103是内网nginx服务器，100和101是边界LB，clinet是1，这个实验是为了实现在LB上虚拟出一个VIP，client通过访问该VIP，来动态负载到两台内网nginx服务器上面来。流量的来回，都需要经过边界LB。

　　我们首先分析一下，这种模式是一种NAT模式，流量的来回都要经过边界LB。回忆一下NAT的工作模式，LB在接收到client的数据请求后，根据自身负载均衡算法，将目标IP地址修改为内网RS地址，然后从内网口扔出去，内网服务器收到这个数据请求之后，目标MAC和IP都是自己，源地址是client，这个时候就会处理该数据包，返回，返回的数据包中，源地址是自己，目标地址是client，在经过LB的时候，会被转换，源地址为VIP地址，目标地址为client地址，然后client就会收到这个回包。

　　这个和物理硬件LB的工作模式中，透明模式相同，透明模式也就是NAT模式。而物理硬件中的反向代理模式，和LVS中的tunnel模式相似，在接收到client的数据请求后，会将整个数据包进行改写，源地址为LB的内网地址，目标地址为某一个RS地址，然后数据包返回的时候，也就会再次进行转换。这种反向代理模式中，服务器正常情况下无法知道client的源，只知道是自己的LB访问了自己，可以通过配置LB，添加字段，来讲该client的地址一并发送给RS。

　　物理硬件LB中的三角模式，也就是LVS的DR模式。需要在服务器上配置lo接口。

　　回到环境中。这个地方采用的是NAT模式，也就是说流量必须要经过LB，最简单的做法，就是将两台NGINX服务器的网关，配置在LB上，而且是以VIP的形式存在，对外，LB也拿出一个VIP，作为client访问的入口，也就是说，LB上，有两个VRRP INSTANCE，而且要保持状态一致。这样才能实现高可用HA。

　　下面开始做配置了，主要是在两台LB上配置keepalived，keepalived是默认安装的，配置文件存放在/etc/keepalived/目录下，启动脚本为/etc/init.d/keepalived，定义100为主，101为备。下面是两台设备的keepalived配置文件。

! Configuration File for keepalived

! The global defines,including notification server and target,router-id and others

global_defs {

    notification_email {

        admin@domain.com

        @domain.com

    }

    notification_email_from keepalived@domain.com

    smtp_server 127.0.0.1

    smtp_connect_timeout

    router_id LB_100

}

! The vrrp sync group,any vrrp instance in the vrrp sync group while keep the same status,

!       no matter the priority ,for example ,VI_1 is master ,VI_2 is backup,the sync group status is backup

vrrp_sync_group VG1 {

    group {

        VI_To_client

        VI_To_server

    }

}

! The VRRP instance for client to connect,clients connect to the VIPs

vrrp_instance VI_To_client {

    state MASTER

    interface eth0

    virtual_router_id

    priority

    nopreempt

    advert_int

    authentication {

        auth_type PASS

        auth_pass

    }

    virtual_ipaddress {

        192.168.245.200

    }

}

! The VRRP instance for real servers,real servers` default gateway

vrrp_instance VI_To_server {

    state MASTER

    interface eth1

    virtual_router_id

    priority

    nopreempt

    advert_int

    authentication {

        auth_type PASS

        auth_pass

    }

    virtual_ipaddress {

        10.0.0.254

    }

}

! The LB configuration,including VS and RS,scheduling argu,and other options,health check

virtual_server 192.168.245.200  {

    delay_loop

    lb_algo rr

    lb_kind NAT

    persistence_timeout

    protocol TCP

    real_server 10.0.0.102  {

        weight

        TCP_CHECK {

            connect_timeout

        }

    }

    real_server 10.0.0.103  {

        weight

        TCP_CHECK {

            connect_timeout

        }

    }

}

　　在100的配置文件中，主要是配置了一个vrrp_sync_group，里面有两个VRRP INSTANCE，分别是面向client和面向RS的，两个VRRP INSTANCE的vrrp-_id是不同的，但是其他属性值都是ixiangtong的。另外，在keepalived的LB配置中，有对VS和RS的配置，同时对RS使用了health check，检查方法很简单，就是TCP的connect，当能够访问到时，将该RS动态的加入到ipvs中，如果访问不到，就从ipvs中移除。这个现象可以通过查看日志看到。

　　slave的配置文件和master的配置文件大同小异，除了必要的字段不同，其他都是相同的。下面看看keepalived的日志信息。

Jul  :: CentOS_101 Keepalived[]: Starting Keepalived v1.2.13 (/,)

Jul  :: CentOS_101 Keepalived[]: Starting Healthcheck child process, pid=

Jul  :: CentOS_101 Keepalived[]: Starting VRRP child process, pid=

Jul  :: CentOS_101 Keepalived_vrrp[]: Netlink reflector reports IP 192.168.245.101 added

Jul  :: CentOS_101 Keepalived_healthcheckers[]: Netlink reflector reports IP 192.168.245.101 added

Jul  :: CentOS_101 Keepalived_healthcheckers[]: Netlink reflector reports IP 10.0.0.101 added

Jul  :: CentOS_101 Keepalived_healthcheckers[]: Netlink reflector reports IP fe80::20c:29ff:fe88: added

Jul  :: CentOS_101 Keepalived_healthcheckers[]: Netlink reflector reports IP fe80::20c:29ff:fe88: added

Jul  :: CentOS_101 Keepalived_healthcheckers[]: Registering Kernel netlink reflector

Jul  :: CentOS_101 Keepalived_healthcheckers[]: Registering Kernel netlink command channel

Jul  :: CentOS_101 Keepalived_healthcheckers[]: Opening file '/etc/keepalived/keepalived.conf'.

Jul  :: CentOS_101 Keepalived_healthcheckers[]: Configuration is using :  Bytes

Jul  :: CentOS_101 Keepalived_healthcheckers[]: Using LinkWatch kernel netlink reflector...

Jul  :: CentOS_101 Keepalived_vrrp[]: Netlink reflector reports IP 10.0.0.101 added

Jul  :: CentOS_101 Keepalived_vrrp[]: Netlink reflector reports IP fe80::20c:29ff:fe88: added

Jul  :: CentOS_101 Keepalived_vrrp[]: Netlink reflector reports IP fe80::20c:29ff:fe88: added

Jul  :: CentOS_101 Keepalived_vrrp[]: Registering Kernel netlink reflector

Jul  :: CentOS_101 Keepalived_vrrp[]: Registering Kernel netlink command channel

Jul  :: CentOS_101 Keepalived_vrrp[]: Registering gratuitous ARP shared channel

Jul  :: CentOS_101 Keepalived_vrrp[]: Opening file '/etc/keepalived/keepalived.conf'.

Jul  :: CentOS_101 Keepalived_vrrp[]: Configuration is using :  Bytes

Jul  :: CentOS_101 Keepalived_vrrp[]: Using LinkWatch kernel netlink reflector...

Jul 15 13:57:01 CentOS_101 Keepalived_healthcheckers[26015]: Activating healthchecker for service [10.0.0.102]:80

Jul 15 13:57:01 CentOS_101 Keepalived_healthcheckers[26015]: Activating healthchecker for service [10.0.0.103]:80

Jul  :: CentOS_101 Keepalived_vrrp[]: VRRP_Instance(VI_To_client) Entering BACKUP STATE

Jul  :: CentOS_101 Keepalived_vrrp[]: VRRP_Instance(VI_To_server) Entering BACKUP STATE

Jul  :: CentOS_101 Keepalived_vrrp[]: VRRP sockpool: [ifindex(), proto(), unicast(), fd(,)]

Jul  :: CentOS_101 Keepalived_vrrp[]: VRRP sockpool: [ifindex(), proto(), unicast(), fd(,)]

　　这个是启动日志，在启动的时候，就对两个RS就开启了健康检查。

Jul  :: CentOS_101 Keepalived_healthcheckers[]: TCP connection to [10.0.0.102]: failed !!!

Jul  :: CentOS_101 Keepalived_healthcheckers[]: Removing service [10.0.0.102]: from VS [192.168.245.200]:

Jul  :: CentOS_101 Keepalived_healthcheckers[]: TCP connection to [10.0.0.102]: success.

Jul  :: CentOS_101 Keepalived_healthcheckers[]: Adding service [10.0.0.102]: to VS [192.168.245.200]:

Jul  :: CentOS_101 Keepalived_healthcheckers[]: TCP connection to [10.0.0.103]: failed !!!

Jul  :: CentOS_101 Keepalived_healthcheckers[]: Removing service [10.0.0.103]: from VS [192.168.245.200]:

Jul  :: CentOS_101 Keepalived_healthcheckers[]: TCP connection to [10.0.0.103]: success.

Jul  :: CentOS_101 Keepalived_healthcheckers[]: Adding service [10.0.0.103]: to VS [192.168.245.200]:

　　这个是测试RS故障时候，keepalived的日志。日志写的很清楚，当主机故障时，就从VS中移除，同时可以看到ipvsadm的状态。

[root@CentOS_100 keepalived]# ipvsadm -L -n

IP Virtual Server version 1.2. (size=)

Prot LocalAddress:Port Scheduler Flags

  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

TCP  192.168.245.200: rr persistent

  -> 10.0.0.102:                Masq

[root@CentOS_100 keepalived]# ipvsadm -L -n

IP Virtual Server version 1.2. (size=)

Prot LocalAddress:Port Scheduler Flags

  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

TCP  192.168.245.200: rr persistent

  -> 10.0.0.102:                Masq

  -> 10.0.0.103:                Masq

[root@CentOS_100 keepalived]# ipvsadm -L -n

　　client在访问的时候，只需要访问200这个虚拟地址就好了。如果不想LB作为RS的网关，可以直接在RS上写一条路由即可。

　　在NAT模式中，需要在LB上开启转发，先查看sysctl -p查看是否开启了转发，如果没有开启，需要在/etc/sysctl.conf中修改net.ipv4.ip_forward为1，默认为0，如果为0，数据包在经过LB的时候，是不会被转发出去的。这个时候的LB，作为的是一个路由器，用linux作为路由器，是需要开启这个属性的。

巴特西

keepalived+LVS实现网站负载均衡和HA

最新文章

热门文章