Linux集群架构（下）

八、DR模式搭建

8.1 准备工作

试验需求三台机器:

• 分发器，也叫调度器（简写为dir) ： 192.168.112.136 ying01

• rs1 ：192.168.112.138 ying02

• rs2 ：192.168.112.139 ying03

• vip ：192.168.112.200

• ying01上设置

新建lvs_dr脚本，按下面内容配置

[root@ying01 ~]# vim /usr/local/sbin/lvs_dr.sh#! /bin/bashecho 1 > /proc/sys/net/ipv4/ip_forwardipv=/usr/sbin/ipvsadmvip=192.168.112.200rs1=192.168.112.138rs2=192.168.112.139#注意这里的网卡名字ifdown wns33ifup  ens33ifconfig ens33:2 $vip broadcast $vip netmask 255.255.255.255 uproute add -host $vip dev ens33:2$ipv -C$ipv -A -t $vip:80 -s wrr$ipv -a -t $vip:80 -r $rs1:80 -g -w 1$ipv -a -t $vip:80 -r $rs2:80 -g -w 1

执行脚本，查看IP，发现vip在ens33上

[root@ying01 ~]# sh /usr/local/sbin/lvs_dr.sh 成功断开设备 'ens33'。连接已成功激活（D-Bus 活动路径：/org/freedesktop/NetworkManager/ActiveConnection/3）[root@ying01 ~]# ip add1: lo: 
    
      mtu 65536 qdisc noqueue state UNKNOWN qlen 1    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00    inet 127.0.0.1/8 scope host lo       valid_lft forever preferred_lft forever    inet6 ::1/128 scope host        valid_lft forever preferred_lft forever2: ens33: 
     
       mtu 1500 qdisc pfifo_fast state UP qlen 1000    link/ether 00:0c:29:87:3f:91 brd ff:ff:ff:ff:ff:ff    inet 192.168.112.136/24 brd 192.168.112.255 scope global ens33       valid_lft forever preferred_lft forever    inet 192.168.112.200/32 brd 192.168.112.200 scope global ens33:2       valid_lft forever preferred_lft forever    inet 192.168.112.158/24 brd 192.168.112.255 scope global secondary ens33:0       valid_lft forever preferred_lft forever    inet6 fe80::16dc:89c:b761:e115/64 scope link        valid_lft forever preferred_lft forever3: ens37: 
      
        mtu 1500 qdisc pfifo_fast state UP qlen 1000    link/ether 00:0c:29:87:3f:9b brd ff:ff:ff:ff:ff:ff    inet 192.168.24.128/24 brd 192.168.24.255 scope global dynamic ens37       valid_lft 1434sec preferred_lft 1434sec    inet6 fe80::ad38:a02e:964e:1b93/64 scope link        valid_lft forever preferred_lft forever
      
     
    

• ying02上

先把网关改为：192.168.112.2，重启网络

[root@ying02 ~]# vim /etc/sysconfig/network-scripts/ifcfg-ens33  GATEWAY=192.168.112.2[root@ying02 ~]# systemctl restart network

新建lvs_rs脚本；

[root@ying02 ~]# vim /usr/local/sbin/lvs_rs.sh#/bin/bashvip=192.168.112.200#把vip绑定在lo上，是为了实现rs直接把结果返回给客户端ifdown loifup loifconfig lo:0 $vip broadcast $vip netmask 255.255.255.255 uproute add -host $vip lo:0#以下操作为更改arp内核参数，目的是为了让rs顺利发送mac地址给客户端#参考文档www.cnblogs.com/lgfeng/archive/2012/10/16/2726308.htmlecho "1" >/proc/sys/net/ipv4/conf/lo/arp_ignoreecho "2" >/proc/sys/net/ipv4/conf/lo/arp_announceecho "1" >/proc/sys/net/ipv4/conf/all/arp_ignoreecho "2" >/proc/sys/net/ipv4/conf/all/arp_announce

执行脚本，并route -n ，查看网络信息

[root@ying02 ~]# sh /usr/local/sbin/lvs_rs.sh[root@ying02 ~]# route -nKernel IP routing tableDestination     Gateway         Genmask         Flags Metric Ref    Use Iface0.0.0.0         192.168.112.2   0.0.0.0         UG    100    0        0 ens33192.168.112.0   0.0.0.0         255.255.255.0   U     100    0        0 ens33192.168.112.200 0.0.0.0         255.255.255.255 UH    0      0        0 lo[root@ying02 ~]# ip add1: lo: 
    
      mtu 65536 qdisc noqueue state UNKNOWN qlen 1    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00    inet 127.0.0.1/8 scope host lo       valid_lft forever preferred_lft forever    inet 192.168.112.200/32 brd 192.168.112.200 scope global lo:0       valid_lft forever preferred_lft forever    inet6 ::1/128 scope host        valid_lft forever preferred_lft forever2: ens33: 
     
       mtu 1500 qdisc pfifo_fast state UP qlen 1000    link/ether 00:0c:29:c6:2c:24 brd ff:ff:ff:ff:ff:ff    inet 192.168.112.138/24 brd 192.168.112.255 scope global ens33       valid_lft forever preferred_lft forever    inet6 fe80::964f:be22:ddf2:54b7/64 scope link        valid_lft forever preferred_lft forever3: ens37: 
      
        mtu 1500 qdisc pfifo_fast state UP qlen 1000    link/ether 00:0c:29:c6:2c:2e brd ff:ff:ff:ff:ff:ff
      
     
    

• ying03

先把网关改为：192.168.112.2，重启网络

[root@ying03 ~]# vim /etc/sysconfig/network-scripts/ifcfg-ens33[root@ying03 ~]# systemctl restart network

新建lvs_rs脚本；

[root@ying03 ~]# vim /usr/local/sbin/lvs_rs.sh#/bin/bashvip=192.168.112.200ifdown loifup lo#把vip绑定在lo上，是为了实现rs直接把结果返回给客户端ifconfig lo:0 $vip broadcast $vip netmask 255.255.255.255 uproute add -host $vip lo:0#以下操作为更改arp内核参数，目的是为了让rs顺利发送mac地址给客户端#参考文档www.cnblogs.com/lgfeng/archive/2012/10/16/2726308.htmlecho "1" >/proc/sys/net/ipv4/conf/lo/arp_ignoreecho "2" >/proc/sys/net/ipv4/conf/lo/arp_announceecho "1" >/proc/sys/net/ipv4/conf/all/arp_ignoreecho "2" >/proc/sys/net/ipv4/conf/all/arp_announce

8.2 测试

执行脚本，vip在lo网卡上

[root@ying03 ~]# sh /usr/local/sbin/lvs_rs.sh[root@ying03 ~]# route -nKernel IP routing tableDestination     Gateway         Genmask         Flags Metric Ref    Use Iface0.0.0.0         192.168.112.2   0.0.0.0         UG    100    0        0 ens33192.168.112.0   0.0.0.0         255.255.255.0   U     100    0        0 ens33192.168.112.200 0.0.0.0         255.255.255.255 UH    0      0        0 lo[root@ying03 ~]# ip add1: lo: 
    
      mtu 65536 qdisc noqueue state UNKNOWN qlen 1    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00    inet 127.0.0.1/8 scope host lo       valid_lft forever preferred_lft forever    inet 192.168.112.200/32 brd 192.168.112.200 scope global lo:0       valid_lft forever preferred_lft forever    inet6 ::1/128 scope host        valid_lft forever preferred_lft forever2: ens33: 
     
       mtu 1500 qdisc pfifo_fast state UP qlen 1000    link/ether 00:0c:29:ed:0f:50 brd ff:ff:ff:ff:ff:ff    inet 192.168.112.139/24 brd 192.168.112.255 scope global ens33       valid_lft forever preferred_lft forever    inet6 fe80::43bd:36bd:3f01:f8e8/64 scope link        valid_lft forever preferred_lft forever    inet6 fe80::964f:be22:ddf2:54b7/64 scope link tentative dadfailed        valid_lft forever preferred_lft forever3: ens37: 
      
        mtu 1500 qdisc pfifo_fast state UP qlen 1000    link/ether 00:0c:29:ed:0f:5a brd ff:ff:ff:ff:ff:ff
      
     
    

在浏览器测试，输入VIP：192.168.112.100；不断刷新浏览器；会出现ying02或者ying03页面

通过ipvsadm -ln查看 规则，可以看出其信息；

[root@ying01 ~]# ipvsadm -lnIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags  -> RemoteAddress:Port           Forward Weight ActiveConn InActConnTCP  192.168.112.200:80 wrr  -> 192.168.112.138:80           Route   1      2          9           -> 192.168.112.139:80           Route   1      2          9

九、 keepalived+LVS

编辑配置文件，把之前的内容清空，按下面配置

[root@ying01 ~]# vim /etc/keepalived/keepalived.conf vrrp_instance VI_1 {    #备用服务器上为 BACKUP    state MASTER    #绑定vip的网卡为ens33    interface ens33    virtual_router_id 51    #备用服务器上为90    priority 100    advert_int 1    authentication {        auth_type PASS        auth_pass aminglinux    }    virtual_ipaddress {        192.168.112.200    }}virtual_server 192.168.112.200 80 {    #(每隔10秒查询realserver状态)    delay_loop 10    #(lvs 算法)    lb_algo wlc    #(DR模式)    lb_kind DR    #(同一IP的连接60秒内被分配到同一台realserver)    persistence_timeout 60    #(用TCP协议检查realserver状态)    protocol TCP    real_server 192.168.112.138 80 {        #(权重)        weight 100        TCP_CHECK {        #(10秒无响应超时)        connect_timeout 10        nb_get_retry 3        delay_before_retry 3        connect_port 80        }    }    real_server 192.168.112.139 80 {        weight 100        TCP_CHECK {        connect_timeout 10        nb_get_retry 3        delay_before_retry 3        connect_port 80        }     }}

查看网卡IP

[root@ying01 ~]# ip add1: lo: 
    
      mtu 65536 qdisc noqueue state UNKNOWN qlen 1    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00    inet 127.0.0.1/8 scope host lo       valid_lft forever preferred_lft forever    inet6 ::1/128 scope host        valid_lft forever preferred_lft forever2: ens33: 
     
       mtu 1500 qdisc pfifo_fast state UP qlen 1000    link/ether 00:0c:29:87:3f:91 brd ff:ff:ff:ff:ff:ff    inet 192.168.112.136/24 brd 192.168.112.255 scope global ens33       valid_lft forever preferred_lft forever    inet 192.168.112.158/24 brd 192.168.112.255 scope global secondary ens33:0       valid_lft forever preferred_lft forever    inet6 fe80::16dc:89c:b761:e115/64 scope link        valid_lft forever preferred_lft forever3: ens37: 
      
        mtu 1500 qdisc pfifo_fast state UP qlen 1000    link/ether 00:0c:29:87:3f:9b brd ff:ff:ff:ff:ff:ff    inet 192.168.24.128/24 brd 192.168.24.255 scope global dynamic ens37       valid_lft 1216sec preferred_lft 1216sec    inet6 fe80::ad38:a02e:964e:1b93/64 scope link        valid_lft forever preferred_lft forever
      
     
    

ipvsadm -ln ：查看其规则，开启与未开启keepalived服务对比；

[root@ying01 ~]# ipvsadm -lnIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn[root@ying01 ~]# systemctl start keepalived[root@ying01 ~]# ipvsadm -lnIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags  -> RemoteAddress:Port           Forward Weight ActiveConn InActConnTCP  192.168.112.200:80 wlc persistent 60  -> 192.168.112.138:80           Route   100    0          0           -> 192.168.112.139:80           Route   100    0          0

开展执行其脚本；做实验，不断刷新浏览器；

[root@ying01 ~]# sh /usr/local/sbin/lvs_dr.sh成功断开设备 'ens33'。连接已成功激活（D-Bus 活动路径：/org/freedesktop/NetworkManager/ActiveConnection/7）[root@ying01 ~]# ipvsadm -lnIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags  -> RemoteAddress:Port           Forward Weight ActiveConn InActConnTCP  192.168.112.200:80 wrr  -> 192.168.112.138:80           Route   1      0          3           -> 192.168.112.139:80           Route   1      0          3

假如ying03宕机了，我们把ying03机器关闭；此时能够剔除ying03机器；

[root@ying01 ~]# ipvsadm -lnIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags  -> RemoteAddress:Port           Forward Weight ActiveConn InActConnTCP  192.168.112.200:80 wrr  -> 192.168.112.138:80           Route   1      3          20        您在 /var/spool/mail/root 中有新邮件

恢复ying03机器，规则里面有添加上ying03

[root@ying01 ~]# ipvsadm -lnIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags  -> RemoteAddress:Port           Forward Weight ActiveConn InActConnTCP  192.168.112.200:80 wrr  -> 192.168.112.138:80           Route   1      3          20          -> 192.168.112.139:80           Route   100    1          1