继续上一章部署。
八、部署高可用组件
本章介绍keepalived和haproxy实现kube-apiserver高可用。
- keepalive 提供 kube-apiserver 对外提供服务的VIP；
- haproxy 监听VIP，后端链接所有 kube-apiserver 实例，提供健康检查和负载均衡功能
- keepalived 一主一备的运行模式，本文档复用 master 节点的两台设备
- haproxy 监听 8443 端口，与 kube-apiserver 的 6443 端口区分开
- keepalived 在运行过程中周期检查本机的 haproxy 进程状态，如果检测到 haproxy 进程异常，则触发重新选主的过程，将VIP飘逸到新选出来的主节点，从而实现 VIP 的高可用
- 所有组件都通过 VIP 和 haproxy 监听的 8443 端口访问 kube-apiserver
8.1 安装软件包
source /opt/k8s/bin/environment.shfor master_ip in ${MASTER_IPS[@]} ?do ???echo ">>> ${master_ip}" ???ssh k8s@${master_ip} "sudo yum install -y keepalived haproxy"done8.2 配置 haproxy
cat > haproxy.cfg <<EOFglobal ???log /dev/log ???local0 ???log /dev/log ???local1 notice ???chroot /var/lib/haproxy ???stats socket /var/run/haproxy-admin.sock mode 660 level admin ???stats timeout 30s ???user haproxy ???group haproxy ???daemon ???nbproc 1defaults ???log ????global ???timeout connect 5000 ???timeout client ?10m ???timeout server ?10mlisten ?admin_stats ???bind 0.0.0.0:10080 ???mode http ???log 127.0.0.1 local0 err ???stats refresh 30s ???stats uri /status ???stats realm welcome login\ Haproxy ???stats auth admin:123456 ???stats hide-version ???stats admin if TRUElisten kube-master ???bind 0.0.0.0:8443 ???mode tcp ???option tcplog ???balance source ???server 192.168.56.20 192.168.56.20:6443 check inter 2000 fall 2 rise 2 weight 1 ???server 192.168.56.21 192.168.56.21:6443 check inter 2000 fall 2 rise 2 weight 1EOF
- haproxy 在10080端口输出 status 信息
- haproxy 监听所有接口的 8443 端口，该端口与环境变量 ${KUBE_APISERVER} 指定的端口必须一致
- server 字段列出所有 kube-apiserver 监听的 IP 和端口
8.3 下发 haproxy.cfg 到所有 master 节点
source /opt/k8s/bin/environment.shfor master_ip in ${MASTER_IPS[@]} ?do ???echo ">>> ${master_ip}" ???scp haproxy.cfg root@${master_ip}:/etc/haproxydone8.4 起 haproxy 服务
source /opt/k8s/bin/environment.shfor master_ip in ${MASTER_IPS[@]} ?do ???echo ">>> ${master_ip}" ???ssh k8s@${master_ip} "sudo systemctl restart haproxy"done8.5 检查服务状态
source /opt/k8s/bin/environment.shfor master_ip in ${MASTER_IPS[@]} ?do ???echo ">>> ${master_ip}" ???ssh k8s@${master_ip} "sudo systemctl status haproxy|grep Active"done确保状态都是 active (running)。
查看日志：journalctl -u haproxy
8.6 检查是否监听了 8443 端口
source /opt/k8s/bin/environment.shfor master_ip in ${MASTER_IPS[@]} ?do ???echo ">>> ${master_ip}" ???ssh k8s@${master_ip} "sudo netstat -lnpt|grep haproxy"done8.7 配置 keeplived
keepalived 是一主一备的运行模式，所以配置文件类型有两种
master 192.168.56.20backup 192.168.56.21
master配置文件：
source /opt/k8s/bin/environment.shcat ?> keepalived-master.conf <<EOFglobal_defs { ???router_id lb-master-105}vrrp_script check-haproxy { ???script "killall -0 haproxy" ???interval 5 ???weight -30}vrrp_instance VI-kube-master { ???state MASTER ???priority 120 ???dont_track_primary ???interface ${VIP_IF} ???virtual_router_id 68 ???advert_int 3 ???track_script { ???????check-haproxy ???} ???virtual_ipaddress { ???????${MASTER_VIP} ???}}EOF- 两个地方需要注意 ${VIP_IF} 和 ${MASTER_VIP}
- 使用 killall -0 haproxy 命令检查所在节点的 haproxy 进程是否正常。如果异常则将权重减少（-30），从而触发重新选主的过程
- route_id、virtual_router_id 用于标识属于该 HA 的keepalived 实例，如果有多套keepalived HA，则必须各不相同
backup 配置文件：
source /opt/k8s/bin/environment.shcat ?> keepalived-backup.conf <<EOFglobal_defs { ???router_id lb-backup-105}vrrp_script check-haproxy { ???script "killall -0 haproxy" ???interval 5 ???weight -30}vrrp_instance VI-kube-master { ???state BACKUP ???priority 110 ???dont_track_primary ???interface ${VIP_IF} ???virtual_router_id 68 ???advert_int 3 ???track_script { ???????check-haproxy ???} ???virtual_ipaddress { ???????${MASTER_VIP} ???}}EOF- priority 的值必须小于 master
8.8 下发keepalived配置文件
[root@k8s-m1 ha_config]# scp keepalived-master.conf root@192.168.56.20:/etc/keepalived/keepalived.conf[root@k8s-m1 ha_config]# scp keepalived-backup.conf root@192.168.56.21:/etc/keepalived/keepalived.conf
8.9 起 keepalived 服务
source /opt/k8s/bin/environment.shfor master_ip in ${MASTER_IPS[@]} ?do ???echo ">>> ${master_ip}" ???ssh root@${master_ip} "systemctl restart keepalived"done8.10 检查服务状态
# 确认服务是 active (running) source /opt/k8s/bin/environment.shfor master_ip in ${MASTER_IPS[@]} ?do ???echo ">>> ${master_ip}" ???ssh root@${master_ip} "systemctl status keepalived|grep Active"done# 查看 VIP 所在的节点，确保可以 ping 通 VIPsource /opt/k8s/bin/environment.shfor master_ip in ${MASTER_IPS[@]} ?do ???echo ">>> ${master_ip}" ???ssh root@${master_ip} "/usr/sbin/ip addr show ${VIP_IF}" ???ssh root@${master_ip} "ping -c 1 ${MASTER_VIP}"done# 查看日志 journalctl -u keepalived8.11 浏览器打开status也没
地址：${MASTER_VIP}:10080/status
用户名和密码在 haproxy.cfg 的 status auth 字段
HAProxy status页面
二进制安装kubernetes v1.11.2 （第七章 部署高可用组件）
原文地址：https://www.cnblogs.com/aast/p/9844068.html