继续前一章的部署。
部署 kube-proxy 组件
14.1 下载和分发二进制文件，参考 第三章
分发到各节点
source /opt/k8s/bin/environment.shfor node_name in ${NODE_NAMES[@]} ?do ???echo ">>> ${node_name}" ???scp k8s/v1.11.2/server/kubernetes/server/bin/{kube-proxy,kubelet} k8s@${node_name}:/opt/k8s/bin/ ???ssh k8s@${node_name} "chmod 755 /opt/k8s/bin/{kube-proxy,kubelet}"done14.2 安装依赖包，参考 第十一章
14.3 创建 kube-proxy 证书
cd /opt/k8s/cert
cat > kube-proxy-csr.json <<EOF{ ?"CN": "system:kube-proxy", ?"key": { ???"algo": "rsa", ???"size": 2048 ?}, ?"names": [ ???{ ?????"C": "CN", ?????"ST": "BeiJing", ?????"L": "BeiJing", ?????"O": "k8s", ?????"OU": "4Paradigm" ???} ?]}EOF
- CN：指定该证书的 User 为 system:kube-proxy；
- 预定义的 RoleBinding system:node-proxier 将User system:kube-proxy 与 Role system:node-proxier 绑定，该 Role 授予了调用 kube-apiserver Proxy 相关 API 的权限；
- 该证书只会被 kube-proxy 当做 client 证书使用，所以 hosts 字段为空；
生成证书和私钥:
cfssl gencert -ca=/etc/kubernetes/cert/ca.pem ??-ca-key=/etc/kubernetes/cert/ca-key.pem ??-config=/etc/kubernetes/cert/ca-config.json ??-profile=kubernetes ?kube-proxy-csr.json | cfssljson -bare kube-proxy
创建和分发 kubeconfig 文件
source /opt/k8s/bin/environment.shkubectl config set-cluster kubernetes ??--certificate-authority=/etc/kubernetes/cert/ca.pem ??--embed-certs=true ??--server=${KUBE_APISERVER} ??--kubeconfig=kube-proxy.kubeconfigkubectl config set-credentials kube-proxy ??--client-certificate=kube-proxy.pem ??--client-key=kube-proxy-key.pem ??--embed-certs=true ??--kubeconfig=kube-proxy.kubeconfigkubectl config set-context default ??--cluster=kubernetes ??--user=kube-proxy ??--kubeconfig=kube-proxy.kubeconfigkubectl config use-context default --kubeconfig=kube-proxy.kubeconfig- --embed-certs-true 将 ca.pem和admin.pem 证书内容嵌入到生成的 kubectl-proxy.kubeconfig 文件中（不加时，写入的是证书文件的路径）
分发 kubeconfig 文件
source /opt/k8s/bin/environment.shfor node_name in ${NODE_NAMES[@]} ?do ???echo ">>> ${node_name}" ???scp kube-proxy.kubeconfig k8s@${node_name}:/etc/kubernetes/done14.3 创建 kube-proxy 配置文件
从 v1.10开始，kube-proxy 部分参数可以在配置文件中配置，可以使用 --write-config-to 选项生成该配置文件，或者参考 kubeproxyconfig 的类型定义源文件：
https://github.com/kubernetes/kubernetes/blob/master/pkg/proxy/apis/kubeproxyconfig/types.go
创建 kube-proxy config 文件模板：
cat >kube-proxy.config.yaml.template <<EOFapiVersion: kubeproxy.config.k8s.io/v1alpha1bindAddress: ##NODE_IP##clientConnection: ?kubeconfig: /etc/kubernetes/kube-proxy.kubeconfigclusterCIDR: ${CLUSTER_CIDR}healthzBindAddress: ##NODE_IP##:10256hostnameOverride: ##NODE_NAME##kind: KubeProxyConfigurationmetricsBindAddress: ##NODE_IP##:10249mode: "ipvs"EOF- bindAddress 监听地址
- clientConnect.kubeconfig 连接 apiserver 的 kubeconfig 文件
- clusterCIDR kube-proxy 根据 --cluster-cidr 判断集群内部和外部流量，指定 --cluster-cidr 或 --masquerade-all 选项后 kube-proxy 才会对访问 Service IP 的请求做 SNAT
- hostnameOverride 参数值必须与 kubelet 的值一致，否则 kube-proxy 启动后会找不到该 Node ，从而不会创建任何 ipvs 规则
- mode 使用 ipvs 模式
为各节点创建和分发 kube-proxy 配置文件
source /opt/k8s/bin/environment.shfor (( i=0; i < 4; i++ )) ?do ????echo ">>> ${NODE_NAMES[i]}" ???sed -e "s/##NODE_NAME##/${NODE_NAMES[i]}/" -e "s/##NODE_IP##/${NODE_IPS[i]}/" kube-proxy.config.yaml.template > kube-proxy-${NODE_NAMES[i]}.config.yaml ???scp kube-proxy-${NODE_NAMES[i]}.config.yaml root@${NODE_NAMES[i]}:/etc/kubernetes/kube-proxy.config.yamldone替换后的 kube-proxy.config.yaml如下
[root@k8s-n2 kubernetes]# cat /etc/kubernetes/kube-proxy.config.yamlapiVersion: kubeproxy.config.k8s.io/v1alpha1bindAddress: 192.168.56.31clientConnection: ?kubeconfig: /etc/kubernetes/kube-proxy.kubeconfigclusterCIDR: 172.30.0.0/16healthzBindAddress: 192.168.56.31:10256hostnameOverride: k8s-n2kind: KubeProxyConfigurationmetricsBindAddress: 192.168.56.31:10249mode: "ipvs"
14.4 创建和分发 kube-proxy systemd unit 文件
source /opt/k8s/bin/environment.shcat > kube-proxy.service <<EOF[Unit]Description=Kubernetes Kube-Proxy ServerDocumentation=https://github.com/GoogleCloudPlatform/kubernetesAfter=network.target[Service]WorkingDirectory=/var/lib/kube-proxyExecStart=/opt/k8s/bin/kube-proxy \ ?--config=/etc/kubernetes/kube-proxy.config.yaml \ ?--alsologtostderr=true \ ?--logtostderr=false \ ?--log-dir=/var/log/kubernetes \ ?--v=2Restart=on-failureRestartSec=5LimitNOFILE=65536[Install]WantedBy=multi-user.targetEOF
分发 kube-proxy systemd unit 文件
source /opt/k8s/bin/environment.shfor node_name in ${NODE_NAMES[@]} ?do ????echo ">>> ${node_name}" ???scp kube-proxy.service root@${node_name}:/etc/systemd/system/done14.5 启动 kube-proxy 服务
source /opt/k8s/bin/environment.shfor node_ip in ${NODE_IPS[@]} ?do ???echo ">>> ${node_ip}" ???ssh root@${node_ip} "mkdir -p /var/lib/kube-proxy" ???ssh root@${node_ip} "mkdir -p /var/log/kubernetes && chown -R k8s /var/log/kubernetes" ???ssh root@${node_ip} "systemctl daemon-reload && systemctl enable kube-proxy && systemctl restart kube-proxy"done检查启动结果：
source /opt/k8s/bin/environment.shfor node_ip in ${NODE_IPS[@]} ?do ???echo ">>> ${node_ip}" ???ssh k8s@${node_ip} "systemctl status kube-proxy|grep Active"done- 确保服务是 Active: active (running) 状态
- 查看日志: journalctl -u kube-proxy
查看监听端口和 metrics
[k8s@k8s-m1 template]$ sudo netstat -lnpt|grep kube-proxtcp ???????0 ?????0 192.168.56.20:10249 ????0.0.0.0:* ??????????????LISTEN ?????729/kube-proxy ?????tcp ???????0 ?????0 192.168.56.20:10256 ????0.0.0.0:* ??????????????LISTEN ?????729/kube-proxy
- 10249：http prometheus metrics port;
- 10256：http healthz port;
查看 ipvs 路由规则
[k8s@k8s-m1 template]$ source /opt/k8s/bin/environment.sh[k8s@k8s-m1 template]$ for node_ip in ${NODE_IPS[@]}> ??do> ????echo ">>> ${node_ip}"> ????ssh root@${node_ip} "/usr/sbin/ipvsadm -ln"> ??done>>> 192.168.56.20IP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags ?-> RemoteAddress:Port ??????????Forward Weight ActiveConn InActConnTCP ?10.254.0.1:443 rr ?-> 192.168.56.20:6443 ??????????Masq ???1 ?????0 ?????????0 ??????????-> 192.168.56.21:6443 ??????????Masq ???1 ?????0 ?????????0 ????????>>> 192.168.56.21IP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags ?-> RemoteAddress:Port ??????????Forward Weight ActiveConn InActConnTCP ?10.254.0.1:443 rr ?-> 192.168.56.20:6443 ??????????Masq ???1 ?????0 ?????????0 ??????????-> 192.168.56.21:6443 ??????????Masq ???1 ?????0 ?????????0 ????????>>> 192.168.56.30IP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags ?-> RemoteAddress:Port ??????????Forward Weight ActiveConn InActConnTCP ?10.254.0.1:443 rr ?-> 192.168.56.20:6443 ??????????Masq ???1 ?????0 ?????????0 ??????????-> 192.168.56.21:6443 ??????????Masq ???1 ?????0 ?????????0 ????????>>> 192.168.56.31IP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags ?-> RemoteAddress:Port ??????????Forward Weight ActiveConn InActConnTCP ?10.254.0.1:443 rr ?-> 192.168.56.20:6443 ??????????Masq ???1 ?????0 ?????????0 ??????????-> 192.168.56.21:6443 ??????????Masq ???1 ?????0 ?????????0 ???可以看到，将所有到 kubernetes cluster ip 443 端口的请求都转发到了 kube-apiserver 的 6443 端口
二进制安装kubernetes v1.11.2 （第十四章 kube-proxy部署）
原文地址：https://www.cnblogs.com/aast/p/9917574.html