kubernetes 1.8 高可用安装（五）

5安装网络组件calico

安装前需要确认kubelet配置是否已经增加--network-plugin=cni
如果没有配置就加到kubelet配置文件里

Environment="KUBELET_NETWORK_ARGS=--network-plugin=cni--cni-conf-dir=/etc/cni/net.d--cni-bin-dir=/opt/cni/bin

5.1先装rbac

官方URL
https://docs.projectcalico.org/v2.6/getting-started/kubernetes/installation/rbac.yaml

calico-rbac.yaml

#CalicoVersionv2.6.1#https://docs.projectcalico.org/v2.6/releases#v2.6.1---kind:ClusterRoleapiVersion:rbac.authorization.k8s.io/v1beta1metadata:name:calico-kube-controllersnamespace:kube-systemrules:-apiGroups:-""-extensionsresources:-pods-namespaces-networkpoliciesverbs:-watch-list---kind:ClusterRoleBindingapiVersion:rbac.authorization.k8s.io/v1beta1metadata:name:calico-kube-controllersroleRef:apiGroup:rbac.authorization.k8s.iokind:ClusterRolename:calico-kube-controllerssubjects:-kind:ServiceAccountname:calico-kube-controllersnamespace:kube-system---kind:ClusterRoleapiVersion:rbac.authorization.k8s.io/v1beta1metadata:name:calico-nodenamespace:kube-systemrules:-apiGroups:[""]resources:-pods-nodesverbs:-get---apiVersion:rbac.authorization.k8s.io/v1beta1kind:ClusterRoleBindingmetadata:name:calico-noderoleRef:apiGroup:rbac.authorization.k8s.iokind:ClusterRolename:calico-nodesubjects:-kind:ServiceAccountname:calico-nodenamespace:kube-system

5.2 创建calico.yaml

官方URL
https://docs.projectcalico.org/v2.6/getting-started/kubernetes/installation/hosted/calico.yaml

配置修改请看下面的参数说明

#CalicoVersionv2.6.1#https://docs.projectcalico.org/v2.6/releases#v2.6.1#Thismanifestincludesthefollowingcomponentversions:#calico/node:v2.6.1#calico/cni:v1.11.0#calico/kube-controllers:v1.0.0#ThisConfigMapisusedtoconfigureaself-hostedCalicoinstallation.kind:ConfigMapapiVersion:v1metadata:name:calico-confignamespace:kube-systemdata:#Configurethiswiththelocationofyouretcdcluster.etcd_endpoints:":2379"#ConfiguretheCalicobackendtouse.calico_backend:"bird"#TheCNInetworkconfigurationtoinstalloneachnode.cni_network_config:|-{"name":"k8s-pod-network","cniVersion":"0.1.0","type":"calico","etcd_endpoints":"__ETCD_ENDPOINTS__","etcd_key_file":"__ETCD_KEY_FILE__","etcd_cert_file":"__ETCD_CERT_FILE__","etcd_ca_cert_file":"__ETCD_CA_CERT_FILE__","log_level":"info","mtu":1500,"ipam":{"type":"calico-ipam"},"policy":{"type":"k8s","k8s_api_root":"https://__KUBERNETES_SERVICE_HOST__:__KUBERNETES_SERVICE_PORT__","k8s_auth_token":"__SERVICEACCOUNT_TOKEN__"},"kubernetes":{"kubeconfig":"__KUBECONFIG_FILEPATH__"}}#Ifyou‘reusingTLSenabledetcduncommentthefollowing.#YoumustalsopopulatetheSecretbelowwiththesefiles.etcd_ca:""#"/calico-secrets/etcd-ca"etcd_cert:""#"/calico-secrets/etcd-cert"etcd_key:""#"/calico-secrets/etcd-key"---#Thefollowingcontainsk8sSecretsforusewithaTLSenabledetcdcluster.#ForinformationonpopulatingSecrets,seehttp://kubernetes.io/docs/user-guide/secrets/apiVersion:v1kind:Secrettype:Opaquemetadata:name:calico-etcd-secretsnamespace:kube-systemdata:#PopulatethefollowingfileswithetcdTLSconfigurationifdesired,butleaveblankif#notusingTLSforetcd.#Thisself-hostedinstallexpectsthreefileswiththefollowingnames.Thevalues#shouldbebase64encodedstringsoftheentirecontentsofeachfile.#etcd-key:null#etcd-cert:null#etcd-ca:null---#Thismanifestinstallsthecalico/nodecontainer,aswell#astheCalicoCNIpluginsandnetworkconfigon#eachmasterandworkernodeinaKubernetescluster.kind:DaemonSetapiVersion:extensions/v1beta1metadata:name:calico-nodenamespace:kube-systemlabels:k8s-app:calico-nodespec:selector:matchLabels:k8s-app:calico-nodetemplate:metadata:labels:k8s-app:calico-nodeannotations:scheduler.alpha.kubernetes.io/critical-pod:‘‘scheduler.alpha.kubernetes.io/tolerations:|[{"key":"dedicated","value":"master","effect":"NoSchedule"},{"key":"CriticalAddonsOnly","operator":"Exists"}]spec:hostNetwork:trueserviceAccountName:calico-nodecontainers:#Runscalico/nodecontaineroneachKubernetesnode.This#containerprogramsnetworkpolicyandroutesoneach#host.-name:calico-nodeimage:quay.io/calico/node:v2.6.1env:#ThelocationoftheCalicoetcdcluster.-name:ETCD_ENDPOINTSvalueFrom:configMapKeyRef:name:calico-configkey:etcd_endpoints#Choosethebackendtouse.-name:CALICO_NETWORKING_BACKENDvalueFrom:configMapKeyRef:name:calico-configkey:calico_backend#Clustertypetoidentifythedeploymenttype-name:CLUSTER_TYPEvalue:"k8s,bgp"#Disablefileloggingso`kubectllogs`works.-name:CALICO_DISABLE_FILE_LOGGINGvalue:"true"#SetFelixendpointtohostdefaultactiontoACCEPT.-name:FELIX_DEFAULTENDPOINTTOHOSTACTIONvalue:"ACCEPT"#ConfiguretheIPPoolfromwhichPodIPswillbechosen.-name:CALICO_IPV4POOL_CIDRvalue:"192.168.0.0/16"-name:CALICO_IPV4POOL_IPIPvalue:"always"#DisableIPv6onKubernetes.-name:FELIX_IPV6SUPPORTvalue:"false"#SetFelixloggingto"info"-name:FELIX_LOGSEVERITYSCREENvalue:"info"#SetMTUfortunneldeviceusedifipipisenabled-name:FELIX_IPINIPMTUvalue:"1440"#Auto-detecttheBGPIPaddress.-name:IPvalue:"autodetect"-name:IP_AUTODETECTION_METHODvalue:"can-reach=www.baidu.com"-name:FELIX_HEALTHENABLEDvalue:"true"securityContext:privileged:trueresources:requests:cpu:250mlivenessProbe:httpGet:path:/livenessport:9099periodSeconds:10initialDelaySeconds:10failureThreshold:6readinessProbe:httpGet:path:/readinessport:9099periodSeconds:10volumeMounts:-mountPath:/lib/modulesname:lib-modulesreadOnly:true-mountPath:/var/run/caliconame:var-run-calicoreadOnly:false-mountPath:/calico-secretsname:etcd-certs#ThiscontainerinstallstheCalicoCNIbinaries#andCNInetworkconfigfileoneachnode.-name:install-cniimage:quay.io/calico/cni:v1.11.0command:["/install-cni.sh"]env:#ThelocationoftheCalicoetcdcluster.-name:ETCD_ENDPOINTSvalueFrom:configMapKeyRef:name:calico-configkey:etcd_endpoints#TheCNInetworkconfigtoinstalloneachnode.-name:CNI_NETWORK_CONFIGvalueFrom:configMapKeyRef:name:calico-configkey:cni_network_configvolumeMounts:-mountPath:/host/opt/cni/binname:cni-bin-dir-mountPath:/host/etc/cni/net.dname:cni-net-dir-mountPath:/calico-secretsname:etcd-certsvolumes:#Usedbycalico/node.-name:lib-moduleshostPath:path:/lib/modules-name:var-run-calicohostPath:path:/var/run/calico#UsedtoinstallCNI.-name:cni-bin-dirhostPath:path:/opt/cni/bin-name:cni-net-dirhostPath:path:/etc/cni/net.d#MountintheetcdTLSsecrets.-name:etcd-certssecret:secretName:calico-etcd-secrets---#ThismanifestdeploystheCalicoKubernetescontrollers.#Seehttps://github.com/projectcalico/kube-controllersapiVersion:extensions/v1beta1kind:Deploymentmetadata:name:calico-kube-controllersnamespace:kube-systemlabels:k8s-app:calico-kube-controllersannotations:scheduler.alpha.kubernetes.io/critical-pod:‘‘scheduler.alpha.kubernetes.io/tolerations:|[{"key":"dedicated","value":"master","effect":"NoSchedule"},{"key":"CriticalAddonsOnly","operator":"Exists"}]spec:#Thecontrollerscanonlyhaveasingleactiveinstance.replicas:1strategy:type:Recreatetemplate:metadata:name:calico-kube-controllersnamespace:kube-systemlabels:k8s-app:calico-kube-controllersspec:#Thecontrollersmustruninthehostnetworknamespacesothat#itisn‘tgovernedbypolicythatwouldpreventitfromworking.hostNetwork:trueserviceAccountName:calico-kube-controllerscontainers:-name:calico-kube-controllersimage:quay.io/calico/kube-controllers:v1.0.0env:#ThelocationoftheCalicoetcdcluster.-name:ETCD_ENDPOINTSvalueFrom:configMapKeyRef:name:calico-configkey:etcd_endpoints---#Thisdeploymentturnsofftheold"policy-controller".Itshouldremainat0replicas,andthen#beremovedentirelyoncethenewkube-controllersdeploymenthasbeendeployedabove.apiVersion:extensions/v1beta1kind:Deploymentmetadata:name:calico-policy-controllernamespace:kube-systemlabels:k8s-app:calico-policyspec:#Turnthisdeploymentoffinfavorofthekube-controllersdeploymentabove.replicas:0strategy:type:Recreatetemplate:metadata:name:calico-policy-controllernamespace:kube-systemlabels:k8s-app:calico-policyspec:hostNetwork:trueserviceAccountName:calico-kube-controllerscontainers:-name:calico-policy-controllerimage:quay.io/calico/kube-controllers:v1.0.0env:#ThelocationoftheCalicoetcdcluster.-name:ETCD_ENDPOINTSvalueFrom:configMapKeyRef:name:calico-configkey:etcd_endpoints---apiVersion:v1kind:ServiceAccountmetadata:name:calico-kube-controllersnamespace:kube-system---apiVersion:v1kind:ServiceAccountmetadata:name:calico-nodenamespace:kube-system

参数说明：

• etcd_endpoints
  改为你自己的etcd集群

• CALICO_IPV4POOL_CIDR
  calico的IP池，不要和集群的cidr，以及机器的其他IP段冲突，比如用：10.10.0.0/16

• IP Autodetection methods
  机器多网卡的时候，安装calico-node会报错，因为calico默认IP的获取方式是first-found，这个ip可能不是你需要的那个。导致网络不成功，导致注册失败

#calico报错日志SkippingdatastoreconnectiontestIPv4address10.96.0.1discoveredoninterfacekube-ipvs0NoASnumberconfiguredonnoderesource,usingglobalvalue

需要修改calico.yaml,修改IP的获取方式为autodetect,注意顺序,修改如下

-name:IPvalue:"autodetect"-name:IP_AUTODETECTION_METHODvalue:"can-reach=www.baidu.com"

IP_AUTODETECTION_METHOD 参数说明
官方文档URL:https://docs.projectcalico.org/v2.6/reference/node/configuration

• 使用通过ip访问的interface
  can-reach=61.135.169.121

• 使用通过域名访问的interface
  can-reach=www.baidu.com

• 使用指定的interface
  interface=ethx

此时node都应该处于Ready状态

[root@kvm-masternetwork]#kubectlgetnodesNAMESTATUSROLESAGEVERSIONnode2Ready<none>23hv1.8.0node1Ready<none>1dv1.8.0

5.3 安装calicoctl管理calico网络

calicoctl.yaml

#CalicoVersionv2.6.1#https://docs.projectcalico.org/v2.6/releases#v2.6.1#Thismanifestincludesthefollowingcomponentversions:#calico/ctl:v1.6.1apiVersion:v1kind:P