Kebernetes学习总结(5) Ingress

虽然kubernetes集群内部署的pod、server都有自己的IP，但是却无法提供外网访问，以前可以通过监听NodePort的方式暴露服务，但是这种方式并不灵活，生产环境也不建议使用。Ingresss是kubernetes集群中的一个API资源对象，扮演边缘路由器(edge router)的角色，也可以理解为集群防火墙、集群网关，可以自定义路由规则来转发、管理、暴露服务(一组pod)，非常灵活，生产环境建议使用这种方式。另外LoadBlancer也可以暴露服务，不过这种方式需要向云平台申请负债均衡器；虽然目前很多云平台都支持，但是这种方式深度耦合了云平台。
Ingress相关组件：
1) Ingress controller
kubernetes中的controller有很多，比如CronJob、DeamonSet、Deployment、ReplicationSet、StatefulSet等，它们的作用就是监控集群的变化，使集群始终保持期望的最终状态(yml文件)。同理Ingress controller的作用就是实时感知Ingress路由规则集合的变化，再与Api Server交互，获取Service、Pod在集群中的IP等信息，然后发送给反向代理web服务器(ingress pods)，刷新其路由配置信息，这就是它的服务发现机制。
2) 反向代理web服务器(ingress pods) 负责转发外部请求到后端service，比如Nginx、Apache、traefik等等
3) Ingress 定义路由规则集合
4) NodePort类型的service 负责将外部流量引入kubernetes cluster，然后由反向代理web服务器处理。

本篇总结中的示例大致如下：首先部署ingress controller及 ingress pods；然后利用NodePort类型的service将外部流最引入kubernetes cluster中；然后再定义后端deployment及后端service；然后定义ingress 转发规则集合；最后访问测试。
1) 部署ingress pods
[root@docker79 ingress]# mkdir example
[root@docker79 ingress]# cd example/
[root@docker79 example]# ?for file in namespace.yaml configmap.yaml rbac.yaml tcp-services-configmap.yaml with-rbac.yaml default-backend.yaml udp-services-configmap.yaml ; do wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/$file ; done
[root@docker79 example]# kubectl apply -f .
configmap/nginx-configuration created
deployment.extensions/default-http-backend created
service/default-http-backend created
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
namespace/ingress-nginx configured
serviceaccount/nginx-ingress-serviceaccount created
clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created
role.rbac.authorization.k8s.io/nginx-ingress-role created
rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created
clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created
configmap/tcp-services created
configmap/udp-services created
deployment.extensions/nginx-ingress-controller created
[root@docker79 example]#
[root@docker79 example]# kubectl get pods -n ingress-nginx
NAME ???????????????????????????????????????READY ????STATUS ???RESTARTS ??AGE
default-http-backend-6586bc58b6-8rxtp ??????1/1 ??????Running ??0 ?????????59m
nginx-ingress-controller-6bd7c597cb-xdrcm ??1/1 ??????Running ??0 ?????????59m
[root@docker79 example]#

2) 在ingress-nginx 名称空间定义nodePort类型的service 引入外部流量，过程如下所示：

[root@docker79 ingress]# cat nginx-frontend-ingress-svc.yamlapiVersion: v1kind: Servicemetadata: ?name: nginx-frontend-ingress-svc ?namespace: ingress-nginx ?labels: ???app.kubernetes.io/name: ingress-nginx ???app.kubernetes.io/part-of: ingress-nginxspec: ?type: NodePort ?ports: ?- name: http ???port: 80 ???targetPort: 80 ???protocol: TCP ???nodePort: 30080 ?- name: https ???port: 443 ???targetPort: 443 ???protocol: TCP ???nodePort: 30443 ?selector: ???app.kubernetes.io/name: ingress-nginx[root@docker79 ingress]#[root@docker79 ingress]# kubectl apply -f nginx-frontend-ingress-svc.yamlservice/nginx-frontend-ingress-svc created[root@docker79 ingress]# kubectl get svc -n ingress-nginxNAME ??????????????????TYPE ???????CLUSTER-IP ????EXTERNAL-IP ??PORT(S) ?????????????????????AGEdefault-http-backend ??ClusterIP ??10.102.71.36 ??<none> ???????80/TCP ??????????????????????1hnginx-frontend-ingress-svc ????NodePort ???10.103.52.44 ??<none> ???????80:30080/TCP,443:30443/TCP ??11s[root@docker79 ingress]#

3) 下例中部署两个后端nginx-backend-deploy、tomcat-backend-deploy。
首先部署nginx-backend-deploy及nginx-backend-svc，过程如下：

[root@docker79 ingress]# vim backend-nginx.yaml[root@docker79 ingress]# cat backend-nginx.yamlapiVersion: v1kind: Servicemetadata: ?name: nginx-backend-svc ?namespace: defaultspec: ?selector: ???app: nginx-backend ???release: canary ?ports: ?- name: http ???targetPort: 80 ???port: 80---apiVersion: apps/v1kind: Deploymentmetadata: ?name: nginx-backend-deploy ?namespace: defaultspec: ?replicas: 2 ?selector: ???matchLabels: ?????app: nginx-backend ?????release: canary ?template: ???metadata: ?????labels: ???????app: nginx-backend ???????release: canary ???spec: ?????containers: ?????- name: nginx-container ???????image: nginx:1.15-alpine ???????ports: ???????- name: http ?????????containerPort: 80[root@docker79 ingress]#[root@docker79 ingress]# kubectl apply -f backend-nginx.yamlservice/nginx-backend-svc createddeployment.apps/nginx-backend-deploy created[root@docker79 ingress]# kubectl get podsNAME ??????????????????????????????????READY ????STATUS ???RESTARTS ??AGEnginx-backend-deploy-9c9b85bf4-6587j ??1/1 ??????Running ??0 ?????????8snginx-backend-deploy-9c9b85bf4-k64tv ??1/1 ??????Running ??0 ?????????8s[root@docker79 ingress]#[root@docker79 ingress]# kubectl get svcNAME ???????????????TYPE ???????CLUSTER-IP ??????EXTERNAL-IP ??PORT(S) ??AGEkubernetes ?????????ClusterIP ??10.96.0.1 ???????<none> ???????443/TCP ??3dnginx-backend-svc ??ClusterIP ??10.110.150.187 ??<none> ???????80/TCP ???1m[root@docker79 ingress]#

然后部署tomcat-backend-deploy及tomcat-backend-svc，过程如下：

[root@docker79 ingress]# vim backend-tomcat.yaml[root@docker79 ingress]# cat backend-tomcat.yamlapiVersion: v1kind: Servicemetadata: ?name: tomcat-backend-svc ?namespace: defaultspec: ?selector: ???app: tomcat-backend ???release: canary ?ports: ?- name: http ???port: 8080 ???targetPort: 8080 ?- name: ajp ???port: 8009 ???targetPort: 8009---apiVersion: apps/v1kind: Deploymentmetadata: ?name: tomcat-deploy ?namespace: defaultspec: ?replicas: 3 ?selector: ???matchLabels: ?????app: tomcat-backend ?????release: canary ?template: ???metadata: ?????labels: ???????app: tomcat-backend ???????release: canary ???spec: ?????containers: ?????- name: tomcat ???????image: tomcat:8.5.32-jre8-alpine ???????ports: ???????- name: http ?????????containerPort: 8080 ???????- name: ajp ?????????containerPort: 8009[root@docker79 ingress]# kubectl apply -f backend-tomcat.yamlservice/tomcat-backend-svc createddeployment.apps/tomcat-deploy created[root@docker79 ingress]# kubectl get podsNAME ??????????????????????????????????READY ????STATUS ???RESTARTS ??AGEnginx-backend-deploy-9c9b85bf4-6587j ??1/1 ??????Running ??0 ?????????24mnginx-backend-deploy-9c9b85bf4-k64tv ??1/1 ??????Running ??0 ?????????24mtomcat-deploy-7b4c97ff9f-fv8rx ????????1/1 ??????Running ??0 ?????????2mtomcat-deploy-7b4c97ff9f-rj94r ????????1/1 ??????Running ??0 ?????????2m[root@docker79 ingress]# kubectl get svcNAME ????????????????TYPE ???????CLUSTER-IP ??????EXTERNAL-IP ??PORT(S) ????????????AGEkubernetes ??????????ClusterIP ??10.96.0.1 ???????<none> ???????443/TCP ????????????3dnginx-backend-svc ???ClusterIP ??10.110.150.187 ??<none> ???????80/TCP ?????????????24mtomcat-backend-svc ??ClusterIP ??10.105.189.191 ??<none> ???????8080/TCP,8009/TCP ??2m[root@docker79 ingress]#

4) 部署 ingress 转发规则集合
首先部署nginx-backend-svc的ingress 转发规则集合，过程如下所示:

[root@docker79 ingress]# vim ingress-rules-ngx.yaml[root@docker79 ingress]# cat ingress-rules-ngx.yamlapiVersion: extensions/v1beta1kind: Ingressmetadata: ?name: ingress-rule-ngx ?namespace: default ?annotations: ???kubernetes.io/ingress.class: "nginx"spec: ?rules: ?- host: nginx.inspiry.com ???http: ?????paths: ?????- path: ???????backend: ?????????serviceName: nginx-backend-svc ?????????servicePort: 80[root@docker79 ingress]#[root@docker79 ingress]# kubectl apply -f ingress-rules.yamlingress.extensions/ingress-rule-ngx created[root@docker79 ingress]# kubectl get ingressNAME ??????????????HOSTS ??????????????ADDRESS ??PORTS ????AGEingress-rule-ngx ??nginx.inspiry.com ????????????80 ???????9s[root@docker79 ingress]#[root@docker79 ingress]# kubectl describe ingress ?ingress-rule-ngxName: ????????????ingress-rule-ngxNamespace: ???????defaultAddress:Default backend: ?default-http-backend:80 (<none>)Rules: ?Host ??????????????Path ?Backends ?---- ??????????????---- ?-------- ?nginx.inspiry.com ???????????????????????nginx-backend-svc:80 (<none>)Annotations: ?kubectl.kubernetes.io/last-applied-configuration: ?{"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"ingress-rule-ngx","namespace":"default"},"spec":{"rules":[{"host":"nginx.inspiry.com","http":{"paths":[{"backend":{"serviceName":"nginx-backend-svc","servicePort":80},"path":null}]}}]}} ?kubernetes.io/ingress.class: ?nginxEvents: ????????????????????????<none>[root@docker79 ingress]#

然后再部署tomcat-backend-svc的ingress 转发规则集合，过程如下所示：

[root@docker79 ingress]# vim ingress-rules-tomcat.yaml[root@docker79 ingress]# cat ingress-rules-tomcat.yamlapiVersion: extensions/v1beta1kind: Ingressmetadata: ?name: ingress-rule-tomcat ?namespace: default ?annotations: ???kubernetes.io/ingress.class: "nginx"spec: ?rules: ?- host: tomcat.inspiry.cn ???http: ?????paths: ?????- path: ???????backend: ?????????serviceName: tomcat-backend-svc ?????????servicePort: 8080[root@docker79 ingress]# kubectl apply -f ingress-rules-tomcat.yamlingress.extensions/ingress-rule-tomcat created[root@docker79 ingress]# kubectl get ingressNAME ?????????????????HOSTS ??????????????ADDRESS ??PORTS ????AGEingress-rule-ngx ?????nginx.inspiry.com ????????????80 ???????11mingress-rule-tomcat ??tomcat.inspiry.cn ????????????80 ???????6s[root@docker79 ingress]#[root@docker79 ingress]# kubectl describe ingress ingress-rule-tomcatName: ????????????ingress-rule-tomcatNamespace: ???????defaultAddress:Default backend: ?default-http-backend:80 (<none>)Rules: ?Host ??????????????Path ?Backends ?---- ??????????????---- ?-------- ?tomcat.inspiry.cn ???????????????????????tomcat-backend-svc:8080 (<none>)Annotations: ?kubectl.kubernetes.io/last-applied-configuration: ?{"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"ingress-rule-tomcat","namespace":"default"},"spec":{"rules":[{"host":"tomcat.inspiry.cn","http":{"paths":[{"backend":{"serviceName":"tomcat-backend-svc","servicePort":8080},"path":null}]}}]}} ?kubernetes.io/ingress.class: ?nginxEvents: ????????????????????????<none>[root@docker79 ingress]#

5) 测试
在kubernetes cluster 外的任何一台主机上使用 ?curl http://nginx.inspiry.com:30080/ ?或 curl http://tomcat.inspiry.cn:30080/ 即可访问相应nginx-backend 、tomcat-backend 的pods。
生产环境中还可以在k8s cluster外再创建一层 LB，将接收到的 nginx.inspiry.com、tomcat.inspiry.cn 域名分别转发到192.168.20.79、192.168.20.78、192.168.20.77为后端的upstream上。

观察ingress controller的内容如下：

[root@docker79 ~]# kubectl get pods -n ingress-nginxNAME ???????????????????????????????????????READY ????STATUS ???RESTARTS ??AGEdefault-http-backend-6586bc58b6-8rxtp ??????1/1 ??????Running ??0 ?????????1hnginx-ingress-controller-6bd7c597cb-xdrcm ??1/1 ??????Running ??0 ?????????1h[root@docker79 ~]# kubectl exec -n ingress-nginx -it nginx-ingress-controller-6bd7c597cb-xdrcm -- /bin/sh$ lsfastcgi.conf ???????geoip ???mime.types ?????nginx.conf ????????scgi_params ?????uwsgi_params.defaultfastcgi.conf.default ???koi-utf ?mime.types.default ?nginx.conf.default ????scgi_params.default ?win-utffastcgi_params ?????koi-win ?modsecurity ????????opentracing.json ??????templatefastcgi_params.default ?lua ?modules ????????owasp-modsecurity-crs ?uwsgi_params$ cat nginx.conf...... (可仔细观察该文件的内容，发现相应的转发规则是随着 ingress rules的变化而变化的)

Kebernetes学习总结(5) Ingress

原文地址：http://blog.51cto.com/caiyuanji/2286809