问题概述:
在通过shell脚本部署K8S集群的过程中,网络插件kube-flannel.yml部署出现了问题,拉取
image: docker.io/flannel/flannel-cni-plugin:v1.1.2
image: docker.io/flannel/flannel:v0.22.0
这2个镜像一直timeout,通过kubectl get pod
去查看STATUS为Init:ImagePullBackOff,再通过kubectl describe pod
去查看报错Error: ErrImagePull,很明显拉取镜像失败,这个默认的镜像仓库在国外被墙掉了,那么网络插件就无法部署,K8S集群状态就不正常了。
问题现象:
各节点状态
[root@hq-t-temp-centos7 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8smaster NotReady control-plane 20h v1.28.2
test-k8s-node01 NotReady <none> 20h v1.28.2
test-k8s-node02 NotReady <none> 20h v1.28.2
journalctl报错信息
Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotRead
kuboard界面节点状态信息
container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
解决方案
通过手动下载flannel相关镜像load到本地,然后重新打tag,并上传到自建的镜像仓库,可以是本地的,也可以是公网的,我这里使用的是阿里云的,上传完成后,再改一下kube-flannel.yml进行部署(kubectl apply -f kube-flannel.yml
)。
我的阿里云镜像仓库地址(公共):
registry.cn-hangzhou.aliyuncs.com/liuk8s/flannel:v0.21.5
registry.cn-hangzhou.aliyuncs.com/liuk8s/flannel-cni-plugin:v1.1.2
阿里云的镜像仓库上传的时候需要自行建立2个仓库名,分别是flannel-cni-plugin和flannel,参考步骤:
docker load -i flannel.tar
docker images
#REPOSITORY TAG IMAGE ID CREATED SIZE
#flannel/flannel v0.21.5 a6c0cb5dbd21 15 months ago 68.9MB
docker tag a6c0cb5dbd21 registry.cn-hangzhou.aliyuncs.com/liuk8s/flannel:v0.21.5
docker push registry.cn-hangzhou.aliyuncs.com/liuk8s/flannel:v0.21.5
完整的kube-flannel.yml配置如下:
apiVersion: v1
kind: Namespace
metadata:labels:k8s-app: flannelpod-security.kubernetes.io/enforce: privilegedname: kube-flannel
---
apiVersion: v1
kind: ServiceAccount
metadata:labels:k8s-app: flannelname: flannelnamespace: kube-flannel
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:labels:k8s-app: flannelname: flannel
rules:
- apiGroups:- ""resources:- podsverbs:- get
- apiGroups:- ""resources:- nodesverbs:- get- list- watch
- apiGroups:- ""resources:- nodes/statusverbs:- patch
- apiGroups:- networking.k8s.ioresources:- clustercidrsverbs:- list- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:labels:k8s-app: flannelname: flannel
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: flannel
subjects:
- kind: ServiceAccountname: flannelnamespace: kube-flannel
---
apiVersion: v1
data:cni-conf.json: |{"name": "cbr0","cniVersion": "0.3.1","plugins": [{"type": "flannel","delegate": {"hairpinMode": true,"isDefaultGateway": true}},{"type": "portmap","capabilities": {"portMappings": true}}]}net-conf.json: |{"Network": "10.244.0.0/16","Backend": {"Type": "vxlan"}}
kind: ConfigMap
metadata:labels:app: flannelk8s-app: flanneltier: nodename: kube-flannel-cfgnamespace: kube-flannel
---
apiVersion: apps/v1
kind: DaemonSet
metadata:labels:app: flannelk8s-app: flanneltier: nodename: kube-flannel-dsnamespace: kube-flannel
spec:selector:matchLabels:app: flannelk8s-app: flanneltemplate:metadata:labels:app: flannelk8s-app: flanneltier: nodespec:affinity:nodeAffinity:requiredDuringSchedulingIgnoredDuringExecution:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/osoperator: Invalues:- linuxcontainers:- args:- --ip-masq- --kube-subnet-mgrcommand:- /opt/bin/flanneldenv:- name: POD_NAMEvalueFrom:fieldRef:fieldPath: metadata.name- name: POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespace- name: EVENT_QUEUE_DEPTHvalue: "5000"image: registry.cn-hangzhou.aliyuncs.com/liuk8s/flannel:v0.21.5name: kube-flannelresources:requests:cpu: 100mmemory: 50MisecurityContext:capabilities:add:- NET_ADMIN- NET_RAWprivileged: falsevolumeMounts:- mountPath: /run/flannelname: run- mountPath: /etc/kube-flannel/name: flannel-cfg- mountPath: /run/xtables.lockname: xtables-lockhostNetwork: trueinitContainers:- args:- -f- /flannel- /opt/cni/bin/flannelcommand:- cpimage: registry.cn-hangzhou.aliyuncs.com/liuk8s/flannel-cni-plugin:v1.1.2name: install-cni-pluginvolumeMounts:- mountPath: /opt/cni/binname: cni-plugin- args:- -f- /etc/kube-flannel/cni-conf.json- /etc/cni/net.d/10-flannel.conflistcommand:- cpimage: registry.cn-hangzhou.aliyuncs.com/liuk8s/flannel:v0.21.5name: install-cnivolumeMounts:- mountPath: /etc/cni/net.dname: cni- mountPath: /etc/kube-flannel/name: flannel-cfgpriorityClassName: system-node-criticalserviceAccountName: flanneltolerations:- effect: NoScheduleoperator: Existsvolumes:- hostPath:path: /run/flannelname: run- hostPath:path: /opt/cni/binname: cni-plugin- hostPath:path: /etc/cni/net.dname: cni- configMap:name: kube-flannel-cfgname: flannel-cfg- hostPath:path: /run/xtables.locktype: FileOrCreatename: xtables-lock