K8S报错异常锦集(持续更新)

K8S报错异常锦集(持续更新)

更新镜像没反应 k8s组件异常

①故障现象:使用脚本更新pod服务镜像,pod无变化,状态没更新;

更新镜像:无变化
kubectl set image deployment/em-api em-api=192.168.90.10/zhufuc/em-api:v1.0-20201110100058
pod状态:
em-api-86855df489-hmvnr   1/1     Running             0          16m   172.18.94.8   k8s-n5   <none>           <none>

解决思路:检查脚本,检查私有仓库,检查步骤,查看日志,查看k8s组件

问题:k8s组件状态异常,controller-manager 异常

经排查脚本没问题,私有仓库没问题,查看日志

journalctl -f -u kubelet
Nov 10 10:11:22 k8s-m1 kubelet[32270]: E1110 10:11:22.165336   32270 kuberuntime_sandbox.go:65] CreatePodSandbox for pod "traefik-ingress-controller-jjss4_kube-system(63eee933-933a-11e9-928a-fefcfe274f71)" failed: rpc error: code = Unknown desc = failed to start sandbox container for pod "traefik-ingress-controller-jjss4": Error response from daemon: driver failed programming external connectivity on endpoint k8s_POD_traefik-ingress-controller-jjss4_kube-system_63eee933-933a-11e9-928a-fefcfe274f71_17529857 (ec7dbca09838629f1e4825175f4be3819723cb1984c9b7d00c2ed499b834fa5a):  (iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 0/0 --dport 8080 -j DNAT --to-destination 172.18.88.14:8080 ! -i docker0: iptables: No chain/target/match by that name.
Nov 10 10:11:22 k8s-m1 kubelet[32270]: (exit status 1))
Nov 10 10:11:22 k8s-m1 kubelet[32270]: E1110 10:11:22.165351   32270 kuberuntime_manager.go:662] createPodSandbox for pod "traefik-ingress-controller-jjss4_kube-system(63eee933-933a-11e9-928a-fefcfe274f71)" failed: rpc error: code = Unknown desc = failed to start sandbox container for pod "traefik-ingress-controller-jjss4": Error response from daemon: driver failed programming external connectivity on endpoint k8s_POD_traefik-ingress-controller-jjss4_kube-system_63eee933-933a-11e9-928a-fefcfe274f71_17529857 (ec7dbca09838629f1e4825175f4be3819723cb1984c9b7d00c2ed499b834fa5a):  (iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 0/0 --dport 8080 -j DNAT --to-destination 172.18.88.14:8080 ! -i docker0: iptables: No chain/target/match by that name.
Nov 10 10:11:22 k8s-m1 kubelet[32270]: (exit status 1))
Nov 10 10:11:22 k8s-m1 kubelet[32270]: E1110 10:11:22.165417   32270 pod_workers.go:190] Error syncing pod 63eee933-933a-11e9-928a-fefcfe274f71 ("traefik-ingress-controller-jjss4_kube-system(63eee933-933a-11e9-928a-fefcfe274f71)"), skipping: failed to "CreatePodSandbox" for "traefik-ingress-controller-jjss4_kube-system(63eee933-933a-11e9-928a-fefcfe274f71)" with CreatePodSandboxError: "CreatePodSandbox for pod \"traefik-ingress-controller-jjss4_kube-system(63eee933-933a-11e9-928a-fefcfe274f71)\" failed: rpc error: code = Unknown desc = failed to start sandbox container for pod \"traefik-ingress-controller-jjss4\": Error response from daemon: driver failed programming external connectivity on endpoint k8s_POD_traefik-ingress-controller-jjss4_kube-system_63eee933-933a-11e9-928a-fefcfe274f71_17529857 (ec7dbca09838629f1e4825175f4be3819723cb1984c9b7d00c2ed499b834fa5a):  (iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 0/0 --dport 8080 -j DNAT --to-destination 172.18.88.14:8080 ! -i docker0: iptables: No chain/target/match by that name.\n (exit status 1))"
Nov 10 10:11:22 k8s-m1 kubelet[32270]: I1110 10:11:22.165608   32270 server.go:459] Event(v1.ObjectReference{Kind:"Pod", Namespace:"kube-system", Name:"traefik-ingress-controller-jjss4", UID:"63eee933-933a-11e9-928a-fefcfe274f71", APIVersion:"v1", ResourceVersion:"13081364", FieldPath:""}): type: 'Warning' reason: 'FailedCreatePodSandBox' Failed create pod sandbox: rpc error: code = Unknown desc = failed to start sandbox container for pod "traefik-ingress-controller-jjss4": Error response from daemon: driver failed programming external connectivity on endpoint k8s_POD_traefik-ingress-controller-jjss4_kube-system_63eee933-933a-11e9-928a-fefcfe274f71_17529857 (ec7dbca09838629f1e4825175f4be3819723cb1984c9b7d00c2ed499b834fa5a):  (iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 0/0 --dport 8080 -j DNAT --to-destination 172.18.88.14:8080 ! -i docker0: iptables: No chain/target/match by that name.

可以看到controller是有问题的,使用命令查看一下k8s的组件

kubectl get cs
 NAME                 STATUS      MESSAGE                                                                                                                                  ERROR
  
  scheduler            Healthy     ok
  
  controller-manager   Unhealthy   Get http://127.0.0.1:10252/healthz: net/http: HTTP/1.x transport connection broken: malformed HTTP response "\x15\x03\x01\x00\x02\x02"
  
  etcd-1               Healthy     {"health":"true"}
  
  etcd-0               Healthy     {"health":"true"}
  
  etcd-2               Healthy     {"health":"true"}

确实是controller-manager问题
查看状态是否有报错信息,根据实际情况,我的直接重启一下就好了

systemctl status kube-controller-manager -l

systemctl restart kube-controller-manager 

测试:

组件状态:
[root@k8s-m1 script]# kubectl get cs
NAME                 STATUS    MESSAGE             ERROR
controller-manager   Healthy   ok                  
scheduler            Healthy   ok                  
etcd-0               Healthy   {"health":"true"}   
etcd-1               Healthy   {"health":"true"}   
etcd-2               Healthy   {"health":"true"}   
更新镜像
kubectl set image deployment/em-api em-api=192.168.90.10/zhufuc/em-api:v1.0-20201110100058
pod状态:
em-api-7c7f76dcdc-kdr5c   0/1     ContainerCreating   0          0s    <none>        k8s-n5   <none>           <none>
em-api-86855df489-hmvnr   1/1     Running             0          16m   172.18.94.8   k8s-n5   <none>           <none>

成功解决!


k8s容器间无法通讯 网络出现问题

node01节点无法ping其他节点容器的IP,同样node01节点的容器也ping不通其他节点容器IP

示例:其他节点的容器去进行ping

node01无法通讯

在这里插入图片描述
用在node05的容器去ping在node02的容器IP和node01的容器IP
在这里插入图片描述

查看一下k8s的组件插件

kubectl get pods --all-namespaces

可以看到有的服务出现异常可以查看详情

kubectl describe pod -n kube-system  [服务名字]

在这里插入图片描述
把服务异常的pod都进行重启

kubectl delete pod -n kube-system kube-proxy-2mzcp

查看状态
在这里插入图片描述

测试

node01去ping其他容器的ip
在这里插入图片描述
成功


版权声明:本文内容由互联网用户自发贡献,该文观点仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 举报,一经查实,本站将立刻删除。

发布者:全栈程序员-用户IM,转载请注明出处:https://javaforall.cn/101938.html原文链接:https://javaforall.cn

【正版授权,激活自己账号】: Jetbrains全家桶Ide使用,1年售后保障,每天仅需1毛

【官方授权 正版激活】: 官方授权 正版激活 支持Jetbrains家族下所有IDE 使用个人JB账号...

(0)


相关推荐

  • 菲尼克斯PSR-SCP- 24DC/FSP2/2X1/1X2耦合继电器

    菲尼克斯PSR-SCP- 24DC/FSP2/2X1/1X2耦合继电器菲尼克斯PSR-SCP-24DC/FSP2/2X1/1X2耦合继电器耦合继电器-PSR-SCP-24DC/FSP2/2X1/1X22986575适用于SIL2高需求和低需求应用的安全耦合继电器,将数字输出信号耦合至I/O端,2个启动电流通路,1个报警触点,用于安全状态关闭应用的模块,内置测试脉冲滤波器,插拔式螺钉连接端子,宽度:17.5mm产品类型 耦合继电器应用 安全关闭高要求低要求机械寿命 10×106开关次数继电器型号 带机械联锁触点的机电式继电器,符合IEC/EN6

  • 通过sql调用procedure_oracle存储过程简单案例

    通过sql调用procedure_oracle存储过程简单案例文章目录1.存储过程和函数在实际项目中的使用2.存储过程与函数的比较2.1.共同点2.2.不同点3.存储过程StoredProcedure3.1.存储过程概述3.1.1.存储过程的优点3.1.2.存储过程的缺点3.2.创建存储过程createprocedure3.3.调用存储过程call3.4.查看存储过程的定义3.5.修改存储过程alterprocedure…

  • HDU 3047 Zjnu Stadium 带权并查集[通俗易懂]

    HDU 3047 Zjnu Stadium 带权并查集

  • linux线程同步有几种方法_shell多线程执行命令

    linux线程同步有几种方法_shell多线程执行命令Linux中的线程同步机制(一)–Futex引子在编译2.6内核的时候,你会在编译选项中看到[*]Enablefutexsupport这一项,上网查,有的资料会告诉你”不选这个内核不一定能正确的运行使用glibc的程序”,那futex是什么?和glibc又有什么关系呢?1.什么是FutexFutex是FastUserspacemuTexes的缩写

  • java string转inputstream(substring截取字符串)

    Apachecommons是一个强大的Java辅助工具包。它提供的IOUtils可以让我们很便捷的实现InputStream转换为String。StringWriterwriter=newStringWriter();IOUtils.copy(inputStream,writer,encoding);StringtheString=writer.toString();首先把in…

  • html页面缩小导航栏隐藏,html – 导航栏缩放问题[通俗易懂]

    html页面缩小导航栏隐藏,html – 导航栏缩放问题[通俗易懂]我有一个问题,我的导航栏似乎与CSS中的.container缩放.现在,我是一个新手,但我已经尝试搞乱CSS中的值,但无济于事.这是HTML和CSS的代码:*{margin:0px;padding:0px;}body{font-family:verdana;background-image:url(images/bg2.jpg);max-width:100%;max-height:…

发表回复

您的电子邮箱地址不会被公开。

关注全栈程序员社区公众号