1 Pod 的状态和探针

官方文档

https://kubernetes.io/zh/docs/concepts/workloads/pods/pod-lifecycle/

1.1 Pod 状态

第一阶段:

  • Pending:正在创建 Pod 但是 Pod 中的容器还没有全部被创建完成,处于此状态的 Pod 应该检查 Pod 依赖的存储是否有权限挂载、镜像是否可以下载、调度是否正常等。
  • Failed:Pod 中有容器启动失败而导致 pod 工作异常。
  • Unknown:由于某种原因无法获得 pod 的当前状态,通常是由于与 pod 所在的 node 节点通信错误。
  • Succeeded:Pod 中的所有容器都被成功终止即 pod 里所有的 containers 均已 terminated。

第二阶段:

  • Unschedulable:Pod不能被调度,kube-scheduler没有匹配到合适的node节点
    • CPU资源不够,内存资源不够
    • 打labels标签
  • PodScheduled:pod 正处于调度中,在 kube-scheduler 刚开始调度的时候,还没有将 pod 分配到指定node,在筛选出合适的节点后就会更新 etcd 数据,将 pod 分配到指定的 node
  • Initialized:所有 pod 中的初始化容器已经完成了
  • ImagePullBackOff:Pod 所在的 node 节点下载镜像失败
    • node 节点无法下载镜像
    • 网络问题
    • 权限问题
    • 镜像地址或者名称写错
  • Running:Pod 内部的容器已经被创建并且启动
  • Ready:表示 pod 中的容器已经可以提供访问服务

1.2 Pod 调度过程

k8s 实战案例中

1.3 Pod 探针

官网文档:

https://kubernetes.io/zh/docs/concepts/workloads/pods/pod-lifecycle/#%E5%AE%B9%E5%99%A8%E6%8E%A2%E9%92%88
https://kubernetes.io/zh/docs/concepts/workloads/pods/pod-lifecycle/#container-probes

1.3.1 探针简介

探针是由 kubelet 对容器执行的定期诊断,以保证 Pod 的状态始终处于运行状态,要执行诊断,kubelet 调用由容器实现的 Handler,有三种类型的处理程序:

  • ExecAction: 在容器内执行指定命令。如果命令退出时返回码为 0 则认为诊断成功。
  • TCPSocketAction: 对容器的 IP 地址上的指定端口执行 TCP 检查。如果端口打开,则诊断被认为是成功的。
  • HTTPGetAction: 对容器的 IP 地址上指定端口和路径执行 HTTP Get 请求。如果响应的状态码大于等于 200 且小于 400,则诊断被认为是成功的。

每次探测都将获得以下三种结果之一:

  • Success(成功):容器通过了诊断。
  • Failure(失败):容器未通过诊断。
  • Unknown(未知):诊断失败,因此不会采取任何行动。

1.3.2 配置探针

基于探针实现对 Pod 的状态检测

1.3.2.1 探针类型

  • livenessProbe:存活探针,检测容器是否正在运行,如果存活探测失败,则 kubelet 会杀死容器,并且容器将受到其重启策略的影响,如果容器不提供存活探针,则默认状态为 Success,livenessProbe 用户控制是否重启 pod。
  • readinessProbe:就绪探针,如果就绪探测失败,端点控制器将从与 Pod 匹配的所有 Service 的端点中删除该 Pod 的 IP 地址,初始延迟之前的就绪状态默认为 Failure,如果容器不提供就绪探针,则默认状态为 Success,readinessProbe 用于控制 pod 是否添加至 service。

1.3.2.2 探针配置

官方文档:

https://kubernetes.io/zh/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#configure-probes

探针有很多配置字段,可以使用这些字段精确的控制存活和就绪检测的行为:

  • initialDelaySeconds: 120
    初始化延迟时间,告诉 kubelet 在执行第一次探测前应该等待多少秒,默认是0秒,最小值是0
  • periodSeconds: 60
    探测周期间隔时间,指定了 kubelet 应该每多少秒秒执行一次存活探测,默认是 10 秒。最小值是 1
  • timeoutSeconds: 5
    单次探测超时时间,探测的超时后等待多少秒,默认值是1秒,最小值是1。
  • successThreshold: 1
    从失败转为成功的重试次数,探测器在失败后,被视为成功的最小连续成功数,默认值是1,存活探测的这个值必须是 1,最小值是 1。
  • failureThreshold: 3
    从成功转为失败的重试次数,当 Pod 启动了并且探测到失败,Kubernetes 的重试次数,存活探测情况下的放弃就意味着重新启动容器,就绪探测情况下的放弃 Pod 会被打上未就绪的标签,默认值是3,最小值是1。

HTTP 探测器可以在 httpGet 上配置额外的字段:

  • host:连接使用的主机名,默认是 Pod 的 IP。也可以在 HTTP 头中设置 “Host” 来代替。
  • scheme :用于设置连接主机的方式(HTTP 还是 HTTPS)。默认是 HTTP。
  • path:访问 HTTP 服务的路径。默认值为 "/"。
  • httpHeaders:请求中自定义的 HTTP 头。HTTP 头字段允许重复。
  • port:访问容器的端口号或者端口名。如果数字必须在 1 ~ 65535 之间。

1.3.2.3 HTTP 探针示例

[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# pwd
/data/kubeadm-waluna/nginx-yml
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# vim nginx-http.yml
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# cat nginx-http.yml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: harbor.waluna.top/baseimages/nginx:1.18
        ports:
        - containerPort: 80
        #readinessProbe:
        livenessProbe:
          httpGet:
            #path: /monitor/monitor.html
            path: /index.html
            port: 80
          initialDelaySeconds: 5
          periodSeconds: 3
          timeoutSeconds: 5
          successThreshold: 1
          failureThreshold: 3

---
kind: Service
apiVersion: v1
metadata:
  labels:
    app: waluna-nginx-service-label
  name: waluna-nginx-service
  namespace: default
spec:
  type: NodePort
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
    nodePort: 30004
  selector:
    app: nginx
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# 

创建pod进行测试

# 创建pod
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# kubectl apply -f nginx-http.yml
deployment.apps/nginx-deployment created
service/waluna-nginx-service created
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# 

# 查看pod
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# kubectl get pod
NAME                                 READY   STATUS    RESTARTS   AGE
nginx-deployment-6889cb644-css2w     1/1     Running   0          23s
tomcat-deployment-6f6bc44c85-2fbh2   1/1     Running   0          2d18h
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# 

# 访问页面
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# curl 10.0.0.69:30004
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# 

# 删除pod
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# kubectl delete -f nginx-http.yml
deployment.apps "nginx-deployment" deleted
service "waluna-nginx-service" deleted
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# 

# 修改yaml文件,将检测文件故意改为不存在的页面
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# vim nginx-http.yml
......
          httpGet:
            path: /monitor/monitor.html
            #path: /index.html
            port: 80
......

# 重新创建pod
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# kubectl apply -f nginx-http.yml
deployment.apps/nginx-deployment created
service/waluna-nginx-service created
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# 

# 查看pod
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# kubectl get pod
NAME                                 READY   STATUS    RESTARTS   AGE
nginx-deployment-5555767bcf-m97hp    1/1     Running   3          47s
tomcat-deployment-6f6bc44c85-2fbh2   1/1     Running   0          2d18h
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# kubectl get pod
NAME                                 READY   STATUS             RESTARTS   AGE
nginx-deployment-5555767bcf-m97hp    0/1     CrashLoopBackOff   3          51s
tomcat-deployment-6f6bc44c85-2fbh2   1/1     Running            0          2d18h
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# 

# 测试网页
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# curl 10.0.0.69:30004
curl: (7) Failed to connect to 10.0.0.69 port 30004: Connection refused
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# 

1.3.2.4 TCP 探针示例

[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# vim nginx-tcp.yml
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# cat nginx-tcp.yml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: harbor.waluna.top/baseimages/nginx:1.18
        ports:
        - containerPort: 80
        #readinessProbe:
        livenessProbe:
          tcpSocket:
            port: 80
            #port: 8080
          initialDelaySeconds: 5
          periodSeconds: 3
          timeoutSeconds: 5
          successThreshold: 1
          failureThreshold: 3

---
kind: Service
apiVersion: v1
metadata:
  labels:
    app: waluna-nginx-service-label
  name: waluna-nginx-service
  namespace: default
spec:
  type: NodePort
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
    nodePort: 30004
  selector:
    app: nginx
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# 

创建pod进行测试

# 删除前面的pod
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# kubectl apply -f nginx-tcp.yml
deployment.apps/nginx-deployment created
service/waluna-nginx-service created
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# 

# 查看pod
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# kubectl get pod -o wide
NAME                                 READY   STATUS    RESTARTS   AGE     IP           NODE               NOMINATED NODE   READINESS GATES
nginx-deployment-58b5899d87-f764z    1/1     Running   0          22s     10.10.3.28   node1.waluna.top   <none>           <none>
tomcat-deployment-6f6bc44c85-2fbh2   1/1     Running   0          2d18h   10.10.5.12   node3.waluna.top   <none>           <none>
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# 

# 访问网页
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# curl 10.0.0.69:30004
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# 

# 删除pod
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# kubectl delete -f nginx-tcp.yml
deployment.apps "nginx-deployment" deleted
service "waluna-nginx-service" deleted
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# 

# 修改yaml文件
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# vim nginx-tcp.yml
......
          tcpSocket:
            #port: 80
            port: 8080
......

# 重新创建pod
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# kubectl apply -f nginx-tcp.yml
deployment.apps/nginx-deployment created
service/waluna-nginx-service created
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# 

# 查看pod
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# kubectl get pod -o wide
NAME                                 READY   STATUS    RESTARTS   AGE     IP           NODE               NOMINATED NODE   READINESS GATES
nginx-deployment-76b8d59975-7mdln    1/1     Running   3          52s     10.10.3.29   node1.waluna.top   <none>           <none>
tomcat-deployment-6f6bc44c85-2fbh2   1/1     Running   0          2d18h   10.10.5.12   node3.waluna.top   <none>           <none>
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# kubectl get pod -o wide
NAME                                 READY   STATUS             RESTARTS   AGE     IP           NODE               NOMINATED NODE   READINESS GATES
nginx-deployment-76b8d59975-7mdln    0/1     CrashLoopBackOff   3          52s     10.10.3.29   node1.waluna.top   <none>           <none>
tomcat-deployment-6f6bc44c85-2fbh2   1/1     Running            0          2d18h   10.10.5.12   node3.waluna.top   <none>           <none>
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# 

# 再次验证网页
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# curl 10.0.0.69:30004
curl: (7) Failed to connect to 10.0.0.69 port 30004: Connection refused
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# 

1.3.2.5 ExecAction 探针示例

可以基于指定的命令对 Pod 进行特定的状态检查。

# 准备镜像
[root@k8s-master1 ~]# docker pull redis:6.0.16
6.0.16: Pulling from library/redis
7d63c13d9b9b: Pull complete 
a2c3b174c5ad: Pull complete 
283a10257b0f: Pull complete 
6c186e04f421: Pull complete 
72348cc8dd1a: Pull complete 
b50866ac277e: Pull complete 
Digest: sha256:50f0934be24ea7916d213cfa90851a558abeb8037f5bd0edb09b35db03c6da94
Status: Downloaded newer image for redis:6.0.16
docker.io/library/redis:6.0.16
[root@k8s-master1 ~]# 

# 打标签并上传镜像
[root@k8s-master1 ~]# docker tag redis:6.0.16 harbor.waluna.top/baseimages/redis:6.0.16
[root@k8s-master1 ~]# docker push harbor.waluna.top/baseimages/redis:6.0.16
The push refers to repository [harbor.waluna.top/baseimages/redis]
5d5805347e15: Pushed 
a4bdf6125258: Pushed 
a61797031d5d: Pushed 
8b9770153666: Pushed 
b43651130521: Pushed 
e8b689711f21: Pushed 
6.0.16: digest: sha256:5ad7c146f7e071335989592dd0e739c7201b64613ae45d557cebfed626c83d88 size: 1573
[root@k8s-master1 ~]# 

# 准备yaml文件
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# vim redis-ExecAction.yml
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# cat redis-ExecAction.yml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis-deployment
  labels:
    app: redis
spec:
  replicas: 1
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
      - name: redis
        image: harbor.waluna.top/baseimages/redis:6.0.16
        ports:
        - containerPort: 6379
        #readinessProbe:
        livenessProbe:
          exec:
            command:
            #- /apps/redis/bin/redis-cli
            - /usr/local/bin/redis-cli
            - quit
          initialDelaySeconds: 5
          periodSeconds: 3
          timeoutSeconds: 5
          successThreshold: 1
          failureThreshold: 3

---
kind: Service
apiVersion: v1
metadata:
  labels:
    app: waluna-redis-service-label
  name: waluna-redis-service
  namespace: default
spec:
  type: NodePort
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 6379
    nodePort: 30006
  selector:
    app: redis
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]#  

创建pod进行测试

# 删除前面的实验
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# kubectl delete -f nginx-tcp.yml
deployment.apps "nginx-deployment" deleted
service "waluna-nginx-service" deleted
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# 

# 创建pod
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# kubectl apply -f redis-ExecAction.yml
deployment.apps/redis-deployment created
service/waluna-redis-service created
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# 

# 查看pod
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# kubectl get pod -o wide
NAME                                 READY   STATUS    RESTARTS   AGE     IP           NODE               NOMINATED NODE   READINESS GATES
redis-deployment-58c7d54b4f-x6t65    1/1     Running   0          2m29s   10.10.3.30   node1.waluna.top   <none>           <none>
tomcat-deployment-6f6bc44c85-2fbh2   1/1     Running   0          2d21h   10.10.5.12   node3.waluna.top   <none>           <none>
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# 

# 删除pod
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# kubectl delete -f redis-ExecAction.yml
deployment.apps "redis-deployment" deleted
service "waluna-redis-service" deleted
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# 

# 修改yaml文件
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# vim redis-ExecAction.yml
......
          exec:
            command:
            - /apps/redis/bin/redis-cli
            #- /usr/local/bin/redis-cli
            - quit
......

# 重新创建pod
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# kubectl apply -f redis-ExecAction.yml
deployment.apps/redis-deployment created
service/waluna-redis-service created
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# 

# 查看pod
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# kubectl get pod
NAME                                 READY   STATUS    RESTARTS   AGE
redis-deployment-7ccc47597f-zl8fl    1/1     Running   3          48s
tomcat-deployment-6f6bc44c85-2fbh2   1/1     Running   0          2d22h
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]# kubectl get pod
NAME                                 READY   STATUS             RESTARTS   AGE
redis-deployment-7ccc47597f-zl8fl    0/1     CrashLoopBackOff   3          49s
tomcat-deployment-6f6bc44c85-2fbh2   1/1     Running            0          2d22h
[root@k8s-master1 /data/kubeadm-waluna/nginx-yml]#