CKA 13: Troubleshooting
* Check the events at pod
kubectl describe pods ${POD_NAME}
* Pod in pending state
- no enough CPU/memory resource on node
- taint and tolerance
- hostport is occupied, if pod is using host network.
* Pod in waiting state
- pod is assigned to node
- pod is not able to pull image. Check image spelling. run command
docker pull IMAGE
* To validate YAML syntax
kubectl apply --validate -f mypod.yaml
Note: all spelling mistakes will be ignore, without validate
* To debug service,
- verify endpoints
kubectl get endpoints ${SERVICE_NAME}
- Verify that the pod's containerPort matches up with the Service's targetPort
- verify DNS entry with nslookup command
- If nslookup for short name fails then check /etc/resolv.conf "search" parameter
//kubelet is invoked with --cluster-dns (set DNS server) and --cluster-domain (set default : cluster-local)
- port value should be numeric, not string
- Is kube-proxy running?
- Is conntrack installed?
kubeadm tool deploy all k8s components as pod
They can be deployed as services too
Master: kube-apiserver, kube-controller-manager, kube-scheduler
Worker: kubelet, kube-proxy
with kubeadm tool logs can be checked with "k logs POD_NAME" command. With service logs can be checked with "journalctl -u SERVICE_NAME" command.
For static pod, its name is suffix with node name.
static pod path
1. /etc/systemd/system/kubelet.service.d/10-kubeadm.conf file has env variable KUBELET_CONFOG_ARGS that point to kubelet config file.
2. kublet config file is at /var/lib/kubelet/config.yaml
3. In this file, we have statisPodPath
4. its default value is generally /etc/kubernetes/manifest
* Node failure
- check cpu with top
- check hdd with df -h
- kubelet status with service status kubelet -l
- kubelet logs with journalctl -u kubelet
- check kubelet certificate with
openssl x509 -in /var/lib/kubelet/worker-1.crt -text
Restart kubelet
1. systemctl deamon-reload
2. systemctl restart kubelet
kubectl describe pods ${POD_NAME}
* Pod in pending state
- no enough CPU/memory resource on node
- taint and tolerance
- hostport is occupied, if pod is using host network.
* Pod in waiting state
- pod is assigned to node
- pod is not able to pull image. Check image spelling. run command
docker pull IMAGE
* To validate YAML syntax
kubectl apply --validate -f mypod.yaml
Note: all spelling mistakes will be ignore, without validate
* To debug service,
- verify endpoints
kubectl get endpoints ${SERVICE_NAME}
- Verify that the pod's containerPort matches up with the Service's targetPort
- verify DNS entry with nslookup command
- If nslookup for short name fails then check /etc/resolv.conf "search" parameter
//kubelet is invoked with --cluster-dns (set DNS server) and --cluster-domain (set default : cluster-local)
- port value should be numeric, not string
- Is kube-proxy running?
- Is conntrack installed?
kubeadm tool deploy all k8s components as pod
They can be deployed as services too
Master: kube-apiserver, kube-controller-manager, kube-scheduler
Worker: kubelet, kube-proxy
with kubeadm tool logs can be checked with "k logs POD_NAME" command. With service logs can be checked with "journalctl -u SERVICE_NAME" command.
For static pod, its name is suffix with node name.
static pod path
1. /etc/systemd/system/kubelet.service.d/10-kubeadm.conf file has env variable KUBELET_CONFOG_ARGS that point to kubelet config file.
2. kublet config file is at /var/lib/kubelet/config.yaml
3. In this file, we have statisPodPath
4. its default value is generally /etc/kubernetes/manifest
* Node failure
- check cpu with top
- check hdd with df -h
- kubelet status with service status kubelet -l
- kubelet logs with journalctl -u kubelet
- check kubelet certificate with
openssl x509 -in /var/lib/kubelet/worker-1.crt -text
Restart kubelet
1. systemctl deamon-reload
2. systemctl restart kubelet
0 comments:
Post a Comment