Simulate and Diagnose Update Failures
In this step, you'll learn how to diagnose potential deployment update failures by simulating an problematic image update and using Kubernetes diagnostic tools.
First, navigate to the project directory:
cd ~/project/k8s-manifests
Create a deployment manifest with an invalid image:
nano problematic-deployment.yaml
Add the following content:
apiVersion: apps/v1
kind: Deployment
metadata:
name: troubleshoot-app
labels:
app: troubleshoot
spec:
replicas: 3
selector:
matchLabels:
app: troubleshoot
template:
metadata:
labels:
app: troubleshoot
spec:
containers:
- name: nginx
image: nginx:non-existent-tag
ports:
- containerPort: 80
Apply the problematic deployment:
kubectl apply -f problematic-deployment.yaml
Example output:
deployment.apps/troubleshoot-app created
Check the deployment status:
kubectl rollout status deployment troubleshoot-app
Example output:
Waiting for deployment "troubleshoot-app" to roll out...
Press Ctrl+C
to exit the rollout status.
Check the pod events and status:
kubectl get pods -l app=troubleshoot
Example output:
NAME READY STATUS RESTARTS AGE
troubleshoot-app-6b8986c555-gcjj9 0/1 ImagePullBackOff 0 2m56s
troubleshoot-app-6b8986c555-p29dp 0/1 ImagePullBackOff 0 2m56s
troubleshoot-app-6b8986c555-vpv5q 0/1 ImagePullBackOff 0 2m56s
Examine pod details and logs:
## Replace 'xxx-yyy' with your actual pod name
POD_NAME=$(kubectl get pods -l app=troubleshoot -o jsonpath='{.items[0].metadata.name}')
kubectl describe pod $POD_NAME
kubectl logs $POD_NAME
You will see the pod status and logs indicating the image pull failure.
Failed to pull image "nginx:non-existent-tag"
Troubleshoot the issue by correcting the image:
nano problematic-deployment.yaml
Update the image to a valid tag:
image: nginx:1.24.0-alpine
Reapply the corrected deployment:
kubectl apply -f problematic-deployment.yaml
Check the pod status again:
kubectl get pods -l app=troubleshoot
Example output:
NAME READY STATUS RESTARTS AGE
troubleshoot-app-5dc9b58d57-bvqbr 1/1 Running 0 5s
troubleshoot-app-5dc9b58d57-tdksb 1/1 Running 0 8s
troubleshoot-app-5dc9b58d57-xdq5n 1/1 Running 0 6s
Key points about diagnosing failures:
- Use
kubectl describe
to view deployment and pod events
- Check pod status for
ImagePullBackOff
or other error states
- Examine pod logs for detailed error information
- Verify image availability and tag correctness