How to Cordon Kubernetes Nodes with kubectl

Introduction

This tutorial will guide you through the process of cordoning Kubernetes nodes using the kubectl command-line tool. Cordoning a node is a useful technique to prevent new pods from being scheduled on a specific node, which is helpful for maintenance tasks or when a node is experiencing issues.

Skills Graph

Kubernetes Nodes Basics

What are Kubernetes Nodes?

Kubernetes Nodes are the worker machines in a Kubernetes cluster. They can be either virtual or physical machines, and they are responsible for running the containerized applications. Each Node is managed by the Kubernetes control plane, and it can have multiple Pods running on it.

Node Components

A Kubernetes Node consists of the following components:

• Kubelet: The Kubelet is the primary "node agent" that runs on each Node. It is responsible for registering the Node with the Kubernetes API server, and for managing the Pods and their containers, volumes, and network interfaces.
• Container Runtime: The container runtime is the software that is responsible for running the containers on the Node. The most common container runtime used with Kubernetes is Docker, but other runtimes like containerd and CRI-O are also supported.
• Kube-proxy: The Kube-proxy is a network proxy that runs on each Node and is responsible for handling network traffic for the Pods running on that Node.

Node Status

The status of a Kubernetes Node can be one of the following:

• Ready: The Node is healthy and ready to accept Pods.
• NotReady: The Node is not healthy and cannot accept Pods.
• Unknown: The Kubernetes control plane cannot communicate with the Node.

Node Scheduling

Kubernetes uses a scheduler to determine which Node to place a Pod on. The scheduler takes into account factors such as resource availability, affinity and anti-affinity rules, and taints and tolerations.

Cordoning Nodes in Kubernetes

What is Cordoning?

Cordoning a Node in Kubernetes is the process of marking a Node as unschedulable, which means that new Pods will not be placed on that Node. This is useful when you need to perform maintenance on a Node, such as upgrading the operating system or hardware.

Why Cordon Nodes?

There are several reasons why you might want to cordon a Node in a Kubernetes cluster:

1. Maintenance: When you need to perform maintenance on a Node, such as upgrading the operating system or hardware, you can cordon the Node to prevent new Pods from being scheduled on it.
2. Draining Nodes: Before you can safely delete a Node, you need to drain it, which means that you need to move all the Pods running on that Node to other Nodes. Cordoning the Node is the first step in the draining process.
3. Load Balancing: You can cordon Nodes to temporarily take them out of the load balancing rotation, for example, to perform a rolling upgrade of your application.

Cordoning Nodes with kubectl

To cordon a Node in Kubernetes, you can use the kubectl cordon command. For example, to cordon a Node named node1, you can run the following command:

kubectl cordon node1

This will mark the Node as unschedulable, and new Pods will not be placed on it.

To verify that the Node has been cordoned, you can run the following command:

kubectl get nodes

The output will show the node1 Node with the SchedulingDisabled condition.

To uncordon a Node and make it schedulable again, you can use the kubectl uncordon command:

kubectl uncordon node1

This will remove the SchedulingDisabled condition from the Node, and it will be able to accept new Pods again.

Cordoning Nodes with kubectl

Cordoning a Node

To cordon a Node in Kubernetes, you can use the kubectl cordon command. For example, to cordon a Node named node1, you can run the following command:

kubectl cordon node1

This will mark the Node as unschedulable, and new Pods will not be placed on it.

You can also cordon multiple Nodes at once by specifying multiple Node names:

kubectl cordon node1 node2 node3

Verifying Node Cordoning

To verify that a Node has been cordoned, you can use the kubectl get nodes command. The output will show the Node with the SchedulingDisabled condition:

NAME     STATUS                     ROLES    AGE   VERSION
node1    Ready,SchedulingDisabled   <none>   1d    v1.20.0
node2    Ready                      <none>   1d    v1.20.0
node3    Ready,SchedulingDisabled   <none>   1d    v1.20.0

You can also use the kubectl describe node command to get more detailed information about a Node's status:

kubectl describe node node1

This will show the SchedulingDisabled condition in the Node's status.

Uncordoning a Node

To uncordon a Node and make it schedulable again, you can use the kubectl uncordon command:

kubectl uncordon node1

This will remove the SchedulingDisabled condition from the Node, and it will be able to accept new Pods again.

As with cordoning, you can also uncordon multiple Nodes at once:

kubectl uncordon node1 node2 node3

Cordoning and Draining Nodes

Cordoning a Node is often the first step in the process of draining a Node. Draining a Node means safely evicting all the Pods running on that Node, so that the Node can be safely decommissioned or upgraded.

To drain a Node, you can use the kubectl drain command:

kubectl drain node1

This will cordon the Node and then evict all the Pods running on it, moving them to other Nodes in the cluster.

Summary

In this tutorial, you have learned how to cordon Kubernetes nodes using the kubectl cordon command. By cordoning nodes, you can protect your cluster by preventing new pods from being scheduled on the affected nodes, which is useful for maintenance tasks or when a node is experiencing issues. Remember, cordoning a node does not automatically drain the existing pods, so you may need to take additional steps to safely move the pods to other nodes.