How do I configure Kubernetes taints and tolerations for node scheduling?

Taints and tolerations let you control which pods can schedule on specific nodes in your cluster. Here’s how to configure them.

---

1. Understand Taints and Tolerations

• Taints are applied to nodes to mark them as unschedulable for certain pods.
• Tolerations are applied to pods to allow them to “tolerate” those taints and be scheduled on the tainted nodes.

---

2. Add a Taint to a Node

To taint a node, use the kubectl taint command.

Syntax:

bash
kubectl taint nodes <node-name> <key>=<value>:<effect>

• <key>: A label key for the taint.
• <value>: A label value for the taint.
• <effect>: Determines the behavior of the taint. Options:
• NoSchedule: Pods without matching tolerations will not be scheduled on the node.
• PreferNoSchedule: Kubernetes will try to avoid scheduling pods without matching tolerations on the node but won’t guarantee it.
• NoExecute: Pods without matching tolerations will be evicted if they are already running on the node.

Example:

bash
kubectl taint nodes node1 key1=value1:NoSchedule
This adds a taint to node1 with the key key1, value value1, and effect NoSchedule. Pods without a toleration for this taint won’t be scheduled on node1.

---

3. Add a Toleration to a Pod

To allow a pod to be scheduled on a node with a taint, add a toleration to the pod’s manifest.

Example Pod Manifest with Toleration:

yaml
apiVersion: v1
kind: Pod
metadata:
name: my-pod
spec:
containers:
- name: my-container
image: nginx
tolerations:
- key: "key1"
operator: "Equal"
value: "value1"
effect: "NoSchedule"

• key: Matches the taint key on the node.
• operator: Can be Equal (default) or Exists. If Exists, the value field is ignored.
• value: Matches the taint value on the node.
• effect: Must match the effect of the taint (NoSchedule, PreferNoSchedule, or NoExecute).

---

4. Remove a Taint from a Node

If you need to remove a taint from a node, use the kubectl taint command with a - at the end.

Example:

bash
kubectl taint nodes node1 key1=value1:NoSchedule-

This removes the taint key1=value1:NoSchedule from node1.

---

5. Taints and Tolerations with NoExecute

The NoExecute effect not only prevents scheduling but also evicts existing pods from the node unless they tolerate the taint.

Example:

yaml
tolerations:
- key: "key1"
operator: "Equal"
value: "value1"
effect: "NoExecute"
tolerationSeconds: 3600

• tolerationSeconds: Specifies how long the pod can remain on the node after the taint is added. After this time, the pod will be evicted.

---

6. Use Cases

• Dedicated Nodes: Assign specific workloads to specific nodes. For example, GPU workloads can be scheduled on GPU-enabled nodes.
• Node Isolation: Prevent certain workloads from running on specific nodes (e.g., critical nodes or nodes with limited resources).
• Eviction Management: Automatically evict pods from nodes when specific conditions are met (e.g., maintenance or resource pressure).

---

7. Verify Configuration

To check the taints applied to a node:
bash
kubectl describe node <node-name> | grep Taints

To check the tolerations of a pod:
bash
kubectl describe pod <pod-name> | grep Tolerations

---

Example Use Case: Taint GPU Nodes

Let’s say you have a node pool with GPU-enabled nodes, and you want to ensure only pods requiring GPU resources are scheduled on them.

Taint the GPU nodes:

bash
kubectl taint nodes gpu-node key=gpu:NoSchedule

Add a toleration to GPU workloads:

yaml
tolerations:
- key: "gpu"
operator: "Equal"
value: "gpu"
effect: "NoSchedule"