How to Enforce Resource Limits with Kubernetes Quotas

Introduction

This tutorial provides a comprehensive understanding of Kubernetes Resource Quotas, a powerful feature that allows you to set limits on the amount of resources a namespace can consume. This is particularly useful in multi-tenant Kubernetes clusters, where you want to ensure that one namespace doesn't monopolize all the available resources, leaving none for other namespaces. The tutorial covers the concepts of Resource Quotas, how to configure them, and how to apply them in practice.

Skills Graph

Understanding Kubernetes Resource Quotas

Kubernetes Resource Quotas are a powerful feature that allow you to set limits on the amount of resources a namespace can consume. This is particularly useful in multi-tenant Kubernetes clusters, where you want to ensure that one namespace doesn't monopolize all the available resources, leaving none for other namespaces.

Resource Quotas can be used to limit the total amount of compute resources (CPU, memory) that can be requested and consumed within a namespace. They can also be used to limit the number of objects (Pods, Services, Secrets, etc.) that can be created in a namespace.

For example, you can set a Resource Quota in a namespace that limits the total CPU request to 2 cores and the total memory request to 4 GB. You can also limit the number of Pods that can be created in the namespace to 100.

apiVersion: v1
kind: ResourceQuota
metadata:
  name: compute-resources
  namespace: default
spec:
  hard:
    requests.cpu: "2"
    requests.memory: 4Gi
    limits.cpu: "4" 
    limits.memory: 8Gi
    pods: "100"

By applying this Resource Quota, Kubernetes will ensure that the total CPU and memory requests in the default namespace do not exceed the specified limits, and the number of Pods created does not exceed 100.

Resource Quotas can be a valuable tool for managing resources in a Kubernetes cluster, ensuring fair allocation and preventing resource hogging by individual namespaces.

Configuring Resource Quotas

Configuring Resource Quotas in Kubernetes involves creating a ResourceQuota object and applying it to a namespace. The ResourceQuota object defines the limits and constraints that will be enforced on the resources consumed within the namespace.

Here's an example of a ResourceQuota manifest that limits the CPU, memory, and object count in a namespace:

apiVersion: v1
kind: ResourceQuota
metadata:
  name: compute-resources
  namespace: my-namespace
spec:
  hard:
    requests.cpu: "2" 
    requests.memory: 4Gi
    limits.cpu: "4"
    limits.memory: 8Gi
    pods: "100"
    services: "50"
    secrets: "100"
    configmaps: "50"

In this example, the ResourceQuota object named compute-resources is applied to the my-namespace namespace. It sets the following limits:

• CPU requests are limited to 2 cores
• Memory requests are limited to 4 GiB
• CPU limits are set to 4 cores
• Memory limits are set to 8 GiB
• The number of Pods is limited to 100
• The number of Services is limited to 50
• The number of Secrets is limited to 100
• The number of ConfigMaps is limited to 50

You can also configure Resource Quotas for specific resource types, such as storage:

apiVersion: v1
kind: ResourceQuota
metadata:
  name: storage-resources
  namespace: my-namespace
spec:
  hard:
    requests.storage: 500Gi
    persistentvolumeclaims: "10"
    configmaps: "10"
    secrets: "10"

This ResourceQuota object limits the total storage requests to 500 GiB and the number of PersistentVolumeClaims, ConfigMaps, and Secrets to 10 each.

Once the ResourceQuota object is created, Kubernetes will automatically enforce the specified limits within the target namespace.

Applying Resource Quotas in Practice

Applying Resource Quotas in a Kubernetes cluster can help you achieve several important goals, such as fair distribution of resources, cost control, and improved cluster performance.

One common use case for Resource Quotas is to ensure that no single namespace can monopolize all the available resources in the cluster. By setting appropriate limits on CPU, memory, and object counts, you can prevent a resource-intensive workload in one namespace from impacting the performance of other namespaces.

Another scenario where Resource Quotas are useful is cost control. By setting limits on the total resources that can be consumed in a namespace, you can ensure that the cost of running workloads in that namespace stays within your budget.

apiVersion: v1
kind: LimitRange
metadata:
  name: limit-range
  namespace: my-namespace
spec:
  limits:
  - default:
      cpu: 500m
      memory: 256Mi
    defaultRequest:
      cpu: 250m
      memory: 128Mi
    type: Container

In this example, the LimitRange object sets default CPU and memory requests and limits for containers in the my-namespace namespace. This can be used in conjunction with a Resource Quota to control the overall resource consumption in the namespace.

By applying Resource Quotas strategically, you can optimize the performance and cost-effectiveness of your Kubernetes cluster, ensuring that resources are allocated fairly and efficiently across all the namespaces.

Summary

Kubernetes Resource Quotas are a valuable tool for managing resources in a Kubernetes cluster, ensuring fair allocation and preventing resource hogging by individual namespaces. By configuring Resource Quotas, you can limit the total amount of compute resources (CPU, memory) that can be requested and consumed within a namespace, as well as the number of objects (Pods, Services, Secrets, etc.) that can be created. This tutorial has provided a comprehensive understanding of Resource Quotas and how to apply them in practice, helping you effectively manage resources in your multi-tenant Kubernetes environment.