0/1 nodes available: insufficient cpu, insufficient memory

July 09, 2026 — LiveStream

Ever found your critical applications stuck in a Pending state, staring at a cryptic error message like "0/1 nodes available: insufficient cpu" or "0/1 nodes available: insufficient memory"? This isn't just an annoying glitch, it's a fundamental signal from your Kubernetes cluster that something is out of whack with your resource allocation. As a DevOps engineer, understanding and resolving these issues is paramount to maintaining a healthy and performant cluster. It's like your cluster is telling you, "Bhai, I need more juice, or maybe you're asking for too much!"

This common Kubernetes scheduling problem indicates that the scheduler couldn't find a node with enough available CPU or memory to host your pod. It's a clear sign that you need to dive deep into your resource requests and the actual capacity of your nodes. Let's grab some chai and figure out why your pods are playing hide-and-seek and how to get them running smoothly.

Decoding the "0/X Nodes Available: Insufficient CPU, Insufficient Memory" Error

When Kubernetes reports "0/1 nodes available: insufficient cpu" or "0/1 nodes available: insufficient memory", it's the cluster scheduler trying to tell you it failed to place your pod. The message typically indicates that none of the nodes in your cluster have enough allocatable resources (CPU or memory) to satisfy the requests made by your pod's containers. This isn't just about a single node; it means all potential nodes were considered and rejected for the given reason.

Let's break down the typical journey of a pod and where things go wrong here. When you create a pod, the Kubernetes scheduler swings into action. Its job is to find the best node for that pod to run on. It does this by going through a series of "predicates" (filters) and "priorities" (scoring). Predicates are like a checklist: "Does this node have enough resources? Is it tainted? Does it meet the pod's node affinity requirements?" If a node fails even one predicate, it's out. If all nodes fail, then your pod remains in a Pending state, and you see errors like the one we're discussing.

You might also see this accompanied by Events messages when you inspect your pod, similar to these:

Events:
  Type     Reason            Age                     From                Message
  ----     ------            ----                    ----                -------
  Warning  FailedScheduling  11m (x2550 over 2d19h)  default-scheduler   0/5 nodes are available: 2 node(s) had volume node affinity conflict, 3 Insufficient cpu.
  Warning  FailedScheduling  65s (x199 over 2d19h)   default-scheduler   0/5 nodes are available: 1 Insufficient memory, 1 node(s) had volume node affinity conflict, 4 Insufficient cpu.

These events provide even more context. They show you *how many* nodes were considered and *why* they were rejected. In the examples above, we see a mix of reasons: Insufficient cpu, Insufficient memory, and even volume node affinity conflict. While our primary focus is CPU and memory, it’s good to remember that scheduling failures can have multiple root causes. The x2550 over 2d19h indicates the scheduler has tried (and failed) to schedule this pod thousands of times over two days!

The immediate symptom is usually pods stuck in Pending:

kubectl get pods | grep Pending
prod-us-west-2-elasticsearch--1                       0/1     Pending   0          2d19h
prod-us-w-2-fluentd-24x                               0/1     Pending   0          2d19h
prod-us-eus-0                                         0/2     Pending   0          2d19h

This is your cue, my friend, to roll up your sleeves and troubleshoot. The core problem is usually a mismatch: either your pods are asking for too much, or your cluster simply doesn't have enough to give. Let's figure out which one it is.

Step 1: Unmasking Your Pod's Resource Demands (YAML Deep Dive)

The first step in resolving insufficient cpu or insufficient memory issues is to understand exactly what your pod is asking for. Remember, Kubernetes scheduler uses the resources.requests defined in your pod's YAML to make placement decisions. If you haven't explicitly set them, or if you've set them incorrectly, that's often where the problem lies.

Extracting Your Pod's YAML Configuration

To inspect a problematic pod, you need its YAML definition. This YAML will tell you everything about the pod's configuration, including its resource requests. You can fetch it using kubectl get:

kubectl get pod <your-pending-pod-name> -o yaml

Replace <your-pending-pod-name> with the actual name of your pod that's stuck in Pending. For example:

kubectl get pod prod-us-west-2-elasticsearch--1 -o yaml

Once you have the YAML, scroll down to the containers section. Inside each container definition, look for the resources field. This is where you specify requests and limits for CPU and memory.

Understanding Resource Requests and Limits

Requests: Think of requests as the minimum guaranteed resources your container needs to function. The Kubernetes scheduler uses these values to decide which node a pod can be placed on. If a node doesn't have enough allocatable resources to satisfy the total requests of all pods it's hosting, including your new one, the scheduler won't place the pod there. This is why "insufficient cpu" or "insufficient memory" occurs.

Limits: limits, on the other hand, define the maximum amount of resources a container is allowed to use. If a container tries to use more CPU than its limit, it will be throttled. If it tries to use more memory than its limit, the container will be terminated (an Out-Of-Memory or OOMKill event). While limits don't directly cause scheduling failures, they are crucial for preventing a single misbehaving application from hogging all resources and impacting other workloads on the same node.

Here’s an example of a simplified pod YAML with explicit resource requests:

apiVersion: v1
kind: Pod
metadata:
  name: my-resource-hungry-app
spec:
  containers:
    - name: app-container
      image: myrepo/my-app:latest
      command: ["sleep", "3600"]
      resources:
        requests:
          memory: "1Gi"  # Requesting 1 Gigabyte of memory
          cpu: "500m"    # Requesting 0.5 CPU cores (500 milliCPU)
        limits:
          memory: "2Gi"  # Limiting to 2 Gigabytes of memory
          cpu: "1"       # Limiting to 1 CPU core
    - name: sidecar-container
      image: myrepo/sidecar:v1
      resources:
        requests:
          memory: "256Mi"
          cpu: "100m"
        limits:
          memory: "512Mi"
          cpu: "200m"

In this example, the my-resource-hungry-app pod, with its two containers, collectively requests 1.25Gi memory and 600m CPU. The scheduler will look for a node that has at least this much available. If your YAML, like the one in the source, shows a massive request:

apiVersion: v1
kind: Pod
metadata:
  name: too-much-mem
spec:
  containers:
    - command:
        - sleep
        - "3600"
      image: busybox
      name: broken-pods-too-much-mem-container
      resources:
        requests:
          memory: "100Gi" # Is this realistic for your cluster?

Then you've likely found your culprit! A 100Gi memory request is enormous. Unless you have extremely large nodes (which are rare and expensive for standard workloads), no node will be able to satisfy this. This directly leads to "0/1 nodes available: insufficient memory".

The Case of Missing Resource Requests (and QoS Classes)

What if your YAML doesn't have a resources section at all? This is an important consideration. If no resource requests are explicitly made in your YAML, Kubernetes assigns default requests based on other configurations in the cluster, such as LimitRanges. Without explicit requests, pods often fall into the "BestEffort" or "Burstable" Quality of Service (QoS) classes. While this might seem convenient, it can lead to unpredictable scheduling and resource contention. BestEffort pods have no guarantees and are the first to be evicted when resources run low. Burstable pods have some requests but can burst up to limits, potentially impacting others.

Pro Tip: Always define explicit requests and limits for your production workloads. This ensures predictable scheduling and prevents resource starvation or overconsumption, giving your pods a "Guaranteed" QoS class if requests equal limits.

Step 2: Peeking into Your Cluster's Resource Reality (Node Metrics)

Once you know what your pod is requesting, the next logical step is to see what your cluster's nodes actually have to offer. This involves inspecting the node's capacity, how much is allocatable for pods, and how much is currently allocated. This is where kubectl describe nodes and kubectl top nodes come into play.

Inspecting Node Capacity with kubectl describe nodes

The kubectl describe nodes command is a goldmine of information about your cluster nodes. It provides details about their hardware, status, allocated resources, and more. Run this command for one of your nodes (or all, if you have a small cluster):

kubectl describe nodes <node-name>

Or, for a quick overview of all nodes:

kubectl describe nodes

Look for sections like Capacity, Allocatable, and Allocated resources. Here's a snippet of what you might see:

$ kubectl describe nodes ip-10-0-0-123.ec2.internal
[...]
Capacity:
  cpu:                4
  ephemeral-storage:  61255492Ki
  hugepages-1Gi:      0
  hugepages-2Mi:      0
  memory:             15456388Ki # Approximately 14.7 GiB

Allocatable:
  cpu:                3900m # 3.9 CPU cores available for pods
  ephemeral-storage:  56494794Ki
  hugepages-1Gi:      0
  hugepages-2Mi:      0
  memory:             14358820Ki # Approximately 13.6 GiB available for pods

Allocated resources:
  (Total limits may exceed capacity; see 'kubectl describe node <node>` for details.)
  Resource           Requests    Limits
  --------           --------    ------
  cpu                2850m (73%)  5000m (128%)
  memory             9Gi (66%)    15Gi (110%)
  ephemeral-storage  0 (0%)      0 (0%)
Events:
  [...]

Let's break these down:

• Capacity: This is the total raw hardware capacity of the node (e.g., 4 CPU cores, 15GiB memory).
• Allocatable: This is the most crucial metric for scheduling. It's the amount of resources that are actually available for pods. Kubernetes reserves a portion of the node's resources for the operating system, the Kubelet, the container runtime (e.g., containerd or Docker), and other system daemons. So, `Allocatable` is always less than `Capacity`. The scheduler will only place pods on a node if its `Allocatable` resources can cover the pod's `requests`.
• Allocated resources: This section tells you how much CPU and memory (based on *requests*) is currently assigned to pods already running on this node. It also shows the *limits* of those running pods. Notice how "Total limits may exceed capacity"; this is normal because limits are not reserved in the same way requests are.

By comparing your pod's resource requests (from Step 1) with the Allocatable resources of your nodes, you can identify two common scenarios:

1. Your resource request cannot fit into any node on the cluster: If your pod requests, say, 5Gi of memory, and all your nodes have an Allocatable memory of 4Gi or less, then no node can ever host your pod. This typically points to your pod requesting too much relative to your cluster's node sizes, or your cluster needs larger nodes.
2. Your resource request can fit on a node in the cluster, but those nodes already have workloads running on them, which block yours from being provisioned: This means individual nodes *do* have enough Allocatable resources for your pod, but when you factor in the Allocated resources by existing pods, there isn't enough contiguous or total available resource left. This often indicates resource fragmentation or simply that the cluster is currently over-utilized.

Monitoring Real-time Usage with kubectl top nodes

While kubectl describe nodes gives you static allocated resources based on requests, kubectl top nodes gives you a real-time snapshot of CPU and memory *usage* on your nodes. You'll need the Kubernetes metrics server deployed in your cluster for this to work.

kubectl top nodes

This command shows you:

NAME                 CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
ip-10-0-0-123.ec2.internal   100m         2%     2048Mi          14%
ip-10-0-0-234.ec2.internal   500m         10%    4096Mi          28%
ip-10-0-0-345.ec2.internal   2000m        50%    8192Mi          56%

This helps you understand if your nodes are genuinely under heavy load (high CPU% or MEMORY%) or if the scheduling issue is purely due to resource requests exceeding allocatable capacity, even if the nodes appear to have low actual usage.

Effective Strategies to Resolve Insufficient CPU and Memory Issues

Once you've identified the mismatch – whether it's over-eager pods or undersized nodes – it's time to implement solutions. There isn't a one-size-fits-all answer, but a combination of these strategies usually helps:

1. Right-Sizing Pod Resource Requests and Limits

This is often the most direct and impactful solution. If your pod is requesting "100Gi" memory for a simple busybox container, that's clearly an error. Adjusting these values is critical.

• Monitor Actual Usage: Use tools like Prometheus, Grafana, or kubectl top pod to monitor your applications' real CPU and memory consumption over time. This data is invaluable for setting realistic requests and limits. Remember to monitor during peak loads.
• Set Requests Conservatively: Start with requests slightly above your observed average usage. This ensures your pod always gets enough resources to run efficiently without over-requesting and hogging valuable node capacity.
• Set Limits Realistically: Limits should be set higher than requests but not excessively high. A good starting point is 1.5x to 2x your requests, depending on the application's burstability. This provides headroom for spikes while preventing runaway resource consumption.
• Iterate and Refine: Resource allocation is rarely a "set it and forget it" task. Continuously monitor your applications, adjust requests and limits, and observe cluster behavior.

Example YAML Adjustment: If your ElasticSearch pod is requesting 10Gi memory and you find it typically uses 4-5Gi, you might adjust the YAML:

# Before (potentially too high or too low leading to issues)
resources:
  requests:
    memory: "10Gi"
    cpu: "2"

# After (based on monitoring, more optimized)
resources:
  requests:
    memory: "5Gi"
    cpu: "1.5"
  limits:
    memory: "7Gi"
    cpu: "2"

2. Scaling Your Kubernetes Cluster (Horizontal and Vertical)

If your pods' requests are reasonable, but your nodes are simply running out of room, then scaling your cluster is the way to go.

• Horizontal Scaling (Adding More Nodes): This is the most common approach. If you have an autoscaling group or a managed Kubernetes service (like EKS, GKE, AKS), configure a Cluster Autoscaler. It automatically adds new nodes to your cluster when there are pending pods due to insufficient resources and removes nodes when they are underutilized. This is the ideal solution for dynamically handling workload changes.
• Vertical Scaling (Using Larger Nodes): Sometimes, horizontal scaling isn't enough, especially if you have very large, monolithic applications that require a lot of CPU or memory on a single node (e.g., a massive database instance). In such cases, you might need to provision larger node types (e.g., from an m5.large to an m5.xlarge in AWS EC2). Just ensure that these larger nodes don't remain underutilized, which can lead to increased costs.

3. Optimizing Existing Workloads and Consolidating Resources

Before adding more nodes, consider if you can optimize your existing applications or consolidate resources.

• Identify Resource Hogs: Use kubectl top pods to identify which pods are consuming the most CPU and memory. Are these usages justified? Can the applications be optimized for lower resource consumption?
• Clean Up Unused Deployments: Sometimes, old or test deployments are left running, consuming resources unnecessarily. Periodically audit your cluster for defunct workloads.
• Review Non-Application Pods: DaemonSets and system pods also consume resources. Ensure they are optimized and necessary.

4. Implementing Resource Quotas and Limit Ranges

To prevent resource contention and scheduling failures proactively, especially in multi-tenant clusters, use Kubernetes' built-in resource management features:

• Resource Quotas: These define the total resource consumption allowed for a namespace (e.g., "this namespace can use a maximum of 10 CPU cores and 20GiB of memory"). This prevents any single team or application from monopolizing cluster resources and helps manage insufficient cpu/memory issues at a higher level.
• Limit Ranges: These define default requests and limits for pods within a namespace if they are not explicitly specified in the pod definition. They also enforce minimum and maximum resource values. This helps ensure that all pods have sensible defaults, reducing the chances of "BestEffort" pods causing instability.

5. Understanding Node Affinity and Taints/Tolerations

While our primary focus is CPU and memory, the error messages sometimes mention "volume node affinity conflict". This means your pod has specific requirements for the node it can run on (e.g., it needs a particular label, or access to a specific storage type) that are not met by available nodes. Or, a node might have a "taint" preventing pods from being scheduled on it unless they have a matching "toleration." If you see such conflicts, you'll need to investigate your pod's nodeSelector, affinity rules, or tolerations and ensure they match your cluster's node configurations.

6. Pod Priority and Preemption (for Critical Workloads)

For truly critical applications, you can assign Pod Priority and Preemption. This mechanism allows high-priority pods to evict (preempt) lower-priority pods from nodes if there are insufficient resources, making room for the critical workload. This should be used sparingly and thoughtfully, as it can disrupt less critical applications.

Prevention is Better Than Cure: Best Practices

Troubleshooting is essential, but preventing these issues from occurring in the first place is the mark of a mature DevOps practice.

• Robust Monitoring and Alerting: Implement comprehensive monitoring with tools like Prometheus and Grafana. Set up alerts for nodes reaching high utilization thresholds (CPU, memory) and for pods stuck in `Pending` state. Early warnings can prevent outages.
• Implement Autoscaling: Utilize both Horizontal Pod Autoscalers (HPA) for scaling individual applications based on metrics, and Cluster Autoscalers (CA) for dynamically adding/removing nodes. This ensures your cluster can adapt to fluctuating demands.
• Regular Resource Audits: Periodically review your deployments and their resource requests/limits. Are they still appropriate? Are there any idle or zombie deployments consuming resources?
• Understand Your Workloads: Gain deep insight into your applications' resource consumption patterns throughout their lifecycle, including startup, peak usage, and idle periods.
• Cost Management Integration: Link your resource optimization efforts to cost management. Over-provisioning resources directly translates to higher cloud bills.
• Standardized Templates: Use Helm charts or other templating tools to standardize resource requests and limits across your organization, making it easier to manage and enforce best practices.

Ultimately, solving "0/1 nodes available: insufficient cpu, insufficient memory" requires a systematic approach. Start by understanding what your pod wants, then see what your cluster has, and finally, adjust either the demand or the supply (or both). With these steps, you'll not only resolve the immediate problem but also build a more resilient and efficient Kubernetes environment. Ab chalein, let's keep those pods running!

Key Takeaways

• The error "0/1 nodes available: insufficient cpu/memory" means no node has enough allocatable resources for your pod.
• Always check the pod's YAML for resources.requests; these determine scheduler decisions.
• Use kubectl describe nodes to see node Capacity, Allocatable (crucial for scheduling), and Allocated resources.
• Use kubectl top nodes and kubectl top pods for real-time resource usage monitoring to inform right-sizing decisions.
• Solutions involve right-sizing pod requests/limits, scaling your cluster (horizontally or vertically), and optimizing existing workloads.
• Preventive measures like autoscaling, resource quotas, limit ranges, and robust monitoring are essential for a healthy cluster.

Frequently Asked Questions

Why are my Kubernetes pods stuck in Pending with "insufficient cpu"?

Your pods are stuck in Pending with "insufficient cpu" because the Kubernetes scheduler cannot find any node in your cluster with enough available CPU resources to satisfy the CPU requests specified in your pod's configuration. This can happen if your pods are requesting too much CPU, or if your cluster nodes are already fully utilized or too small.

What's the difference between node Capacity and Allocatable in Kubernetes?

Capacity refers to the total hardware resources (CPU, memory, storage) of a node. Allocatable is the portion of those resources that are available for Kubernetes to assign to pods. The difference accounts for resources reserved by the operating system, kubelet, container runtime, and other system components running on the node itself.

How do I check resource usage on my Kubernetes nodes and pods?

You can check real-time resource usage using the Kubernetes Metrics Server. For nodes, use kubectl top nodes. For individual pods, use kubectl top pods or kubectl top pod <pod-name>. For historical data and more detailed insights, integrate with external monitoring solutions like Prometheus and Grafana.

Should I set resource requests or limits (or both) for my Kubernetes pods?

It's best practice to set both resource requests and limits for your production Kubernetes pods. Requests guarantee a minimum amount of resources for scheduling and stability, while limits prevent your application from consuming excessive resources and impacting other workloads on the same node. Setting requests equal to limits grants your pod a "Guaranteed" Quality of Service (QoS) class, offering the highest level of reliability.

Solving resource-related issues in Kubernetes is a fundamental skill for any DevOps engineer. This deep dive should give you a solid foundation to diagnose and resolve instances of "0/1 nodes available: insufficient cpu, insufficient memory". For a visual walkthrough and more practical tips, don't forget to watch the original video on the @explorenystream YouTube channel. Subscribe for more expert insights!