No Vacancy

intermediatePending

Objective

Escape Room: No Vacancy

The application pod is stuck in Pending state. The previous team migrated these manifests from a cloud cluster, but something about this environment doesn't match what the pod expects.

Your Mission

Identify why the pod cannot be scheduled
Understand what the pod's storage needs and what the cluster provides
Fix the storage configuration so the pod can run

Success Criteria

The escape-app-0 pod is in Running state and shows 1/1 Ready
The PersistentVolumeClaim is Bound

Getting Started

# Check the pod status
kubectl get pods -n escape-room-no-vacancy

# Check all resources including PVCs
kubectl get all,pvc -n escape-room-no-vacancy

# What storage infrastructure does this cluster have?
kubectl get storageclass

Useful Reference

A StorageClass tells Kubernetes how to provision storage. You can inspect any existing StorageClass to see how it's configured:

kubectl get sc <name> -o yaml

The key fields are provisioner (which backend creates the volumes) and volumeBindingMode (when to bind). If you need to create a new StorageClass, a minimal definition looks like:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: <name>
provisioner: <copy-from-existing-sc>
volumeBindingMode: <copy-from-existing-sc>

Namespace

All resources are in the escape-room-no-vacancy namespace.

Good luck, engineer. The pod needs its storage before it can start.

Quick Start

Run this command in your terminal to set up the room:

$ make room-apply ROOM=room-no-vacancy

This creates the namespace escape-room-no-vacancy with the broken resources.

Other useful commands:

$ make room-test ROOM=room-no-vacancy

Verify the room is in the expected broken state

$ make room-escape-test ROOM=room-no-vacancy

Test if you have successfully fixed all issues

$ make room-reset ROOM=room-no-vacancy

Reset the room to try again

Useful Commands

Check pod status

$ kubectl get pods -n escape-room-no-vacancy

See the current state of pods in the namespace

View events

$ kubectl get events -n escape-room-no-vacancy --sort-by='.lastTimestamp'

Check recent events for error details

Describe pods

$ kubectl describe pods -n escape-room-no-vacancy

Get detailed information about pods

Check logs

$ kubectl logs -l app.kubernetes.io/part-of=K8sEscapeRoom -n escape-room-no-vacancy

View the application logs

Hints

0/4 revealed

Submit Proof

View Solution (Spoiler)

Solution preview locked

Complete the room to unlock the full solution here

Run this to see the full solution:

$ make room-solution ROOM=room-no-vacancy

Show solution anyway (spoiler)

Solution: PVC Pending

Root Cause

The StatefulSet's volumeClaimTemplates references a StorageClass named fast-storage that doesn't exist in this cluster:

volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      storageClassName: "fast-storage"   # <-- doesn't exist!

The manifests were migrated from a cloud cluster that had a fast-storage StorageClass for SSD-backed persistent volumes. This cluster only has the standard StorageClass.

Result: PVCs stay Pending → pods stay Pending.

Diagnosis Steps

# Step 1: Pod is Pending
kubectl get pods -n escape-room-no-vacancy
# NAME           READY   STATUS    RESTARTS   AGE
# escape-app-0   0/1     Pending   0          5m

# Step 2: Describe pod — waiting for volume
kubectl describe pod escape-app-0 -n escape-room-no-vacancy
# Events:
#   Warning  FailedScheduling  pod has unbound immediate PersistentVolumeClaims

# Step 3: Check PVC — also Pending
kubectl get pvc -n escape-room-no-vacancy
# NAME                STATUS    STORAGECLASS   AGE
# data-escape-app-0   Pending   fast-storage   5m

# Step 4: Describe PVC — StorageClass not found
kubectl describe pvc data-escape-app-0 -n escape-room-no-vacancy
# Events:
#   Warning  ProvisioningFailed  storageclass.storage.k8s.io "fast-storage" not found

# Step 5: What StorageClasses exist? What provisioner do they use?
kubectl get sc
# NAME                 PROVISIONER             ...
# standard (default)   rancher.io/local-path   ...

The Fix

Create a StorageClass named fast-storage with the same provisioner as standard:

First, check what provisioner and settings the existing standard StorageClass uses:

kubectl get sc standard -o yaml

Then create a StorageClass named fast-storage with the same provisioner and volumeBindingMode:

kubectl apply -f - <<'EOF'
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: fast-storage
provisioner: rancher.io/local-path
volumeBindingMode: WaitForFirstConsumer
EOF

That's it — no pods or PVCs to delete. The PVC binds once the StorageClass exists, and the pod schedules automatically.

Verification

# Check PVC is now Bound
kubectl get pvc -n escape-room-no-vacancy
# NAME                STATUS   VOLUME       CAPACITY   STORAGECLASS
# data-escape-app-0   Bound    pvc-xxxxx    1Gi        fast-storage

# Check pod is Running
kubectl get pods -n escape-room-no-vacancy
# NAME           READY   STATUS    RESTARTS   AGE
# escape-app-0   1/1     Running   0          30s

Lessons Learned

StorageClasses are cluster-scoped — they don't travel with namespace manifests
Always check kubectl get sc when PVCs are stuck Pending
Manifests from other clusters may reference infrastructure that doesn't exist — StorageClasses, IngressClasses, etc.
StatefulSet volumeClaimTemplates are immutable — you can't edit the StatefulSet to change the StorageClass; you must make the environment match
Inspect existing resources — kubectl get sc standard -o yaml tells you the provisioner you need

Real-World Considerations

This happens constantly when:

Migrating from cloud (EKS/GKE/AKS) to on-prem or local clusters
Copying manifests between environments (staging → production)
Using Helm charts that assume specific StorageClass names
Setting up new clusters without matching the storage configuration

Prevention:

Document required StorageClasses in your deployment prerequisites
Use Helm values or Kustomize overlays to parameterize storageClassName
Include StorageClass manifests in your infrastructure-as-code
Use the default StorageClass where possible instead of naming one explicitly