Out of Service

intermediateNoEndpoints

Objective

Escape Room: Out of Service

The application is running fine but traffic never reaches it. Everything appears to be configured, but something isn't connecting.

Your Mission

Verify the application pod is running
Investigate why traffic isn't reaching the pod
Fix the networking so the application is accessible

Success Criteria

The pod escape-app remains in Running state
The Service escape-service has at least 1 endpoint
You can curl the service and get a response

Getting Started

# Check the pod status - it should be Running
kubectl get pods -n escape-room-out-of-service

# Check the service
kubectl get svc -n escape-room-out-of-service

# Check the endpoints - this is the key!
kubectl get endpoints -n escape-room-out-of-service

Namespace

All resources are in the escape-room-out-of-service namespace.

Good luck, engineer. The app is running, but nobody can reach it.

Quick Start

Run this command in your terminal to set up the room:

$ make room-apply ROOM=room-out-of-service

This creates the namespace escape-room-out-of-service with the broken resources.

Other useful commands:

$ make room-test ROOM=room-out-of-service

Verify the room is in the expected broken state

$ make room-escape-test ROOM=room-out-of-service

Test if you have successfully fixed all issues

$ make room-reset ROOM=room-out-of-service

Reset the room to try again

Useful Commands

Check pod status

$ kubectl get pods -n escape-room-out-of-service

See the current state of pods in the namespace

View events

$ kubectl get events -n escape-room-out-of-service --sort-by='.lastTimestamp'

Check recent events for error details

Describe pods

$ kubectl describe pods -n escape-room-out-of-service

Get detailed information about pods

Check logs

$ kubectl logs -l app.kubernetes.io/part-of=K8sEscapeRoom -n escape-room-out-of-service

View the application logs

Hints

0/4 revealed

Submit Proof

View Solution (Spoiler)

Solution preview locked

Complete the room to unlock the full solution here

Run this to see the full solution:

$ make room-solution ROOM=room-out-of-service

Show solution anyway (spoiler)

Solution: Service Selector Mismatch

Root Cause

The Service has a selector that doesn't match the pod's labels:

# Service selector (WRONG)
selector:
  app: escapeapp    # Missing hyphen!

# Pod labels (CORRECT)
labels:
  app: escape-app   # Has hyphen

Because the selector doesn't match, Kubernetes cannot associate any pods with the Service, resulting in zero endpoints. The pod runs fine, but traffic sent to the Service has nowhere to go.

Diagnosis Steps

# Step 1: Verify pod is running (it is!)
kubectl get pods -n escape-room-out-of-service
# Output: escape-app-xxxxx   1/1   Running   0   ...

# Step 2: Check service exists
kubectl get svc -n escape-room-out-of-service
# Output: escape-service   ClusterIP   10.x.x.x   <none>   80/TCP   ...

# Step 3: Check endpoints - THIS IS THE KEY
kubectl get endpoints escape-service -n escape-room-out-of-service
# Output: escape-service   <none>   ← NO ENDPOINTS!

# Step 4: Compare labels and selectors
kubectl get pods -n escape-room-out-of-service --show-labels
# Shows: app=escape-app

kubectl get svc escape-service -n escape-room-out-of-service -o jsonpath='{.spec.selector}'
# Shows: {"app":"escapeapp"} ← MISMATCH!

The Fix

Option 1: Patch the Service Selector

kubectl patch svc escape-service -n escape-room-out-of-service \
  --type='json' \
  -p='[{"op": "replace", "path": "/spec/selector/app", "value": "escape-app"}]'

Option 2: Edit the Service Directly

kubectl edit svc escape-service -n escape-room-out-of-service

Change:

selector:
  app: escapeapp

To:

selector:
  app: escape-app

Option 3: Replace the Service

kubectl get svc escape-service -n escape-room-out-of-service -o yaml > svc.yaml
# Edit svc.yaml to fix the selector
kubectl replace -f svc.yaml

Verification

# Check endpoints now exist
kubectl get endpoints escape-service -n escape-room-out-of-service
# Should show: escape-service   10.x.x.x:80

# Test connectivity
kubectl run test-curl --rm -it --image=curlimages/curl --restart=Never -n escape-room-out-of-service -- curl -s http://escape-service
# Should return nginx welcome page HTML

Lessons Learned

Always check endpoints when debugging Service connectivity issues
Labels and selectors must match exactly (case-sensitive, hyphen-sensitive)
A Service with no endpoints means the selector doesn't match any pods
Pod running ≠ Service working - they're independent

Real-World Considerations

This commonly happens when:

Copy-paste errors in YAML
Refactoring label names without updating all references
Different teams manage Deployments and Services
Auto-generated names vs manual names don't match

Best practices:

Use consistent labeling conventions across your team
Use Helm/Kustomize to ensure labels are consistent
Add kubectl get endpoints to your debugging checklist
Consider using label validation in CI/CD
Use kubectl describe svc which shows endpoint count