Kubernetes: from a pod to an operated service

Before you begin

Complete the first Docker lessons. Install kubectl and a local cluster such as minikube from their official instructions. Start the cluster. These commands require a dedicated local context; confirm kubectl config current-context before modifying resources.

Working toward: Deploy and recover a small application with probes, resource requests, storage, access controls, and observable rollouts.

Read each explanation, run the example in your own lab, and attempt the exercise before opening its answer. Published lessons are ready to study; unfinished roadmap topics remain planned.

Validation: New lessons 5–7 were reviewed against current Kubernetes documentation on 14 September 2026. YAML was parsed and checked for matching selectors, named ports, namespaces, probes and resource/rollout settings. No kubectl, API server or local cluster was available: admission, scheduling, image pulls, probe transitions and rollbacks were not execution-tested. Use a dedicated local cluster and record its client/server versions; schema acceptance is not inferred from YAML parsing.

1. Declare a desired state

A pod is the scheduling unit containing one or more containers. A Deployment manages replicated pods through a ReplicaSet. You declare the desired number of replicas; controllers work to reconcile actual state. A namespace groups names and provides a scope for policy.

kubectl config current-context
kubectl get nodes
kubectl create namespace auwen-lab
kubectl -n auwen-lab create deployment web --image=nginx:stable
kubectl -n auwen-lab rollout status deployment/web
kubectl -n auwen-lab get pods

What to expect

A node is Ready and a web pod becomes Running/Ready. Image pulls and scheduling can take time.

Your turn

Explain why a Deployment is preferable to manually recreating a deleted pod.

Show answer and reasoning

Its controller knows the desired replica count and creates replacement pods. The replacement has a different identity; applications must tolerate that.

Watch for: A namespace is not a complete security boundary. Cluster context mistakes can send valid commands to the wrong environment.

Link to this lesson

2. Reach a changing set of pods

A Service gives a selected set of pods a stable networking abstraction. Pod addresses can change. Here ClusterIP stays inside the cluster; port-forward provides a temporary local development connection.

kubectl -n auwen-lab expose deployment web --port=80 --target-port=80
kubectl -n auwen-lab get service web
kubectl -n auwen-lab port-forward service/web 8084:80

What to expect

http://localhost:8084 displays nginx while port-forward runs. Ctrl+C ends the forwarding process, not the Deployment.

Your turn

Find which labels the Service selects using kubectl -n auwen-lab describe service web.

Show answer and reasoning

The Service created by expose selects the Deployment’s app=web label. A selector mismatch leaves the Service with no matching ready backends.

Watch for: Port-forward is a local debugging tool, not a production ingress design.

Link to this lesson

3. Observe recovery and scaling

Increasing replicas changes desired capacity. Deleting one pod demonstrates reconciliation. Use a pod name you just inspected; do not delete the Deployment, which would remove the controller and its desired state.

kubectl -n auwen-lab scale deployment web --replicas=2
kubectl -n auwen-lab get pods
# Replace POD_NAME with one web pod from the output:
kubectl -n auwen-lab delete pod POD_NAME
kubectl -n auwen-lab get pods -w

What to expect

The controller creates a replacement and returns toward two ready replicas. Ctrl+C stops watching.

Your turn

Explain why two replicas alone do not guarantee high availability on a one-node laptop cluster.

Show answer and reasoning

Both replicas share one node and its failure domain. Node failure can take down both. Availability also depends on scheduling, dependencies, networking, and capacity.

Watch for: A replica count is not proof of resilience. Test failure scenarios and application readiness.

Link to this lesson

4. Diagnose before restarting

Status, events, and logs answer different questions. Pending often points to scheduling or storage. ImagePullBackOff points to an image retrieval problem. CrashLoopBackOff means the process keeps exiting and restarting. Look at the preceding evidence before changing anything.

kubectl -n auwen-lab get pods
kubectl -n auwen-lab describe deployment web
kubectl -n auwen-lab get events --sort-by=.metadata.creationTimestamp
kubectl -n auwen-lab logs deployment/web
# Inspect a specific pod with describe pod POD_NAME

What to expect

Record observed state and a cause hypothesis. The healthy lab should show available replicas and request/startup logs.

Your turn

Where would you look first if an image name were misspelled? Clean up only the lab namespace when finished.

Show answer and reasoning

Describe the affected pod and inspect its events for image pull errors. Cleanup: kubectl delete namespace auwen-lab. This deletes resources within that namespace, so verify the name first.

Watch for: Do not paste secrets from logs into public issue reports. A restart can erase useful evidence without solving the cause.

Link to this lesson

5. Keep desired state in a file you can review

Before this lesson: Complete Kubernetes objects, Service, reconciliation and diagnosis (lessons 1–4). Use a dedicated local minikube or equivalent learning cluster, not a work cluster. Outcome: explain and apply a Namespace, Deployment and Service from one file and make a repeatable replica change.

Imperative commands are useful for discovery, but remembering every command is a poor configuration record. A manifest states desired object fields in YAML. apiVersion identifies the API schema; kind identifies the object type; metadata gives identity; spec describes desired behavior. The separator --- allows several YAML documents in one file. This file is not a script: document order alone is not a general application dependency mechanism.

The Deployment's selector identifies the pods it manages. Its pod-template labels match that selector. The Service independently selects matching pod labels; it is not attached to a Deployment by name. A named container port http is resolved by the Service's targetPort. containerPort documents a port; it does not make an application listen or publish a host port by itself.

Save notes.yaml below. This creates a new namespace, auwen-next-lab, so the earlier auwen-lab exercises remain separate. nginx:stable is a mutable teaching tag: record the resolved image ID/digest, and use a reviewed immutable reference when reproducibility matters. Do not infer that a currently documented Kubernetes feature exists on your own server; check kubectl version and your cluster's supported APIs.

Applying a file reconciles the fields you declare. Repeating the same apply should not create duplicate objects because kind/namespace/name identify them. Editing the file preserves intent; an unrelated imperative edit may later be overwritten by that file. kubectl diff is a preview with a meaningful nonzero status when differences exist, not automatically a failed release.

notes.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: auwen-next-lab
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: notes
  namespace: auwen-next-lab
spec:
  replicas: 2
  selector:
    matchLabels:
      app: auwen-notes
  template:
    metadata:
      labels:
        app: auwen-notes
    spec:
      containers:
        - name: web
          image: nginx:stable
          ports:
            - name: http
              containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: notes
  namespace: auwen-next-lab
spec:
  selector:
    app: auwen-notes
  ports:
    - port: 80
      targetPort: http

kubectl config current-context
kubectl version
# Stop unless this is your dedicated local learning cluster.
kubectl apply -f notes.yaml
kubectl -n auwen-next-lab rollout status deployment/notes --timeout=120s
kubectl -n auwen-next-lab get pods -l app=auwen-notes
kubectl -n auwen-next-lab get service notes
kubectl -n auwen-next-lab get endpointslices -l kubernetes.io/service-name=notes
kubectl -n auwen-next-lab port-forward service/notes 8087:80

Run it

Keep port-forward running and visit http://localhost:8087. Ctrl+C ends forwarding, not the Deployment. Apply the unchanged file again and inspect object counts. Save the manifest alongside a short record of client/server versions and the pulled image ID.

What to expect

Two pods become ready if scheduling and image retrieval succeed; the Deployment reaches its desired replicas. The Service has a cluster-internal address and EndpointSlices reflecting its selected backends. Reapplying the same file does not create a second notes Deployment. Port-forward provides a temporary local route; it is not a production ingress.

Your turn

Edit replicas from 2 to 3 in notes.yaml. Run kubectl diff -f notes.yaml, apply it, and verify three pods. Then deliberately change only the Service selector to app: wrong-label. Predict whether the Deployment or Service breaks, inspect EndpointSlices, and restore the selector in the file.

Show answer and reasoning

kubectl diff -f notes.yaml
kubectl apply -f notes.yaml
kubectl -n auwen-next-lab get pods -l app=auwen-notes
kubectl -n auwen-next-lab get endpointslices -l kubernetes.io/service-name=notes

# Three replicas belong to the Deployment. A wrong Service selector
# does not stop those pods, but leaves that Service without matching
# backends. Restore app: auwen-notes in the Service and reapply.
# Restore replicas: 2 for the following resource example.
# diff status 1 means differences; status greater than 1 signals error.
# Checkpoint: explain three separate roles for the Deployment selector,
# template labels and Service selector.

Watch for: Do not change an existing Deployment selector casually: it is immutable. YAML parsing alone cannot validate an API schema or admission policy. Deleting this manifest includes deleting its namespace and everything inside it, so reserve cleanup for the end of these dedicated labs. Never copy an unrelated namespace into a deletion command.

Lesson references

Link to this lesson

6. Give startup, readiness and liveness different jobs

Before this lesson: Complete the manifest lesson and keep auwen-next-lab. Save the separate probe-notes.yaml below. Outcome: remove a pod from ready Service backends without restarting it, then distinguish that from liveness-driven restart behavior.

A startup probe protects a slow-starting application by withholding its regular liveness/readiness checks until startup succeeds. A readiness probe asks whether the container should receive ordinary Service traffic. A liveness probe asks whether restarting the container is an appropriate recovery response. Reusing one shallow check for all three can hide failures or cause restart loops.

The teaching Deployment uses nginx for HTTP and a marker file for readiness. Its shell first creates /tmp/ready, then exec replaces the shell with nginx so the server is the main process. Startup and liveness check the HTTP endpoint. Readiness tests only for the marker. This deliberately simple signal lets us make the pod unready without breaking HTTP or triggering liveness. A real readiness check must represent application readiness, not merely copy this marker trick.

The startup settings allow roughly 60 seconds of failed checks (30 at a two-second period) before a failed start is treated as a failure; exact timing includes scheduling and probe execution. Readiness has one allowed failure at a three-second period. Liveness needs three consecutive failures at a ten-second period, each bounded by its timeout. These are different operational decisions, not arbitrary identical defaults.

A not-ready pod can remain Running with an unchanged restart count. Kubernetes updates endpoint readiness so normal Service traffic avoids it. Probe effects are asynchronous; inspect pod state, events and EndpointSlice conditions rather than relying solely on the browser. Port-forward can reach a selected pod directly and is not a reliable test of Service load-balancing readiness.

probe-notes.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: probe-notes
  namespace: auwen-next-lab
spec:
  replicas: 1
  selector:
    matchLabels:
      app: auwen-probe-notes
  template:
    metadata:
      labels:
        app: auwen-probe-notes
    spec:
      containers:
        - name: web
          image: nginx:stable
          command: ["/bin/sh", "-c"]
          args:
            - "touch /tmp/ready; exec nginx -g 'daemon off;'"
          ports:
            - name: http
              containerPort: 80
          startupProbe:
            httpGet:
              path: /
              port: http
            periodSeconds: 2
            failureThreshold: 30
          readinessProbe:
            exec:
              command: ["/bin/sh", "-c", "test -f /tmp/ready"]
            periodSeconds: 3
            failureThreshold: 1
          livenessProbe:
            httpGet:
              path: /
              port: http
            periodSeconds: 10
            timeoutSeconds: 2
            failureThreshold: 3
---
apiVersion: v1
kind: Service
metadata:
  name: probe-notes
  namespace: auwen-next-lab
spec:
  selector:
    app: auwen-probe-notes
  ports:
    - port: 80
      targetPort: http

kubectl config current-context
kubectl apply -f probe-notes.yaml
kubectl -n auwen-next-lab rollout status deployment/probe-notes --timeout=120s
kubectl -n auwen-next-lab get pods -l app=auwen-probe-notes
# Remove only the teaching readiness marker:
kubectl -n auwen-next-lab exec deployment/probe-notes -- rm /tmp/ready
kubectl -n auwen-next-lab get pods -l app=auwen-probe-notes
kubectl -n auwen-next-lab get endpointslices -l kubernetes.io/service-name=probe-notes -o yaml
# Restore the marker:
kubectl -n auwen-next-lab exec deployment/probe-notes -- touch /tmp/ready
kubectl -n auwen-next-lab get pods -l app=auwen-probe-notes

Run it

Wait for initial readiness, record restart count, remove the marker, then wait for a readiness check before inspecting again. EndpointSlices may retain an endpoint with ready: false rather than removing the address entirely. Restore the marker and watch readiness recover. Do not restart the Deployment to conceal the cause.

What to expect

Initially one pod is Running and 1/1 Ready. Removing /tmp/ready eventually changes it to 0/1 Ready while it remains Running and its restart count stays unchanged. HTTP liveness still succeeds. Restoring the file makes it ready again after a successful check. These are predicted cluster observations; no cluster execution was available here.

Your turn

In a copy of the manifest, add an exec liveness probe that tests /tmp/ready instead of the HTTP liveness probe; remove the original httpGet from that liveness block because a probe has one handler. Predict what deleting the marker now does. Would this be a sensible response to a brief outage in an external database?

Show answer and reasoning

livenessProbe:
  exec:
    command: ["/bin/sh", "-c", "test -f /tmp/ready"]
  periodSeconds: 10
  timeoutSeconds: 2
  failureThreshold: 3

# Apply the experimental manifest, wait for its new pod, then delete
# the marker. Repeated liveness failures trigger a container restart.
# The startup shell recreates the marker; the restart count increases.
# Restarting every client may worsen an external database outage.
# Use readiness for inability to serve; use liveness only where a
# restart is a justified recovery. Restore the HTTP liveness version.
# Checkpoint: predict Ready, phase and restart count independently.

Watch for: An exec probe runs in the container and needs the requested executable. Probe failures do not all have the same effect. Overaggressive liveness can amplify overload; a huge startup allowance can hide a broken start for too long. Default Service endpoint behavior has exceptions such as publishNotReadyAddresses, not used here.

Lesson references

Link to this lesson

7. Reserve capacity and recover a stalled rollout

Before this lesson: Complete manifests and probes. Keep a dedicated local cluster and the notes Deployment from lesson 5; probe-notes is independent. Save this complete replacement as notes.yaml. Outcome: calculate requested capacity, distinguish Pending from a crashing process, and recover a rollout while restoring the source file.

A resource request tells the scheduler how much capacity to account for when placing a pod. A limit bounds runtime use under the platform's enforcement rules. 100m CPU means one tenth of a CPU unit; 32Mi is 32 × 2^20 bytes. CPU pressure can cause throttling; memory-limit failures may cause termination. These are not interchangeable outcomes, and a request is not a promise of dedicated low-latency hardware.

Two replicas requesting 100m and 32Mi each request 200m CPU and 64Mi total for these containers. The rolling strategy permits one extra pod (maxSurge: 1), so an update can temporarily need 300m CPU and 96Mi of requests, plus cluster/system overhead and other workloads. maxUnavailable: 0 keeps the controller from intentionally reducing available replicas during a normal rolling update, but does not guarantee uptime against node or application failure.

A readiness probe prevents a new pod from counting as ready merely because its process started. progressDeadlineSeconds reports a stalled rollout after the configured deadline; it does not automatically roll the Deployment back. A rollout-status timeout is the command giving up waiting, not proof that Kubernetes undid the change.

Save a known-good copy of notes.yaml before experimenting. A deliberately nonexistent image tag produces a controlled pull failure. Existing ready replicas can remain while the surge pod cannot start. Inspect events and the actual image string; increasing memory will not correct a misspelled image. Recovery requires both restoring runtime state and correcting the file that the next apply will use.

notes.yaml — replace the lesson 5 file

apiVersion: v1
kind: Namespace
metadata:
  name: auwen-next-lab
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: notes
  namespace: auwen-next-lab
spec:
  replicas: 2
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  progressDeadlineSeconds: 120
  selector:
    matchLabels:
      app: auwen-notes
  template:
    metadata:
      labels:
        app: auwen-notes
    spec:
      containers:
        - name: web
          image: nginx:stable
          resources:
            requests:
              cpu: "100m"
              memory: "32Mi"
            limits:
              cpu: "500m"
              memory: "128Mi"
          readinessProbe:
            httpGet:
              path: /
              port: http
            periodSeconds: 3
          ports:
            - name: http
              containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: notes
  namespace: auwen-next-lab
spec:
  selector:
    app: auwen-notes
  ports:
    - port: 80
      targetPort: http

kubectl config current-context
kubectl apply -f notes.yaml
kubectl -n auwen-next-lab rollout status deployment/notes --timeout=120s
kubectl -n auwen-next-lab get pods -l app=auwen-notes
kubectl -n auwen-next-lab rollout history deployment/notes

# Controlled bad image, local lab only:
kubectl -n auwen-next-lab set image deployment/notes web=nginx:auwen-deliberately-missing-20260914
kubectl -n auwen-next-lab rollout status deployment/notes --timeout=60s
kubectl -n auwen-next-lab describe deployment notes
kubectl -n auwen-next-lab get events --sort-by=.metadata.creationTimestamp

# Restore the unchanged known-good source file:
kubectl apply -f notes.yaml
kubectl -n auwen-next-lab rollout status deployment/notes --timeout=120s

Run it

Use a valid pulled nginx:stable image and sufficient local-cluster capacity for the baseline. Record rollout history before the bad-image change. If your registry unexpectedly contains the deliberately unusual tag, choose a verified nonexistent lab tag instead. The intended test is image retrieval failure, not running an unknown image.

What to expect

The baseline should reach two ready replicas. The bad update normally creates an unready surge pod with image-pull events and stalls; old ready pods remain when capacity and the previous workload are healthy. Reapplying the known-good file restores the intended image and eventually the desired ready replica count. Event order, pod names and timing vary. No automatic rollback is claimed.

Your turn

Calculate requests for three replicas plus one surge. Then propose a temporary CPU request larger than any node's allocatable CPU in your local cluster: should the pod be Pending or CrashLoopBackOff, and why? Explain how rollout undo differs from correcting a bad image still written in notes.yaml.

Show answer and reasoning

Three steady replicas: 300m CPU, 96Mi memory.
With one surge replica: 400m CPU, 128Mi memory, plus other overhead.

An unschedulable request prevents placement: Pending with scheduling
events is expected, not a running process repeatedly crashing.
Inspect the specific pod's events before changing limits.

kubectl rollout undo can restore an earlier Deployment pod template,
but does not rewrite your local YAML or reverse database migrations.
A subsequent apply of a bad file can reintroduce the failure.
Correct the file and verify the image, replica readiness and workload.
Cleanup AFTER all Kubernetes labs:
kubectl delete namespace auwen-next-lab
This removes every resource in that dedicated namespace.

Watch for: Do not use kubectl top as your only capacity evidence: it requires a metrics service and shows observations, not scheduling requests. A readiness endpoint returning 200 is not a full application acceptance test. Mutable image tags weaken rollback reproducibility; pin reviewed digests for real releases. No workload, policy or schema validation is implied by simply parsing YAML.

Lesson references

Link to this lesson

Path to advanced

In-progress stages identify the lessons already published. All other listed topics remain planned. Each addition needs teaching, a reproducible lab, failure cases, and a checkpoint before the capstone.

Declarative workloads
Published: complete manifests, startup/readiness/liveness probes, requests/limits and rollout recovery (lessons 5–7). Planned: deeper scheduling, resource tuning and multi-service release exercises; the operated-service capstone remains planned.
IN PROGRESS · PUBLISHED LESSONS ABOVE
Platform foundations
ConfigMaps, Secrets, storage classes, persistent volumes, DNS, ingress/Gateway API, and scheduling.
PLANNED
Operate securely
RBAC, network policies, upgrades, metrics, backups, recovery, and policy enforcement.
PLANNED
Capstone
Operate a versioned application, deliberately break it, and recover from a written runbook.
PLANNED

References

Original AUWEN lessons, with upstream documentation for further study and version checks.

All learning paths and update notes →

Before you begin

1. Declare a desired state

What to expect

Your turn

2. Reach a changing set of pods

What to expect

Your turn

3. Observe recovery and scaling

What to expect

Your turn

4. Diagnose before restarting

What to expect

Your turn

5. Keep desired state in a file you can review

notes.yaml

Run it

What to expect

Your turn

Lesson references

6. Give startup, readiness and liveness different jobs

probe-notes.yaml

Run it

What to expect

Your turn

Lesson references

7. Reserve capacity and recover a stalled rollout

notes.yaml — replace the lesson 5 file

Run it

What to expect

Your turn

Lesson references

Path to advanced

Declarative workloads

Platform foundations

Operate securely

Capstone

References