This capstone lesson wires together everything from the tutorial: Ingress with TLS termination, NetworkPolicy micro-segmentation, and a PersistentVolumeClaim for durable storage — all on a single realistic application stack. The sample is a Guestbook API (a stateless Node.js backend paired with a Redis instance that holds persistent data), but every technique applies directly to production Java services, Python data pipelines, or any stateful workload you deploy at scale.
What you will build: Two Deployments (api + redis), two Services, one Ingress with a Let's Encrypt TLS certificate, one NetworkPolicy that restricts ingress to only the api → redis path, and one PersistentVolumeClaim that survives pod restarts and node failures.
Step 1 — Namespace & Resource Isolation
Always give a real application its own namespace. This gives you a clean RBAC boundary, makes kubectl output readable, and lets NetworkPolicy selectors be scoped to only the pods you care about.
Redis is an in-memory store, but its RDB/AOF persistence writes to disk. Without a PVC, every pod restart loses all data. The PVC below requests a 10Gi volume from whichever StorageClass is the cluster default — on AWS that is gp3, on GKE it is standard-rwo. The ReadWriteOnce access mode is correct: Redis is a single-writer process and must not share the volume with another replica.
Why headless for Redis? A headless Service (clusterIP: None) makes DNS return the Pod IP directly instead of a VIP. This lets Redis clients use their own consistent-hashing logic or connection pooling without an extra hop through kube-proxy.
Step 3 — Stateless API Deployment & Service
The API is horizontally scalable: three replicas, no local state. It reads the REDIS_HOST environment variable to locate Redis. The Service type is ClusterIP — it must not be LoadBalancer because the Ingress controller will route traffic to it internally.
This is where external traffic enters the cluster. The manifest below assumes you are running the ingress-nginx controller and the cert-manager with a ClusterIssuer named letsencrypt-prod. The cert-manager.io/cluster-issuer annotation is the only thing you need to add — cert-manager watches for it, provisions a TLS certificate via ACME HTTP-01, and stores it in the Secret named guestbook-tls. The Ingress controller then reads that Secret and terminates TLS before forwarding plain HTTP to the backend Service.
DNS must resolve before cert-manager can issue the certificate. Create a DNS A record pointing guestbook.example.com to the Ingress controller's external IP before applying this manifest. cert-manager will fail ACME HTTP-01 validation — and back off exponentially — if the domain does not resolve. Use kubectl describe certificate guestbook-tls -n guestbook to watch the issuance state.
By default, Kubernetes allows all pod-to-pod traffic within a cluster. In production, that means a compromised API pod could directly connect to any database, secret store, or internal control-plane endpoint it can route to. NetworkPolicies enforce a least-privilege network posture. The two policies below implement a strict allow-list:
Redis accepts TCP/6379 only from pods labeled tier: api in the same namespace.
The API accepts port 8080 from the Ingress controller's namespace (labeled by the controller's DaemonSet), and nothing else inbound.
End-to-end request path: HTTPS traffic terminates at the Ingress controller, forwards to the API pods (protected by NetworkPolicy), which write durable data to Redis backed by a PVC.
Step 6 — Apply & Validate
Apply everything in dependency order, then confirm each layer is healthy before moving to the next.
# Apply all manifests
kubectl apply -f redis-pvc.yaml
kubectl apply -f redis-deployment.yaml
kubectl apply -f api-deployment.yaml
kubectl apply -f ingress.yaml
kubectl apply -f network-policy.yaml
# Verify PVC is Bound (StorageClass provisioned the volume)
kubectl get pvc -n guestbook
# Verify pods are Running and Ready
kubectl get pods -n guestbook -w
# Verify Ingress has an ADDRESS (the LB external IP)
kubectl get ingress -n guestbook
# Watch cert-manager issue the TLS certificate (~60s on a healthy cluster)
kubectl describe certificate guestbook-tls -n guestbook
# Smoke-test the live endpoint
curl -v https://guestbook.example.com/healthz
# Test NetworkPolicy enforcement: this should time out (no route from default → redis)
kubectl run test-pod --rm -it --image=busybox --namespace=default \
-- sh -c "nc -zv redis.guestbook.svc.cluster.local 6379"
# This should succeed (api tier pod → redis)
kubectl exec -n guestbook deploy/guestbook-api \
-- sh -c "nc -zv redis.guestbook.svc.cluster.local 6379"
Production Failure Modes & Hardening
Every step in this project has a corresponding failure mode that manifests in real clusters:
PVC stuck in Pending — the StorageClass does not exist or the CSI driver is not installed. Run kubectl describe pvc redis-data -n guestbook; look at the Events section for the provisioner error.
Certificate stuck in Pending — DNS not propagated, or the ACME HTTP-01 challenge is blocked by the NetworkPolicy. cert-manager creates a temporary Ingress and an HTTP challenge pod in the cert-manager namespace. Ensure your NetworkPolicy for the API namespace does not block inbound port 80 from cert-manager.
502 Bad Gateway from Ingress — the readiness probe is failing, so Endpoints has zero members. Describe the pod and check the probe logs: kubectl describe pod -n guestbook -l app=guestbook-api.
Data loss after Redis pod restart — you forgot to mount the PVC, or the PVC was deleted. Always set a reclaimPolicy: Retain on production StorageClasses to protect against accidental PVC deletion.
GitOps this entire manifest set. Commit all YAML to a repository and manage it with Argo CD or Flux. That way, a kubectl delete namespace guestbook (accidental or malicious) is a self-healing event — the GitOps controller recreates the namespace and reconciles every resource within seconds.