Kubernetes Architecture Explained

Use It, Don't Just Say It

Follow a Pod from an API request to scheduling and execution, then examine what recovery does and does not guarantee.

1. The Control Plane (The Brain)

These components manage the cluster's global state. They make decisions (scheduling, scaling, self-healing) and run on dedicated "master" nodes.

A. API Server (kube-apiserver)

• Role: The single front door to the entire cluster. Every interaction — kubectl, the dashboard, other control plane components — goes through the API Server.
• Function: Validates requests (kubectl create pod), authenticates callers, authorizes requests (for example with RBAC), and persists state to Etcd. It is the only component that talks directly to Etcd.
• Design: Stateless and horizontally scalable. You can run multiple API Server instances behind a load balancer for high availability.

B. Etcd

• Role: The cluster's source of truth. A distributed, consistent Key-Value store (based on the Raft consensus algorithm).
• Data Stored: The entire cluster state — pod specs, service definitions, secrets, ConfigMaps, namespace info, RBAC policies.
• Criticality: If Etcd is corrupted or lost, the cluster is effectively dead. Three- or five-member etcd clusters are common HA choices; backups and restore drills remain necessary.
• Performance: Writes require consensus. Read consistency modes differ; disk latency, object churn and request patterns determine capacity. Benchmark the actual workload.

C. Scheduler (kube-scheduler)

Interactive: The Scheduling Algorithm

• Role: Watches for newly created Pods with no assigned Node and decides where to run them.
• Algorithm (two phases):
  1. Filtering: Which nodes satisfy the pod's constraints?
     • Does the node have enough CPU and RAM?
     • Does it match nodeSelector or nodeAffinity rules?
     • Do the pod’s tolerations match applicable node taints?
     • Is it tainted to reject this pod?
  2. Scoring: Among feasible nodes, which is "best"?
     • Use configured scoring plugins, including resource requests and topology goals; instantaneous CPU use alone is not the scheduler’s capacity model.
     • Prefer nodes that already have the container image cached.
     • Apply custom priority functions.
  3. Binding: The scheduler binds the Pod to the winning Node by updating Etcd via the API Server.

D. Controller Manager

2. The Data Plane (Worker Nodes)

Where the actual application containers run. Each worker node has three components:

A. Kubelet

• Role: The agent that runs on every node (both master and worker).
• Function: Talks to the API Server every few seconds: "Do you have work for me?" → "Yes, ensure these Pods are running." → Tells the Container Runtime to start/stop containers.
• Pod lifecycle management: Runs liveness and readiness probes. If a container's liveness probe fails, Kubelet restarts it.

B. Kube-proxy

• Role: The networking agent. Runs on every node.
• Function: Maintains network rules (iptables or IPVS) that enable Services. When you create a Kubernetes Service, kube-proxy ensures that traffic to the Service's ClusterIP is routed to one of the Service's healthy Pods.
• Modes:
  • iptables (default): Creates iptables rules for load balancing. Simple but can get slow with thousands of services.
  • IPVS: Uses kernel-level IPVS for faster load balancing at scale.

C. Container Runtime

• Role: The engine that actually runs containers.
• containerd: A common CRI-compatible runtime, not a universal Kubernetes default. Kubernetes 1.24 removed the in-tree dockershim adapter. Docker-built images still work with compatible runtimes; Docker Engine can use an external adapter. Kubernetes runtime documentation.
• CRI-O: An alternative lightweight runtime designed specifically for Kubernetes.

3. The Complete Flow: kubectl run nginx

What actually happens when you run this command?

kubectl run nginx --image=nginx:latest

1. kubectl sends an HTTP POST request to the API Server with the Pod spec.
2. API Server validates the request (authentication, RBAC, admission controllers), then stores "Pending Nginx Pod" in Etcd.
3. Scheduler watches for unassigned Pods. It sees the new Pod, evaluates nodes (filter + score), picks "Node-1", and updates the Pod's nodeName in Etcd via the API Server.
4. Kubelet on Node-1 notices (via its API Server watch) that it has been assigned a new Pod.
5. Kubelet tells containerd to pull the nginx:latest image and start the container.
6. The configured network implementation provides Pod connectivity. kube-proxy or its replacement handles Service routing when a Service selects this Pod; kubectl run alone does not create a Service.
7. Kubelet reports Pod status as Running back to the API Server → Etcd is updated.

4. Self-Healing in Action

What happens when Node-1 crashes?

1. Node Controller stops receiving heartbeats from Node-1.
2. After the configured health grace period, its readiness condition changes.
3. Eviction timing depends on taints, tolerations and controller settings; it is not an unconditional five-minute deadline for every Pod.
4. ReplicaSet Controller detects that the desired replica count (3) > actual running replicas (2).
5. It creates a new Pod spec.
6. Scheduler assigns the new Pod to a healthy node (Node-2 or Node-3).
7. Kubelet on the chosen node starts the container.

The entire process is automatic — no human intervention needed.

5. Key Kubernetes Objects

Interview Tips 💡

1. Don't just say what Kubernetes is: Explain the control loop — "K8s continuously compares desired state (Etcd) with actual state (kubelet reports) and reconciles."
2. Know the scheduling algorithm: "Filter feasible nodes, score them, bind the best one."
3. Explain the flow: Walk through what happens when kubectl apply is run — API Server → Etcd → Scheduler → Kubelet → Container Runtime.
4. Discuss high availability: "API Server is stateless and load-balanced. Etcd runs as a 3-node Raft cluster. Workers are inherently redundant."
5. Networking: "Services provide stable IPs via kube-proxy (iptables/IPVS). Ingress controllers handle HTTP routing."

Summary

• API Server: Central hub — the only gateway to Etcd.
• Etcd: Source of truth — stores all cluster state.
• Scheduler: Decides where to run new Pods (filter → score → bind).
• Controllers: Decide what to do (self-healing via reconciliation loops).
• Kubelet: The worker agent — ensures Pods are running on each node.
• Kube-proxy: Networking — routes traffic to healthy Pods.

Failure exercise: a Pod is Pending but CPU looks idle

Check the scheduler’s event for that Pod before changing cluster size. A node can show low measured CPU while its allocatable resources are already reserved by other Pods’ requests. A taint, volume affinity or topology rule can also eliminate it. Compare requests with allocatable capacity, then inspect the specific failed constraint. Removing all constraints may make the Pod run but violate the availability design.

A Pod created directly with kubectl run has no Deployment controller to replace it after deletion. Use a workload controller when replacement is required; see Kubernetes Services for the separate networking abstraction.

Related Concepts

• Docker Internals — Container images and runtimes
• Service Discovery — How services find each other
• CI/CD Pipeline — Deploying to Kubernetes
• Blue-Green Deployment — Deployment strategies in K8s
• Observability Stack — Monitoring K8s clusters