Workloads & Scheduling

Deployments, StatefulSets, DaemonSets — and how pods land on nodes

Deployment

A Deployment manages ReplicaSets, which manage Pods. It's the standard way to run stateless applications. The key value is automated rolling updates with rollback capability.

1. Deployment

   Desired state
   image, replicas, strategy

2. ReplicaSet

   Ensures N replicas
   one per revision

3. Pod

   Actual running
   container(s)

Rolling Update Strategy

When you update a Deployment (e.g., change the image tag), it creates a new ReplicaSet, scales it up gradually while scaling the old one down.

# Deployment YAML with key fields highlighted
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
  namespace: production
spec:
  replicas: 3                           # desired pod count
  revisionHistoryLimit: 10                # old ReplicaSets to keep (for rollback)
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1                        # 1 extra pod during update
      maxUnavailable: 0                  # no downtime — always 3 available
  selector:
    matchLabels:
      app: web-app
  template:                               # pod template
    metadata:
      labels:
        app: web-app
    spec:
      containers:
        - name: web
          image: myapp:v2.1.0             # change this to trigger rollout
          ports:
            - containerPort: 8080
          resources:
            requests:
              cpu: "250m"
              memory: "256Mi"
            limits:
              cpu: "500m"
              memory: "512Mi"

StatefulSet

StatefulSets are for stateful applications that need stable identities and persistent storage — databases, message queues, distributed systems like etcd itself.

Ordered Operations

1. pod-0

   Created first
   Deleted last

2. pod-1

   Created second
   After pod-0 is Ready

3. pod-2

   Created third
   After pod-1 is Ready

• Scale up: pods created in order 0 → 1 → 2, each waiting for the previous to be Running and Ready
• Scale down: pods deleted in reverse order 2 → 1 → 0
• Opt out: set podManagementPolicy: Parallel to create/delete all at once

Use cases: PostgreSQL (primary/replica topology), Kafka (broker IDs), etcd (peer discovery), ZooKeeper (ensemble membership), Redis Sentinel, Elasticsearch

# StatefulSet YAML
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres
spec:
  serviceName: postgres-headless        # required headless service
  replicas: 3
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
        - name: postgres
          image: postgres:16
          volumeMounts:
            - name: data
              mountPath: /var/lib/postgresql/data
  volumeClaimTemplates:                  # one PVC per pod
    - metadata:
        name: data
      spec:
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: "50Gi"
# Creates PVCs: data-postgres-0, data-postgres-1, data-postgres-2

DaemonSet

A DaemonSet ensures that exactly one pod runs on every node (or every node matching a selector). When nodes are added, the DaemonSet automatically creates pods on them. When nodes are removed, those pods are garbage collected.

Job & CronJob

Scheduling Mechanisms

Kubernetes provides multiple mechanisms to control where pods are placed. They range from simple label matching to complex topology-aware distribution.

nodeSelector (Simple)

The simplest scheduling constraint — pod runs only on nodes with matching labels.

# Simple node selector — pod only runs on SSD nodes
spec:
  nodeSelector:
    disk: ssd
    region: us-east-1

Node Affinity (Expressive)

More powerful than nodeSelector — supports operators (In, NotIn, Exists, DoesNotExist, Gt, Lt) and soft/hard rules.

affinity:
  nodeAffinity:
    requiredDuringScheduling
    IgnoredDuringExecution:
      nodeSelectorTerms:
        - matchExpressions:
            - key: zone
              operator: In
              values:
                - us-east-1a
                - us-east-1b

affinity:
  nodeAffinity:
    preferredDuringScheduling
    IgnoredDuringExecution:
      - weight: 80
        preference:
          matchExpressions:
            - key: gpu-type
              operator: In
              values:
                - a100

Pod Affinity / Anti-Affinity

Schedule pods relative to other pods, not just node labels. topologyKey defines what "near" means.

podAffinity:
  requiredDuringScheduling
  IgnoredDuringExecution:
    - labelSelector:
        matchLabels:
          app: redis-cache
      topologyKey: kubernetes.io/hostname

podAntiAffinity:
  requiredDuringScheduling
  IgnoredDuringExecution:
    - labelSelector:
        matchLabels:
          app: web-app
      topologyKey: topology.kubernetes.io/zone

Taints and Tolerations

Taints are applied to nodes to repel pods. Tolerations are applied to pods to allow scheduling on tainted nodes. This is the inverse of affinity — instead of attracting pods, taints repel them.

          $ kubectl taint nodes gpu-node-1 \

            nvidia.com/gpu=true:NoSchedule
        

tolerations:
  - key: "nvidia.com/gpu"
    operator: "Equal"
    value: "true"
    effect: "NoSchedule"

Topology Spread Constraints

Distribute pods evenly across failure domains (zones, nodes, racks). More granular than pod anti-affinity.

topologySpreadConstraints:
  - maxSkew: 1                            # max difference between domains
    topologyKey: topology.kubernetes.io/zone
    whenUnsatisfiable: DoNotSchedule       # or ScheduleAnyway
    labelSelector:
      matchLabels:
        app: web-app

PriorityClass and Preemption

Assign priority values to pods. Higher-priority pods can preempt (evict) lower-priority pods when cluster resources are scarce.

# Define a PriorityClass
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: high-priority
value: 1000000                           # higher number = higher priority
preemptionPolicy: PreemptLowerPriority    # or Never
globalDefault: false
description: "For critical production workloads"
---
# Use in a pod spec
spec:
  priorityClassName: high-priority

Scheduling Decision Flow

Putting it all together — here is the complete flow the scheduler follows for every unscheduled pod.

1. Unscheduled Pod

   Scheduler detects pod
   with no nodeName

2. Filter

   Eliminate nodes:
   resources, selectors,
   taints, affinity

3. Score

   Rank nodes 0-100:
   least requested,
   balanced, topology

4. Bind

   Assign pod to
   highest-scoring node

5. Kubelet

   Picks up pod,
   calls CRI, starts
   containers