Introduction
Services give clients one stable address for a set of Pods. That becomes especially valuable when demand changes: a Deployment can add or remove replicas while the Service identity stays the same.
In this lab, each backend returns its own Pod hostname. You will scale from two replicas to four, send independent requests through one Service, and see responses from multiple Pods. Then you will scale back to two and watch both the Deployment and EndpointSlice converge on the new desired state.
This is manual horizontal scaling. Automatic scaling with HPA depends on resource requests, metrics, and a control policy, so it belongs after manual replica behavior is fully understood.
Build an Observable Replicated Application
In this step, you will create two backends that reveal which Pod handled each request. This makes Service traffic distribution visible instead of abstract.
Environment startup: This lab starts a complete Kubernetes cluster for you. Configuring its control plane, node, and networking components usually takes 2–3 minutes. Please wait patiently for the environment to finish loading before running the commands below.
The first command uses cd to enter the prepared workspace. The next uses a here-document: cat <<'EOF' > hostname-web.yaml writes every following line through the closing EOF into the YAML file, replacing any previous contents.
cd /home/labex/project/scale-lab
cat <<'EOF' > hostname-web.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: hostname-web
spec:
replicas: 2
selector:
matchLabels:
app: hostname-web
template:
metadata:
labels:
app: hostname-web
spec:
containers:
- name: web
image: busybox:1.36
imagePullPolicy: IfNotPresent
command: ["sh", "-c"]
args:
- mkdir -p /www; hostname > /www/index.html; exec httpd -f -p 8080 -h /www
ports:
- name: http
containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: hostname-web
spec:
selector:
app: hostname-web
ports:
- name: http
port: 80
targetPort: http
EOF
kubectl apply -f hostname-web.yaml
kubectl rollout status deployment/hostname-web --timeout=60s
kubectl get pods -l app=hostname-web
After the closing EOF, kubectl apply -f sends every object in the file to the API server. rollout status waits up to 60 seconds for both desired Pods, and -l app=hostname-web lists only Pods with that label.
Inside the container command, semicolons run actions in sequence: create /www, redirect the hostname into index.html, then use exec to make the web server the container's main process. The two Pod names are different, so each backend returns a different page value.
Establish the Baseline Backends
In this step, you will connect the Deployment's replica count to the Service's ready backend count.
Read three related views. get deployment shows desired and ready replicas; -l selects application Pods and -o wide adds IP/node columns; the final label selector finds the EndpointSlice created for this Service:
kubectl get deployment hostname-web
kubectl get pods -l app=hostname-web -o wide
kubectl get endpointslices -l kubernetes.io/service-name=hostname-web
The Deployment shows 2/2, and the EndpointSlice shows two addresses. Request the Service several times from one long-running client Pod.
kubectl run creates a standalone Pod because --restart=Never is set. The -- separator ends kubectl options; sleep 3600 is the container command that keeps it alive. The for loop uses seq 1 6 to generate six iterations, and kubectl exec POD -- COMMAND runs wget inside the client each time:
kubectl run load-client --image=busybox:1.36 --image-pull-policy=IfNotPresent --restart=Never -- sleep 3600
kubectl wait --for=condition=Ready pod/load-client --timeout=30s
for i in $(seq 1 6); do kubectl exec load-client -- wget -qO- http://hostname-web; done
Each response is a Pod name. You may see one or both names in a short sample; Service distribution is not a promise of strict round-robin order.
Scale Up Declaratively
In this step, you will change the saved desired state from two replicas to four. Editing the manifest keeps the file and live object consistent.
sed -i 's/old/new/' file replaces matching text directly in a file. grep -n then searches for replicas: and shows its line number, giving you a quick check before applying the change:
cd /home/labex/project/scale-lab
sed -i 's/replicas: 2/replicas: 4/' hostname-web.yaml
grep -n 'replicas:' hostname-web.yaml
kubectl apply -f hostname-web.yaml
kubectl rollout status deployment/hostname-web --timeout=60s
kubectl get deployment hostname-web
The Deployment should show 4/4. The controller created two additional Pods because actual state was below the new desired state.
Watch the Service Backend Set Expand
In this step, you will verify that the unchanged Service automatically discovers the new Pods.
The EndpointSlice command uses JSONPath because the normal table may abbreviate details. range repeats the template for every endpoint; each repetition prints its first address, the literal word ready, the readiness value, and a newline. The backslash continues one shell command across display lines:
kubectl get pods -l app=hostname-web -o wide
kubectl get endpointslices -l kubernetes.io/service-name=hostname-web \
-o jsonpath='{range .items[*].endpoints[*]}{.addresses[0]}{" ready="}{.conditions.ready}{"\n"}{end}'
There are now four ready addresses. You did not edit the Service: its selector continued to match every ready Pod with app=hostname-web.
Compare the stable Service IP with the expanded backend set:
kubectl get service hostname-web -o wide
The ClusterIP stays stable while the EndpointSlice membership changes.
Observe Requests Reaching Multiple Pods
In this step, you will send independent HTTP requests through one Service and summarize which Pod hostnames respond.
First, rm -f removes an old result file if it exists; -f also makes a missing file harmless. The loop sends 20 requests. >> appends each hostname instead of replacing earlier results. Finally, the pipe sends sorted lines to uniq -c, which collapses adjacent duplicates and prefixes each hostname with its count:
rm -f /tmp/hostname-responses.txt
for i in $(seq 1 20); do
kubectl exec load-client -- wget -qO- http://hostname-web >> /tmp/hostname-responses.txt
done
sort /tmp/hostname-responses.txt | uniq -c
You should see more than one hostname. Counts may be uneven, and not every short run must hit all four backends. Kubernetes Service routing distributes connections; it does not guarantee a perfectly even or ordered sequence.
Confirm how many unique backends your sample reached. sort -u keeps one copy of each hostname, the pipe sends those lines to wc -l, and wc -l counts lines:
sort -u /tmp/hostname-responses.txt | wc -l
A value greater than one is direct evidence that the stable Service address routed requests to multiple Pods.
Scale Down Imperatively
In this step, you will use kubectl scale for a quick live adjustment from four replicas back to two.
kubectl scale changes the live Deployment's desired replica count immediately. --replicas=2 supplies the new count; it does not edit your YAML file:
kubectl scale deployment/hostname-web --replicas=2
kubectl rollout status deployment/hostname-web --timeout=60s
kubectl get pods -l app=hostname-web
Kubernetes terminates two Pods and retains two. Wait until the Service backend set also converges.
This bounded polling loop tries at most 30 times. Each pass stores the ready endpoint count in count; jq filters EndpointSlice JSON to ready endpoints and returns the array length. [ "$count" -eq 2 ] is a numeric shell test, && break exits the loop when it succeeds, and sleep 1 pauses before retrying:
for i in $(seq 1 30); do
count=$(kubectl get endpointslices -l kubernetes.io/service-name=hostname-web -o json | jq '[.items[].endpoints[] | select(.conditions.ready == true)] | length')
[ "$count" -eq 2 ] && break
sleep 1
done
echo "Ready backends: $count"
The live Deployment now requests two replicas, but the file still says four. That difference is intentional for the next step.
Reconcile the Manifest and Read the Controller Trail
In this step, you will make the saved manifest agree with the live two-replica state and connect scaling actions to controller evidence.
First compare the two desired states. grep -n shows the saved line; JSONPath extracts only the live .spec.replicas field and adds a newline:
grep -n 'replicas:' /home/labex/project/scale-lab/hostname-web.yaml
kubectl get deployment hostname-web -o jsonpath='Live replicas: {.spec.replicas}{"\n"}'
Change the file from four back to two and apply it:
sed -i 's/replicas: 4/replicas: 2/' /home/labex/project/scale-lab/hostname-web.yaml
kubectl apply -f /home/labex/project/scale-lab/hostname-web.yaml
Because live state already has two replicas, this apply should not create more Pods. Read the Deployment events. The pipe passes the full describe output to sed -n; /Events:/,$p means “print from the line containing Events: through the end”:
kubectl describe deployment hostname-web | sed -n '/Events:/,$p'
Look for ScalingReplicaSet messages showing scale-up and scale-down decisions. Finally remove the temporary client. --ignore-not-found makes cleanup succeed even if the Pod has already disappeared:
kubectl delete pod load-client --ignore-not-found
Manual scaling changes desired replica count; the Deployment controller creates or terminates Pods, and the Service automatically tracks ready members. Keeping the manifest synchronized prevents a later kubectl apply from unexpectedly restoring an old count.
Summary
You manually scaled a Deployment from two replicas to four and back to two. You watched the controller reconcile Pods, observed EndpointSlice membership follow ready backends without changing the Service, and gathered direct evidence that one Service address routed independent requests to multiple Pods.
The durable mental model is: replica count is desired state, the Deployment controller reconciles actual Pods, and the Service follows ready labeled backends. Declarative files should be reconciled with deliberate live changes so future applies remain predictable.
