• English icon
  • German icon
Explainer

My pods are Running. Why does the Service answer nothing?

· Series: Containers on Azure

azureakskuberneteskubernetes-servicekubectldevops

Both pods said Running, one of one ready. The Service had a public IP. And every call I made to that IP timed out: ten in a row, with no error in any status.

This explainer breaks a Kubernetes Service on Azure Kubernetes Service twice, on my own cluster, and shows what really connects a Service to its pods: the labels it selects, the port it calls, and the address you call it on.

Key points

  • A Service follows labels, not names. It never looks at your Deployment. Its selector is a list of labels, and only pods that carry every one of them, with the same values, go on its endpoint list.
  • One missing label is silent. With no matching pod, the endpoint list is empty: the pods are Running and ready, the Service has its IP, and no warning or event tells you anything.
  • Nobody checks the target port. port is what callers use, targetPort is where the pod listens. A wrong target port keeps the pods on the list, and calls get "connection refused", then "timed out" once the Azure load balancer's own check gives up on the node.
  • Inside the cluster, call the name; from outside, the load balancer. ClusterIP gives a name and an internal address, NodePort adds a port on every node (which has no public IP on AKS), LoadBalancer adds the public IP.

Labels, not names

A Service is one stable address in front of your pods, and it passes each call on to one of them. Most people picture it pointing at a Deployment. It doesn't: it has a selector, a short list of labels. Labels are name tags, a key and a value, that the pod template puts on every pod. Kubernetes compares each pod's labels with the selector; a pod that carries all of them goes on the Service's endpoint list (its address and port), and the Service only sends calls to the ready pods on that list.

For the run I took the Service from the Microsoft Learn module, which selects two labels, app and version, and put it in front of my own deployment web, whose pod template only sets app: web:

# Service
spec:
type: LoadBalancer
selector:
app: web
version: v1
ports:
- port: 80
targetPort: http
---
# Deployment: the pod template (the broken version)
template:
metadata:
labels:
app: web

Everything looked fine:

> kubectl get pods,svc -l app=web
NAME READY STATUS RESTARTS AGE
pod/web-6555d997fb-vpl2j 1/1 Running 0 53s
pod/web-6555d997fb-xhnn2 1/1 Running 0 53s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/web LoadBalancer 10.0.160.151 203.0.113.12 80:31713/TCP 53s

The Service's own description showed the problem, and nothing else did:

> kubectl describe svc web
...
Selector: app=web,version=v1
Type: LoadBalancer
...
TargetPort: http/TCP
NodePort: http 31713/TCP
Endpoints:
...

The Learn module shows Endpoints: <none> for this case; with kubectl 1.36 the line was simply empty. The pods' labels:

> kubectl get pods -l app=web --show-labels
NAME ... LABELS
web-6555d997fb-vpl2j ... app=web,pod-template-hash=6555d997fb,...
web-6555d997fb-xhnn2 ... app=web,pod-template-hash=6555d997fb,...

No version. One missing label, and neither pod made the list. The fix is one line in the pod template, version: v1. A changed template means new pods, so the Deployment rolled out two new ones with both labels and removed the old ones:

> python watch.py up web --secs 180
21.5s web-654b6b7b85-sjwzj Running 1/1
22.0s svc web ready endpoints 1
32.0s GET http://203.0.113.12/ 200 web-654b6b7b85-sjwzj

Two ports

A Service has two ports. port is what callers use (80). targetPort is where the pod listens. In my manifest the target port is a name, http, and the container gives its port that name:

# Deployment: the container
ports:
- name: http
containerPort: 8000

So the Service follows the container. Then I broke it on purpose: targetPort: 8080, a number you see in many examples, the module's too, on an app that listens on 8000. The labels still matched, so both pods stayed on the list. Four calls, three seconds apart:

> python watch.py get web --tries 4 --gap 3
2.4s GET http://203.0.113.12/ connection refused
7.7s GET http://203.0.113.12/ connection refused
12.8s GET http://203.0.113.12/ connection refused
20.8s GET http://203.0.113.12/ timed out

Refused means the call reached a pod and nothing listened on 8080; Kubernetes never checks that port. The timeout comes from the Azure load balancer. Your pods run on nodes, the virtual machines of the cluster; for a Service with a public IP, every node opens a port of its own, the node port, and passes what arrives there on to the target port. The load balancer checks that node port on every node, every five seconds by default (with externalTrafficPolicy: Cluster, AKS's default probe mode). Its check lands on the same wrong port and fails, and after two misses the load balancer stops sending anything to the node.

The same check explains the first break: with an empty endpoint list the check failed too, so those calls only timed out. And after the label fix, the first answer came ten seconds after the pod was ready: the check had to pass again first. Back to targetPort: http, and the app answered again after about eight seconds.

Give the target port the container port's name, and the Service follows your container.

Which address

The Service type decides who can reach it, and the three types build on each other:

TypeWhat it addsReachable from
ClusterIP (the default)an internal address and a DNS nameinside the cluster
NodePorta port on every node, 30000–32767the network the nodes are in
LoadBalanceran Azure load balancer with a public IP in front of the node portsthe internet

I added a ClusterIP and a NodePort Service for the same pods:

> kubectl get svc
NAME TYPE EXTERNAL-IP PORT(S)
web LoadBalancer 203.0.113.12 80:31713/TCP
web-internal ClusterIP <none> 80/TCP
web-np NodePort <none> 80:30412/TCP

Inside the cluster you call a Service by its name. From one of my own pods:

> kubectl exec deploy/web -- python get.py http://web-internal
200 {"pod": "web-654b6b7b85-sjwzj", "version": "v1"}

The full name web-internal.default.svc.cluster.local answered too. No port in the address means 80, the Service's port. The module's example puts the target port in the address (…svc.cluster.local:8080); callers always use the Service's port.

The module also says: reach a NodePort from outside on the node's IP and the node port. My node had no public address:

> kubectl get nodes -o wide
NAME INTERNAL-IP EXTERNAL-IP ...
aks-nodepool1-24214868-vmss000000 10.224.0.4 <none> ...

AKS nodes get no public IP unless you turn it on (--enable-node-public-ip). A node port works inside the network, and it's the door the load balancer uses. Inside the cluster, call the name. From outside, call the load balancer. For public HTTP apps, Microsoft recommends an ingress controller or the Gateway API in front of ClusterIP Services instead of one public IP per Service.

The one line to read first

The endpoints line of kubectl describe service doesn't only tell you whether the list is empty; it shows the port. When my target port was wrong, I listed the same list directly, where Kubernetes keeps it, in endpoint slices:

> kubectl get endpointslices -l kubernetes.io/service-name=web
NAME ADDRESSTYPE PORTS ENDPOINTS AGE
web-cdxkl IPv4 8080 10.244.0.154,10.244.0.165 110s

Two pod addresses, and port 8080 for an app on 8000. An empty list: check the labels. The wrong port: check the target port. One line, both mistakes.

What happened when I ran it

Tested on October 26, 2026 in my own Azure subscription (AKS 1.36 in westeurope, Azure CLI 2.88.0, kubectl 1.36); everything was deleted afterwards. One AKS cluster served four videos of this series; this page uses its Service steps. The cluster had one Standard_D2s_v5 node and the free tier; the app is a small Python API shaped like the module exercise's (port 8000, /healthz, /readyz), without the exercise's model in Microsoft Foundry.

  • With the missing label, the watcher stopped after "10 failed GETs in a row while the pods are ready" at 52 seconds; the pods had been ready since 22.8 seconds.
  • kubectl describe svc web showed only two Normal events (EnsuringLoadBalancer, EnsuredLoadBalancer): nothing about the empty selector match.
  • After the label fix the Deployment rolled out pods with the new template hash (654b6b7b85); the first new pod was ready at 21.5 seconds and on the endpoint list at 22.0.
  • With targetPort: 8080, the change from refused to timed out happened between about 13 and 16 seconds into the watcher, which started right after the apply.

What the video leaves out

  • ExternalName Services, network policies and kube-proxy or DNS failures (Microsoft's troubleshooting guide covers them).
  • externalTrafficPolicy: Local and the shared health probe mode of the AKS load balancer (preview), which change how the load balancer checks nodes.
  • Debug pods (kubectl run) and ephemeral debug containers; the run used kubectl exec into the app's own pod.

Further reading

Video transcript

Both pods said Running, one of one ready. The Service had a public IP. And every call I made to that IP timed out. Ten calls in a row, and not one error in any status. So where did my requests go?

In this video, you'll see how a Service on AKS finds its pods, which port it calls, and which address you call. And at the end: the one line I should have read first.

How does a Service find its pods? A Service is one stable address in front of your pods, and it passes each call on to one of them. Most people picture a Service pointing at a Deployment, the object that creates your pods, the small wrappers your containers run in. It doesn't. A Service never looks at the Deployment at all. It has a selector, a short list of labels. Labels are name tags, a key and a value, that the pod template, the stencil for new pods, puts on every pod. Kubernetes compares each pod's tags with the selector. A pod that carries all of them, with the same values, goes on the Service's endpoint list: its address and its port. The Service only sends calls to the ready pods on that list. For my run, I took the Service from the Microsoft Learn module, which selects two labels, app and version, and put it in front of my own deployment, called web. Then I asked Kubernetes for everything with the label app equals web: the pods and the service. Two pods, both one of one ready: their container passed its readiness check. The Service, with its public IP. Everything looks fine. So both pods are ready. How many of them get traffic from the Service? Zero. The endpoint list was empty. I asked Kubernetes to describe the service: its selector, its ports, and the endpoints it found. The selector wants app equals web and version v1. And the endpoints line: empty. No warning, no event, nothing red. Then I listed the two pods again, this time with every label they carry. App equals web, and a few labels Kubernetes added on its own. No version. One tag missing, and neither pod made the list. The fix is one line in the deployment's pod template: version v1. A changed template means new pods, so the deployment rolled out two new ones with both tags, and removed the old ones. I applied the fixed deployment and started a small script of mine. It watches the pods and the Service, calls the public IP, and prints only what changes. At twenty-two seconds, the first new pod was ready and on the list. At thirty-two, the first answer. A Service doesn't follow the deployment. It follows labels, and every label must match.

Which port does the Service call on your pod? A Service has two ports. The first, port, is what callers use: eighty. The second, target port, is where the pod listens. In my manifest, the target port isn't a number. It's a name: http. The container gives its port that name, and the port is eight thousand. So the Service follows the container. Then I broke it on purpose. I set the target port to eighty eighty, a number you see in many examples, the module's too. My app listens on eight thousand. The labels still match, so both pods stay on the list. What does a call get now? Connection refused. Three times. I called the public IP four times, three seconds apart, with the same script. Refused, refused, refused. Then: timed out. Refused means the call reached a pod, and nothing was listening on eighty eighty. Kubernetes never checks that port. This series runs one Azure service at a time, for real. Subscribe to follow the series. So why the timeout? Your pods run on nodes, the virtual machines of the cluster. For a Service with a public IP, like mine, every node opens a port of its own, the node port, and passes what arrives there on to the target port. The Azure load balancer checks that node port on every node, every five seconds. Its check lands on the same wrong port and fails. After two misses, the load balancer stops sending anything to the node. I showed that check in the last video. The same check explains my first break. With an empty list, the check failed too, so those calls only timed out. And after the fix, the first answer came ten seconds after the pod was ready: the check had to pass again first. Back to the name http, and eight seconds later, the app answered again. Give the target port the container port's name, and the Service follows your container.

Which address do you actually call? The Service type decides who can reach it, and the three types build on each other. ClusterIP, the default, gives the Service an internal address and a name, inside the cluster only. Every other type keeps that name. NodePort adds the node port on every node, a number between thirty thousand and thirty-two thousand seven hundred sixty-seven. And LoadBalancer adds an Azure load balancer with a public IP in front of those node ports. I added a ClusterIP and a NodePort service for the same pods, next to the LoadBalancer, and listed all services. Only the LoadBalancer has an external IP. The NodePort service got port thirty thousand four hundred twelve. Inside the cluster, you call a Service by its name. From inside one of my own pods, I called the ClusterIP service by its short name, with no port. Two hundred, from one of the pods behind it. The full name answered too: web-internal, then the namespace, default, then svc dot cluster dot local. No port in the address means eighty, the Service's port. Don't put the target port in the address, like the module's example does with eighty eighty. Callers use the Service's port. Now the module's advice for NodePort: from outside, call the node's IP and the node port. So what is my node's public IP? There isn't one. I asked Kubernetes for my node, with its internal and its external address. Internal: ten dot two two four dot zero dot four. External: none. AKS nodes get no public IP unless you turn it on. A node port works inside the network, and it's the door the load balancer uses. From the internet, you call the load balancer's IP. Inside the cluster, call the name. From outside, call the load balancer.

To sum it up: a Service finds pods by their labels, and every label in the selector must match. It calls them on the target port, and Kubernetes never checks that your app listens there. And on AKS, you call it by its name inside the cluster, or on the load balancer's IP from outside.

And the one line I should have read first? The endpoints line of kubectl describe service. It doesn't only tell you whether the list is empty. It shows the port. When my target port was wrong, I asked for that list directly. Kubernetes keeps it in endpoint slices, one set per Service. Two pod addresses, and port eighty eighty. My app listens on eight thousand. Empty list: check the labels. Wrong port: check the target port. One line, both mistakes.

The video on screen now shows what happens between kubectl apply and the first answer, second by second. If this was useful, subscribe for more practical .NET and Azure.

Need this in your own Azure environment?

I build, migrate and run .NET platforms on Azure, with Azure DevOps and Terraform. If your team wants this set up properly, see what I offer.

See services