Who is Ehsan Tatasadi?

Ehsan Tatasadi is a freelance Senior .NET & Azure Engineer based in Hamburg, Germany. He builds and modernizes .NET platforms on Azure, connects them with Dynamics 365, and runs them in production. He has 8+ years on the Microsoft stack, more than 7 of them for a DAX-listed energy utility and a regional grid operator.

What technologies does Ehsan Tatasadi specialize in?

C#, .NET 8 and ASP.NET Core with SQL Server; Azure (App Service, Functions, Key Vault, Private Endpoints, Application Insights); Dynamics 365 CE integration via the XRM SDK, C# plugins and JavaScript form scripts; Terraform and Azure DevOps YAML pipelines; React and TypeScript. He is certified in AZ-400, AZ-204, AI-103 and GH-300.

Where is Ehsan Tatasadi located?

Ehsan Tatasadi is based in Hamburg, Germany. He works remotely, on-site in Hamburg, and on-site anywhere in Germany on demand.

Explainer

My pod won't start. Which command shows why?

November 5, 2026 · Series: Containers on Azure

azureakskuberneteskubectltroubleshootingdevops

I broke four pods on Azure Kubernetes Service on purpose. Two said ImagePullBackOff, one said CreateContainerConfigError, and one crashed four times in a minute and a half. Every time the question was the same: which command shows you why?

This explainer follows the line every pod walks before your code runs, shows where each status stops it, and which command finds the reason at each stop, with the real output from my own cluster.

Key points

The status names the stop. The kubelet pulls the image, fills in the container's settings (environment variables and Secrets), then starts the container. ImagePullBackOff stops at the first, CreateContainerConfigError at the second, CrashLoopBackOff after the third: your app ran and died.
BackOff means waiting, not giving up. Kubernetes keeps trying, and the wait between two tries grows up to five minutes. A fix (a pushed tag, registry access, a created Secret) is picked up on the next try, without a redeploy.
Before your app runs, only Kubernetes knows the reason. kubectl describe pod ends with the events. Read the first error: a wrong tag also prints 401 Unauthorized, because AKS always tries an anonymous pull as well.
After your app ran, its own logs have the answer. In a crash loop, plain kubectl logs shows the container that just crashed; --previous only helps once the container runs again. The RESTARTS column picks the command: zero and not running → describe, more than zero → logs.

Stuck at the image

A pod is the small wrapper Kubernetes runs your container in. Before your code runs, the kubelet, the agent on every machine of the cluster, pulls the container image from the registry, fills in the container's settings, and starts the container. The status in kubectl get pods names the stop where the pod is waiting.

My first break was a typo: the tag v1.0, the tag in the Microsoft Learn module's example, while my registry only had v1. A small script of mine printed every status change with the second it happened, like kubectl get pods --watch with timestamps:

> python watch.py pods api-typo --secs 75
0.4s  api-typo-…  ContainerCreating 0/1
1.1s  api-typo-…  ErrImagePull 0/1
16.4s  api-typo-…  ImagePullBackOff 0/1
27.4s  api-typo-…  ErrImagePull 0/1
39.0s  api-typo-…  ImagePullBackOff 0/1

ErrImagePull means a try just failed; ImagePullBackOff means Kubernetes is waiting before the next try, and each wait is longer than the last (up to five minutes). Nine and a half minutes later the pod still said ImagePullBackOff, with zero restarts. The tries never stop.

Your app never ran, so it has nothing to say. Kubernetes does: kubectl describe pod ends with the events, what the kubelet did and what failed:

> kubectl describe pod -l app=api-typo
Status:  Pending
Reason:  ImagePullBackOff
Restart Count:  0
Failed to pull image "…/aks-api:v1.0": … not found, … 401 Unauthorized

Unauthorized doesn't mean permissions here. The first error is the real one: not found. AKS always tries a pull without credentials as well, and that try gets the 401 (Microsoft's troubleshooting guide says so for wrong image names). Check the image name and the tag against your registry; fix the tag in your YAML and apply it again, or push the missing tag, which Kubernetes picks up on its next try.

My second break used the same image in a second registry the cluster was never given access to. Now the event had the 401 and nothing else:

> kubectl describe pod -l app=api-private
Reason:  ErrImagePull
Restart Count:  0
Failed to pull image "otherregistry…": … 401 Unauthorized

The module calls registry access "usually automatic". It isn't: the cluster pulls images with its own identity (the kubelet's managed identity), and that identity needs the AcrPull role on each registry. My first registry had it only because I attached it when I created the cluster. For an existing cluster:

az aks update -g rg-demo -n aks-demo --attach-acr otherregistry

That took 106 seconds. I didn't redeploy and didn't touch the pod; 32 seconds after I started watching, the app was running.

Stuck at the settings

The module's manifest reads an API key from a Secret, a Kubernetes object that holds keys and passwords:

env:
- name: MODEL_NAME
  value: "gpt-5-mini"
- name: API_KEY
  valueFrom:
    secretKeyRef:
      name: api-secrets     # never created
      key: api-key

The module lists Secrets under CrashLoopBackOff, as if a missing one makes the app crash. It doesn't: the kubelet fills in the settings before the container exists, and a required Secret that's missing stops it right there. The status was CreateContainerConfigError after half a second, and then nothing: no crash, no restart. The container was never created, so there are no logs:

> kubectl logs deploy/api-secret
Error from server (BadRequest): container "api" in pod "…"
is waiting to start: CreateContainerConfigError

The reason is in the events again:

> kubectl describe pod -l app=api-secret
Reason:  CreateContainerConfigError
Restart Count:  0
API_KEY:  <set to the key 'api-key' in secret 'api-secrets'>
Failed (x5 over 42s)  Error: secret "api-secrets" not found

The fix is the Secret itself (kubectl create secret generic api-secrets --from-literal=api-key=demo). Nine seconds later the same pod was running, without a redeploy or a restart.

The app ran, and died

My app needs one setting, MODEL_NAME; without it, it prints an error and exits with code 1. I deployed it without that setting and watched it for a hundred seconds:

> python watch.py pods api-crash --secs 100
1.2s  api-crash-…  Running 1/1
1.9s  api-crash-…  Error 0/1
2.6s  api-crash-…  Running 1/1  restarts 1
4.1s  api-crash-…  Error 0/1  restarts 1
18.0s  api-crash-…  CrashLoopBackOff 0/1  restarts 1
94.2s  api-crash-…  Running 1/1  restarts 4

This time the container did start, so Kubernetes restarts it: the first restart comes at once, then the wait doubles (10 s, 20 s, 40 s, up to five minutes; it resets after ten minutes without problems). My restarts came a little later than that, but further and further apart. CrashLoopBackOff means Kubernetes is sitting out one of those waits.

This time your app ran, so it has something to say:

> kubectl logs deploy/api-crash
error: MODEL_NAME is not set

Most guides, the module too, say: for a crash loop, ask for the previous container's logs. Right after the plain logs, that gave me an error of its own:

> kubectl logs deploy/api-crash --previous
unable to retrieve container logs for containerd://74a2e527a39d…

kubectl describe pod shows why: the pod had two terminated containers, the current one and the one before.

State:  Terminated
Exit Code:  1
Last State:  Terminated
Exit Code:  1
Restart Count:  4

While a crashed container waits for its restart, it is still the pod's latest container, so plain kubectl logs shows it. --previous asks for the one before, and the kubelet had already cleaned that one up (it garbage-collects dead containers). Use --previous once the container is running again, when the crash is behind it.

The one column that picks the command

kubectl get pods has a RESTARTS column. At the end of my run:

> kubectl get pods
NAME  READY  STATUS  RESTARTS  AGE
api-crash-565c6974d4-r5xzq  0/1  Error  4 (116s ago)  2m37s

The pods that never got to run their app had zero restarts in every check. Zero restarts and not running: your app never ran, so ask Kubernetes with kubectl describe pod. More than zero: it ran and died, so ask your app with kubectl logs.

Status	Where it stops	Restarts	Read
ErrImagePull / ImagePullBackOff	pulling the image	0	`describe` → events (the first error)
CreateContainerConfigError	filling in the settings (a missing Secret or ConfigMap)	0	`describe` → events
CrashLoopBackOff	after the start: the app exited	> 0	`logs` (plain first, `--previous` once it runs again)

What happened when I ran it

Tested on October 26, 2026 in my own Azure subscription (AKS 1.36 in westeurope, Azure CLI 2.88.0, kubectl 1.36); everything was deleted afterwards. One AKS cluster served four videos of this series; this page uses its pod status steps. The cluster had one Standard_D2s_v5 node and the free tier; the app is a small Python API shaped like the module exercise's (port 8000, MODEL_NAME required), without the exercise's model in Microsoft Foundry.

During the attach of the second registry, Kubernetes replaced the waiting pod with a new one of the same ReplicaSet (a new pod name; the run doesn't show why). No redeploy was needed either way: the app ran 31.7 seconds after the watcher started.
My first try of the crash and Secret steps never got that far: both pods stayed Pending with 0/1 nodes are available: 1 Insufficient cpu, because my own earlier deployments had filled the node's CPU requests. That's the next video's topic (requests and limits); after cleaning up, the rerun showed the statuses above.
In the crash loop the restarts came at 2.6, 18.7, 40.4 and 94.2 seconds of the watcher.

What the video leaves out

Pending pods and Insufficient cpu (requests and limits: the next video).
Image pull secrets (imagePullSecrets), ABAC-enabled registries (where --attach-acr doesn't assign the right role) and image architecture mismatches, which also show 401 Unauthorized (Microsoft's troubleshooting guide covers them).
Liveness probes as a cause of restarts, kubectl get events and kubectl debug.

Links

Watch on YouTube

Video transcript

I broke four pods on Azure Kubernetes Service, on purpose. Two said ImagePullBackOff. One said CreateContainerConfigError. And one crashed four times in a minute and a half. Three statuses, and the same question every time: which command shows you why?

In this video, you'll see the three places a pod's start can stop, and which command finds the reason at each. And at the end: the one column that picks the command for you.

What does ImagePullBackOff actually tell you? A pod is the small wrapper Kubernetes runs your container in, and every pod walks the same line before your code runs. The kubelet, the agent on every machine of the cluster, pulls the container image from the registry. It fills in the container's settings: environment variables, and Secrets for keys and passwords. Then it starts the container. An earlier video timed this line second by second; the card is up there now. The status column of kubectl get pods, the Kubernetes command line, names the stop where the pod is waiting. My first break was a typo. I gave the image the tag v1.0, the tag in the example of the Microsoft Learn module I followed. My registry only had v1. I watched the pod with a small script of mine. Like kubectl get pods in watch mode, it prints every status change, plus the second it happened. ErrImagePull after one second. ImagePullBackOff at sixteen. Then back and forth. ErrImagePull means a try just failed. BackOff means Kubernetes is waiting before the next try, and each wait is longer than the last. So will Kubernetes ever give up on this image? No. It keeps trying. Nine and a half minutes later, I listed my pods again, with their status and their restarts. Still ImagePullBackOff. The wait between two tries grows up to five minutes, but the tries never stop. So where is the reason? Your app never ran, so it has nothing to say. Kubernetes does: kubectl describe pod ends with the events, what the kubelet did and what failed. I asked Kubernetes to describe the pod, and went to the events at the bottom. Failed to pull image: not found. And in the same line: 401, Unauthorized. Is that 401 a permission problem? Here it isn't. AKS, Azure Kubernetes Service, always tries a pull without credentials as well, and that try gets the 401. The first error is the real one: not found. So check the image name and the tag against your registry. Fix the tag in your YAML and apply it again, or push the missing tag to the registry: Kubernetes picks that up on its next try, at most five minutes later. My second break: the same image, in a second registry the cluster was never given access to. I described that pod too, and went straight to the events at the bottom of its output. Four oh one, Unauthorized, and nothing else. No not found. Now it really is access. So why did the first registry work? The module says registry access is usually automatic. It isn't. The cluster pulls images with its own Azure identity, and that identity needs permission on each registry. My first registry had it only because I attached it when I created the cluster. So I attached the second one too: az aks update with the attach ACR option, ACR for Azure Container Registry. A hundred and six seconds, and the cluster could pull from it. Then I only watched with my script. I didn't redeploy, and I didn't touch the pod. Thirty-two seconds after I started watching, the app was running. Stuck at the image: your app never ran. Read the events, and trust the first error.

What happens when a Secret doesn't exist? The module's YAML for this app reads an API key from a Secret. The pod names the Secret, api-secrets, and the key inside it. The module lists missing Secrets under CrashLoopBackOff, the status of an app that keeps crashing. So I deployed it without creating the Secret. I watched this pod for forty seconds, waiting for the first crash. CreateContainerConfigError after half a second. Then nothing. No crash at all. So what does kubectl logs show? Nothing from my app. I asked for the pod's logs, the way you would for any app that misbehaves. Bad request: the container is waiting to start. Logs are what your app prints. The image was pulled just fine, but the kubelet fills in the settings before the container exists, and a missing Secret stops it right there. The container was never created, so the app never printed a thing. This series runs one Azure service at a time, for real. Subscribe to follow the series. So the reason is in the events again. I described the pod and went to the bottom. Secret api-secrets not found. Five times in forty-two seconds: the kubelet kept trying. The fix is the Secret itself. I created it with one kubectl command, with a demo value for the key. Secret created. No restart, no new deployment. Then I watched the same pod again, without touching anything else. Nine seconds later, it was running. Stuck at the settings: the app never started, so there are no logs. The reason is in the events.

And when the app itself is the problem? My app needs one setting, the model name. Without it, it prints an error and exits. So I deployed it without that setting. I watched this pod for a hundred seconds, and counted the restarts. Running after one second, then Error. Running, Error. CrashLoopBackOff at eighteen seconds, and the fourth restart at ninety-four. This time the container did start, so Kubernetes restarts it. The first restart comes at once. After that, the wait doubles: ten seconds, twenty, forty, up to five minutes. In my run, the restarts came a little later than that, but further and further apart. CrashLoopBackOff means Kubernetes is sitting out one of those waits. And this time your app ran, so it had something to say. I asked for its logs. Error: MODEL_NAME is not set. The reason, in the app's own words. For a crash loop, most guides, the module too, say: ask for the previous container's logs. So what did that give me? An error of its own. I ran kubectl logs with previous, on the same pod, right after the plain logs. Unable to retrieve container logs. Describe shows the two containers. I asked for the pod's container states: the current one, and the one before. State: terminated, exit code one. That's the crash I had just read. Last state: the one before it. While a crashed container waits for its restart, it's still the pod's latest container, so plain kubectl logs shows it. Previous asks for the one before, and the kubelet had already cleaned that one up. Use previous once the container is running again, and the crash is behind it. The app ran, and died: now its own logs have the answer.

To sum it up: the status names the stop. Stuck at the image or at the settings, your app never ran: describe the pod, and read the first error in the events. And if the app ran and crashed, read its own logs, plain logs first.

And the one column that picks the command for you? Restarts, in kubectl get pods. At the end of my run, I listed the pods one last time. The crash loop: four restarts. The pods that never got to run their app had zero, in every check I made. Zero restarts and not running: your app never ran, so ask Kubernetes, with describe. More than zero: it ran and died, so ask your app, with logs.

Every pod's start begins with its image. The video on screen now shows how tags work in Azure Container Registry, and why one tag can give you different code tomorrow. If this was useful, subscribe for more practical .NET and Azure.

Need this in your own Azure environment?

I build, migrate and run .NET platforms on Azure, with Azure DevOps and Terraform. If your team wants this set up properly, see what I offer.

See services