Who is Ehsan Tatasadi?

Ehsan Tatasadi is a freelance Senior .NET & Azure Engineer based in Hamburg, Germany. He builds and modernizes .NET platforms on Azure, connects them with Dynamics 365, and runs them in production. He has 8+ years on the Microsoft stack, more than 7 of them for a DAX-listed energy utility and a regional grid operator.

What technologies does Ehsan Tatasadi specialize in?

C#, .NET 8 and ASP.NET Core with SQL Server; Azure (App Service, Functions, Key Vault, Private Endpoints, Application Insights); Dynamics 365 CE integration via the XRM SDK, C# plugins and JavaScript form scripts; Terraform and Azure DevOps YAML pipelines; React and TypeScript. He is certified in AZ-400, AZ-204, AI-103 and GH-300.

Where is Ehsan Tatasadi located?

Ehsan Tatasadi is based in Hamburg, Germany. He works remotely, on-site in Hamburg, and on-site anywhere in Germany on demand.

Explainer

Kubernetes Secrets on AKS: base64, RBAC and Azure Key Vault

November 9, 2026 · Series: Containers on Azure

azureakskubernetessecretssecuritykubectl

Kubernetes calls it a Secret. When I described one on Azure Kubernetes Service (AKS), kubectl showed only a size: 21 bytes. One command later, I had the value in plain text. No password, no key, nothing to break.

This explainer shows what a Kubernetes Secret really protects, who can read it, and what keeps it safe. Everything below comes from the Microsoft Learn exercise, run on my own cluster.

Key points

A Secret is encoded, not encrypted. Kubernetes stores Secret values as base64. Base64 has no key: anyone allowed to read the Secret turns it back with one command. Listing Secrets returns their values too.
Inside the pod it's plain text. The kubelet hands the decoded value to the container. An app can mask it in its own output, but anyone who may run kubectl exec on the pod reads the full value.
RBAC is the lock, not the pod's identity. The pod's own service account could not read a single Secret, and it doesn't need to: the kubelet fetches Secrets for the pods on its node. So whoever may create pods in a namespace can mount any Secret there.
On AKS the disks are encrypted; each Secret inside etcd only with KMS. Azure Storage always encrypts the data at rest. Encrypting each Secret before it's written to etcd is an optional layer, off by default. The Azure Key Vault provider for the Secrets Store CSI Driver keeps the value out of the cluster altogether.

Encoded, not encrypted

A Secret is a Kubernetes object for sensitive values, like passwords and keys. It works like a ConfigMap, the object for plain settings, but it has its own type and its own permissions. The Microsoft Learn exercise writes one in a YAML manifest, with the values in plain text under stringData:

apiVersion: v1
kind: Secret
metadata:
  name: api-secrets
type: Opaque
stringData:
  secret-endpoint: "SecretEndpointValue"
  secret-access-key: "SecretAccessKey123456"

These are the exercise's demo values, not a real key. A manifest like this is the first leak: it should never go into source control. Kubernetes merges stringData into the data field, and data holds base64.

The module checks a Secret with kubectl describe, which shows the keys and their sizes, not the values:

> kubectl describe secret api-secrets
Type:  Opaque
Data
====
secret-access-key:  21 bytes
secret-endpoint:    19 bytes

That looks protected. It isn't. Asking for the raw data field returns the stored value:

> kubectl get secret api-secrets -o jsonpath="{.data.secret-access-key}"
U2VjcmV0QWNjZXNzS2V5MTIzNDU2

Base64 is an encoding, not encryption. It turns any bytes into plain letters so they fit into YAML and JSON: every four letters stand for three bytes (U2Vj is Sec, cmV0 is ret, and so on). There is no key. kubectl's go-template output decodes it in the same command:

> kubectl get secret api-secrets -o go-template='{{index .data "secret-access-key" | base64decode}}'
SecretAccessKey123456

A Secret is an envelope with a label, not a safe. kubectl describe just doesn't open it for you.

Plain text in the pod

In the exercise, a Deployment runs the app in a pod, and its template maps each key to an environment variable with secretKeyRef:

env:
- name: SECRET_ACCESS_KEY
  valueFrom:
    secretKeyRef:
      name: api-secrets
      key: secret-access-key

When the container starts, the kubelet, the Kubernetes agent on the node, fetches the Secret and hands the decoded value to the container. Like a ConfigMap read as environment variables, it's a copy taken at the start: a rotated Secret reaches the app only after a rollout.

The exercise's app is careful. Its /secrets endpoint masks the value:

> python cfg.py get /secrets
GET /secrets -> 200
… "secret_access_key":{…,"value":"***3456","length":21} …

But masking is something the app does. The container itself holds the real value:

> kubectl exec deploy/aks-config-api -- printenv SECRET_ACCESS_KEY
SecretAccessKey123456

Inside the pod, a Secret is plain text. Anyone allowed to run kubectl exec on that pod can read it.

Who may read it

Kubernetes decides access with RBAC, role-based access control: which user or identity may get, list or watch which kind of object. list counts as reading: listing Secrets returns their values. kubectl auth can-i asks the cluster that question:

> kubectl auth can-i get secrets
yes
> kubectl auth can-i get secrets --as system:serviceaccount:default:default
no

The first answer is mine. My kubeconfig came from az aks get-credentials, on a cluster without Microsoft Entra sign-in; there, that credential has the same effect as cluster admin. The second answer is the pod's: the exercise sets no service account, so its pods run as default, and that identity may not read a single Secret.

It doesn't need to. The pod never asks the API for its Secret; the kubelet does, and the cluster lets each kubelet read only the Secrets of pods bound to its own node. So the pod's identity isn't the lock. The lock is who may read Secrets, and who may get into the pods that use them.

Who	Reads the value how
Anyone with `get`, `list` or `watch` on Secrets	`kubectl get secret … -o jsonpath` + a base64 decode
Anyone who may `exec` into the pod	`printenv` or the mounted file
Anyone who may create pods (or Deployments) in the namespace	a new pod that mounts the Secret
The pod's own service account (default)	no API access needed: the kubelet delivers it

Encrypted where?

Kubernetes keeps every object, Secrets included, in etcd, its database. On AKS, etcd runs in the control plane, the part of the cluster that Microsoft runs for you. Is each Secret encrypted there by default? No, only if you turn it on.

Azure platform encryption: Azure Storage always encrypts the data at rest. That protects the disk, not each Secret.
KMS encryption: an optional layer that encrypts each Secret before it's written to etcd, with a key in Azure Key Vault (platform-managed or customer-managed keys).

My cluster's security profile showed that layer off, which is the default:

> az aks show -g rg-demo -n aks-demo --query securityProfile -o json
{
  "azureKeyVaultKms": null,
  "customCaTrustCertificates": null,
  "defender": null,
  "imageCleaner": null,
  "workloadIdentity": null
}

Even with KMS on, kubectl still gets the plain value: the API server decrypts it for anyone RBAC lets through. Encryption at rest protects the stored copy, not the API.

The stronger move keeps the value out of the cluster: the Azure Key Vault provider for the Secrets Store CSI Driver, an add-on that mounts secrets from Key Vault into pods. The value lives in Key Vault; when the pod starts, the driver fetches it with a managed identity and mounts it as a file. It never sits in etcd, unless you ask the driver to sync it into a Kubernetes Secret as well. Key Vault can log every read, and with rotation turned on (it's off by default), the driver checks for a new value every two minutes. Inside the pod it's still a plain file, so the rules above still apply.

The permission that reads every Secret

The kubelet hands a Secret to any pod that names it. So whoever can create a pod, or a Deployment, in a namespace can mount any Secret in it, and print it, without ever being allowed to read Secrets. In my run, the app's service account couldn't:

> kubectl auth can-i create pods --as system:serviceaccount:default:default
no

When you hand out roles, treat create pods like read Secrets.

What happened when I ran it

Tested on October 28, 2026 in my own Azure subscription (AKS 1.36 in westeurope, Azure CLI 2.88.0, kubectl 1.36); everything was deleted afterwards. The cluster had one Standard_D2s_v5 node and no Microsoft Entra integration (local accounts), as in the exercise. Every command above ran in under a second, except the az aks show call (3.1 s). The checks on the default service account exited with code 1, which is how kubectl auth can-i says no. The Secret held the exercise's demo values, so the decoded output on this page is safe to show.

What the video leaves out

Other Secret types: kubernetes.io/dockerconfigjson for registry credentials and kubernetes.io/tls for certificates, and kubectl create secret generic --from-literal.
Azure App Configuration with Key Vault references: its Kubernetes provider resolves the references and generates Kubernetes Secrets, which then live in etcd again.
Microsoft Entra integration and disabling local accounts, so cluster access goes through Entra ID instead of a cluster-admin certificate.
Workload identity for the CSI driver instead of the add-on's managed identity, and immutable Secrets.
Persistent storage, the third part of the module (a separate explainer).

Links

Watch on YouTube

Video transcript

Kubernetes calls it a Secret. I described one on Azure Kubernetes Service, and it showed only a size: twenty-one bytes. One command later, I had the value, in plain text. No password, no key, nothing to break.

In this video, you'll see who can really read a Kubernetes Secret, and what keeps it safe. At the end: one permission that reads every Secret in a namespace, without permission to read Secrets.

So what does a Secret actually store? A Secret is a Kubernetes object for sensitive values, like passwords and keys. It works like a ConfigMap, the object for plain settings, but with its own type and its own permissions. The Microsoft Learn module's exercise writes one in a YAML manifest, called api-secrets: two keys, an endpoint and an access key, in plain text, under stringData. I applied it, then asked kubectl, the Kubernetes command line, to describe the Secret, the way the module checks it. Two keys, and only their sizes: twenty-one bytes and nineteen bytes. No values. The value went in as plain text. What did Kubernetes store? The same value, in base64. I asked kubectl for the raw data field of that one key, and nothing else. A string of letters and digits. Base64 is an encoding, not encryption. It turns any bytes into plain letters, so they fit into YAML and JSON. There's no key: anyone can turn it back. So I decoded it, with kubectl's go-template output, a small template language, and its built-in base64decode function. Secret access key, one two three four five six. The whole value. A Secret is an envelope with a label, not a safe. kubectl describe just doesn't open it for you.

Then what does the pod that uses it see? In the exercise, a deployment runs the app in a pod, the unit that wraps your container. Its template maps each key to an environment variable, with secretKeyRef: the Secret's name, and the key inside it. When the container starts, the kubelet, the Kubernetes agent on the node, fetches the Secret and hands the decoded value to the container. Just like a ConfigMap, it's a copy taken at the start. The ConfigMap video showed what that means when the value changes. The exercise's app is careful with it. With a small script of mine, I called its secrets endpoint over the public address. Masked. Three stars, the last four characters, and the length: twenty-one. But masking is something the app does. So I asked the container itself, with kubectl exec and printenv. Secret access key, one two three four five six. The full value, in plain text. Kubernetes hands the real value to every process in the container. So anyone allowed to run kubectl exec on that pod can read it. Inside the pod, a Secret is plain text. Getting into a pod is as good as reading its Secrets. This series runs one Azure service at a time, for real. Subscribe to follow the series.

So who is allowed to read a Secret? Kubernetes decides that with RBAC, role-based access control: which user or identity may get, list or watch which kind of object. And list counts as reading: listing Secrets returns their values too. kubectl auth can-i asks the cluster that question for you. First, for me. Yes. My login file came from az aks get-credentials, the usual way to connect. On a cluster without Microsoft Entra sign-in, that file is a cluster admin: allowed to do everything. Pods have an identity too: a service account. The exercise sets none, so its pods run as default. The pod uses this Secret all the time. Can its own service account read it? No. I asked again with --as, which checks the question for another identity: the pod's service account. A plain no, exit code one. That identity can't read a single Secret. It doesn't need to. The pod never asks the API for its Secret. The kubelet does, and the cluster lets each kubelet read only the Secrets of pods on its own node. So the pod's identity isn't the lock. The lock is who may read Secrets, and who may get into the pods that use them.

So is it at least encrypted where it's stored? Kubernetes keeps every object, Secrets included, in etcd, its database. On Azure Kubernetes Service, etcd runs in the control plane, the part of the cluster that Microsoft runs for you. On AKS, is each Secret encrypted inside etcd by default? No, only if you turn it on. The disks under etcd are always encrypted by Azure Storage. But that protects the disk, not each Secret. But encrypting each Secret before it's written to etcd is an extra layer: KMS, the Key Management Service plugin, with a key in Azure Key Vault. I asked AKS for my cluster's security profile, the place where that layer would show up. azureKeyVaultKms: null. The extra layer was off, as it is by default. And even with it on, kubectl still gets the plain value: the API server decrypts it for anyone RBAC lets through. Encryption at rest protects the stored copy, not the API. So how do you keep the value out of the cluster altogether? With the Azure Key Vault provider for the Secrets Store CSI Driver: an add-on that mounts secrets from Key Vault into pods. The value lives in Key Vault. When the pod starts, the driver fetches it with a managed identity, an Azure identity with no password to store, and mounts it as a file. It never sits in etcd, unless you ask the driver to copy it into a Kubernetes Secret as well. Key Vault can log every read. And with rotation turned on, the driver checks for a new value every two minutes. Inside the pod, it's still a plain file, so the rules from before still apply. Base64 hides nothing. RBAC decides who reads it, KMS protects the stored copy, and Key Vault keeps it out of the cluster.

To sum it up: a Secret is base64, and one command decodes it. In the pod, it's plain text for anyone who can exec in. RBAC decides who reads Secrets. Encryption inside etcd is an extra on AKS, and Key Vault keeps the value out of the cluster.

And the permission that reads every Secret in a namespace, without permission to read Secrets? Creating pods. The kubelet hands a Secret to any pod that names it. So whoever can create a pod, or a deployment, in a namespace, a folder of objects in the cluster, can mount any Secret in it, and print it. So I checked the app's service account one last time: may it create pods in the namespace? No. Good. When you hand out roles, treat create pods like read Secrets.

The safest secret is one you never store: the video on screen swaps a connection string for a managed identity. If this was useful, subscribe for more practical .NET and Azure.

Need this in your own Azure environment?

I build, migrate and run .NET platforms on Azure, with Azure DevOps and Terraform. If your team wants this set up properly, see what I offer.

See services