☁️ Kubernetes Node Access

Logging In to Kubernetes Nodes with kubectl debug (and what to do when Gatekeeper blocks it)
kubectl -n kube-system debug node/NODE_NAME -it --image=mcr.microsoft.com/azurelinux/busybox:1.36

Sometimes you must look INSIDE a node: the kubelet will not start, a node is out of disk, containers will not pull, DNS or networking is odd, or you need logs
that the Kubernetes API does not show. Managed clusters such as AKS do not give you SSH keys to nodes that live on private networks. The supported, no-SSH way in is
kubectl debug node/..., which starts a privileged "debug pod" on that node with the node's filesystem mounted at /host. This guide shows the exact commands
for AKS (Azure Kubernetes Service), how to use it when an admission policy (OPA Gatekeeper / Azure Policy for Kubernetes) refuses the debug pod and you must
run it in the kube-system namespace, what to do once you are on the node, how to clean up, and safer alternatives. Other clouds are noted at the end.
Use this only on clusters you are authorized to administer, and follow your organization's access rules. Node access is powerful and is usually audited.

1. Find the node:            kubectl get nodes -o wide
2. Open a shell on it:       kubectl debug node/aks-nodepool1-37663765-vmss000000 -it --image=mcr.microsoft.com/azurelinux/busybox:1.36
3. Blocked by Gatekeeper?    Add the kube-system namespace:
                             kubectl -n kube-system debug node/aks-nodepool1-37663765-vmss000000 -it --image=mcr.microsoft.com/azurelinux/busybox:1.36
4. Become the node's shell:  chroot /host
5. Look around:              journalctl -u kubelet --no-pager | tail -50      crictl ps      df -h      exit  (twice)
6. Clean up (same namespace!):   kubectl -n kube-system get pods | grep node-debugger      then     kubectl -n kube-system delete pod node-debugger-NODE-xxxxx

The command is:  --image=mcr.microsoft.com/azurelinux/busybox:1.36
The tag is  1.36  (a DOT, not a colon: "1:36" is a typo). This is the image used in Microsoft's current AKS "Connect to cluster nodes" documentation.
- azurelinux/busybox is a tiny Azure Linux image with BusyBox tools (sh, ls, cat, tar, vi, and so on). It is small, so it pulls fast
- Tags follow the BusyBox version: 1.36 always points to the newest 1.36.x build. There are also pinned tags with a build date, like 1.36.1-azl3.0.20260909 (and -amd64 / -arm64 variants)
- There is no  latest  tag for this image. Check the currently published tags before you script it:
     curl -s https://mcr.microsoft.com/v2/azurelinux/busybox/tags/list | jq -r '.tags[]' | sort -V | tail -10
- Older docs and blog posts use  mcr.microsoft.com/cbl-mariner/busybox:2.0  (CBL-Mariner was renamed Azure Linux). If you see that, use the azurelinux name above
- In a locked-down or private cluster the node may not be able to pull from mcr.microsoft.com. Copy the image into your own registry (Azure Container Registry) and use that:
     az acr import --name myregistry --source mcr.microsoft.com/azurelinux/busybox:1.36 --image azurelinux/busybox:1.36
     kubectl debug node/NODE -it --image=myregistry.azurecr.io/azurelinux/busybox:1.36
  The cluster must be allowed to pull from your registry (attached ACR or an image pull secret). Your policies may also restrict which registries are allowed (see the Gatekeeper section)
- You need more tools than BusyBox offers? Use  chroot /host  (below) to use the node's own tools, or pick another image such as ubuntu or a network toolbox image (for example nicolaka/netshoot).
  Every extra image must pass your organization's image policy, and public images may be blocked

[ ] kubectl installed, and your kubeconfig pointing at the cluster:   az aks get-credentials --resource-group myRG --name myCluster      kubectl config current-context
[ ] Permission to create pods (and, if you use the kube-system route, permission to create pods IN kube-system). Check:
      kubectl auth can-i create pods
      kubectl auth can-i create pods -n kube-system
      kubectl auth can-i create pods/exec -n kube-system
    With Microsoft Entra ID and Azure RBAC on the cluster, the role needs to allow these (for example Azure Kubernetes Service RBAC Admin, or a custom role)
[ ] A node that is Linux. Windows nodes use a different method (see the Windows section)
[ ] Network access to the Kubernetes API (private clusters: from inside the virtual network, over VPN or from a jump box)
[ ] Optional but useful: Azure CLI 2.0.64 or newer (az --version) for cluster commands
kubectl version --client                     # a recent kubectl gives you more debug options (the --profile flag needs 1.27 or newer)

kubectl get nodes -o wide                                    # names, status, OS image, internal IPs
kubectl get nodes -l agentpool=nodepool1                     # nodes of one AKS node pool (the label is agentpool)
kubectl get nodes --show-labels | head                       # see labels such as kubernetes.io/os, agentpool, topology.kubernetes.io/zone
kubectl get pod my-app-7d9c -n prod -o wide                  # which node runs a particular pod (the NODE column)
kubectl top nodes                                            # CPU and memory (needs metrics-server, on by default in AKS)
kubectl describe node NODE_NAME | less                       # conditions (MemoryPressure, DiskPressure), taints, allocated resources, recent events
Typical AKS node names look like  aks-nodepool1-37663765-vmss000000  (Windows nodes like aksnpwin000000). Copy the name exactly.

kubectl debug node/aks-nodepool1-37663765-vmss000000 -it --image=mcr.microsoft.com/azurelinux/busybox:1.36
Expected output:
    Creating debugging pod node-debugger-aks-nodepool1-37663765-vmss000000-bkmmx with container debugger on node aks-nodepool1-37663765-vmss000000.
    If you don't see a command prompt, try pressing enter.
    /#
What the pieces mean:
kubectl debug node/NAME    Create a temporary pod scheduled on THAT node
-it                        -i keeps input open and -t gives you a terminal, so you get an interactive shell
--image=...                The container image the debug pod runs (BusyBox here)
-n / --namespace           The namespace the debug pod is created in. The default is your current namespace (usually "default"). This is what you change when a policy blocks it
--profile=sysadmin         (kubectl 1.27+) Explicitly ask for the "sysadmin" profile: a privileged pod with the host namespaces and the host filesystem. Recent kubectl versions warn that the default "legacy" profile is deprecated, and suggest choosing one. For node debugging use sysadmin if you need full access
--quiet                    Hide the "Creating debugging pod" message        --image-pull-policy=IfNotPresent    Reuse a cached image
-- COMMAND ARGS            (after two dashes) run a command instead of the default shell
What the debug pod is: it runs with the node's process namespace, network and IPC, in a PRIVILEGED container, with the node's whole root filesystem mounted at /host.
That is why it can see everything, and also why policies dislike it.

Inside the debug container you only have BusyBox tools. To use the node's own shell, programs and services, switch into the mounted node filesystem:
chroot /host                        # now / is the node's real root. (If bash is missing use:  chroot /host /bin/sh)
cat /etc/os-release                 # confirm: Ubuntu or Azure Linux
hostname
systemctl status kubelet
exit                                # leaves chroot and returns to the BusyBox container
exit                                # ends the debug session
You can also skip the extra step and go straight in:
kubectl debug node/NODE -it --image=mcr.microsoft.com/azurelinux/busybox:1.36 -- chroot /host /bin/sh
Files without chroot: the node's files are under /host, for example  /host/var/log  and  /host/etc/kubernetes .

Health and resources
    uptime ; free -m ; df -h ; df -i                       # load, memory, disk space, and inodes (a full disk or no inodes breaks pulls and pods)
    top -b -n1 | head -20        or       ps aux --sort=-%mem | head
    dmesg -T | tail -50                                    # kernel messages: out-of-memory kills, disk errors, network driver problems
    journalctl -k --no-pager | grep -i -E 'oom|killed process' | tail
Kubelet and the container runtime
    systemctl status kubelet ; systemctl status containerd
    journalctl -u kubelet --no-pager | tail -100           # why pods are not starting, image pull errors, PLEG issues
    journalctl -u kubelet -f                               # follow live (Ctrl+C to stop)
    journalctl -u containerd --no-pager | tail -50
    crictl ps -a                                           # every container on the node (crictl talks to the runtime, docker is not used on modern nodes)
    crictl pods                                            # pod sandboxes
    crictl images                                          # cached images and sizes                 crictl rmi IMAGE_ID   remove one
    crictl logs CONTAINER_ID                               # a container's logs (works even if the API server cannot serve them)
    crictl inspect CONTAINER_ID | less                     # detailed container info
    crictl stats                                           # CPU and memory per container
    ls /var/log/pods /var/log/containers                   # log files on disk
Networking
    ip addr ; ip route ; ip -br link
    ss -tulpn                                              # listening ports
    cat /etc/resolv.conf ; cat /run/systemd/resolve/resolv.conf     # DNS settings the node uses
    nslookup mcr.microsoft.com ; curl -sI https://mcr.microsoft.com/v2/     # can the node resolve and reach registries?
    iptables-save | head -50 ; nft list ruleset | head     # firewall and kube-proxy rules      conntrack -S    connection tracking counters
    curl -sk https://YOUR-API-SERVER-FQDN/version           # can the node reach the API server? Use the server address from your kubeconfig. A 401 or 403 still proves connectivity (nodes do not resolve cluster-internal names like kubernetes.default.svc)
Kubernetes files
    ls /etc/kubernetes /var/lib/kubelet                    # kubelet config, certificates, pod volumes (handle with care)
    cat /var/lib/kubelet/config.yaml                       # kubelet settings (eviction thresholds, cluster DNS)
Disk hogs
    du -xh /var --max-depth=2 2>/dev/null | sort -rh | head -15
    du -sh /var/lib/containerd /var/log /var/lib/kubelet
    journalctl --disk-usage
Everything is read as root. Prefer looking to changing. If you free space or restart kubelet, write down what you did (see the cautions below).

While the debug pod is running (in another terminal):
kubectl get pods -n kube-system | grep node-debugger                                      # or the namespace you used
kubectl cp -n kube-system node-debugger-NODE-xxxxx:/host/var/log/syslog ./node-syslog     # download (needs tar in the image; BusyBox has it)
kubectl cp ./script.sh -n kube-system node-debugger-NODE-xxxxx:/tmp/script.sh              # upload to the debug container
If kubectl cp fails, stream the file:
kubectl exec -n kube-system node-debugger-NODE-xxxxx -- cat /host/var/log/kubelet.log > kubelet.log
kubectl exec -n kube-system node-debugger-NODE-xxxxx -- tar czf - -C /host/var/log . > node-logs.tgz
Collect a log bundle without an interactive shell:
kubectl -n kube-system debug node/NODE --image=mcr.microsoft.com/azurelinux/busybox:1.36 -- sh -c 'chroot /host journalctl -u kubelet --no-pager | tail -500'
(without -it the output prints and the pod stays until you delete it.) Never copy secrets or personal data off the node unless you are authorized.

When you type exit, the debug pod is NOT deleted automatically. It stays around in Completed state, and it can clutter the namespace, keep an old privileged pod visible to policy tools, and keep failing audits.
kubectl get pods -A | grep node-debugger                                     # find leftovers in every namespace
kubectl delete pod node-debugger-aks-nodepool1-37663765-vmss000000-bkmmx                # the debug pod in your CURRENT namespace
kubectl -n kube-system delete pod node-debugger-aks-nodepool1-37663765-vmss000000-bkmmx  # ...if you created it with -n kube-system, delete it with -n kube-system
for p in $(kubectl -n kube-system get pods -o name | grep node-debugger); do kubectl -n kube-system delete "$p"; done       # remove all of your debug pods in kube-system
Forgetting the -n on the delete is the classic mistake: "pods ... not found" means you are looking in the wrong namespace.
Always confirm with the get pods commands above that nothing is left behind.

WHEN GATEKEEPER BLOCKS THE DEBUG POD

On AKS with the Azure Policy add-on (or a plain OPA Gatekeeper install), an admission webhook checks every new pod. A debug pod is privileged, uses host namespaces and mounts the host's root filesystem,
so common policies refuse it in normal namespaces. The command fails immediately, before you get a shell. The error looks similar to:
    Error from server (Forbidden): admission webhook "validation.gatekeeper.sh" denied the request:
    [azurepolicy-k8sazurev3noprivilege-...] Privileged container is not allowed: debugger, securityContext: {"privileged": true}
    [azurepolicy-k8sazurev1hostfilesystem-...] HostPath volume {"hostPath": {"path": "/", "type": ""}, "name": "host-root"} is not allowed, pod: node-debugger-...
    [azurepolicy-k8sazurev2blockhostnamespace-...] Sharing the host namespace is not allowed: debugger
(The constraint names and wording depend on your policies. The key parts are  admission webhook "validation.gatekeeper.sh"  and  denied the request .)
Other reasons a debug pod can be refused:  a restricted image registry (the image is not on the allowed list),  required labels,  resource limits required,  a disallowed image tag,  or an ordinary RBAC error (that one says "forbidden: User cannot create resource pods").
RBAC error = you lack permission. Gatekeeper error = a policy rejected the pod, even though you have permission.

Azure Policy for Kubernetes and most Gatekeeper setups EXCLUDE some system namespaces (by default kube-system, gatekeeper-system and azure-arc) so that the cluster's own components can run privileged. A debug pod created there
falls under the exclusion and is admitted. That is why the fix your team uses is to add the namespace flag:
kubectl -n kube-system debug node/aks-nodepool1-37663765-vmss000000 -it --image=mcr.microsoft.com/azurelinux/busybox:1.36
(The order of  -n kube-system  does not matter:  kubectl debug node/NODE -n kube-system -it --image=...  is equivalent.)
If you use a different namespace for break-glass work, the policy exclusion must cover it. Some organizations create a dedicated, excluded namespace instead of using kube-system.
Before you rely on it, know the rules:
- Using kube-system to get around a policy is a workaround for a control someone put there on purpose. Do it only if your organization's runbook or security team allows it, and record why (a ticket or change record)
- Your RBAC must let you create pods in kube-system (kubectl auth can-i create pods -n kube-system). Many teams restrict this to a small admin group
- The kube-system route is NOT guaranteed: if your policy does not exclude kube-system (or excludes different namespaces), you are still blocked. See "Find out what is blocking you"

kubectl get pods -n gatekeeper-system                                      # is Gatekeeper running? (AKS Azure Policy add-on: pods named gatekeeper-controller and azure-policy)
kubectl get constrainttemplates                                            # the policy templates installed
kubectl get constraints                                                    # every constraint, across kinds. Look for names starting with azurepolicy-
kubectl get constraints -o wide
kubectl describe <kind> <constraint-name>                                  # example:  kubectl describe k8sazurev3noprivilege azurepolicy-k8sazurev3noprivilege-xxxx
    Look at:   Spec > Match > Excluded Namespaces  (which namespaces are exempt),   Spec > Enforcement Action (deny, warn or dryrun),   Spec > Parameters,   Status > Violations (existing offenders)
kubectl get constraints -o json | jq -r '.items[] | [.kind, .metadata.name, .spec.enforcementAction, (.spec.match.excludedNamespaces // [] | join(","))] | @tsv'      # a one-line summary of each constraint and its excluded namespaces
kubectl auth can-i create pods -n kube-system
In Azure: Portal > Policy > Assignments > (the assignment) > Parameters shows "Namespace exclusions". Policy > Compliance shows which assignment denies the request.
The policy definition names are usually like "Kubernetes cluster should not allow privileged containers" or "Kubernetes cluster containers should only use allowed images".
Test without creating a node debug pod (a dry run shows whether the API server and the webhook would accept the object):
kubectl -n kube-system debug node/NODE -it --image=mcr.microsoft.com/azurelinux/busybox:1.36 --dry-run=server -o yaml      # (support depends on your kubectl version)

- Ask the platform or security team for a temporary Azure Policy EXEMPTION scoped to the cluster or a namespace, with an expiry date (Azure Policy > Exemptions). This is the supported way, and it leaves an audit trail
- Ask for a dedicated break-glass namespace that is excluded from the policy, with strict RBAC and logging
- Use another access path that does not create a pod (SSH by private IP, Azure Bastion, or az vmss run-command: see Alternatives)
- Do NOT delete, disable or edit the Gatekeeper constraints yourself to get around them, and do not remove the Azure Policy add-on. That removes the protection for the whole cluster, and it will be flagged
- If the policy also blocks the IMAGE, use an image from an allowed registry (import it into your ACR), or ask for the allowed list to include it

- The debug pod is root on the node. It can read every file on the node, including the service account tokens and mounted secrets of OTHER pods that run there, and kubelet credentials. Treat the session like a root login on a production server
- Kubernetes audit logs record who created the pod (and AKS diagnostic settings can send them to Log Analytics). Expect to be asked why. Query example:  AKSAudit | where ObjectRef contains "node-debugger"  (the table name depends on your logging setup)
- Keep sessions short, delete the pod afterward, and do not leave privileged pods running
- Do not paste or copy secrets into tickets. Redact tokens and keys from logs
- Look, do not change. Nodes in a node pool are replaceable and can be recreated at any time (upgrades, autoscaling, reimage), so manual changes are lost. If you find a fix that needs a config change, apply it through the node pool settings, a DaemonSet, or the cluster configuration
- Microsoft's support policy for AKS generally does not cover manual changes made on nodes. Always tell support what you changed
- Do not drain, cordon or delete nodes unless you understand the effect on workloads (PodDisruptionBudgets, single replicas)
- A Windows node cannot be reached this way. See below

1. SSH to the node's private IP (only from inside the cluster's virtual network, and it needs an SSH key configured on the node pool):
     az aks machine list --resource-group myRG --cluster-name myCluster --nodepool-name nodepool1 -o table        # names and private IPs
     ssh -i ~/.ssh/id_rsa azureuser@10.224.0.33
   Manage the SSH keys as described in the AKS documentation ("Manage SSH access"). Nodes are not reachable from the internet.
2. Use an existing node as a jump host: debug one node (above), then port-forward and SSH from there to the target node (see the Windows proxy method)
3. Azure Bastion: if a node or proxy node is not reachable, connect through an Azure Bastion host set up for the cluster's virtual network
4. Run one command on the node VM without a shell (VM Scale Set node pools): az vmss run-command invoke -g MC_myRG_myCluster_region -n aks-nodepool1-37663765-vmss --command-id RunShellScript --instance-id 0 --scripts "journalctl -u kubelet --no-pager | tail -50"
   (the node resource group is named  MC_<resourceGroup>_<clusterName>_<region> , or the custom one you chose)
5. Private cluster with no direct API access: az aks command invoke runs kubectl inside Azure for you (it is not interactive, so -it does not work):
     az aks command invoke --resource-group myRG --name myCluster --command "kubectl get nodes -o wide"
     az aks command invoke --resource-group myRG --name myCluster --command "kubectl -n kube-system debug node/NODE --image=mcr.microsoft.com/azurelinux/busybox:1.36 -- chroot /host journalctl -u kubelet --no-pager -n 100"
6. Do you really need the node? Often the answer is elsewhere:
     kubectl describe node NODE ; kubectl get events -A --sort-by=.lastTimestamp | tail -30 ; kubectl logs -n kube-system -l component=kube-proxy ; kubectl logs POD --previous
     AKS Diagnose and solve problems (Azure Portal), Container insights, and Log Analytics queries for kubelet and container logs, node metrics and events
     Node problem detector and az aks kollect / the AKS periscope tool collect node logs into a storage account without an interactive session

kubectl debug node/... does not work for Windows Server nodes. Use a host process container pod (HPC) that runs on the node. Save as hostprocess.yaml, replace AKSWINDOWSNODENAME, and if a policy blocks it add  namespace: kube-system :
apiVersion: v1
kind: Pod
metadata:
  name: hpc
  namespace: kube-system                 # only if a policy blocks the default namespace, and only if that is allowed
  labels:
    pod: hpc
spec:
  securityContext:
    windowsOptions:
      hostProcess: true
      runAsUserName: "NT AUTHORITY\\SYSTEM"
  hostNetwork: true
  containers:
    - name: hpc
      image: mcr.microsoft.com/windows/nanoserver:ltsc2022        # use nanoserver:1809 for Windows Server 2019 nodes
      command: ["powershell.exe", "-Command", "Start-Sleep 2147483"]
      imagePullPolicy: IfNotPresent
  nodeSelector:
    kubernetes.io/os: windows
    kubernetes.io/hostname: AKSWINDOWSNODENAME
  tolerations:
    - effect: NoSchedule
      key: node.kubernetes.io/unschedulable
      operator: Exists
kubectl apply -f hostprocess.yaml
kubectl -n kube-system exec -it hpc -- powershell           # you are SYSTEM on the Windows node (switch to C:\ to see its files)
kubectl -n kube-system delete pod hpc                       # clean up
Another way in: debug a Linux node, port-forward its debug pod, and SSH to the Windows node's private IP through it (see the AKS documentation, "Windows Server proxy connection for SSH").

Any conformant cluster:    kubectl debug node/NODE -it --image=busybox            (or ubuntu, for more tools). Then  chroot /host
EKS (AWS):     kubectl debug works the same way. You can also use AWS Systems Manager Session Manager to reach the EC2 instance without SSH:   aws ssm start-session --target i-0abc123
GKE (Google):  kubectl debug works. Or  gcloud compute ssh NODE_NAME --zone ZONE  (needs a network path and OS Login or keys)
OpenShift:     oc debug node/NODE     (it mounts the host at /host as well: chroot /host)
Minikube / kind: minikube ssh  or  docker exec -it kind-control-plane bash
Gatekeeper and Pod Security Admission behave the same way everywhere: a privileged, host-mounting pod is refused in restricted namespaces. Kubernetes Pod Security Admission can also block it:
the error then mentions  violates PodSecurity "baseline:latest"  or "restricted". Fix it with the same approach (an exempt namespace such as kube-system, or a namespace labeled pod-security.kubernetes.io/enforce=privileged, if your admins allow it).

"Error from server (Forbidden): ... admission webhook "validation.gatekeeper.sh" denied the request"        A Gatekeeper policy blocked it. Add  -n kube-system  if allowed, or ask for an exemption (see the Gatekeeper section)
"pods is forbidden: User ... cannot create resource "pods" in API group "" in the namespace "kube-system""      RBAC: you lack permission there. Ask for access or use another route. Check  kubectl auth can-i create pods -n kube-system
"violates PodSecurity ..."                                                    Pod Security Admission on the namespace. Use an allowed namespace, or the admins can label a break-glass namespace as privileged
The prompt never appears / hangs after "Creating debugging pod"               The image is still pulling or cannot be pulled. In another terminal:  kubectl get pods -n kube-system -w   and   kubectl describe pod node-debugger-...   (look for ImagePullBackOff or "Unschedulable")
ImagePullBackOff / ErrImagePull                                               The node cannot reach mcr.microsoft.com (private cluster, firewall, or DNS), the tag is wrong (use 1.36, with a dot), or your registry needs credentials. Import the image to your ACR
"Unschedulable" / stays Pending                                               The node is cordoned, tainted, or out of resources. Debug pods tolerate most taints; check kubectl describe pod. A node with full disk or pressure can also refuse pods
"error: unknown flag: --profile"                                              Your kubectl is older than 1.27. Upgrade it, or leave the flag out
Warning: Profile "legacy" is deprecated ...                                  A notice only. Add  --profile=sysadmin  (or another profile) to choose explicitly
"error: ephemeral containers are disabled" / debug not supported             Old clusters or restricted setups. Use another access method
"chroot: can't execute '/bin/bash': No such file or directory"                Use /bin/sh:  chroot /host /bin/sh
"command not found" inside the debug container                                You are in the BusyBox container. Run  chroot /host  to use the node's tools, or install what you need (not possible in a minimal image)
"crictl: command not found" after chroot                                      Try  /usr/local/bin/crictl  or check  ls /usr/bin | grep crictl . On some node images use  ctr -n k8s.io containers list
Cannot write to the node filesystem                                           The /host mount can be read-only in some profiles. Reading is the usual goal. Do not remount the root filesystem read-write on a production node
The session closed by itself                                                  Idle timeouts on the API connection or a bastion. Reconnect (a new debug pod is created)
Leftover node-debugger pods everywhere                                        Delete them (see Cleaning up). Use  kubectl get pods -A | grep node-debugger
I can create the pod but exec fails                                           kubectl auth can-i create pods/exec -n kube-system. You need the exec permission as well as create
"Unable to connect to the server" / timeouts                                  Private cluster: connect from inside the VNet, or use az aks command invoke. Check az aks get-credentials and your VPN
Everything works but the node looks fine                                      Compare with another node of the same pool. Check events and the control plane side (kube-apiserver, admission webhooks, CNI pods in kube-system)

    The image tag is 1.36 with a dot: mcr.microsoft.com/azurelinux/busybox:1.36. Check the current tags on the registry before you script it, and pin a dated tag if you need repeatable behavior.

    kubectl debug node/NODE -it --image=... gives you a privileged pod. Run chroot /host to use the node's own tools (systemctl, journalctl, crictl).

    If a Gatekeeper policy blocks the pod, add -n kube-system (many policies exempt it), but only when your organization allows that. Otherwise request a policy exemption or a break-glass namespace.

    Delete the debug pod when you finish, using the same -n you created it with. It is not removed automatically.

    Look but do not change: nodes are replaceable, and manual edits are lost and may not be supported. Fix causes in the node pool or cluster configuration.

    Treat every node session as a root login: keep it short, do not copy secrets, and expect it to be audited.

    Check RBAC first with kubectl auth can-i create pods -n kube-system, and use kubectl get constraints to see what a policy actually excludes.

    In a private cluster, import the image into your own registry so the node can pull it, and use az aks command invoke if you have no direct API access.

    Windows nodes need a host process container pod instead of kubectl debug.

    Prefer the other diagnostics first (kubectl describe node, events, logs, Container insights). You often find the answer without touching the node.