Last updated: October 2026
In this lesson, you will learn how to login to a Vmware Tanzu TKG/VKS Node
Understanding The Subject Matter
There are situations where you may need to log in directly to a VMware Tanzu Kubernetes Grid Service (Tanzu VKS) node for troubleshooting or investigation.
For example, you may need to investigate:
- High CPU or memory utilization
- Disk or filesystem issues
- Container runtime problems
- Kubernetes node-level issues
- Network connectivity problems
- Operating system-level issues
In this post, I will show you how to log in to a VKS or TKG node using the Supervisor Cluster.
Important: Direct SSH access to VKS nodes should normally be used for troubleshooting and investigation. Avoid making changes directly on a VKS node unless you understand the impact and the change is supported.
A VKS cluster consists of Kubernetes control-plane and worker nodes. These nodes are managed by VMware’s Kubernetes infrastructure.
When you have access to the Supervisor Cluster, you can use kubectl to retrieve information about the VKS cluster and its virtual machines.
The general process is:
- Connect to the Supervisor/ Supervisor Cluster.
- Identify the Supervisor namespace, and the VKS cluster (Note: VKS Supervisor namespace is different from VKS Kubernetes namespace)
- Identify the VM/node you want to access.
- Retrieve the SSH credentials associated with the VKS cluster.
- Decode the credential if necessary.
- Connect to the node using SSH.
- If password authentication fails, use the SSH private key.
ACTION TIME
Step 1: Connect to the Supervisor Cluster
First, switch your kubectl context to the Supervisor Cluster.
[tekneed_Victor@vks01 ~]$ k config use-context reg1supv.tekneed.com
Switched to context "reg1supv.tekneed.com".
You can confirm that you are connected to the Supervisor Cluster by listing the namespaces (supervisor namespaces not VKS k8s namespaces):
[tekneed_Victor@vks01 ~]$ k get ns
NAME STATUS AGE
production-lag-01 Active 92d
production-lag-02 Active 94d
production-lag-03 Active 92d
production-lag-04 Active 92d
argocd Active 109d
default Active 117d
kube-node-lease Active 117d
kube-public Active 117d
kube-system Active 117d
production-lag-05 Active 115d
svc-argocd-service-domain-xxxxxxx Active 109d
svc-cci-ns-domain-xxxxxxx Active 116d
svc-contour-domain-xxxxxxxx Active 104d
svc-harbor-domain-xxxxxxx Active 99d
svc-tkg-domain-xxxxxxx Active 117d
svc-tmc-xxxxxxx Active 117d
svc-velero-domain-xxxxxxxx Active 117d
test Active 91d
velero Active 96d
vgcharborregistry Active 98d
production-lag-06 Active 91d
vmware-system-ako Active 117d
vmware-system-appplatform-operator-system Active 117d
vmware-system-cert-manager Active 117d
vmware-system-csi Active 117d
vmware-system-imageregistry Active 117d
vmware-system-kubeimage Active 117d
vmware-system-license-operator Active 117d
vmware-system-logging Active 117d
vmware-system-monitoring Active 117d
vmware-system-netop Active 117d
vmware-system-nsop Active 117d
vmware-system-pinniped Active 117d
vmware-system-supervisor-services Active 117d
vmware-system-vks-public Active 116d
vmware-system-vmop Active 117d

Step 1B: Identify the VKS Node you want to log in to (optional step)
Once you know the namespace containing the Supervisor cluster, you can list the virtual machines in that namespace.
[tekneed_Victor@vks01 ~]$ k get vm -o wide -n production-lag-03
NAME POWER-STATE CLASS IMAGE PRIMARY-IP4
production-lag-03-xxxx-xxxxx PoweredOn master-vm vmi-xxxxxxxxxxxxxxxx 10.10.10.86
production-lag-03-xxxx-xxxxx PoweredOn master-vm vmi-xxxxxxxxxxxxxxxx 10.10.10.199 production-lag-03-xxxx-xxxxx PoweredOn master-vm vmi-xxxxxxxxxxxxxxxx 10.10.10.92 production-lag-03-xxxx-xxxxx PoweredOn master-vm vmi-xxxxxxxxxxxxxxxx 10.10.10.231
Step 2: Retrieve the SSH Password
[tekneed_Victor@vks01 ~]$ k get secrets production-lag-03-ssh-password -o yaml -n production-lag-03
apiVersion: v1
data:
ssh-passwordkey: gejwjwgeg3288wwbeddweqheggehskegs=
kind: Secret
metadata:
annotations:
kubernetes.vmware.com/password-update-last-timestamp: "2026-08-27T13:33:19Z"
creationTimestamp: "2026-06-05T13:33:12Z"
name: production-lag-03-ssh-password
namespace: production-lag-03
ownerReferences:
- apiVersion: cluster.x-k8s.io/v1beta2
kind: Cluster
name: production-lag-03
uid: hdhskks-dnenbsw-dnssmm-mnea-heh283
resourceVersion: "122356262"
uid: hdhskks-dnenbsw-dnssmm-mnea
type: Opaque
step 3: Decode the Base64 Password
Notice that the value of ssh-passwordkey is Base64 encoded.
You can decode it with:
echo "<BASE64_ENCODED_PASSWORD>" | base64 -d
[tekneed_Victor@vks01 ~]$ echo "gejwjwgeg3288wwbeddweqheggehskegs=" |base64 -d
tebaq7dwwkkwkewiehfqhlhqqhhqhw372a[tekneed_Victor@vks01 ~]$
Step 4: SSH Using the Password
[tekneed_Victor@vks01 ~]$ ssh vmware-system-user@10.10.10.231
vmware-system-user@10.10.10.231's password:
Permission denied, please try again.
vmware-system-user@10.10.10.231's password:
Permission denied, please try again.
vmware-system-user@10.10.10.231's password:
vmware-system-user@10.10.10.231: Permission denied (publickey,password).
[tekneed_Victor@vks01 ~]$
Depending on the state of the cluster and its SSH configuration, password authentication may not work, just as we have seen above.
For example:
Permission denied, please try again. Permission denied, please try again. Permission denied (publickey,password).
If this happens, you can use the SSH private key associated with the VKS cluster.
Using the SSH Private Key
Step 5: Retrieve the SSH Private Key
The VKS cluster also has an SSH secret containing the private key.
[tekneed_Victor@vks01 ~]$ k get secret production-lag-03-ssh -n production-lag-03 -o yaml
apiVersion: v1
data:
ssh-privatekey: 22whwwejelweowjehdfh74203wjenw8034wjnwnsn0wn3nwnsnsnnsssjsjsjsbbsbssbsbsbbssmsdcffsjjedswsjsrjsjerwkrhshhhskrhrh740308e58305830853843hhhekdkhhdkshdhskshfskhfsskskskkshhrhrrhhhss84830848308583939838nbdgdhifsuehhdsiuhdisuheisigiiskhshhedhhw9e978s9s9sshhhdhs88337w937474747shsbgdhjsgdgsjsggdssjsejs88308384843030038484803030483hbbsss..................................
kind: Secret
metadata:
creationTimestamp: "2026-06-05T13:33:12Z"
name: production-lag-03-ssh
namespace: production-lag-03
ownerReferences:
- apiVersion: cluster.x-k8s.io/v1beta2
kind: Cluster
name: production-lag-03
uid: hdhskks-dnenbsw-dnssmm-mnea-heh283
resourceVersion: "23626856"
uid: hdhskks-dnenbsw-dnssmm-mnea-heh283
type: kubernetes.io/ssh-auth
Step 6: Decode the Private Key
You can decode the private key and save it to a file.
[tekneed_Victor@vks01 ~]$ echo 22whwwejelweowjehdfh74203wjenw8034wjnwnsn0wn3nwnsnsnnsssjsjsjsbbsbssbsbsbbssmsdcffsjjedswsjsrjsjerwkrhshhhskrhrh740308e58305830853843hhhekdkhhdkshdhskshfskhfsskskskkshhrhrrhhhss84830848308583939838nbdgdhifsuehhdsiuhdisuheisigiiskhshhedhhw9e978s9s9sshhhdhs88337w937474747shsbgdhjsgdgsjsggdssjsejs88308384843030038484803030483hbbsss |base64 -d > production-lag-03.keys
The private key is now stored in: production-lag-03.keys
Step 7: SSH Into the VKS Node Using the Private Key
[tekneed_Victor@vks01 ~]$ ssh -i production-lag-03.keys vmware-system-user@10.10.10.231
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: UNPROTECTED PRIVATE KEY FILE! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0664 for 'production-lag-03.keys' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
Load key "production-lag-03.keys": bad permissions
vmware-system-user@10.10.10.231's password:
If you try to use the private key without changing its permissions, SSH may reject it, WHICH IT DID in this case
This happens because SSH does not want a private key to be readable or writable by other users. Hence, you need to change the permission.
Step 8: Set the Correct Permissions
[tekneed_Victor@vks01 ~]$ chmod 400 production-lag-03.keys
You can verify the permissions with; ls -l production-lag-03.keys. The important part is that only the owner can read the file.
Step 8: SSH Into the VKS Node Using the Private Key
You can now use the private key to connect to the node
[tekneed_Victor@vks01 ~]$ ssh -i production-lag-03.keys vmware-system-user@10.10.10.231
Welcome to Ubuntu 24.04.3 LTS (GNU/Linux 6.8.0-90-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/pro
System information as of Tue Sep 8 01:11:50 PM UTC 2026
System load: 0.75 Processes: 204
Usage of /: 49.1% of 19.02GB Users logged in: 0
Memory usage: 45% IPv4 address for eth0: 10.10.10.231
Swap usage: 0%
* Canonical Workshop gives developers fast, composable, reproducible, and
secure developer environments that are perfect for agentic workflows.
https://ubuntu.com/workshop
Expanded Security Maintenance for Applications is not enabled.
0 updates can be applied immediately.
Enable ESM Apps to receive additional future security updates.
See https://ubuntu.com/esm or run: sudo pro status
The list of available updates is more than a week old.
To check for new updates run: sudo apt update
Last login: Thu Sep 3 12:42:30 2026 from 10.10.10.14
vmware-system-user@production-lag-03-XXXX-XXXX:~$
Conclusion
Logging directly into a VMware Tanzu TKGI/VKS node is useful when Kubernetes-level commands are not enough and you need to investigate what is happening at the operating-system or VM level.
The important thing to remember is that you can retrieve the VKS node’s SSH credentials from the Supervisor Cluster. If password-based authentication does not work, the SSH private key stored in the cluster’s SSH secret can be decoded and used for authentication.
Also, always protect the private key with appropriate file permissions before using it:
chmod 400 production-lag-03.keys
Once connected, commands such as free -h, df -h, top, ip addr, systemctl, and container-runtime commands can help you investigate issues directly from the node.
Leave a Reply