A lot of the example commands in the troubleshooting section of the portworx website return data with the node ID string only. I am left wondering which node has the problem. Is there a pxctl syntax that will translate 4e20943c-a748-4836-928a-bf8306838d2b into the node’s hostname or IP address?
There is a command line utility of portworx where you can simply check it, you can login to portworx pod or worker node and run the following command : /opt/pwx/bin/pxctl status
IP ID SchedulerNodeName Auth StorageNode Used Capacity Status StorageStatus Version Kernel OS
10.229.192.69 e893a103-0bcf-4beb-a64e-5e6321ea7e11 mizo-portworx-worker02 Disabled Yes 8.5 GiB 150 GiB Online Up 2.8.0.0-1ef62f8 3.10.0-1160.15.2.el7.x86_64 CentOS Linux 7 (Core)
10.229.193.234 7c4d00fc-68e4-4d94-ae04-d4bb5a1782e2 mizo-portworx-worker03 Disabled Yes 8.5 GiB 150 GiB Online Up 2.8.0.0-1ef62f8 3.10.0-1160.15.2.el7.x86_64 CentOS Linux 7 (Core)
10.229.192.100 40a9f2bd-5848-4282-bc23-9117f656ab74 mizo-portworx-worker01 Disabled Yes 8.5 GiB 150 GiB Online Up (This node) 2.8.0.0-1ef62f8 3.10.0-1160.15.2.el7.x86_64 CentOS Linux 7 (Core)
This will give you a detail of IP/OS/ and other status of node is up/down.
pxctl status shows all kinds of confusing information. Here’s an example.
root@kube1:~# /opt/pwx/bin/pxctl status
PX stopped working 4.8s ago. Last status: Could not init boot manager (error=“error in obtaining etcd version: Get “http://kube-kvdb.sp.yo:2379/version”: EOF”)
Failed to initialize PX with the provided kvdb endpoint(s) http://kube-kvdb.sp.yo:2379.
Please ensure you have provided the correct endpoint(s) to a functional kvdb database.
List of last known failures:
NODE NodeStartFailure kube1 ALARM 1 Jan 23 17:49:21 UTC 2022 Jan 23 17:49:21 UTC 2022 Failed to start Portworx: error in obtaining etcd version: Get “http://kube-kvdb.sp.yo:2379/version”: EOF
NODE KvdbConnectionFailed kube1 ALARM 1 Jan 23 17:49:21 UTC 2022 Jan 23 17:49:21 UTC 2022 Failed to parse kvdb parameters: error in obtaining etcd version: Get “http://kube-kvdb.sp.yo:2379/version”: EOF
It looks like your ETCD endpoint is not reachable, you can try to do from worker node curl http://kube-kvdb.sp.yo:2379/version.
Do you want to use external etcd ? I would recommend to use Internal KVDB you have an option when you generate the portworx spec from central.portworx.com.