I am trying to test drive this on a 4 node cluster each with 1x100G drive. the install completed with the following log messages. Clearly “127.0.0.1:9001: connect: connection refused” is the problem, but do I need to install this service separately? Any pointers would be greatly appreciated!
time=“2021-01-03T18:59:49Z” level=warning msg=“Could not retrieve PX node status” error=“Get http://127.0.0.1:9001/v1/cluster/nodehealth: dial tcp 127.0.0.1:9001: connect: connection refused”
After trying to uninstall and reinstall based on the online docs posted a few more times. Now, I have 1 out of the 3 nodes running.
The other 2 nodes are still logging. time=“2021-01-04T14:20:51Z” level=warning msg=“Could not retrieve PX node status” error=“Get http://127.0.0.1:9001/v1/cluster/nodehealth: dial tcp 127.0.0.1:9001: connect: connection refused”. Tried to reboot both nodes that are not working… Is there a way to troubleshoot exactly what the root causue is?
portworx-25dfv 1/1 Running 0 121m 172.18.98.12 node2
portworx-api-d5krf 1/1 Running 0 121m 172.18.98.12 node2
portworx-api-x2bfq 0/1 Running 1 121m 172.18.98.13 node3
portworx-api-xj2dg 0/1 Running 0 121m 172.18.98.14 node4
portworx-hv2ph 0/1 Running 3 55m 172.18.98.13 node3
portworx-v56v7 0/1 Running 7 121m 172.18.98.14 node4
px-lighthouse-777fbbf998-bj4x6 3/3 Running 0 121m 10.34.0.3 node2
stork-855cff789d-m6gdk 1/1 Running 1 121m 10.32.0.2 node3
stork-855cff789d-s8lk4 1/1 Running 0 121m 10.34.0.2 node2
stork-855cff789d-sf97z 1/1 Running 0 121m 10.38.0.2 node4
stork-scheduler-844b5948b9-kqqgd 1/1 Running 0 121m 10.38.0.4 node4
stork-scheduler-844b5948b9-lmm8n 1/1 Running 1 121m 10.32.0.3 node3
stork-scheduler-844b5948b9-ts5tv 1/1 Running 0 121m 10.34.0.1 node2
Hi
Can you post the output of kubectl describe of the one of the uninitialised pod e.g. portworx-hv2ph
can collect the entire portworx logs from that node by journalctl -lu portworx*
and also can you please post the storagecluster yaml file. and describe one of the portworx pod