NFS errors on install on-prem

Portworx Install
1

NFS dependency errors when installing and portworx essentials, on-prem/vcenter/flasharray. Probably something i did incorrect with the spec, but I’ve not been able to track it down, any help appreciated.

kubectl exec px-cluster-4abbef0c-9499-4a5d-b191-d179431dc64e-nbh5b -n portworx – /opt/pwx/bin/pxctl cluster provision-status
time=“2024-09-18T15:18:28Z” level=error msg=“Failed to get /status: Get "http://localhost:9001/status\”: dial tcp 127.0.0.1:9001: connect: connection refused"
PX stopped working 33.6s ago. Last status: Could not initiate cloud provider

List of last known failures:

Type ID Resource Severity Count LastSeen FirstSeen Description
NODE NfsDependencyNotEnabled user-host-t2-03 ALARM 1 Sep 16 20:11:19 UTC 2024 Sep 16 20:11:19 UTC 2024 Could not configure NFS service
NODE NfsDependencyInstallFailure user-host-t2-03 ALARM 1 Sep 16 20:11:19 UTC 2024 Sep 16 20:11:19 UTC 2024 could not install NFS service: Command ‘DEBIAN_FRONTEND=noninteractive apt-get install -yq dbus nfs-common rpcbind nfs-kernel-server’ failed: exit status 100
command terminated with exit code 1

$ kubectl -n portworx get storagenodes -l name=portworx
NAME ID STATUS VERSION AGE
user-host-t2-01 Initializing 43h
user-host-t2-02 Initializing 43h
user-host-t2-03 Initializing 42h

2

journalctl -lu portworx*

Sep 18 15:45:18 user-host-t2-03 portworx[638764]: 2024-09-18 15:45:18,027 INFO spawned: ‘pxdaemon’ with pid 502376
Sep 18 15:45:18 user-host-t2-03 portworx[638764]: PX_STORAGE_IO_FLUSHER=yes
Sep 18 15:45:18 user-host-t2-03 portworx[638764]: Starting as an IOFlusher process : /usr/local/bin/start_pxcontroller_pxstorage.py
Sep 18 15:45:18 user-host-t2-03 portworx[638764]: Process with PID 502376, is a IO Flusher
Sep 18 15:45:23 user-host-t2-03 portworx[638764]: 2024-09-18 15:45:23,080 INFO success: pxdaemon entered RUNNING state, process has stayed up for > than 5 seconds (star>
Sep 18 15:45:48 user-host-t2-03 portworx[638764]: PXPROCS[INFO]: lttng is not running. Starting without it…
Sep 18 15:45:48 user-host-t2-03 portworx[638764]: PXPROCS[INFO]: Validating container binaries…
Sep 18 15:45:48 user-host-t2-03 portworx[638764]: PXPROCS[INFO]: Started px-storage with pid 502450
Sep 18 15:45:48 user-host-t2-03 portworx[638764]: bash: connect: Connection refused
Sep 18 15:45:48 user-host-t2-03 portworx[638764]: bash: /dev/tcp/localhost/9009: Connection refused
Sep 18 15:45:48 user-host-t2-03 portworx[638764]: PXPROCS[INFO]: px-storage not started yet on ‘localhost’…sleeping
Sep 18 15:45:51 user-host-t2-03 portworx[638764]: 2024-09-18 15:45:51,870 INFO reaped unknown pid 502351
Sep 18 15:45:51 user-host-t2-03 portworx[638764]: PXPROCS[INFO]: px-storage startup on ‘::1’…
Sep 18 15:45:51 user-host-t2-03 portworx[638764]: PXPROCS[INFO]: Started px with pid 502463
Sep 18 15:45:51 user-host-t2-03 portworx[638764]: PXPROCS[INFO]: Started watchdog with pid 502464
Sep 18 15:45:51 user-host-t2-03 portworx[638764]: 2024-09-18_15:45:51: PX-Watchdog: Starting watcher
Sep 18 15:45:51 user-host-t2-03 portworx[638764]: 2024-09-18_15:45:51: PX-Watchdog: Waiting for px process to start
Sep 18 15:45:51 user-host-t2-03 portworx[638764]: 2024-09-18_15:45:51: PX-Watchdog: (pid 502463): Begin monitoring
Sep 18 15:45:51 user-host-t2-03 portworx[638764]: 2024-09-18_15:45:51 * ERROR * : LTTNG Channel files do not exist. LTTNG tracing is not being done.
Sep 18 15:45:52 user-host-t2-03 portworx[638764]: time=“2024-09-18T15:45:52Z” level=info msg=“[Metering (AirGapped)]: registered the Air-Gapped Metering Agent…”
Sep 18 15:45:52 user-host-t2-03 portworx[638764]: time=“2024-09-18T15:45:52Z” level=info msg=“Registered the default Metering Agent…”
Sep 18 15:45:52 user-host-t2-03 portworx[638764]: time=“2024-09-18T15:45:52Z” level=info msg=“[Metering (UsageAgent)] Registered the Usage based Metering Agent…”
Sep 18 15:45:52 user-host-t2-03 portworx[638764]: time=“2024-09-18T15:45:52Z” level=info msg=“Registering [kernel] as a volume driver”
Sep 18 15:45:52 user-host-t2-03 portworx[638764]: time=“2024-09-18T15:45:52Z” level=info msg=“Registering [blockFADriver] as a volume driver”
Sep 18 15:45:52 user-host-t2-03 portworx[638764]: time=“2024-09-18T15:45:52Z” level=info msg=“Setting log level to info(4)”
Sep 18 15:45:52 user-host-t2-03 portworx[638764]: md_fixup: running version 0.3
Sep 18 15:45:52 user-host-t2-03 portworx[638764]: building global dm targets
Sep 18 15:45:52 user-host-t2-03 portworx[638764]: discovered dm targets: |3|
Sep 18 15:45:52 user-host-t2-03 portworx[638764]: building node information
Sep 18 15:45:52 user-host-t2-03 portworx[638764]: assembling arrays:
Sep 18 15:45:52 user-host-t2-03 portworx[638764]: checking thin pool on arrays:
Sep 18 15:45:52 user-host-t2-03 portworx[638764]: time=“2024-09-18T15:45:52Z” level=info msg=“Read config from env var” file=“boot.go:736” component=porx/px/boot fields>
Sep 18 15:45:52 user-host-t2-03 portworx[638764]: time=“2024-09-18T15:45:52Z” level=info msg=“Read config from config.json” file=“boot.go:744” component=porx/px/boot fi>
Sep 18 15:45:52 user-host-t2-03 portworx[638764]: time=“2024-09-18T15:45:52Z” level=info msg=“Alerts initialized successfully for this cluster” file=“boot.go:774” compo>
Sep 18 15:46:20 user-host-t2-03 portworx[638764]: time=“2024-09-18T15:46:20Z” level=error msg=“Could not initiate cloud provider” file=“terminate.go:87” component=porx/>
Sep 18 15:46:20 user-host-t2-03 portworx[638764]: PXPROCS[INFO]: px daemon exited with code: 1
Sep 18 15:46:21 user-host-t2-03 portworx[638764]: 2024-09-18 15:46:21,146 INFO exited: pxdaemon (exit status 1; not expected)
Sep 18 15:46:21 user-host-t2-03 portworx[638764]: 2024-09-18 15:46:21,146 INFO reaped unknown pid 502450