Portworx on OCP4.5 IBM Cloud VPC

Portworx on Kubernetes
1

Hi!

I have installed Portworx onto an OCP 4.5 cluster running on IBM Cloud in a VPC (gen2). I am using the install process that comes with IBM Cloud Pak for Data which uses the image cpdv3.0.1_portworx.tgz. The install completes successfully and the nodes report an OK status. However when I try to provision a PVC, it fails to connect to the API:
Failed to provision volume with StorageClass “portworx-low”: Get http://172.21.13.108:9001/v1/osd-volumes/versions: dial tcp 172.21.13.108:9001: i/o timeout

Would appreciate any assistance. Output from status on a worker node below:

kubectl exec $PX_POD -n kube-system -- /opt/pwx/bin/pxctl status
Status: PX is operational
License: IBM Cloud Pak for Data
Node ID: 8d8e25b9-45f2-415c-b52b-c7885329a1d9
	IP: 10.242.129.12
 	Local Storage Pool: 1 pool
	POOL	IO_PRIORITY	RAID_LEVEL	USABLE		USED	STATUS	ZONE	REGION
	0	LOW		raid0		1000 GiB	12 GiB	Online	eu-gb-3	eu-gb
	Local Storage Devices: 1 device
	Device	Path		Media Type		Size			Last-Scan
	0:1	/dev/vde	STORAGE_MEDIUM_MAGNETIC	1000 GiB		23 Oct 20 18:51 UTC
	total			-			1000 GiB
	Cache Devices:
	No cache devices
	Metadata Device:
	1	/dev/vdd	STORAGE_MEDIUM_MAGNETIC
Cluster Summary
	Cluster ID: px-storage-cluster
	Cluster UUID: 753a5e21-c41b-4aa7-bc33-a242a64bf479
	Scheduler: kubernetes
	Nodes: 3 node(s) with storage (3 online)
	IP		ID					SchedulerNodeName	StorageNode	Used	Capacity	Status	StorageStatus	VersionKernel				OS
	10.242.129.10	d010340f-8959-47f7-992d-d72c1d26d358	10.242.129.10	Yes		12 GiB	1000 GiB	Online	Up		2.5.5.0-bef56913.10.0-1160.2.1.el7.x86_64	Red Hat
	10.242.129.11	9d0f95d3-d353-4e9a-b4f4-2c151cb06898	10.242.129.11	Yes		12 GiB	1000 GiB	Online	Up		2.5.5.0-bef56913.10.0-1160.2.1.el7.x86_64	Red Hat
	10.242.129.12	8d8e25b9-45f2-415c-b52b-c7885329a1d9	10.242.129.12	Yes		12 GiB	1000 GiB	Online	Up (This node)	2.5.5.0-bef56913.10.0-1160.2.1.el7.x86_64	Red Hat
	Warnings:
		 WARNING: Persistent journald logging is not enabled on this node.
Global Storage Pool
	Total Used    	:  36 GiB
	Total Capacity	:  2.9 TiB
2

Judging from the port the error is coming from, 9001 - it suggests to me that the issue is that you are attempting to use the OpenShift 3.11 specific Portworx installation on an OpenShift 4.x environment (where you mention you are attempting to perform the installation). In OpenShift 4.3 there was a new service introduced that hijacked the port 9001 we have traditionally used, hence we needed to switch over to a port range starting with 17001.

To confirm the deployment approach, can you please paste the output:

oc get po -n kube-system -lname=portworx

?

Additionally, it’s required that there is unrestricted network connectivity between nodes on ports 17001-17022 on TCP and 17002 on udp - so please ensure any ibm cloud based firewall rules (as well as linux iptables) are allowing this traffic.

3

Thanks aleks - here is the output

$ oc get po -n kube-system

NAME READY STATUS RESTARTS AGE
portworx-api-bfc52 1/1 Running 0 2d20h
portworx-api-qcqfn 1/1 Running 0 2d20h
portworx-api-t6l7b 1/1 Running 0 2d20h
portworx-operator-7489bf486-6dc5c 1/1 Running 1 2d20h
px-storage-cluster-5n2hn 1/1 Running 0 2d20h
px-storage-cluster-vnvz4 1/1 Running 0 2d20h
px-storage-cluster-vv7pm 1/1 Running 0 2d20h
stork-bd9897c8-hbgf6 1/1 Running 1 2d20h
stork-bd9897c8-w9b8v 1/1 Running 0 2d20h
stork-bd9897c8-xqsqx 1/1 Running 1 2d20h
stork-scheduler-59d7977cb5-2h4tt 1/1 Running 0 2d20h
stork-scheduler-59d7977cb5-fgwmj 1/1 Running 1 2d20h
stork-scheduler-59d7977cb5-tw4r8 1/1 Running 1 2d20h