Panic while using corporate proxy credentials during first setup

Portworx Essentials
1

Hi,

Kubernetes (rancher RKE): 1.19.6
Px essential : 2.6.0.2

I’m getting the following panic error when trying to perform an air gapped installation using a proxy (the proxy env variables was properly set according the documentation). The kubernetes nodes are using a proxy to access Internet. I just copied the first error lines according the stack trace is huge

@kube01 portworx[31665]: time="2022-04-28T14:33:31Z" level=info msg="Trying to dial pxessentials.portworx.com:http, proxy: http://myuser%40mydomain:mypassword@127.0.0.1:9000"
@kube01 portworx[31665]: 2022/04/28 14:33:33 [ERR] memberlist: Push/Pull with f9cbbdfb-6805-4f9e-b5be-f9f38f35ccd0v2 failed: dial tcp 10.126.25.227:9002: connect: connection refused
@kube01 portworx[31665]: time="2022-04-28T14:33:33Z" level=info msg="Stopping updates collector"
@kube01 portworx[31665]: panic: runtime error: invalid memory address or nil pointer dereference
@kube01 portworx[31665]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x40 pc=0x1f88ac8]
@kube01 portworx[31665]: goroutine 1 [running]:
@kube01 portworx[31665]: panic(0x30eee00, 0x6795d40)
@kube01 portworx[31665]: /usr/local/go/src/runtime/panic.go:551 +0x3c1 fp=0xc420d650e8 sp=0xc420d65048 pc=0x441a51
@kube01 portworx[31665]: runtime.panicmem()
@kube01 portworx[31665]: /usr/local/go/src/runtime/panic.go:63 +0x5e fp=0xc420d65108 sp=0xc420d650e8 pc=0x44092e
@kube01 portworx[31665]: runtime.sigpanic()
@kube01 portworx[31665]: /usr/local/go/src/runtime/signal_unix.go:388 +0x17a fp=0xc420d65158 sp=0xc420d65108 pc=0x4578da
@kube01 portworx[31665]: github.com/portworx/porx/pkg/apiclient.(*httpProxy).Dial(0xc420ccdb80, 0x37ece98, 0x3, 0xc420ca78a0, 0x1e, 0x0, 0x0, 0x37ec679, 0x2)
@kube01 portworx[31665]: /home/carson/jenkins/jenkins.portworx.co/workspace/elease-build-jobs_gs-rel-2.6.0.2/go/src/github.com/portworx/porx/pkg/apiclient/proxyDialer.go:94 +0x3f8 fp=0xc420d65278 sp=0xc420d65158 pc=0x1f88ac8
@kube01 portworx[31665]: github.com/portworx/porx/pkg/apiclient.IsURLReachable(0xc4217715c0, 0x39, 0x5, 0x0, 0x0)
@kube01 portworx[31665]: /home/carson/jenkins/jenkins.portworx.co/workspace/elease-build-jobs_gs-rel-2.6.0.2/go/src/github.com/portworx/porx/pkg/apiclient/proxyDialer.go:149 +0x778 fp=0xc420d65530 sp=0xc420d65278 pc=0x1f89558
@kube01 portworx[31665]: github.com/portworx/porx/pkg/metering/essential.(*pxEssential).registerEssentialAgent(0xc420352240, 0xb, 0x4791440)
@kube01 portworx[31665]: /home/carson/jenkins/jenkins.portworx.co/workspace/elease-build-jobs_gs-rel-2.6.0.2/go/src/github.com/portworx/porx/pkg/metering/essential/essential.go:154 +0x164 fp=0xc420d65780 sp=0xc420d65530 pc=0x2acb894
@kube01 portworx[31665]: github.com/portworx/porx/pkg/metering/essential.(*pxEssential).Initialize(0xc420352240, 0x0, 0x0)

Any idea to solve this?
Is it due to the fact i’m using credentials?

2

Additional information if it can help:

the 3 portworx container are failing the same way (same error). When i check the pxctl status, it seems to keep restarting in loop:

a bad status:

[root@kube01 ~]#  /opt/pwx/bin/pxctl status
Status: PX is initializing...
License: PX-Essential (lease renewal in 23h, 59m)
Node ID: bdd478ee-aa8e-41b5-a4ce-e8ebcbfbf16c
        IP: 10.126.25.220
        Local Storage Pool: 1 pool
        POOL    IO_PRIORITY     RAID_LEVEL      USABLE  USED    STATUS  ZONE    REGION
        0       MEDIUM          raid0           80 GiB  6.0 GiB Online  default default
        Local Storage Devices: 1 device
        Device  Path                            Media Type              Size            Last-Scan
        0:1     /dev/mapper/pxdatavg-pxdatalv   STORAGE_MEDIUM_MAGNETIC 80 GiB          28 Apr 22 16:54 CEST
        total                                   -                       80 GiB
        Cache Devices:
         * No cache devices
        Kvdb Device:
        Device Path                     Size
        /dev/mapper/pxkvdbvg-pxkvdblv   24 GiB
         * Internal kvdb on this node is using this dedicated kvdb device to store its data.
Cluster Summary
        Cluster ID: px-cluster-523fd839-bd9e-4203-8e7e-3a60e176c7c7
        Cluster UUID: be1924ce-3fc0-4829-bcf0-303dd183a01a
        Scheduler: kubernetes
        Nodes: 3 node(s) with storage (0 online)
        IP              ID                                      SchedulerNodeName       StorageNode     Used            Capacity        Status          StorageStatus   Version         Kernel                  OS
        10.126.25.227   f9cbbdfb-6805-4f9e-b5be-f9f38f35ccd0    kube02                  Yes             Unavailable     Unavailable     Not Available   Up              Unavailable     Unavailable             Unavailable
        10.126.25.220   bdd478ee-aa8e-41b5-a4ce-e8ebcbfbf16c    kube01                  Yes             0 B             0 B             Initializing    Up (This node)  2.6.0.2-d505d8d 3.10.0-1160.el7.x86_64  Oracle Linux Server 7.9
        10.126.26.52    61c40ee2-8660-4cb6-9c4b-ff81de32fad1    kube04                  Yes             Unavailable     Unavailable     Not Available   Up              Unavailable     Unavailable             Unavailable
Global Storage Pool
        Total Used      :  0 B
        Total Capacity  :  0 B

or

[root@kube01 ~]#  /opt/pwx/bin/pxctl status
PX is not running on  host: Could not reach 'HealthMonitor'
3

I solved it by installing cntml (proxy) that permits to add my proxy credentials (user/domain/password)
After removing the proxy credentials and specifying my cntml proxy adress in portworx env it’s working properly