PLCnext MQTT client and WSS broker (TLS)

Hi,

my question basically is 'How to connect the Mqtt client on the plcnext to an mqtt broker using WSS protocol with TLS.
After filling in the settings json file and rebooting i got errors in the broker logs when the client is connecting.
The errors, see below, stated that the username flag isnt set, which is strange because login info is provided from the settings json file.
FYI, i am aware that by default, port 8883 is used for the SSL:// protocol but we had to use this port for the WSS protocol, i assume that isn't a problem.

2019-09-23 15:21:11.228 [error] emulator Error in process on node 'VerneMQ@127.0.0.1' with exit value:
{[
{reason,{case_clause,{{error,username_flag_not_set},<<>>}}},{mfa,{vmq_websocket,websocket_handle,3}},{stacktrace,[{vmq_mqtt_pre_init,data_in,2,[{file,"/opt/vernemq/apps/vmq_server/src/vmq_mqtt_pre_init.erl"},{line,48}]},
{vmq_websocket,websocket_handle,3,[{file,"/opt/vernemq/apps/vmq_server/src/vmq_websocket.erl"},{line,82}]},{cowboy_websocket,handler_call,7,[{file,"/opt/vernemq/_build/default/lib/cowboy/src/cowboy_websocket.erl"},{line,588}]},
{cowboy_protocol,execute,4,[{file,"/opt/vernemq/_build/default/lib/cowboy/src/cowboy_protocol.erl"},{line,442}]}]},
{msg,{binary,<<16,63,0,4,77,81,84,84,4,70,0,30,0,7,80,76,67,110,101,120,116,0,24,67,65,78,65,82,89,47,68,79,77,73,67,65,47,83,78,47,79,70,70,76,73,78,69,0,1,49,0,13,100,106,104,101,54,51,57,48,102,107,100,115,57>>}},
{req,[{socket,{sslsocket,{gen_tcp,#Port,tls_connection,},[,]}},{transport,ranch_ssl},{connection,keepalive},{pid,},{method,<<"GET">>},{version,'HTTP/1.1'},
{peer,{{81,82,206,197},43696}},{host,<<"our.broker">>},{host_info,undefined},{port,8883},{path,<<"/mqtt">>},{path_info,undefined},{qs,<<>>},{qs_vals,undefined},{bindings,[]},
{headers,[{<<"host">>,<<"our.broker:8883">>},{<<"upgrade">>,<<"websocket">>},{<<"connection">>,<<"Upgrade">>},{<<"origin">>,<<"http://our.broker:8883">>},{<<"sec-websocket-key">>,<<"W9cP16ImTQ6m4HwLXhgh5A==">>},
{<<"sec-websocket-version">>,<<"13">>},{<<"sec-websocket-protocol">>,<<"mqtt">>}]},{p_headers,[{<<"sec-websocket-protocol">>,[<<"mqtt">>]},{<<"upgrade">>,[<<"websocket">>]},{<<"connection">>,[<<"upgrade">>]}]},{cookies,undefined},
{meta,[{websocket_version,13},{websocket_compress,false}]},{body_state,waiting},{buffer,<<>>},{multipart,undefined},{resp_compress,false},{resp_state,done},{resp_headers,[]},{resp_body,<<>>},{onresponse,undefined}]},
{state,{st,<<>>,vmq_mqtt_pre_init,{state,{{81,82,206,197},43696},[{mountpoint,[]},{allowed_protocol_versions,[3,4,131]},{buffer_sizes,undefined},{use_identity_as_username,false}],0,#Ref},
{{1569,244871,208337},0},{{1569,244871,208338},0}}}],[{cowboy_websocket,handler_call,7,[{file,"/opt/vernemq/_build/default/lib/cowboy/src/cowboy_websocket.erl"},{line,642}]},{cowboy_protocol,execute,4,
[{file,"/opt/vernemq/_build/default/lib/cowboy/src/cowboy_protocol.erl"},{line,442}]}
]}

2019-09-23 15:21:11.228 [error] @vmq_mqtt_pre_init:data_in:48 Ranch listener {{0,0,0,0},8883} terminated in vmq_websocket:websocket_handle/3 with reason:
no case clause matching {{error,username_flag_not_set},<<>>} in vmq_mqtt_pre_init:data_in/2 line 48

For TLS i am using a server cert that is signed against LetsEncrypt, I put this certificate in a pem file along with Let's Encrypt Authority X3 and DST Root CA X3. Its readable by admin.

admin@axcf2152:~$ ls -l /opt/plcnext/Security/MQTT/
total 8
-rw-rw-r-- 1 admin plcnext 4850 Sep 23 12:14 final.pem

I set ssl_options as follows:

"ssl_options":
{
"trust_store": "/opt/plcnext/Security/MQTT/final.pem",
"enable_server_cert_auth": false
},

I don't know if this is a valid way of doing this? I also tested connecting using openssl s_client -showcerts -CAfile /opt/plcnext/Security/MQTT_Aviary/final.pem -connect our.broker:8883 and this looked OK...

CONNECTED(00000003)
depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
verify return:1
depth=0 CN = our.broker
verify return:1
---
Certificate chain
0 s:/CN=our.broker
i:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
1 s:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
i:/O=Digital Signature Trust Co./CN=DST Root CA X3
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
---
Server certificate
subject=/CN=our.broker
issuer=/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 3063 bytes and written 433 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES256-GCM-SHA384
Session-ID: 84D4B9795B84378FE1C1FE71AB014E6BB110342C6197DDDE20E35E78E12CCFF7
Session-ID-ctx:
Master-Key: B58769A2AF277D25C9EC7B18BF3113507980D2D19A67F3DEEBA1E32233A4F2319ABB2B90A8D091F235C28647E39BF0A5
Key-Arg : None
PSK identity: None
PSK identity hint: None
SRP username: None
Start Time: 1569257800
Timeout : 300 (sec)
Verify return code: 0 (ok)
---
closed

After searching i tried by also passing the auth info via the URI; wss://user:pass@our.broker:8883 and the error on the broker went away, but i dont see any info about a client connecting so thats probably not it.

After reboot, on the broker i see in netstat an established connection on port 8883 but not on the client itself

admin@axcf2152:~$ tail -n +1 -f /opt/plcnext/logs/Output.log | grep 'Mqtt\|MQTT\|mqtt'
23.09.19 16:34:46.768 Arp.System.Acf.Internal.Sm.ProcessesController INFO - Library 'PxceTcs.MqttClient' in process 'MainProcess' loaded.
23.09.19 16:34:47.020 Arp.System.Acf.Internal.Sm.ComponentsController INFO - Component 'MqttClientComponent-1' in process 'MainProcess' created.
23.09.19 16:34:59.721 PxceTcs.MqttClient.MqttClientComponent INFO - Publishing MQTT Client service
23.09.19 16:34:59.722 PxceTcs.MqttClient.MqttClientComponent INFO - Published MQTT Client service
23.09.19 16:34:59.784 PxceTcs.Mqtt.GdsConnectorComponent INFO - Subscribed to MQTT Client Service.
23.09.19 16:34:59.785 PxceTcs.Mqtt.GdsConnectorComponent INFO - Subscribed to GDS Data Access Service.
23.09.19 16:34:59.786 PxceTcs.Mqtt.GdsConnectorComponent INFO - Loaded settings: Path=/opt/plcnext/projects/MqttClient/mqtt_gds.settings.json
23.09.19 16:35:00.964 PxceTcs.Mqtt.GdsConnectorComponent INFO - Loaded configuration from file: /opt/plcnext/projects/MqttClient/mqtt_gds.settings.json
23.09.19 16:35:00.967 PxceTcs.Mqtt.GdsConnectorComponent INFO - Loaded configuration schema.
23.09.19 16:35:00.975 PxceTcs.Mqtt.GdsConnectorComponent INFO - Configuration is valid.
23.09.19 16:35:00.983 PxceTcs.Mqtt.GdsConnectorComponent INFO - Created MQTT Client with ID: 48616
23.09.19 16:35:00.984 PxceTcs.Mqtt.GdsConnectorComponent INFO - Global timeout value has been set to 60 milliseconds
23.09.19 16:35:00.986 PxceTcs.MqttClient.MqttClientManager INFO - Connecting to MQTT server
23.09.19 16:35:01.050 PxceTcs.MqttClient.MqttClientManager ERROR - Exception in operation: Connect(48616, ConnectOptions)
23.09.19 16:35:01.052 PxceTcs.MqttClient.MqttClientManager ERROR - MQTT error [-1]:
23.09.19 16:35:01.053 PxceTcs.Mqtt.GdsConnectorComponent ERROR - Error connecting to MQTT Client 48616

So I hope anyone can help me get a plcnext device to connect to a wss-enabled MQTT broker...

Best regards,

Thomas