You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Vincent Hermes <vi...@gmail.com> on 2020/06/25 01:50:28 UTC

HTTPS Broken with 4.14

Hi Guys,

so we updated our Cloudstack from 4.11.3 to 4.14 in production and now our HTTPS Webserver including Primate is not responding anymore. HTTP pn Port 8080 works though.

It does not show any certificate error, in fact it even says the right issuer but the site just never loads. There is also no access.log action when connecting via HTTPS. There are no error logs in management-server.log. It just reads "https=true, port=8443", etc and loads the keystore file correctly as it seems.

We have changed the port to 8444, used various browsers, also in private mode, cache cleared, reinstalled the certificates both on the HTTP UI and the keystore file and we even installed a completely fresh MGMT Server on a CentOS7 which shows the exact same behaviour after enabling HTTPS.

Is there anything new in 4.14 regarding https? Is someone having similar issues?

Regards
Vincent

Re: HTTPS Broken with 4.14

Posted by Rohit Yadav <ro...@shapeblue.com>.
Hi Vincent,

We'll need to investigate deeper as it could be a regression/backend issue, can you log your issue here:
https://github.com/apache/cloudstack/issues

Thanks.


Regards.

________________________________
From: Vincent Hermes <vi...@gmail.com>
Sent: Monday, June 29, 2020 15:50
To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
Subject: Re: HTTPS Broken with 4.14

Hi Rohit,

keytool -list -keystore /etc/cloudstack/management/keystore.pkcs12 (Password same as in server.properties and works)
--------------------
Keystore type: PKCS12
Keystore provider: SUN

Your keystore contains 1 entry

1, 11 Dec 2019, PrivateKeyEntry,
Certificate fingerprint (SHA-256): xx:xx:xx:xx...etc
--------------------

Converted pkcs12 to jks via "keytool -importkeystore -srckeystore combined.pkcs12 -destkeystore combined.jks -deststoretype jks" (I renamed the copy of keystore.pkcs12 to "combined.pkcs12")
Choose same password, output ok
Changed server.properties to https.keystore=/etc/cloudstack/management/combined.jks
-> Management-Server behaviour is the same
--------------------
Logs regarding keystore:
2020-06-29 12:01:02,052 INFO  [o.e.j.s.h.ContextHandler] (main:null) (logid:) Started o.e.j.w.WebAppContext@311bf055{/client,file:///usr/share/cloudstack-management/webapp/,AVAILABLE}{/usr/share/cloudstack-management/webapp}
2020-06-29 12:01:02,053 INFO  [o.e.j.s.h.ContextHandler] (main:null) (logid:) Started o.e.j.s.h.MovedContextHandler@451001e5{/,null,AVAILABLE}
2020-06-29 12:01:02,076 INFO  [o.e.j.s.AbstractConnector] (main:null) (logid:) Started ServerConnector@6f46426d{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
2020-06-29 12:01:02,090 INFO  [o.e.j.u.s.SslContextFactory] (main:null) (logid:) x509=X509@25c6abfa(1,h=[our acual domain name],w=[our domain name again]) for SslContextFactory@4991c0f7[provider=null,keyStore=file:///etc/cloudstack/management/combined.jks,trustStore=null]
--------------------

Management Server is listening on 8080 and 8443, though it only says tcp6 for some reason:
tcp6       3      0 :::8443                 :::*                    LISTEN
tcp6       0      0 :::8080                 :::*                    LISTEN

I removed IPv6 config from the network scripts but it remains. IPv4 on 8080 is working anyway.

Is there anything strange here?

Regards
Vincent


rohit.yadav@shapeblue.com 
www.shapeblue.com
3 London Bridge Street,  3rd floor, News Building, London  SE1 9SGUK
@shapeblue
  
 


Re: HTTPS Broken with 4.14

Posted by Vincent Hermes <vi...@gmail.com>.
Hi Rohit,

keytool -list -keystore /etc/cloudstack/management/keystore.pkcs12 (Password same as in server.properties and works)
--------------------
Keystore type: PKCS12
Keystore provider: SUN

Your keystore contains 1 entry

1, 11 Dec 2019, PrivateKeyEntry,
Certificate fingerprint (SHA-256): xx:xx:xx:xx...etc
--------------------

Converted pkcs12 to jks via "keytool -importkeystore -srckeystore combined.pkcs12 -destkeystore combined.jks -deststoretype jks" (I renamed the copy of keystore.pkcs12 to "combined.pkcs12")
Choose same password, output ok
Changed server.properties to https.keystore=/etc/cloudstack/management/combined.jks
-> Management-Server behaviour is the same
--------------------
Logs regarding keystore:
2020-06-29 12:01:02,052 INFO  [o.e.j.s.h.ContextHandler] (main:null) (logid:) Started o.e.j.w.WebAppContext@311bf055{/client,file:///usr/share/cloudstack-management/webapp/,AVAILABLE}{/usr/share/cloudstack-management/webapp}
2020-06-29 12:01:02,053 INFO  [o.e.j.s.h.ContextHandler] (main:null) (logid:) Started o.e.j.s.h.MovedContextHandler@451001e5{/,null,AVAILABLE}
2020-06-29 12:01:02,076 INFO  [o.e.j.s.AbstractConnector] (main:null) (logid:) Started ServerConnector@6f46426d{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
2020-06-29 12:01:02,090 INFO  [o.e.j.u.s.SslContextFactory] (main:null) (logid:) x509=X509@25c6abfa(1,h=[our acual domain name],w=[our domain name again]) for SslContextFactory@4991c0f7[provider=null,keyStore=file:///etc/cloudstack/management/combined.jks,trustStore=null]
--------------------

Management Server is listening on 8080 and 8443, though it only says tcp6 for some reason:
tcp6       3      0 :::8443                 :::*                    LISTEN
tcp6       0      0 :::8080                 :::*                    LISTEN

I removed IPv6 config from the network scripts but it remains. IPv4 on 8080 is working anyway.

Is there anything strange here?

Regards
Vincent


Re: HTTPS Broken with 4.14

Posted by Rohit Yadav <ro...@shapeblue.com>.
Hi Vincent,

Can you try the following:

  *   Using keytool check if the certificates exists in the keystore and are valid, and provided passphrase is also correct
  *   Try creating a keystore in jks format
  *   Check if there's any firewall rule blocking https port 8443 and verify (ex using netstat -nl | grep 8443) if management server is even listening on port 8443
  *   On any changes restart the management server

If you're able to confirm the bug, then you may log an issue here: https://github.com/apache/cloudstack/issues/new

Hope this helps.


Regards.

________________________________
From: Vincent Hermes <vi...@gmail.com>
Sent: Thursday, June 25, 2020 15:46
To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
Subject: Re: HTTPS Broken with 4.14

Hi Rohit,

yes I have set the following in my /etc/cloudstack/management/server.properties:

# The service context path where URL requests should be served
context.path=/client

# The HTTP port to be used by the management server
http.port=8080

# Max inactivity time in minutes for the session
session.timeout=1440

# Options to configure and enable HTTPS on management server
#
# For management server to pickup these configuration settings, the configured
# keystore file should exists and be readable by the management server.
https.enable=true
https.port=8443
# The keystore and manager passwords are assumed to be same.
https.keystore=/etc/cloudstack/management/keystore.pkcs12
https.keystore.password=**********

# The path to webapp directory
webapp.dir=/usr/share/cloudstack-management/webapp

# The path to access log file
access.log=/var/log/cloudstack/management/access.log

Regards
Vincent


rohit.yadav@shapeblue.com 
www.shapeblue.com
3 London Bridge Street,  3rd floor, News Building, London  SE1 9SGUK
@shapeblue
  
 


Re: HTTPS Broken with 4.14

Posted by Vincent Hermes <vi...@gmail.com>.
Hi Rohit,

yes I have set the following in my /etc/cloudstack/management/server.properties:

# The service context path where URL requests should be served
context.path=/client

# The HTTP port to be used by the management server
http.port=8080

# Max inactivity time in minutes for the session
session.timeout=1440

# Options to configure and enable HTTPS on management server
#
# For management server to pickup these configuration settings, the configured
# keystore file should exists and be readable by the management server.
https.enable=true
https.port=8443
# The keystore and manager passwords are assumed to be same.
https.keystore=/etc/cloudstack/management/keystore.pkcs12
https.keystore.password=**********

# The path to webapp directory
webapp.dir=/usr/share/cloudstack-management/webapp

# The path to access log file
access.log=/var/log/cloudstack/management/access.log

Regards
Vincent


Re: HTTPS Broken with 4.14

Posted by Rohit Yadav <ro...@shapeblue.com>.
Hi Vincent,

Have you configured your management server SSL using server.properties file?
Refer http://docs.cloudstack.apache.org/en/latest/installguide/optional_installation.html#ssl-optional


Regards.

________________________________
From: Vincent Hermes <vi...@gmail.com>
Sent: Thursday, June 25, 2020 07:20
To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
Subject: HTTPS Broken with 4.14

Hi Guys,

so we updated our Cloudstack from 4.11.3 to 4.14 in production and now our HTTPS Webserver including Primate is not responding anymore. HTTP pn Port 8080 works though.

It does not show any certificate error, in fact it even says the right issuer but the site just never loads. There is also no access.log action when connecting via HTTPS. There are no error logs in management-server.log. It just reads "https=true, port=8443", etc and loads the keystore file correctly as it seems.

We have changed the port to 8444, used various browsers, also in private mode, cache cleared, reinstalled the certificates both on the HTTP UI and the keystore file and we even installed a completely fresh MGMT Server on a CentOS7 which shows the exact same behaviour after enabling HTTPS.

Is there anything new in 4.14 regarding https? Is someone having similar issues?

Regards
Vincent

rohit.yadav@shapeblue.com 
www.shapeblue.com
3 London Bridge Street,  3rd floor, News Building, London  SE1 9SGUK
@shapeblue