You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by GitBox <gi...@apache.org> on 2020/06/19 11:26:20 UTC
[GitHub] [cloudstack] Spaceman1984 opened a new pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renwew
Spaceman1984 opened a new pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156
## Description
<!--- Describe your changes in detail -->
When a host connects to a management server the host IP address and the certificate are stored in memory. This mapping is checked periodically to determine if any certificates are due to expire.
Before a certificate is renewed a few checks are done to determine if the host is connected to the management server by fetching the host record from the database. The problem here is if the wrong record is fetched, the host is not checked for renewal.
This PR improves the host record fetch from the database by looking only at hosts that are not removed.
<!-- For new features, provide link to FS, dev ML discussion etc. -->
<!-- In case of bug fix, the expected and actual behaviours, steps to reproduce. -->
<!-- When "Fixes: #<id>" is specified, the issue/PR will automatically be closed when this PR gets merged -->
<!-- For addressing multiple issues/PRs, use multiple "Fixes: #<id>" -->
<!-- Fixes: # -->
Fixes: #4129
## Types of changes
<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] New feature (non-breaking change which adds functionality)
- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
## Screenshots (if appropriate):
## How Has This Been Tested?
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to -->
<!-- see how your change affects other areas of the code, etc. -->
This has been tested by setting the ca.framework.cert.validity.period and ca.framework.cert.expiry.alert.period to the same value. This is to ensure that a certificate is up for renewal as soon as it is issued.
Then watch the management server logs to see if auto-renewal happens.
This has also been tested by using two management servers and reprovision host security keys from the second management server and still having the certs auto-renew.
<!-- Please read the [CONTRIBUTING](https://github.com/apache/cloudstack/blob/master/CONTRIBUTING.md) document -->
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Spaceman1984 edited a comment on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
Spaceman1984 edited a comment on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-648800553
@nvazquez there is no problem with the certs map losing certs when hosts disconnect because the cert map is repopulated when a host connects again.
If you provision keys from a different management server, the host will disconnect but can reconnect to any available management server in it's list.
If you test with 2 management servers and attach your debugger to both at the same time, you will see the cert map being populated on the management server the host is connecting to.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Spaceman1984 commented on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
Spaceman1984 commented on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-648821226
Auto-renewal can only happen if the host has a valid certificate. As soon as the certificate becomes invalid the host can't communicate with the management server anymore and therefore wouldn't be able to get a new certificate. So before the validity period of the certificate would run out and the host is not able to communicate with the management server, auto-renewal must happen. - The auto-renewal process was failing and therefore certificates were not being renewed. If auto-renewal is fixed, then there wouldn't be a problem with disconnected hosts.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-648648063
@Spaceman1984 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-646615551
Packaging result: ✔centos7 ✔debian. JID-1421
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-648552888
@Spaceman1984 can you kick packaging and tests again; was purgeHost being called on a host already removed in DB? But how that does affect hosts that are already connected to a management server (i.e. host not removed in DB)?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-652159406
@nvazquez yes it's because the code which adds the X509 cert to the hashmap (currently) uses the incoming client's IP address as the key. It may need checking or use host UUID/ID instead of IP address if that corrects the implementation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-651767145
@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-655317151
Packaging result: ✔centos7 ✔debian. JID-1554
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Spaceman1984 commented on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
Spaceman1984 commented on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-648648942
@rhtyd I didn't see purgeHost being called on a host already removed in my testing. What I observed, was the host being ignored because a check was done on an empty management server field.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-655371515
@blueorangutan test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Spaceman1984 commented on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renwew
Posted by GitBox <gi...@apache.org>.
Spaceman1984 commented on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-646604013
@blueorangutan package
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez commented on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
nvazquez commented on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-651766821
@blueorangutan test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-648661132
Packaging result: ✔centos7 ✔debian. JID-1443
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd merged pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
rhtyd merged pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Spaceman1984 commented on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
Spaceman1984 commented on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-648800553
@nvazquez there is no problem with the certs map losing certs when hosts disconnect because the cert map is repopulated when a host connects again.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Spaceman1984 edited a comment on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
Spaceman1984 edited a comment on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-648800553
@nvazquez there is no problem with the certs map losing certs when hosts disconnect because the cert map is repopulated when a host connects again.
If you provision keys from a different management server, the host will disconnect but can reconnect to any available management server in it's list.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-653725080
ping @nvazquez please review
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renwew
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-646604598
@Spaceman1984 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez commented on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
nvazquez commented on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-648950178
Thanks @Spaceman1984, tested manually and looks good.
@rhtyd I noticed the internal active certificates map uses the host IP as the key, which is then used for querying in the DB for that IP. Is there any reason for not using the internal host ID or the UUID as the map key and using the host IP instead?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Spaceman1984 commented on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
Spaceman1984 commented on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-648661556
@blueorangutan test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-655303763
@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-655303521
@blueorangutan package
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-651749218
@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-655764802
<b>Trillian test result (tid-2034)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 44441 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4156-t2034-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 82 look OK, 1 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL | `Failure` | 362.40 | test_vpc_redundant.py
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-651760910
Packaging result: ✔centos7 ✔debian. JID-1492
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Spaceman1984 commented on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
Spaceman1984 commented on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-646624426
@blueorangutan test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Spaceman1984 edited a comment on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
Spaceman1984 edited a comment on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-648821226
Auto-renewal can only happen if the host has a valid certificate. As soon as the certificate becomes invalid, the host can't communicate with the management server anymore and therefore wouldn't be able to get a new certificate. So before the validity period of the certificate would run out and the host is not able to communicate with the management server, auto-renewal must happen. - The auto-renewal process was failing and therefore certificates were not being renewed. If auto-renewal is fixed, then there wouldn't be a problem with disconnected hosts.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Spaceman1984 commented on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
Spaceman1984 commented on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-648647548
@blueorangutan package
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-646625054
@Spaceman1984 a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez commented on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
nvazquez commented on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-651748735
@blueorangutan package
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-648661944
@Spaceman1984 a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4156: Fixed removal of hosts from certsmap when running certificate auto-renew
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4156:
URL: https://github.com/apache/cloudstack/pull/4156#issuecomment-655372449
@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org