You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by GitBox <gi...@apache.org> on 2020/10/06 19:56:27 UTC
[GitHub] [cloudstack] ustcweizhou opened a new pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
ustcweizhou opened a new pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386
## Description
<!--- Describe your changes in detail -->
in 4.15, keepalived in redundant VRs keeps restarting every minute.
After debugging, I found it happens when update config for monitor service.
it is because keepalived process is changed in Debian 10.
in Debian 9 (systemvm for 4.14),
```
root@r-1969-VM:~# ps -ef|grep keepalived
root 16324 1 0 09:53 ? 00:00:04 /usr/sbin/keepalived
root 16325 16324 0 09:53 ? 00:00:04 /usr/sbin/keepalived
root 16326 16324 0 09:53 ? 00:00:14 /usr/sbin/keepalived
```
in Debian 10 (systemvm for 4.15), processes end with "--dont-fork"
```
root@r-2040-VM:~# ps -ef|grep keepalived
root 5237 1 0 16:40 ? 00:00:00 /usr/sbin/keepalived --dont-fork
root 5239 5237 0 16:40 ? 00:00:03 /usr/sbin/keepalived --dont-fork
```
<!-- For new features, provide link to FS, dev ML discussion etc. -->
<!-- In case of bug fix, the expected and actual behaviours, steps to reproduce. -->
<!-- When "Fixes: #<id>" is specified, the issue/PR will automatically be closed when this PR gets merged -->
<!-- For addressing multiple issues/PRs, use multiple "Fixes: #<id>" -->
<!-- Fixes: # -->
## Types of changes
<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] New feature (non-breaking change which adds functionality)
- [X] Bug fix (non-breaking change which fixes an issue)
- [ ] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
## Screenshots (if appropriate):
## How Has This Been Tested?
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to -->
<!-- see how your change affects other areas of the code, etc. -->
<!-- Please read the [CONTRIBUTING](https://github.com/apache/cloudstack/blob/master/CONTRIBUTING.md) document -->
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on a change in pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on a change in pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386#discussion_r501492064
##########
File path: systemvm/debian/opt/cloud/bin/cs/CsRedundant.py
##########
@@ -194,7 +194,7 @@ def _redundant_on(self):
heartbeat_cron.commit()
proc = CsProcess(['/usr/sbin/keepalived'])
Review comment:
@ustcweizhou specifying "/usr/sbin/keepalived" here may not be required as grep is used instead.
I understand this issue is with _keepalived_ process, but on a broader view, other processes might have similar issue later, can this be fixed in _CsProcess.py find()_ ?
##########
File path: systemvm/debian/opt/cloud/bin/cs/CsRedundant.py
##########
@@ -194,7 +194,7 @@ def _redundant_on(self):
heartbeat_cron.commit()
proc = CsProcess(['/usr/sbin/keepalived'])
Review comment:
ok @weizhouapache , got it. Note that, if the same issue repeats for other processes later, then _CsProcess.py_ have to be fixed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386#issuecomment-707613367
@blueorangutan test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386#issuecomment-707108041
@DaanHoogland a Trillian-Jenkins test job (ubuntu mgmt + kvm-ubuntu) has been kicked to run smoke tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] weizhouapache commented on a change in pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
weizhouapache commented on a change in pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386#discussion_r501614873
##########
File path: systemvm/debian/opt/cloud/bin/cs/CsRedundant.py
##########
@@ -194,7 +194,7 @@ def _redundant_on(self):
heartbeat_cron.commit()
proc = CsProcess(['/usr/sbin/keepalived'])
Review comment:
@sureshanaparti yes, of course.
To be sure that is not regression issue, I choosed a easier way to fix it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386#issuecomment-707178252
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386#issuecomment-706935043
ok @weizhouapache , will run tests once
@blueorangutan package
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] weizhouapache commented on pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
weizhouapache commented on pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386#issuecomment-706380544
@rhtyd @DaanHoogland this needs to be merged asap.
the test failure should be fixed by this pr.
Test | Result | Time (s) | Test File
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers | Failure | 290.08 | test_vpc_redundant.py
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386#issuecomment-707613639
@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386#issuecomment-706935726
@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on a change in pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on a change in pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386#discussion_r501618064
##########
File path: systemvm/debian/opt/cloud/bin/cs/CsRedundant.py
##########
@@ -194,7 +194,7 @@ def _redundant_on(self):
heartbeat_cron.commit()
proc = CsProcess(['/usr/sbin/keepalived'])
Review comment:
ok @weizhouapache , got it. Note that, if the same issue repeats for other processes later, then _CsProcess.py_ have to be fixed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] weizhouapache commented on a change in pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
weizhouapache commented on a change in pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386#discussion_r501614873
##########
File path: systemvm/debian/opt/cloud/bin/cs/CsRedundant.py
##########
@@ -194,7 +194,7 @@ def _redundant_on(self):
heartbeat_cron.commit()
proc = CsProcess(['/usr/sbin/keepalived'])
Review comment:
@sureshanaparti yes, of course.
To be sure that is not regression issue, I choosed a easier way to fix it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386#issuecomment-707112130
@blueorangutan test ubuntu kvm-ubuntu
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386#issuecomment-708213341
none of these errors are related, @rhtyd @sureshanaparti or others, can we merge?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386#issuecomment-706954882
Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2158
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386#issuecomment-707096930
@blueorangutan test ubuntu kvm-ubuntu keepEnv
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland removed a comment on pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
DaanHoogland removed a comment on pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386#issuecomment-707158283
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386#issuecomment-707159076
@DaanHoogland unsupported parameters provided. Supported mgmt server os are: `centos6, centos7, ubuntu`. Supported hypervisors are: `kvm-centos6, kvm-centos7, kvm-ubuntu, xenserver-71, xenserver-65sp1, xenserver-62sp1, vmware-65u2, vmware-60u2, vmware-55u3, vmware-51u1, vmware-50u1`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland removed a comment on pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
DaanHoogland removed a comment on pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386#issuecomment-707112130
@blueorangutan test ubuntu kvm-ubuntu
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386#issuecomment-708310468
The same PR was sent to 4.14 https://github.com/apache/cloudstack/pull/4384 - should we close this one and merge the other one?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on a change in pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on a change in pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386#discussion_r501492064
##########
File path: systemvm/debian/opt/cloud/bin/cs/CsRedundant.py
##########
@@ -194,7 +194,7 @@ def _redundant_on(self):
heartbeat_cron.commit()
proc = CsProcess(['/usr/sbin/keepalived'])
Review comment:
@ustcweizhou specifying "/usr/sbin/keepalived" here may not be required as grep is used instead.
I understand this issue is with _keepalived_ process, but on a broader view, other processes might have similar issue later, can this be fixed in _CsProcess.py find()_ ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386#issuecomment-707995506
<b>Trillian test result (tid-2930)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 38742 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4386-t2930-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 83 look OK, 2 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_07_deploy_kubernetes_ha_cluster | `Failure` | 3609.57 | test_kubernetes_clusters.py
test_08_deploy_and_upgrade_kubernetes_ha_cluster | `Failure` | 0.09 | test_kubernetes_clusters.py
test_09_delete_kubernetes_ha_cluster | `Failure` | 0.06 | test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown | `Error` | 75.82 | test_kubernetes_clusters.py
test_hostha_kvm_host_fencing | `Error` | 175.07 | test_hostha_kvm.py
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] weizhouapache closed pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
weizhouapache closed pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386#issuecomment-707112843
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386#issuecomment-707174978
using this PR for some testing seems not fair :(
@blueorangutan test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland removed a comment on pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
DaanHoogland removed a comment on pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386#issuecomment-707096930
@blueorangutan test ubuntu kvm-ubuntu keepEnv
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386#issuecomment-707159076
@DaanHoogland unsupported parameters provided. Supported mgmt server os are: `centos6, centos7, ubuntu`. Supported hypervisors are: `kvm-centos6, kvm-centos7, kvm-ubuntu, xenserver-71, xenserver-65sp1, xenserver-62sp1, vmware-65u2, vmware-60u2, vmware-55u3, vmware-51u1, vmware-50u1`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386#issuecomment-707112843
@DaanHoogland a Trillian-Jenkins test job (ubuntu mgmt + kvm-ubuntu) has been kicked to run smoke tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386#issuecomment-707178254
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #4386: systemvm: fix keepalived is always restarted when update config for monitor service
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #4386:
URL: https://github.com/apache/cloudstack/pull/4386#issuecomment-707158283
@blueorangutan test keepEnv
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org