You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by GitBox <gi...@apache.org> on 2021/02/02 11:10:54 UTC

[GitHub] [cloudstack] ustcweizhou opened a new pull request #4623: server: Fix update capacity for hosts take long time if there are many service offerings

ustcweizhou opened a new pull request #4623:
URL: https://github.com/apache/cloudstack/pull/4623


   ### Description
   
   This PR fixes the issue that update capacity for hosts take long time if there are many service offerings.
   
   Steps to reproduce the issue:
   
   (1)Create 10000 service offerings (by db changes or cloudmonkey).
   
   (2) Check the total time of periodical capacity check in cloudstack.
   
   Without this patch, it spend 2.5 seconds (2 hosts)
   ```
   2021-01-15 16:10:12,793 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-5d5f3b3b) (logid:f5eb68ba) Running Capacity Checker ...
   2021-01-15 16:10:15,287 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-5d5f3b3b) (logid:f5eb68ba) Done running Capacity Checker ...
   ```
   
   With this patch ,it spend 1.3 seconds (2 hosts)
   ```
   2021-01-15 16:12:43,604 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-a2a7f3f1) (logid:f7e0a4c5) Running Capacity Checker ...
   2021-01-15 16:12:44,927 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-a2a7f3f1) (logid:f7e0a4c5) Done running Capacity Checker ...
   ```
   
   If there are 100 hosts, the total time will be reduced from 100+ seconds to around 10 seconds.
   This helps a lot to reduce the execution time of prometheus exporter.
   
   
   <!--- Describe your changes in DETAIL - And how has behaviour functionally changed. -->
   
   <!-- For new features, provide link to FS, dev ML discussion etc. -->
   <!-- In case of bug fix, the expected and actual behaviours, steps to reproduce. -->
   
   <!-- When "Fixes: #<id>" is specified, the issue/PR will automatically be closed when this PR gets merged -->
   <!-- For addressing multiple issues/PRs, use multiple "Fixes: #<id>" -->
   <!-- Fixes: # -->
   
   <!--- ********************************************************************************* -->
   <!--- NOTE: AUTOMATATION USES THE DESCRIPTIONS TO SET LABELS AND PRODUCE DOCUMENTATION. -->
   <!--- PLEASE PUT AN 'X' in only **ONE** box -->
   <!--- ********************************************************************************* -->
   
   ### Types of changes
   
   - [ ] Breaking change (fix or feature that would cause existing functionality to change)
   - [ ] New feature (non-breaking change which adds functionality)
   - [X] Bug fix (non-breaking change which fixes an issue)
   - [ ] Enhancement (improves an existing feature and functionality)
   - [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
   
   ### Feature/Enhancement Scale or Bug Severity
   
   #### Feature/Enhancement Scale
   
   - [ ] Major
   - [ ] Minor
   
   #### Bug Severity
   
   - [ ] BLOCKER
   - [ ] Critical
   - [ ] Major
   - [X] Minor
   - [ ] Trivial
   
   
   ### Screenshots (if appropriate):
   
   
   ### How Has This Been Tested?
   <!-- Please describe in detail how you tested your changes. -->
   <!-- Include details of your testing environment, and the tests you ran to -->
   <!-- see how your change affects other areas of the code, etc. -->
   
   
   <!-- Please read the [CONTRIBUTING](https://github.com/apache/cloudstack/blob/master/CONTRIBUTING.md) document -->
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] blueorangutan commented on pull request #4623: server: Fix update capacity for hosts take long time if there are many service offerings

Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4623:
URL: https://github.com/apache/cloudstack/pull/4623#issuecomment-768319524


   @rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] shwstppr commented on pull request #4623: server: Fix update capacity for hosts take long time if there are many service offerings

Posted by GitBox <gi...@apache.org>.
shwstppr commented on pull request #4623:
URL: https://github.com/apache/cloudstack/pull/4623#issuecomment-771562637


   Running failed test manually @weizhouapache @rhtyd 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] blueorangutan commented on pull request #4623: server: Fix update capacity for hosts take long time if there are many service offerings

Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4623:
URL: https://github.com/apache/cloudstack/pull/4623#issuecomment-768362004


   Packaging result: ✔centos7 ✖centos8 ✔debian. JID-2607


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] blueorangutan commented on pull request #4623: server: Fix update capacity for hosts take long time if there are many service offerings

Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4623:
URL: https://github.com/apache/cloudstack/pull/4623#issuecomment-771561977


   @rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] weizhouapache commented on pull request #4623: server: Fix update capacity for hosts take long time if there are many service offerings

Posted by GitBox <gi...@apache.org>.
weizhouapache commented on pull request #4623:
URL: https://github.com/apache/cloudstack/pull/4623#issuecomment-771560401


   > @ustcweizhou can you review the `test_01_nic` failure?
   
   @rhtyd I ran the test on 4.14 and 4.15, both succeed.
   can you re-kick the test ?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] rhtyd commented on pull request #4623: server: Fix update capacity for hosts take long time if there are many service offerings

Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4623:
URL: https://github.com/apache/cloudstack/pull/4623#issuecomment-768318576


   @blueorangutan package
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] shwstppr commented on pull request #4623: server: Fix update capacity for hosts take long time if there are many service offerings

Posted by GitBox <gi...@apache.org>.
shwstppr commented on pull request #4623:
URL: https://github.com/apache/cloudstack/pull/4623#issuecomment-771676587


   > @ustcweizhou can you review the `test_01_nic` failure?
   
   Failing test verified manually,
   ```
   ==== Marvin Init Started ====
   
   === Marvin Parse Config Successful ===
   
   === Marvin Setting TestData Successful===
   
   ==== Log Folder Path: /marvin/MarvinLogs/Feb_02_2021_14_27_56_UX96H4. All logs will be available here ====
   
   === Marvin Init Logging Successful===
   
   ==== Marvin Init Successful ====
   === TestName: test_01_nic | Status : SUCCESS ===
   
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] rhtyd merged pull request #4623: server: Fix update capacity for hosts take long time if there are many service offerings

Posted by GitBox <gi...@apache.org>.
rhtyd merged pull request #4623:
URL: https://github.com/apache/cloudstack/pull/4623


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] rhtyd commented on pull request #4623: server: Fix update capacity for hosts take long time if there are many service offerings

Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4623:
URL: https://github.com/apache/cloudstack/pull/4623#issuecomment-770682156


   @blueorangutan test


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] blueorangutan commented on pull request #4623: server: Fix update capacity for hosts take long time if there are many service offerings

Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4623:
URL: https://github.com/apache/cloudstack/pull/4623#issuecomment-771094777


   <b>Trillian test result (tid-3457)</b>
   Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
   Total time taken: 36001 seconds
   Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4623-t3457-kvm-centos7.zip
   Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
   Intermittent failure detected: /marvin/tests/smoke/test_nic.py
   Smoke tests completed. 81 look OK, 2 have error(s)
   Only failed tests results shown below:
   
   
   Test | Result | Time (s) | Test File
   --- | --- | --- | ---
   test_03_deploy_and_upgrade_kubernetes_cluster | `Failure` | 258.93 | test_kubernetes_clusters.py
   test_01_nic | `Error` | 50.06 | test_nic.py
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] weizhouapache commented on pull request #4623: server: Fix update capacity for hosts take long time if there are many service offerings

Posted by GitBox <gi...@apache.org>.
weizhouapache commented on pull request #4623:
URL: https://github.com/apache/cloudstack/pull/4623#issuecomment-771560401


   > @ustcweizhou can you review the `test_01_nic` failure?
   
   @rhtyd I ran the test on 4.14 and 4.15, both succeed.
   can you re-kick the test ?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] blueorangutan commented on pull request #4623: server: Fix update capacity for hosts take long time if there are many service offerings

Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4623:
URL: https://github.com/apache/cloudstack/pull/4623#issuecomment-771584413


   Packaging result: ✔centos7 ✖centos8 ✔debian. JID-2633


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] blueorangutan commented on pull request #4623: server: Fix update capacity for hosts take long time if there are many service offerings

Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4623:
URL: https://github.com/apache/cloudstack/pull/4623#issuecomment-771561977






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] blueorangutan commented on pull request #4623: server: Fix update capacity for hosts take long time if there are many service offerings

Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4623:
URL: https://github.com/apache/cloudstack/pull/4623#issuecomment-772201153


   <b>Trillian test result (tid-3474)</b>
   Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
   Total time taken: 47885 seconds
   Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4623-t3474-kvm-centos7.zip
   Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
   Smoke tests completed. 82 look OK, 1 have error(s)
   Only failed tests results shown below:
   
   
   Test | Result | Time (s) | Test File
   --- | --- | --- | ---
   test_01_deploy_kubernetes_cluster | `Failure` | 3602.88 | test_kubernetes_clusters.py
   test_03_deploy_and_upgrade_kubernetes_cluster | `Failure` | 251.15 | test_kubernetes_clusters.py
   test_08_deploy_and_upgrade_kubernetes_ha_cluster | `Failure` | 166.53 | test_kubernetes_clusters.py
   ContextSuite context=TestKubernetesCluster>:teardown | `Error` | 760.79 | test_kubernetes_clusters.py
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] blueorangutan commented on pull request #4623: server: Fix update capacity for hosts take long time if there are many service offerings

Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4623:
URL: https://github.com/apache/cloudstack/pull/4623#issuecomment-770683231


   @rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] shwstppr commented on pull request #4623: server: Fix update capacity for hosts take long time if there are many service offerings

Posted by GitBox <gi...@apache.org>.
shwstppr commented on pull request #4623:
URL: https://github.com/apache/cloudstack/pull/4623#issuecomment-771562637






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] rhtyd commented on pull request #4623: server: Fix update capacity for hosts take long time if there are many service offerings

Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4623:
URL: https://github.com/apache/cloudstack/pull/4623#issuecomment-771561447


   @blueorangutan package 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] blueorangutan commented on pull request #4623: server: Fix update capacity for hosts take long time if there are many service offerings

Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4623:
URL: https://github.com/apache/cloudstack/pull/4623#issuecomment-771652658


   @DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] weizhouapache closed pull request #4623: server: Fix update capacity for hosts take long time if there are many service offerings

Posted by GitBox <gi...@apache.org>.
weizhouapache closed pull request #4623:
URL: https://github.com/apache/cloudstack/pull/4623


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] DaanHoogland commented on pull request #4623: server: Fix update capacity for hosts take long time if there are many service offerings

Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #4623:
URL: https://github.com/apache/cloudstack/pull/4623#issuecomment-771652184


   @blueorangutan test


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] rhtyd commented on pull request #4623: server: Fix update capacity for hosts take long time if there are many service offerings

Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4623:
URL: https://github.com/apache/cloudstack/pull/4623#issuecomment-771541715






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] rhtyd commented on pull request #4623: server: Fix update capacity for hosts take long time if there are many service offerings

Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4623:
URL: https://github.com/apache/cloudstack/pull/4623#issuecomment-771541715


   @ustcweizhou can you review the `test_01_nic` failure?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] weizhouapache closed pull request #4623: server: Fix update capacity for hosts take long time if there are many service offerings

Posted by GitBox <gi...@apache.org>.
weizhouapache closed pull request #4623:
URL: https://github.com/apache/cloudstack/pull/4623


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] rhtyd merged pull request #4623: server: Fix update capacity for hosts take long time if there are many service offerings

Posted by GitBox <gi...@apache.org>.
rhtyd merged pull request #4623:
URL: https://github.com/apache/cloudstack/pull/4623


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] DaanHoogland commented on pull request #4623: server: Fix update capacity for hosts take long time if there are many service offerings

Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #4623:
URL: https://github.com/apache/cloudstack/pull/4623#issuecomment-771652184


   @blueorangutan test


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org