You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by GitBox <gi...@apache.org> on 2020/10/14 22:35:17 UTC

[GitHub] [cloudstack] rhtyd opened a new pull request #4407: packaging: enable Parallel Collector GC for management server

rhtyd opened a new pull request #4407:
URL: https://github.com/apache/cloudstack/pull/4407


   The default GC algorithm G1 that is enabled by default with Java11
   serves well on multiprocessor machines with large amount of memory where
   GC is probablistic with low pauses, where response time is more
   important than throughput and GC is kept shorter.
   
   The CloudStack management server is largely a multi-threaded server
   application that handles and orchestrates several network requests, and
   has the default max. heap size of only 2G that can be considered a
   small/medium application from a heap size perspective. Perhaps a more
   aggresive GC algorithm such as ParallelGC as used in Java8 and before
   (that is previous CloudStack releases) would serve better for throughput
   and cause more aggressive GC.
   
   This PR propose a change in default GC algorithm to avoid OOM issues.
   
   Reference: https://docs.oracle.com/en/java/javase/11/gctuning/available-collectors.html#GUID-13943556-F521-4287-AAAA-AE5DE68777CD
   
   ## Types of changes
   <!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
   - [ ] Breaking change (fix or feature that would cause existing functionality to change)
   - [ ] New feature (non-breaking change which adds functionality)
   - [x] Bug fix (non-breaking change which fixes an issue)
   - [x] Enhancement (improves an existing feature and functionality)
   - [ ] Cleanup (Code refactoring and cleanup, that may add test cases)


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] rhtyd merged pull request #4407: packaging: enable Parallel Collector GC for management server

Posted by GitBox <gi...@apache.org>.
rhtyd merged pull request #4407:
URL: https://github.com/apache/cloudstack/pull/4407


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] sureshanaparti commented on pull request #4407: packaging: enable Parallel Collector GC for management server

Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on pull request #4407:
URL: https://github.com/apache/cloudstack/pull/4407#issuecomment-708980393


   @rhtyd Good to see change in the GC for management server process. Even though Parallel GC is the fastest, there are some side effects of using Parallel GC.
   
   - Parallel GC is the default GC in Java 8, which is changed to G1 from Java 9 [1], due to higher GC pauses.
   - All application threads are stopped during GC, can impact response time / latency. Check some use cases for Parallel GC here [2].
   
   [1] http://openjdk.java.net/jeps/248
   [2] https://www.informit.com/articles/article.aspx?p=2496621&seqNum=2
   
   I think, it is better to control such pause times using option "-XX:MaxGCPauseMillis", when we opt for Parallel GC.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] blueorangutan commented on pull request #4407: packaging: enable Parallel Collector GC for management server

Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4407:
URL: https://github.com/apache/cloudstack/pull/4407#issuecomment-709626317


   <b>Trillian test result (tid-2967)</b>
   Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
   Total time taken: 42818 seconds
   Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4407-t2967-vmware-67u3.zip
   Smoke tests completed. 83 look OK, 0 have error(s)
   Only failed tests results shown below:
   
   
   Test | Result | Time (s) | Test File
   --- | --- | --- | ---
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] blueorangutan commented on pull request #4407: packaging: enable Parallel Collector GC for management server

Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4407:
URL: https://github.com/apache/cloudstack/pull/4407#issuecomment-708709449


   Packaging result: ✔centos7 ✖centos8 ✔debian. JID-2182


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] blueorangutan commented on pull request #4407: packaging: enable Parallel Collector GC for management server

Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4407:
URL: https://github.com/apache/cloudstack/pull/4407#issuecomment-709082432


   Packaging result: ✔centos7 ✖centos8 ✔debian. JID-2185


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] blueorangutan commented on pull request #4407: packaging: enable Parallel Collector GC for management server

Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4407:
URL: https://github.com/apache/cloudstack/pull/4407#issuecomment-709043596


   @rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] blueorangutan commented on pull request #4407: packaging: enable Parallel Collector GC for management server

Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4407:
URL: https://github.com/apache/cloudstack/pull/4407#issuecomment-709555744


   <b>Trillian test result (tid-2968)</b>
   Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
   Total time taken: 31997 seconds
   Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4407-t2968-kvm-centos7.zip
   Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
   Smoke tests completed. 83 look OK, 0 have error(s)
   Only failed tests results shown below:
   
   
   Test | Result | Time (s) | Test File
   --- | --- | --- | ---
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] rhtyd commented on pull request #4407: packaging: enable Parallel Collector GC for management server

Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4407:
URL: https://github.com/apache/cloudstack/pull/4407#issuecomment-709113447


   Let's do one KVM too
   @blueorangutan test


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] blueorangutan commented on pull request #4407: packaging: enable Parallel Collector GC for management server

Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4407:
URL: https://github.com/apache/cloudstack/pull/4407#issuecomment-709116591


   @rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] blueorangutan commented on pull request #4407: packaging: enable Parallel Collector GC for management server

Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4407:
URL: https://github.com/apache/cloudstack/pull/4407#issuecomment-709084616


   @rhtyd a Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] rhtyd commented on pull request #4407: packaging: enable Parallel Collector GC for management server

Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4407:
URL: https://github.com/apache/cloudstack/pull/4407#issuecomment-708910333


   Smoketests against vmware kicked


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] blueorangutan commented on pull request #4407: packaging: enable Parallel Collector GC for management server

Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4407:
URL: https://github.com/apache/cloudstack/pull/4407#issuecomment-709016612


   <b>Trillian test result (tid-2957)</b>
   Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
   Total time taken: 32877 seconds
   Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4407-t2957-vmware-67u3.zip
   Smoke tests completed. 83 look OK, 0 have error(s)
   Only failed tests results shown below:
   
   
   Test | Result | Time (s) | Test File
   --- | --- | --- | ---
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] rhtyd commented on pull request #4407: packaging: enable Parallel Collector GC for management server

Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4407:
URL: https://github.com/apache/cloudstack/pull/4407#issuecomment-709083094


   @blueorangutan test centos7 vmware-67u3


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] rhtyd commented on pull request #4407: packaging: enable Parallel Collector GC for management server

Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4407:
URL: https://github.com/apache/cloudstack/pull/4407#issuecomment-710022187


   Heap status seen under control, after over 24hrs:
   ![Screenshot from 2020-10-16 18-02-27](https://user-images.githubusercontent.com/95203/96259453-3f593f00-0fdb-11eb-9b35-f3b104bfc011.png)
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] rhtyd commented on pull request #4407: packaging: enable Parallel Collector GC for management server

Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4407:
URL: https://github.com/apache/cloudstack/pull/4407#issuecomment-709025481


   @sureshanaparti okay I'll add that and do some tests, however, the max. heap size is very small (2G) the pauses would affect memory-heavy application such as databases or some sort of application that have huge heaps (10-100s of GBs). I think given CloudStack is largely a network application with small heap of 2G (default) even the pauses won't affect it (or its threads) much.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] rhtyd commented on pull request #4407: packaging: enable Parallel Collector GC for management server

Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4407:
URL: https://github.com/apache/cloudstack/pull/4407#issuecomment-711773061


   After 94+hrs with the changes:
   ![Screenshot from 2020-10-19 13-06-02](https://user-images.githubusercontent.com/95203/96415560-6a76a500-120c-11eb-91c9-341d2ad678c7.png)
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] rhtyd commented on pull request #4407: packaging: enable Parallel Collector GC for management server

Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4407:
URL: https://github.com/apache/cloudstack/pull/4407#issuecomment-709042595


   @sureshanaparti can you check the changes now? I've added the `-XX:MaxGCPauseMillis=500` Thanks
   
   @blueorangutan package


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] rhtyd commented on pull request #4407: packaging: enable Parallel Collector GC for management server

Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4407:
URL: https://github.com/apache/cloudstack/pull/4407#issuecomment-708696183


   @blueorangutan package


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org