You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@helix.apache.org by GitBox <gi...@apache.org> on 2020/02/11 17:41:43 UTC
[GitHub] [helix] nikhilvavs opened a new issue #748: Huge CPU spike after
upgrading from 0.8.2 to 0.8.4
nikhilvavs opened a new issue #748: Huge CPU spike after upgrading from 0.8.2 to 0.8.4
URL: https://github.com/apache/helix/issues/748
We were using apache helix 0.8.2 and upgraded to using 0.8.4.
After upgrading, we saw a huge spike in the CPU usage where it jumped from around 80% to 180% with the newer library.
On profiling the service, we saw that the upgraded version was spending a lot more time in helix code.
And caused by introducing a synchronized block in [this commit](https://github.com/apache/helix/commit/8c8f79c5e545818c0e4478addea453fc42b6d003)
Is there thing we can do to improve performance?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] nikhilvavs commented on issue #748: Huge CPU spike after
upgrading from 0.8.2 to 0.8.4
Posted by GitBox <gi...@apache.org>.
nikhilvavs commented on issue #748: Huge CPU spike after upgrading from 0.8.2 to 0.8.4
URL: https://github.com/apache/helix/issues/748#issuecomment-585063817
We took flame graphs of the CPU with and without the upgraded version.
The upgraded version had 25.7% samples from `org/apache/helix/common/DedupEventProcessor.run` compared to 3.23% in the normal revision.
**Flame graph from 0.8.2:**
Although not very clear from this image, the small box next to `java/lang/Thread.run` (second row from bottom) is the box corresponding to helix code
**Flame graph from the upgraded version (0.8.4):**
You can see a significant increase in the number of samples in `DedupEventProcessor.run`
**Flame graph zoomed in on details with helix (version 0.8.4):**
Apologies for not being able to attach the complete svg.
We checked the commits in `DedupEventProcessor.java` and saw that the commit was introduced in 0.8.3. (It may have been some other function inside as well.)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] nikhilvavs edited a comment on issue #748: Huge CPU spike
after upgrading from 0.8.2 to 0.8.4 (0.8.3)
Posted by GitBox <gi...@apache.org>.
nikhilvavs edited a comment on issue #748: Huge CPU spike after upgrading from 0.8.2 to 0.8.4 (0.8.3)
URL: https://github.com/apache/helix/issues/748#issuecomment-602420386
I tested once again and found that this started in 0.8.3.
The flame graph showed that it was taking a lot of time in the deserializer, but I don't see any major changes that can cause this b/w 0.8.2 and 0.8.3 - [Diff](https://github.com/apache/helix/compare/helix-0.8.2...helix-0.8.3?diff=unified)
I would be happy to help debug this further, it would be great if you can give me some steps that I can take.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] jiajunwang commented on issue #748: Huge CPU spike after
upgrading from 0.8.2 to 0.8.4
Posted by GitBox <gi...@apache.org>.
jiajunwang commented on issue #748: Huge CPU spike after upgrading from 0.8.2 to 0.8.4
URL: https://github.com/apache/helix/issues/748#issuecomment-584794700
@nikhilvavs Thanks for reporting the issue. We will take a look at this. Could you please share more about your profiling? What makes you think it is related to the synchronized block?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] nikhilvavs commented on issue #748: Huge CPU spike after
upgrading from 0.8.2 to 0.8.4 (0.8.3)
Posted by GitBox <gi...@apache.org>.
nikhilvavs commented on issue #748: Huge CPU spike after upgrading from 0.8.2 to 0.8.4 (0.8.3)
URL: https://github.com/apache/helix/issues/748#issuecomment-602420386
I tested once again and found that this started in 0.8.3.
The flame graph showed that it was taking a lot of time in the deserializer, but I don't see any major changes that can cause this b/w 0.8.2 and 0.8.3 - [Diff](https://github.com/apache/helix/compare/helix-0.8.2...helix-0.8.3?diff=unified)
I would be happy to help debug this further, but if you can give me some steps that I can take.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org