You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@uniffle.apache.org by GitBox <gi...@apache.org> on 2022/08/18 08:03:17 UTC
[GitHub] [incubator-uniffle] xianjingfeng opened a new issue, #164: [Improvement] Limit the speed of memory release when drop pending events
xianjingfeng opened a new issue, #164:
URL: https://github.com/apache/incubator-uniffle/issues/164
In `org.apache.uniffle.server.ShuffleFlushManager#processPendingEvents`,OOM will happen if a large number of events need to be dropped, because `usedMemory` release immediately, but the speed of GC is not fast enough.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] jerqi commented on issue #164: [Improvement] Limit the speed of memory release when drop pending events
Posted by GitBox <gi...@apache.org>.
jerqi commented on issue #164:
URL: https://github.com/apache/incubator-uniffle/issues/164#issuecomment-1220484950
https://stackoverflow.com/questions/8719071/is-it-possible-to-get-outofmemoryerror-because-garbage-collection-too-slow I'm not sure the effect of pr.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] xianjingfeng commented on issue #164: [Improvement] Limit the speed of memory release when drop pending events
Posted by GitBox <gi...@apache.org>.
xianjingfeng commented on issue #164:
URL: https://github.com/apache/incubator-uniffle/issues/164#issuecomment-1220462077
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] xianjingfeng commented on issue #164: [Improvement] Limit the speed of memory release when drop pending events
Posted by GitBox <gi...@apache.org>.
xianjingfeng commented on issue #164:
URL: https://github.com/apache/incubator-uniffle/issues/164#issuecomment-1220708226
@zuston I misunderstood what you said just now. You are right, but GC will not delete all unused objects at once.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] xianjingfeng commented on issue #164: [Improvement] Limit the speed of memory release when drop pending events
Posted by GitBox <gi...@apache.org>.
xianjingfeng commented on issue #164:
URL: https://github.com/apache/incubator-uniffle/issues/164#issuecomment-1220691083
When to trigger GC? The direct reason for GC is that it is written too fast. Maybe it will be solved after replace grpc with netty in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] jerqi commented on issue #164: [Improvement] Limit the speed of memory release when drop pending events
Posted by GitBox <gi...@apache.org>.
jerqi commented on issue #164:
URL: https://github.com/apache/incubator-uniffle/issues/164#issuecomment-1221749615
Could you explain the two diagrams? I can't get your point.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] xianjingfeng commented on issue #164: [Improvement] Limit the speed of memory release when drop pending events
Posted by GitBox <gi...@apache.org>.
xianjingfeng commented on issue #164:
URL: https://github.com/apache/incubator-uniffle/issues/164#issuecomment-1220483174
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] xianjingfeng commented on issue #164: [Improvement] Limit the speed of memory release when drop pending events
Posted by GitBox <gi...@apache.org>.
xianjingfeng commented on issue #164:
URL: https://github.com/apache/incubator-uniffle/issues/164#issuecomment-1220488837
The purpose of this PR is to reserve more time for GC and reduce the speed of receiving data. Maybe this is not a good way, but I feel it is effective.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] jerqi commented on issue #164: [Improvement] Limit the speed of memory release when drop pending events
Posted by GitBox <gi...@apache.org>.
jerqi commented on issue #164:
URL: https://github.com/apache/incubator-uniffle/issues/164#issuecomment-1222444842
Em......If we can't write data to storage, we will occupy the more memory. It's normal. But the two diagrams can't persuade me that the gc speed lead to OOM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] zuston commented on issue #164: [Improvement] Limit the speed of memory release when drop pending events
Posted by GitBox <gi...@apache.org>.
zuston commented on issue #164:
URL: https://github.com/apache/incubator-uniffle/issues/164#issuecomment-1220434115
I'm curious about how to find OOM is caused by dropped events.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] xianjingfeng closed issue #164: [Improvement] Limit the speed of memory release when drop pending events
Posted by GitBox <gi...@apache.org>.
xianjingfeng closed issue #164: [Improvement] Limit the speed of memory release when drop pending events
URL: https://github.com/apache/incubator-uniffle/issues/164
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] xianjingfeng commented on issue #164: [Improvement] Limit the speed of memory release when drop pending events
Posted by GitBox <gi...@apache.org>.
xianjingfeng commented on issue #164:
URL: https://github.com/apache/incubator-uniffle/issues/164#issuecomment-1221920689
When `total_dropped_event_num` rises rapidly, `jvm_memory_bytes_used` rises rapidly together at the same time. Does it means that it is triggered by processing pending events?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] zuston commented on issue #164: [Improvement] Limit the speed of memory release when drop pending events
Posted by GitBox <gi...@apache.org>.
zuston commented on issue #164:
URL: https://github.com/apache/incubator-uniffle/issues/164#issuecomment-1220665611
From my understanding, if memory is not enough, it will firstly trigger GC to remove unused objects. This will stop the world.
After that, when memory is still not enough, it will OOM.
What do u think so?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org