You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/04/27 04:33:42 UTC
[GitHub] [pulsar] yangwenjie88 opened a new issue #6829: pulsar-client MultiTopicsConsumerImpl frequently fullgc
yangwenjie88 opened a new issue #6829:
URL: https://github.com/apache/pulsar/issues/6829
300 partitions topic 1200W/s, pulsar-client 2.5.1
failover SubscriptionType, When using 150 2core 4G vm, fullgc frequently appears, the connection has been reconnected, and the consumption cannot be achieved at all.
Some parameter configuration:
receiverQueueSize = 1000
maxTotalReceiverQueueSizeAcrossPartitions = 60000
Instead of using 20 16core 32G VMs when consuming, the memory usage is relatively high at startup, and there will be reconnection, which can be consumed after a while, but the speed is only 3000ms / s per partition. The performance is far from enough,
when the receiverQueueSize=3000, fullgc will appear
When the specified partition is used for single consumption, the performance is normal. MultiTopicsConsumerImpl Multi-partition consumption is estimated to have performance problems
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on issue #6829: pulsar-client MultiTopicsConsumerImpl frequently fullgc
Posted by GitBox <gi...@apache.org>.
lhotari commented on issue #6829:
URL: https://github.com/apache/pulsar/issues/6829#issuecomment-699129291
This might be related to #8138.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] jiazhai commented on issue #6829: pulsar-client MultiTopicsConsumerImpl frequently fullgc
Posted by GitBox <gi...@apache.org>.
jiazhai commented on issue #6829:
URL: https://github.com/apache/pulsar/issues/6829#issuecomment-620327431
@yangwenjie88 Would you please have a try on use larger receiverQueueSize?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yangwenjie88 commented on issue #6829: pulsar-client MultiTopicsConsumerImpl frequently fullgc
Posted by GitBox <gi...@apache.org>.
yangwenjie88 commented on issue #6829:
URL: https://github.com/apache/pulsar/issues/6829#issuecomment-620335502
> @yangwenjie88 Would you please have a try on use larger receiverQueueSize?
Have tried to increase the size, after the increase, the memory is insufficient, resulting in more frequent fullgc
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org