You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/06/02 03:56:32 UTC
[GitHub] [pulsar-client-go] LvBay opened a new issue #265: High cpu press because of the default BatchingMaxPublishDelay
LvBay opened a new issue #265:
URL: https://github.com/apache/pulsar-client-go/issues/265
#### Expected behavior
When using the default configuration to create producers, it shows lower CPU usage
#### Actual behavior
When I create producers for two topics with 61 partitions, the CPU usage is high
#### Steps to reproduce
1. Create two topics with 61 partitions
2. Create failover producer with default configuration
#### System configuration
**Pulsar version**: docker image for latest
When I checked pprof, I set BatchingMaxPublishDelay to 1s, and the situation improved. But how much is the default value?
@wolfstudy
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] sijie commented on issue #265: High cpu press because of the default BatchingMaxPublishDelay
Posted by GitBox <gi...@apache.org>.
sijie commented on issue #265:
URL: https://github.com/apache/pulsar-client-go/issues/265#issuecomment-637901614
`batchMaxPublishDelay` will affect the publish latency. If your workload is latency-sensitive, you probably want to make sure the value is around milliseconds. Otherwise you will see huge publish latency.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] LvBay commented on issue #265: High cpu press because of the default BatchingMaxPublishDelay
Posted by GitBox <gi...@apache.org>.
LvBay commented on issue #265:
URL: https://github.com/apache/pulsar-client-go/issues/265#issuecomment-637256564
How much better is the default value of BatchingMaxPublishDelay?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] LvBay commented on issue #265: High cpu press because of the default BatchingMaxPublishDelay
Posted by GitBox <gi...@apache.org>.
LvBay commented on issue #265:
URL: https://github.com/apache/pulsar-client-go/issues/265#issuecomment-638572155
> 2. Share the same ticker for multiple producers (maybe using a pool of them)
Agree with this
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] LvBay closed issue #265: High cpu press because of the default BatchingMaxPublishDelay
Posted by GitBox <gi...@apache.org>.
LvBay closed issue #265:
URL: https://github.com/apache/pulsar-client-go/issues/265
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] LvBay commented on issue #265: High cpu press because of the default BatchingMaxPublishDelay
Posted by GitBox <gi...@apache.org>.
LvBay commented on issue #265:
URL: https://github.com/apache/pulsar-client-go/issues/265#issuecomment-638199281
If I use Send() instead of SendAsync(), does it mean that batchMaxPublishDelay can be set larger without affecting publish latency? @sijie
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] kyrozetera commented on issue #265: High cpu press because of the default BatchingMaxPublishDelay
Posted by GitBox <gi...@apache.org>.
kyrozetera commented on issue #265:
URL: https://github.com/apache/pulsar-client-go/issues/265#issuecomment-797626590
> 1. Do not schedule the task until the first SendAsync() call
>2. Share the same ticker for multiple producers (maybe using a pool of them)
Have either of these options been pushed forward? This seems like a good idea, especially if SendAsync is used very sparingly, or if all producers have the same publish delay. All those tickers can eat up a lot of CPU for an idle service.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] merlimat commented on issue #265: High cpu press because of the default BatchingMaxPublishDelay
Posted by GitBox <gi...@apache.org>.
merlimat commented on issue #265:
URL: https://github.com/apache/pulsar-client-go/issues/265#issuecomment-638381043
Yes, it will not affect the publish latency of `Send()` operations.
There are few possible improvements here:
1. Do not schedule the task until the first `SendAsync()` call
2. Share the same ticker for multiple producers (maybe using a pool of them)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] LvBay edited a comment on issue #265: High cpu press because of the default BatchingMaxPublishDelay
Posted by GitBox <gi...@apache.org>.
LvBay edited a comment on issue #265:
URL: https://github.com/apache/pulsar-client-go/issues/265#issuecomment-638572155
> 2. Share the same ticker for multiple producers (maybe using a pool of them)
Agree with this
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org