You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/08/07 09:09:48 UTC
[GitHub] [pulsar] hangc0276 opened a new pull request #7782: make pulsar executor pool size configurable
hangc0276 opened a new pull request #7782:
URL: https://github.com/apache/pulsar/pull/7782
#### Motivation
The pulsar executor pool size number is hard code to `20` when pulsar service start, it should be configurable in broker.conf.
```
private final ScheduledExecutorService executor = Executors.newScheduledThreadPool(20,
new DefaultThreadFactory("pulsar"));
```
#### Changes
make the executor pool size configurable in broker.conf
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] hangc0276 commented on pull request #7782: make pulsar executor pool size configurable
Posted by GitBox <gi...@apache.org>.
hangc0276 commented on pull request #7782:
URL: https://github.com/apache/pulsar/pull/7782#issuecomment-671066345
> As a followup to these changes in thread pool sizes, it would be nice to reduce the size of these pools when running the unit tests
Yes, for the unit tests framework, i will reduce the pool size.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui merged pull request #7782: make pulsar executor pool size configurable
Posted by GitBox <gi...@apache.org>.
codelipenghui merged pull request #7782:
URL: https://github.com/apache/pulsar/pull/7782
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] hangc0276 commented on a change in pull request #7782: make pulsar executor pool size configurable
Posted by GitBox <gi...@apache.org>.
hangc0276 commented on a change in pull request #7782:
URL: https://github.com/apache/pulsar/pull/7782#discussion_r467597382
##########
File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/PulsarService.java
##########
@@ -170,8 +170,8 @@
private LocalZooKeeperConnectionService localZooKeeperConnectionProvider;
private Compactor compactor;
- private final ScheduledExecutorService executor = Executors.newScheduledThreadPool(20,
- new DefaultThreadFactory("pulsar"));
Review comment:
Thanks for your review, i will update the pr.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] hangc0276 commented on pull request #7782: make pulsar executor pool size configurable
Posted by GitBox <gi...@apache.org>.
hangc0276 commented on pull request #7782:
URL: https://github.com/apache/pulsar/pull/7782#issuecomment-671134311
> > As a followup to these changes in thread pool sizes, it would be nice to reduce the size of these pools when running the unit tests
>
> Yes, for the unit tests framework, i will reduce the pool size.
@merlimat @jiazhai I have changed the default executor pool size to Runtime.getRuntime().availableProcessors() and update unit test executor pool size to 5, please take a look, thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] merlimat commented on pull request #7782: make pulsar executor pool size configurable
Posted by GitBox <gi...@apache.org>.
merlimat commented on pull request #7782:
URL: https://github.com/apache/pulsar/pull/7782#issuecomment-670948487
As a followup to these changes in thread pool sizes, it would be nice to reduce the size of these pools when running the unit tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] jiazhai commented on a change in pull request #7782: make pulsar executor pool size configurable
Posted by GitBox <gi...@apache.org>.
jiazhai commented on a change in pull request #7782:
URL: https://github.com/apache/pulsar/pull/7782#discussion_r467591829
##########
File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/PulsarService.java
##########
@@ -170,8 +170,8 @@
private LocalZooKeeperConnectionService localZooKeeperConnectionProvider;
private Compactor compactor;
- private final ScheduledExecutorService executor = Executors.newScheduledThreadPool(20,
- new DefaultThreadFactory("pulsar"));
Review comment:
@hangc0276 would you please help changed this value as Matteo suggested? `Runtime.getRuntime().availableProcessors()`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] hangc0276 commented on pull request #7782: make pulsar executor pool size configurable
Posted by GitBox <gi...@apache.org>.
hangc0276 commented on pull request #7782:
URL: https://github.com/apache/pulsar/pull/7782#issuecomment-671197751
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] hangc0276 commented on a change in pull request #7782: make pulsar executor pool size configurable
Posted by GitBox <gi...@apache.org>.
hangc0276 commented on a change in pull request #7782:
URL: https://github.com/apache/pulsar/pull/7782#discussion_r467597454
##########
File path: conf/broker.conf
##########
@@ -61,6 +61,12 @@ numIOThreads=
# Number of threads to use for HTTP requests processing. Default is set to 2 * Runtime.getRuntime().availableProcessors()
numHttpServerThreads=
+# Number of thread pool size to use for pulsar broker service.
+# The executor in thread pool will do basic broker operation like load/unload bundle, update managedLedgerConfig,
+# update topic/subscription/replicator message dispatch rate, do leader election etc.
+# Default is 20
+numExecutorThreadPoolSize=
Review comment:
ok, it's better to change it to Runtime.getRuntime().availableProcessors(). it will update the pr.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] merlimat commented on a change in pull request #7782: make pulsar executor pool size configurable
Posted by GitBox <gi...@apache.org>.
merlimat commented on a change in pull request #7782:
URL: https://github.com/apache/pulsar/pull/7782#discussion_r467481796
##########
File path: conf/broker.conf
##########
@@ -61,6 +61,12 @@ numIOThreads=
# Number of threads to use for HTTP requests processing. Default is set to 2 * Runtime.getRuntime().availableProcessors()
numHttpServerThreads=
+# Number of thread pool size to use for pulsar broker service.
+# The executor in thread pool will do basic broker operation like load/unload bundle, update managedLedgerConfig,
+# update topic/subscription/replicator message dispatch rate, do leader election etc.
+# Default is 20
+numExecutorThreadPoolSize=
Review comment:
I would be better to default to something like `Runtime.getRuntime().availableProcessors()`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org