You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/07/26 03:29:11 UTC
[GitHub] [pulsar] hangc0276 opened a new pull request, #15000: [improve][broker]Expose Netty threads num config to bookkeeper.conf
hangc0276 opened a new pull request, #15000:
URL: https://github.com/apache/pulsar/pull/15000
### Motivation
https://github.com/apache/bookkeeper/pull/1612 this PR make the Netty io threads number configurable, but doesn't expose to the `bookkeeper.conf`. This parameter will be usually changed in performance tuning especially deploy this bookie on bare metal which have many cpus. For operation engineer who is unfamiliar with the source code will be hard to do performance tuning.
### Modification
Expose this parameter configuration into `bookkeeper.conf` and add more explanation about it.
### Documentation
Check the box below or label this PR directly (if you have committer privilege).
Need to update docs?
- [x] `doc-required`
(If you need help on updating docs, create a doc issue)
- [ ] `no-need-doc`
(Please explain why)
- [ ] `doc`
(If this PR contains doc changes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] tisonkun commented on pull request #15000: [improve][broker] Expose Netty threads num config to bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
tisonkun commented on PR #15000:
URL: https://github.com/apache/pulsar/pull/15000#issuecomment-1366355222
CI steadily fails. Closed.
You can resubmit this patch and try to fix the tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] momo-jun commented on a diff in pull request #15000: [improve][broker]Expose Netty threads num config to bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
momo-jun commented on code in PR #15000:
URL: https://github.com/apache/pulsar/pull/15000#discussion_r867728312
##########
conf/bookkeeper.conf:
##########
@@ -158,6 +158,10 @@ numReadWorkerThreads=8
# (i.e. recovery reads and adds, and fencing).
numHighPriorityWorkerThreads=8
+# This is the number of threads used by Netty to handle TCP connections.
+# Default is 2 * Runtime.getRuntime().availableProcessors()
+serverNumIOThreads=
Review Comment:
The parameter name (`serverNumIOThreads`) is different from that of broker (`numIOThreads`). Does it make sense to keep it aligned?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] hangc0276 closed pull request #15000: [improve][broker]Expose Netty threads num config to bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
hangc0276 closed pull request #15000: [improve][broker]Expose Netty threads num config to bookkeeper.conf
URL: https://github.com/apache/pulsar/pull/15000
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Anonymitaet commented on pull request #15000: [improve][broker]Expose Netty threads num config to bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
Anonymitaet commented on PR #15000:
URL: https://github.com/apache/pulsar/pull/15000#issuecomment-1089584940
@momo-jun a soft reminder: here is a PR targeted at 2.11 w/ doc-required label, could u pls follow up? Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #15000: [improve][broker]Expose Netty threads num config to bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #15000:
URL: https://github.com/apache/pulsar/pull/15000#issuecomment-1150594413
The pr had no activity for 30 days, mark with Stale label.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] hangc0276 commented on pull request #15000: [improve][broker]Expose Netty threads num config to bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
hangc0276 commented on PR #15000:
URL: https://github.com/apache/pulsar/pull/15000#issuecomment-1196177137
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Jason918 commented on pull request #15000: [improve][broker]Expose Netty threads num config to bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
Jason918 commented on PR #15000:
URL: https://github.com/apache/pulsar/pull/15000#issuecomment-1200997302
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codecov-commenter commented on pull request #15000: [improve][broker]Expose Netty threads num config to bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #15000:
URL: https://github.com/apache/pulsar/pull/15000#issuecomment-1342494662
# [Codecov](https://codecov.io/gh/apache/pulsar/pull/15000?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15000](https://codecov.io/gh/apache/pulsar/pull/15000?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (fddc44f) into [master](https://codecov.io/gh/apache/pulsar/commit/68ca60cea7665e6a4bd9f07518b038c17893fe02?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (68ca60c) will **decrease** coverage by `2.65%`.
> The diff coverage is `20.00%`.
[](https://codecov.io/gh/apache/pulsar/pull/15000?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15000 +/- ##
============================================
- Coverage 50.05% 47.40% -2.66%
+ Complexity 11024 9392 -1632
============================================
Files 703 623 -80
Lines 68814 59068 -9746
Branches 7378 6143 -1235
============================================
- Hits 34446 28000 -6446
+ Misses 30621 28036 -2585
+ Partials 3747 3032 -715
```
| Flag | Coverage Δ | |
|---|---|---|
| unittests | `47.40% <20.00%> (-2.66%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pulsar/pull/15000?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...r/stats/prometheus/PrometheusMetricsGenerator.java](https://codecov.io/gh/apache/pulsar/pull/15000/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zdGF0cy9wcm9tZXRoZXVzL1Byb21ldGhldXNNZXRyaWNzR2VuZXJhdG9yLmphdmE=) | `0.00% <0.00%> (-67.40%)` | :arrow_down: |
| [...ersistentStickyKeyDispatcherMultipleConsumers.java](https://codecov.io/gh/apache/pulsar/pull/15000/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL3BlcnNpc3RlbnQvUGVyc2lzdGVudFN0aWNreUtleURpc3BhdGNoZXJNdWx0aXBsZUNvbnN1bWVycy5qYXZh) | `55.39% <25.00%> (-6.99%)` | :arrow_down: |
| [...n/java/org/apache/pulsar/broker/admin/v3/Sink.java](https://codecov.io/gh/apache/pulsar/pull/15000/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9hZG1pbi92My9TaW5rLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...java/org/apache/pulsar/broker/admin/v3/Source.java](https://codecov.io/gh/apache/pulsar/pull/15000/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9hZG1pbi92My9Tb3VyY2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...a/org/apache/pulsar/broker/admin/v3/Functions.java](https://codecov.io/gh/apache/pulsar/pull/15000/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9hZG1pbi92My9GdW5jdGlvbnMuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ar/broker/stats/prometheus/ManagedLedgerStats.java](https://codecov.io/gh/apache/pulsar/pull/15000/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zdGF0cy9wcm9tZXRoZXVzL01hbmFnZWRMZWRnZXJTdGF0cy5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...oker/stats/prometheus/PrometheusMetricStreams.java](https://codecov.io/gh/apache/pulsar/pull/15000/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zdGF0cy9wcm9tZXRoZXVzL1Byb21ldGhldXNNZXRyaWNTdHJlYW1zLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [.../stats/prometheus/AggregatedSubscriptionStats.java](https://codecov.io/gh/apache/pulsar/pull/15000/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zdGF0cy9wcm9tZXRoZXVzL0FnZ3JlZ2F0ZWRTdWJzY3JpcHRpb25TdGF0cy5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...metheus/AggregatedTransactionCoordinatorStats.java](https://codecov.io/gh/apache/pulsar/pull/15000/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zdGF0cy9wcm9tZXRoZXVzL0FnZ3JlZ2F0ZWRUcmFuc2FjdGlvbkNvb3JkaW5hdG9yU3RhdHMuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...broker/stats/prometheus/TransactionAggregator.java](https://codecov.io/gh/apache/pulsar/pull/15000/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zdGF0cy9wcm9tZXRoZXVzL1RyYW5zYWN0aW9uQWdncmVnYXRvci5qYXZh) | `0.00% <0.00%> (-96.06%)` | :arrow_down: |
| ... and [183 more](https://codecov.io/gh/apache/pulsar/pull/15000/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #15000: [improve][broker]Expose Netty threads num config to bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #15000:
URL: https://github.com/apache/pulsar/pull/15000#issuecomment-1120561996
The pr had no activity for 30 days, mark with Stale label.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] tisonkun closed pull request #15000: [improve][broker] Expose Netty threads num config to bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
tisonkun closed pull request #15000: [improve][broker] Expose Netty threads num config to bookkeeper.conf
URL: https://github.com/apache/pulsar/pull/15000
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Jason918 commented on pull request #15000: [improve][broker]Expose Netty threads num config to bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
Jason918 commented on PR #15000:
URL: https://github.com/apache/pulsar/pull/15000#issuecomment-1229148320
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] momo-jun commented on pull request #15000: [improve][broker]Expose Netty threads num config to bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
momo-jun commented on PR #15000:
URL: https://github.com/apache/pulsar/pull/15000#issuecomment-1092548973
Hi @hangc0276 will you commit the doc changes (by updating the `reference-configuration.md` file?) in this PR or in a follow-up doc PR? Just let me know your plan or anything I can help with.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] congbobo184 commented on pull request #15000: [improve][broker]Expose Netty threads num config to bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
congbobo184 commented on PR #15000:
URL: https://github.com/apache/pulsar/pull/15000#issuecomment-1318538089
@hangc0276 hi, I move this PR to release/2.9.5, if you have any questions, please ping me. thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] hangc0276 closed pull request #15000: [improve][broker]Expose Netty threads num config to bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
hangc0276 closed pull request #15000: [improve][broker]Expose Netty threads num config to bookkeeper.conf
URL: https://github.com/apache/pulsar/pull/15000
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] hangc0276 commented on a diff in pull request #15000: [improve][broker]Expose Netty threads num config to bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
hangc0276 commented on code in PR #15000:
URL: https://github.com/apache/pulsar/pull/15000#discussion_r929491467
##########
conf/bookkeeper.conf:
##########
@@ -158,6 +158,10 @@ numReadWorkerThreads=8
# (i.e. recovery reads and adds, and fencing).
numHighPriorityWorkerThreads=8
+# This is the number of threads used by Netty to handle TCP connections.
+# Default is 2 * Runtime.getRuntime().availableProcessors()
+serverNumIOThreads=
Review Comment:
They are different parameters
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Jason918 commented on pull request #15000: [improve][broker]Expose Netty threads num config to bookkeeper.conf
Posted by GitBox <gi...@apache.org>.
Jason918 commented on PR #15000:
URL: https://github.com/apache/pulsar/pull/15000#issuecomment-1196608871
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org