You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "codelipenghui (via GitHub)" <gi...@apache.org> on 2023/06/19 14:12:50 UTC
[GitHub] [pulsar] codelipenghui opened a new pull request, #20607: [fix][broker]fix the publish latency spike issue with large number of producers
codelipenghui opened a new pull request, #20607:
URL: https://github.com/apache/pulsar/pull/20607
### Motivation
The issue should only happen on the topic with many producers, and the max producer limitation is enabled. When a new producer is added, the broker will check all the existing producers and filter out the remote producer (from the replicator), which will run out of the CPU resource for a short period of time.
<img width="1907" alt="image" src="https://github.com/apache/pulsar/assets/12592133/14e74ed2-7ace-429b-9352-c643995e0e27">
[profiler.html.txt](https://github.com/apache/pulsar/files/11788992/profiler.html.txt)
### Modifications
A counter will be much more efficient instead of going through all the producers. So a new userCreatedProducerCount is added to the topic.
### Verifying this change
The existing tests already covered the max producer limitation. The change will not affect the correctness.
### Does this pull request potentially affect one of the following parts:
<!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
*If the box was checked, please highlight the changes*
- [ ] Dependencies (add or upgrade a dependency)
- [ ] The public API
- [ ] The schema
- [ ] The default values of configurations
- [ ] The threading model
- [ ] The binary protocol
- [ ] The REST endpoints
- [ ] The admin CLI options
- [ ] The metrics
- [ ] Anything that affects deployment
### Documentation
<!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
- [ ] `doc` <!-- Your PR contains doc changes. -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [x] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Technoboy- merged pull request #20607: [fix][broker]fix the publish latency spike issue with large number of producers
Posted by "Technoboy- (via GitHub)" <gi...@apache.org>.
Technoboy- merged PR #20607:
URL: https://github.com/apache/pulsar/pull/20607
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #20607: [fix][broker]fix the publish latency spike issue with large number of producers
Posted by "codelipenghui (via GitHub)" <gi...@apache.org>.
codelipenghui commented on PR #20607:
URL: https://github.com/apache/pulsar/pull/20607#issuecomment-1597271021
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #20607: [fix][broker]fix the publish latency spike issue with large number of producers
Posted by "codelipenghui (via GitHub)" <gi...@apache.org>.
codelipenghui commented on PR #20607:
URL: https://github.com/apache/pulsar/pull/20607#issuecomment-1597922536
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codecov-commenter commented on pull request #20607: [fix][broker]fix the publish latency spike issue with large number of producers
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #20607:
URL: https://github.com/apache/pulsar/pull/20607#issuecomment-1597992914
## [Codecov](https://app.codecov.io/gh/apache/pulsar/pull/20607?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
> Merging [#20607](https://app.codecov.io/gh/apache/pulsar/pull/20607?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (f601751) into [master](https://app.codecov.io/gh/apache/pulsar/commit/8125d1f6f63014a2aba1b53d48e98b9546afd001?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (8125d1f) will **increase** coverage by `0.76%`.
> The diff coverage is `90.00%`.
[](https://app.codecov.io/gh/apache/pulsar/pull/20607?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
```diff
@@ Coverage Diff @@
## master #20607 +/- ##
============================================
+ Coverage 72.29% 73.05% +0.76%
+ Complexity 31951 31889 -62
============================================
Files 1854 1867 +13
Lines 138462 138663 +201
Branches 15221 15237 +16
============================================
+ Hits 100095 101302 +1207
+ Misses 30384 29330 -1054
- Partials 7983 8031 +48
```
| Flag | Coverage Δ | |
|---|---|---|
| inttests | `24.19% <80.00%> (?)` | |
| systests | `25.00% <60.00%> (?)` | |
| unittests | `72.34% <90.00%> (+0.05%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://app.codecov.io/gh/apache/pulsar/pull/20607?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...rg/apache/pulsar/broker/service/AbstractTopic.java](https://app.codecov.io/gh/apache/pulsar/pull/20607?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL0Fic3RyYWN0VG9waWMuamF2YQ==) | `88.72% <90.00%> (+0.66%)` | :arrow_up: |
... and [186 files with indirect coverage changes](https://app.codecov.io/gh/apache/pulsar/pull/20607/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org