You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2022/04/28 23:22:47 UTC
[GitHub] [pinot] KKcorps opened a new pull request, #8612: Store batch message ids vs calculating them
KKcorps opened a new pull request, #8612:
URL: https://github.com/apache/pinot/pull/8612
Currently, Pulsar batch message ids to start the consumer are calculated by assuming each batch is of exact BATCH_SIZE.
However, in pulsar producer configuration, that is the max size and sometimes the actual batches can be smaller than the particular size.
The only way is to keep track of all batch ids in a map. This will help reduce the instances of flakiness in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] KKcorps commented on pull request #8612: Store batch message ids vs calculating them
Posted by GitBox <gi...@apache.org>.
KKcorps commented on PR #8612:
URL: https://github.com/apache/pinot/pull/8612#issuecomment-1112742234
@walterddr @Jackie-Jiang can you please review this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter commented on pull request #8612: Store batch message ids vs calculating them
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #8612:
URL: https://github.com/apache/pinot/pull/8612#issuecomment-1112764107
# [Codecov](https://codecov.io/gh/apache/pinot/pull/8612?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#8612](https://codecov.io/gh/apache/pinot/pull/8612?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e10f403) into [master](https://codecov.io/gh/apache/pinot/commit/71a6e6379b9cfaef02b756c9442763e15c334220?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (71a6e63) will **increase** coverage by `5.60%`.
> The diff coverage is `n/a`.
> :exclamation: Current head e10f403 differs from pull request most recent head 15ecd70. Consider uploading reports for the commit 15ecd70 to get more accurate results
```diff
@@ Coverage Diff @@
## master #8612 +/- ##
============================================
+ Coverage 63.96% 69.57% +5.60%
- Complexity 4321 4325 +4
============================================
Files 1648 1693 +45
Lines 86880 88799 +1919
Branches 13263 13472 +209
============================================
+ Hits 55576 61778 +6202
+ Misses 27269 22664 -4605
- Partials 4035 4357 +322
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `27.24% <ø> (?)` | |
| unittests1 | `66.99% <ø> (+0.04%)` | :arrow_up: |
| unittests2 | `14.23% <ø> (+0.03%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8612?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...l/segment/index/readers/OnHeapFloatDictionary.java](https://codecov.io/gh/apache/pinot/pull/8612/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC9zZWdtZW50L2luZGV4L3JlYWRlcnMvT25IZWFwRmxvYXREaWN0aW9uYXJ5LmphdmE=) | `90.00% <0.00%> (-5.00%)` | :arrow_down: |
| [.../aggregation/function/ModeAggregationFunction.java](https://codecov.io/gh/apache/pinot/pull/8612/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9xdWVyeS9hZ2dyZWdhdGlvbi9mdW5jdGlvbi9Nb2RlQWdncmVnYXRpb25GdW5jdGlvbi5qYXZh) | `88.10% <0.00%> (-0.55%)` | :arrow_down: |
| [...t/plugin/metrics/yammer/YammerMetricsRegistry.java](https://codecov.io/gh/apache/pinot/pull/8612/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1tZXRyaWNzL3Bpbm90LXlhbW1lci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcGx1Z2luL21ldHJpY3MveWFtbWVyL1lhbW1lck1ldHJpY3NSZWdpc3RyeS5qYXZh) | `60.00% <0.00%> (ø)` | |
| [...starter/helix/DefaultHelixStarterServerConfig.java](https://codecov.io/gh/apache/pinot/pull/8612/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zZXJ2ZXIvc3RhcnRlci9oZWxpeC9EZWZhdWx0SGVsaXhTdGFydGVyU2VydmVyQ29uZmlnLmphdmE=) | `100.00% <0.00%> (ø)` | |
| [.../pinot/plugin/metrics/yammer/YammerMetricName.java](https://codecov.io/gh/apache/pinot/pull/8612/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1tZXRyaWNzL3Bpbm90LXlhbW1lci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcGx1Z2luL21ldHJpY3MveWFtbWVyL1lhbW1lck1ldHJpY05hbWUuamF2YQ==) | `33.33% <0.00%> (ø)` | |
| [...ot/server/api/resources/PinotServerAppConfigs.java](https://codecov.io/gh/apache/pinot/pull/8612/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zZXJ2ZXIvYXBpL3Jlc291cmNlcy9QaW5vdFNlcnZlckFwcENvbmZpZ3MuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...inot/server/api/resources/ServerResourceUtils.java](https://codecov.io/gh/apache/pinot/pull/8612/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zZXJ2ZXIvYXBpL3Jlc291cmNlcy9TZXJ2ZXJSZXNvdXJjZVV0aWxzLmphdmE=) | `45.45% <0.00%> (ø)` | |
| [...pache/pinot/plugin/metrics/yammer/YammerMeter.java](https://codecov.io/gh/apache/pinot/pull/8612/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1tZXRyaWNzL3Bpbm90LXlhbW1lci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcGx1Z2luL21ldHJpY3MveWFtbWVyL1lhbW1lck1ldGVyLmphdmE=) | `55.55% <0.00%> (ø)` | |
| [.../pinot/plugin/inputformat/csv/CSVRecordReader.java](https://codecov.io/gh/apache/pinot/pull/8612/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1pbnB1dC1mb3JtYXQvcGlub3QtY3N2L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9wbHVnaW4vaW5wdXRmb3JtYXQvY3N2L0NTVlJlY29yZFJlYWRlci5qYXZh) | `0.00% <0.00%> (ø)` | |
| [.../java/org/apache/pinot/server/conf/ServerConf.java](https://codecov.io/gh/apache/pinot/pull/8612/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zZXJ2ZXIvY29uZi9TZXJ2ZXJDb25mLmphdmE=) | `69.23% <0.00%> (ø)` | |
| ... and [338 more](https://codecov.io/gh/apache/pinot/pull/8612/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8612?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8612?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [71a6e63...15ecd70](https://codecov.io/gh/apache/pinot/pull/8612?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang merged pull request #8612: Store batch message ids vs calculating them
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang merged PR #8612:
URL: https://github.com/apache/pinot/pull/8612
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on pull request #8612: Store batch message ids vs calculating them
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on PR #8612:
URL: https://github.com/apache/pinot/pull/8612#issuecomment-1116756998
Related to #8537
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org