You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2021/04/08 18:36:46 UTC
[GitHub] [incubator-pinot] fx19880617 opened a new pull request #6759: Fix the issue that PinotFs doesn't init before the first get in SegmentGenerationJobRunner
fx19880617 opened a new pull request #6759:
URL: https://github.com/apache/incubator-pinot/pull/6759
## Description
Fix the issue that PinotFs doesn't init before the first get in SegmentGenerationJobRunner
## Upgrade Notes
Does this PR prevent a zero down-time upgrade? (Assume upgrade order: Controller, Broker, Server, Minion)
* [ ] Yes (Please label as **<code>backward-incompat</code>**, and complete the section below on Release Notes)
Does this PR fix a zero-downtime upgrade introduced earlier?
* [ ] Yes (Please label this as **<code>backward-incompat</code>**, and complete the section below on Release Notes)
Does this PR otherwise need attention when creating release notes? Things to consider:
- New configuration options
- Deprecation of configurations
- Signature changes to public methods/interfaces
- New plugins added or old plugins removed
* [ ] Yes (Please label this PR as **<code>release-notes</code>** and complete the section on Release Notes)
## Release Notes
If you have tagged this as either backward-incompat or release-notes,
you MUST add text here that you would like to see appear in release notes of the
next release.
If you have a series of commits adding or enabling a feature, then
add this section only in final commit that marks the feature completed.
Refer to earlier release notes to see examples of text
## Documentation
If you have introduced a new feature or configuration, please add it to the documentation as well.
See https://docs.pinot.apache.org/developers/developers-and-contributors/update-document
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang commented on pull request #6759: Fix the issue that PinotFs doesn't init before the first get in SegmentGenerationJobRunner
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on pull request #6759:
URL: https://github.com/apache/incubator-pinot/pull/6759#issuecomment-817031887
I think PinotFS should always be initialized first before launching the job. Is there a scenario where we don't get a chance to init the PinotFS?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 merged pull request #6759: Fix the issue that PinotFs doesn't init before the first get in SegmentGenerationJobRunner
Posted by GitBox <gi...@apache.org>.
fx19880617 merged pull request #6759:
URL: https://github.com/apache/incubator-pinot/pull/6759
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] codecov-io commented on pull request #6759: Fix the issue that PinotFs doesn't init before the first get in SegmentGenerationJobRunner
Posted by GitBox <gi...@apache.org>.
codecov-io commented on pull request #6759:
URL: https://github.com/apache/incubator-pinot/pull/6759#issuecomment-817231959
# [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6759?src=pr&el=h1) Report
> Merging [#6759](https://codecov.io/gh/apache/incubator-pinot/pull/6759?src=pr&el=desc) (77ab7aa) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/e9170aa8e61a00032f036d6e56bf21f6bebef1e6?el=desc) (e9170aa) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/incubator-pinot/pull/6759?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #6759 +/- ##
============================================
- Coverage 43.35% 43.34% -0.01%
Complexity 7 7
============================================
Files 1411 1411
Lines 68666 68666
Branches 9918 9918
============================================
- Hits 29768 29765 -3
- Misses 36373 36386 +13
+ Partials 2525 2515 -10
```
| Flag | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| integration | `43.34% <ø> (-0.01%)` | `0.00 <ø> (ø)` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/6759?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...ion/groupby/AggregationGroupByTrimmingService.java](https://codecov.io/gh/apache/incubator-pinot/pull/6759/diff?src=pr&el=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9xdWVyeS9hZ2dyZWdhdGlvbi9ncm91cGJ5L0FnZ3JlZ2F0aW9uR3JvdXBCeVRyaW1taW5nU2VydmljZS5qYXZh) | `66.66% <0.00%> (-27.54%)` | `0.00% <0.00%> (ø%)` | |
| [...e/pinot/core/segment/index/loader/LoaderUtils.java](https://codecov.io/gh/apache/incubator-pinot/pull/6759/diff?src=pr&el=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9zZWdtZW50L2luZGV4L2xvYWRlci9Mb2FkZXJVdGlscy5qYXZh) | `57.77% <0.00%> (-13.34%)` | `0.00% <0.00%> (ø%)` | |
| [.../filter/predicate/InPredicateEvaluatorFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6759/diff?src=pr&el=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvcHJlZGljYXRlL0luUHJlZGljYXRlRXZhbHVhdG9yRmFjdG9yeS5qYXZh) | `55.17% <0.00%> (-11.50%)` | `0.00% <0.00%> (ø%)` | |
| [...t/core/segment/index/readers/DoubleDictionary.java](https://codecov.io/gh/apache/incubator-pinot/pull/6759/diff?src=pr&el=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9zZWdtZW50L2luZGV4L3JlYWRlcnMvRG91YmxlRGljdGlvbmFyeS5qYXZh) | `60.00% <0.00%> (-10.00%)` | `0.00% <0.00%> (ø%)` | |
| [.../controller/helix/core/SegmentDeletionManager.java](https://codecov.io/gh/apache/incubator-pinot/pull/6759/diff?src=pr&el=tree#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9oZWxpeC9jb3JlL1NlZ21lbnREZWxldGlvbk1hbmFnZXIuamF2YQ==) | `53.27% <0.00%> (-9.84%)` | `0.00% <0.00%> (ø%)` | |
| [...che/pinot/core/segment/memory/PinotDataBuffer.java](https://codecov.io/gh/apache/incubator-pinot/pull/6759/diff?src=pr&el=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9zZWdtZW50L21lbW9yeS9QaW5vdERhdGFCdWZmZXIuamF2YQ==) | `54.16% <0.00%> (-7.50%)` | `0.00% <0.00%> (ø%)` | |
| [...er/starter/helix/SegmentMessageHandlerFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6759/diff?src=pr&el=tree#diff-cGlub3Qtc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zZXJ2ZXIvc3RhcnRlci9oZWxpeC9TZWdtZW50TWVzc2FnZUhhbmRsZXJGYWN0b3J5LmphdmE=) | `69.33% <0.00%> (-6.67%)` | `0.00% <0.00%> (ø%)` | |
| [...operator/filter/RangeIndexBasedFilterOperator.java](https://codecov.io/gh/apache/incubator-pinot/pull/6759/diff?src=pr&el=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvUmFuZ2VJbmRleEJhc2VkRmlsdGVyT3BlcmF0b3IuamF2YQ==) | `40.42% <0.00%> (-4.26%)` | `0.00% <0.00%> (ø%)` | |
| [...rg/apache/pinot/core/query/utils/idset/IdSets.java](https://codecov.io/gh/apache/incubator-pinot/pull/6759/diff?src=pr&el=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9xdWVyeS91dGlscy9pZHNldC9JZFNldHMuamF2YQ==) | `25.00% <0.00%> (-2.78%)` | `0.00% <0.00%> (ø%)` | |
| [...server/starter/helix/HelixInstanceDataManager.java](https://codecov.io/gh/apache/incubator-pinot/pull/6759/diff?src=pr&el=tree#diff-cGlub3Qtc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zZXJ2ZXIvc3RhcnRlci9oZWxpeC9IZWxpeEluc3RhbmNlRGF0YU1hbmFnZXIuamF2YQ==) | `78.84% <0.00%> (-2.57%)` | `0.00% <0.00%> (ø%)` | |
| ... and [23 more](https://codecov.io/gh/apache/incubator-pinot/pull/6759/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6759?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6759?src=pr&el=footer). Last update [e9170aa...77ab7aa](https://codecov.io/gh/apache/incubator-pinot/pull/6759?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request #6759: Fix the issue that PinotFs doesn't init before the first get in SegmentGenerationJobRunner
Posted by GitBox <gi...@apache.org>.
fx19880617 commented on a change in pull request #6759:
URL: https://github.com/apache/incubator-pinot/pull/6759#discussion_r611803605
##########
File path: pinot-plugins/pinot-stream-ingestion/pinot-kafka-2.0/src/test/java/org/apache/pinot/plugin/stream/kafka20/KafkaPartitionLevelConsumerTest.java
##########
@@ -54,11 +54,11 @@
private static final String TEST_TOPIC_2 = "bar";
private static final int NUM_MSG_PRODUCED_PER_PARTITION = 1000;
- private static MiniKafkaCluster kafkaCluster;
- private static String brokerAddress;
+ private MiniKafkaCluster kafkaCluster;
Review comment:
I was seeing everytime the tests failed on this file, so just modify it to play around
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] mayankshriv commented on a change in pull request #6759: Fix the issue that PinotFs doesn't init before the first get in SegmentGenerationJobRunner
Posted by GitBox <gi...@apache.org>.
mayankshriv commented on a change in pull request #6759:
URL: https://github.com/apache/incubator-pinot/pull/6759#discussion_r611117561
##########
File path: pinot-plugins/pinot-stream-ingestion/pinot-kafka-2.0/src/test/java/org/apache/pinot/plugin/stream/kafka20/KafkaPartitionLevelConsumerTest.java
##########
@@ -54,11 +54,11 @@
private static final String TEST_TOPIC_2 = "bar";
private static final int NUM_MSG_PRODUCED_PER_PARTITION = 1000;
- private static MiniKafkaCluster kafkaCluster;
- private static String brokerAddress;
+ private MiniKafkaCluster kafkaCluster;
Review comment:
Curious on how the changes in these files are related to the fix?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org