You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "swaminathanmanish (via GitHub)" <gi...@apache.org> on 2024/05/09 05:53:12 UTC
[PR] Revert "Using local copy of segment instead of downloading from remote" [pinot]
swaminathanmanish opened a new pull request, #13114:
URL: https://github.com/apache/pinot/pull/13114
Reverts apache/pinot#12863 until further verification.
Seeing an error after this change, during metadata push.
2024/05/09 00:43:51.276 WARN [PinotSegmentUploadDownloadRestletResource] [jersey-server-managed-async-executor-0] Could not fetch segment size for metadata push
java.lang.IllegalArgumentException: File system element for parameter 'path' does not exist: '/home/pinot/data/minionData/SegmentRefreshTask/tmp-f19f9d8f-5f15-4165-8f5b-72673845d172/convertedTarredSegmentDir/refreshed_1715215417793_2_taxitest_0.tar.gz'
at org.apache.commons.io.file.PathUtils.requireExists(PathUtils.java:1467) ~[startree-pinot-all-1.2.0-ST.35-jar-with-dependencies.jar:1.2.0-ST.35-50452797ff61e418ecd172cdf7f7090fbf331dec]
at org.apache.commons.io.file.PathUtils.sizeOf(PathUtils.java:1634) ~[startree-pinot-all-1.2.0-ST.35-jar-with-dependencies.jar:1.2.0-ST.35-50452797ff61e418ecd172cdf7f7090fbf331dec]
at org.apache.commons.io.FileUtils.lambda$sizeOf$13(FileUtils.java:2863) ~[startree-pinot-all-1.2.0-ST.35-jar-with-dependencies.jar:1.2.0-ST.35-50452797ff61e418ecd172cdf7f7090fbf331dec]
at org.apache.commons.io.function.Uncheck.get(Uncheck.java:197) ~[startree-pinot-all-1.2.0-ST.35-jar-with-dependencies.jar:1.2.0-ST.35-50452797ff61e418ecd172cdf7f7090fbf331dec]
at org.apache.commons.io.FileUtils.sizeOf(FileUtils.java:2863) ~[startree-pinot-all-1.2.0-ST.35-jar-with-dependencies.jar:1.2.0-ST.35-50452797ff61e418ecd172cdf7f7090fbf331dec]
at org.apache.pinot.spi.filesystem.LocalPinotFS.length(LocalPinotFS.java:106) ~[startree-pinot-all-1.2.0-ST.35-jar-with-dependencies.jar:1.2.0-ST.35-50452797ff61e418ecd172cdf7f7090fbf331dec]
at org.apache.pinot.spi.filesystem.NoClosePinotFS.length(NoClosePinotFS.java:80) ~[startree-pinot-all-1.2.0-ST.35-jar-with-dependencies.jar:1.2.0-ST.35-50452797ff61e418ecd172cdf7f7090fbf331dec]
at org.apache.pinot.controller.api.resources.PinotSegmentUploadDownloadRestletResource.uploadSegment(PinotSegmentUploadDownloadRestletResource.java:308) ~[startree-pinot-all-1.2.0-ST.35-jar-with-dependencies.jar:1.2.0-ST.35-50452797ff61e418ecd172cdf7f7090fbf331dec]
at org.apache.pinot.controller.api.resources.PinotSegmentUploadDownloadRestletResource.uploadSegmentAsMultiPartV2(PinotSegmentUploadDownloadRestletResource.java:639) ~[startree-pinot-all-1.2.0-ST.35-jar-with-dependencies.jar:1.2.0-ST.35-50452797ff61e418ecd172cdf7f7090fbf331dec]
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) ~[?:?]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Revert "Using local copy of segment instead of downloading from remote" [pinot]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #13114:
URL: https://github.com/apache/pinot/pull/13114#issuecomment-2102043625
## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/13114?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
Attention: Patch coverage is `0%` with `10 lines` in your changes are missing coverage. Please review.
> Project coverage is 62.05%. Comparing base [(`59551e4`)](https://app.codecov.io/gh/apache/pinot/commit/59551e45224f1535c4863fd577622b37366ccc97?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`ee89454`)](https://app.codecov.io/gh/apache/pinot/pull/13114?dropdown=coverage&src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
> Report is 420 commits behind head on master.
| [Files](https://app.codecov.io/gh/apache/pinot/pull/13114?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../tasks/BaseMultipleSegmentsConversionExecutor.java](https://app.codecov.io/gh/apache/pinot/pull/13114?src=pr&el=tree&filepath=pinot-plugins%2Fpinot-minion-tasks%2Fpinot-minion-builtin-tasks%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpinot%2Fplugin%2Fminion%2Ftasks%2FBaseMultipleSegmentsConversionExecutor.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvQmFzZU11bHRpcGxlU2VnbWVudHNDb252ZXJzaW9uRXhlY3V0b3IuamF2YQ==) | 0.00% | [10 Missing :warning: ](https://app.codecov.io/gh/apache/pinot/pull/13114?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #13114 +/- ##
============================================
+ Coverage 61.75% 62.05% +0.30%
+ Complexity 207 198 -9
============================================
Files 2436 2514 +78
Lines 133233 137790 +4557
Branches 20636 21320 +684
============================================
+ Hits 82274 85511 +3237
- Misses 44911 45887 +976
- Partials 6048 6392 +344
```
| [Flag](https://app.codecov.io/gh/apache/pinot/pull/13114/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [custom-integration1](https://app.codecov.io/gh/apache/pinot/pull/13114/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `<0.01% <0.00%> (-0.01%)` | :arrow_down: |
| [integration](https://app.codecov.io/gh/apache/pinot/pull/13114/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `<0.01% <0.00%> (-0.01%)` | :arrow_down: |
| [integration1](https://app.codecov.io/gh/apache/pinot/pull/13114/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `<0.01% <0.00%> (-0.01%)` | :arrow_down: |
| [integration2](https://app.codecov.io/gh/apache/pinot/pull/13114/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <0.00%> (ø)` | |
| [java-11](https://app.codecov.io/gh/apache/pinot/pull/13114/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `27.74% <0.00%> (-33.97%)` | :arrow_down: |
| [java-21](https://app.codecov.io/gh/apache/pinot/pull/13114/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `35.24% <0.00%> (-26.38%)` | :arrow_down: |
| [skip-bytebuffers-false](https://app.codecov.io/gh/apache/pinot/pull/13114/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `62.03% <0.00%> (+0.28%)` | :arrow_up: |
| [skip-bytebuffers-true](https://app.codecov.io/gh/apache/pinot/pull/13114/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `35.21% <0.00%> (+7.48%)` | :arrow_up: |
| [temurin](https://app.codecov.io/gh/apache/pinot/pull/13114/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `62.05% <0.00%> (+0.30%)` | :arrow_up: |
| [unittests](https://app.codecov.io/gh/apache/pinot/pull/13114/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `62.05% <0.00%> (+0.30%)` | :arrow_up: |
| [unittests1](https://app.codecov.io/gh/apache/pinot/pull/13114/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `46.70% <ø> (-0.19%)` | :arrow_down: |
| [unittests2](https://app.codecov.io/gh/apache/pinot/pull/13114/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `27.74% <0.00%> (+0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/pinot/pull/13114?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Revert "Using local copy of segment instead of downloading from remote" [pinot]
Posted by "klsince (via GitHub)" <gi...@apache.org>.
klsince merged PR #13114:
URL: https://github.com/apache/pinot/pull/13114
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Revert "Using local copy of segment instead of downloading from remote" [pinot]
Posted by "swaminathanmanish (via GitHub)" <gi...@apache.org>.
swaminathanmanish commented on PR #13114:
URL: https://github.com/apache/pinot/pull/13114#issuecomment-2101988578
@klsince - Could you please review ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org