You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2022/10/26 23:46:10 UTC
[GitHub] [pinot] amrishlal opened a new pull request, #9669: Avoid setting result column types to STRING when resultset is empty.
amrishlal opened a new pull request, #9669:
URL: https://github.com/apache/pinot/pull/9669
The current codebase sets datatype of columns to STRING while generating an empty resultset after all segments have been pruned out (see Issue #9660 for more details). In this PR, empty result set column datatypes are not set when all segments are pruned out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on pull request #9669: [Work In Progress] Avoid setting result column types to STRING when resultset is empty.
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on PR #9669:
URL: https://github.com/apache/pinot/pull/9669#issuecomment-1302922378
There are 2 types of pruning, one is to prune the invalid segments (empty or column does not exist), the other is value based. We cannot keep the invalid one because query engine can throw exception against it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on pull request #9669: Avoid setting result column types to STRING when resultset is empty.
Posted by GitBox <gi...@apache.org>.
amrishlal commented on PR #9669:
URL: https://github.com/apache/pinot/pull/9669#issuecomment-1294074963
> This is backward-incompatible change, and user might want to get the column names back. Currently it happens when all the segments are pruned. One work-around would be to always keep at least one segment, so that we can get the actual data types
We can set column names in `buildEmptySelectionQueryResults` functions also, but if we are ok with keeping at least one segment, then it might be possible to set proper column types as well. I can try it out...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter commented on pull request #9669: Avoid setting result column types to STRING when resultset is empty.
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #9669:
URL: https://github.com/apache/pinot/pull/9669#issuecomment-1293986226
# [Codecov](https://codecov.io/gh/apache/pinot/pull/9669?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#9669](https://codecov.io/gh/apache/pinot/pull/9669?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7e7b04d) into [master](https://codecov.io/gh/apache/pinot/commit/9f5aa89809bdad5c4a7a84f3611963778921314f?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (9f5aa89) will **increase** coverage by `1.21%`.
> The diff coverage is `66.66%`.
```diff
@@ Coverage Diff @@
## master #9669 +/- ##
============================================
+ Coverage 24.62% 25.83% +1.21%
+ Complexity 53 44 -9
============================================
Files 1935 1934 -1
Lines 103815 103930 +115
Branches 15758 15767 +9
============================================
+ Hits 25560 26851 +1291
+ Misses 75635 74354 -1281
- Partials 2620 2725 +105
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `25.83% <66.66%> (?)` | |
| integration2 | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/9669?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...t/core/query/selection/SelectionOperatorUtils.java](https://codecov.io/gh/apache/pinot/pull/9669/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9xdWVyeS9zZWxlY3Rpb24vU2VsZWN0aW9uT3BlcmF0b3JVdGlscy5qYXZh) | `61.07% <50.00%> (-5.93%)` | :arrow_down: |
| [...ore/operator/blocks/results/ResultsBlockUtils.java](https://codecov.io/gh/apache/pinot/pull/9669/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9ibG9ja3MvcmVzdWx0cy9SZXN1bHRzQmxvY2tVdGlscy5qYXZh) | `63.04% <100.00%> (-5.48%)` | :arrow_down: |
| [.../apache/pinot/server/access/RequesterIdentity.java](https://codecov.io/gh/apache/pinot/pull/9669/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VydmVyL2FjY2Vzcy9SZXF1ZXN0ZXJJZGVudGl0eS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...t/core/plan/StreamingInstanceResponsePlanNode.java](https://codecov.io/gh/apache/pinot/pull/9669/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9wbGFuL1N0cmVhbWluZ0luc3RhbmNlUmVzcG9uc2VQbGFuTm9kZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ore/operator/streaming/StreamingResponseUtils.java](https://codecov.io/gh/apache/pinot/pull/9669/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9zdHJlYW1pbmcvU3RyZWFtaW5nUmVzcG9uc2VVdGlscy5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...server/starter/helix/SegmentReloadStatusValue.java](https://codecov.io/gh/apache/pinot/pull/9669/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zZXJ2ZXIvc3RhcnRlci9oZWxpeC9TZWdtZW50UmVsb2FkU3RhdHVzVmFsdWUuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ager/realtime/PeerSchemeSplitSegmentCommitter.java](https://codecov.io/gh/apache/pinot/pull/9669/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9kYXRhL21hbmFnZXIvcmVhbHRpbWUvUGVlclNjaGVtZVNwbGl0U2VnbWVudENvbW1pdHRlci5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...form/function/IsDistinctFromTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/9669/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vSXNEaXN0aW5jdEZyb21UcmFuc2Zvcm1GdW5jdGlvbi5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...m/function/IsNotDistinctFromTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/9669/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vSXNOb3REaXN0aW5jdEZyb21UcmFuc2Zvcm1GdW5jdGlvbi5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...urces/ServerReloadControllerJobStatusResponse.java](https://codecov.io/gh/apache/pinot/pull/9669/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9hcGkvcmVzb3VyY2VzL1NlcnZlclJlbG9hZENvbnRyb2xsZXJKb2JTdGF0dXNSZXNwb25zZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| ... and [374 more](https://codecov.io/gh/apache/pinot/pull/9669/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal closed pull request #9669: [Work In Progress] Avoid setting result column types to STRING when resultset is empty.
Posted by "amrishlal (via GitHub)" <gi...@apache.org>.
amrishlal closed pull request #9669: [Work In Progress] Avoid setting result column types to STRING when resultset is empty.
URL: https://github.com/apache/pinot/pull/9669
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on pull request #9669: [Work In Progress] Avoid setting result column types to STRING when resultset is empty.
Posted by GitBox <gi...@apache.org>.
amrishlal commented on PR #9669:
URL: https://github.com/apache/pinot/pull/9669#issuecomment-1298876212
@Jackie-Jiang @siddharthteotia Things are now setup so that a query will always run against at least one segment and this will allow for properly setting result metadata column names and types. I am not seeing a case where `ResultsBlockUtils.buildEmptyQueryResults` will be called anymore, so this function could potentially be removed if we are ok with query running against at least one segment on the server side.
The change is not backward compatible only if we assume that a client is relying on the earlier incorrect behavior of returning all column types as STRING when resultset is empty otherwise for a large majority of clients we should be good. Let me know what you think.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on pull request #9669: Avoid setting result column types to STRING when resultset is empty.
Posted by GitBox <gi...@apache.org>.
amrishlal commented on PR #9669:
URL: https://github.com/apache/pinot/pull/9669#issuecomment-1294129180
> One work-around would be to always keep at least one segment, so that we can get the actual data types
@Jackie-Jiang This works very well :-) Great Idea. Allows for properly setting types (even when expressions are used in the select list) when the rest is empty. This is same as what Postgres and Mysql do. The only caveat is that the query will get executed against at least one segment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org