You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by GitBox <gi...@apache.org> on 2021/01/26 04:45:52 UTC
[GitHub] [carbondata] VenuReddy2103 opened a new pull request #4081: [WIP]Scan best Fix Secondary Index table on Index Server and prune main table
VenuReddy2103 opened a new pull request #4081:
URL: https://github.com/apache/carbondata/pull/4081
### Why is this PR needed?
This PR is to enable Carbon to make use of secndary indexes without plan rewrite.
### What changes were proposed in this PR?
This approach works only with Index Server. Upon get splits for main table, Index Server driver scans the best suitable Secondary Index table based on filter expression, append existing filter expression for main table to have implicit expression for positionId as right child. Then does distributed prune RDD like we do as before this PR.
In fact, we can avoid distributed prune RDD itself after scannig best feasible Secondary Index table based on filter. Because Secondary Index table gives the exact available blocklets for the given filter. So we can directly form the extended blocklets and return. Number of Splits cannot be reduced further, even if we call distributed prune RDD. Still working on it.
### Does this PR introduce any user interface change?
- No
- Yes. (please explain the change and update document)
### Is any new testcase added?
- No
- Yes
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4081: [WIP]Secondary Index based pruning without spark query plan modification
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4081:
URL: https://github.com/apache/carbondata/pull/4081#issuecomment-776634120
Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3690/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] VenuReddy2103 closed pull request #4081: [WIP]Secondary Index based pruning without spark query plan modification
Posted by GitBox <gi...@apache.org>.
VenuReddy2103 closed pull request #4081:
URL: https://github.com/apache/carbondata/pull/4081
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4081: [WIP]Secondary Index based pruning without spark query plan modification
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4081:
URL: https://github.com/apache/carbondata/pull/4081#issuecomment-776134616
Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3688/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4081: [WIP]Secondary Index based pruning without spark query plan modification
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4081:
URL: https://github.com/apache/carbondata/pull/4081#issuecomment-781565780
Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/3326/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4081: [WIP]Secondary Index based pruning without spark query plan modification
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4081:
URL: https://github.com/apache/carbondata/pull/4081#issuecomment-779082825
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5467/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4081: [WIP]Scan best Fix Secondary Index table on Index Server and prune main table
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4081:
URL: https://github.com/apache/carbondata/pull/4081#issuecomment-776017835
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5448/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4081: [WIP]Scan best Fix Secondary Index table on Index Server and prune main table
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4081:
URL: https://github.com/apache/carbondata/pull/4081#issuecomment-767352402
Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3594/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4081: [WIP]Secondary Index based pruning without spark query plan modification
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4081:
URL: https://github.com/apache/carbondata/pull/4081#issuecomment-778984796
Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3702/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4081: [WIP]Secondary Index based pruning without spark query plan modification
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4081:
URL: https://github.com/apache/carbondata/pull/4081#issuecomment-781563045
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/5083/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4081: [WIP]Secondary Index based pruning without spark query plan modification
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4081:
URL: https://github.com/apache/carbondata/pull/4081#issuecomment-779086163
Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3703/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4081: [WIP]Scan best Fix Secondary Index table on Index Server and prune main table
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4081:
URL: https://github.com/apache/carbondata/pull/4081#issuecomment-767352151
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5354/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4081: [WIP]Secondary Index based pruning without spark query plan modification
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4081:
URL: https://github.com/apache/carbondata/pull/4081#issuecomment-776642321
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5451/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4081: [WIP]Secondary Index based pruning without spark query plan modification
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4081:
URL: https://github.com/apache/carbondata/pull/4081#issuecomment-776133890
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5449/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4081: [WIP]Secondary Index based pruning without spark query plan modification
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4081:
URL: https://github.com/apache/carbondata/pull/4081#issuecomment-777598587
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/5074/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4081: [WIP]Secondary Index based pruning without spark query plan modification
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4081:
URL: https://github.com/apache/carbondata/pull/4081#issuecomment-776020699
Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3687/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4081: [WIP]Secondary Index based pruning without spark query plan modification
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4081:
URL: https://github.com/apache/carbondata/pull/4081#issuecomment-795239061
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/5052/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4081: [WIP]Secondary Index based pruning without spark query plan modification
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4081:
URL: https://github.com/apache/carbondata/pull/4081#issuecomment-777596931
Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/3316/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4081: [WIP]Secondary Index based pruning without spark query plan modification
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4081:
URL: https://github.com/apache/carbondata/pull/4081#issuecomment-795230826
Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/3294/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4081: [WIP]Secondary Index based pruning without spark query plan modification
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4081:
URL: https://github.com/apache/carbondata/pull/4081#issuecomment-778984431
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5466/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org