You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@carbondata.apache.org by GitBox <gi...@apache.org> on 2022/03/31 13:06:15 UTC
[GitHub] [carbondata] ShreelekhyaG opened a new pull request #4257: [WIP] Incremental Dataload of Average aggregate in MV
ShreelekhyaG opened a new pull request #4257:
URL: https://github.com/apache/carbondata/pull/4257
### Why is this PR needed?
Currently, whenever MV is created with average aggregate, a full refresh is done meaning it reloads the whole MV for any newly added segments. This will slow down the loading. With incremental data load, only the segments that are newly added can be loaded to the MV.
### What changes were proposed in this PR?
If avg is present, rewrite the query with the sum and count of the columns to create MV and use them to derive avg.
Refer: https://docs.google.com/document/d/1kPEMCX50FLZcmyzm6kcIQtUH9KXWDIqh-Hco7NkTp80/edit
### Does this PR introduce any user interface change?
- No
### Is any new testcase added?
- Yes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4257: [WIP] Incremental Dataload of Average aggregate in MV
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4257:
URL: https://github.com/apache/carbondata/pull/4257#issuecomment-1084695408
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6260/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4257: [WIP] Incremental Dataload of Average aggregate in MV
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4257:
URL: https://github.com/apache/carbondata/pull/4257#issuecomment-1084810428
Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4515/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4257: [WIP] Incremental Dataload of Average aggregate in MV
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4257:
URL: https://github.com/apache/carbondata/pull/4257#issuecomment-1084822246
Build Failed with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/650/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org