You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by GitBox <gi...@apache.org> on 2020/10/25 06:21:59 UTC
[GitHub] [carbondata] marchpure opened a new pull request #3997: [WIP]Add TPCDS TestCase
marchpure opened a new pull request #3997:
URL: https://github.com/apache/carbondata/pull/3997
### Why is this PR needed?
### What changes were proposed in this PR?
### Does this PR introduce any user interface change?
- No
- Yes. (please explain the change and update document)
### Is any new testcase added?
- No
- Yes
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] marchpure edited a comment on pull request #3997: [CARBONDATA-4045] Add TPCDS TestCase for Spark on CarbonData Integration Test
Posted by GitBox <gi...@apache.org>.
marchpure edited a comment on pull request #3997:
URL: https://github.com/apache/carbondata/pull/3997#issuecomment-717045718
> @marchpure : The reason why we don't have TPCH and TPCDS in UT is we need a huge data set, loading huge data takes time.
>
> What is the reason behind adding this? we can anyways have separate TPCH or TPCDS machines that can have automation script to give performance benchmark on every release
>
> Also no need to run TPCH and TPCDS on every PR builder. Running once per release is enough.
>
> @QiangCai , @kunal642 : What's your opinion on this?
1. The TPCDS dataset in this PR is really small(totally 33KB). it won"t took so much time to load and query. It may help to avoid possible issues with accepted overhead.
2. it also help us to debug tpcds. explain plan ～analyse in local environment.
the inspiration to add tpcds test case is CARBONDATA 4008. Whose issue is Spark on CarbonData will fail in TPCDS Query 83. This issue seems has been there for a log time, which implies that our UT is not enough.
I believe that we can add a profile to turn on/off of TPCDS test in the future if the automatic TPCDS machine is ready.
**Maybe we can have a module name 'carbondata-integretion-test'?**
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3997: [WIP] Add TPCDS TestCase
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on pull request #3997:
URL: https://github.com/apache/carbondata/pull/3997#issuecomment-716562736
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4693/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3997: [WIP] Add TPCDS TestCase
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on pull request #3997:
URL: https://github.com/apache/carbondata/pull/3997#issuecomment-716276174
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4684/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3997: [WIP] Add TPCDS TestCase
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on pull request #3997:
URL: https://github.com/apache/carbondata/pull/3997#issuecomment-716230854
Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2924/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3997: [CARBONDATA-4045] Add TPCDS TestCase
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on pull request #3997:
URL: https://github.com/apache/carbondata/pull/3997#issuecomment-717016610
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4697/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] ajantha-bhat commented on pull request #3997: [CARBONDATA-4045] Add TPCDS TestCase for Spark on CarbonData Integration Test
Posted by GitBox <gi...@apache.org>.
ajantha-bhat commented on pull request #3997:
URL: https://github.com/apache/carbondata/pull/3997#issuecomment-717031195
@marchpure : The reason why we don't have TPCH and TPCDS in UT is we need a huge data set, loading huge data takes time.
What is the reason behind adding this? we can anyways have separate TPCH or TPCDS machines that can have automation script to give performance benchmark on every release
Also no need to run TPCH and TPCDS on every PR builder. Running once per release is enough.
@QiangCai , @kunal642 : What's your opinion on this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3997: [WIP] Add TPCDS TestCase
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on pull request #3997:
URL: https://github.com/apache/carbondata/pull/3997#issuecomment-716175123
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4680/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3997: [WIP] Add TPCDS TestCase
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on pull request #3997:
URL: https://github.com/apache/carbondata/pull/3997#issuecomment-716563917
Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2936/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3997: [WIP] Add TPCDS TestCase
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on pull request #3997:
URL: https://github.com/apache/carbondata/pull/3997#issuecomment-716455767
Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2933/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3997: [WIP] Add TPCDS TestCase
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on pull request #3997:
URL: https://github.com/apache/carbondata/pull/3997#issuecomment-716175045
Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2923/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3997: [WIP]Add TPCDS TestCase
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on pull request #3997:
URL: https://github.com/apache/carbondata/pull/3997#issuecomment-716103357
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] marchpure commented on pull request #3997: [WIP]Add TPCDS TestCase
Posted by GitBox <gi...@apache.org>.
marchpure commented on pull request #3997:
URL: https://github.com/apache/carbondata/pull/3997#issuecomment-716167647
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3997: [WIP]Add TPCDS TestCase
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on pull request #3997:
URL: https://github.com/apache/carbondata/pull/3997#issuecomment-716166118
Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2921/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3997: [WIP] Add TPCDS TestCase
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on pull request #3997:
URL: https://github.com/apache/carbondata/pull/3997#issuecomment-716275742
Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2927/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3997: [WIP] Add TPCDS TestCase
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on pull request #3997:
URL: https://github.com/apache/carbondata/pull/3997#issuecomment-716230413
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4681/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3997: [CARBONDATA-4045] Add TPCDS TestCase
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on pull request #3997:
URL: https://github.com/apache/carbondata/pull/3997#issuecomment-717017353
Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2940/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] marchpure commented on pull request #3997: [CARBONDATA-4045] Add TPCDS TestCase for Spark on CarbonData Integration Test
Posted by GitBox <gi...@apache.org>.
marchpure commented on pull request #3997:
URL: https://github.com/apache/carbondata/pull/3997#issuecomment-717045718
> @marchpure : The reason why we don't have TPCH and TPCDS in UT is we need a huge data set, loading huge data takes time.
>
> What is the reason behind adding this? we can anyways have separate TPCH or TPCDS machines that can have automation script to give performance benchmark on every release
>
> Also no need to run TPCH and TPCDS on every PR builder. Running once per release is enough.
>
> @QiangCai , @kunal642 : What's your opinion on this?
1. The TPCDS dataset in this PR is really small(totally 33KB). it won"t took so much time to load and query. It may help to avoid possible issues with accepted overhead.
2. it also help us to debug tpcds. explain plan ～analyse in local environment.
the inspiration to add tpcds test case is CARBONDATA 4008. Whose issue is Spark on CarbonData will fail in TPCDS Query 83. This issue seems has been there for a log time, which implies that our UT is not enough.
I believe that we can add a profile to turn on/off of TPCDS test in the future if the automatic TPCDS machine is ready.
Maybe we can have a module name 'carbondata-integretion-test'?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3997: [WIP]Add TPCDS TestCase
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on pull request #3997:
URL: https://github.com/apache/carbondata/pull/3997#issuecomment-716166185
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4678/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3997: [WIP] Add TPCDS TestCase
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on pull request #3997:
URL: https://github.com/apache/carbondata/pull/3997#issuecomment-716454832
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4690/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org