You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by mayunSaicmotor <gi...@git.apache.org> on 2017/04/05 10:03:29 UTC
[GitHub] incubator-carbondata pull request #732: [CARBONDATA-754] improve performance...
GitHub user mayunSaicmotor opened a pull request:
https://github.com/apache/incubator-carbondata/pull/732
[CARBONDATA-754] improve performance when order by prefix columns of mdk + limit
the improvement scenario is for order by prefix columns of mdk + limit
the logical is to leverage the mdk sort feature to get the sorted data. The performance is much better.
for example, order by prefix columns of mdk + limit on 20,000,000 data, the performance can be from 10s to about 1s.
if do not want to use this feature, can also set CarbonCommonConstants.ORDER_BY_MDK_OPTIMIZATION_FLG = false.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mayunSaicmotor/incubator-carbondata orderby-mdk
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-carbondata/pull/732.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #732
----
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #732: [CARBONDATA-754] improve performance when o...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/732
Build Success with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1457/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #732: [CARBONDATA-754] improve performance when o...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/732
Build Success with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1455/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #732: [CARBONDATA-754] improve performance when o...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/732
Build Success with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1494/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #732: [CARBONDATA-754] improve performance...
Posted by mayunSaicmotor <gi...@git.apache.org>.
Github user mayunSaicmotor closed the pull request at:
https://github.com/apache/incubator-carbondata/pull/732
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #732: [CARBONDATA-754] improve performance when o...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/732
Build Success with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1456/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #732: [CARBONDATA-754] improve performance...
Posted by mayunSaicmotor <gi...@git.apache.org>.
Github user mayunSaicmotor closed the pull request at:
https://github.com/apache/incubator-carbondata/pull/732
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #732: [CARBONDATA-754] improve performance when o...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/732
Build Success with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1452/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #732: [CARBONDATA-754] improve performance...
Posted by mayunSaicmotor <gi...@git.apache.org>.
GitHub user mayunSaicmotor reopened a pull request:
https://github.com/apache/incubator-carbondata/pull/732
[CARBONDATA-754] improve performance when order by prefix columns of mdk + limit
the improvement scenario is for order by prefix columns of mdk + limit
1. order by prefix columns of mdk asc + limit
2. order by prefix columns of mdk desc + limit
3. order by prefix columns of mdk asc + limit + filter
4. order by prefix columns of mdk desc + limit + filter
the logical is to leverage the mdk sort feature to get the sorted data. The performance is much better.
for example, order by prefix columns of mdk + limit on 20,000,000 data, the performance can be from 10s to about 1s.
if do not want to use this feature, can also set CarbonCommonConstants.ORDER_BY_MDK_OPTIMIZATION_FLG = false.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mayunSaicmotor/incubator-carbondata orderby-mdk
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-carbondata/pull/732.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #732
----
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #732: [CARBONDATA-754] improve performance when o...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/732
Build Failed with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1439/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #732: [CARBONDATA-754] improve performance when o...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/732
Build Failed with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1435/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #732: [CARBONDATA-754] improve performance...
Posted by mayunSaicmotor <gi...@git.apache.org>.
GitHub user mayunSaicmotor reopened a pull request:
https://github.com/apache/incubator-carbondata/pull/732
[CARBONDATA-754] improve performance when order by prefix columns of mdk + limit
the improvement scenario is for order by prefix columns of mdk + limit
the logical is to leverage the mdk sort feature to get the sorted data. The performance is much better.
for example, order by prefix columns of mdk + limit on 20,000,000 data, the performance can be from 10s to about 1s.
if do not want to use this feature, can also set CarbonCommonConstants.ORDER_BY_MDK_OPTIMIZATION_FLG = false.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mayunSaicmotor/incubator-carbondata orderby-mdk
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-carbondata/pull/732.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #732
----
commit 8a623995a9c9385223be740c56ecb6c9b1dfb38d
Author: \u9a6c\u4e91 <si...@163.com>
Date: 2017-03-31T14:07:35Z
Merge pull request #5 from apache/master
merge from master
commit 9294f5c2b77052f44272e43a432182b9c0b84643
Author: mayun <si...@163.com>
Date: 2017-04-02T03:28:16Z
order by mdk optimization
commit 33ed5d7e20bdefc085411137a2263fd6bbf87a1d
Author: mayun <si...@163.com>
Date: 2017-04-05T05:51:53Z
order by mdk optimization
commit 1af457b517c0ed26f3ee80e8235f16625f03134f
Author: \u9a6c\u4e91 <si...@163.com>
Date: 2017-04-05T08:45:56Z
Merge pull request #6 from apache/master
merge from apache master
commit f73495fab356e52676200e69fe299fd61806777d
Author: mayun <si...@163.com>
Date: 2017-04-05T09:35:51Z
add test data
commit 8c7a190be441937739bc8ea331954c1ffb4e8160
Author: mayun <si...@163.com>
Date: 2017-04-05T09:37:41Z
add load test data class
commit fcbc4b6233bd0d55eaa9d81bf14fa9f96f6c1653
Author: mayun <si...@163.com>
Date: 2017-04-05T11:58:08Z
change limit default value from 0 to -1
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---