You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by kumarvishal09 <gi...@git.apache.org> on 2018/04/23 11:57:02 UTC
[GitHub] carbondata pull request #2213: [CARBONDATA-2386]Fix Query performance issue ...
GitHub user kumarvishal09 opened a pull request:
https://github.com/apache/carbondata/pull/2213
[CARBONDATA-2386]Fix Query performance issue with PreAggregate table
**Problem:** Query on pre aggregate table is consuming too much time.
**Root cause:** Time consumption to calculate size of selecting the smallest Pre-Aggregate table is approximately 76 seconds. This is index file is being read when segment file is present to compute the size of Pre-Aggregate table
**Solution:** Read table status and get the size of data file and index file for valid segments. For older segments were datasize and indexsize is not present calculate the size of store folder
- [ ] Any interfaces changed?
- [ ] Any backward compatibility impacted?
- [ ] Document update required?
- [ ] Testing done
Tested in Cluster with 2 billion rows and 16 aggregate tables
Older Time:76 seconds
New Time: 6 seconds
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kumarvishal09/incubator-carbondata preagg_perfix
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2213.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2213
----
commit b8857c9e4064f30ec69fd55a6dfb47306e95616e
Author: kumarvishal09 <ku...@...>
Date: 2018-04-23T11:33:00Z
Fix performance issue
----
---
[GitHub] carbondata issue #2213: [CARBONDATA-2386]Fix Query performance issue with Pr...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2213
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4196/
---
[GitHub] carbondata issue #2213: [CARBONDATA-2386]Fix Query performance issue with Pr...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2213
LGTM
---
[GitHub] carbondata issue #2213: [CARBONDATA-2386]Fix Query performance issue with Pr...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2213
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4484/
---
[GitHub] carbondata issue #2213: [CARBONDATA-2386]Fix Query performance issue with Pr...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2213
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5367/
---
[GitHub] carbondata issue #2213: [CARBONDATA-2386]Fix Query performance issue with Pr...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2213
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5345/
---
[GitHub] carbondata pull request #2213: [CARBONDATA-2386]Fix Query performance issue ...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/carbondata/pull/2213
---
[GitHub] carbondata issue #2213: [CARBONDATA-2386]Fix Query performance issue with Pr...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on the issue:
https://github.com/apache/carbondata/pull/2213
retest this please
---
[GitHub] carbondata issue #2213: [CARBONDATA-2386]Fix Query performance issue with Pr...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2213
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4161/
---