You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by ravipesala <gi...@git.apache.org> on 2017/01/27 07:27:37 UTC
[GitHub] incubator-carbondata pull request #576: [WIP] Improve query performance with...
GitHub user ravipesala opened a pull request:
https://github.com/apache/incubator-carbondata/pull/576
[WIP] Improve query performance with read a head blocklet and changed number compressor.
Need to work out with backward compatability
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ravipesala/incubator-carbondata vector_perform
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-carbondata/pull/576.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #576
----
commit 1c64874f18610be5de1490fe4f450ea6ca2127b0
Author: ravipesala <ra...@gmail.com>
Date: 2017-01-12T06:37:41Z
Added prefetch to scanner
commit d61ef31ff3867bd0a1e032e4753a99a9db54e519
Author: ravipesala <ra...@gmail.com>
Date: 2017-01-26T09:54:24Z
Added read a head blocklet
commit 37d7ca08c7184d01eba2eb717f9c204ed8817142
Author: ravipesala <ra...@gmail.com>
Date: 2017-01-27T07:13:22Z
Added read a head
commit 6b3af88bf8f19c3156d9c3e5c8208bc13cf10018
Author: ravipesala <ra...@gmail.com>
Date: 2017-01-27T07:19:12Z
Fixed style
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #576: [WIP] Improve query performance with read a...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/576
Build Failed with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/757/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #576: [WIP] Improve query performance with...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala closed the pull request at:
https://github.com/apache/incubator-carbondata/pull/576
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #576: [WIP] Improve query performance with read a...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/576
Build Failed with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/759/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #576: [WIP] Improve query performance with read a...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/576
Build Failed with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/801/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #576: [WIP] Improve query performance with read a...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/incubator-carbondata/pull/576
Same has been handled in https://github.com/apache/incubator-carbondata/pull/584, so closing this PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #576: [WIP] Improve query performance with...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/576#discussion_r98163965
--- Diff: core/src/main/java/org/apache/carbondata/core/keygenerator/mdkey/NumberCompressor.java ---
@@ -137,12 +161,31 @@ public NumberCompressor(int cardinality) {
return words;
}
+ // public int[] unCompress(byte[] key, int offset, int length) {
+ // int ls = length;
+ // int arrayLength = (ls * BYTE_LENGTH) / bitsLength;
+ // long[] words = new long[getWordsSizeFromBytesSize(ls)];
+ // unCompressVal(key, ls, words, offset);
+ // return getArray(words, arrayLength);
+ // }
+
public int[] unCompress(byte[] key, int offset, int length) {
- int ls = length;
- int arrayLength = (ls * BYTE_LENGTH) / bitsLength;
- long[] words = new long[getWordsSizeFromBytesSize(ls)];
- unCompressVal(key, ls, words, offset);
- return getArray(words, arrayLength);
+ int len = length / bytesPerKey;
+ if (length % bytesPerKey != 0) {
+ throw new IllegalArgumentException(
+ "Some thing wrong while decompress" + length + " " + bytesPerKey);
--- End diff --
Both length is read from footer, right? If comes here, mean the file is corrupted.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #576: [WIP] Improve query performance with read a...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/576
Build Failed with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/762/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #576: [WIP] Improve query performance with read a...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/576
Build Failed with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/761/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #576: [WIP] Improve query performance with read a...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/576
Build Failed with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/790/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---