You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by xuchuanyin <gi...@git.apache.org> on 2017/12/22 04:05:40 UTC
[GitHub] carbondata pull request #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and opt...
GitHub user xuchuanyin opened a pull request:
https://github.com/apache/carbondata/pull/1707
[CARBONDATA-1839] [DataLoad] Fix bugs and optimize in compressing sort temp files
1. fix bugs in compressing sort temp file, use file-level compression
instead of batch-record-level compression
2. reduce duplicate code in reading & writing sort temp file
and make it more readable
3. optimize sort procedure:
Before:
raw row that has been converted(call it 'RawRow' for short) ->
sort on RawRow ->
write RawRow to temp sort file ->
read RawRow from temp sort file ->
sort on RawRow -> ... ->
at the final sort, sort on RawRow and convert the RawRow to 3 'PartedRow' ->
write 'PartedRow' to DataFile in write procedure.
After:
raw row that has been converted(call it 'RawRow' for short) ->
convert RawRow to 3 'PartedRow' ->
sort on PartedRow ->
write PartedRow to temp sort file ->
read PartedRow from temp sort file ->
sort on PartedRow -> ... ->
at the final sort, sort on PartedRow ->
write 'PartedRow' to DataFile in write procedure.
4. add tests
5. remove unused code
6. update docs, add property to configure the compressor
Please refer to [maillist](http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/Discussion-Compression-for-sort-temp-files-in-Carbomdata-td31747.html) to get more information
Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:
- [X] Any interfaces changed?
`YES, ONLY CHANGE INTERNAL INTERFACES`
- [X] Any backward compatibility impacted?
`NO`
- [X] Document update required?
`YES, RELATED DOCUMENT HAS BEEN UPDATED`
- [X] Testing done
Please provide details on
- Whether new unit test cases have been added or why no new tests are required?
`ADDED TESTS`
- How it is tested? Please attach test report.
`TESTED IN LOCAL CLUSTER`
- Is it a performance related change? Please attach the performance test report.
`YES`
- Any additional information to help reviewers in testing this change.
`The key point lies in` **`SortStepRowHandler`**`. It is used to convert raw row to 3-parted row and read/write row from/to sort temp file/unsafe memory`
- [X] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
`NOT RELATED`
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/xuchuanyin/carbondata bug_compress_sort_temp_1222
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/1707.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1707
----
commit 78684a172f0346584ee992bfc40750b03a9f814b
Author: xuchuanyin <xu...@...>
Date: 2017-12-07T08:31:58Z
Fix bugs in compressing sort temp file
1. fix bugs in compressing sort temp file, use file-level compression
instead of batch-record-level compression
2. reduce duplicate code in reading & writing sort temp file
and make it more readable
3. optimize sort procedure:
Before:
raw row that has been converted(call it 'RawRow' for short) ->
sort on RawRow ->
write RawRow to temp sort file ->
read RawRow from temp sort file ->
sort on RawRow -> ... ->
at the final sort, sort on RawRow and convert the RawRow to 3 'PartedRow' ->
write 'PartedRow' to DataFile in write procedure.
After:
raw row that has been converted(call it 'RawRow' for short) ->
convert RawRow to 3 'PartedRow' ->
sort on PartedRow ->
write PartedRow to temp sort file ->
read PartedRow from temp sort file ->
sort on PartedRow -> ... ->
at the final sort, sort on PartedRow ->
write 'PartedRow' to DataFile in write procedure.
4. add tests
5. remove unused code
6. update docs, add property to configure the compressor
----
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1707
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2565/
---
[GitHub] carbondata pull request #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and opt...
Posted by xuchuanyin <gi...@git.apache.org>.
Github user xuchuanyin commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1707#discussion_r158902533
--- Diff: core/src/main/java/org/apache/carbondata/core/datastore/filesystem/AbstractDFSCarbonFile.java ---
@@ -315,6 +319,49 @@ public boolean delete() {
return new DataInputStream(new BufferedInputStream(stream));
}
+ @Override public DataInputStream getDataInputStream(String path, FileFactory.FileType fileType,
--- End diff --
Fine, I'll reduce the duplicate code.
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by xuchuanyin <gi...@git.apache.org>.
Github user xuchuanyin commented on the issue:
https://github.com/apache/carbondata/pull/1707
retest this please
---
[GitHub] carbondata pull request #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and opt...
Posted by xuchuanyin <gi...@git.apache.org>.
Github user xuchuanyin commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1707#discussion_r158927598
--- Diff: processing/src/main/java/org/apache/carbondata/processing/loading/sort/unsafe/holder/UnsafeSortTempFileChunkHolder.java ---
@@ -312,7 +217,7 @@ private void fillDataForPrefetch() {
}
// write complex dimensions here.
- for (; dimCount < dimensionCount; dimCount++) {
+ for (; dimCount < sortParameters.getComplexDimColCount(); dimCount++) {
--- End diff --
yeah, I made a mistake when applying the changes to this PR.
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1707
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2393/
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1707
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2340/
---
[GitHub] carbondata pull request #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and opt...
Posted by xuchuanyin <gi...@git.apache.org>.
Github user xuchuanyin commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1707#discussion_r158927577
--- Diff: processing/src/main/java/org/apache/carbondata/processing/loading/sort/unsafe/holder/UnsafeSortTempFileChunkHolder.java ---
@@ -296,11 +200,12 @@ private void fillDataForPrefetch() {
* @throws CarbonSortKeyAndGroupByException
*/
private Object[] getRowFromStream() throws CarbonSortKeyAndGroupByException {
- Object[] row = new Object[dimensionCount + measureCount];
+ Object[] row = new Object[sortParameters.getDimColCount()
+ + sortParameters.getComplexDimColCount() + sortParameters.getMeasureColCount()];
--- End diff --
Fine~ I also find the same issues in other files and will fix it at the same time.
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by xuchuanyin <gi...@git.apache.org>.
Github user xuchuanyin commented on the issue:
https://github.com/apache/carbondata/pull/1707
This PR is to replace PR #1632
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1707
Build Success with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1108/
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1707
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2466/
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1707
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2356/
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1707
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2306/
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by xuchuanyin <gi...@git.apache.org>.
Github user xuchuanyin commented on the issue:
https://github.com/apache/carbondata/pull/1707
retest this please
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1707
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2443/
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by xuchuanyin <gi...@git.apache.org>.
Github user xuchuanyin commented on the issue:
https://github.com/apache/carbondata/pull/1707
retest this please
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1707
Build Failed with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1079/
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1707
Build Success with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1223/
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by xuchuanyin <gi...@git.apache.org>.
Github user xuchuanyin commented on the issue:
https://github.com/apache/carbondata/pull/1707
retest this please
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1707
Build Success with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1132/
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on the issue:
https://github.com/apache/carbondata/pull/1707
LGTM
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by xuchuanyin <gi...@git.apache.org>.
Github user xuchuanyin commented on the issue:
https://github.com/apache/carbondata/pull/1707
retest this please
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1707
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2294/
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1707
Build Success with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1119/
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1707
Build Success with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1090/
---
[GitHub] carbondata pull request #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and opt...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1707#discussion_r158901473
--- Diff: processing/src/main/java/org/apache/carbondata/processing/loading/sort/unsafe/holder/UnsafeSortTempFileChunkHolder.java ---
@@ -296,11 +200,12 @@ private void fillDataForPrefetch() {
* @throws CarbonSortKeyAndGroupByException
*/
private Object[] getRowFromStream() throws CarbonSortKeyAndGroupByException {
- Object[] row = new Object[dimensionCount + measureCount];
+ Object[] row = new Object[sortParameters.getDimColCount()
+ + sortParameters.getComplexDimColCount() + sortParameters.getMeasureColCount()];
--- End diff --
I think better keep the old of keeping variables at class level instead of calling method for each row.
---
[GitHub] carbondata pull request #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and opt...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1707#discussion_r158901253
--- Diff: processing/src/main/java/org/apache/carbondata/processing/loading/sort/unsafe/holder/UnsafeSortTempFileChunkHolder.java ---
@@ -296,11 +200,12 @@ private void fillDataForPrefetch() {
* @throws CarbonSortKeyAndGroupByException
*/
private Object[] getRowFromStream() throws CarbonSortKeyAndGroupByException {
- Object[] row = new Object[dimensionCount + measureCount];
+ Object[] row = new Object[sortParameters.getDimColCount()
+ + sortParameters.getComplexDimColCount() + sortParameters.getMeasureColCount()];
--- End diff --
I think there is difference with changed code compare to old code. Please double check the changes. Complex dimensions may not work
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1707
Build Failed with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1170/
---
[GitHub] carbondata pull request #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and opt...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1707#discussion_r158900792
--- Diff: core/src/main/java/org/apache/carbondata/core/datastore/filesystem/AbstractDFSCarbonFile.java ---
@@ -331,6 +378,26 @@ public boolean delete() {
return fs.create(pt, true, bufferSize, fs.getDefaultReplication(pt), blockSize);
}
+ @Override public DataOutputStream getDataOutputStream(String path, FileFactory.FileType fileType,
--- End diff --
It already has a method in this class , change that method to use codec as per the file extension
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1707
Build Success with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1242/
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by xuchuanyin <gi...@git.apache.org>.
Github user xuchuanyin commented on the issue:
https://github.com/apache/carbondata/pull/1707
retest this please
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1707
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2408/
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1707
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2506/
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1707
Build Success with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1141/
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by xuchuanyin <gi...@git.apache.org>.
Github user xuchuanyin commented on the issue:
https://github.com/apache/carbondata/pull/1707
retest this please
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1707
Build Failed with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1184/
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1707
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2372/
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1707
Build Success with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1024/
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1707
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2325/
---
[GitHub] carbondata pull request #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and opt...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1707#discussion_r158900586
--- Diff: core/src/main/java/org/apache/carbondata/core/datastore/filesystem/AbstractDFSCarbonFile.java ---
@@ -315,6 +319,49 @@ public boolean delete() {
return new DataInputStream(new BufferedInputStream(stream));
}
+ @Override public DataInputStream getDataInputStream(String path, FileFactory.FileType fileType,
--- End diff --
It already has a method in this class which chooses codec depends on the extension, Better use that instead of duplicating.
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1707
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2333/
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by xuchuanyin <gi...@git.apache.org>.
Github user xuchuanyin commented on the issue:
https://github.com/apache/carbondata/pull/1707
retest this please
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1707
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2596/
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by xuchuanyin <gi...@git.apache.org>.
Github user xuchuanyin commented on the issue:
https://github.com/apache/carbondata/pull/1707
@ravipesala Thanks for your carefully review. I've fixed the issues and tested in my local cluster to make sure it's OK.
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by xuchuanyin <gi...@git.apache.org>.
Github user xuchuanyin commented on the issue:
https://github.com/apache/carbondata/pull/1707
retest this please
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1707
Build Success with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1124/
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1707
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2247/
---
[GitHub] carbondata pull request #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and opt...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1707#discussion_r158901262
--- Diff: processing/src/main/java/org/apache/carbondata/processing/loading/sort/unsafe/holder/UnsafeSortTempFileChunkHolder.java ---
@@ -312,7 +217,7 @@ private void fillDataForPrefetch() {
}
// write complex dimensions here.
- for (; dimCount < dimensionCount; dimCount++) {
+ for (; dimCount < sortParameters.getComplexDimColCount(); dimCount++) {
--- End diff --
I think there is difference with changed code compare to old code. Please double check the changes. Complex dimensions may not work
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1707
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2350/
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1707
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2339/
---
[GitHub] carbondata pull request #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and opt...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/carbondata/pull/1707
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1707
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2386/
---
[GitHub] carbondata issue #1707: [CARBONDATA-1839] [DataLoad] Fix bugs and optimize i...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1707
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2605/
---