You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@carbondata.apache.org by ma...@apache.org on 2017/09/20 09:11:37 UTC
carbondata git commit: [CARBONDATA-1490] Fixed memory allocation for
carbon row
Repository: carbondata
Updated Branches:
refs/heads/master e9c24c505 -> d4eabbe56
[CARBONDATA-1490] Fixed memory allocation for carbon row
While creating carbon row the space for measures is assigned based on outputLength and not on measureCount which will tend to allocate more space than is actually needed.
outputLength = measureCount + ((noDictCount + complexCount) > 0 ? 1 : 0) + 1.
When the table has 1 complex column then measureCount=0, noDictCount=0, complexCount=1 therefore, outputLength = 2. Even if there are no measure columns the measure object is created with 2 null values.
This closes #1373
Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/d4eabbe5
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/d4eabbe5
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/d4eabbe5
Branch: refs/heads/master
Commit: d4eabbe56aa6825d84933ab5a0fc78f26762cea9
Parents: e9c24c5
Author: kunal642 <ku...@gmail.com>
Authored: Wed Sep 20 11:02:00 2017 +0530
Committer: manishgupta88 <to...@gmail.com>
Committed: Wed Sep 20 14:41:38 2017 +0530
----------------------------------------------------------------------
.../newflow/steps/CarbonRowDataWriterProcessorStepImpl.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/carbondata/blob/d4eabbe5/processing/src/main/java/org/apache/carbondata/processing/newflow/steps/CarbonRowDataWriterProcessorStepImpl.java
----------------------------------------------------------------------
diff --git a/processing/src/main/java/org/apache/carbondata/processing/newflow/steps/CarbonRowDataWriterProcessorStepImpl.java b/processing/src/main/java/org/apache/carbondata/processing/newflow/steps/CarbonRowDataWriterProcessorStepImpl.java
index 765e0ed..643d2ba 100644
--- a/processing/src/main/java/org/apache/carbondata/processing/newflow/steps/CarbonRowDataWriterProcessorStepImpl.java
+++ b/processing/src/main/java/org/apache/carbondata/processing/newflow/steps/CarbonRowDataWriterProcessorStepImpl.java
@@ -254,7 +254,7 @@ public class CarbonRowDataWriterProcessorStepImpl extends AbstractDataLoadProces
nonDicArray[nonDicIndex++] = (byte[]) row.getObject(dimCount);
}
- Object[] measures = new Object[outputLength];
+ Object[] measures = new Object[measureCount];
for (int i = 0; i < this.measureCount; i++) {
measures[i] = row.getObject(i + this.dimensionWithComplexCount);
}