[jira] [Created] (CARBONDATA-1946) Exception thrown after alter data type change operation on dictionary exclude integer type column

["Manish Gupta (JIRA)" <ji...@apache.org>]

Manish Gupta created CARBONDATA-1946:
----------------------------------------

             Summary: Exception thrown after alter data type change operation on dictionary exclude integer type column
                 Key: CARBONDATA-1946
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-1946
             Project: CarbonData
          Issue Type: Bug
            Reporter: Manish Gupta
            Assignee: Manish Gupta
            Priority: Minor


Problem: After restructure change data type operation (INT to BIGINT) on dictionary exclude INT type column if select query is triggered then exception is thrown.

Analysis: This is happening because while retrieving the data the vector is created for BIGINT type (size 8 bytes) which but the actual length of each data is 4 bytes and there is length check while reading the data which is failing.

Steps to reproduce
---------------------------
1. create table table_sort(age int) stored by 'carbondata' TBLPROPERTIES('DICTIONARY_EXCLUDE'='age')
2. insert into table_sort select 32794
3. alter table table_sort change age age bigint
4. select * from table_sort

Exception Trace
--------------------------
17/12/28 12:03:15 ERROR Executor: Exception in task 0.0 in stage 2.0 (TID 2)
java.lang.IllegalArgumentException: Wrong length: 4, expected 8
	at org.apache.carbondata.core.util.ByteUtil.explainWrongLengthOrOffset(ByteUtil.java:581)
	at org.apache.carbondata.core.util.ByteUtil.toLong(ByteUtil.java:553)
	at org.apache.carbondata.core.datastore.chunk.store.impl.safe.SafeVariableLengthDimensionDataChunkStore.fillRow(SafeVariableLengthDimensionDataChunkStore.java:156)
	at org.apache.carbondata.core.datastore.chunk.impl.VariableLengthDimensionDataChunk.fillConvertedChunkData(VariableLengthDimensionDataChunk.java:112)
	at org.apache.carbondata.core.scan.result.AbstractScannedResult.fillColumnarNoDictionaryBatch(AbstractScannedResult.java:246)
	at org.apache.carbondata.core.scan.collector.impl.DictionaryBasedVectorResultCollector.scanAndFillResult(DictionaryBasedVectorResultCollector.java:163)
	at org.apache.carbondata.core.scan.collector.impl.DictionaryBasedVectorResultCollector.collectVectorBatch(DictionaryBasedVectorResultCollector.java:155)
	at org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.processNextBatch(DataBlockIteratorImpl.java:65)
	at org.apache.carbondata.core.scan.result.iterator.VectorDetailQueryResultIterator.processNextBatch(VectorDetailQueryResultIterator.java:46)
	at org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextBatch(VectorizedCarbonRecordReader.java:267)
	at org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextKeyValue(VectorizedCarbonRecordReader.java:155)
	at org.apache.carbondata.spark.rdd.CarbonScanRDD$$anon$1.hasNext(CarbonScanRDD.scala:370)
	at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.scan_nextBatch$(Unknown Source)
	at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown Source)
	at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
	at org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:395)
	at org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:234)
	at org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:228)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:827)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:827)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
	at org.apache.spark.scheduler.Task.run(Task.scala:108)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)