[jira] [Resolved] (CARBONDATA-1946) Exception thrown after alter data type change operation on dictionary exclude integer type column

["Ravindra Pesala (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CARBONDATA-1946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ravindra Pesala resolved CARBONDATA-1946.
-----------------------------------------
       Resolution: Fixed
    Fix Version/s: 1.3.0

> Exception thrown after alter data type change operation on dictionary exclude integer type column
> -------------------------------------------------------------------------------------------------
>
>                 Key: CARBONDATA-1946
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-1946
>             Project: CarbonData
>          Issue Type: Bug
>            Reporter: Manish Gupta
>            Assignee: Manish Gupta
>            Priority: Minor
>             Fix For: 1.3.0
>
>          Time Spent: 5h
>  Remaining Estimate: 0h
>
> Problem: After restructure change data type operation (INT to BIGINT) on dictionary exclude INT type column if select query is triggered then exception is thrown.
> Analysis: This is happening because while retrieving the data the vector is created for BIGINT type (size 8 bytes) which but the actual length of each data is 4 bytes and there is length check while reading the data which is failing.
> Steps to reproduce
> ---------------------------
> 1. create table table_sort(age int) stored by 'carbondata' TBLPROPERTIES('DICTIONARY_EXCLUDE'='age')
> 2. insert into table_sort select 32794
> 3. alter table table_sort change age age bigint
> 4. select * from table_sort
> Exception Trace
> --------------------------
> 17/12/28 12:03:15 ERROR Executor: Exception in task 0.0 in stage 2.0 (TID 2)
> java.lang.IllegalArgumentException: Wrong length: 4, expected 8
> 	at org.apache.carbondata.core.util.ByteUtil.explainWrongLengthOrOffset(ByteUtil.java:581)
> 	at org.apache.carbondata.core.util.ByteUtil.toLong(ByteUtil.java:553)
> 	at org.apache.carbondata.core.datastore.chunk.store.impl.safe.SafeVariableLengthDimensionDataChunkStore.fillRow(SafeVariableLengthDimensionDataChunkStore.java:156)
> 	at org.apache.carbondata.core.datastore.chunk.impl.VariableLengthDimensionDataChunk.fillConvertedChunkData(VariableLengthDimensionDataChunk.java:112)
> 	at org.apache.carbondata.core.scan.result.AbstractScannedResult.fillColumnarNoDictionaryBatch(AbstractScannedResult.java:246)
> 	at org.apache.carbondata.core.scan.collector.impl.DictionaryBasedVectorResultCollector.scanAndFillResult(DictionaryBasedVectorResultCollector.java:163)
> 	at org.apache.carbondata.core.scan.collector.impl.DictionaryBasedVectorResultCollector.collectVectorBatch(DictionaryBasedVectorResultCollector.java:155)
> 	at org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.processNextBatch(DataBlockIteratorImpl.java:65)
> 	at org.apache.carbondata.core.scan.result.iterator.VectorDetailQueryResultIterator.processNextBatch(VectorDetailQueryResultIterator.java:46)
> 	at org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextBatch(VectorizedCarbonRecordReader.java:267)
> 	at org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextKeyValue(VectorizedCarbonRecordReader.java:155)
> 	at org.apache.carbondata.spark.rdd.CarbonScanRDD$$anon$1.hasNext(CarbonScanRDD.scala:370)
> 	at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.scan_nextBatch$(Unknown Source)
> 	at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown Source)
> 	at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
> 	at org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:395)
> 	at org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:234)
> 	at org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:228)
> 	at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:827)
> 	at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:827)
> 	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> 	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> 	at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> 	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
> 	at org.apache.spark.scheduler.Task.run(Task.scala:108)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)