You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by "anubhav tarar (JIRA)" <ji...@apache.org> on 2018/01/15 05:50:00 UTC
[jira] [Closed] (CARBONDATA-1798) class cast exception when loading
data from a hive table to carbondata table
[ https://issues.apache.org/jira/browse/CARBONDATA-1798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
anubhav tarar closed CARBONDATA-1798.
-------------------------------------
Resolution: Fixed
> class cast exception when loading data from a hive table to carbondata table
> ----------------------------------------------------------------------------
>
> Key: CARBONDATA-1798
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1798
> Project: CarbonData
> Issue Type: Bug
> Components: spark-integration
> Affects Versions: 1.3.0
> Environment: spark2.1
> Reporter: anubhav tarar
> Assignee: anubhav tarar
> Priority: Major
>
> 1. i created the hive tpch table named lineitem with around 10 gb data
> CREATE TABLE LINEITEM ( L_ORDERKEY INT ,L_PARTKEY INT ,L_SUPPKEY INT ,L_LINENUMBER INT ,L_QUANTITY DECIMAL(15,2) ,L_EXTENDEDPRICE DECIMAL(15,2) ,L_DISCOUNT DECIMAL(15,2) ,L_TAX DECIMAL(15,2) ,L_RETURNFLAG STRING ,L_LINESTATUS STRING ,L_SHIPDATE Date ,L_COMMITDATE Date ,L_RECEIPTDATE Date ,L_SHIPINSTRUCT STRING ,L_SHIPMODE STRING ,L_COMMENT STRING) stored as orc;
> 2.tried to insert the data from that hive table into carbondata table
> CREATE TABLE LINEITEM ( L_ORDERKEY INT ,L_PARTKEY INT ,L_SUPPKEY INT ,L_LINENUMBER INT ,L_QUANTITY DECIMAL(15,2) ,L_EXTENDEDPRICE DECIMAL(15,2) ,L_DISCOUNT DECIMAL(15,2) ,L_TAX DECIMAL(15,2) ,L_RETURNFLAG STRING ,L_LINESTATUS STRING ,L_SHIPDATE Date ,L_COMMITDATE Date ,L_RECEIPTDATE Date ,L_SHIPINSTRUCT STRING ,L_SHIPMODE STRING ,L_COMMENT STRING) stored by 'carbondata';
> then i used INSERT INTO LINEITEM SELECT * FROM LINEITEM_ORC;
> here are error logs
> 17/11/22 16:56:40 INFO BlockManagerInfo: Added broadcast_22_piece0 in memory on slave1:41401 (size: 25.2 KB, free: 3.0 GB)
> 17/11/22 16:59:12 WARN TaskSetManager: Lost task 0.0 in stage 15.0 (TID 20, hadoop-master, executor 2): org.apache.carbondata.processing.loading.exception.CarbonDataLoadingException:
> at org.apache.carbondata.processing.loading.sort.impl.UnsafeParallelReadMergeSorterImpl.sort(UnsafeParallelReadMergeSorterImpl.java:114)
> at org.apache.carbondata.processing.loading.steps.SortProcessorStepImpl.execute(SortProcessorStepImpl.java:62)
> at org.apache.carbondata.processing.loading.steps.DataWriterProcessorStepImpl.execute(DataWriterProcessorStepImpl.java:87)
> at org.apache.carbondata.processing.loading.DataLoadExecutor.execute(DataLoadExecutor.java:50)
> at org.apache.carbondata.spark.rdd.NewDataFrameLoaderRDD$$anon$2.<init>(NewCarbonDataLoadRDD.scala:391)
> at org.apache.carbondata.spark.rdd.NewDataFrameLoaderRDD.internalCompute(NewCarbonDataLoadRDD.scala:354)
> at org.apache.carbondata.spark.rdd.CarbonRDD.compute(CarbonRDD.scala:60)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
> at org.apache.spark.scheduler.Task.run(Task.scala:99)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.carbondata.processing.sort.exception.CarbonSortKeyAndGroupByException: org.apache.carbondata.processing.sort.exception.CarbonSortKeyAndGroupByException:
> at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeIntermediateMerger.checkForFailure(UnsafeIntermediateMerger.java:188)
> at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeIntermediateMerger.finish(UnsafeIntermediateMerger.java:171)
> at org.apache.carbondata.processing.loading.sort.impl.UnsafeParallelReadMergeSorterImpl.sort(UnsafeParallelReadMergeSorterImpl.java:107)
> ... 14 more
> Caused by: java.util.concurrent.ExecutionException: org.apache.carbondata.processing.sort.exception.CarbonSortKeyAndGroupByException:
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeIntermediateMerger.checkForFailure(UnsafeIntermediateMerger.java:185)
> ... 16 more
> Caused by: org.apache.carbondata.processing.sort.exception.CarbonSortKeyAndGroupByException:
> at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeIntermediateFileMerger.call(UnsafeIntermediateFileMerger.java:142)
> at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeIntermediateFileMerger.call(UnsafeIntermediateFileMerger.java:45)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ... 3 more
> Caused by: java.lang.ClassCastException: java.math.BigDecimal cannot be cast to [B
> at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeIntermediateFileMerger.writeDataTofile(UnsafeIntermediateFileMerger.java:333)
> at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeIntermediateFileMerger.call(UnsafeIntermediateFileMerger.java:113)
> ... 5 more
> 17/11/22 16:59:12 INFO TaskSetManager: Starting task 0.1 in stage 15.0 (TID 22, hadoop-master, executor 2, partition 0, NODE_LOCAL, 7696 bytes)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)