You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by "anubhav tarar (JIRA)" <ji...@apache.org> on 2017/11/22 12:43:00 UTC
[jira] [Created] (CARBONDATA-1798) class cast exception when
loading data from a hive table to carbondata table
anubhav tarar created CARBONDATA-1798:
-----------------------------------------
Summary: class cast exception when loading data from a hive table to carbondata table
Key: CARBONDATA-1798
URL: https://issues.apache.org/jira/browse/CARBONDATA-1798
Project: CarbonData
Issue Type: Bug
Components: spark-integration
Affects Versions: 1.3.0
Environment: spark2.1
Reporter: anubhav tarar
Assignee: anubhav tarar
1. i created the hive tpch table named lineitem with around 10 gb data
CREATE TABLE LINEITEM ( L_ORDERKEY INT ,L_PARTKEY INT ,L_SUPPKEY INT ,L_LINENUMBER INT ,L_QUANTITY DECIMAL(15,2) ,L_EXTENDEDPRICE DECIMAL(15,2) ,L_DISCOUNT DECIMAL(15,2) ,L_TAX DECIMAL(15,2) ,L_RETURNFLAG STRING ,L_LINESTATUS STRING ,L_SHIPDATE Date ,L_COMMITDATE Date ,L_RECEIPTDATE Date ,L_SHIPINSTRUCT STRING ,L_SHIPMODE STRING ,L_COMMENT STRING) stored as orc;
2.tried to insert the data from that hive table into carbondata table
CREATE TABLE LINEITEM ( L_ORDERKEY INT ,L_PARTKEY INT ,L_SUPPKEY INT ,L_LINENUMBER INT ,L_QUANTITY DECIMAL(15,2) ,L_EXTENDEDPRICE DECIMAL(15,2) ,L_DISCOUNT DECIMAL(15,2) ,L_TAX DECIMAL(15,2) ,L_RETURNFLAG STRING ,L_LINESTATUS STRING ,L_SHIPDATE Date ,L_COMMITDATE Date ,L_RECEIPTDATE Date ,L_SHIPINSTRUCT STRING ,L_SHIPMODE STRING ,L_COMMENT STRING) stored by 'carbondata';
then i used INSERT INTO LINEITEM SELECT * FROM LINEITEM_ORC;
here are error logs
17/11/22 16:56:40 INFO BlockManagerInfo: Added broadcast_22_piece0 in memory on slave1:41401 (size: 25.2 KB, free: 3.0 GB)
17/11/22 16:59:12 WARN TaskSetManager: Lost task 0.0 in stage 15.0 (TID 20, hadoop-master, executor 2): org.apache.carbondata.processing.loading.exception.CarbonDataLoadingException:
at org.apache.carbondata.processing.loading.sort.impl.UnsafeParallelReadMergeSorterImpl.sort(UnsafeParallelReadMergeSorterImpl.java:114)
at org.apache.carbondata.processing.loading.steps.SortProcessorStepImpl.execute(SortProcessorStepImpl.java:62)
at org.apache.carbondata.processing.loading.steps.DataWriterProcessorStepImpl.execute(DataWriterProcessorStepImpl.java:87)
at org.apache.carbondata.processing.loading.DataLoadExecutor.execute(DataLoadExecutor.java:50)
at org.apache.carbondata.spark.rdd.NewDataFrameLoaderRDD$$anon$2.<init>(NewCarbonDataLoadRDD.scala:391)
at org.apache.carbondata.spark.rdd.NewDataFrameLoaderRDD.internalCompute(NewCarbonDataLoadRDD.scala:354)
at org.apache.carbondata.spark.rdd.CarbonRDD.compute(CarbonRDD.scala:60)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
at org.apache.spark.scheduler.Task.run(Task.scala:99)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.carbondata.processing.sort.exception.CarbonSortKeyAndGroupByException: org.apache.carbondata.processing.sort.exception.CarbonSortKeyAndGroupByException:
at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeIntermediateMerger.checkForFailure(UnsafeIntermediateMerger.java:188)
at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeIntermediateMerger.finish(UnsafeIntermediateMerger.java:171)
at org.apache.carbondata.processing.loading.sort.impl.UnsafeParallelReadMergeSorterImpl.sort(UnsafeParallelReadMergeSorterImpl.java:107)
... 14 more
Caused by: java.util.concurrent.ExecutionException: org.apache.carbondata.processing.sort.exception.CarbonSortKeyAndGroupByException:
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeIntermediateMerger.checkForFailure(UnsafeIntermediateMerger.java:185)
... 16 more
Caused by: org.apache.carbondata.processing.sort.exception.CarbonSortKeyAndGroupByException:
at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeIntermediateFileMerger.call(UnsafeIntermediateFileMerger.java:142)
at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeIntermediateFileMerger.call(UnsafeIntermediateFileMerger.java:45)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Caused by: java.lang.ClassCastException: java.math.BigDecimal cannot be cast to [B
at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeIntermediateFileMerger.writeDataTofile(UnsafeIntermediateFileMerger.java:333)
at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeIntermediateFileMerger.call(UnsafeIntermediateFileMerger.java:113)
... 5 more
17/11/22 16:59:12 INFO TaskSetManager: Starting task 0.1 in stage 15.0 (TID 22, hadoop-master, executor 2, partition 0, NODE_LOCAL, 7696 bytes)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)