You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by "Ajantha Bhat (JIRA)" <ji...@apache.org> on 2018/11/27 14:55:00 UTC
[jira] [Updated] (CARBONDATA-3136) JVM crash with preaggregate
datamap
[ https://issues.apache.org/jira/browse/CARBONDATA-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ajantha Bhat updated CARBONDATA-3136:
-------------------------------------
Description:
JVM crash with preaggregate datamap.
callstack:
Stack: [0x00007efebd49a000,0x00007efebd59b000], sp=0x00007efebd598dc8, free space=1019k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x7b2b50]
J 7620 sun.misc.Unsafe.copyMemory(Ljava/lang/Object;JLjava/lang/Object;JJ)V (0 bytes) @ 0x00007eff4a3479e1 [0x00007eff4a347900+0xe1]
j org.apache.spark.unsafe.Platform.copyMemory(Ljava/lang/Object;JLjava/lang/Object;JJ)V+34
j org.apache.spark.sql.catalyst.expressions.UnsafeRow.getBinary(I)[B+54
j org.apache.spark.sql.catalyst.expressions.UnsafeRow.getDecimal(III)Lorg/apache/spark/sql/types/Decimal;+30
j org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection.apply(Lorg/apache/spark/sql/catalyst/InternalRow;)Lorg/apache/spark/sql/catalyst/expressions/UnsafeRow;+36
j org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection.apply(Ljava/lang/Object;)Ljava/lang/Object;+5
J 3104 C1 scala.collection.Iterator$$anon$11.next()Ljava/lang/Object; (19 bytes) @ 0x00007eff49154724 [0x00007eff49154560+0x1c4]
j org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(Lscala/collection/Iterator;)Lscala/collection/Iterator;+78
j org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(Ljava/lang/Object;)Ljava/lang/Object;+5
J 14007 C1 org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object; (17 bytes) @ 0x00007eff4a6ed204 [0x00007eff4a6ecc40+0x5c4]
J 11684 C1 org.apache.spark.rdd.MapPartitionsRDD.compute(Lorg/apache/spark/Partition;Lorg/apache/spark/TaskContext;)Lscala/collection/Iterator; (36 bytes) @ 0x00007eff4ad11274 [0x00007eff4ad10f60+0x314]
J 13771 C1 org.apache.spark.rdd.RDD.iterator(Lorg/apache/spark/Partition;Lorg/apache/spark/TaskContext;)Lscala/collection/Iterator; (46 bytes) @ 0x00007eff4b39dd3c [0x00007eff4b39d160+0xbdc]
test({color:#008000}"Test Pre_aggregate with decimal column with order by"{color}) {
sql({color:#008000}"drop table if exists maintable"{color})
sql({color:#008000}"create table maintable(name string, decimal_col decimal(30,16)) stored by 'carbondata'"{color})
sql({color:#008000}"insert into table maintable select 'abc',452.564"{color})
sql(
{color:#008000}"create datamap ag1 on table maintable using 'preaggregate' as select name,avg(decimal_col)" {color}+
{color:#008000}" from maintable group by name"{color})
checkAnswer(sql({color:#008000}"select avg(decimal_col) from maintable group by name order by name"{color}),
{color:#660e7a}Seq{color}(Row({color:#0000ff}452.56400000000000000000{color})))
}
was:
JVM crash with preaggregate datamap.
callstack:
Stack: [0x00007efebd49a000,0x00007efebd59b000], sp=0x00007efebd598dc8, free space=1019k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x7b2b50]
J 7620 sun.misc.Unsafe.copyMemory(Ljava/lang/Object;JLjava/lang/Object;JJ)V (0 bytes) @ 0x00007eff4a3479e1 [0x00007eff4a347900+0xe1]
j org.apache.spark.unsafe.Platform.copyMemory(Ljava/lang/Object;JLjava/lang/Object;JJ)V+34
j org.apache.spark.sql.catalyst.expressions.UnsafeRow.getBinary(I)[B+54
j org.apache.spark.sql.catalyst.expressions.UnsafeRow.getDecimal(III)Lorg/apache/spark/sql/types/Decimal;+30
j org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection.apply(Lorg/apache/spark/sql/catalyst/InternalRow;)Lorg/apache/spark/sql/catalyst/expressions/UnsafeRow;+36
j org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection.apply(Ljava/lang/Object;)Ljava/lang/Object;+5
J 3104 C1 scala.collection.Iterator$$anon$11.next()Ljava/lang/Object; (19 bytes) @ 0x00007eff49154724 [0x00007eff49154560+0x1c4]
j org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(Lscala/collection/Iterator;)Lscala/collection/Iterator;+78
j org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(Ljava/lang/Object;)Ljava/lang/Object;+5
J 14007 C1 org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object; (17 bytes) @ 0x00007eff4a6ed204 [0x00007eff4a6ecc40+0x5c4]
J 11684 C1 org.apache.spark.rdd.MapPartitionsRDD.compute(Lorg/apache/spark/Partition;Lorg/apache/spark/TaskContext;)Lscala/collection/Iterator; (36 bytes) @ 0x00007eff4ad11274 [0x00007eff4ad10f60+0x314]
J 13771 C1 org.apache.spark.rdd.RDD.iterator(Lorg/apache/spark/Partition;Lorg/apache/spark/TaskContext;)Lscala/collection/Iterator; (46 bytes) @ 0x00007eff4b39dd3c [0x00007eff4b39d160+0xbdc]
> JVM crash with preaggregate datamap
> -----------------------------------
>
> Key: CARBONDATA-3136
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3136
> Project: CarbonData
> Issue Type: Improvement
> Reporter: Ajantha Bhat
> Assignee: Ajantha Bhat
> Priority: Major
>
> JVM crash with preaggregate datamap.
> callstack:
> Stack: [0x00007efebd49a000,0x00007efebd59b000], sp=0x00007efebd598dc8, free space=1019k
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
> V [libjvm.so+0x7b2b50]
> J 7620 sun.misc.Unsafe.copyMemory(Ljava/lang/Object;JLjava/lang/Object;JJ)V (0 bytes) @ 0x00007eff4a3479e1 [0x00007eff4a347900+0xe1]
> j org.apache.spark.unsafe.Platform.copyMemory(Ljava/lang/Object;JLjava/lang/Object;JJ)V+34
> j org.apache.spark.sql.catalyst.expressions.UnsafeRow.getBinary(I)[B+54
> j org.apache.spark.sql.catalyst.expressions.UnsafeRow.getDecimal(III)Lorg/apache/spark/sql/types/Decimal;+30
> j org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection.apply(Lorg/apache/spark/sql/catalyst/InternalRow;)Lorg/apache/spark/sql/catalyst/expressions/UnsafeRow;+36
> j org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection.apply(Ljava/lang/Object;)Ljava/lang/Object;+5
> J 3104 C1 scala.collection.Iterator$$anon$11.next()Ljava/lang/Object; (19 bytes) @ 0x00007eff49154724 [0x00007eff49154560+0x1c4]
> j org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(Lscala/collection/Iterator;)Lscala/collection/Iterator;+78
> j org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(Ljava/lang/Object;)Ljava/lang/Object;+5
> J 14007 C1 org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object; (17 bytes) @ 0x00007eff4a6ed204 [0x00007eff4a6ecc40+0x5c4]
> J 11684 C1 org.apache.spark.rdd.MapPartitionsRDD.compute(Lorg/apache/spark/Partition;Lorg/apache/spark/TaskContext;)Lscala/collection/Iterator; (36 bytes) @ 0x00007eff4ad11274 [0x00007eff4ad10f60+0x314]
> J 13771 C1 org.apache.spark.rdd.RDD.iterator(Lorg/apache/spark/Partition;Lorg/apache/spark/TaskContext;)Lscala/collection/Iterator; (46 bytes) @ 0x00007eff4b39dd3c [0x00007eff4b39d160+0xbdc]
>
>
> test({color:#008000}"Test Pre_aggregate with decimal column with order by"{color}) {
> sql({color:#008000}"drop table if exists maintable"{color})
> sql({color:#008000}"create table maintable(name string, decimal_col decimal(30,16)) stored by 'carbondata'"{color})
> sql({color:#008000}"insert into table maintable select 'abc',452.564"{color})
> sql(
> {color:#008000}"create datamap ag1 on table maintable using 'preaggregate' as select name,avg(decimal_col)" {color}+
> {color:#008000}" from maintable group by name"{color})
> checkAnswer(sql({color:#008000}"select avg(decimal_col) from maintable group by name order by name"{color}),
> {color:#660e7a}Seq{color}(Row({color:#0000ff}452.56400000000000000000{color})))
> }
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)