You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Reynold Xin (JIRA)" <ji...@apache.org> on 2016/06/02 17:58:59 UTC
[jira] [Updated] (SPARK-15604) Spark-SQL: Get
com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
when runing query_1.sql of TPC-DS
[ https://issues.apache.org/jira/browse/SPARK-15604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Reynold Xin updated SPARK-15604:
--------------------------------
Fix Version/s: 2.0.0
> Spark-SQL: Get com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException when runing query_1.sql of TPC-DS
> ------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-15604
> URL: https://issues.apache.org/jira/browse/SPARK-15604
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Environment: sparkļ¼Master development branch
> hadoop:2.7.1
> hive:1.2.1
> spark:Standalone, 1 Master,3 Worker
> Reporter: wenjing wang
> Labels: performance
> Fix For: 2.0.0
>
> Attachments: 1_1.png, 1_2.png, 1_3.png, 2_1.png, 2_2.png, 3_1.png, 3_2.png, 4_1.png, 4_2.png, test4.txt
>
>
> When I run query_1.sql of TPC-DS like:
> bin/spark-sql --master spark://192.168.30.78:7077 -i /data/disk1/data/100G/query_1.sql
> The process fails with a exception throwing:
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> The error details are:
> {code}
> bin/spark-sql --master spark://192.168.30.78:7077 -i /data/disk1/data/100G/query_1.sql
> 16/05/27 06:16:52 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where
> applicable
> 16/05/27 06:16:57 WARN ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so
> recording the schema version 1.2.0
> 16/05/27 06:16:57 WARN ObjectStore: Failed to get database default, returning NoSuchObjectException
> 16/05/27 06:16:58 WARN AbstractHandler: No Server set for org.spark_project.jetty.server.handler.ErrorHandler@55a354d8
> 16/05/27 06:16:59 WARN SparkContext: Use an existing SparkContext, some configuration may not take effect.
> [Stage 9:=================================> (123 + 42) / 200]16/05/27 06:17:38 ERROR TaskResultGetter: Exception while
> getting task result
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
> at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
> at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:25)
> at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:19)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> at org.apache.spark.serializer.KryoSerializerInstance.deserialize(KryoSerializer.scala:312)
> at org.apache.spark.scheduler.DirectTaskResult.value(TaskResult.scala:87)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:66)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
> at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:159)
> at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:150)
> at scala.math.Ordering$$anon$4.compare(Ordering.scala:111)
> at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649)
> at java.util.PriorityQueue.siftUp(PriorityQueue.java:627)
> at java.util.PriorityQueue.offer(PriorityQueue.java:329)
> at java.util.PriorityQueue.add(PriorityQueue.java:306)
> at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:78)
> at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:31)
> at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
> at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
> ... 15 more
> 16/05/27 06:17:38 ERROR TaskResultGetter: Exception while getting task result
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
> at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
> at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:25)
> at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:19)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> at org.apache.spark.serializer.KryoSerializerInstance.deserialize(KryoSerializer.scala:312)
> at org.apache.spark.scheduler.DirectTaskResult.value(TaskResult.scala:87)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:66)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
> at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:159)
> at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:150)
> at scala.math.Ordering$$anon$4.compare(Ordering.scala:111)
> at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649)
> at java.util.PriorityQueue.siftUp(PriorityQueue.java:627)
> at java.util.PriorityQueue.offer(PriorityQueue.java:329)
> at java.util.PriorityQueue.add(PriorityQueue.java:306)
> at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:78)
> at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:31)
> at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
> at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
> ... 15 more
> 16/05/27 06:17:38 ERROR TaskResultGetter: Exception while getting task result
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
> at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
> at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:25)
> at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:19)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> at org.apache.spark.serializer.KryoSerializerInstance.deserialize(KryoSerializer.scala:312)
> at org.apache.spark.scheduler.DirectTaskResult.value(TaskResult.scala:87)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:66)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
> at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:159)
> at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:150)
> at scala.math.Ordering$$anon$4.compare(Ordering.scala:111)
> at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649)
> at java.util.PriorityQueue.siftUp(PriorityQueue.java:627)
> at java.util.PriorityQueue.offer(PriorityQueue.java:329)
> at java.util.PriorityQueue.add(PriorityQueue.java:306)
> at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:78)
> at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:31)
> at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
> at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
> ... 15 more
> 16/05/27 06:17:38 ERROR TaskResultGetter: Exception while getting task result
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
> at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
> at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:25)
> at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:19)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> at org.apache.spark.serializer.KryoSerializerInstance.deserialize(KryoSerializer.scala:312)
> at org.apache.spark.scheduler.DirectTaskResult.value(TaskResult.scala:87)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:66)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
> at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:159)
> at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:150)
> at scala.math.Ordering$$anon$4.compare(Ordering.scala:111)
> at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649)
> at java.util.PriorityQueue.siftUp(PriorityQueue.java:627)
> at java.util.PriorityQueue.offer(PriorityQueue.java:329)
> at java.util.PriorityQueue.add(PriorityQueue.java:306)
> at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:78)
> at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:31)
> at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
> at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
> ... 15 more
> 16/05/27 06:17:38 ERROR TaskResultGetter: Exception while getting task result
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
> at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
> at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:25)
> at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:19)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> at org.apache.spark.serializer.KryoSerializerInstance.deserialize(KryoSerializer.scala:312)
> at org.apache.spark.scheduler.DirectTaskResult.value(TaskResult.scala:87)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:66)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
> at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:159)
> at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:150)
> at scala.math.Ordering$$anon$4.compare(Ordering.scala:111)
> at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649)
> at java.util.PriorityQueue.siftUp(PriorityQueue.java:627)
> at java.util.PriorityQueue.offer(PriorityQueue.java:329)
> at java.util.PriorityQueue.add(PriorityQueue.java:306)
> at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:78)
> at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:31)
> at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
> at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
> ... 15 more
> 16/05/27 06:17:38 ERROR TaskResultGetter: Exception while getting task result
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
> at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
> at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:25)
> at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:19)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> at org.apache.spark.serializer.KryoSerializerInstance.deserialize(KryoSerializer.scala:312)
> at org.apache.spark.scheduler.DirectTaskResult.value(TaskResult.scala:87)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:66)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
> at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:159)
> at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:150)
> at scala.math.Ordering$$anon$4.compare(Ordering.scala:111)
> at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649)
> at java.util.PriorityQueue.siftUp(PriorityQueue.java:627)
> at java.util.PriorityQueue.offer(PriorityQueue.java:329)
> at java.util.PriorityQueue.add(PriorityQueue.java:306)
> at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:78)
> at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:31)
> at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
> at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
> ... 15 more
> 16/05/27 06:17:38 ERROR SparkSQLDriver: Failed in [with customer_total_return as
> (select sr_customer_sk as ctr_customer_sk
> ,sr_store_sk as ctr_store_sk
> ,sum(SR_FEE) as ctr_total_return
> from store_returns
> ,date_dim
> where sr_returned_date_sk = d_date_sk
> and d_year =2000
> group by sr_customer_sk
> ,sr_store_sk)
> select c_customer_id
> from customer_total_return ctr1
> ,store
> ,customer
> where ctr1.ctr_total_return > (select avg(ctr_total_return)*1.2
> from customer_total_return ctr2
> where ctr1.ctr_store_sk = ctr2.ctr_store_sk)
> and s_store_sk = ctr1.ctr_store_sk
> and s_state = 'SD'
> and ctr1.ctr_customer_sk = c_customer_sk
> order by c_customer_id
> limit 100]
> org.apache.spark.SparkException: Job aborted due to stage failure: Exception while getting task result:
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
> at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages
> (DAGScheduler.scala:1435)
> at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1423)
> at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1422)
> at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
> at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1422)
> at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:806)
> at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:806)
> at scala.Option.foreach(Option.scala:257)
> at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:806)
> at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1644)
> at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1603)
> at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1592)
> at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
> at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:632)
> at org.apache.spark.SparkContext.runJob(SparkContext.scala:1872)
> at org.apache.spark.SparkContext.runJob(SparkContext.scala:1935)
> at org.apache.spark.rdd.RDD$$anonfun$reduce$1.apply(RDD.scala:974)
> at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:357)
> at org.apache.spark.rdd.RDD.reduce(RDD.scala:956)
> at org.apache.spark.rdd.RDD$$anonfun$takeOrdered$1.apply(RDD.scala:1371)
> at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:357)
> at org.apache.spark.rdd.RDD.takeOrdered(RDD.scala:1358)
> at org.apache.spark.sql.execution.TakeOrderedAndProjectExec.executeCollect(limit.scala:127)
> at org.apache.spark.sql.execution.SparkPlan.executeCollectPublic(SparkPlan.scala:310)
> at org.apache.spark.sql.execution.QueryExecution.hiveResultString(QueryExecution.scala:126)
> at org.apache.spark.sql.hive.thriftserver.SparkSQLDriver.run(SparkSQLDriver.scala:63)
> at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.processCmd(SparkSQLCLIDriver.scala:323)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:311)
> at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:409)
> at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:425)
> at org.apache.hadoop.hive.cli.CliDriver.processInitFiles(CliDriver.java:436)
> at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkSQLCLIDriver.scala:157)
> at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQLCLIDriver.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:724)
> at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
> at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> org.apache.spark.SparkException: Job aborted due to stage failure: Exception while getting task result:
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
> at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages
> (DAGScheduler.scala:1435)
> at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1423)
> at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1422)
> at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
> at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1422)
> at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:806)
> at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:806)
> at scala.Option.foreach(Option.scala:257)
> at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:806)
> at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1644)
> at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1603)
> at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1592)
> at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
> at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:632)
> at org.apache.spark.SparkContext.runJob(SparkContext.scala:1872)
> at org.apache.spark.SparkContext.runJob(SparkContext.scala:1935)
> at org.apache.spark.rdd.RDD$$anonfun$reduce$1.apply(RDD.scala:974)
> at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:357)
> at org.apache.spark.rdd.RDD.reduce(RDD.scala:956)
> at org.apache.spark.rdd.RDD$$anonfun$takeOrdered$1.apply(RDD.scala:1371)
> at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:357)
> at org.apache.spark.rdd.RDD.takeOrdered(RDD.scala:1358)
> at org.apache.spark.sql.execution.TakeOrderedAndProjectExec.executeCollect(limit.scala:127)
> at org.apache.spark.sql.execution.SparkPlan.executeCollectPublic(SparkPlan.scala:310)
> at org.apache.spark.sql.execution.QueryExecution.hiveResultString(QueryExecution.scala:126)
> at org.apache.spark.sql.hive.thriftserver.SparkSQLDriver.run(SparkSQLDriver.scala:63)
> at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.processCmd(SparkSQLCLIDriver.scala:323)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:311)
> at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:409)
> at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:425)
> at org.apache.hadoop.hive.cli.CliDriver.processInitFiles(CliDriver.java:436)
> at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkSQLCLIDriver.scala:157)
> at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQLCLIDriver.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:724)
> at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
> at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> 16/05/27 06:17:38 ERROR TaskResultGetter: Exception while getting task result
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
> at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
> at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:25)
> at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:19)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> at org.apache.spark.serializer.KryoSerializerInstance.deserialize(KryoSerializer.scala:312)
> at org.apache.spark.scheduler.DirectTaskResult.value(TaskResult.scala:87)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:66)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
> at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:159)
> at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:150)
> at scala.math.Ordering$$anon$4.compare(Ordering.scala:111)
> at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649)
> at java.util.PriorityQueue.siftUp(PriorityQueue.java:627)
> at java.util.PriorityQueue.offer(PriorityQueue.java:329)
> at java.util.PriorityQueue.add(PriorityQueue.java:306)
> at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:78)
> at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:31)
> at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
> at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
> ... 15 more
> 16/05/27 06:17:38 ERROR TaskResultGetter: Exception while getting task result
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
> at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
> at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:25)
> at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:19)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> at org.apache.spark.serializer.KryoSerializerInstance.deserialize(KryoSerializer.scala:312)
> at org.apache.spark.scheduler.DirectTaskResult.value(TaskResult.scala:87)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:66)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
> at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:159)
> at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:150)
> at scala.math.Ordering$$anon$4.compare(Ordering.scala:111)
> at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649)
> at java.util.PriorityQueue.siftUp(PriorityQueue.java:627)
> at java.util.PriorityQueue.offer(PriorityQueue.java:329)
> at java.util.PriorityQueue.add(PriorityQueue.java:306)
> at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:78)
> at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:31)
> at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
> at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
> ... 15 more
> 16/05/27 06:17:38 ERROR TaskResultGetter: Exception while getting task result
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
> at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
> at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:25)
> at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:19)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> at org.apache.spark.serializer.KryoSerializerInstance.deserialize(KryoSerializer.scala:312)
> at org.apache.spark.scheduler.DirectTaskResult.value(TaskResult.scala:87)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:66)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
> at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:159)
> at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:150)
> at scala.math.Ordering$$anon$4.compare(Ordering.scala:111)
> at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649)
> at java.util.PriorityQueue.siftUp(PriorityQueue.java:627)
> at java.util.PriorityQueue.offer(PriorityQueue.java:329)
> at java.util.PriorityQueue.add(PriorityQueue.java:306)
> at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:78)
> at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:31)
> at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
> at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
> ... 15 more
> 16/05/27 06:17:38 ERROR TaskResultGetter: Exception while getting task result
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
> at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
> at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:25)
> at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:19)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> at org.apache.spark.serializer.KryoSerializerInstance.deserialize(KryoSerializer.scala:312)
> at org.apache.spark.scheduler.DirectTaskResult.value(TaskResult.scala:87)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:66)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
> at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:159)
> at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:150)
> at scala.math.Ordering$$anon$4.compare(Ordering.scala:111)
> at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649)
> at java.util.PriorityQueue.siftUp(PriorityQueue.java:627)
> at java.util.PriorityQueue.offer(PriorityQueue.java:329)
> at java.util.PriorityQueue.add(PriorityQueue.java:306)
> at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:78)
> at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:31)
> at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
> at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
> ... 15 more
> 16/05/27 06:17:38 ERROR TaskResultGetter: Exception while getting task result
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
> at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
> at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:25)
> at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:19)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> at org.apache.spark.serializer.KryoSerializerInstance.deserialize(KryoSerializer.scala:312)
> at org.apache.spark.scheduler.DirectTaskResult.value(TaskResult.scala:87)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:66)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
> at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:159)
> at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:150)
> at scala.math.Ordering$$anon$4.compare(Ordering.scala:111)
> at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649)
> at java.util.PriorityQueue.siftUp(PriorityQueue.java:627)
> at java.util.PriorityQueue.offer(PriorityQueue.java:329)
> at java.util.PriorityQueue.add(PriorityQueue.java:306)
> at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:78)
> at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:31)
> at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
> at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
> ... 15 more
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 38.0 in stage 10.0 (TID 900, 192.168.30.72): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 40.0 in stage 10.0 (TID 902, 192.168.30.72): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 5.0 in stage 10.0 (TID 867, 192.168.30.79): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 32.0 in stage 10.0 (TID 894, 192.168.30.79): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 26.0 in stage 10.0 (TID 888, 192.168.30.79): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 11.0 in stage 10.0 (TID 873, 192.168.30.79): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 8.0 in stage 10.0 (TID 870, 192.168.30.79): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 2.0 in stage 10.0 (TID 864, 192.168.30.79): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 ERROR TaskResultGetter: Exception while getting task result
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
> at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
> at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:25)
> at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:19)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> at org.apache.spark.serializer.KryoSerializerInstance.deserialize(KryoSerializer.scala:312)
> at org.apache.spark.scheduler.DirectTaskResult.value(TaskResult.scala:87)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:66)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
> at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:159)
> at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:150)
> at scala.math.Ordering$$anon$4.compare(Ordering.scala:111)
> at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649)
> at java.util.PriorityQueue.siftUp(PriorityQueue.java:627)
> at java.util.PriorityQueue.offer(PriorityQueue.java:329)
> at java.util.PriorityQueue.add(PriorityQueue.java:306)
> at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:78)
> at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:31)
> at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
> at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
> ... 15 more
> 16/05/27 06:17:38 ERROR TaskResultGetter: Exception while getting task result
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
> at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
> at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:25)
> at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:19)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> at org.apache.spark.serializer.KryoSerializerInstance.deserialize(KryoSerializer.scala:312)
> at org.apache.spark.scheduler.DirectTaskResult.value(TaskResult.scala:87)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:66)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
> at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793)
> at org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:159)
> at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:150)
> at scala.math.Ordering$$anon$4.compare(Ordering.scala:111)
> at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649)
> at java.util.PriorityQueue.siftUp(PriorityQueue.java:627)
> at java.util.PriorityQueue.offer(PriorityQueue.java:329)
> at java.util.PriorityQueue.add(PriorityQueue.java:306)
> at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:78)
> at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:31)
> at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
> at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
> ... 15 more
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 29.0 in stage 10.0 (TID 891, 192.168.30.79): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 37.0 in stage 10.0 (TID 899, 192.168.30.72): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 42.0 in stage 10.0 (TID 904, 192.168.30.72): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 17.0 in stage 10.0 (TID 879, 192.168.30.79): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 22.0 in stage 10.0 (TID 885, 192.168.30.79): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 20.0 in stage 10.0 (TID 882, 192.168.30.79): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 45.0 in stage 10.0 (TID 907, 192.168.30.72): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 35.0 in stage 10.0 (TID 897, 192.168.30.79): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 39.0 in stage 10.0 (TID 901, 192.168.30.72): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 43.0 in stage 10.0 (TID 905, 192.168.30.72): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 14.0 in stage 10.0 (TID 876, 192.168.30.79): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 41.0 in stage 10.0 (TID 903, 192.168.30.72): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 44.0 in stage 10.0 (TID 906, 192.168.30.72): TaskKilled (killed intentionally)
> {code}
> {code}
> query_1.sql
> with customer_total_return as
> (select sr_customer_sk as ctr_customer_sk
> ,sr_store_sk as ctr_store_sk
> ,sum(SR_FEE) as ctr_total_return
> from store_returns
> ,date_dim
> where sr_returned_date_sk = d_date_sk
> and d_year =2000
> group by sr_customer_sk
> ,sr_store_sk)
> select c_customer_id
> from customer_total_return ctr1
> ,store
> ,customer
> where ctr1.ctr_total_return > (select avg(ctr_total_return)*1.2
> from customer_total_return ctr2
> where ctr1.ctr_store_sk = ctr2.ctr_store_sk)
> and s_store_sk = ctr1.ctr_store_sk
> and s_state = 'SD'
> and ctr1.ctr_customer_sk = c_customer_sk
> order by c_customer_id
> limit 100;
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org