[Spark SQL Continue] Sorry, it is not only limited in SQL, may due to network

[Trident <cw...@vip.qq.com>]

Dear Community,

       Please ignore my last post about Spark SQL.

       When I run:
val file = sc.textFile("./README.md")
val count = file.flatMap(line => line.split(" ")).map(word => (word, 1)).reduceByKey(_+_)
count.collect()
‍
        it happends too.

        is there any possible reason for that? we make have some adjustment in network last night

                                                                                             Chen Weikeng
14/10/09 20:45:23 ERROR Executor: Exception in task ID 1 java.lang.NullPointerException 	at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:571) 	at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:571) 	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35) 	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262) 	at org.apache.spark.rdd.RDD.iterator(RDD.scala:229) 	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35) 	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262) 	at org.apache.spark.rdd.RDD.iterator(RDD.scala:229) 	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35) 	at org.apache.spark.sql.SchemaRDD.compute(SchemaRDD.scala:116) 	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262) 	at org.apache.spark.rdd.RDD.iterator(RDD.scala:229) 	at org.apache.spark.rdd.MappedRDD.compute(MappedRDD.scala:31) 	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262) 	at org.apache.spark.rdd.RDD.iterator(RDD.scala:229) 	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:111) 	at org.apache.spark.scheduler.Task.run(Task.scala:51) 	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:183) 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 	at java.lang.Thread.run(Thread.java:745) 14/10/09 20:45:23 INFO CoarseGrainedExecutorBackend: Got assigned task 2 14/10/09 20:45:23 INFO Executor: Running task ID 2 14/10/09 20:45:23 DEBUG BlockManager: Getting local block broadcast_0 14/10/09 20:45:23 DEBUG BlockManager: Level for block broadcast_0 is StorageLevel(true, true, false, true, 1) 14/10/09 20:45:23 DEBUG BlockManager: Getting block broadcast_0 from memory 14/10/09 20:45:23 INFO BlockManager: Found block broadcast_0 locally 14/10/09 20:45:23 DEBUG Executor: Task 2's epoch is 0 14/10/09 20:45:23 INFO HadoopRDD: Input split: file:/public/rdma14/app/spark-rdma/examples/src/main/resources/people.txt:16+16 14/10/09 20:45:23 ERROR Executor: Exception in task ID 2 java.lang.NullPointerException 	at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:571) 	at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:571) 	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35) 	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262) 	at org.apache.spark.rdd.RDD.iterator(RDD.scala:229) 	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35) 	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262) 	at org.apache.spark.rdd.RDD.iterator(RDD.scala:229) 	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35) 	at org.apache.spark.sql.SchemaRDD.compute(SchemaRDD.scala:116) 	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262) 	at org.apache.spark.rdd.RDD.iterator(RDD.scala:229) 	at org.apache.spark.rdd.MappedRDD.compute(MappedRDD.scala:31) 	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262) 	at org.apache.spark.rdd.RDD.iterator(RDD.scala:229) 	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:111) 	at org.apache.spark.scheduler.Task.run(Task.scala:51) 	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:183) 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 	at java.lang.Thread.run(Thread.java:745)‍