[jira] [Created] (SPARK-4471) blockManagerIdFromJson function throws exception while BlockManagerId be null in MetadataFetchFailedException

["SuYan (JIRA)" <ji...@apache.org>]

SuYan created SPARK-4471:
----------------------------

             Summary:  blockManagerIdFromJson function throws exception while BlockManagerId be null in MetadataFetchFailedException
                 Key: SPARK-4471
                 URL: https://issues.apache.org/jira/browse/SPARK-4471
             Project: Spark
          Issue Type: Bug
          Components: Spark Core
    Affects Versions: 1.1.0, 1.1.1
            Reporter: SuYan
            Priority: Minor


1. when throws MetadataFetchFailedException, it will create a FetchFailedException, which BlockManagerId is null.
  FetchFailedException(null, shuffleId, -1, reduceId) 

2. JsonProtocol.TaskEndReasonToJson, mark BlockManagerId as JNothing

3. while use JsonProtocol.TaskEndReasonFromJson, it didn't take  JNothing into account, so BlockManagerIdFromJson will throw exception

error log belows:

org.json4s.package$MappingException: Did not find value which can be converted into java.lang.String
        at org.json4s.reflect.package$.fail(package.scala:96)
        at org.json4s.Extraction$.convert(Extraction.scala:554)
        at org.json4s.Extraction$.extract(Extraction.scala:331)
        at org.json4s.Extraction$.extract(Extraction.scala:42)
        at org.json4s.ExtractableJsonAstNode.extract(ExtractableJsonAstNode.scala:21)
        at org.apache.spark.util.JsonProtocol$.blockManagerIdFromJson(JsonProtocol.scala:662)
        at org.apache.spark.util.JsonProtocol$.taskEndReasonFromJson(JsonProtocol.scala:643)
        at org.apache.spark.util.JsonProtocol$.taskEndFromJson(JsonProtocol.scala:450)
        at org.apache.spark.util.JsonProtocol$.sparkEventFromJson(JsonProtocol.scala:410)
        at org.apache.spark.scheduler.ReplayListenerBus$$anonfun$replay$2$$anonfun$apply$1.apply(ReplayListenerBus.scala:71)
        at org.apache.spark.scheduler.ReplayListenerBus$$anonfun$replay$2$$anonfun$apply$1.apply(ReplayListenerBus.scala:69)




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org