You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Andrew Or (JIRA)" <ji...@apache.org> on 2014/11/26 00:52:14 UTC
[jira] [Closed] (SPARK-4471) blockManagerIdFromJson function
throws exception while BlockManagerId be null in
MetadataFetchFailedException
[ https://issues.apache.org/jira/browse/SPARK-4471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Or closed SPARK-4471.
----------------------------
Resolution: Fixed
Fix Version/s: 1.2.0
Target Version/s: 1.2.0
> blockManagerIdFromJson function throws exception while BlockManagerId be null in MetadataFetchFailedException
> --------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-4471
> URL: https://issues.apache.org/jira/browse/SPARK-4471
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.1.0, 1.1.1
> Reporter: SuYan
> Assignee: SuYan
> Priority: Minor
> Fix For: 1.2.0
>
>
> 1. when throws MetadataFetchFailedException, it will create a FetchFailedException, which BlockManagerId is null.
> FetchFailedException(null, shuffleId, -1, reduceId)
> 2. JsonProtocol.TaskEndReasonToJson, mark BlockManagerId as JNothing
> 3. while use JsonProtocol.TaskEndReasonFromJson, it didn't take JNothing into account, so BlockManagerIdFromJson will throw exception
> error log belows:
> org.json4s.package$MappingException: Did not find value which can be converted into java.lang.String
> at org.json4s.reflect.package$.fail(package.scala:96)
> at org.json4s.Extraction$.convert(Extraction.scala:554)
> at org.json4s.Extraction$.extract(Extraction.scala:331)
> at org.json4s.Extraction$.extract(Extraction.scala:42)
> at org.json4s.ExtractableJsonAstNode.extract(ExtractableJsonAstNode.scala:21)
> at org.apache.spark.util.JsonProtocol$.blockManagerIdFromJson(JsonProtocol.scala:662)
> at org.apache.spark.util.JsonProtocol$.taskEndReasonFromJson(JsonProtocol.scala:643)
> at org.apache.spark.util.JsonProtocol$.taskEndFromJson(JsonProtocol.scala:450)
> at org.apache.spark.util.JsonProtocol$.sparkEventFromJson(JsonProtocol.scala:410)
> at org.apache.spark.scheduler.ReplayListenerBus$$anonfun$replay$2$$anonfun$apply$1.apply(ReplayListenerBus.scala:71)
> at org.apache.spark.scheduler.ReplayListenerBus$$anonfun$replay$2$$anonfun$apply$1.apply(ReplayListenerBus.scala:69)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org