You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Imran Rashid (JIRA)" <ji...@apache.org> on 2015/04/25 03:21:38 UTC
[jira] [Comment Edited] (SPARK-6980) Akka timeout exceptions
indicate which conf controls them
[ https://issues.apache.org/jira/browse/SPARK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512148#comment-14512148 ]
Imran Rashid edited comment on SPARK-6980 at 4/25/15 1:20 AM:
--------------------------------------------------------------
Hi [~bryanc] [~harshg],
sorry I didn't notice these comments until just now. I have a feeling I was originally thinking of something much simpler, I probably didn't explain it very well. My idea was that we change the signature of {{AkkaUtils.askWithReply}}, so that instead of taking a {{FiniteDuration}}, it would take some new {{NamedDuration}} (or something like that, I'm not sold on the name). It would basically just be a little wrapper around {{FiniteDuration}} and a name. Then {{SparkConf}} would have util methods to get {{NamedDuration}}. My goal for this specific issue is just to change the exception message -- so that it says something like "Future timed out after 30 seconds. You can increase the value of this timeout with spark.foo.bar.timeout"
For testing, I'd suggest you first just write unit tests that don't involve Spark jobs at all. Just create an actor system, two very simple Actors, have one actor use {{AkkaUtils.askWithReply}} to send a message to the other one with a modest timeout. Try with the second actor responding right away, and everything should be fine; try again with the second actor never replying, catch the exception and check the message.
Does this all make sense? Also open to any other ideas you have on how to do this, just giving my thoughts.
was (Author: irashid):
Hi [~bryanc] [~harshg],
sorry I didn't notice these comments until just now. I have a feeling I was originally thinking of something much simpler, I probably didn't explain it very well. My idea was that we change the signature of {{AkkaUtils.askWithReply}}, so that instead of taking a {{FiniteDuration}}, it would take some new {{NamedDuration}} (or something like that, I'm not sold on the name). It would basically just be a little wrapper around {{FiniteDuration}} and a name. Then {{SparkConf}} would have util methods to get {{NamedDuration}}. My goal for this specific issue is just to change the exception message -- so that it says something like "Future timed out after 30 seconds. You can increase the value of this timeout with spark.foo.bar.timeout"
For testing, I'd suggest you first just write unit tests that don't involve Spark jobs at all. Just create an actor system, two very simple Actors, have one actor use {{AkkaUtils.askWithReply}} to send a message to the other one with a modest timeout. Try with the second actor responding write away, and everything should be fine; try again with the second actor never replying, catch the exception and check the message.
Does this all make sense? Also open to any other ideas you have on how to do this, just giving my thoughts.
> Akka timeout exceptions indicate which conf controls them
> ---------------------------------------------------------
>
> Key: SPARK-6980
> URL: https://issues.apache.org/jira/browse/SPARK-6980
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Reporter: Imran Rashid
> Assignee: Harsh Gupta
> Priority: Minor
> Labels: starter
> Attachments: Spark-6980-Test.scala
>
>
> If you hit one of the akka timeouts, you just get an exception like
> {code}
> java.util.concurrent.TimeoutException: Futures timed out after [30 seconds]
> {code}
> The exception doesn't indicate how to change the timeout, though there is usually (always?) a corresponding setting in {{SparkConf}} . It would be nice if the exception including the relevant setting.
> I think this should be pretty easy to do -- we just need to create something like a {{NamedTimeout}}. It would have its own {{await}} method, catches the akka timeout and throws its own exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org