You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@reef.apache.org by Markus Weimer <ma...@weimo.de> on 2017/04/18 18:14:51 UTC
[Discuss] How to deal with `ITask.Dispose()` / `Task.close()` exceptions
Hi,
I noticed an imprecision in how we deal with exceptions in
`Task.close()` in Java or `ITask.Dispose()`:
* REEF.NET today crashes the Evaluator when such an exception
occurs, but after a result or other exception is sent to the Driver.
* REEF/Java sends such a exception to the Driver, but potentially
after also sending a result or prior exception to the Driver. This
probably leads to Driver failure.
How do we want to deal with this? My gut feeling is that throwing
exceptions in `.Dispose()` or `.Close()` should constitute a Task
failure. However, when I change REEF.NET that way, many of our own
tests fail.
WDYT?
Markus
Re: [Discuss] How to deal with `ITask.Dispose()` / `Task.close()` exceptions
Posted by Douglas Service <ds...@gmail.com>.
Since the you get the completed task it would make the most sense to give a
FailedEvaluator in both REEF.NET and REEF/Java.
On Wed, Apr 19, 2017 at 8:34 AM, Markus Weimer <ma...@weimo.de> wrote:
> On Tue, Apr 18, 2017 at 12:57 PM, Douglas Service <ds...@gmail.com>
> wrote:
> > From an application programmers perspective using REEF, I would want to
> get
> > the result back if possible (maybe it is a long running task) and be
> > informed that something is going wrong disposing of resources after
> running
> > a task.
>
> That is a valid point, and the current behavior of both our Java and
> C# code. They both give you a completed task, followed by a failure.
> REEF.NET gives you a FailedEvaluator, REEF/Java gives you a
> FailedTask.
>
> Which of those would you prefer? Or is there another sequence of
> events that makes more sense?
>
> Markus
>
Re: [Discuss] How to deal with `ITask.Dispose()` / `Task.close()` exceptions
Posted by Markus Weimer <ma...@weimo.de>.
On Tue, Apr 18, 2017 at 12:57 PM, Douglas Service <ds...@gmail.com> wrote:
> From an application programmers perspective using REEF, I would want to get
> the result back if possible (maybe it is a long running task) and be
> informed that something is going wrong disposing of resources after running
> a task.
That is a valid point, and the current behavior of both our Java and
C# code. They both give you a completed task, followed by a failure.
REEF.NET gives you a FailedEvaluator, REEF/Java gives you a
FailedTask.
Which of those would you prefer? Or is there another sequence of
events that makes more sense?
Markus
Re: [Discuss] How to deal with `ITask.Dispose()` / `Task.close()` exceptions
Posted by Douglas Service <ds...@gmail.com>.
Are you differentiating between a coding error disposing of a resource and
an abnormal resource disposal failure?
From an application programmers perspective using REEF, I would want to get
the result back if possible (maybe it is a long running task) and be
informed that something is going wrong disposing of resources after running
a task.
On Tue, Apr 18, 2017 at 12:49 PM, Markus Weimer <ma...@weimo.de> wrote:
> > I remember the original thinking was if dispose throws exception,
> meaning the resource was not cleared most possibly, therefore we should
> crash the Evaluator.
>
> That makes sense and justifies the current implementation in REEF.NET.
> In that current implementation, we return a result and crash the
> Evaluator. Is that the desired behavior?
>
> > In fault tolerant scenarios, we need to know if an evaluator is still
> re-usable in recovery case.
>
> An alternative would be to consider this something the application
> Driver has to deal with. If we treated an exception from `.Dispose`
> and `.close` as a Task failure, the Driver would receive a
> `FailedTask`. The exception within would allow it to make a
> determination whether or not the Evaluator is still usable.
>
> Markus
>
Re: [Discuss] How to deal with `ITask.Dispose()` / `Task.close()` exceptions
Posted by Markus Weimer <ma...@weimo.de>.
> I remember the original thinking was if dispose throws exception, meaning the resource was not cleared most possibly, therefore we should crash the Evaluator.
That makes sense and justifies the current implementation in REEF.NET.
In that current implementation, we return a result and crash the
Evaluator. Is that the desired behavior?
> In fault tolerant scenarios, we need to know if an evaluator is still re-usable in recovery case.
An alternative would be to consider this something the application
Driver has to deal with. If we treated an exception from `.Dispose`
and `.close` as a Task failure, the Driver would receive a
`FailedTask`. The exception within would allow it to make a
determination whether or not the Evaluator is still usable.
Markus
RE: [Discuss] How to deal with `ITask.Dispose()` / `Task.close()`
exceptions
Posted by "Julia Wang (QIUHE)" <Qi...@microsoft.com.INVALID>.
In fault tolerant scenarios, we need to know if an evaluator is still re-usable in recovery case.
-----Original Message-----
From: Julia Wang (QIUHE) [mailto:Qiuhe.Wang@microsoft.com.INVALID]
Sent: Tuesday, April 18, 2017 11:34 AM
To: dev@reef.apache.org
Subject: RE: [Discuss] How to deal with `ITask.Dispose()` / `Task.close()` exceptions
I remember the original thinking was if dispose throws exception, meaning the resource was not cleared most possibly, therefore we should crash the Evaluator.
-----Original Message-----
From: Markus Weimer [mailto:markus@weimo.de]
Sent: Tuesday, April 18, 2017 11:31 AM
To: REEF Developers Mailinglist <de...@reef.apache.org>
Subject: Re: [Discuss] How to deal with `ITask.Dispose()` / `Task.close()` exceptions
Before we jump to solutions, let's first decide on the right semantics:
Should Exceptions thrown in `ITask.Dispose()` and `Task.close()` constitute a `TaskFailure`? That is: Even if the `Call` / `call` method returned without failure, we still declare the Task a failure?
Markus
On Tue, Apr 18, 2017 at 11:24 AM, Julia Wang (QIUHE) <Qi...@microsoft.com.invalid> wrote:
> Looks like we shall call SetException(e) in the catch block of the Dispose() instead of throw InvalidOperationException. That makes more sense than crashing the evaluator. Is this what you tried? We can look at tests failures after the change. We have some test cases that throw exceptions inside Dispose. If behavior is changed, the expectation would also need to be modified.
>
> Julia
>
> -----Original Message-----
> From: Markus Weimer [mailto:markus@weimo.de]
> Sent: Tuesday, April 18, 2017 11:15 AM
> To: REEF Developers Mailinglist <de...@reef.apache.org>
> Subject: [Discuss] How to deal with `ITask.Dispose()` / `Task.close()`
> exceptions
>
> Hi,
>
> I noticed an imprecision in how we deal with exceptions in `Task.close()` in Java or `ITask.Dispose()`:
>
> * REEF.NET today crashes the Evaluator when such an exception occurs, but after a result or other exception is sent to the Driver.
>
> * REEF/Java sends such a exception to the Driver, but potentially after also sending a result or prior exception to the Driver. This probably leads to Driver failure.
>
> How do we want to deal with this? My gut feeling is that throwing exceptions in `.Dispose()` or `.Close()` should constitute a Task failure. However, when I change REEF.NET that way, many of our own tests fail.
>
> WDYT?
>
> Markus
RE: [Discuss] How to deal with `ITask.Dispose()` / `Task.close()`
exceptions
Posted by "Julia Wang (QIUHE)" <Qi...@microsoft.com.INVALID>.
I remember the original thinking was if dispose throws exception, meaning the resource was not cleared most possibly, therefore we should crash the Evaluator.
-----Original Message-----
From: Markus Weimer [mailto:markus@weimo.de]
Sent: Tuesday, April 18, 2017 11:31 AM
To: REEF Developers Mailinglist <de...@reef.apache.org>
Subject: Re: [Discuss] How to deal with `ITask.Dispose()` / `Task.close()` exceptions
Before we jump to solutions, let's first decide on the right semantics:
Should Exceptions thrown in `ITask.Dispose()` and `Task.close()` constitute a `TaskFailure`? That is: Even if the `Call` / `call` method returned without failure, we still declare the Task a failure?
Markus
On Tue, Apr 18, 2017 at 11:24 AM, Julia Wang (QIUHE) <Qi...@microsoft.com.invalid> wrote:
> Looks like we shall call SetException(e) in the catch block of the Dispose() instead of throw InvalidOperationException. That makes more sense than crashing the evaluator. Is this what you tried? We can look at tests failures after the change. We have some test cases that throw exceptions inside Dispose. If behavior is changed, the expectation would also need to be modified.
>
> Julia
>
> -----Original Message-----
> From: Markus Weimer [mailto:markus@weimo.de]
> Sent: Tuesday, April 18, 2017 11:15 AM
> To: REEF Developers Mailinglist <de...@reef.apache.org>
> Subject: [Discuss] How to deal with `ITask.Dispose()` / `Task.close()`
> exceptions
>
> Hi,
>
> I noticed an imprecision in how we deal with exceptions in `Task.close()` in Java or `ITask.Dispose()`:
>
> * REEF.NET today crashes the Evaluator when such an exception occurs, but after a result or other exception is sent to the Driver.
>
> * REEF/Java sends such a exception to the Driver, but potentially after also sending a result or prior exception to the Driver. This probably leads to Driver failure.
>
> How do we want to deal with this? My gut feeling is that throwing exceptions in `.Dispose()` or `.Close()` should constitute a Task failure. However, when I change REEF.NET that way, many of our own tests fail.
>
> WDYT?
>
> Markus
Re: [Discuss] How to deal with `ITask.Dispose()` / `Task.close()` exceptions
Posted by Markus Weimer <ma...@weimo.de>.
Before we jump to solutions, let's first decide on the right semantics:
Should Exceptions thrown in `ITask.Dispose()` and `Task.close()`
constitute a `TaskFailure`? That is: Even if the `Call` / `call`
method returned without failure, we still declare the Task a failure?
Markus
On Tue, Apr 18, 2017 at 11:24 AM, Julia Wang (QIUHE)
<Qi...@microsoft.com.invalid> wrote:
> Looks like we shall call SetException(e) in the catch block of the Dispose() instead of throw InvalidOperationException. That makes more sense than crashing the evaluator. Is this what you tried? We can look at tests failures after the change. We have some test cases that throw exceptions inside Dispose. If behavior is changed, the expectation would also need to be modified.
>
> Julia
>
> -----Original Message-----
> From: Markus Weimer [mailto:markus@weimo.de]
> Sent: Tuesday, April 18, 2017 11:15 AM
> To: REEF Developers Mailinglist <de...@reef.apache.org>
> Subject: [Discuss] How to deal with `ITask.Dispose()` / `Task.close()` exceptions
>
> Hi,
>
> I noticed an imprecision in how we deal with exceptions in `Task.close()` in Java or `ITask.Dispose()`:
>
> * REEF.NET today crashes the Evaluator when such an exception occurs, but after a result or other exception is sent to the Driver.
>
> * REEF/Java sends such a exception to the Driver, but potentially after also sending a result or prior exception to the Driver. This probably leads to Driver failure.
>
> How do we want to deal with this? My gut feeling is that throwing exceptions in `.Dispose()` or `.Close()` should constitute a Task failure. However, when I change REEF.NET that way, many of our own tests fail.
>
> WDYT?
>
> Markus
RE: [Discuss] How to deal with `ITask.Dispose()` / `Task.close()`
exceptions
Posted by "Julia Wang (QIUHE)" <Qi...@microsoft.com.INVALID>.
Looks like we shall call SetException(e) in the catch block of the Dispose() instead of throw InvalidOperationException. That makes more sense than crashing the evaluator. Is this what you tried? We can look at tests failures after the change. We have some test cases that throw exceptions inside Dispose. If behavior is changed, the expectation would also need to be modified.
Julia
-----Original Message-----
From: Markus Weimer [mailto:markus@weimo.de]
Sent: Tuesday, April 18, 2017 11:15 AM
To: REEF Developers Mailinglist <de...@reef.apache.org>
Subject: [Discuss] How to deal with `ITask.Dispose()` / `Task.close()` exceptions
Hi,
I noticed an imprecision in how we deal with exceptions in `Task.close()` in Java or `ITask.Dispose()`:
* REEF.NET today crashes the Evaluator when such an exception occurs, but after a result or other exception is sent to the Driver.
* REEF/Java sends such a exception to the Driver, but potentially after also sending a result or prior exception to the Driver. This probably leads to Driver failure.
How do we want to deal with this? My gut feeling is that throwing exceptions in `.Dispose()` or `.Close()` should constitute a Task failure. However, when I change REEF.NET that way, many of our own tests fail.
WDYT?
Markus