[jira] [Commented] (HADOOP-13716) Add an Eventually class to await eventual completion of lambda expressions

["Steve Loughran (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-13716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15569723#comment-15569723 ] 

Steve Loughran commented on HADOOP-13716:
-----------------------------------------

example from a test
{code}
    int reps = eventually(
        () -> ++count == 4,
        50,
        () -> 10,
        (timeout, ex) -> new Exception("timeout after " + count)
    );
{code}
This will evaluate the first closure until it succeeds or the total wait time exceeds 50; the operation for interval calculation is always 10. Returns the number of iterations it took.

Another example, evaluate a closure until it succeeds. Here it doesn't; the e
{code}
  @Test
  public void testEvalLambdaFailures() throws Throwable {
    try {
      evaluate(() -> {
            throw new IOException("oops");
          },
          TIMEOUT,
          retry);
      fail("should not have got here");
    } catch (IOException expected) {
      assertExceptionContains("oops", expected);
    }
  }
{code}

As well as being Lambda friendly, the patch is just invoking {{Callable<T>}}, so works with Java & and normal interface/implementation classes; that's how I plan to initially use it for branch-2 code. I just want to be confident we have something which will move to lambda-expressions with ease.

I'm looking at doing some S3A work, in particular handling inconsistency problems in some of the FS contract root tests, though I think someone ought to be able to migrate {{waitFor()}} code to it when they want better exception propagation, failure handling or alternative backoff strategies

> Add an Eventually class to await eventual completion of lambda expressions
> --------------------------------------------------------------------------
>
>                 Key: HADOOP-13716
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13716
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: test
>    Affects Versions: 2.8.0
>            Reporter: Steve Loughran
>
> To make our tests robust against timing problems and eventual consistent stores, we need to do more spin & wait for state.
> We have some code in {{GenericTestUtils.waitFor}} to await a condition being met, but the predicate it calls doesn't throw exceptions, there's no way for a probe to throw an exception, and all you get is the eventual "timed out" message. 
> We can do better, and in closure-ready languages (scala & scalatest, groovy and some slider code) we've examples to follow. Some of that work has been reimplemented slightly in {{S3ATestUtils.eventually}}
> I propose adding a class in the test tree, {{Eventually}} to be a successor/replacement for these.
> # has an eventually/waitfor operation taking a predicate that throws an exception
> # has an "evaluate" exception which tries to evaluate an answer until the operation stops raising an exception. (again, from scalatest)
> # plugin backoff strategies (from Scalatest; lets you do exponential as well as linear)
> # option of adding a special handler to generate the failure exception (e.g. run more detailed diagnostics for the exception text, etc).
> # be Java 8 lambda expression friendly
> # be testable and tested itself.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org