PAssertTest#runExpectingAssertionFailure() and waitUntilFinish()

[Stas Levin <st...@gmail.com>]

Hi,

PAssertTest#runExpectingAssertionFailure() contains the following block:

try {
  pipeline.run();
} catch (AssertionError exc) {
  return exc;
}

I was wondering if in some cases this might not produce the desired effect,
particularly if the run() method returns before the pipeline has ended.
While some runners may choose to inherently wait for the pipeline to end in
their run() method (e.g., DirectRunner if options.isBlockOnRun() is set to
true), this seems to be an optional behavior that may be absent in other
runners.

Is this indeed an issue (in which case the above will benefit from calling
waitUntilFinish)?

Regards,
Stas

Re: PAssertTest#runExpectingAssertionFailure() and waitUntilFinish()

[Kenneth Knowles <kl...@google.com.INVALID>]

Hi Stas,

This is something special to TestPipeline and the test configuration for a
runner.

If runExpectingAssertionFailure() does not succeed, then our whole suite of
RunnableOnService tests is not going to work, because they all have an
assumption that TestPipeline#run() waits until the assertions are known to
all succeed or one of them fails. This can be different than waiting until
the pipeline itself terminates.

Today, unfortunately, every runner is responsible for making that work in
their own way, thus we have TestDataflowRunner, TestFlinkRunner,
TestApexRunner, and TestSparkRunner.

Eventually the logic should be centralized in TestPipeline, and
TestPipeline become a JUnit @Rule. These issues are roughly covered by
[BEAM-85] and [BEAM-298]

Kenn

[BEAM-85] https://issues.apache.org/jira/browse/BEAM-85
[BEAM-298] https://issues.apache.org/jira/browse/BEAM-298

On Mon, Dec 5, 2016 at 3:30 AM, Stas Levin <st...@gmail.com> wrote:

> Hi,
>
> PAssertTest#runExpectingAssertionFailure() contains the following block:
>
> try {
>   pipeline.run();
> } catch (AssertionError exc) {
>   return exc;
> }
>
> I was wondering if in some cases this might not produce the desired effect,
> particularly if the run() method returns before the pipeline has ended.
> While some runners may choose to inherently wait for the pipeline to end in
> their run() method (e.g., DirectRunner if options.isBlockOnRun() is set to
> true), this seems to be an optional behavior that may be absent in other
> runners.
>
> Is this indeed an issue (in which case the above will benefit from calling
> waitUntilFinish)?
>
> Regards,
> Stas
>