[jira] [Commented] (BEAM-3261) Apex runner does not detect pipeline failure

["Tyson Hamilton (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-3261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192994#comment-17192994 ] 

Tyson Hamilton commented on BEAM-3261:
--------------------------------------

Apex runner has been removed.

> Apex runner does not detect pipeline failure
> --------------------------------------------
>
>                 Key: BEAM-3261
>                 URL: https://issues.apache.org/jira/browse/BEAM-3261
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-apex
>            Reporter: Eugene Kirpichov
>            Priority: P1
>          Time Spent: 1h
>  Remaining Estimate: 0h
>
> I was looking at https://github.com/apache/beam/pull/4074/files and asked myself "Don't we already have a ValidatesRunner test for this?"
> Turns out we do: https://github.com/apache/beam/blob/3b79b6298e84711528b5ad1302200cb8acbac07e/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/ViewTest.java#L793
> I ran this test with TestApexRunner, and I observed the same exception as in https://stackoverflow.com/questions/46982827/error-when-using-side-input-withsideinputs-method-not-accepting-kv-type-as-inp?noredirect=1#comment81040223_46982827 , however the test passed.
> It seems that it is passing because ApexRunnerResult.waitUntilFinish() detects only PAssert assertion errors, but not any other errors: https://github.com/apache/beam/blob/3b79b6298e84711528b5ad1302200cb8acbac07e/runners/apex/src/main/java/org/apache/beam/runners/apex/ApexRunnerResult.java#L62
> This seems very problematic, as it means that 1) the status of Apex ValidatesRunner tests cannot be trusted: if they fail in any other way other than a PAssert error, the failure will be undetected, and 2) in general, a user can not trust a successful pipeline.run() from Apex runner.
> For tests in particular, some other TestXRunner's guard against such a failure mode by verifying not only that there were no assertion failures, but also that all assertions succeeded - using metrics: e.g. https://github.com/apache/beam/blob/3b79b6298e84711528b5ad1302200cb8acbac07e/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/TestDataflowRunner.java#L248 . Fixing this would be optimal, but meanwhile, the runner should at least fail the pipeline in case of error.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)