[jira] [Updated] (BEAM-3261) Apex runner does not detect pipeline failure

["Kenneth Knowles (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-3261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kenneth Knowles updated BEAM-3261:
----------------------------------
    Resolution: Fixed
        Status: Resolved  (was: Resolved)

Hello! Due to a bug in our Jira configuration, this issue had status:Resolved but resolution:Unresolved.

I am bulk editing these issues to have resolution:Fixed

If a different resolution is appropriate, please change it. To do this, click the "Resolve" button (you can do this even for closed issues) and set the Resolution field to the right value.

> Apex runner does not detect pipeline failure
> --------------------------------------------
>
>                 Key: BEAM-3261
>                 URL: https://issues.apache.org/jira/browse/BEAM-3261
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-apex
>            Reporter: Eugene Kirpichov
>            Priority: P1
>          Time Spent: 1h
>  Remaining Estimate: 0h
>
> I was looking at https://github.com/apache/beam/pull/4074/files and asked myself "Don't we already have a ValidatesRunner test for this?"
> Turns out we do: https://github.com/apache/beam/blob/3b79b6298e84711528b5ad1302200cb8acbac07e/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/ViewTest.java#L793
> I ran this test with TestApexRunner, and I observed the same exception as in https://stackoverflow.com/questions/46982827/error-when-using-side-input-withsideinputs-method-not-accepting-kv-type-as-inp?noredirect=1#comment81040223_46982827 , however the test passed.
> It seems that it is passing because ApexRunnerResult.waitUntilFinish() detects only PAssert assertion errors, but not any other errors: https://github.com/apache/beam/blob/3b79b6298e84711528b5ad1302200cb8acbac07e/runners/apex/src/main/java/org/apache/beam/runners/apex/ApexRunnerResult.java#L62
> This seems very problematic, as it means that 1) the status of Apex ValidatesRunner tests cannot be trusted: if they fail in any other way other than a PAssert error, the failure will be undetected, and 2) in general, a user can not trust a successful pipeline.run() from Apex runner.
> For tests in particular, some other TestXRunner's guard against such a failure mode by verifying not only that there were no assertion failures, but also that all assertions succeeded - using metrics: e.g. https://github.com/apache/beam/blob/3b79b6298e84711528b5ad1302200cb8acbac07e/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/TestDataflowRunner.java#L248 . Fixing this would be optimal, but meanwhile, the runner should at least fail the pipeline in case of error.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)