[jira] [Commented] (BEAM-5529) Dataflow runner raises AssertionError if job takes > 50 seconds to go from PENDING to RUNNING

["Ahmet Altay (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634786#comment-16634786 ] 

Ahmet Altay commented on BEAM-5529:
-----------------------------------

Thank you for reporting. Sent out #6535 to fix this.

> Dataflow runner raises AssertionError if job takes > 50 seconds to go from PENDING to RUNNING
> ---------------------------------------------------------------------------------------------
>
>                 Key: BEAM-5529
>                 URL: https://issues.apache.org/jira/browse/BEAM-5529
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-py-core
>    Affects Versions: 2.6.0
>            Reporter: Joar Wandborg
>            Assignee: Ahmet Altay
>            Priority: Major
>          Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> To reproduce:
>  - Try to run a Pipeline that takes longer than 50 seconds to go from JOB_STATE_PENDING to JOB_STATE_RUNNING using "with Pipeline() as pipeline:" (automatically executes {{pipeline.run().wait_until_finish()}} via {{Pipeline.__exit__}}) on Dataflow.
>  - Your script will raise "AssertionError: Job did not reach to a terminal state after waiting indefinitely." (from [https://github.com/apache/beam/blob/766a1dc5b6523a0b7a3ce0a9ab78901cf17c9013/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py#L1152-L1153])
> I think the issue is that the block at [https://github.com/apache/beam/blob/766a1dc5b6523a0b7a3ce0a9ab78901cf17c9013/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py#L147-L160] unintentionally exits if it takes > 50 seconds for a job to go from JOB_STATE_PENDING to JOB_STATE_RUNNING.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)