You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2018/10/01 23:56:00 UTC
[jira] [Work logged] (BEAM-5529) Dataflow runner raises
AssertionError if job takes > 50 seconds to go from PENDING to RUNNING
[ https://issues.apache.org/jira/browse/BEAM-5529?focusedWorklogId=150268&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-150268 ]
ASF GitHub Bot logged work on BEAM-5529:
----------------------------------------
Author: ASF GitHub Bot
Created on: 01/Oct/18 23:55
Start Date: 01/Oct/18 23:55
Worklog Time Spent: 10m
Work Description: aaltay opened a new pull request #6535: [BEAM-5529] Before starting the final countdown, check that job is not in pre-running state
URL: https://github.com/apache/beam/pull/6535
Before starting the final countdown, check that job is not in pre-running state
------------------------
Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
It will help us expedite review of your Pull Request if you tag someone (e.g. `@username`) to look at it.
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/) | --- | --- | --- | --- | --- | ---
Java | [](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/) </br> [](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/) | --- | --- | ---
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 150268)
Time Spent: 10m
Remaining Estimate: 0h
> Dataflow runner raises AssertionError if job takes > 50 seconds to go from PENDING to RUNNING
> ---------------------------------------------------------------------------------------------
>
> Key: BEAM-5529
> URL: https://issues.apache.org/jira/browse/BEAM-5529
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Affects Versions: 2.6.0
> Reporter: Joar Wandborg
> Assignee: Ahmet Altay
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> To reproduce:
> - Try to run a Pipeline that takes longer than 50 seconds to go from JOB_STATE_PENDING to JOB_STATE_RUNNING using "with Pipeline() as pipeline:" (automatically executes {{pipeline.run().wait_until_finish()}} via {{Pipeline.__exit__}}) on Dataflow.
> - Your script will raise "AssertionError: Job did not reach to a terminal state after waiting indefinitely." (from [https://github.com/apache/beam/blob/766a1dc5b6523a0b7a3ce0a9ab78901cf17c9013/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py#L1152-L1153])
> I think the issue is that the block at [https://github.com/apache/beam/blob/766a1dc5b6523a0b7a3ce0a9ab78901cf17c9013/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py#L147-L160] unintentionally exits if it takes > 50 seconds for a job to go from JOB_STATE_PENDING to JOB_STATE_RUNNING.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)