You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Alan Myrvold (JIRA)" <ji...@apache.org> on 2019/01/28 23:14:00 UTC
[jira] [Commented] (BEAM-6528) Dataflow jobs failing in multiple
PostCommits, including Java PostCommit.
[ https://issues.apache.org/jira/browse/BEAM-6528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16754435#comment-16754435 ]
Alan Myrvold commented on BEAM-6528:
------------------------------------
Environmental.
> Dataflow jobs failing in multiple PostCommits, including Java PostCommit.
> -------------------------------------------------------------------------
>
> Key: BEAM-6528
> URL: https://issues.apache.org/jira/browse/BEAM-6528
> Project: Beam
> Issue Type: Bug
> Components: runner-dataflow, test-failures
> Reporter: Daniel Oliveira
> Assignee: Alan Myrvold
> Priority: Critical
> Labels: currently-failing, flake
>
> * [Jenkins Job|https://builds.apache.org/job/beam_PostCommit_Python_Verify/7219/]
> * [Gradle Build Scan|https://scans.gradle.com/s/cdjxkpm7dgtxm]
> Initial investigation:
> It seems this test has failed four times just today with flakes in various integration tests. I've linked to the first instance of this failure above.
> In the Dataflow dashboard I see this error:
> {noformat}
> Startup of the worker pool in zone us-central1-f failed to bring up any of the desired 1 workers. ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS: The zone 'projects/apache-beam-testing/zones/us-central1-f' does not have enough resources available to fulfill the request. '(resource type:pd-standard)'.{noformat}
> Possibly some quota issue?
>
> Edit:
> I'm updating this bug because it seems like this isn't restricted to beam_PostCommit_Python_Verify after all. I've now seen this happening in various java postcommits:
> [beam_PostCommit_Java|https://builds.apache.org/job/beam_PostCommit_Java/2466/]
> [beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow|https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow/617/]
> [beam_PostCommit_Java_ValidatesRunner_Dataflow|https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/1959/]
> In each case the error looks something like this:
> {noformat}
> java.lang.RuntimeException: Startup of the worker pool in zone us-central1-f failed to bring up any of the desired 1 workers. ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS: The zone 'projects/apache-beam-testing/zones/us-central1-f' does not have enough resources available to fulfill the request. '(resource type:pd-standard)'.Workflow failed.{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)