You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Lukasz Gajowy (Jira)" <ji...@apache.org> on 2019/12/19 16:02:00 UTC
[jira] [Assigned] (BEAM-8938) Tests end up leaving stale Dataflow
jobs in apache-beam-testing project and exhaust GCP resources
[ https://issues.apache.org/jira/browse/BEAM-8938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lukasz Gajowy reassigned BEAM-8938:
-----------------------------------
Assignee: Kamil Wasilewski
> Tests end up leaving stale Dataflow jobs in apache-beam-testing project and exhaust GCP resources
> -------------------------------------------------------------------------------------------------
>
> Key: BEAM-8938
> URL: https://issues.apache.org/jira/browse/BEAM-8938
> Project: Beam
> Issue Type: Bug
> Components: testing
> Reporter: Lukasz Gajowy
> Assignee: Kamil Wasilewski
> Priority: Blocker
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Some tests (I'm not sure if this is the exhaustive list but they seem to appear in the dataflow console repeatedly) that seem to not be killed and eat our resources:
> - [test_reshuffle_preserves_timestamps|https://github.com/apache/beam/blob/719b8cc5e51dcd3e98425ecae5ec246657d46eca/sdks/python/apache_beam/transforms/util_test.py#L487] (spotted multiple times in the dataflow console) (Python SDK)
> - [test_flatten_same_pcollections|https://github.com/apache/beam/blob/44d456830442e5f13b7fd3bd684695e2b69e2c0d/sdks/python/apache_beam/transforms/ptransform_test.py#L596] (Python SDK)
> - [testPairWithIndexWindowedTimestampedBounded|https://github.com/apache/beam/blob/c2f0d282337f3ae0196a7717712396a5a41fdde1/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/SplittableDoFnTest.java#L158] (Java SDK)
> - [testPairWithIndexBasicBounded|https://github.com/apache/beam/blob/c2f0d282337f3ae0196a7717712396a5a41fdde1/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/SplittableDoFnTest.java#L125] (Java SDK)
>
> Temporary solution is to ignore them. Real solution requires greater investigation.
> Please see the devlist thread for more context: [https://lists.apache.org/thread.html/01eb33ae9c05d12bb0698f91adc0021662fdfe2978cfdfde28dc56b2%40%3Cdev.beam.apache.org%3E]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)