You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Ismaël Mejía (Jira)" <ji...@apache.org> on 2020/01/03 00:27:00 UTC
[jira] [Resolved] (BEAM-8938) Tests end up leaving stale Dataflow
jobs in apache-beam-testing project and exhaust GCP resources
[ https://issues.apache.org/jira/browse/BEAM-8938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ismaël Mejía resolved BEAM-8938.
--------------------------------
Fix Version/s: Not applicable
Resolution: Not A Problem
Java part is not failing anymore
Python part covered as part of BEAM-8877
> Tests end up leaving stale Dataflow jobs in apache-beam-testing project and exhaust GCP resources
> -------------------------------------------------------------------------------------------------
>
> Key: BEAM-8938
> URL: https://issues.apache.org/jira/browse/BEAM-8938
> Project: Beam
> Issue Type: Bug
> Components: testing
> Reporter: Lukasz Gajowy
> Assignee: Kamil Wasilewski
> Priority: Blocker
> Fix For: Not applicable
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Some tests (I'm not sure if this is the exhaustive list but they seem to appear in the dataflow console repeatedly) that seem to not be killed and eat our resources:
> - [test_reshuffle_preserves_timestamps|https://github.com/apache/beam/blob/719b8cc5e51dcd3e98425ecae5ec246657d46eca/sdks/python/apache_beam/transforms/util_test.py#L487] (spotted multiple times in the dataflow console) (Python SDK)
> - [test_flatten_same_pcollections|https://github.com/apache/beam/blob/44d456830442e5f13b7fd3bd684695e2b69e2c0d/sdks/python/apache_beam/transforms/ptransform_test.py#L596] (Python SDK)
> - [testPairWithIndexWindowedTimestampedBounded|https://github.com/apache/beam/blob/c2f0d282337f3ae0196a7717712396a5a41fdde1/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/SplittableDoFnTest.java#L158] (Java SDK)
> - [testPairWithIndexBasicBounded|https://github.com/apache/beam/blob/c2f0d282337f3ae0196a7717712396a5a41fdde1/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/SplittableDoFnTest.java#L125] (Java SDK)
>
> Temporary solution is to ignore them. Real solution requires greater investigation.
> Please see the devlist thread for more context: [https://lists.apache.org/thread.html/01eb33ae9c05d12bb0698f91adc0021662fdfe2978cfdfde28dc56b2%40%3Cdev.beam.apache.org%3E]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)