You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Alex Amato <aj...@google.com> on 2021/09/15 01:42:22 UTC
beam_LoadTests_Go_Combine_Flink_Batch failing possibly leaking vms.
Details here:
https://issues.apache.org/jira/browse/BEAM-12898?filter=-2
I think that this test may also be responsible for leaking some gce vms on
apache-beam-testing. As this morning we discovered several vms that were
not torn down. Though I am not sure if there is a clear link.
01:43:59 2021/09/14 08:43:59 ():
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException:
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException:
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException:
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.IllegalArgumentException: Encountered unsupported logical type
URN: int
01:43:59 at
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2050)
01:43:59 at
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952)
01:43:59 at
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
01:43:59 at
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958)
01:43:59 at
org.apache.beam.runners.core.construction.RehydratedComponents.getCoder(RehydratedComponents.java:168)
01:43:59 at
org.apache.beam.runners.fnexecution.wire.WireCoders.instantiateRunnerWireCoder(WireCoders.java:94)
01:43:59 at
org.apache.beam.runners.fnexecution.wire.WireCoders.instantiateRunnerWireCoder(WireCoders.java:75)
01:43:59 at
org.apache.beam.runners.flink.FlinkBatchPortablePipelineTranslator.translateExecutableStage(FlinkBatchPortablePipelineTranslator.java:311)
01:43:59 at
org.apache.beam.runners.flink.FlinkBatchPortablePipelineTranslator.translate(FlinkBatchPortablePipelineTranslator.java:272)
01:43:59 at
org.apache.beam.runners.flink.FlinkBatchPortablePipelineTranslator.translate(FlinkBatchPortablePipelineTranslator.java:118)
01:43:59 at
org.apache.beam.runners.flink.FlinkPipelineRunner.runPipelineWithTranslator(FlinkPipelineRunner.java:115)
01:43:59 at
org.apache.beam.runners.flink.FlinkPipelineRunner.run(FlinkPipelineRunner.java:85)
01:43:59 at
org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
01:43:59 at
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
01:43:59 at
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
01:43:59 at
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
01:43:59 at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
01:43:59 at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
01:43:59 at java.lang.Thread.run(Thread.java:748)
Re: beam_LoadTests_Go_Combine_Flink_Batch failing possibly leaking vms.
Posted by Alex Amato <aj...@google.com>.
FWIW these tests were disabled. details here:
https://issues.apache.org/jira/browse/BEAM-12898
Is there anyone who can take ownership of these Flink tests?
Test either must be debugged before turning back on
Or one could implement a solution to cleanup GCP resources in beam testing
projects (https://issues.apache.org/jira/browse/BEAM-12903)
On Tue, Sep 14, 2021 at 6:42 PM Alex Amato <aj...@google.com> wrote:
> Details here:
> https://issues.apache.org/jira/browse/BEAM-12898?filter=-2
>
> I think that this test may also be responsible for leaking some gce vms on
> apache-beam-testing. As this morning we discovered several vms that were
> not torn down. Though I am not sure if there is a clear link.
>
>
> 01:43:59 2021/09/14 08:43:59 ():
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException:
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException:
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException:
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException:
> java.lang.IllegalArgumentException: Encountered unsupported logical type
> URN: int
> 01:43:59 at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2050)
> 01:43:59 at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952)
> 01:43:59 at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
> 01:43:59 at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958)
> 01:43:59 at
> org.apache.beam.runners.core.construction.RehydratedComponents.getCoder(RehydratedComponents.java:168)
> 01:43:59 at
> org.apache.beam.runners.fnexecution.wire.WireCoders.instantiateRunnerWireCoder(WireCoders.java:94)
> 01:43:59 at
> org.apache.beam.runners.fnexecution.wire.WireCoders.instantiateRunnerWireCoder(WireCoders.java:75)
> 01:43:59 at
> org.apache.beam.runners.flink.FlinkBatchPortablePipelineTranslator.translateExecutableStage(FlinkBatchPortablePipelineTranslator.java:311)
> 01:43:59 at
> org.apache.beam.runners.flink.FlinkBatchPortablePipelineTranslator.translate(FlinkBatchPortablePipelineTranslator.java:272)
> 01:43:59 at
> org.apache.beam.runners.flink.FlinkBatchPortablePipelineTranslator.translate(FlinkBatchPortablePipelineTranslator.java:118)
> 01:43:59 at
> org.apache.beam.runners.flink.FlinkPipelineRunner.runPipelineWithTranslator(FlinkPipelineRunner.java:115)
> 01:43:59 at
> org.apache.beam.runners.flink.FlinkPipelineRunner.run(FlinkPipelineRunner.java:85)
> 01:43:59 at
> org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
> 01:43:59 at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
> 01:43:59 at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
> 01:43:59 at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
> 01:43:59 at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> 01:43:59 at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> 01:43:59 at java.lang.Thread.run(Thread.java:748)
>