You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Etienne Chauchot <ec...@apache.org> on 2018/03/22 08:49:57 UTC
Build breaks on examples on jenkins with dataflow runner
Hi all,
java PreCommit test fails on jenkins on the examples module (woundCountIT). It gives incorrect signal on the build of
PRs.
It seems to be related to communication issues with dataflow service
org.apache.beam.examples.WindowedWordCountIT.testWindowedWordCountInBatchStaticSharding or org.apache.beam.examples.Wind
owedWordCountIT.testWindowedWordCountInBatchDynamicSharding
A work item was attempted 4 times without success. Each time
the worker eventually lost contact with the service. The work item was attempted on:
testpipeline-jenkins-0321-03210922-9f05-harness-qxtj,
testpipeline-jenkins-0321-03210922-9f05-harness-98n1,
testpipeline-jenkins-0321-03210922-9f05-harness-47mf,
testpipeline-jenkins-0321-03210922-9f05-harness-n1vb
org.apache.beam.examples.WordCountIT.testE2EWordCount
java.lang.RuntimeException: Workflow failed. Causes: The Dataflow appears to be stuck. You can get help with Cloud Dataflow at https://cloud.google.com/dataflow/support.
at org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:134)
at org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:90)
at org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:55)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:311)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
at org.apache.beam.examples.WordCount.runWordCount(WordCount.java:185)
at org.apache.beam.examples.WordCountIT.testE2EWordCount(WordCountIT.java:70)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.apache.maven.surefire.junitcore.pc.Scheduler$1.run(Scheduler.java:410)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Does anyone has a clue?
Etienne
Re: Build breaks on examples on jenkins with dataflow runner
Posted by Etienne Chauchot <ec...@apache.org>.
Thanks
Yes, we changed the API. I think we need to rebuild dataflow worker like we did there (1) for example
[1] https://issues.apache.org/jira/browse/BEAM-2012
Etienne
Le mercredi 28 mars 2018 à 20:32 +0000, Alan Myrvold a écrit :
> Thomas logged this issue as https://issues.apache.org/jira/projects/BEAM/issues/BEAM-3964
>
> On Wed, Mar 28, 2018 at 10:09 AM Alan Myrvold <am...@google.com> wrote:
> > I looked at one failure, and saw this error in the log. Could it be related to https://github.com/apache/beam/pull/4
> > 918 ?
> >
> >
> > 2018-03-28 09:45:26.603 PDTThread 25 died.
> > Expand all | Collapse all
> > {
> > insertId: "7765450222281849523:822129:0:57106"
> > jsonPayload: {
> > exception: "java.lang.NoSuchMethodError: org.apache.beam.sdk.metrics.MetricName.name()Ljava/lang/String;
> > at
> > com.google.cloud.dataflow.worker.MetricsToCounterUpdateConverter.structuredNameAndMetadata(MetricsToCounterUpdateCon
> > verter.java:99)
> > at
> > com.google.cloud.dataflow.worker.MetricsToCounterUpdateConverter.fromCounter(MetricsToCounterUpdateConverter.java:68
> > )
> > at com.google.cloud.dataflow.worker.BatchModeExecutionContext.lambda$null$1(BatchModeExecutionContext.java:463)
> > at
> > com.google.cloud.dataflow.worker.repackaged.com.google.common.collect.Iterators$7.transform(Iterators.java:750)
> > at
> > com.google.cloud.dataflow.worker.repackaged.com.google.common.collect.TransformedIterator.next(TransformedIterator.j
> > ava:47)
> > at
> > com.google.cloud.dataflow.worker.repackaged.com.google.common.collect.MultitransformedIterator.next(Multitransformed
> > Iterator.java:66)
> > at
> > com.google.cloud.dataflow.worker.repackaged.com.google.common.collect.ImmutableCollection$Builder.addAll(ImmutableCo
> > llection.java:388)
> > at
> > com.google.cloud.dataflow.worker.repackaged.com.google.common.collect.ImmutableCollection$ArrayBasedBuilder.addAll(I
> > mmutableCollection.java:472)
> > at
> > com.google.cloud.dataflow.worker.repackaged.com.google.common.collect.ImmutableList$Builder.addAll(ImmutableList.jav
> > a:669)
> > at com.google.cloud.dataflow.worker.WorkItemStatusClient.populateCounterUpdates(WorkItemStatusClient.java:256)
> > at com.google.cloud.dataflow.worker.WorkItemStatusClient.createStatusUpdate(WorkItemStatusClient.java:240)
> > at com.google.cloud.dataflow.worker.WorkItemStatusClient.reportError(WorkItemStatusClient.java:94)
> > at com.google.cloud.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:358)
> > at com.google.cloud.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:284)
> > at
> > com.google.cloud.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:134)
> > at
> > com.google.cloud.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:114)
> > at
> > com.google.cloud.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:101)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > at java.lang.Thread.run(Thread.java:745)
> >
> >
> >
> > On Wed, Mar 28, 2018 at 6:32 AM Etienne Chauchot <ec...@apache.org> wrote:
> > > Hi all,
> > > Please know that I have submitted a PR to skip IT tests from de PreCommit java tests (1). They still run in
> > > PostCommit.
> > > [1] https://github.com/apache/beam/pull/4967
> > >
> > > Etienne
> > >
> > > Le jeudi 22 mars 2018 à 11:54 +0100, Etienne Chauchot a écrit :
> > > > Also, WDYT about running these tests as PostCommit instead of preCommit as they are integration tests?
> > > >
> > > > Etienne
> > > >
> > > > Le jeudi 22 mars 2018 à 09:49 +0100, Etienne Chauchot a écrit :
> > > > > Hi all,
> > > > > java PreCommit test fails on jenkins on the examples module (woundCountIT). It gives incorrect signal on the
> > > > > build of PRs.
> > > > > It seems to be related to communication issues with dataflow service
> > > > >
> > > > > org.apache.beam.examples.WindowedWordCountIT.testWindowedWordCountInBatchStaticSharding
> > > > > or org.apache.beam.examples.WindowedWordCountIT.testWindowedWordCountInBatchDynamicSharding
> > > > > A work item was attempted 4 times without success. Each time the worker eventually lost contact with the
> > > > > service. The work item was attempted on:
> > > > > testpipeline-jenkins-0321-03210922-9f05-harness-qxtj,
> > > > > testpipeline-jenkins-0321-03210922-9f05-harness-98n1,
> > > > > testpipeline-jenkins-0321-03210922-9f05-harness-47mf,
> > > > > testpipeline-jenkins-0321-03210922-9f05-harness-n1vb
> > > > >
> > > > >
> > > > > org.apache.beam.examples.WordCountIT.testE2EWordCount
> > > > > java.lang.RuntimeException: Workflow failed. Causes: The Dataflow appears to be stuck. You can get help with
> > > > > Cloud Dataflow at https://cloud.google.com/dataflow/support.
> > > > > at org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:134)
> > > > > at org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:90)
> > > > > at org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:55)
> > > > > at org.apache.beam.sdk.Pipeline.run(Pipeline.java:311)
> > > > > at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
> > > > > at org.apache.beam.examples.WordCount.runWordCount(WordCount.java:185)
> > > > > at org.apache.beam.examples.WordCountIT.testE2EWordCount(WordCountIT.java:70)
> > > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > > > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > > > > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > > > at java.lang.reflect.Method.invoke(Method.java:498)
> > > > > at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> > > > > at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> > > > > at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> > > > > at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> > > > > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> > > > > at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> > > > > at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> > > > > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> > > > > at org.apache.maven.surefire.junitcore.pc.Scheduler$1.run(Scheduler.java:410)
> > > > > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > > > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > > > > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > > at java.lang.Thread.run(Thread.java:748)
> > > > >
> > > > > Does anyone has a clue?
> > > > >
> > > > > Etienne
Re: Build breaks on examples on jenkins with dataflow runner
Posted by Alan Myrvold <am...@google.com>.
Thomas logged this issue as
https://issues.apache.org/jira/projects/BEAM/issues/BEAM-3964
On Wed, Mar 28, 2018 at 10:09 AM Alan Myrvold <am...@google.com> wrote:
> I looked at one failure, and saw this error in the log. Could it be
> related to https://github.com/apache/beam/pull/4918 ?
>
>
> 2018-03-28 09:45:26.603 PDTThread 25 died.
> Expand all | Collapse all
> {
> insertId: "7765450222281849523:822129:0:57106"
> jsonPayload: {
> exception: "java.lang.NoSuchMethodError:
> org.apache.beam.sdk.metrics.MetricName.name()Ljava/lang/String; at
> com.google.cloud.dataflow.worker.MetricsToCounterUpdateConverter.structuredNameAndMetadata(MetricsToCounterUpdateConverter.java:99)
> at
> com.google.cloud.dataflow.worker.MetricsToCounterUpdateConverter.fromCounter(MetricsToCounterUpdateConverter.java:68)
> at
> com.google.cloud.dataflow.worker.BatchModeExecutionContext.lambda$null$1(BatchModeExecutionContext.java:463)
> at
> com.google.cloud.dataflow.worker.repackaged.com.google.common.collect.Iterators$7.transform(Iterators.java:750)
> at
> com.google.cloud.dataflow.worker.repackaged.com.google.common.collect.TransformedIterator.next(TransformedIterator.java:47)
> at
> com.google.cloud.dataflow.worker.repackaged.com.google.common.collect.MultitransformedIterator.next(MultitransformedIterator.java:66)
> at
> com.google.cloud.dataflow.worker.repackaged.com.google.common.collect.ImmutableCollection$Builder.addAll(ImmutableCollection.java:388)
> at
> com.google.cloud.dataflow.worker.repackaged.com.google.common.collect.ImmutableCollection$ArrayBasedBuilder.addAll(ImmutableCollection.java:472)
> at
> com.google.cloud.dataflow.worker.repackaged.com.google.common.collect.ImmutableList$Builder.addAll(ImmutableList.java:669)
> at
> com.google.cloud.dataflow.worker.WorkItemStatusClient.populateCounterUpdates(WorkItemStatusClient.java:256)
> at
> com.google.cloud.dataflow.worker.WorkItemStatusClient.createStatusUpdate(WorkItemStatusClient.java:240)
> at
> com.google.cloud.dataflow.worker.WorkItemStatusClient.reportError(WorkItemStatusClient.java:94)
> at
> com.google.cloud.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:358)
> at
> com.google.cloud.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:284)
> at
> com.google.cloud.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:134)
> at
> com.google.cloud.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:114)
> at
> com.google.cloud.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:101)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>
>
>
> On Wed, Mar 28, 2018 at 6:32 AM Etienne Chauchot <ec...@apache.org>
> wrote:
>
>> Hi all,
>> Please know that I have submitted a PR to skip IT tests from de PreCommit
>> java tests (1). They still run in PostCommit.
>> [1] https://github.com/apache/beam/pull/4967
>>
>> Etienne
>>
>> Le jeudi 22 mars 2018 à 11:54 +0100, Etienne Chauchot a écrit :
>>
>> Also, WDYT about running these tests as PostCommit instead of preCommit
>> as they are integration tests?
>>
>> Etienne
>>
>> Le jeudi 22 mars 2018 à 09:49 +0100, Etienne Chauchot a écrit :
>>
>> Hi all,
>> java PreCommit test fails on jenkins on the examples module
>> (woundCountIT). It gives incorrect signal on the build of PRs.
>> It seems to be related to communication issues with dataflow service
>>
>> org.apache.beam.examples.WindowedWordCountIT.testWindowedWordCountInBatchStaticSharding
>> or
>> org.apache.beam.examples.WindowedWordCountIT.testWindowedWordCountInBatchDynamicSharding
>>
>> A work item was attempted 4 times without success. Each time the worker eventually lost contact with the service. The work item was attempted on:
>> testpipeline-jenkins-0321-03210922-9f05-harness-qxtj,
>> testpipeline-jenkins-0321-03210922-9f05-harness-98n1,
>> testpipeline-jenkins-0321-03210922-9f05-harness-47mf,
>> testpipeline-jenkins-0321-03210922-9f05-harness-n1vb
>>
>>
>>
>> org.apache.beam.examples.WordCountIT.testE2EWordCount
>>
>> java.lang.RuntimeException: Workflow failed. Causes: The Dataflow appears to be stuck. You can get help with Cloud Dataflow at https://cloud.google.com/dataflow/support.
>> at org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:134)
>> at org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:90)
>> at org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:55)
>> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:311)
>> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
>> at org.apache.beam.examples.WordCount.runWordCount(WordCount.java:185)
>> at org.apache.beam.examples.WordCountIT.testE2EWordCount(WordCountIT.java:70)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>> at org.apache.maven.surefire.junitcore.pc.Scheduler$1.run(Scheduler.java:410)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> at java.lang.Thread.run(Thread.java:748)
>>
>>
>> Does anyone has a clue?
>>
>> Etienne
>>
>>
Re: Build breaks on examples on jenkins with dataflow runner
Posted by Alan Myrvold <am...@google.com>.
I looked at one failure, and saw this error in the log. Could it be related
to https://github.com/apache/beam/pull/4918 ?
2018-03-28 09:45:26.603 PDTThread 25 died.
Expand all | Collapse all
{
insertId: "7765450222281849523:822129:0:57106"
jsonPayload: {
exception: "java.lang.NoSuchMethodError:
org.apache.beam.sdk.metrics.MetricName.name()Ljava/lang/String; at
com.google.cloud.dataflow.worker.MetricsToCounterUpdateConverter.structuredNameAndMetadata(MetricsToCounterUpdateConverter.java:99)
at
com.google.cloud.dataflow.worker.MetricsToCounterUpdateConverter.fromCounter(MetricsToCounterUpdateConverter.java:68)
at
com.google.cloud.dataflow.worker.BatchModeExecutionContext.lambda$null$1(BatchModeExecutionContext.java:463)
at
com.google.cloud.dataflow.worker.repackaged.com.google.common.collect.Iterators$7.transform(Iterators.java:750)
at
com.google.cloud.dataflow.worker.repackaged.com.google.common.collect.TransformedIterator.next(TransformedIterator.java:47)
at
com.google.cloud.dataflow.worker.repackaged.com.google.common.collect.MultitransformedIterator.next(MultitransformedIterator.java:66)
at
com.google.cloud.dataflow.worker.repackaged.com.google.common.collect.ImmutableCollection$Builder.addAll(ImmutableCollection.java:388)
at
com.google.cloud.dataflow.worker.repackaged.com.google.common.collect.ImmutableCollection$ArrayBasedBuilder.addAll(ImmutableCollection.java:472)
at
com.google.cloud.dataflow.worker.repackaged.com.google.common.collect.ImmutableList$Builder.addAll(ImmutableList.java:669)
at
com.google.cloud.dataflow.worker.WorkItemStatusClient.populateCounterUpdates(WorkItemStatusClient.java:256)
at
com.google.cloud.dataflow.worker.WorkItemStatusClient.createStatusUpdate(WorkItemStatusClient.java:240)
at
com.google.cloud.dataflow.worker.WorkItemStatusClient.reportError(WorkItemStatusClient.java:94)
at
com.google.cloud.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:358)
at
com.google.cloud.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:284)
at
com.google.cloud.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:134)
at
com.google.cloud.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:114)
at
com.google.cloud.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:101)
at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
On Wed, Mar 28, 2018 at 6:32 AM Etienne Chauchot <ec...@apache.org>
wrote:
> Hi all,
> Please know that I have submitted a PR to skip IT tests from de PreCommit
> java tests (1). They still run in PostCommit.
> [1] https://github.com/apache/beam/pull/4967
>
> Etienne
>
> Le jeudi 22 mars 2018 à 11:54 +0100, Etienne Chauchot a écrit :
>
> Also, WDYT about running these tests as PostCommit instead of preCommit as
> they are integration tests?
>
> Etienne
>
> Le jeudi 22 mars 2018 à 09:49 +0100, Etienne Chauchot a écrit :
>
> Hi all,
> java PreCommit test fails on jenkins on the examples module
> (woundCountIT). It gives incorrect signal on the build of PRs.
> It seems to be related to communication issues with dataflow service
>
> org.apache.beam.examples.WindowedWordCountIT.testWindowedWordCountInBatchStaticSharding
> or
> org.apache.beam.examples.WindowedWordCountIT.testWindowedWordCountInBatchDynamicSharding
>
> A work item was attempted 4 times without success. Each time the worker eventually lost contact with the service. The work item was attempted on:
> testpipeline-jenkins-0321-03210922-9f05-harness-qxtj,
> testpipeline-jenkins-0321-03210922-9f05-harness-98n1,
> testpipeline-jenkins-0321-03210922-9f05-harness-47mf,
> testpipeline-jenkins-0321-03210922-9f05-harness-n1vb
>
>
>
> org.apache.beam.examples.WordCountIT.testE2EWordCount
>
> java.lang.RuntimeException: Workflow failed. Causes: The Dataflow appears to be stuck. You can get help with Cloud Dataflow at https://cloud.google.com/dataflow/support.
> at org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:134)
> at org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:90)
> at org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:55)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:311)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
> at org.apache.beam.examples.WordCount.runWordCount(WordCount.java:185)
> at org.apache.beam.examples.WordCountIT.testE2EWordCount(WordCountIT.java:70)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.apache.maven.surefire.junitcore.pc.Scheduler$1.run(Scheduler.java:410)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>
>
> Does anyone has a clue?
>
> Etienne
>
>
Re: Build breaks on examples on jenkins with dataflow runner
Posted by Etienne Chauchot <ec...@apache.org>.
Hi all,
Please know that I have submitted a PR to skip IT tests from de PreCommit java tests (1). They still run in PostCommit.
[1] https://github.com/apache/beam/pull/4967
Etienne
Le jeudi 22 mars 2018 à 11:54 +0100, Etienne Chauchot a écrit :
> Also, WDYT about running these tests as PostCommit instead of preCommit as they are integration tests?
>
> Etienne
>
> Le jeudi 22 mars 2018 à 09:49 +0100, Etienne Chauchot a écrit :
> > Hi all,
> > java PreCommit test fails on jenkins on the examples module (woundCountIT). It gives incorrect signal on the build
> > of PRs.
> > It seems to be related to communication issues with dataflow service
> >
> > org.apache.beam.examples.WindowedWordCountIT.testWindowedWordCountInBatchStaticSharding
> > or org.apache.beam.examples.WindowedWordCountIT.testWindowedWordCountInBatchDynamicSharding
> > A work item was attempted 4 times without success. Each time the worker eventually lost contact with the service.
> > The work item was attempted on:
> > testpipeline-jenkins-0321-03210922-9f05-harness-qxtj,
> > testpipeline-jenkins-0321-03210922-9f05-harness-98n1,
> > testpipeline-jenkins-0321-03210922-9f05-harness-47mf,
> > testpipeline-jenkins-0321-03210922-9f05-harness-n1vb
> >
> >
> > org.apache.beam.examples.WordCountIT.testE2EWordCount
> > java.lang.RuntimeException: Workflow failed. Causes: The Dataflow appears to be stuck. You can get help with Cloud
> > Dataflow at https://cloud.google.com/dataflow/support.
> > at org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:134)
> > at org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:90)
> > at org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:55)
> > at org.apache.beam.sdk.Pipeline.run(Pipeline.java:311)
> > at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
> > at org.apache.beam.examples.WordCount.runWordCount(WordCount.java:185)
> > at org.apache.beam.examples.WordCountIT.testE2EWordCount(WordCountIT.java:70)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:498)
> > at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> > at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> > at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> > at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> > at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> > at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> > at org.apache.maven.surefire.junitcore.pc.Scheduler$1.run(Scheduler.java:410)
> > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > at java.lang.Thread.run(Thread.java:748)
> >
> > Does anyone has a clue?
> >
> > Etienne
Re: Build breaks on examples on jenkins with dataflow runner
Posted by Etienne Chauchot <ec...@apache.org>.
Also, WDYT about running these tests as PostCommit instead of preCommit as they are integration tests?
Etienne
Le jeudi 22 mars 2018 à 09:49 +0100, Etienne Chauchot a écrit :
> Hi all,
> java PreCommit test fails on jenkins on the examples module (woundCountIT). It gives incorrect signal on the build of
> PRs.
> It seems to be related to communication issues with dataflow service
>
> org.apache.beam.examples.WindowedWordCountIT.testWindowedWordCountInBatchStaticSharding
> or org.apache.beam.examples.WindowedWordCountIT.testWindowedWordCountInBatchDynamicSharding
> A work item was attempted 4 times without success. Each time the worker eventually lost contact with the service. The
> work item was attempted on:
> testpipeline-jenkins-0321-03210922-9f05-harness-qxtj,
> testpipeline-jenkins-0321-03210922-9f05-harness-98n1,
> testpipeline-jenkins-0321-03210922-9f05-harness-47mf,
> testpipeline-jenkins-0321-03210922-9f05-harness-n1vb
>
>
> org.apache.beam.examples.WordCountIT.testE2EWordCount
> java.lang.RuntimeException: Workflow failed. Causes: The Dataflow appears to be stuck. You can get help with Cloud
> Dataflow at https://cloud.google.com/dataflow/support.
> at org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:134)
> at org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:90)
> at org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:55)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:311)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
> at org.apache.beam.examples.WordCount.runWordCount(WordCount.java:185)
> at org.apache.beam.examples.WordCountIT.testE2EWordCount(WordCountIT.java:70)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.apache.maven.surefire.junitcore.pc.Scheduler$1.run(Scheduler.java:410)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>
> Does anyone has a clue?
>
> Etienne