You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/06/19 18:18:08 UTC
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Flink #3
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/3/display/redirect>
------------------------------------------
[...truncated 3.33 MB...]
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState
INFO: CHAIN MapPartition (MapPartition at Query12/Query12.Stamp/ParMultiDo(Anonymous)) -> FlatMap (FlatMap at Query12/Query12.Stamp/ParMultiDo(Anonymous).output) (1/1) (4e2d22e9cef6f5bed31c9beea3427e7d) switched from RUNNING to FINISHED.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.taskmanager.Task transitionState
INFO: CHAIN MapPartition (MapPartition at Query12.Format/ParMultiDo(Anonymous)) -> FlatMap (FlatMap at Query12.Format/ParMultiDo(Anonymous).output) (1/1) (329acacf6c989b2b88f035c9b60ca392) switched from RUNNING to FINISHED.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.taskmanager.Task run
INFO: Freeing task resources for CHAIN MapPartition (MapPartition at Query12.Format/ParMultiDo(Anonymous)) -> FlatMap (FlatMap at Query12.Format/ParMultiDo(Anonymous).output) (1/1) (329acacf6c989b2b88f035c9b60ca392).
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.taskmanager.Task run
INFO: Ensuring all FileSystem streams are closed for task CHAIN MapPartition (MapPartition at Query12.Format/ParMultiDo(Anonymous)) -> FlatMap (FlatMap at Query12.Format/ParMultiDo(Anonymous).output) (1/1) (329acacf6c989b2b88f035c9b60ca392) [FINISHED]
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.taskmanager.Task transitionState
INFO: CHAIN MapPartition (MapPartition at Query12.DevNull/ParMultiDo(Anonymous)) -> FlatMap (FlatMap at Query12.DevNull/ParMultiDo(Anonymous).output) (1/1) (3a9c4bb7d97dce8fc457491630517d74) switched from RUNNING to FINISHED.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.taskmanager.Task run
INFO: Freeing task resources for CHAIN MapPartition (MapPartition at Query12.DevNull/ParMultiDo(Anonymous)) -> FlatMap (FlatMap at Query12.DevNull/ParMultiDo(Anonymous).output) (1/1) (3a9c4bb7d97dce8fc457491630517d74).
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState
INFO: DataSink (org.apache.flink.api.java.io.DiscardingOutputFormat@3ce067bd) (1/1) (4fdebb3dabee8e78d4af1fc205473968) switched from CREATED to SCHEDULED.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.taskmanager.Task run
INFO: Ensuring all FileSystem streams are closed for task CHAIN MapPartition (MapPartition at Query12.DevNull/ParMultiDo(Anonymous)) -> FlatMap (FlatMap at Query12.DevNull/ParMultiDo(Anonymous).output) (1/1) (3a9c4bb7d97dce8fc457491630517d74) [FINISHED]
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.taskexecutor.TaskExecutor unregisterTaskAndNotifyFinalState
INFO: Un-registering task and sending final execution state FINISHED to JobManager for task CHAIN MapPartition (MapPartition at Query12.Format/ParMultiDo(Anonymous)) -> FlatMap (FlatMap at Query12.Format/ParMultiDo(Anonymous).output) 329acacf6c989b2b88f035c9b60ca392.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState
INFO: DataSink (org.apache.flink.api.java.io.DiscardingOutputFormat@3ce067bd) (1/1) (4fdebb3dabee8e78d4af1fc205473968) switched from SCHEDULED to DEPLOYING.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.executiongraph.Execution deploy
INFO: Deploying DataSink (org.apache.flink.api.java.io.DiscardingOutputFormat@3ce067bd) (1/1) (attempt #0) to localhost
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.taskexecutor.TaskExecutor unregisterTaskAndNotifyFinalState
INFO: Un-registering task and sending final execution state FINISHED to JobManager for task CHAIN MapPartition (MapPartition at Query12.DevNull/ParMultiDo(Anonymous)) -> FlatMap (FlatMap at Query12.DevNull/ParMultiDo(Anonymous).output) 3a9c4bb7d97dce8fc457491630517d74.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState
INFO: CHAIN MapPartition (MapPartition at Query12.Format/ParMultiDo(Anonymous)) -> FlatMap (FlatMap at Query12.Format/ParMultiDo(Anonymous).output) (1/1) (329acacf6c989b2b88f035c9b60ca392) switched from RUNNING to FINISHED.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState
INFO: CHAIN MapPartition (MapPartition at Query12.DevNull/ParMultiDo(Anonymous)) -> FlatMap (FlatMap at Query12.DevNull/ParMultiDo(Anonymous).output) (1/1) (3a9c4bb7d97dce8fc457491630517d74) switched from RUNNING to FINISHED.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.taskexecutor.TaskExecutor submitTask
INFO: Received task DataSink (org.apache.flink.api.java.io.DiscardingOutputFormat@3ce067bd) (1/1).
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.taskmanager.Task transitionState
INFO: DataSink (org.apache.flink.api.java.io.DiscardingOutputFormat@3ce067bd) (1/1) (4fdebb3dabee8e78d4af1fc205473968) switched from CREATED to DEPLOYING.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.taskmanager.Task run
INFO: Creating FileSystem stream leak safety net for task DataSink (org.apache.flink.api.java.io.DiscardingOutputFormat@3ce067bd) (1/1) (4fdebb3dabee8e78d4af1fc205473968) [DEPLOYING]
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.taskmanager.Task run
INFO: Loading JAR files for task DataSink (org.apache.flink.api.java.io.DiscardingOutputFormat@3ce067bd) (1/1) (4fdebb3dabee8e78d4af1fc205473968) [DEPLOYING].
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.taskmanager.Task run
INFO: Registering task at network: DataSink (org.apache.flink.api.java.io.DiscardingOutputFormat@3ce067bd) (1/1) (4fdebb3dabee8e78d4af1fc205473968) [DEPLOYING].
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.taskmanager.Task transitionState
INFO: DataSink (org.apache.flink.api.java.io.DiscardingOutputFormat@3ce067bd) (1/1) (4fdebb3dabee8e78d4af1fc205473968) switched from DEPLOYING to RUNNING.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState
INFO: DataSink (org.apache.flink.api.java.io.DiscardingOutputFormat@3ce067bd) (1/1) (4fdebb3dabee8e78d4af1fc205473968) switched from DEPLOYING to RUNNING.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.taskmanager.Task transitionState
INFO: DataSink (org.apache.flink.api.java.io.DiscardingOutputFormat@3ce067bd) (1/1) (4fdebb3dabee8e78d4af1fc205473968) switched from RUNNING to FINISHED.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.taskmanager.Task run
INFO: Freeing task resources for DataSink (org.apache.flink.api.java.io.DiscardingOutputFormat@3ce067bd) (1/1) (4fdebb3dabee8e78d4af1fc205473968).
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.taskmanager.Task run
INFO: Ensuring all FileSystem streams are closed for task DataSink (org.apache.flink.api.java.io.DiscardingOutputFormat@3ce067bd) (1/1) (4fdebb3dabee8e78d4af1fc205473968) [FINISHED]
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.taskexecutor.TaskExecutor unregisterTaskAndNotifyFinalState
INFO: Un-registering task and sending final execution state FINISHED to JobManager for task DataSink (org.apache.flink.api.java.io.DiscardingOutputFormat@3ce067bd) 4fdebb3dabee8e78d4af1fc205473968.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState
INFO: DataSink (org.apache.flink.api.java.io.DiscardingOutputFormat@3ce067bd) (1/1) (4fdebb3dabee8e78d4af1fc205473968) switched from RUNNING to FINISHED.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.executiongraph.ExecutionGraph transitionState
INFO: Job main-jenkins-0619181647-9efedbe (d0e0b3138e3b9ae1f46483741f04da56) switched from state RUNNING to FINISHED.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.dispatcher.Dispatcher jobReachedGloballyTerminalState
INFO: Job d0e0b3138e3b9ae1f46483741f04da56 reached globally terminal state FINISHED.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.jobmaster.JobMaster postStop
INFO: Stopping the JobMaster for job main-jenkins-0619181647-9efedbe(d0e0b3138e3b9ae1f46483741f04da56).
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.taskexecutor.TaskExecutor closeJobManagerConnection
INFO: Close JobManager connection for job d0e0b3138e3b9ae1f46483741f04da56.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.jobmaster.JobMaster dissolveResourceManagerConnection
INFO: Close ResourceManager connection 5753988c3776c086ec6d3fb546afa270: JobManager is shutting down..
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.jobmaster.slotpool.SlotPool suspend
INFO: Suspending SlotPool.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.resourcemanager.ResourceManager closeJobManagerConnection
INFO: Disconnect job manager a124400c4315be29ea482efdeb1a4920@akka://flink/user/jobmanager_25 for job d0e0b3138e3b9ae1f46483741f04da56 from the resource manager.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.jobmaster.slotpool.SlotPool postStop
INFO: Stopping SlotPool.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.minicluster.MiniCluster closeAsync
INFO: Shutting down Flink Mini Cluster
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.taskexecutor.TaskExecutor$JobLeaderListenerImpl jobManagerLostLeadership
INFO: JobManager for job d0e0b3138e3b9ae1f46483741f04da56 with leader id a124400c4315be29ea482efdeb1a4920 lost leadership.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.rest.RestServerEndpoint closeAsync
INFO: Shutting down rest endpoint.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.dispatcher.Dispatcher postStop
INFO: Stopping dispatcher akka://flink/user/dispatcher462aee94-e784-48a7-bd69-82230dc8df35.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.dispatcher.Dispatcher terminateJobManagerRunners
INFO: Stopping all currently running jobs of dispatcher akka://flink/user/dispatcher462aee94-e784-48a7-bd69-82230dc8df35.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager close
INFO: Closing the SlotManager.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager suspend
INFO: Suspending the SlotManager.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager unregisterTaskManager
INFO: Unregister TaskManager c1e7b6d21c6f320ce89345634562bb1b from the SlotManager.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.taskexecutor.slot.TaskSlotTable freeSlot
INFO: Free slot TaskSlot(index:0, state:ALLOCATED, resource profile: ResourceProfile{cpuCores=1.0, heapMemoryInMB=42, directMemoryInMB=0, nativeMemoryInMB=0, networkMemoryInMB=0}, allocationId: a44996e0cf028a79b888587b0539f2d0, jobId: d0e0b3138e3b9ae1f46483741f04da56).
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.taskexecutor.JobLeaderService removeJob
INFO: Remove job d0e0b3138e3b9ae1f46483741f04da56 from job leader monitoring.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.taskexecutor.TaskExecutor closeJobManagerConnection
INFO: Close JobManager connection for job d0e0b3138e3b9ae1f46483741f04da56.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.taskexecutor.TaskExecutor postStop
INFO: Stopping TaskExecutor akka://flink/user/taskmanager_24.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager shutdown
INFO: Shutting down TaskExecutorLocalStateStoresManager.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.io.disk.iomanager.IOManager shutdown
INFO: I/O manager removed spill file directory /tmp/flink-io-85bbd478-4def-43bb-b297-475b8340cfd1
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.io.network.NetworkEnvironment shutdown
INFO: Shutting down the network environment and its components.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.rest.handler.legacy.backpressure.StackTraceSampleCoordinator shutDown
INFO: Shutting down stack trace sample coordinator.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.webmonitor.WebMonitorEndpoint lambda$shutDownInternal$4
INFO: Removing cache directory /tmp/flink-web-ui
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.taskexecutor.JobLeaderService stop
INFO: Stop job leader service.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.dispatcher.Dispatcher lambda$postStop$0
INFO: Stopped dispatcher akka://flink/user/dispatcher462aee94-e784-48a7-bd69-82230dc8df35.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.taskexecutor.TaskExecutor postStop
INFO: Stopped TaskExecutor akka://flink/user/taskmanager_24.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.rpc.akka.AkkaRpcService stopService
INFO: Stopping Akka RPC service.
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.blob.AbstractBlobCache close
INFO: Shutting down BLOB cache
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.blob.AbstractBlobCache close
INFO: Shutting down BLOB cache
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.blob.BlobServer close
INFO: Stopped BLOB server at 0.0.0.0:46867
Jun 19, 2018 6:18:03 PM org.apache.flink.runtime.rpc.akka.AkkaRpcService lambda$stopService$4
INFO: Stopped Akka RPC service.
Jun 19, 2018 6:18:03 PM org.apache.beam.runners.flink.FlinkRunner createPipelineResult
INFO: Execution finished in 5312 msecs
Jun 19, 2018 6:18:03 PM org.apache.beam.runners.flink.FlinkRunner createPipelineResult
INFO: Final accumulator values:
Jun 19, 2018 6:18:03 PM org.apache.beam.runners.flink.FlinkRunner createPipelineResult
INFO: __metricscontainers : org.apache.beam.runners.core.metrics.MetricsContainerStepMap@ea391e7d
2018-06-19T18:18:03.418Z Waiting for main pipeline to 'finish'
2018-06-19T18:18:03.418Z DONE Query12
==========================================================================================
Run started 2018-06-19T18:16:46.944Z and ran for PT76.474S
Default configuration:
{"debug":true,"query":0,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:0; exportSummaryToBigQuery:true; streamTimeout:60
0001 query:1; exportSummaryToBigQuery:true; streamTimeout:60
0002 query:2; exportSummaryToBigQuery:true; streamTimeout:60
0003 query:3; exportSummaryToBigQuery:true; streamTimeout:60
0004 query:4; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
0005 query:5; exportSummaryToBigQuery:true; streamTimeout:60
0006 query:6; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
0007 query:7; exportSummaryToBigQuery:true; streamTimeout:60
0008 query:8; exportSummaryToBigQuery:true; streamTimeout:60
0009 query:9; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
0010 query:10; exportSummaryToBigQuery:true; streamTimeout:60
0011 query:11; exportSummaryToBigQuery:true; streamTimeout:60
0012 query:12; exportSummaryToBigQuery:true; streamTimeout:60
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 0.7 153609.8 100000
0001 0.5 189753.3 92000
0002 0.4 251256.3 351
0003 10.9 9160.9 580
0004 5.3 1881.8 40
0005 5.2 19241.9 12
0006 4.7 2115.1 103
0007 6.0 16556.3 1
0008 7.8 12810.7 6000
0009 5.1 1960.0 298
0010 5.8 17199.9 1
0011 6.6 15101.2 1919
0012 5.3 18932.2 1919
==========================================================================================
==========================================================================================
Run started 2018-06-19T18:16:46.944Z and ran for PT76.785S
Default configuration:
{"debug":true,"query":0,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:0; exportSummaryToBigQuery:true; streamTimeout:60
0001 query:1; exportSummaryToBigQuery:true; streamTimeout:60
0002 query:2; exportSummaryToBigQuery:true; streamTimeout:60
0003 query:3; exportSummaryToBigQuery:true; streamTimeout:60
0004 query:4; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
0005 query:5; exportSummaryToBigQuery:true; streamTimeout:60
0006 query:6; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
0007 query:7; exportSummaryToBigQuery:true; streamTimeout:60
0008 query:8; exportSummaryToBigQuery:true; streamTimeout:60
0009 query:9; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
0010 query:10; exportSummaryToBigQuery:true; streamTimeout:60
0011 query:11; exportSummaryToBigQuery:true; streamTimeout:60
0012 query:12; exportSummaryToBigQuery:true; streamTimeout:60
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 0.7 153609.8 100000
0001 0.5 189753.3 92000
0002 0.4 251256.3 351
0003 10.9 9160.9 580
0004 5.3 1881.8 40
0005 5.2 19241.9 12
0006 4.7 2115.1 103
0007 6.0 16556.3 1
0008 7.8 12810.7 6000
0009 5.1 1960.0 298
0010 5.8 17199.9 1
0011 6.6 15101.2 1919
Exception in thread "main" 0012 5.3 18932.2 1919
==========================================================================================
java.lang.IllegalArgumentException: BigQueryIO.Write needs a GCS temp location to store temp files.
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)
at org.apache.beam.sdk.io.gcp.bigquery.BatchLoads.validate(BatchLoads.java:194)
at org.apache.beam.sdk.Pipeline$ValidateVisitor.enterCompositeTransform(Pipeline.java:640)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:656)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:660)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:660)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600(TransformHierarchy.java:311)
at org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:245)
at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:458)
at org.apache.beam.sdk.Pipeline.validate(Pipeline.java:575)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:310)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
at org.apache.beam.sdk.nexmark.Main.savePerfsToBigQuery(Main.java:182)
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:102)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:395)
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 13,5,main]) completed. Took 1 mins 20.026 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_152/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
See https://docs.gradle.org/4.8/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 44s
62 actionable tasks: 58 executed, 4 from cache
Publishing build scan...
https://gradle.com/s/wbib4nqxat3pq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
Not sending mail to unregistered user jiangkai@gmail.com
Jenkins build is back to normal :
beam_PostCommit_Java_Nexmark_Flink #13
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/13/display/redirect>
Jenkins build is back to normal :
beam_PostCommit_Java_Nexmark_Flink #12
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/12/display/redirect?page=changes>
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Flink #11
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/11/display/redirect>
------------------------------------------
[...truncated 3.82 MB...]
INFO: Shutting down BLOB cache
Jun 21, 2018 6:26:05 PM org.apache.flink.runtime.blob.AbstractBlobCache close
INFO: Shutting down BLOB cache
Jun 21, 2018 6:26:05 PM org.apache.flink.runtime.blob.BlobServer close
INFO: Stopped BLOB server at 0.0.0.0:35713
Jun 21, 2018 6:26:05 PM org.apache.flink.runtime.rpc.akka.AkkaRpcService lambda$stopService$4
INFO: Stopped Akka RPC service.
Jun 21, 2018 6:26:05 PM org.apache.beam.runners.flink.FlinkRunner run
SEVERE: Pipeline execution failed
org.apache.flink.runtime.client.JobExecutionException: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins06211823329c2bc4b5_a5602a2cefce41389146b6e3cba9073b_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:625)
at org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:234)
at org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:91)
at org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:114)
at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:116)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:311)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
at org.apache.beam.sdk.nexmark.Main.savePerfsToBigQuery(Main.java:182)
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:102)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:395)
Caused by: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins06211823329c2bc4b5_a5602a2cefce41389146b6e3cba9073b_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:36)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Unable to insert job: beam_load_mainjenkins06211823329c2bc4b5_a5602a2cefce41389146b6e3cba9073b_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:231)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Source URI must be a Google Cloud Storage location: <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/ws/src/sdks/java/nexmark/nexmark-temp/BigQueryWriteTemp/beam_load_mainjenkins06211823329c2bc4b5_a5602a2cefce41389146b6e3cba9073b/acc55f8b-58ae-4956-a1a2-5b544c70b0e3",>
"reason" : "invalid"
} ],
"message" : "Source URI must be a Google Cloud Storage location: <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/ws/src/sdks/java/nexmark/nexmark-temp/BigQueryWriteTemp/beam_load_mainjenkins06211823329c2bc4b5_a5602a2cefce41389146b6e3cba9073b/acc55f8b-58ae-4956-a1a2-5b544c70b0e3">
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1065)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:216)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
==========================================================================================
Run started 2018-06-21T18:23:31.110Z and ran for PT154.026S
Default configuration:
{"debug":true,"query":0,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Exception in thread "main" Configurations:
Conf Description
0000 query:0; exportSummaryToBigQuery:true; streamTimeout:60
0001 query:1; exportSummaryToBigQuery:true; streamTimeout:60
0002 query:2; exportSummaryToBigQuery:true; streamTimeout:60
0003 query:3; exportSummaryToBigQuery:true; streamTimeout:60
0004 query:4; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
0005 query:5; exportSummaryToBigQuery:true; streamTimeout:60
0006 query:6; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
0007 query:7; exportSummaryToBigQuery:true; streamTimeout:60
0008 query:8; exportSummaryToBigQuery:true; streamTimeout:60
0009 query:9; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
0010 query:10; exportSummaryToBigQuery:true; streamTimeout:60
0011 query:11; exportSummaryToBigQuery:true; streamTimeout:60
0012 query:12; exportSummaryToBigQuery:true; streamTimeout:60
java.lang.RuntimeException: Pipeline execution failed
Performance:
at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:119)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:311)
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
0000 0.7 141043.7 100000
at org.apache.beam.sdk.nexmark.Main.savePerfsToBigQuery(Main.java:182)
0001 0.4 228833.0 92000
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:102)
0002 0.4 273224.0 351
at org.apache.beam.sdk.nexmark.Main.main(Main.java:395)
0003 12.5 8009.0 580
0004 5.4 1849.8 40
Caused by: org.apache.flink.runtime.client.JobExecutionException: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins06211823329c2bc4b5_a5602a2cefce41389146b6e3cba9073b_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:625)
0005 4.8 20772.7 12
at org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:234)
0006 4.3 2301.5 103
at org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:91)
0007 6.2 16105.7 1
at org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:114)
0008 6.5 15448.8 6000
at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:116)
0009 4.9 2056.3 298
... 5 more
0010 5.4 18508.2 1
Caused by: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins06211823329c2bc4b5_a5602a2cefce41389146b6e3cba9073b_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:36)
0011 5.9 16852.0 1919
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
0012 4.8 20898.6 1919
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
==========================================================================================
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Unable to insert job: beam_load_mainjenkins06211823329c2bc4b5_a5602a2cefce41389146b6e3cba9073b_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:231)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Source URI must be a Google Cloud Storage location: <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/ws/src/sdks/java/nexmark/nexmark-temp/BigQueryWriteTemp/beam_load_mainjenkins06211823329c2bc4b5_a5602a2cefce41389146b6e3cba9073b/acc55f8b-58ae-4956-a1a2-5b544c70b0e3",>
"reason" : "invalid"
} ],
"message" : "Source URI must be a Google Cloud Storage location: <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/ws/src/sdks/java/nexmark/nexmark-temp/BigQueryWriteTemp/beam_load_mainjenkins06211823329c2bc4b5_a5602a2cefce41389146b6e3cba9073b/acc55f8b-58ae-4956-a1a2-5b544c70b0e3">
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1065)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:216)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 15,5,main]) completed. Took 2 mins 37.321 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_152/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
See https://docs.gradle.org/4.8/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4m 54s
62 actionable tasks: 58 executed, 4 from cache
Publishing build scan...
https://gradle.com/s/axpsozp7sxrhm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user migryz@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user marek.simunek@firma.seznam.cz
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user sourabhbajaj@google.com
Not sending mail to unregistered user mergebot@apache.org
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user thomas.haines@practiceinsight.io
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user ddonnelly@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user devon.meunier@shopify.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user cademarkegard@gmail.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user basti.lj@alibaba-inc.com
Not sending mail to unregistered user mrtn@nrd.io
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user azur@paypal.com
Not sending mail to unregistered user jamalone@gmail.com
Not sending mail to unregistered user terrencehan@tencent.com
Not sending mail to unregistered user hadarh@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user fjp@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user taro.murao@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user wnsdud1861@gmail.com
Not sending mail to unregistered user peihe0@gmail.com
Not sending mail to unregistered user sisk@google.com
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Flink #10
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/10/display/redirect?page=changes>
Changes:
[ankurgoenka] Fixing key lookup in FlinkBatchSideInputHandlerFactory.SideInputHandler
------------------------------------------
[...truncated 3.82 MB...]
INFO: Shutting down BLOB cache
Jun 21, 2018 6:02:06 PM org.apache.flink.runtime.blob.AbstractBlobCache close
INFO: Shutting down BLOB cache
Jun 21, 2018 6:02:06 PM org.apache.flink.runtime.blob.BlobServer close
INFO: Stopped BLOB server at 0.0.0.0:37281
Jun 21, 2018 6:02:06 PM org.apache.flink.runtime.rpc.akka.AkkaRpcService lambda$stopService$4
INFO: Stopped Akka RPC service.
Jun 21, 2018 6:02:06 PM org.apache.beam.runners.flink.FlinkRunner run
SEVERE: Pipeline execution failed
org.apache.flink.runtime.client.JobExecutionException: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins0621175931124187c0_25df23acc7034fd28b6d30c21f079ede_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:625)
at org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:234)
at org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:91)
at org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:114)
at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:116)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:311)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
at org.apache.beam.sdk.nexmark.Main.savePerfsToBigQuery(Main.java:182)
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:102)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:395)
Caused by: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins0621175931124187c0_25df23acc7034fd28b6d30c21f079ede_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:36)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Unable to insert job: beam_load_mainjenkins0621175931124187c0_25df23acc7034fd28b6d30c21f079ede_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:231)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Source URI must be a Google Cloud Storage location: <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/ws/src/sdks/java/nexmark/nexmark-temp/BigQueryWriteTemp/beam_load_mainjenkins0621175931124187c0_25df23acc7034fd28b6d30c21f079ede/5d6c9bfc-6c80-48d2-8727-6ce45697feab",>
"reason" : "invalid"
} ],
"message" : "Source URI must be a Google Cloud Storage location: <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/ws/src/sdks/java/nexmark/nexmark-temp/BigQueryWriteTemp/beam_load_mainjenkins0621175931124187c0_25df23acc7034fd28b6d30c21f079ede/5d6c9bfc-6c80-48d2-8727-6ce45697feab">
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1065)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:216)
==========================================================================================
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
Run started 2018-06-21T17:59:30.580Z and ran for PT155.892S
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
Default configuration:
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
{"debug":true,"query":0,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1} at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
Configurations:
Conf Description
0000 query:0; exportSummaryToBigQuery:true; streamTimeout:60
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
0001 query:1; exportSummaryToBigQuery:true; streamTimeout:60
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
0002 query:2; exportSummaryToBigQuery:true; streamTimeout:60
0003 query:3; exportSummaryToBigQuery:true; streamTimeout:60
at org.apache.flink.runtime.operato 0004 query:4; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
0005 query:5; exportSummaryToBigQuery:true; streamTimeout:60
0006 query:6; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
rs.MapPartitionDriver.run(MapPartitionDriver.java:103)
0007 query:7; exportSummaryToBigQuery:true; streamTimeout:60
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
0008 query:8; exportSummaryToBigQuery:true; streamTimeout:60
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
0009 query:9; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
0010 query:10; exportSummaryToBigQuery:true; streamTimeout:60
at java.lang.Thread.run(Thread.java:748)
0011 query:11; exportSummaryToBigQuery:true; streamTimeout:60
0012 query:12; exportSummaryToBigQuery:true; streamTimeout:60
Exception in thread "main" java.lang.RuntimeException: Pipeline execution failed
Performance:
at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:119)
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:311)
0000 1.0 100806.5 100000
0001 0.6 173611.1 92000
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
0002 0.4 250626.6 351
at org.apache.beam.sdk.nexmark.Main.savePerfsToBigQuery(Main.java:182)
0003 10.9 9155.8 580
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:102)
0004 5.5 1832.5 40
at org.apache.beam.sdk.nexmark.Main.main(Main.java:395)
0005 6.0 16775.7 12
Caused by: org.apache.flink.runtime.client.JobExecutionException: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins0621175931124187c0_25df23acc7034fd28b6d30c21f079ede_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
0006 5.8 1733.7 103
at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:625)
0007 7.5 13412.0 1
at org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:234)
0008 7.1 14160.3 6000
at org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:91)
0009 5.2 1906.9 298
at org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:114)
0010 6.2 16257.5 1
at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:116)
0011 7 ... 5 more
.6 13159.6 1919
Caused by: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins0621175931124187c0_25df23acc7034fd28b6d30c21f079ede_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:36)
0012 5.6 17790.4 1919
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
==========================================================================================
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.jav
a:185)
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Unable to insert job: beam_load_mainjenkins0621175931124187c0_25df23acc7034fd28b6d30c21f079ede_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:231)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Source URI must be a Google Cloud Storage location: <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/ws/src/sdks/java/nexmark/nexmark-temp/BigQueryWriteTemp/beam_load_mainjenkins0621175931124187c0_25df23acc7034fd28b6d30c21f079ede/5d6c9bfc-6c80-48d2-8727-6ce45697feab",>
"reason" : "invalid"
} ],
"message" : "Source URI must be a Google Cloud Storage location: <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/ws/src/sdks/java/nexmark/nexmark-temp/BigQueryWriteTemp/beam_load_mainjenkins0621175931124187c0_25df23acc7034fd28b6d30c21f079ede/5d6c9bfc-6c80-48d2-8727-6ce45697feab">
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1065)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:216)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 14,5,main]) completed. Took 2 mins 39.795 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_152/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
See https://docs.gradle.org/4.8/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 17s
62 actionable tasks: 58 executed, 4 from cache
Publishing build scan...
https://gradle.com/s/prbzr66rgxhwu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user migryz@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user marek.simunek@firma.seznam.cz
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user sourabhbajaj@google.com
Not sending mail to unregistered user mergebot@apache.org
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user thomas.haines@practiceinsight.io
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user ddonnelly@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user devon.meunier@shopify.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user cademarkegard@gmail.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user basti.lj@alibaba-inc.com
Not sending mail to unregistered user mrtn@nrd.io
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user azur@paypal.com
Not sending mail to unregistered user jamalone@gmail.com
Not sending mail to unregistered user terrencehan@tencent.com
Not sending mail to unregistered user hadarh@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user fjp@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user taro.murao@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user wnsdud1861@gmail.com
Not sending mail to unregistered user peihe0@gmail.com
Not sending mail to unregistered user sisk@google.com
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Flink #9
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/9/display/redirect?page=changes>
Changes:
[migryz] Point gearpump to master
------------------------------------------
[...truncated 3.83 MB...]
INFO: Shutting down BLOB cache
Jun 21, 2018 5:40:18 PM org.apache.flink.runtime.blob.AbstractBlobCache close
INFO: Shutting down BLOB cache
Jun 21, 2018 5:40:18 PM org.apache.flink.runtime.blob.BlobServer close
INFO: Stopped BLOB server at 0.0.0.0:37221
Jun 21, 2018 5:40:18 PM org.apache.flink.runtime.rpc.akka.AkkaRpcService lambda$stopService$4
INFO: Stopped Akka RPC service.
Jun 21, 2018 5:40:18 PM org.apache.beam.runners.flink.FlinkRunner run
SEVERE: Pipeline execution failed
org.apache.flink.runtime.client.JobExecutionException: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins062117373038c86f64_273fe27a22ec4886aa299a013fcd62c1_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:625)
at org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:234)
at org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:91)
at org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:114)
at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:116)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:311)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
at org.apache.beam.sdk.nexmark.Main.savePerfsToBigQuery(Main.java:182)
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:102)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:395)
Caused by: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins062117373038c86f64_273fe27a22ec4886aa299a013fcd62c1_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:36)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Unable to insert job: beam_load_mainjenkins062117373038c86f64_273fe27a22ec4886aa299a013fcd62c1_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:231)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Source URI must be a Google Cloud Storage location: <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/ws/src/sdks/java/nexmark/nexmark-temp/BigQueryWriteTemp/beam_load_mainjenkins062117373038c86f64_273fe27a22ec4886aa299a013fcd62c1/a0349ffc-e847-4b08-bc09-9bba0fe8f37f",>
"reason" : "invalid"
} ],
"message" : "Source URI must be a Google Cloud Storage location: <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/ws/src/sdks/java/nexmark/nexmark-temp/BigQueryWriteTemp/beam_load_mainjenkins062117373038c86f64_273fe27a22ec4886aa299a013fcd62c1/a0349ffc-e847-4b08-bc09-9bba0fe8f37f">
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1065)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:216)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
==========================================================================================
Run started 2018-06-21T17:37:29.653Z and ran for PT169.014S
Default configuration:
Exception in thread "main" {"debug":true,"query":0,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:0; exportSummaryToBigQuery:true; streamTimeout:60
0001 query:1; exportSummaryToBigQuery:true; streamTimeout:60
0002 query:2; exportSummaryToBigQuery:true; streamTimeout:60
0003 query:3; exportSummaryToBigQuery:true; streamTimeout:60
0004 query:4; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
0005 query:5; exportSummaryToBigQuery:true; streamTimeout:60
0006 query:6; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
0007 query:7; exportSummaryToBigQuery:true; streamTimeout:60
0008 query:8; exportSummaryToBigQuery:true; streamTimeout:60
0009 query:9; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
0010 query:10; exportSummaryToBigQuery:true; streamTimeout:60
0011 query:11; exportSummaryToBigQuery:true; streamTimeout:60
0012 query:12; exportSummaryToBigQuery:true; streamTimeout:60
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)java.lang.RuntimeException: Pipeline execution failed
at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:119)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:311)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
at org.apache.beam.sdk.nexmark.Main.savePerfsToBigQuery(Main.java:182)
0000 0.9 108577.6 100000
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:102)
0001 0.6 173310.2 92000
at org.apache.beam.sdk.nexmark.Main.main(Main.java:395)
0002 0.4 244498.8 351
Caused by: org.apache.flink.runtime.client.JobExecutionException: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins062117373038c86f64_273fe27a22ec4886aa299a013fcd62c1_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
0003 10.8 9297.1 580
at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:625)
0004 5.3 1881.8 40
at org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:234)
0005 5.9 16986.6 12
at org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:91)
0006 6.4 1569.4 103
0007 6.3 15910.9 1
at org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:114)
0008 8.0 12551.8 6000
0009 5.3 1893.9 298
at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:116)
0010 6.3 15807.8 1
... 5 more
0011 7.6 13164.8 1919
Caused by: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins062117373038c86f64_273fe27a22ec4886aa299a013fcd62c1_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
0012 5.6 17711.7 1919
at org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:36)
==========================================================================================
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Unable to insert job: beam_load_mainjenkins062117373038c86f64_273fe27a22ec4886aa299a013fcd62c1_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:231)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Source URI must be a Google Cloud Storage location: <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/ws/src/sdks/java/nexmark/nexmark-temp/BigQueryWriteTemp/beam_load_mainjenkins062117373038c86f64_273fe27a22ec4886aa299a013fcd62c1/a0349ffc-e847-4b08-bc09-9bba0fe8f37f",>
"reason" : "invalid"
} ],
"message" : "Source URI must be a Google Cloud Storage location: <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/ws/src/sdks/java/nexmark/nexmark-temp/BigQueryWriteTemp/beam_load_mainjenkins062117373038c86f64_273fe27a22ec4886aa299a013fcd62c1/a0349ffc-e847-4b08-bc09-9bba0fe8f37f">
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1065)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:216)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 11,5,main]) completed. Took 2 mins 52.929 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_152/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
See https://docs.gradle.org/4.8/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 51s
62 actionable tasks: 59 executed, 3 from cache
Publishing build scan...
https://gradle.com/s/33et3nkunfhjq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user migryz@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user marek.simunek@firma.seznam.cz
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user sourabhbajaj@google.com
Not sending mail to unregistered user mergebot@apache.org
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user thomas.haines@practiceinsight.io
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user ddonnelly@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user devon.meunier@shopify.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user cademarkegard@gmail.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user basti.lj@alibaba-inc.com
Not sending mail to unregistered user mrtn@nrd.io
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user azur@paypal.com
Not sending mail to unregistered user jamalone@gmail.com
Not sending mail to unregistered user terrencehan@tencent.com
Not sending mail to unregistered user hadarh@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user fjp@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user taro.murao@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user wnsdud1861@gmail.com
Not sending mail to unregistered user peihe0@gmail.com
Not sending mail to unregistered user sisk@google.com
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Flink #8
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/8/display/redirect?page=changes>
Changes:
[robertwb] [BEAM-4565] Fix hot key fanout in the face of combiner lifting.
[robertwb] Fix and test global combine with fanout.
------------------------------------------
[...truncated 3.82 MB...]
INFO: Shutting down BLOB cache
Jun 21, 2018 5:34:06 PM org.apache.flink.runtime.blob.AbstractBlobCache close
INFO: Shutting down BLOB cache
Jun 21, 2018 5:34:06 PM org.apache.flink.runtime.blob.BlobServer close
INFO: Stopped BLOB server at 0.0.0.0:41055
Jun 21, 2018 5:34:06 PM org.apache.flink.runtime.rpc.akka.AkkaRpcService lambda$stopService$4
INFO: Stopped Akka RPC service.
Jun 21, 2018 5:34:06 PM org.apache.beam.runners.flink.FlinkRunner run
SEVERE: Pipeline execution failed
org.apache.flink.runtime.client.JobExecutionException: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins0621173120cd70133_cb2e2441cf7a4f0da20800047075b555_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:625)
at org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:234)
at org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:91)
at org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:114)
at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:116)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:311)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
at org.apache.beam.sdk.nexmark.Main.savePerfsToBigQuery(Main.java:182)
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:102)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:395)
Caused by: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins0621173120cd70133_cb2e2441cf7a4f0da20800047075b555_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:36)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Unable to insert job: beam_load_mainjenkins0621173120cd70133_cb2e2441cf7a4f0da20800047075b555_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:231)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Source URI must be a Google Cloud Storage location: <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/ws/src/sdks/java/nexmark/nexmark-temp/BigQueryWriteTemp/beam_load_mainjenkins0621173120cd70133_cb2e2441cf7a4f0da20800047075b555/e899cd38-5b67-4092-92b2-7a246a87aea6",>
"reason" : "invalid"
} ],
"message" : "Source URI must be a Google Cloud Storage location: <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/ws/src/sdks/java/nexmark/nexmark-temp/BigQueryWriteTemp/beam_load_mainjenkins0621173120cd70133_cb2e2441cf7a4f0da20800047075b555/e899cd38-5b67-4092-92b2-7a246a87aea6">
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1065)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:216)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
==========================================================================================
Run started 2018-06-21T17:31:19.082Z and ran for PT167.287S
Default configuration:
{"debug":true,"query":0,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:0; exportSummaryToBigQuery:true; streamTimeout:60
Exception in thread "main" 0001 query:1; exportSummaryToBigQuery:true; streamTimeout:60
0002 query:2; exportSummaryToBigQuery:true; streamTimeout:60
0003 query:3; exportSummaryToBigQuery:true; streamTimeout:60
0004 query:4; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
0005 query:5; exportSummaryToBigQuery:true; streamTimeout:60
0006 query:6; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
0007 query:7; exportSummaryToBigQuery:true; streamTimeout:60
0008 query:8; exportSummaryToBigQuery:true; streamTimeout:60
0009 query:9; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
0010 query:10; exportSummaryToBigQuery:true; streamTimeout:60
0011 query:11; exportSummaryToBigQuery:true; streamTimeout:60
0012 query:12; exportSummaryToBigQuery:true; streamTimeout:60
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 0.9 114155.3 100000
0001 0.6 172117.0 92000
0002 0.4 228310.5 351
0003 10.7 9379.1 580
0004 5.6 1794.0 40
0005 5.2 19413.7 12
java.lang.RuntimeException: Pipeline execution failed
0006 4.8 2080.3 103
at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:119)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:311)
0007 5.7 17491.7 1
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
0008 8.6 11627.9 6000
at org.apache.beam.sdk.nexmark.Main.savePerfsToBigQuery(Main.java:182)
0009 5.0 1988.9 298
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:102)
0010 5.8 17199.9 1
at org.apache.beam.sdk.nexmark.Main.main(Main.java:395)
0011 7.5 13317.4 1919
Caused by: org.apache.flink.runtime.client.JobExecutionException: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins0621173120cd70133_cb2e2441cf7a4f0da20800047075b555_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
0012 5.6 17979.1 1919
at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:625)
at org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:234)
==========================================================================================
at org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:91)
at org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:114)
at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:116)
... 5 more
Caused by: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins0621173120cd70133_cb2e2441cf7a4f0da20800047075b555_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:36)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Unable to insert job: beam_load_mainjenkins0621173120cd70133_cb2e2441cf7a4f0da20800047075b555_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:231)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Source URI must be a Google Cloud Storage location: <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/ws/src/sdks/java/nexmark/nexmark-temp/BigQueryWriteTemp/beam_load_mainjenkins0621173120cd70133_cb2e2441cf7a4f0da20800047075b555/e899cd38-5b67-4092-92b2-7a246a87aea6",>
"reason" : "invalid"
} ],
"message" : "Source URI must be a Google Cloud Storage location: <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/ws/src/sdks/java/nexmark/nexmark-temp/BigQueryWriteTemp/beam_load_mainjenkins0621173120cd70133_cb2e2441cf7a4f0da20800047075b555/e899cd38-5b67-4092-92b2-7a246a87aea6">
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1065)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:216)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 15,5,main]) completed. Took 2 mins 51.225 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_152/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
See https://docs.gradle.org/4.8/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 50s
62 actionable tasks: 59 executed, 3 from cache
Publishing build scan...
https://gradle.com/s/4dqfnfx3epd3w
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user migryz@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user marek.simunek@firma.seznam.cz
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user sourabhbajaj@google.com
Not sending mail to unregistered user mergebot@apache.org
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user thomas.haines@practiceinsight.io
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user ddonnelly@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user devon.meunier@shopify.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user cademarkegard@gmail.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user basti.lj@alibaba-inc.com
Not sending mail to unregistered user mrtn@nrd.io
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user azur@paypal.com
Not sending mail to unregistered user jamalone@gmail.com
Not sending mail to unregistered user terrencehan@tencent.com
Not sending mail to unregistered user hadarh@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user fjp@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user taro.murao@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user wnsdud1861@gmail.com
Not sending mail to unregistered user peihe0@gmail.com
Not sending mail to unregistered user sisk@google.com
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Flink #7
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/7/display/redirect?page=changes>
Changes:
[matthias] Futurize examples subpackage
[kirpichov] A couple more trivial changes extracted from #5662
[katarzyna.kucharczyk] Created static retry method which accepts retriable method, amount of
[rober] Fix build break from PR 5676, which modified method signatures.
[migryz] Implemented working prototype for PostCommit jobs
[migryz] Update post commit jobs to utilize new helper methods
[ankurgoenka] Fix cache references in portable flink runner
[Pablo] Adding ignore parameters for gitattributes
[swegner] Remove submodule
[migryz] Fix typo in _ghprb suffix
[migryz] Update suffix to be PullRequest
[ccy] Revert #5689 to fix build
[ankurgoenka] Remove unused import
[github] Make suffix shorter
[kirpichov] Make ImmutableExecutableStage constructors public
[kirpichov] [BEAM-4285] Implement Flink batch side input handler
[ekirpichov] Introduces PipelineValidator that checks the well-formedness of a
[cademarkegard] [BEAM-4325] Enforce ErrorProne analysis in the SQL project
[echauchot] [BEAM-4283] Fix naming of the BigQuery fields
[coheigea] Removing some null checks, where we already know that the variable in
[iemejia] [BEAM-3314] Set correctly host and port on RedisIO
[iemejia] [BEAM-3314] Fix error-prone warnings and add extra test for Read
------------------------------------------
[...truncated 3.83 MB...]
INFO: Shutting down BLOB cache
Jun 21, 2018 2:45:15 PM org.apache.flink.runtime.blob.AbstractBlobCache close
INFO: Shutting down BLOB cache
Jun 21, 2018 2:45:15 PM org.apache.flink.runtime.blob.BlobServer close
INFO: Stopped BLOB server at 0.0.0.0:37159
Jun 21, 2018 2:45:15 PM org.apache.flink.runtime.rpc.akka.AkkaRpcService lambda$stopService$4
INFO: Stopped Akka RPC service.
Jun 21, 2018 2:45:15 PM org.apache.beam.runners.flink.FlinkRunner run
SEVERE: Pipeline execution failed
org.apache.flink.runtime.client.JobExecutionException: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins062114421865f5fecf_c0cd4ad2cd3a46569d678f11a548b866_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:625)
at org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:234)
at org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:91)
at org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:114)
at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:116)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:311)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
at org.apache.beam.sdk.nexmark.Main.savePerfsToBigQuery(Main.java:182)
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:102)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:395)
Caused by: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins062114421865f5fecf_c0cd4ad2cd3a46569d678f11a548b866_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:36)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Unable to insert job: beam_load_mainjenkins062114421865f5fecf_c0cd4ad2cd3a46569d678f11a548b866_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:231)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Source URI must be a Google Cloud Storage location: <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/ws/src/sdks/java/nexmark/nexmark-temp/BigQueryWriteTemp/beam_load_mainjenkins062114421865f5fecf_c0cd4ad2cd3a46569d678f11a548b866/2b1cebd6-8d28-486c-a208-6d4f7f3123a0",>
"reason" : "invalid"
} ],
"message" : "Source URI must be a Google Cloud Storage location: <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/ws/src/sdks/java/nexmark/nexmark-temp/BigQueryWriteTemp/beam_load_mainjenkins062114421865f5fecf_c0cd4ad2cd3a46569d678f11a548b866/2b1cebd6-8d28-486c-a208-6d4f7f3123a0">
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1065)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:216)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
==========================================================================================
Run started 2018-06-21T14:42:16.966Z and ran for PT178.429S
Default configuration:
{"debug":true,"query":0,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:0; exportSummaryToBigQuery:true; streamTimeout:60
0001 query:1; exportSummaryToBigQuery:true; streamTimeout:60
0002 query:2; exportSummaryToBigQuery:true; streamTimeout:60
Exception in thread "main" 0003 query:3; exportSummaryToBigQuery:true; streamTimeout:60
0004 query:4; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
0005 query:5; exportSummaryToBigQuery:true; streamTimeout:60
0006 query:6; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
0007 query:7; exportSummaryToBigQuery:true; streamTimeout:60
0008 query:8; exportSummaryToBigQuery:true; streamTimeout:60
0009 query:9; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
0010 query:10; exportSummaryToBigQuery:true; streamTimeout:60
0011 query:11; exportSummaryToBigQuery:true; streamTimeout:60
0012 query:12; exportSummaryToBigQuery:true; streamTimeout:60
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 0.9 112359.6 100000
java.lang.RuntimeException: Pipeline execution failed
0001 0.5 184162.1 92000
at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:119)
0002 0.4 227790.4 351
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:311)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
at org.apache.beam.sdk.nexmark.Main.savePerfsToBigQuery(Main.java:182)
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:102)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:395)
0003 10.9 9168.4 580
Caused by: org.apache.flink.runtime.client.JobExecutionException: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins062114421865f5fecf_c0cd4ad2cd3a46569d678f11a548b866_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:625)
0004 6.1 1631.1 40
at org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:234)
0005 5.8 17385.3 12
at org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:91)
0006 6.1 1649.6 103
at org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:114)
at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:116)
... 5 more
0007 6.4 15511.1 1
Caused by: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins062114421865f5fecf_c0cd4ad2cd3a46569d678f11a548b866_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
0008 7.8 12822.2 6000
at org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:36)
0009 5.6 1800.5 298
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
0010 6.6 15089.8 1
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
0011 7.3 13719.3 1919
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
0012 5.9 16946.3 1919
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
==========================================================================================
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Unable to insert job: beam_load_mainjenkins062114421865f5fecf_c0cd4ad2cd3a46569d678f11a548b866_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:231)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Source URI must be a Google Cloud Storage location: <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/ws/src/sdks/java/nexmark/nexmark-temp/BigQueryWriteTemp/beam_load_mainjenkins062114421865f5fecf_c0cd4ad2cd3a46569d678f11a548b866/2b1cebd6-8d28-486c-a208-6d4f7f3123a0",>
"reason" : "invalid"
} ],
"message" : "Source URI must be a Google Cloud Storage location: <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/ws/src/sdks/java/nexmark/nexmark-temp/BigQueryWriteTemp/beam_load_mainjenkins062114421865f5fecf_c0cd4ad2cd3a46569d678f11a548b866/2b1cebd6-8d28-486c-a208-6d4f7f3123a0">
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1065)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:216)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 6,5,main]) completed. Took 3 mins 1.984 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_152/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
See https://docs.gradle.org/4.8/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 6m 7s
62 actionable tasks: 62 executed
Publishing build scan...
https://gradle.com/s/bamg3reczunnk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user katarzyna.kucharczyk@polidea.com
Not sending mail to unregistered user migryz@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user marek.simunek@firma.seznam.cz
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user sourabhbajaj@google.com
Not sending mail to unregistered user mergebot@apache.org
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user thomas.haines@practiceinsight.io
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user ddonnelly@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user devon.meunier@shopify.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user cademarkegard@gmail.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user basti.lj@alibaba-inc.com
Not sending mail to unregistered user mrtn@nrd.io
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user azur@paypal.com
Not sending mail to unregistered user jamalone@gmail.com
Not sending mail to unregistered user terrencehan@tencent.com
Not sending mail to unregistered user hadarh@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user fjp@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user taro.murao@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user wnsdud1861@gmail.com
Not sending mail to unregistered user peihe0@gmail.com
Not sending mail to unregistered user sisk@google.com
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Flink #6
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/6/display/redirect>
------------------------------------------
[...truncated 3.83 MB...]
Jun 20, 2018 6:29:54 PM org.apache.flink.runtime.webmonitor.WebMonitorEndpoint lambda$shutDownInternal$4
INFO: Removing cache directory /tmp/flink-web-ui
Jun 20, 2018 6:29:54 PM org.apache.flink.runtime.rpc.akka.AkkaRpcService stopService
INFO: Stopping Akka RPC service.
Jun 20, 2018 6:29:54 PM org.apache.flink.runtime.blob.AbstractBlobCache close
INFO: Shutting down BLOB cache
Jun 20, 2018 6:29:54 PM org.apache.flink.runtime.blob.AbstractBlobCache close
INFO: Shutting down BLOB cache
Jun 20, 2018 6:29:54 PM org.apache.flink.runtime.blob.BlobServer close
INFO: Stopped BLOB server at 0.0.0.0:41337
Jun 20, 2018 6:29:54 PM org.apache.flink.runtime.rpc.akka.AkkaRpcService lambda$stopService$4
INFO: Stopped Akka RPC service.
Jun 20, 2018 6:29:54 PM org.apache.beam.runners.flink.FlinkRunner run
SEVERE: Pipeline execution failed
org.apache.flink.runtime.client.JobExecutionException: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins06201827005b72bb68_e13eb641db0c4777a9722634eed9ff84_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:625)
at org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:234)
at org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:91)
at org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:114)
at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:116)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:311)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
at org.apache.beam.sdk.nexmark.Main.savePerfsToBigQuery(Main.java:182)
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:102)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:395)
Caused by: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins06201827005b72bb68_e13eb641db0c4777a9722634eed9ff84_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:36)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Unable to insert job: beam_load_mainjenkins06201827005b72bb68_e13eb641db0c4777a9722634eed9ff84_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:231)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Invalid field name \"Runtime(sec)\". Fields must contain only letters, numbers, and underscores, start with a letter or underscore, and be at most 128 characters long.",
"reason" : "invalid"
} ],
"message" : "Invalid field name \"Runtime(sec)\". Fields must contain only letters, numbers, and underscores, start with a letter or underscore, and be at most 128 characters long."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1065)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:216)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
==========================================================================================
Run started 2018-06-20T18:26:59.618Z and ran for PT174.852S
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
Exception in thread "main" java.lang.RuntimeException: Pipeline execution failed
at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:119)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:311)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
at org.apache.beam.sdk.nexmark.Main.savePerfsToBigQuery(Main.java:182)
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:102)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:395)
Caused by: org.apache.flink.runtime.client.JobExecutionException: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins06201827005b72bb68_e13eb641db0c4777a9722634eed9ff84_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:625)
at org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:234)
at org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:91)
at org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:114)
at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:116)
... 5 more
Caused by: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins06201827005b72bb68_e13eb641db0c4777a9722634eed9ff84_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:36)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Unable to insert job: beam_load_mainjenkins06201827005b72bb68_e13eb641db0c4777a9722634eed9ff84_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:231)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Invalid field name \"Runtime(sec)\". Fields must contain only letters, numbers, and underscores, start with a letter or underscore, and be at most 128 characters long.",
"reason" : "invalid"
} ],
"message" : "Invalid field name \"Runtime(sec)\". Fields must contain only letters, numbers, and underscores, start with a letter or underscore, and be at most 128 characters long."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1065)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:216)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
Default configuration:
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
{"debug":true,"query":0,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
Configurations:
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
Conf Description
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
0000 query:0; exportSummaryToBigQuery:true; streamTimeout:60
0001 query:1; exportSummaryToBigQuery:true; streamTimeout:60
0002 query:2; exportSummaryToBigQuery:true; streamTimeout:60
0003 query:3; exportSummaryToBigQuery:true; streamTimeout:60
0004 query:4; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
0005 query:5; exportSummaryToBigQuery:true; streamTimeout:60
0006 query:6; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
0007 query:7; exportSummaryToBigQuery:true; streamTimeout:60
0008 query:8; exportSummaryToBigQuery:true; streamTimeout:60
0009 query:9; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
0010 query:10; exportSummaryToBigQuery:true; streamTimeout:60
0011 query:11; exportSummaryToBigQuery:true; streamTimeout:60
0012 query:12; exportSummaryToBigQuery:true; streamTimeout:60
Performance:
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline) at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
0000 0.8 118764.8 100000
0001 0.6 158227.8 92000
0002 0.5 220264.3 351
0003 13.6 7360.5 580
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
0004 5.6 1774.0 40
0005 5.8 17253.3 12
0006 6.7 1484.6 103
0007 6.7 15021.8 1
0008 8.0 12518.8 6000
0009 5.8 1728.6 298
0010 6.5 15330.4 1
0011 7.6 13182.2 1919
0012 6.0 16611.3 1919
==========================================================================================
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 2,5,main]) completed. Took 2 mins 58.608 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_152/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
See https://docs.gradle.org/4.8/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 56s
62 actionable tasks: 58 executed, 4 from cache
Publishing build scan...
https://gradle.com/s/ippvn6mudqrxy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user marek.simunek@firma.seznam.cz
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user sourabhbajaj@google.com
Not sending mail to unregistered user mergebot@apache.org
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user thomas.haines@practiceinsight.io
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user ddonnelly@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user devon.meunier@shopify.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user basti.lj@alibaba-inc.com
Not sending mail to unregistered user mrtn@nrd.io
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user azur@paypal.com
Not sending mail to unregistered user jamalone@gmail.com
Not sending mail to unregistered user terrencehan@tencent.com
Not sending mail to unregistered user hadarh@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user fjp@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user taro.murao@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user wnsdud1861@gmail.com
Not sending mail to unregistered user peihe0@gmail.com
Not sending mail to unregistered user sisk@google.com
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Flink #5
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/5/display/redirect?page=changes>
Changes:
[ehudm] Jupyter notebook for monitoring precommit job times
------------------------------------------
[...truncated 3.83 MB...]
Jun 20, 2018 5:44:10 PM org.apache.flink.runtime.webmonitor.WebMonitorEndpoint lambda$shutDownInternal$4
INFO: Removing cache directory /tmp/flink-web-ui
Jun 20, 2018 5:44:10 PM org.apache.flink.runtime.rpc.akka.AkkaRpcService stopService
INFO: Stopping Akka RPC service.
Jun 20, 2018 5:44:10 PM org.apache.flink.runtime.blob.AbstractBlobCache close
INFO: Shutting down BLOB cache
Jun 20, 2018 5:44:10 PM org.apache.flink.runtime.blob.AbstractBlobCache close
INFO: Shutting down BLOB cache
Jun 20, 2018 5:44:10 PM org.apache.flink.runtime.blob.BlobServer close
INFO: Stopped BLOB server at 0.0.0.0:43917
Jun 20, 2018 5:44:10 PM org.apache.flink.runtime.rpc.akka.AkkaRpcService lambda$stopService$4
INFO: Stopped Akka RPC service.
Jun 20, 2018 5:44:10 PM org.apache.beam.runners.flink.FlinkRunner run
SEVERE: Pipeline execution failed
org.apache.flink.runtime.client.JobExecutionException: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins0620174139e4c5cb3c_c2ee7d3a0dd5423ea360c315b3f90f7e_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:625)
at org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:234)
at org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:91)
at org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:114)
at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:116)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:311)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
at org.apache.beam.sdk.nexmark.Main.savePerfsToBigQuery(Main.java:182)
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:102)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:395)
Caused by: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins0620174139e4c5cb3c_c2ee7d3a0dd5423ea360c315b3f90f7e_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:36)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Unable to insert job: beam_load_mainjenkins0620174139e4c5cb3c_c2ee7d3a0dd5423ea360c315b3f90f7e_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:231)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Invalid field name \"Runtime(sec)\". Fields must contain only letters, numbers, and underscores, start with a letter or underscore, and be at most 128 characters long.",
"reason" : "invalid"
} ],
"message" : "Invalid field name \"Runtime(sec)\". Fields must contain only letters, numbers, and underscores, start with a letter or underscore, and be at most 128 characters long."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1065)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:216)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
==========================================================================================
Run started 2018-06-20T17:41:38.301Z and ran for PT151.775S
Default configuration:
{"debug":true,"query":0,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:0; exportSummaryToBigQuery:true; streamTimeout:60
0001 query:1; exportSummaryToBigQuery:true; streamTimeout:60
0002 query:2; exportSummaryToBigQuery:true; streamTimeout:60
0003 query:3; exportSummaryToBigQuery:true; streamTimeout:60
0004 query:4; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
0005 query:5; exportSummaryToBigQuery:true; streamTimeout:60
0006 query:6; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
0007 query:7; exportSummaryToBigQuery:true; streamTimeout:60
0008 query:8; exportSummaryToBigQuery:true; streamTimeout:60
0009 query:9; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
Exception in thread "main" 0010 query:10; exportSummaryToBigQuery:true; streamTimeout:60
0011 query:11; exportSummaryToBigQuery:true; streamTimeout:60
0012 query:12; exportSummaryToBigQuery:true; streamTimeout:60
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 0.8 122100.1 100000
0001 0.6 165016.5 92000
0002 0.5 218340.6 351
0003 11.3 8837.0 580
0004 6.5 1528.1 40
0005 6.4 15666.6 12
0006 6.6 1504.9 103
0007 7.0 14251.1 1
0008 8.3 11984.7 6000
0009 5.9 1691.8 298
0010 6.7 15024.0 1
0011 8.1 12336.5 1919
0012 6.2 16084.9 1919
==========================================================================================
java.lang.RuntimeException: Pipeline execution failed
at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:119)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:311)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
at org.apache.beam.sdk.nexmark.Main.savePerfsToBigQuery(Main.java:182)
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:102)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:395)
Caused by: org.apache.flink.runtime.client.JobExecutionException: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins0620174139e4c5cb3c_c2ee7d3a0dd5423ea360c315b3f90f7e_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:625)
at org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:234)
at org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:91)
at org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:114)
at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:116)
... 5 more
Caused by: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins0620174139e4c5cb3c_c2ee7d3a0dd5423ea360c315b3f90f7e_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:36)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Unable to insert job: beam_load_mainjenkins0620174139e4c5cb3c_c2ee7d3a0dd5423ea360c315b3f90f7e_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:231)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Invalid field name \"Runtime(sec)\". Fields must contain only letters, numbers, and underscores, start with a letter or underscore, and be at most 128 characters long.",
"reason" : "invalid"
} ],
"message" : "Invalid field name \"Runtime(sec)\". Fields must contain only letters, numbers, and underscores, start with a letter or underscore, and be at most 128 characters long."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1065)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:216)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 4,5,main]) completed. Took 2 mins 35.471 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_152/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
See https://docs.gradle.org/4.8/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 32s
62 actionable tasks: 58 executed, 4 from cache
Publishing build scan...
https://gradle.com/s/xcfxck3ndgcia
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user marek.simunek@firma.seznam.cz
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user sourabhbajaj@google.com
Not sending mail to unregistered user mergebot@apache.org
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user thomas.haines@practiceinsight.io
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user ddonnelly@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user devon.meunier@shopify.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user basti.lj@alibaba-inc.com
Not sending mail to unregistered user mrtn@nrd.io
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user azur@paypal.com
Not sending mail to unregistered user jamalone@gmail.com
Not sending mail to unregistered user terrencehan@tencent.com
Not sending mail to unregistered user hadarh@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user fjp@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user taro.murao@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user wnsdud1861@gmail.com
Not sending mail to unregistered user peihe0@gmail.com
Not sending mail to unregistered user sisk@google.com
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Flink #4
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Flink/4/display/redirect?page=changes>
Changes:
[jamalone] Initial add
[stefano] Fix duplicated Stephan Ewen
[stefano] Fix broken mailto
[jamalone] Fix for timezone
[jamalone] Addition of Twitter and Blog details to index.
[klk] Add apache ID 'kenn' for Kenneth Knowles
[jamalone] New blog post; changes to logos.
[jamalone] Fix for blog post.
[jamalone] Edit to index page
[Max] change output directory to "content"
[jbonofre] Fix jbonofre e-mail
[mxm] Team Updates
[takidau] Capability matrix page + blog post: - Content as discussed in this
[davor] Add content files missed in PR2 due to an overly aggressive .gitignore
[takidau] Blog page "read more" links + README.md for the site.
[takidau] Remove trailing slash in blog "read more" links
[mrtn] Fix 'getting started page' link in homepage alert message
[davor] Add apache_ids for everyone that has one at this time.
[davor] Add contribution guide
[takidau] Capability matrix updates: - Add sliding window support for batch Spark
[davor] Addition of public materials page/post
[jamalone] Addition of public meetings page
[jamalone] Update for second meeting
[jamalone] Update for email list
[jamalone] Update to meeting page
[jamalone] Addition of material page; nav fixes
[ddonnelly] Added the /docs/ directory and a landing page at docs/index.md. Added
[iemejia] Add button to make the navigation bar accesible in mobile browsing
[jamalone] Addition of PCollection blog post; blog fix
[jamalone] Addition of sAF blog; delay of PCollection post.
[mxm] [BEAM-103] update capability matrix
[jamalone] Update date for blog post
[robertwb] Remove duplicate paragraph.
[robertwb] Fix blog post typo.
[davor] Add Flink Batch Runner Blog Post
[davor] Add a blog post about the first release
[davor] Add a page about releases
[davor] Add Javadoc for version 0.1.0-incubating
[jamalone] Fixed HTML errors; added link and HTML tests via rake
[github] added incubation disclaimer
[github] fixed typo
[fjp] Merged in pullhttps://github.com/apache/beam-site/pull/25 and
[fjp] Adding Travis CI YAML config
[dhalperi] [BEAM-500] Finished skeleton of new website layout
[dhalperi] [BEAM-510] releases: use ASF download script to balance load on Apache
[fjp] Fixed remaining rake failure.
[klk] Moved javadoc under /learn/sdks/java and added the links back
[fjp] Addressed BEAM-514 (add required asf links)
[fjp] Addressed Beam-515 (apache logos)
[dhalperi] Minor fixups
[dhalperi] Navbar: move ASF dropdown to right side
[dhalperi] Minor style tweaks for alignment
[dhalperi] Fix ASF header collapsing
[dhalperi] Added a half birthday blog post, as discussed on dev@.
[dhalperi] Update to today
[dhalperi] Add a link to the Apache Code of Conduct
[klk] Revise Beam programming guide for new DoFn
[jasonkuster] Add testing guide to website.
[mxm] [BEAM-102] update capability matrix
[jason] Updated per PR comments
[fjp] Added toc
[dhalperi] Update downloads link to 0.2.0-incubating
[dhalperi] Upgrade Javadoc to 0.2.0-incubating
[dhalperi] Build fix for Travis
[terrencehan] minor: remove duplicate words
[fjp] Added blog post about Strata+Hadoop World tutorial. Added Jesse Anderson
[fjp] Regenerate html and make 'rake test' pass.
[fjp] Added bundler gem to README.
[fjp] Update mailing lists to use new lists site.
[fjp] Add Design Principles (take from the original Beam technical vision
[fjp] Remove page title from TOCs
[fjp] Updated Contribution Guide & added Work In Progress * Added section on
[fjp] Add Blog Post on TestStream
[dhalperi] Add release guide to the website
[fjp] Updated website instructions to use Bundler correctly for managing ruby
[fjp] Turn on existing syntax highlighting styles.
[fjp] Enable syntax highlighting in @tgroh's blog.
[davor] Update to Beam committers
[fjp] Removed TriggerTester, obsoleted in
[fjp] Add IDE setup instructions
[fjp] Reordered IDE section after code checkout and tweaked heading levels.
[davor] [BEAM-501] New skin proposal
[fjp] Add a first version of the WordCount walkthrough.
[fjp] Temporarily disable tests for v2/, since the links are all fake
[fjp] Add downstream merge exemption to contribution guide
[swegner] Add IntelliJ Checkstyle documentation
[aljoscha.krettek] Add pointers to 0.3.0-incubating release artifacts
[davor] Flink compatibility matrix: drop redundant text
[davor] [BEAM-752] Infrastructure for toggling code snippets in documentation
[davor] Add Mobile gaming walkthrough
[davor] New structure
[davor] Move Javadoc from learn/sdks to documentation/sdks.
[davor] [BEAM-890] Update compatibility matrix for Spark.
[dhalperi] Add missing site.baseurls
[dhalperi] Build in a separate content directory to catch bad links
[davor] Move files
[davor] Update README.md and _config.yml after moving the source to the src/
[fjp] Initial pass on a real Quickstart
[davor] Remove unnecessary build timestamps in the generated site
[davor] Fix typo in file name (materix -> matrix)
[davor] BEAM-845 Update navigation and runner capability matrix to include Apex.
[davor] [BEAM-902] Added runner toggles [BEAM-752] Infrastructure for toggling
[davor] Introduce site.release_latest global variable to facilitate Javadoc
[dhalperi] Add note about gpg-agent to release-guide
[dhalperi] Ignore more generated/internal Javadoc in Release Guide
[dhalperi] Tweak gpg-agent text, move to GPG setup instructions
[davor] Add tool to fix links.
[jason] Remove parser since it's not on the executors.
[davor] Update beam_team Committer IDs
[thw] Update Apex runner info after merge to master.
[github] "before" instead of "behind" for "early" elements
[davor] [BEAM-505] Fill in the documentation/runners/direct portion of the
[davor] [BEAM-508] Fill in the documentation/runners/dataflow portion of the
[davor] Use jekyll site.release_latest variable in direct runner sample pom.xml
[davor] Add Jenkins job definitions for existing projects using Jenkins DSL
[davor] Update content on v2 landing page
[fjp] Add Python SDK page and add it to /documentation/index.md
[fjp] Rename /get-started/releases to get-started/downloads. Delete old
[aljoscha.krettek] [BEAM-899] Add Flink Instructions to quickstart.md
[davor] Remove Apex runner from work in progress
[aljoscha.krettek] [BEAM-1007] Fixed toggles code, to work in safari
[davor] Update download page to specify backward-compatibility guarantees
[davor] Quickstart: suggest using nightly build temporarily
[davor] Add maturity model assessment
[davor] Add commiter/PMC bar information
[davor] Add contribution diversity
[davor] Update documentation/pipelines/design-your-pipeline.md. Add 5 images for
[davor] Add external link icon to Java SDK API reference, and change to open in
[davor] Update main landing page and Beam Overview page
[davor] [BEAM-277] Add transforms section to the programming guide
[dhalperi] [BEAM-506] Fill in the documentation/runners/flink portion of the
[dhalperi] Website: minor typographical corrections
[davor] Simplify Flink Runner instructions for running on cluster
[davor] fixed command to run WordCount example on direct runner, now in the site
[davor] Update Additional Resources page
[davor] [BEAM-507] Fill in the documentation/runners/spark portion of the
[klk] Rename PTransform apply to expand throughout
[tgroh] [BEAM-835] Added IntelliJ instructions to the contribution guide
[tgroh] Reword IntelliJ Annotation Processor section
[davor] [BEAM-900] Spark quickstart instructions.
[davor] Dataflow quickstart fix
[davor] Add DOAP file for the project
[davor] Remove incubating references
[davor] [BEAM-1220] Update team.md with post-graduation committer list
[davor] [BEAM-1163] Add signature keys to the vote template (release guide)
[davor] Add clarification to original pull request
[davor] Update website pull request instructions in the Release Guide
[davor] Additional clarifications to pull request
[davor] Update Release Guide: exclude apex runner translation package from the
[davor] Updated Wordcount, Mobile gaming, and Using the Docs pages.
[dhalperi] Added python supports in Programming Guide
[davor] Update Release Guide as part of the post graduation tasks
[dhalperi] Rename apache_beam.utils.options to apache_beam.utils.pipeline_options
[davor] Add .htaccess to automatically forward HTTP to HTTP
[jbonofre] Add 0.4.0 release
[dhalperi] test-your-pipeline: use site.baseurl, site.release_latest, canonical
[davor] Added post for 0.4.0 release with Apex runner addition.
[davor] Publish a blog about Apache Apex runner
[davor] Publish a blog about Apache Apex, second attempt
[davor] testing.md: fix links to project
[davor] Pass over the contribution guide
[github] Limit the number of blog posts listed on the main page
[github] Update copyright on each page
[davor] Update maturity model to reflect graduation
[davor] Blog post on Beam graduation
[robertwb] Add plugin for snippet extraction.
[robertwb] Replace literal code samples with extracted, tested snippets.
[dhalperi] [BEAM-278] Add I/O section to programming guide
[dhalperi] Update according to reviewer comments
[dhalperi] minor fixes
[davor] Clarify maturity of various runners.
[jbonofre] Use PMC instead of PPMC
[davor] Adds PTransform style guide
[davor] WIP: Add Python to WordCount documentation
[davor] Added Python Quickstart
[davor] Update documentation to remove python-sdk branch references
[altay] Add new committers to the team.
[davor] Cleanup of links: relative instead of absolute, https instead of http,
[davor] Blog: "Media recap of the Apache Beam graduation"
[davor] Disable Travis coverage
[davor] Standardize Jenkins suites into .jenkins directory
[github] Fix typo in programming-guide.md
[davor] Delete jobs due to move to main repo.
[altay] Add page about python type safety and type hints
[davor] Add 0.5.0 release
[klk] Update the capability matrix for state and timers
[fjp] Update programming guide with Python snippets and data encoding section.
[davor] Add documentation to build the site with a jekyll container
[melissapa] Add back composite transforms anchor to fix broken links
[dhalperi] processBatch() is now called processBundle()
[altay] Add new Managing Python Pipeline Dependencies page
[altay] Update ParDo documentation for Python
[davor] Add klk as a blog author
[davor] Add blog post "Stateful Processing with Apache Beam"
[klk] Fix typo in stateful processing post
[iemejia] Fix some typos and small formatting issues.
[klk] Touch up some punctuation in state blog post
[davor] Clarify state further in capability matrix
[altay] Add custom sources and sinks for Python page
[dhalperi] [BEAM-475] update release instructions for javadoc
[dhalperi] Minor tweaks
[melissapa] Fix broken links due to code path changes
[davor] Add Apache in front of initial Beam reference
[sourabhbajaj] [BEAM-1320] Update release guide to create pydocs
[sourabhbajaj] Fix broken links
[staslevin] Adding testing related info: TestPipeline, API surface.
[altay] Fix the quickstart documentation
[altay] Update quickstart to have runner specific examples
[thomas.haines] BEAM-1559 add HBase docs
[davor] Add contrib guide comments about using squash & force push.
[davor] Comment Ostatic link as it has been broken for a few weeks
[iemejia] [BEAM-325] Add Slack details to website
[mairbek] Typo fix in "Reading Data Into Your Pipeline" example.
[wnsdud1861] Fixed typos (from distributed to distributed)
[altay] Finding a typo in a code block
[davor] Fix possible erranous naming conventions reference
[azur] Add IntelliJ files to .gitignore
[davor] [BEAM-1652] Code style instructions for IntelliJ in contribution guide.
[davor] Documentation changes for 0.6.0 release.
[sourabhbajaj] Add htaccess back
[altay] Changes to web site related to recently released Python SDK
[devon.meunier] Typo fix
[davor] Blog post for 0.6.0 release with python sdk
[kirpichov] Add jkff as committer
[iemejia] Add iemejia as committer
[davor] Add Pipeline I/O section to website - outline + move some existing
[davor] Add chamikara as a committer
[azur] Add aviemzur as committer
[iemejia] Improve committing website changes section in the contribution guide
[iemejia] Fix timezone to America/Los_Angeles
[peihe0] Change peihe company affiliation to Alibaba.
[davor] Simplify merge process section
[davor] Add windowing section to programming guide
[altay] Simplify python quick start.
[davor] add code snippet to page design-your-pipeline
[klk] Reword RunnableOnService to ValidatesRunner
[aljoscha.krettek] Update Capability Matrix: Flink supports state/timers in batch mode
[sourabhbajaj] Fix the link to the apache ICLA
[sourabhbajaj] [BEAM-1711] Add information about extra requires in Python Quickstart
[altay] Docker server command should run non-incremental build
[altay] Add "In-Progress" info to the Built-in I/O Transforms page
[altay] Add Read Transform content for Authoring I/O Transforms - Overview
[sisk] Add to in-progress IO list and alphabetize
[azur] Add JSON as a planned IO to built-in IOs page.
[github] Update quickstart-java.md
[dhalperi] [BEAM-1949] Associated doc changes for renaming sideOutput to output
[dhalperi] add CoGroupByKey to chapter 'Using GroupByKey'
[aljoscha.krettek] remove extra streaming refs in capability matrix: Flink state/timers
[davor] Track RestIO in In-Progress I/O Transforms
[dhalperi] [BEAM-1272] Align the naming of "generateInitialSplits" and
[davor] [BEAM-1949] Doc changes for Python SideOutputValue to OutputValue rename
[davor] [BEAM-1741] Update direct and Cloud Dataflow runner pages with Python
[davor] Update overview page.
[davor] update release notes post 0.6.0 release
[aljoscha.krettek] Update flink page ref from language-python to language-py and add toggle
[davor] [BEAM-919] Remove remaining old use/learn links from website src
[davor] [BEAM-825] Fill in the documentation/runners/apex portion of the website
[davor] Fix Apex runner instructions (pending review comments and other changes)
[davor] [BEAM-1452] Add composite transforms section to programming guide
[davor] [BEAM-1011] Add triggers content to the programming guide
[altay] [BEAM-1695] Improve Python-SDK's programming guide
[lcwik] Fix broken HDFS link on built-in IO page
[altay] [BEAM-2142] Add composite transforms Python snippets to programming
[altay] Add Python snippets to Mobile Gaming.
[klk] [BEAM-969] Add a gearpump runner web page
[klk] Minor touch-ups to Gearpump listing on overview
[davor] add SQL_DSL to page 'work-in-progress'
[davor] Transfer some content from Create Your Pipeline to the Programming
[davor] Add trademark symbol to the Downloads page
[altay] Remove README.md references from the website
[altay] Migrate hadoop inputformat to website
[altay] Move extension README files to website
[melissapa] Add quickstart step re: Python 2.7.x requirement
[altay] fix typos in hadoop io documentation
[davor] Release guide: fix instructions
[davor] Revise WordCount Example Walkthrough
[altay] Quick start should not default to java quickstart
[kirpichov] Rewrites the section on Coders to not talk about them as a parsing
[kirpichov] [BEAM-1353] Style Guide fixups
[dhalperi] [BEAM-501] Large visual refresh of Apache Beam website homepage
[dhalperi] Shorten quotes and fix a typo
[dhalperi] [BEAM-501] fix quotes
[dhalperi] Update latest version metadata information
[dhalperi] Add version 2.0.0 to Downloads page
[dhalperi] Python: fix links to PipelineOptions after move from utils to options
[dhalperi] Update Java Quickstart to stop using snapshot repos
[davor] Add a draft of the Beam FSR blog post
[github] TikaIO JIRA number fix
[github] Added Cloud Spanner Connector
[hadarh] Add 'how to run' directions to WordCount for all runners.
[dhalperi] remove note about JIRA
[dhalperi] Changing Aggregators to Metrics in capability matrix
[dhalperi] fixups
[dhalperi] Do not use BeautifulSoup prettify, which is known to corrupt HTML
[chamikara] Updates contribution guide to include development and testing
[iemejia] Add HadoopInputFormatIO example to read from Hive's HCatalog
[klk] Add support for "no-toggle" code snippets
[klk] Add Beam Runner Guide
[iemejia] Update IO with CassandraIO
[iemejia] Add Hive (HCatalog) to the list of available I/O transforms
[altay] Mention python setup.py sdist and link to its docs
[altay] [BEAM-2437] Input path should be flexible in quickstart
[iemejia] Add Amazon DynamoDB example using HadoopInputFormatIO
[aljoscha.krettek] Update to more recent Flink version in example
[aljoscha.krettek] Add note about Scala version when setting up Flink cluster
[github] Remove references to Travis.
[github] No longer working title
[mergebot] Update build-in IO list with AMQP
[mergebot] Remove AMQPIO from in-progress list
[mergebot] Fix missing closing element
[mergebot] Add JingsongLee as committer
[mergebot] Restore index.html and fix order
[thw] update apex runner properties example
[github] Remove incorrect info from testing page.
[github] Drop the word reviewer from the contribution guide
[mariagh] Fix broken links
[mariagh] Update to latest version
[mariagh] Address review comments
[mariagh] Address review comments
[mergebot] Includes Splittable DoFn in the capability matrix
[mergebot] [BEAM-2500] List Amazon S3 File System as a planned I/O.
[mergebot] Port of Google doc
[mergebot] IO Testing, unit tests: update after readthrough
[mergebot] fixup! IO Testing, unit tests: update after readthrough
[mergebot] fixup! fixup! IO Testing, unit tests: update after readthrough
[mergebot] [BEAM-2638] Add Elasticsearch (v2 and upcoming v5) and MQTT to Built-in
[mergebot] [BEAM-2126] Add JStorm runner to ongoing projects.
[mergebot] Add initial text for I/O Integration Test documentation
[mergebot] fixup! Add initial text for I/O Integration Test documentation
[mergebot] Updated IO IT docs based on PR feedback
[mergebot] Update JDBC IO IT docs based on PR feedback.
[kirpichov] Update a dead link to Jenkins builds.
[kirpichov] Add Apache Solr to the list of Built-In I/O Transforms
[kirpichov] A couple of fixes to website
[kirpichov] Expands the sections on Coders and Validation in Style Guide
[mxm] Update company affiliation
[azur] Update PMC
[altay] Update PMC
[mergebot] Add MapReduce and Tez runners to ongoing projects.
[mergebot] Splittable DoFn blog post.
[mergebot] Add Manu Zhang to committers list
[mergebot] add a page for DSLs:SQL
[mergebot] fix typo issues
[mergebot] add intro of BeamSqlRow, trigger settings of aggregation/join,
[mergebot] typo fix
[mergebot] change page dsls/sql.md into a blog post
[mergebot] split into 1 blog + 1 development doc.
[mergebot] fix broken link
[mergebot] update to latest API/usages
[mergebot] remove post page move internal imple to the end of page
[altay] Publish release 2.1.0
[altay] Fix quote
[mergebot] [BEAM-2780] Fix Window sample code in programming guide
[takidau] [BEAM-2800] Revamp sections 1 and 2 of DSL SQL docs: - Expand BeamRecord
[takidau] Remove TODO
[altay] Add tfrecord io to builtin io list
[mingmxu] add SQL grammar in BNF
[mingmxu] correct the syntax of ORDER_BY LIMIT
[klk] Add blog post with timely processing
[kirpichov] Adds guidance on how to test transform configuration/validation
[altay] [rebase] add MingminXu to team.md
[altay] Add James Xu(xumingming) as committer
[altay] Add Reuven Lax to committers list.
[kirpichov] Fixes SDF blog post after rename of Match to FileIO.match
[altay] Document stale PR policy
[huafengw] Update Gearpump runner related pages
[melissapa] [BEAM-2510] Link to SDK pages from the quickstarts
[mergebot] [BEAM-2869] Add dates to releases on downloads page
[mergebot] [BEAM-1534] Add docker images documentation to the website
[mergebot] [BEAM-2726] Fix typo in programming guide
[mergebot] [BEAM-893] Update mobile gaming example
[mergebot] Update with review feedback
[mergebot] [BEAM-2839, BEAM-2838] Add MapReduce runner to Beam asf-site.
[mergebot] fixup: address comments.
[mergebot] fixup: address comments.
[mergebot] fixup: address comments.
[melissapa] [BEAM-667] Verify and update wordcount snippets
[melissapa] Update with review feedback
[melissapa] Update with review feedback
[melissapa] [BEAM-2039] Number programming guide chapters
[melissapa] Updates with review feedback
[mergebot] Update Mapreduce capability matrix when/how entries
[mergebot] Remove extraneous type variable from StateSpec in state blog post
[melissapa] Add Cloud Resource Manager API to the required API list
[melissapa] Add new execution model page and associated images
[melissapa] Update with review feedback and new diagrams
[melissapa] More updates from review feedback
[basti.lj] [BEAM-2666] Add JStorm runner to Beam asf-site.
[mergebot] Update source-repository.md
[mergebot] [BEAM-2335] Document how to run NeedsRunner tests.
[mergebot] fixup! [BEAM-2335] Document how to run NeedsRunner tests.
[mergebot] Manually add missed files from content dir
[iemejia] [BEAM-2727] Add build information to the vote email template
[altay] [BEAM-2853] Move the nexmark documentation to the website
[altay] fixups
[melissapa] Fix dead link
[mergebot] [MINOR] Updated hero and cards background to use background-size: cover
[jbonofre] Add RedisIO in the built-in set
[jbonofre] Move Elasticsearch v5.x from in-progress to built-in
[altay] [BEAM-664] Update docs: MinimalWordCount in Java is intentionally
[herohde] [BEAM-3121] Remove broken docker script and documentation
[klk] New top menu plus side nav layout
[mergebot] [BEAM-1934] Add more CoGroupByKey content/examples
[mergebot] Update with Java snippet tags
[mergebot] Add melap as committer
[mergebot] Add page for the portability framework
[mergebot] Add portability to contribute section
[mergebot] Update PMC
[mergebot] [BEAM-3123] Update status of BEAM-1035 and BEAM-1037 in Nexmark.
[mergebot] Update IntelliJ Checkstyle instructions
[mergebot] Address
[mergebot] Address
[pgerver] [BEAM-3162] Add IBM Streams Runner to compatability matrix
[github] Update testing docs to account for Python.
[echauchot] Add Etienne Chauchot as a committer
[jbonofre] Update SCM URL to use gitbox instead of git-wip-us
[echauchot] [BEAM-2853] Fix left menu in nexmark page
[echauchot] [BEAM-2853] Move nexmark page to documentation/sdks/java/nexmark
[melissapa] Fix ptransform guide links to use site variable
[altay] Capabilities Matrix: Stateful processing support in Apex Runner
[relax] documentation changes for 2.2.0-RC1
[relax] Cherry pick for pr/3578
[herohde] Add links to the Go SDK
[mergebot] Add Reuven Lax as PMC member.
[mergebot] Update release date for 2.2.0 release
[melissapa] Fix broken SQL branch links
[klk] Move reporter.apache.org step to finalize section
[klk] Expand instructions for pushing release version tag
[klk] Make contribution-guide.md markdown readable
[klk] Revamp the contribution guide
[mergebot] [BEAM-3338] Add SDK specific pages to SDK left nav
[mergebot] [BEAM-3374] Fix missing/stretched images, improve alt text
[mergebot] [BEAM-2664] Make latest stable version more visible
[mergebot] Adding update to Wikipedia to release guide.
[mergebot] Fixed an example of shading the application jar to it run on Spark
[mergebot] Add link to Scio (Scala) interface.
[mergebot] Manually set hight of scala logo.
[iemejia] [BEAM-3467] Remove Java 7 from the docker development images
[melissapa] Fix broken JIRA component links
[mergebot] fix language tabs after a language specific paragraph
[melissapa] [BEAM-3117] Fix reference to bundles and windows
[jbonofre] Add 2017 look back blog post
[melissapa] Fix broken links due to code directory moves
[github] Replaces sources/sinks in programming guide with IOs
[jbonofre] Announce 2.3.0 release
[mergebot] Add blog post for the release of Beam 2.3.0
[aromanenko.dev] Fixed number of contributors and duplication of name of Alexey Romanenko
[taro.murao] Java variable names should be in CamelCase
[mergebot] Clarify runtime type checking documentation
[mergebot] Add nightly snapshot build to the release guide
[mergebot] [BEAM-3430] Update workdcount example for Java8 for Java SDK
[mergebot] Add procedure to create a self executing Jar for DataFlow
[mergebot] Don't pass the pom.xml as an input
[mergebot] Updates list of built-in IO transforms
[klk] Update Gemfile.lock
[klk] Explicitly define section id due to kramdown id generation changes
[altay] Update site to mention the 2 Python lint envs
[lcwik] [BEAM-3846] Add a link to be able to self join the Apache Beam slack
[lcwik] [BEAM-3852] Update slack channel link and slack invite for new channel.
[robertwb] Update instructions on cryptographic hashes.
[mergebot] Increase link checking timeouts
[mergebot] Update SQL doc to match new APIs
[mergebot] Update current release to 2.4.0.
[mergebot] Update download page (including signature and checksum).
[mergebot] Add instructions on how to build and use the Python tarball.
[mergebot] Update nexmark support matrix for timer and state APIs
[echauchot] Make minor fixes to Nexmark page
[echauchot] Use {{ site.release_latest }} instead of [version]
[echauchot] Make gs paths coherent for runs on dataflow runner
[mergebot] expand and collpase navigation side bar new feature
[mergebot] mark active item on navigation side bar
[melissapa] [BEAM-4026] Remove link to deleted Go SDK branch
[melissapa] parallelise test
[mergebot] [BEAM-3252] Updating contributor's guide to reference Gradle.
[mergebot] Fix project names to reflect changes required for release.
[mergebot] Drop using gradle clean in favor of --rerun-tasks
[mergebot] [BEAM-3252] Adding a step to the IntelliJ setup for gradle.
[mergebot] [BEAM-3252] Removing Annotation Processing section.
[mergebot] Stop recommending we use auto-import
[mergebot] Update checkstyle comment to correspond to the current version
[mergebot] Update instructions to run linter.
[mergebot] Change first occurance of MyOptions to PipelineOptions
[ehudm] Document Gradle command for testing Beam SDK.
[melissapa] [BEAM-4101] Fix broken links
[melissapa] Update Eclipse documentation for Gradle build.
[melissapa] [BEAM-3036] Adding Checkstyle version warning.
[melissapa] Add TODO with linked JIRA to update Docker documentation for Gradle.
[robertwb] Ignore jstorm.io to unblock presubmits.
[mergebot] Friendlier contribution guide; link to starter tasks
[mergebot] Cleanup now that GitHub is not the mirror.
[mergebot] Update contribution guide to reflect current practices.
[mergebot] Further refinement.
[mergebot] Update Nexmark launch instructions for Gradle
[lukasz.gajowy] [BEAM-4119] Update i/o testing documentation to use gradle
[mergebot] Add Jason Kuster as Committer
[github] Adding pabloem to the list of team members.
[github] Sorting by last name
[github] Adding gris as committer as well.
[mergebot] Document Java extensions for parsing Apache HTTPD logfiles and Useragent
[mergebot] Fixed some layout issues
[mergebot] Moved the 3rd party extensions to a separate page
[mergebot] [BEAM-4177] Clarify thread contraint in Programming Guide 4.3.2
[coheigea] Typo fix in the programming guide
[klk] Add "Become a Committer" with PMC guidelines
[klk] Remove Team page in favor of link to ASF roster
[klk] Remove/correct references to Team page
[klk] Do not validate links from datatorrent.com
[melissapa] Updates for Python streaming
[klk] Add top-level Community section
[klk] Remove obsolete Maturity Model section
[klk] Move Release Guide to committers section
[klk] Move committer section to the bottom (we can find it :-)
[klk] Streamline the contribution section
[klk] Google has scrapped research, now only AI
[klk] Temporarily suppress validation of ai.google links
[mergebot] Add link to portability support sheet
[valentyn] Add a Python 3 entry to the list of ongoing projects.
[mergebot] Add Go SDK quickstart and minimal documentation page
[mergebot] Add Go to the Wordcount example
[mergebot] Fix mailing list links and docker image
[github] Update index.md
[klk] Reorganize SQL documentation into sections
[swegner] Remove DoFnTester from testing guide.
[mergebot] Fix typo in becoming-a-committer docs
[timrobertson100] [BEAM-4361] Document usage of HBase TableSnapshotInputFormat
[mergebot] Add Apache Kudu IO to the planned IOs table
[mergebot] Add next Java LTS to the WIP section of the contribution guide
[mergebot] Add PubSubIO to Python IO table row
[mergebot] Change ParquetIO status in built-in I/O section
[lcwik] Add link to design docs and template to the contribution guide.
[lukasz.gajowy] Add Performance Testing to "Work in Progress" section
[marek.simunek] [BEAM-4368] add euphoria java 8 dsl documentation section
[aromanenko.dev] Added design docs page
[aromanenko.dev] Added first intro sentence.
[aromanenko.dev] Moved one doc to other section and added several new docs
[aromanenko.dev] Added more docs
[mergebot] Add SQL area to ongoing projects
[mergebot] [BEAM-4488] How to install Python SDK from source [BEAM-4489] Document
[mergebot] More changes
[mergebot] updates after review
[mergebot] Updating release guide to use Gradle commands.
[mergebot] Addressing comments. Improving commands.
[mergebot] Addressing comments. Improving commands.
[mergebot] Update release guide for publishing source zip from GitHub
[mergebot] Addressing TODOs
[mergebot] Address comments.
[mergebot] Improving.
[mergebot] Add more updates
[mergebot] More updates
[mergebot] Addressed Alan's comments
[mergebot] Add instructions about how to stage python binaries
[mergebot] copy renamed zi[ file to sdks/python/staging subdir
[mergebot] Update gradle publish with --no-parallel --no-daemon
[mergebot] Add Python Kafka to list of in-progress I/O transforms.
[echauchot] Add Etienne Chauchot's ongoing work to the contribute page.
[mergebot] [BEAM-2852] Update Nexmark documentation with Kafka support
[mergebot] [BEAM-4479] Fixed document for Coder
[swegner] Fix ErrorProne violations in fn harness and enable failOnWarnings
[swegner] fixup! Fix ErrorProne violations in fn harness and enable failOnWarnings
[swegner] Temporarily exclude website sources from RAT
[swegner] fixup! Fix ErrorProne violations in fn harness and enable failOnWarnings
[iemejia] [BEAM-4570] Use pagination in
[apilloud] [SQL] Add parse code for SET
[apilloud] [SQL] Plumb PipelineOptions through BeamRelNode
[apilloud] [SQL] Plumb PipelineOptions through BeamSqlCli
[kirpichov] Propagates artifact retrieval token in Flink runner and to the Java
[ankurgoenka] load beam fileSystems in FnHarness
[axelmagn] Implement new staging token contract in java SDK
[robertwb] Add Impulse primitive to Python SDK
[robertwb] Expand Read to Impulse | ParDo(Split) | ParDo(Read) in Python SDK
[robertwb] Fix default docker image name.
[apilloud] [SQL] Set application name to BeamSql
[apilloud] [SQL] Correctly represent pipeline
[apilloud] [SQL] Cleanup table IO functions
[apilloud] [SQL] Simplify overly verbose names
[apilloud] [SQL] Add each Calcite node to Beam graph once
[apilloud] [SQL] Seperate parse from pipeline construction
[swegner] fixup! Fix ErrorProne violations in fn harness and enable failOnWarnings
[lcwik] [BEAM-4592] Make Dataflow understand kind:varint as a well known since
[chamikara] Fixes null pointer exception on empty pubsub message data (#5016)
[lcwik] Update Dataflow container images
[robertwb] Update impulse-based read.
[kenn] Make BeamRelNode take a PCollectionList<Row> always
------------------------------------------
[...truncated 3.84 MB...]
INFO: Stopping Akka RPC service.
Jun 20, 2018 5:21:02 PM org.apache.flink.runtime.blob.AbstractBlobCache close
INFO: Shutting down BLOB cache
Jun 20, 2018 5:21:02 PM org.apache.flink.runtime.blob.AbstractBlobCache close
INFO: Shutting down BLOB cache
Jun 20, 2018 5:21:02 PM org.apache.flink.runtime.blob.BlobServer close
INFO: Stopped BLOB server at 0.0.0.0:33283
Jun 20, 2018 5:21:02 PM org.apache.flink.runtime.rpc.akka.AkkaRpcService lambda$stopService$4
INFO: Stopped Akka RPC service.
Jun 20, 2018 5:21:02 PM org.apache.beam.runners.flink.FlinkRunner run
SEVERE: Pipeline execution failed
org.apache.flink.runtime.client.JobExecutionException: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins0620171721eb26bd5b_6f1d33e4e1ba4b86a9f97a8c2d34393d_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:625)
at org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:234)
at org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:91)
at org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:114)
at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:116)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:311)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
at org.apache.beam.sdk.nexmark.Main.savePerfsToBigQuery(Main.java:182)
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:102)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:395)
Caused by: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins0620171721eb26bd5b_6f1d33e4e1ba4b86a9f97a8c2d34393d_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:36)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Unable to insert job: beam_load_mainjenkins0620171721eb26bd5b_6f1d33e4e1ba4b86a9f97a8c2d34393d_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:231)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Invalid field name \"Runtime(sec)\". Fields must contain only letters, numbers, and underscores, start with a letter or underscore, and be at most 128 characters long.",
"reason" : "invalid"
} ],
"message" : "Invalid field name \"Runtime(sec)\". Fields must contain only letters, numbers, and underscores, start with a letter or underscore, and be at most 128 characters long."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1065)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:216)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
==========================================================================================
Exception in thread "main" Run started 2018-06-20T17:17:20.075Z and ran for PT222.058S
java.lang.RuntimeException: Pipeline execution failed
at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:119)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:311)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
at org.apache.beam.sdk.nexmark.Main.savePerfsToBigQuery(Main.java:182)
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:102)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:395)
Default configuration:
Caused by: org.apache.flink.runtime.client.JobExecutionException: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins0620171721eb26bd5b_6f1d33e4e1ba4b86a9f97a8c2d34393d_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:625)
at org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:234)
{"debug":true,"query":0,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
at org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:91)
0000 query:0; exportSummaryToBigQuery:true; streamTimeout:60
at org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:114)
0001 query:1; exportSummaryToBigQuery:true; streamTimeout:60
0002 query:2; exportSummaryToBigQuery:true; streamTimeout:60
at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:116)
... 5 more
0003 query:3; exportSummaryToBigQuery:true; streamTimeout:60
Caused by: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: Unable to insert job: beam_load_mainjenkins0620171721eb26bd5b_6f1d33e4e1ba4b86a9f97a8c2d34393d_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
at org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:36)
0004 query:4; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
0005 query:5; exportSummaryToBigQuery:true; streamTimeout:60
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
0006 query:6; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
0007 query:7; exportSummaryToBigQuery:true; streamTimeout:60
0008 query:8; exportSummaryToBigQuery:true; streamTimeout:60
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
0009 query:9; exportSummaryToBigQuery:true; numEvents:10000; streamTimeout:60
Caused by: java.io.IOException: Unable to insert job: beam_load_mainjenkins0620171721eb26bd5b_6f1d33e4e1ba4b86a9f97a8c2d34393d_6831ef916a10e30d9ef1c0e5f0e1ec7d_00001_00000-0, aborting after 9 .
0010 query:10; exportSummaryToBigQuery:true; streamTimeout:60
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:231)
0011 query:11; exportSummaryToBigQuery:true; streamTimeout:60
0012 query:12; exportSummaryToBigQuery:true; streamTimeout:60
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
Performance:
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline) at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
0000 0.9 115074.8 100000
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
0001 0.6 172117.0 92000
{
"code" : 400,
"errors" : [ {
"domain" : "global",
0002 0.5 207468.9 351
"message" : "Invalid field name \"Runtime(sec)\". Fields must contain only letters, numbers, and underscores, start with a letter or underscore, and be at most 128 characters long.",
"reason" : "invalid"
} ],
0003 11.6 8655.8 580
"message" : "Invalid field name \"Runtime(sec)\". Fields must contain only letters, numbers, and underscores, start with a letter or underscore, and be at most 128 characters long."
}
0004 5.7 1744.0 40
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
0005 5.8 17208.7 12
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
0006 5.6 1798.9 103
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClie 0007 6.9 14405.1 1
ntRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
0008 7.4 13426.4 6000
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321)
0009 5.7 1752.2 298
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1065)
0010 6.6 15232.3 1
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
0011 8.0 12534.5 1919
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
0012 6.0 16677.8 1919
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
==========================================================================================
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:216)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:202)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startLoadJob(BigQueryServicesImpl.java:142)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.load(WriteTables.java:269)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables.access$600(WriteTables.java:80)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:159)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:185)
at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:146)
at org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.processElement(DoFnRunnerWithMetricsUpdate.java:66)
at org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction.mapPartition(FlinkDoFnFunction.java:120)
at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:748)
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 14,5,main]) completed. Took 3 mins 45.741 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_152/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
See https://docs.gradle.org/4.8/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 6m 36s
62 actionable tasks: 58 executed, 4 from cache
Publishing build scan...
https://gradle.com/s/57zb3spqkum5q
Build cache (/home/jenkins/.gradle/caches/build-cache-1) remove files older than Wed Jun 13 17:14:30 UTC 2018.
Build cache (/home/jenkins/.gradle/caches/build-cache-1) removing 232 cache entries (26 MB reclaimed).
Build cache (/home/jenkins/.gradle/caches/build-cache-1) cleaned up in 0.035 secs.
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user marek.simunek@firma.seznam.cz
Not sending mail to unregistered user sourabhbajaj@google.com
Not sending mail to unregistered user lukasz.gajowy@polidea.com
Not sending mail to unregistered user mergebot@apache.org
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user thomas.haines@practiceinsight.io
Not sending mail to unregistered user axelmagn@gmail.com
Not sending mail to unregistered user ddonnelly@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user relax@relax-macbookpro2.roam.corp.google.com
Not sending mail to unregistered user devon.meunier@shopify.com
Not sending mail to unregistered user timrobertson100@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user mrtn@nrd.io
Not sending mail to unregistered user basti.lj@alibaba-inc.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user azur@paypal.com
Not sending mail to unregistered user jamalone@gmail.com
Not sending mail to unregistered user terrencehan@tencent.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user hadarh@google.com
Not sending mail to unregistered user fjp@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user pgerver@us.ibm.com
Not sending mail to unregistered user jasonkuster@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user taro.murao@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user wnsdud1861@gmail.com
Not sending mail to unregistered user peihe0@gmail.com
Not sending mail to unregistered user sisk@google.com