You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2019/10/24 22:21:40 UTC
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#4101
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/4101/display/redirect?page=changes>
Changes:
[xinyuliu.us] [BEAM-8341]: basic bundling support for portable runner (#9777)
------------------------------------------
[...truncated 53.35 KB...]
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true; numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:1000000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:1000000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true; numEvents:10000000
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true; numEvents:10000000
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true; numEvents:10000000
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true; numEvents:10000000
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true; numEvents:10000000
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true; numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 49.1 203823.7 10000000
0001 46.3 215838.2 9200000
0002 *** not run ***
0003 *** not run ***
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 39.5 252908.4 92833
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 35.9 278800.0 199929
0013 *** not run ***
0014 *** not run ***
==========================================================================================
2019-10-24T22:16:23.978Z Waiting for main pipeline to 'finish'
2019-10-24T22:16:23.990Z Waiting for main pipeline to 'finish'
2019-10-24T22:16:24.023Z RUNNING Query2
2019-10-24T22:16:24.045Z RUNNING BoundedSideInputJoin
2019-10-24T22:16:24.817Z new perf {"snapshots":null,"runtimeSec":28.707,"numEvents":10000000,"eventsPerSec":348347.09304350853,"eventBytesPerSec":4.471261006026404E7,"numResults":83527,"resultsPerSec":2909.638764064514,"resultBytesPerSec":46554.22022503222,"startupDelaySec":-174.935,"processingDelaySec":0.729,"shutdownDelaySec":146.234,"timeDilation":34.8326192217926,"errors":null,"jobId":null}
2019-10-24T22:16:25.371Z new perf {"snapshots":null,"runtimeSec":120.17,"numEvents":10000000,"eventsPerSec":83215.44478655238,"eventBytesPerSec":1.068124238162603E7,"numResults":9200000,"resultsPerSec":76558.2092036282,"resultBytesPerSec":6556809.478239161,"startupDelaySec":-181.15,"processingDelaySec":61.656,"shutdownDelaySec":60.98,"timeDilation":8.320961970541733,"errors":null,"jobId":null}
2019-10-24T22:16:27.083Z Waiting for main pipeline to 'finish'
2019-10-24T22:16:27.136Z RUNNING BoundedSideInputJoin
2019-10-24T22:16:28.139Z new perf {"snapshots":null,"runtimeSec":46.369,"numEvents":10000000,"eventsPerSec":215661.32545450624,"eventBytesPerSec":2.7681530699389677E7,"numResults":9200000,"resultsPerSec":198408.41941814576,"resultBytesPerSec":7072220.556837542,"startupDelaySec":-151.351,"processingDelaySec":0.858,"shutdownDelaySec":104.982,"timeDilation":21.565054238823354,"errors":null,"jobId":null}
2019-10-24T22:16:36.545Z Waiting for main pipeline to 'finish'
2019-10-24T22:16:36.545Z DONE Query3
==========================================================================================
Run started 2019-10-24T22:10:39.112Z and ran for PT357.523S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true; numEvents:10000000
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true; numEvents:10000000
0002 query:SELECTION; exportSummaryToBigQuery:true; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true; numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:1000000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:1000000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true; numEvents:10000000
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true; numEvents:10000000
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true; numEvents:10000000
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true; numEvents:10000000
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true; numEvents:10000000
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true; numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 49.1 203823.7 10000000
0001 46.3 215838.2 9200000
0002 *** not run ***
0003 40.9 244379.3 59507
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 39.5 252908.4 92833
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 35.9 278800.0 199929
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Exception in thread "main" ==========================================================================================
Run started 2019-10-24T22:10:39.112Z and ran for PT358.576S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true; numEvents:10000000
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true; numEvents:10000000
0002 query:SELECTION; exportSummaryToBigQuery:true; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true; numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:1000000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:1000000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true; numEvents:10000000
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true; numEvents:10000000
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true; numEvents:10000000
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true; numEvents:10000000
java.lang.RuntimeException: java.lang.RuntimeException: Failed to create a workflow job: Quota exceeded for quota metric 'dataflow.googleapis.com/create_requests' and limit 'CreateRequestsPerMinutePerUser' of service 'dataflow.googleapis.com' for consumer 'project_number:844138762903'.
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true; numEvents:10000000
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:128)
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true; numEvents:10000000
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 49.1 203823.7 10000000
0001 46.3 215838.2 9200000
0002 *** not run ***
0003 40.9 244379.3 59507
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 39.5 252908.4 92833
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 35.9 278800.0 199929
0013 *** not run ***
0014 *** not run ***
==========================================================================================
Caused by: java.lang.RuntimeException: Failed to create a workflow job: Quota exceeded for quota metric 'dataflow.googleapis.com/create_requests' and limit 'CreateRequestsPerMinutePerUser' of service 'dataflow.googleapis.com' for consumer 'project_number:844138762903'.
at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:941)
at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:186)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:315)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:301)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1178)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:90)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 429 Too Many Requests
{
"code" : 429,
"errors" : [ {
"domain" : "global",
"message" : "Quota exceeded for quota metric 'dataflow.googleapis.com/create_requests' and limit 'CreateRequestsPerMinutePerUser' of service 'dataflow.googleapis.com' for consumer 'project_number:844138762903'.",
"reason" : "rateLimitExceeded"
} ],
"message" : "Quota exceeded for quota metric 'dataflow.googleapis.com/create_requests' and limit 'CreateRequestsPerMinutePerUser' of service 'dataflow.googleapis.com' for consumer 'project_number:844138762903'.",
"status" : "RESOURCE_EXHAUSTED"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:401)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1097)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:499)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:549)
at org.apache.beam.runners.dataflow.DataflowClient.createJob(DataflowClient.java:61)
at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:927)
... 12 more
2019-10-24T22:16:39.891Z DONE Query2
2019-10-24T22:16:39.973Z Waiting for main pipeline to 'finish'
2019-10-24T22:16:40.043Z RUNNING Query5
2019-10-24T22:16:40.406Z RUNNING BoundedSideInputJoin
2019-10-24T22:16:40.952Z new perf {"snapshots":null,"runtimeSec":62.042,"numEvents":10000000,"eventsPerSec":161181.13535991748,"eventBytesPerSec":2.0688644740659554E7,"numResults":4567,"resultsPerSec":73.61142451887432,"resultBytesPerSec":1177.782792301989,"startupDelaySec":-183.919,"processingDelaySec":61.894,"shutdownDelaySec":121.877,"timeDilation":16.118113535991746,"errors":null,"jobId":null}
2019-10-24T22:16:41.696Z no activity
2019-10-24T22:16:42.807Z Waiting for main pipeline to 'finish'
2019-10-24T22:16:42.807Z DONE Query9
2019-10-24T22:16:43.210Z RUNNING BoundedSideInputJoin
2019-10-24T22:16:44.314Z no activity
2019-10-24T22:16:49.621Z Waiting for main pipeline to 'finish'
2019-10-24T22:16:49.624Z Waiting for main pipeline to 'finish'
2019-10-24T22:16:49.656Z RUNNING Query10
2019-10-24T22:16:49.658Z RUNNING Query7
2019-10-24T22:16:50.564Z new perf {"snapshots":null,"runtimeSec":86.803,"numEvents":10000000,"eventsPerSec":115203.39158784835,"eventBytesPerSec":1.4787102945750723E7,"numResults":100,"resultsPerSec":1.1520339158784836,"resultBytesPerSec":116.38998652120318,"startupDelaySec":-177.957,"processingDelaySec":68.067,"shutdownDelaySec":91.154,"timeDilation":11.329101528749007,"errors":null,"jobId":null}
2019-10-24T22:16:50.746Z new perf {"snapshots":null,"runtimeSec":92.88,"numEvents":10000000,"eventsPerSec":107665.80534022395,"eventBytesPerSec":1.381960483419466E7,"numResults":100,"resultsPerSec":1.0766580534022394,"resultBytesPerSec":49.52627045650302,"startupDelaySec":-178.559,"processingDelaySec":92.608,"shutdownDelaySec":85.679,"timeDilation":10.658914728682172,"errors":null,"jobId":null}
2019-10-24T22:16:56.044Z DONE Query5
2019-10-24T22:16:56.752Z RUNNING BoundedSideInputJoin
2019-10-24T22:16:57.881Z no activity
2019-10-24T22:16:59.393Z RUNNING BoundedSideInputJoin
2019-10-24T22:17:00.392Z no activity
2019-10-24T22:17:05.640Z RUNNING Query7
2019-10-24T22:17:05.811Z RUNNING Query10
2019-10-24T22:17:06.679Z no activity
2019-10-24T22:17:06.714Z no activity
2019-10-24T22:17:12.965Z RUNNING BoundedSideInputJoin
2019-10-24T22:17:13.365Z Waiting for main pipeline to 'finish'
2019-10-24T22:17:13.399Z RUNNING Query6
2019-10-24T22:17:14.163Z no activity
2019-10-24T22:17:14.243Z new perf {"snapshots":null,"runtimeSec":41.962,"numEvents":1000000,"eventsPerSec":23831.085267623086,"eventBytesPerSec":3059150.3741480387,"numResults":8902,"resultsPerSec":212.14432105238072,"resultBytesPerSec":3394.3091368380915,"startupDelaySec":-173.447,"processingDelaySec":41.455,"shutdownDelaySec":131.485,"timeDilation":-1.0,"errors":null,"jobId":null}
2019-10-24T22:17:15.462Z RUNNING BoundedSideInputJoin
2019-10-24T22:17:16.347Z no activity
2019-10-24T22:17:21.722Z RUNNING Query10
2019-10-24T22:17:21.797Z RUNNING Query7
2019-10-24T22:17:22.575Z no activity
2019-10-24T22:17:22.852Z no activity
2019-10-24T22:17:29.226Z RUNNING BoundedSideInputJoin
2019-10-24T22:17:29.308Z RUNNING Query6
2019-10-24T22:17:30.269Z no activity
2019-10-24T22:17:30.414Z no activity
2019-10-24T22:17:31.404Z DONE BoundedSideInputJoin
2019-10-24T22:17:37.624Z DONE Query10
2019-10-24T22:17:37.955Z DONE Query7
2019-10-24T22:17:45.411Z DONE Query6
2019-10-24T22:17:45.478Z DONE BoundedSideInputJoin
2019-10-24T22:20:49.849Z Waiting for main pipeline to 'finish'
2019-10-24T22:20:49.887Z RUNNING Query11
2019-10-24T22:20:50.759Z new perf {"snapshots":null,"runtimeSec":53.958,"numEvents":10000000,"eventsPerSec":185329.33021980058,"eventBytesPerSec":2.3788222265465733E7,"numResults":1839657,"resultsPerSec":34094.23996441677,"resultBytesPerSec":545507.8394306683,"startupDelaySec":-174.377,"processingDelaySec":40.128,"shutdownDelaySec":120.419,"timeDilation":18.53200637532896,"errors":null,"jobId":null}
2019-10-24T22:21:05.916Z RUNNING Query11
2019-10-24T22:21:06.708Z no activity
2019-10-24T22:21:21.767Z RUNNING Query11
2019-10-24T22:21:22.699Z no activity
2019-10-24T22:21:37.800Z DONE Query11
> Task :sdks:java:testing:nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:nexmark:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 12m 4s
80 actionable tasks: 60 executed, 20 from cache
Publishing build scan...
https://gradle.com/s/5x2dvrjin3rdk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal :
beam_PostCommit_Java_Nexmark_Dataflow #4102
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/4102/display/redirect>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org