You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/08/16 16:50:40 UTC
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #281
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/281/display/redirect?page=changes>
Changes:
[pablo] Make .gitignore rules work correctly with rat.
------------------------------------------
[...truncated 710.13 KB...]
Aug 16, 2018 4:49:56 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON `B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Aug 16, 2018 4:49:56 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4])
LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], extra=[$4], starttime=[$5])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 10000)])
BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 10000)], price=[$2])
BeamIOSourceRel(table=[[beam, Bid]])
Aug 16, 2018 4:49:56 PM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCalcRel(expr#0..5=[{inputs}], proj#0..5=[{exprs}])
BeamAggregationRel(group=[{0, 1, 2, 3, 4, 5}])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[10000], expr#6=[TUMBLE($t3, $t5)], auction=[$t0], price=[$t2], bidder=[$t1], dateTime=[$t3], extra=[$t4], $f5=[$t6])
BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxprice=[$t1], starttime=[$t0])
BeamAggregationRel(group=[{0}], maxprice=[MAX($1)])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[10000], expr#6=[TUMBLE($t3, $t5)], $f0=[$t6], price=[$t2])
BeamIOSourceRel(table=[[beam, Bid]])
2018-08-16T16:50:35.084Z Waiting for main pipeline to 'finish'
2018-08-16T16:50:35.084Z DONE SqlQuery7
2018-08-16T16:50:35.085Z Running query:8; exportSummaryToBigQuery:true
2018-08-16T16:50:35.085Z skipping since configuration is not implemented
2018-08-16T16:50:35.085Z Running query:9; exportSummaryToBigQuery:true; numEvents:10000
2018-08-16T16:50:35.085Z skipping since configuration is not implemented
2018-08-16T16:50:35.085Z Running query:10; exportSummaryToBigQuery:true
2018-08-16T16:50:35.086Z skipping since configuration is not implemented
2018-08-16T16:50:35.086Z Running query:11; exportSummaryToBigQuery:true
==========================================================================================
Run started 2018-08-16T16:49:01.911Z and ran for PT93.174S
2018-08-16T16:50:35.086Z skipping since configuration is not implemented
Default configuration:
2018-08-16T16:50:35.086Z Running query:12; exportSummaryToBigQuery:true
{"debug":true,"query":0,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:0; exportSummaryToBigQuery:true
0001 query:1; exportSummaryToBigQuery:true
0002 query:2; exportSummaryToBigQuery:true
0003 query:3; exportSummaryToBigQuery:true
0004 query:4; exportSummaryToBigQuery:true; numEvents:10000
0005 query:5; exportSummaryToBigQuery:true
0006 query:6; exportSummaryToBigQuery:true; numEvents:10000
0007 query:7; exportSummaryToBigQuery:true
0008 query:8; exportSummaryToBigQuery:true
0009 query:9; exportSummaryToBigQuery:true; numEvents:10000
0010 query:10; exportSummaryToBigQuery:true
0011 query:11; exportSummaryToBigQuery:true
2018-08-16T16:50:35.086Z skipping since configuration is not implemented
0012 query:12; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 9.5 10545.2 92000
0001 10.5 9525.6 92000
0002 5.4 18375.6 351
0003 5.6 18011.5 580
0004 *** not run ***
0005 16.4 6087.5 12
0006 *** not run ***
0007 38.4 2601.3 1
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
==========================================================================================
Aug 16, 2018 4:50:35 PM org.apache.beam.sdk.io.gcp.bigquery.BatchLoads$4 getTempFilePrefix
INFO: Writing BigQuery temporary files to gs://temp-storage-for-perf-tests/nexmark/BigQueryWriteTemp/beam_load_mainjenkins0816165035c7e11f56_32b741d697b34d1cb512d376b94e7f56/ before loading them.
Aug 16, 2018 4:50:35 PM org.apache.beam.sdk.io.gcp.bigquery.TableRowWriter <init>
INFO: Opening TableRowWriter to gs://temp-storage-for-perf-tests/nexmark/BigQueryWriteTemp/beam_load_mainjenkins0816165035c7e11f56_32b741d697b34d1cb512d376b94e7f56/8dab66d8-9285-42ba-a6c9-a609845ae1ee.
Aug 16, 2018 4:50:35 PM org.apache.beam.sdk.io.gcp.bigquery.TableRowWriter <init>
INFO: Opening TableRowWriter to gs://temp-storage-for-perf-tests/nexmark/BigQueryWriteTemp/beam_load_mainjenkins0816165035c7e11f56_32b741d697b34d1cb512d376b94e7f56/f2b2e85a-7273-48a3-bb3e-6335b6a44cd8.
Aug 16, 2018 4:50:35 PM org.apache.beam.sdk.io.gcp.bigquery.TableRowWriter <init>
INFO: Opening TableRowWriter to gs://temp-storage-for-perf-tests/nexmark/BigQueryWriteTemp/beam_load_mainjenkins0816165035c7e11f56_32b741d697b34d1cb512d376b94e7f56/f27e3f54-b1a1-4779-a86a-6822d861a7ab.
Aug 16, 2018 4:50:35 PM org.apache.beam.sdk.io.gcp.bigquery.TableRowWriter <init>
INFO: Opening TableRowWriter to gs://temp-storage-for-perf-tests/nexmark/BigQueryWriteTemp/beam_load_mainjenkins0816165035c7e11f56_32b741d697b34d1cb512d376b94e7f56/e5fdd8d5-d1ca-4c0f-a5e3-227ac09a4169.
Aug 16, 2018 4:50:35 PM org.apache.beam.sdk.io.gcp.bigquery.TableRowWriter <init>
INFO: Opening TableRowWriter to gs://temp-storage-for-perf-tests/nexmark/BigQueryWriteTemp/beam_load_mainjenkins0816165035c7e11f56_32b741d697b34d1cb512d376b94e7f56/3f2ec898-24e2-404c-b33a-41b3e8e4a336.
Aug 16, 2018 4:50:35 PM org.apache.beam.sdk.io.gcp.bigquery.TableRowWriter <init>
INFO: Opening TableRowWriter to gs://temp-storage-for-perf-tests/nexmark/BigQueryWriteTemp/beam_load_mainjenkins0816165035c7e11f56_32b741d697b34d1cb512d376b94e7f56/61df5eed-e37d-4d55-9755-8a2ff07a085d.
Aug 16, 2018 4:50:37 PM org.apache.beam.sdk.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse
WARNING: Request failed with code 410, performed 0 retries due to IOExceptions, performed 1 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://www.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?name=nexmark/BigQueryWriteTemp/beam_load_mainjenkins0816165035c7e11f56_32b741d697b34d1cb512d376b94e7f56/8dab66d8-9285-42ba-a6c9-a609845ae1ee&uploadType=resumable&upload_id=AEnB2UqjdW--9U4wHlfHyqIqKLGcgAQVONEaT4Mv2ySNZ71yAItpPpKRyqGsBtLJQ9MyBWVsufNW0G2CwTghxCkyICMx-fP4XA
Aug 16, 2018 4:50:37 PM com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation call
SEVERE: Exception not convertible into handled response
com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
{
"code" : 503,
"errors" : [ {
"domain" : "global",
"message" : "Backend Error",
"reason" : "backendError"
} ],
"message" : "Backend Error"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
==========================================================================================
Run started 2018-08-16T16:49:01.911Z and ran for PT95.416S
Default configuration:
{"debug":true,"query":0,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:0; exportSummaryToBigQuery:true
0001 query:1; exportSummaryToBigQuery:true
0002 query:2; exportSummaryToBigQuery:true
0003 query:3; exportSummaryToBigQuery:true
Exception in thread "main" 0004 query:4; exportSummaryToBigQuery:true; numEvents:10000
0005 query:5; exportSummaryToBigQuery:true
0006 query:6; exportSummaryToBigQuery:true; numEvents:10000
0007 query:7; exportSummaryToBigQuery:true
0008 query:8; exportSummaryToBigQuery:true
0009 query:9; exportSummaryToBigQuery:true; numEvents:10000
0010 query:10; exportSummaryToBigQuery:true
0011 query:11; exportSummaryToBigQuery:true
0012 query:12; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 9.5 10545.2 92000
org.apache.beam.sdk.Pipeline$PipelineExecutionException: java.io.IOException: Failed to close some writers 0001 10.5 9525.6 92000
0002 5.4 18375.6 351
at org.apache.beam.runners.direct.DirectRunner$DirectPipelineResult.waitUntilFinish(DirectRunner.java:332)
0003 5.6 18011.5 580
at org.apache.beam.runners.direct.DirectRunner$DirectPipelineResult.waitUntilFinish(DirectRunner.java:302)
0004 *** not run ** at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:197)
*
at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:64)
0005 16.4 6087.5 12
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:313)
0006 *** not run ***
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:299)
0007 38.4 2601.3 1
at org.apache.beam.sdk.nexmark.Main.savePerfsToBigQuery(Main.java:250)
0008 *** not run ***
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:162)
0009 *** not run ***
at org.apache.beam.sdk.nexmark.Main.main(Main.java:477)
0010 *** not run ***
Caused by: java.io.IOException: Failed to close some writers
0011 *** not run ***
at org.apache.beam.sdk.io.gcp.bigquery.WriteBundlesToFiles.finishBundle(WriteBundlesToFiles.java:248)
0012 *** not run ***
==========================================================================================
Suppressed: java.io.IOException: com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
{
"code" : 503,
"errors" : [ {
"domain" : "global",
"message" : "Backend Error",
"reason" : "backendError"
} ],
"message" : "Backend Error"
}
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:432)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:287)
at org.apache.beam.sdk.io.gcp.bigquery.TableRowWriter.close(TableRowWriter.java:81)
at org.apache.beam.sdk.io.gcp.bigquery.WriteBundlesToFiles.finishBundle(WriteBundlesToFiles.java:242)
at org.apache.beam.sdk.io.gcp.bigquery.WriteBundlesToFiles$DoFnInvoker.invokeFinishBundle(Unknown Source)
at org.apache.beam.repackaged.beam_runners_direct_java.runners.core.SimpleDoFnRunner.finishBundle(SimpleDoFnRunner.java:285)
at org.apache.beam.repackaged.beam_runners_direct_java.runners.core.SimplePushbackSideInputDoFnRunner.finishBundle(SimplePushbackSideInputDoFnRunner.java:118)
at org.apache.beam.runners.direct.ParDoEvaluator.finishBundle(ParDoEvaluator.java:225)
at org.apache.beam.runners.direct.DoFnLifecycleManagerRemovingTransformEvaluator.finishBundle(DoFnLifecycleManagerRemovingTransformEvaluator.java:74)
at org.apache.beam.runners.direct.DirectTransformExecutor.finishBundle(DirectTransformExecutor.java:188)
at org.apache.beam.runners.direct.DirectTransformExecutor.run(DirectTransformExecutor.java:126)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
{
"code" : 503,
"errors" : [ {
"domain" : "global",
"message" : "Backend Error",
"reason" : "backendError"
} ],
"message" : "Backend Error"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
... 4 more
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 2,5,main]) completed. Took 1 mins 36.875 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_152/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
See https://docs.gradle.org/4.8/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 49s
62 actionable tasks: 2 executed, 60 up-to-date
Publishing build scan...
https://gradle.com/s/wpiyreapaw7w2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
Jenkins build is back to normal :
beam_PostCommit_Java_Nexmark_Direct #282
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/282/display/redirect?page=changes>