You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/04/12 14:11:02 UTC
Build failed in Jenkins:
beam_LoadTests_Java_CoGBK_SparkStructuredStreaming_Batch #361
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_SparkStructuredStreaming_Batch/361/display/redirect?page=changes>
Changes:
[noreply] docs: incremental --> patch
[anant.damle] [BEAM-12151] Bump Apache Parquet to 1.12.0
[Elliotte Rusty Harold] use Oxford comma
------------------------------------------
[...truncated 37.93 KB...]
21/04/12 14:05:51 INFO org.apache.beam.runners.spark.structuredstreaming.SparkStructuredStreamingRunner: *** SparkStructuredStreamingRunner is based on spark structured streaming framework and is no more
based on RDD/DStream API. See
https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html
It is still experimental, its coverage of the Beam model is partial. ***
21/04/12 14:05:54 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
21/04/12 14:05:56 INFO org.apache.beam.runners.spark.structuredstreaming.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()
21/04/12 14:05:56 INFO org.apache.beam.runners.spark.structuredstreaming.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:
Load test results for test (ID): 9bcc1bdd-17f0-4d47-9d29-4f24ac747806 and timestamp: 2021-04-12T14:05:51.102000000Z:
Metric: Value:
sparkstructuredstreaming_runtime_sec 125.528
sparkstructuredstreaming_total_bytes_count 2.2E9
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 2m 55s
81 actionable tasks: 52 executed, 29 from cache
Publishing build scan...
https://gradle.com/s/qkdpklpzvx3f6
Build step 'Invoke Gradle script' changed build result to SUCCESS
[beam_LoadTests_Java_CoGBK_SparkStructuredStreaming_Batch] $ /bin/bash -xe /tmp/jenkins1465379391127933578.sh
+ echo '*** Load test: CoGBK 2GB 100 byte records - multiple keys ***'
*** Load test: CoGBK 2GB 100 byte records - multiple keys ***
[Gradle] - Launching build.
[src] $ <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_SparkStructuredStreaming_Batch/ws/src/gradlew> -PloadTest.mainClass=org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest -Prunner=:runners:spark:2 -PwithDataflowWorkerJar=false '-PloadTest.args=--project=apache-beam-testing --appName=load_tests_Java_SparkStructuredStreaming_batch_CoGBK_2 --tempLocation=gs://temp-storage-for-perf-tests/loadtests --publishToBigQuery=true --bigQueryDataset=load_test --bigQueryTable=java_sparkstructuredstreaming_batch_CoGBK_2 --influxMeasurement=java_batch_cogbk_2 --publishToInfluxDB=true --sourceOptions={"numRecords":20000000,"keySizeBytes":10,"valueSizeBytes":90,"numHotKeys":5} --coSourceOptions={"numRecords":2000000,"keySizeBytes":10,"valueSizeBytes":90,"numHotKeys":1000} --iterations=1 --streaming=false --influxDatabase=beam_test_metrics --influxHost=http://10.128.0.96:8086 --runner=SparkStructuredStreamingRunner' --continue --max-****s=12 -Dorg.gradle.jvmargs=-Xms2g -Dorg.gradle.jvmargs=-Xmx4g -Pdocker-pull-licenses :sdks:java:testing:load-tests:run
> Task :buildSrc:compileJava NO-SOURCE
> Task :buildSrc:compileGroovy UP-TO-DATE
> Task :buildSrc:pluginDescriptors UP-TO-DATE
> Task :buildSrc:processResources UP-TO-DATE
> Task :buildSrc:classes UP-TO-DATE
> Task :buildSrc:jar UP-TO-DATE
> Task :buildSrc:assemble UP-TO-DATE
> Task :buildSrc:spotlessGroovy UP-TO-DATE
> Task :buildSrc:spotlessGroovyCheck UP-TO-DATE
> Task :buildSrc:spotlessGroovyGradle UP-TO-DATE
> Task :buildSrc:spotlessGroovyGradleCheck UP-TO-DATE
> Task :buildSrc:spotlessCheck UP-TO-DATE
> Task :buildSrc:pluginUnderTestMetadata UP-TO-DATE
> Task :buildSrc:compileTestJava NO-SOURCE
> Task :buildSrc:compileTestGroovy NO-SOURCE
> Task :buildSrc:processTestResources NO-SOURCE
> Task :buildSrc:testClasses UP-TO-DATE
> Task :buildSrc:test NO-SOURCE
> Task :buildSrc:validatePlugins UP-TO-DATE
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :runners:core-java:createCheckerFrameworkManifest UP-TO-DATE
> Task :model:pipeline:createCheckerFrameworkManifest UP-TO-DATE
> Task :model:job-management:createCheckerFrameworkManifest UP-TO-DATE
> Task :sdks:java:core:createCheckerFrameworkManifest UP-TO-DATE
> Task :runners:core-construction-java:createCheckerFrameworkManifest UP-TO-DATE
> Task :model:fn-execution:createCheckerFrameworkManifest UP-TO-DATE
> Task :sdks:java:expansion-service:createCheckerFrameworkManifest UP-TO-DATE
> Task :runners:java-job-service:createCheckerFrameworkManifest UP-TO-DATE
> Task :sdks:java:harness:createCheckerFrameworkManifest UP-TO-DATE
> Task :runners:java-fn-execution:createCheckerFrameworkManifest UP-TO-DATE
> Task :sdks:java:extensions:google-cloud-platform-core:createCheckerFrameworkManifest UP-TO-DATE
> Task :sdks:java:harness:processResources NO-SOURCE
> Task :sdks:java:core:generateAvroProtocol NO-SOURCE
> Task :runners:core-construction-java:processResources NO-SOURCE
> Task :sdks:java:fn-execution:createCheckerFrameworkManifest UP-TO-DATE
> Task :runners:core-java:processResources NO-SOURCE
> Task :runners:java-fn-execution:processResources NO-SOURCE
> Task :sdks:java:fn-execution:processResources NO-SOURCE
> Task :model:job-management:extractProto UP-TO-DATE
> Task :sdks:java:expansion-service:processResources NO-SOURCE
> Task :runners:java-job-service:processResources NO-SOURCE
> Task :sdks:java:extensions:google-cloud-platform-core:processResources NO-SOURCE
> Task :sdks:java:core:generateAvroJava NO-SOURCE
> Task :runners:spark:2:copyResourcesOverrides NO-SOURCE
> Task :model:fn-execution:extractProto UP-TO-DATE
> Task :model:job-management:processResources UP-TO-DATE
> Task :sdks:java:io:google-cloud-platform:createCheckerFrameworkManifest UP-TO-DATE
> Task :sdks:java:extensions:protobuf:createCheckerFrameworkManifest UP-TO-DATE
> Task :sdks:java:io:synthetic:createCheckerFrameworkManifest UP-TO-DATE
> Task :model:fn-execution:processResources UP-TO-DATE
> Task :runners:spark:2:copySourceOverrides UP-TO-DATE
> Task :sdks:java:io:google-cloud-platform:processResources NO-SOURCE
> Task :sdks:java:io:synthetic:processResources NO-SOURCE
> Task :runners:spark:2:copyTestResourcesOverrides NO-SOURCE
> Task :sdks:java:testing:load-tests:createCheckerFrameworkManifest UP-TO-DATE
> Task :sdks:java:testing:test-utils:createCheckerFrameworkManifest UP-TO-DATE
> Task :sdks:java:io:kinesis:createCheckerFrameworkManifest UP-TO-DATE
> Task :sdks:java:io:kafka:createCheckerFrameworkManifest UP-TO-DATE
> Task :sdks:java:io:kafka:processResources NO-SOURCE
> Task :sdks:java:extensions:protobuf:extractProto UP-TO-DATE
> Task :sdks:java:io:kinesis:processResources NO-SOURCE
> Task :runners:spark:2:createCheckerFrameworkManifest UP-TO-DATE
> Task :sdks:java:testing:test-utils:processResources NO-SOURCE
> Task :sdks:java:testing:load-tests:processResources NO-SOURCE
> Task :sdks:java:core:generateGrammarSource UP-TO-DATE
> Task :sdks:java:extensions:protobuf:processResources NO-SOURCE
> Task :runners:spark:2:processResources UP-TO-DATE
> Task :sdks:java:core:processResources UP-TO-DATE
> Task :model:pipeline:extractIncludeProto UP-TO-DATE
> Task :model:pipeline:extractProto UP-TO-DATE
> Task :model:pipeline:generateProto UP-TO-DATE
> Task :model:pipeline:compileJava UP-TO-DATE
> Task :model:pipeline:processResources UP-TO-DATE
> Task :model:pipeline:classes UP-TO-DATE
> Task :model:pipeline:jar UP-TO-DATE
> Task :model:fn-execution:extractIncludeProto UP-TO-DATE
> Task :model:job-management:extractIncludeProto UP-TO-DATE
> Task :model:pipeline:shadowJar UP-TO-DATE
> Task :model:fn-execution:generateProto UP-TO-DATE
> Task :model:job-management:generateProto UP-TO-DATE
> Task :model:fn-execution:compileJava UP-TO-DATE
> Task :model:fn-execution:classes UP-TO-DATE
> Task :model:job-management:compileJava UP-TO-DATE
> Task :model:job-management:classes UP-TO-DATE
> Task :model:job-management:shadowJar UP-TO-DATE
> Task :model:fn-execution:shadowJar UP-TO-DATE
> Task :sdks:java:core:compileJava UP-TO-DATE
> Task :sdks:java:core:classes UP-TO-DATE
> Task :sdks:java:core:shadowJar UP-TO-DATE
> Task :sdks:java:extensions:protobuf:extractIncludeProto UP-TO-DATE
> Task :sdks:java:extensions:protobuf:generateProto NO-SOURCE
> Task :sdks:java:io:synthetic:compileJava UP-TO-DATE
> Task :sdks:java:io:synthetic:classes UP-TO-DATE
> Task :sdks:java:fn-execution:compileJava UP-TO-DATE
> Task :sdks:java:fn-execution:classes UP-TO-DATE
> Task :sdks:java:io:synthetic:jar UP-TO-DATE
> Task :sdks:java:fn-execution:jar UP-TO-DATE
> Task :sdks:java:io:kinesis:compileJava UP-TO-DATE
> Task :sdks:java:io:kinesis:classes UP-TO-DATE
> Task :sdks:java:io:kinesis:jar UP-TO-DATE
> Task :sdks:java:extensions:protobuf:compileJava UP-TO-DATE
> Task :sdks:java:extensions:protobuf:classes UP-TO-DATE
> Task :sdks:java:testing:test-utils:compileJava UP-TO-DATE
> Task :sdks:java:testing:test-utils:classes UP-TO-DATE
> Task :sdks:java:extensions:protobuf:jar UP-TO-DATE
> Task :sdks:java:testing:test-utils:jar UP-TO-DATE
> Task :runners:core-construction-java:compileJava UP-TO-DATE
> Task :runners:core-construction-java:classes UP-TO-DATE
> Task :runners:core-construction-java:jar UP-TO-DATE
> Task :runners:core-java:compileJava UP-TO-DATE
> Task :runners:core-java:classes UP-TO-DATE
> Task :runners:core-java:jar UP-TO-DATE
> Task :sdks:java:extensions:google-cloud-platform-core:compileJava UP-TO-DATE
> Task :sdks:java:extensions:google-cloud-platform-core:classes UP-TO-DATE
> Task :sdks:java:extensions:google-cloud-platform-core:jar UP-TO-DATE
> Task :sdks:java:harness:compileJava UP-TO-DATE
> Task :sdks:java:harness:classes UP-TO-DATE
> Task :sdks:java:harness:jar UP-TO-DATE
> Task :sdks:java:harness:shadowJar UP-TO-DATE
> Task :runners:java-fn-execution:compileJava UP-TO-DATE
> Task :runners:java-fn-execution:classes UP-TO-DATE
> Task :runners:java-fn-execution:jar UP-TO-DATE
> Task :sdks:java:expansion-service:compileJava UP-TO-DATE
> Task :sdks:java:expansion-service:classes UP-TO-DATE
> Task :sdks:java:expansion-service:jar UP-TO-DATE
> Task :runners:java-job-service:compileJava UP-TO-DATE
> Task :runners:java-job-service:classes UP-TO-DATE
> Task :runners:java-job-service:jar UP-TO-DATE
> Task :sdks:java:io:kafka:compileJava UP-TO-DATE
> Task :sdks:java:io:kafka:classes UP-TO-DATE
> Task :sdks:java:io:kafka:jar UP-TO-DATE
> Task :sdks:java:io:google-cloud-platform:compileJava UP-TO-DATE
> Task :sdks:java:io:google-cloud-platform:classes UP-TO-DATE
> Task :sdks:java:io:google-cloud-platform:jar UP-TO-DATE
> Task :runners:spark:2:compileJava UP-TO-DATE
> Task :runners:spark:2:classes UP-TO-DATE
> Task :runners:spark:2:jar UP-TO-DATE
> Task :sdks:java:testing:load-tests:compileJava UP-TO-DATE
> Task :sdks:java:testing:load-tests:classes UP-TO-DATE
> Task :sdks:java:testing:load-tests:jar UP-TO-DATE
> Task :sdks:java:testing:load-tests:run
21/04/12 14:08:26 WARN org.apache.beam.sdk.Pipeline: The following transforms do not have stable unique names: Window.Into()
21/04/12 14:08:26 INFO org.apache.beam.runners.spark.structuredstreaming.SparkStructuredStreamingRunner: *** SparkStructuredStreamingRunner is based on spark structured streaming framework and is no more
based on RDD/DStream API. See
https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html
It is still experimental, its coverage of the Beam model is partial. ***
21/04/12 14:08:27 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
21/04/12 14:08:28 INFO org.apache.beam.runners.spark.structuredstreaming.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()
21/04/12 14:08:28 INFO org.apache.beam.runners.spark.structuredstreaming.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:
Load test results for test (ID): cd3c86fa-570a-4918-aea7-1564ac65b3fe and timestamp: 2021-04-12T14:08:25.465000000Z:
Metric: Value:
sparkstructuredstreaming_runtime_sec 124.299
sparkstructuredstreaming_total_bytes_count 2.2E9
Exception in thread "main" com.google.cloud.bigquery.BigQueryException: Read timed out
at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.translate(HttpBigQueryRpc.java:115)
at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.insertAll(HttpBigQueryRpc.java:494)
at com.google.cloud.bigquery.BigQueryImpl.insertAll(BigQueryImpl.java:1036)
at org.apache.beam.sdk.testutils.publishing.BigQueryClient.insertAll(BigQueryClient.java:123)
at org.apache.beam.sdk.testutils.publishing.BigQueryClient.insertAll(BigQueryClient.java:110)
at org.apache.beam.sdk.testutils.publishing.BigQueryResultsPublisher.publish(BigQueryResultsPublisher.java:57)
at org.apache.beam.sdk.loadtests.LoadTest.publishResultsToBigQuery(LoadTest.java:177)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:140)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:457)
at sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:68)
at sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1095)
at sun.security.ssl.SSLSocketImpl.access$200(SSLSocketImpl.java:72)
at sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:815)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:735)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1593)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:352)
at com.google.api.client.http.javanet.NetHttpResponse.<init>(NetHttpResponse.java:36)
at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:149)
at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:84)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1012)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:514)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:455)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:565)
at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.insertAll(HttpBigQueryRpc.java:492)
... 8 more
> Task :sdks:java:testing:load-tests:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2m 41s
81 actionable tasks: 1 executed, 80 up-to-date
Publishing build scan...
https://gradle.com/s/5a6abqldvglom
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal :
beam_LoadTests_Java_CoGBK_SparkStructuredStreaming_Batch #362
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_SparkStructuredStreaming_Batch/362/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org