[jira] [Commented] (BEAM-12500) Dataflow SocketException (SSLException) error while trying to send message from Cloud Pub/Sub to BigQuery

["Chamikara Madhusanka Jayalath (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-12500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426424#comment-17426424 ] 

Chamikara Madhusanka Jayalath commented on BEAM-12500:
------------------------------------------------------

Usually such errors are resolved through retries. Wasn't that the case here ?

> Dataflow SocketException (SSLException) error while trying to send message from Cloud Pub/Sub to BigQuery
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: BEAM-12500
>                 URL: https://issues.apache.org/jira/browse/BEAM-12500
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-java-core, sdk-java-harness
>    Affects Versions: 2.26.0, 2.27.0, 2.28.0, 2.29.0
>         Environment: GCP services used (such as BigQuery): Pub/Sub -->Dataflow-->BigQuery
>            Reporter: steven zhang
>            Priority: P1
>              Labels: SSLException, sdk-consistency
>             Fix For: 2.20.0, 2.25.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> *Issue*: I found an issue of seeing SSLException when using beam version (>2.25) and could not upgrade with advanced version of beam in the dataflow job (streaming). Only versions below 2.25 is possible to work. Could not be sure if this issue is connected to [1]
> [1]https://bugs.openjdk.java.net/browse/JDK-8241372
> *Log*: {{Error message from worker: java.lang.RuntimeException: java.net.SocketException: Connection reset org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.insertAll(BigQueryServicesImpl.java:932) org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.insertAll(BigQueryServicesImpl.java:972) org.apache.beam.sdk.io.gcp.bigquery.StreamingWriteFn.flushRows(StreamingWriteFn.java:195) org.apache.beam.sdk.io.gcp.bigquery.StreamingWriteFn.finishBundle(StreamingWriteFn.java:149) Caused by: java.net.SocketException: Connection reset java.net.SocketInputStream.read(SocketInputStream.java:210) java.net.SocketInputStream.read(SocketInputStream.java:141) sun.security.ssl.InputRecord.readFully(InputRecord.java:465) sun.security.ssl.InputRecord.read(InputRecord.java:503) sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983) sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:940) sun.security.ssl.AppInputStream.read(AppInputStream.java:105) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:552) sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:609) sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:696) java.io.FilterInputStream.read(FilterInputStream.java:133) sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:3393) com.google.api.client.http.javanet.NetHttpResponse$SizeValidatingInputStream.read(NetHttpResponse.java:164) java.io.FilterInputStream.read(FilterInputStream.java:133) java.io.FilterInputStream.read(FilterInputStream.java:107) com.google.common.io.ByteStreams.exhaust(ByteStreams.java:273) com.google.api.client.http.ConsumingInputStream.close(ConsumingInputStream.java:40) java.util.zip.InflaterInputStream.close(InflaterInputStream.java:227) java.util.zip.GZIPInputStream.close(GZIPInputStream.java:136) com.fasterxml.jackson.core.json.UTF8StreamJsonParser._closeInput(UTF8StreamJsonParser.java:254) com.fasterxml.jackson.core.base.ParserBase.close(ParserBase.java:369) com.google.api.client.json.jackson2.JacksonParser.close(JacksonParser.java:47) com.google.api.client.json.JsonParser.parse(JsonParser.java:363) com.google.api.client.json.JsonParser.parse(JsonParser.java:335) com.google.api.client.json.JsonObjectParser.parseAndClose(JsonObjectParser.java:79) com.google.api.client.json.JsonObjectParser.parseAndClose(JsonObjectParser.java:73) com.google.api.client.http.HttpResponse.parseAs(HttpResponse.java:451) com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:591) org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.lambda$insertAll$1(BigQueryServicesImpl.java:857) org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$BoundedExecutorService$SemaphoreCallable.call(BigQueryServicesImpl.java:1304) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:748)}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)