[jira] [Created] (BEAM-1953) Google Credential refresh failures aren't consistently retried

["Luke Cwik (JIRA)" <ji...@apache.org>]

Luke Cwik created BEAM-1953:
-------------------------------

             Summary: Google Credential refresh failures aren't consistently retried
                 Key: BEAM-1953
                 URL: https://issues.apache.org/jira/browse/BEAM-1953
             Project: Beam
          Issue Type: Bug
          Components: runner-dataflow, sdk-java-gcp
            Reporter: Luke Cwik
            Priority: Minor


Each request to a Google service requires an access token to authenticate.

This token is periodically refreshed but this refresh may fail (request failure or IO failure). For example:
Exception in thread "main" javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:992)
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
        at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
        at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1316)
        at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1291)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:250)
        at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:77)
        at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:981)
        at com.google.api.client.auth.oauth2.TokenRequest.executeUnparsed(TokenRequest.java:283)
        at com.google.api.client.auth.oauth2.TokenRequest.execute(TokenRequest.java:307)
        at com.google.api.client.googleapis.auth.oauth2.GoogleCredential.executeRefreshToken(GoogleCredential.java:384)
        at com.google.api.client.auth.oauth2.Credential.refreshToken(Credential.java:489)
        at com.google.api.client.auth.oauth2.Credential.intercept(Credential.java:217)
        at com.google.cloud.hadoop.util.ChainingHttpRequestInitializer$2.intercept(ChainingHttpRequestInitializer.java:98)
        at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:868)
        at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
        at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
        at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
        ...
Caused by: java.io.EOFException: SSL peer shut down incorrectly
        at sun.security.ssl.InputRecord.read(InputRecord.java:505)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
        ... 27 more

Higher level retries may mask some of these failures but the credential refresh should have some retry policy.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)