[jira] [Created] (BEAM-8851) Dataflow BiqQuery TimePartitioning integer overflow

["Per Böckman (Jira)" <ji...@apache.org>]

Per Böckman created BEAM-8851:
---------------------------------

             Summary: Dataflow BiqQuery TimePartitioning integer overflow
                 Key: BEAM-8851
                 URL: https://issues.apache.org/jira/browse/BEAM-8851
             Project: Beam
          Issue Type: Bug
          Components: io-java-gcp
    Affects Versions: 2.16.0
         Environment: GCP
            Reporter: Per Böckman


BigQueryIO.writeTableRows()....withTimePartitioning() seems to fail to create a new BQ table if TimePartitioning expirationMs is bigger than max int. 

TimePartitioning.setExpirationMs accepts a long and if the expiry time is set to one month (86400 * 1000 * 30 = 2592000000 ms) the BQ server responds with a 400 and the error indicates an integer overflow.

 

exception: "org.apache.beam.sdk.util.UserCodeException: java.lang.RuntimeException: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request \{ "code" : 400, "errors" : [ { "domain" : "global", "message" : "TTL for interval partition must be a positive number.", "reason" : "invalid" } ], "message" : "TTL for interval partition must be a positive number.", "status" : "INVALID_ARGUMENT" }

 

at org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:34) at org.apache.beam.sdk.io.gcp.bigquery.CreateTables$CreateTablesFn$DoFnInvoker.invokeProcessElement(Unknown Source) at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:218)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)