[jira] [Created] (BEAM-10021) BigQueryUtils DATETIME conversion requires unix millis

["Ben Birt (Jira)" <ji...@apache.org>]

Ben Birt created BEAM-10021:
-------------------------------

             Summary: BigQueryUtils DATETIME conversion requires unix millis
                 Key: BEAM-10021
                 URL: https://issues.apache.org/jira/browse/BEAM-10021
             Project: Beam
          Issue Type: Bug
          Components: beam-model
            Reporter: Ben Birt


I'm trying to run a very simple GCP Dataflow SQL job (copy all columns from table A to B), and it's failing with the following exception:
{code:java}
Caused by: java.lang.ClassCastException: org.apache.avro.util.Utf8 cannot be cast to java.lang.Longat org.apache.beam.sdk.io.gcp.bigquery.BigQueryUtils.truncateToMillis (BigQueryUtils.java:643)at org.apache.beam.sdk.io.gcp.bigquery.BigQueryUtils.convertAvroFormat (BigQueryUtils.java:597)at org.apache.beam.sdk.io.gcp.bigquery.BigQueryUtils.lambda$toBeamRow$5 (BigQueryUtils.java:389)
{code}
It looks to me like this is because BigQueryUtils.convertAvroFormat expects that any value whose (beam) type is "DATETIME" must be expressed in unix millis. However, these values come out of BigQuery as a string, (AFAICT) in ISO 8601 format.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)