[jira] [Commented] (BEAM-9613) BigQuery IO not support convert double type for beam row

["Matthew Ouyang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-9613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17306301#comment-17306301 ] 

Matthew Ouyang commented on BEAM-9613:
--------------------------------------

Hello, after doing some more investigation the issue appears to be linked to the transition from JSON to Avro for reading in BigQueryIO that happened in BEAM-2879.  BigQueryAvroUtils returns Avro boolean and floats as Java booleans and doubles, and BigQueryUtils.toBeamValue were not respectively updated.  Anything with REPEATED mode will not be processed correctly either because it also assumes the JSON format.

> BigQuery IO not support convert double type for beam row
> --------------------------------------------------------
>
>                 Key: BEAM-9613
>                 URL: https://issues.apache.org/jira/browse/BEAM-9613
>             Project: Beam
>          Issue Type: Bug
>          Components: io-java-gcp
>            Reporter: MAKSIM TSYGAN
>            Priority: P3
>
> If execute query with double column  via BigQueryIO.readFrom(), I get exception:
> Caused by: java.lang.UnsupportedOperationException: Converting BigQuery type 'class java.lang.Double' to 'FieldType\{typeName=DOUBLE, nullable=true, logicalType=null, collectionElementType=null, mapKeyType=null, mapValueType=null, rowSchema=null, metadata={}}' is not supported
>  at org.apache.beam.sdk.io.gcp.bigquery.BigQueryUtils.toBeamValue(BigQueryUtils.java:532)
>  at org.apache.beam.sdk.io.gcp.bigquery.BigQueryUtils.toBeamRowFieldValue(BigQueryUtils.java:483)
>  at org.apache.beam.sdk.io.gcp.bigquery.BigQueryUtils.lambda$toBeamRow$6(BigQueryUtils.java:469)
>  at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
>  at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
>  at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
>  at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
>  at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
>  at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
>  at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
>  at org.apache.beam.sdk.io.gcp.bigquery.BigQueryUtils.toBeamRow(BigQueryUtils.java:470)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)