You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/06/04 22:43:07 UTC
[GitHub] [beam] damccorm opened a new issue, #21327: BigQuery Storage Write API fails with type mismatch error for DATE type
damccorm opened a new issue, #21327:
URL: https://github.com/apache/beam/issues/21327
If `DATE` type is specified in schema, `BigQueryIO` with Storage Write API may fail with the following error:
> Got error com.google.api.gax.rpc.InvalidArgumentException: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: The proto field mismatched with BigQuery field at <table\>, the proto field type string, BigQuery field type DATE Entity: <write_stream_name\>
> .
> `DATE` type in BEAM for BigQuery is converted to `string` in proto request [1], while BigQuery Storage Write API requires an integer type for `DATE` type in BigQuery [2].
A workaround is to use `INTEGER` in BEAM and manually compute the number of days since epoch.
Actually, a feature request to BigQuery is filed at [3]. If it's implemented, we can use `DATE` type successfully.
[1] [https://github.com/apache/beam/blob/v2.35.0/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/TableRowToStorageApiProto.java#L55-L73](https://github.com/apache/beam/blob/v2.35.0/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/TableRowToStorageApiProto.java#L55-L73)
[2] [https://cloud.google.com/bigquery/docs/write-api#data_type_conversions](https://cloud.google.com/bigquery/docs/write-api#data_type_conversions)
[3] [https://issuetracker.google.com/issues/205174128](https://issuetracker.google.com/issues/205174128)
Imported from Jira [BEAM-13753](https://issues.apache.org/jira/browse/BEAM-13753). Original Jira may contain additional context.
Reported by: baeminbo.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] closed issue #21327: BigQuery Storage Write API fails with type mismatch error for DATE type
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed issue #21327: BigQuery Storage Write API fails with type mismatch error for DATE type
URL: https://github.com/apache/beam/issues/21327
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ahmedabu98 commented on issue #21327: BigQuery Storage Write API fails with type mismatch error for DATE type
Posted by GitBox <gi...@apache.org>.
ahmedabu98 commented on issue #21327:
URL: https://github.com/apache/beam/issues/21327#issuecomment-1310957645
This was fixed by #17404
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ahmedabu98 commented on issue #21327: BigQuery Storage Write API fails with type mismatch error for DATE type
Posted by GitBox <gi...@apache.org>.
ahmedabu98 commented on issue #21327:
URL: https://github.com/apache/beam/issues/21327#issuecomment-1310957679
.close-issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org