[GitHub] [beam] ahmedabu98 commented on a diff in pull request #26889: Update document Storage API Support in `Google BigQuery I/O connector` for Python SDK

["ahmedabu98 (via GitHub)" <gi...@apache.org>]

ahmedabu98 commented on code in PR #26889:
URL: https://github.com/apache/beam/pull/26889#discussion_r1207044220


##########
sdks/python/apache_beam/examples/snippets/snippets.py:
##########
@@ -1181,6 +1189,32 @@ def model_bigqueryio(
       create_disposition=beam.io.BigQueryDisposition.CREATE_IF_NEEDED)
   # [END model_bigqueryio_write]
 
+  # [START model_bigqueryio_write_with_storage_write_api]
+  quotes | beam.io.WriteToBigQuery(
+      table_spec,
+      schema=table_schema,
+      method=beam.io.WriteToBigQuery.Method.STORAGE_WRITE_API)
+  # [END model_bigqueryio_write_with_storage_write_api]

Review Comment:
   Writing with storage api currently has some limitations on what data types are allowed in the schema. Important to note that the python `decimal.Decimal` is needed to write a `NUMERIC`  BigQuery type. Similarly, the the Beam `Timestamp` type is needed to write a `TIMESTAMP` BigQuery type. 
   
   Also, some other types are not yet supported, like "DATETIME". Might be worth mentioning some of that, here is the full allowed list:
   
   https://github.com/apache/beam/blob/0b430748cdd2e25edc553747ce018195e9cce888/sdks/python/apache_beam/io/gcp/bigquery_tools.py#L112-L123



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org