You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "ng-oliver (via GitHub)" <gi...@apache.org> on 2025/06/17 14:15:15 UTC
[I] To include `additional_bq_parameters` in StorageWriteToBigQuery() [beam]
ng-oliver opened a new issue, #35329:
URL: https://github.com/apache/beam/issues/35329
https://github.com/apache/beam/blob/297ba99e96ed65ee154a8c4327023dd9a3c31f98/sdks/python/apache_beam/io/gcp/bigquery.py#L2555
Currently one can configure (say clustering and partition) for `create_depositions` when method is `STREAMING_INSERTS` or `FILE_LOADS` via the `additional_bq_parameters` argument.
However this argument is absent when method = `STORAGE_WRITE_API`. This leads to an awkward situation that the dataflow job can have dynamic destination (which is great) but none of the destinations are partitioned nor clustered upon table creation (which is not great).
One will then need to manually delete the created tables and run a DDL statement to re-create the table with partition and clustering. This has eliminated the benefit of letting dataflow to create destination dynamically.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Task] To include `additional_bq_parameters` in StorageWriteToBigQuery() [beam]
Posted by "TanuSharma2511 (via GitHub)" <gi...@apache.org>.
TanuSharma2511 commented on issue #35329:
URL: https://github.com/apache/beam/issues/35329#issuecomment-2989718306
.take-issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Task] To include `additional_bq_parameters` in StorageWriteToBigQuery() [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #35329: [Task] To include `additional_bq_parameters` in StorageWriteToBigQuery()
URL: https://github.com/apache/beam/issues/35329
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Task] To include `additional_bq_parameters` in StorageWriteToBigQuery() [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #35329:
URL: https://github.com/apache/beam/issues/35329#issuecomment-3541667618
This issue has been marked as stale due to 150 days of inactivity. It will be closed in 30 days if no further activity occurs. If you think that’s incorrect or this issue still needs to be addressed, please simply write any comment. If closed, you can reopen the issue at any time. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Task] To include `additional_bq_parameters` in StorageWriteToBigQuery() [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #35329:
URL: https://github.com/apache/beam/issues/35329#issuecomment-3665194256
This issue has been closed due to lack of activity. If you think that is incorrect, you can reopen the issue at any time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org