You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/06/03 22:52:29 UTC

[GitHub] [beam] kennknowles opened a new issue, #19174: BigQuery IO Source is not Exporting to GCS as written in documentation

kennknowles opened a new issue, #19174:
URL: https://github.com/apache/beam/issues/19174

   Did some check on the beam code and find out that DataFlow is querying BigQuery and retrieve the result using pagination [1]. As per our understanding, this means no parallelism on reading BigQuery table. It is contradictory to what the documentation is telling us [2].
    
   Is this some kind of work in progress? I'm filing as a bug since documentation telling me that it is using GCS meanwhile it's using NativeSourceReader which yield data per row as iterator.
    
   [1] [https://github.com/apache/beam/blob/520b3a24e49306c30940ceab09100d775a04d28e/sdks/python/apache_beam/io/gcp/bigquery.py#L1083](https://github.com/apache/beam/blob/520b3a24e49306c30940ceab09100d775a04d28e/sdks/python/apache_beam/io/gcp/bigquery.py#L1083)
   [2] [https://github.com/apache/beam/blob/520b3a24e49306c30940ceab09100d775a04d28e/sdks/python/apache_beam/io/gcp/bigquery.py#L60](https://github.com/apache/beam/blob/520b3a24e49306c30940ceab09100d775a04d28e/sdks/python/apache_beam/io/gcp/bigquery.py#L60)
   
   Imported from Jira [BEAM-5352](https://issues.apache.org/jira/browse/BEAM-5352). Original Jira may contain additional context.
   Reported by: rendybjunior.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] rendybjunior commented on issue #19174: BigQuery IO Source is not Exporting to GCS as written in documentation

Posted by GitBox <gi...@apache.org>.
rendybjunior commented on issue #19174:
URL: https://github.com/apache/beam/issues/19174#issuecomment-1146435378

   Thanks for migrating the issue.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org