You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@beam.apache.org by Chengxuan Wang <wc...@gmail.com> on 2019/04/23 16:51:47 UTC

questions about BigQuery temp dataset

Hi,

I am using Apache Beam python sdk (apache-beam==2.11.0) to run a dataflow
job with BigQuerySource. Even though I checked the code, BigQueryReader
will delete the temporary dataset after the query is done.
https://github.com/apache/beam/blob/1ad61fd384bcd1edd11086a3cf9d7dddb154d934/sdks/python/apache_beam/io/gcp/bigquery_tools.py#L900

But I still see some temporary dataset in my GCP console. Could you help me
look into it?

Another thing is is that possible to set expiration for the temporary
dataset? right now I see is never.

Thanks,
Tom