[beam] branch master updated: Warn if temp dataset cleanup permission is denied

[pa...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

pabloem pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
     new 3d23f47  Warn if temp dataset cleanup permission is denied
     new bee6602  Merge pull request #13433 from Warn if temp dataset cleanup permission is denied
3d23f47 is described below

commit 3d23f471c2d98ad06e720eb385abef532b7d4be7
Author: Frank Zhao <fr...@frankzhao.net>
AuthorDate: Thu Nov 26 14:01:00 2020 +1100

    Warn if temp dataset cleanup permission is denied
    
    This will mean users no longer require BigQuery dataset delete
    permission for the execution user.
---
 sdks/python/apache_beam/io/gcp/bigquery_tools.py | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/sdks/python/apache_beam/io/gcp/bigquery_tools.py b/sdks/python/apache_beam/io/gcp/bigquery_tools.py
index da1b7fd..1544588 100644
--- a/sdks/python/apache_beam/io/gcp/bigquery_tools.py
+++ b/sdks/python/apache_beam/io/gcp/bigquery_tools.py
@@ -783,7 +783,17 @@ class BigQueryWrapper(object):
         return
       else:
         raise
-    self._delete_dataset(temp_table.projectId, temp_table.datasetId, True)
+    try:
+      self._delete_dataset(temp_table.projectId, temp_table.datasetId, True)
+    except HttpError as exn:
+      if exn.status_code == 403:
+        _LOGGER.warning(
+            'Permission denied to delete temporary dataset %s:%s for clean up',
+            temp_table.projectId,
+            temp_table.datasetId)
+        return
+      else:
+        raise
 
   @retry.with_exponential_backoff(
       num_retries=MAX_RETRIES,