[GitHub] [airflow] jithin97 commented on a change in pull request #7126: [AIRFLOW-6405] Add GCP BigQuery Table Upsert Operator

[GitBox <gi...@apache.org>]

jithin97 commented on a change in pull request #7126: [AIRFLOW-6405] Add GCP BigQuery Table Upsert Operator
URL: https://github.com/apache/airflow/pull/7126#discussion_r372935998
 
 

 ##########
 File path: airflow/gcp/hooks/bigquery.py
 ##########
 @@ -1109,6 +1110,16 @@ def run_table_upsert(self, dataset_id: str, table_resource: Dict,
         """
         service = self.get_service()
         # check to see if the table exists
+        if 'tableReference' not in table_resource:
+            raise AirflowBadRequest(
+                '"tableReference" is required within table_resource parameter. '
+                'See https://cloud.google.com/bigquery/docs/reference/v2/tables#resource'
+            )
+        if 'tableId' not in table_resource["tableReference"]:
+            raise AirflowBadRequest(
+                '"tableId" is required within table_resource["tableReference"]. '
+                'See https://cloud.google.com/bigquery/docs/reference/v2/tables#resource'
+            )
 
 Review comment:
   @nuclearpinguin If your concern is about the operators then I think we are safe because currently none of the operators are using this method. The only calls are from test cases.
   
   Memo:
   ```
   :~/Desktop/search_work/airflow|master 
   ⇒  git grep run_table_upsert
   
   CHANGELOG.txt:- [AIRFLOW-314] Fix BigQuery cursor run_table_upsert method
   CHANGELOG.txt:- Add two methods to bigquery hook's base cursor: run_table_upsert, which adds a table or updates an existing table; and run_grant_dataset_view_access, which grants view access to a given dataset for a given table.
   UPDATING.md:| airflow.gcp.hooks.bigquery.BigQueryBaseCursor.run_table_upsert              | airflow.gcp.hooks.bigquery.BigQueryHook.run_table_upsert              |
   airflow/gcp/hooks/bigquery.py:    def run_table_upsert(self, dataset_id: str, table_resource: Dict,
   airflow/gcp/hooks/bigquery.py:    def run_table_upsert(self, *args, **kwargs) -> Dict:
   airflow/gcp/hooks/bigquery.py:        Please use `airflow.gcp.hooks.bigquery.BigQueryHook.run_table_upsert`
   airflow/gcp/hooks/bigquery.py:            "Please use `airflow.gcp.hooks.bigquery.BigQueryHook.run_table_upsert`",
   airflow/gcp/hooks/bigquery.py:        return self.hook.run_table_upsert(*args, **kwargs)
   tests/gcp/hooks/test_bigquery.py:        bq_hook.run_table_upsert(dataset_id=DATASET_ID, table_resource=table_resource)
   tests/gcp/hooks/test_bigquery.py:        bq_hook.run_table_upsert(dataset_id=DATASET_ID, table_resource=table_resource)
   tests/gcp/hooks/test_bigquery.py:        ("run_table_upsert",),
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services