[GitHub] [airflow] eladkal commented on a diff in pull request #28677: Support partition_columns in BaseSQLToGCSOperator

[GitBox <gi...@apache.org>]

eladkal commented on code in PR #28677:
URL: https://github.com/apache/airflow/pull/28677#discussion_r1064393076


##########
airflow/providers/google/cloud/transfers/sql_to_gcs.py:
##########
@@ -77,6 +78,10 @@ class BaseSQLToGCSOperator(BaseOperator):
         account from the list granting this role to the originating account (templated).
     :param upload_metadata: whether to upload the row count metadata as blob metadata
     :param exclude_columns: set of columns to exclude from transmission
+    :param partition_columns: list of columns to use for file partitioning. In order to use
+        this parameter, you must sort your dataset by partition_columns. Do this by
+        passing an ORDER BY clause to the sql query. Files are uploaded to GCS as objects
+        with a hive style partitioning directory structure.

Review Comment:
   ```suggestion
           with a hive style partitioning directory structure (templated).
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org