You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by "Thomas H (JIRA)" <ji...@apache.org> on 2017/04/18 17:44:41 UTC

[jira] [Created] (AIRFLOW-1119) Redshift to S3 operator - headers not on first row

Thomas H created AIRFLOW-1119:
---------------------------------

             Summary: Redshift to S3 operator - headers not on first row
                 Key: AIRFLOW-1119
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1119
             Project: Apache Airflow
          Issue Type: Bug
            Reporter: Thomas H
         Attachments: airflow-bug.png

The RedshiftToS3 operator adds headers when unloading data from redshift to s3. However, there is a bug where the headers can appear in any row (see screenshot). Need to add `ORDER BY 1 DESC` to the query below to ensure headers are on first row.

https://github.com/apache/incubator-airflow/blob/master/airflow/operators/redshift_to_s3_operator.py#L93:L102

More info regarding unloading data with headers:
http://stackoverflow.com/questions/24681214/unloading-from-redshift-to-s3-with-headers
https://medium.com/carwow-product-engineering/unloading-a-file-from-redshift-to-s3-with-headers-fb707f5480f7



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)