You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/07/15 09:57:12 UTC

[GitHub] [airflow] feluelle commented on a change in pull request #8936: Fixed S3FileTransformOperator to support S3 Select transformation only

feluelle commented on a change in pull request #8936:
URL: https://github.com/apache/airflow/pull/8936#discussion_r454933227



##########
File path: tests/providers/amazon/aws/operators/test_s3_file_transform.py
##########
@@ -123,26 +127,24 @@ def test_execute_with_select_expression(self, mock_select_key):
             expression=select_expression
         )
 
+        conn = boto3.client('s3')
+        result = conn.get_object(Bucket=self.bucket, Key=self.output_key)
+        self.assertEqual(b'input', result['Body'].read())

Review comment:
       ```suggestion
           assert self.bio == result['Body']
   ```
   Does this work?
   

##########
File path: airflow/providers/amazon/aws/operators/s3_file_transform.py
##########
@@ -167,7 +167,7 @@ def execute(self, context):
             self.log.info("Uploading transformed file to S3")
             f_dest.flush()
             dest_s3.load_file(
-                filename=f_dest.name,
+                filename=f_dest.name if self.transform_script is not None else f_source.name,

Review comment:
       ```suggestion
                   filename=f_dest.name if self.transform_script else f_source.name,
   ```




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org