[jira] [Created] (AIRFLOW-2045) S3FileTransformOperator doesn't adhere to current boto3 API

["Zachary Rubenstein (JIRA)" <ji...@apache.org>]

Zachary Rubenstein created AIRFLOW-2045:
-------------------------------------------

             Summary: S3FileTransformOperator doesn't adhere to current boto3 API
                 Key: AIRFLOW-2045
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2045
             Project: Apache Airflow
          Issue Type: Bug
          Components: aws, boto3
    Affects Versions: 1.9.1
            Reporter: Zachary Rubenstein


When using S3FileTransformOperator with boto3 >= 1.5.0, I get an error:

 

 Traceback (most recent call last):
 File "/home/airflow/data/airflow/.venv/bin/airflow", line 27, in <module>
 args.func(args)
 File "/home/airflow/data/airflow/.venv/local/lib/python2.7/site-packages/airflow/bin/cli.py", line 528, in test
 ti.run(ignore_task_deps=True, ignore_ti_state=True, test_mode=True)
 File "/home/airflow/data/airflow/.venv/local/lib/python2.7/site-packages/airflow/utils/db.py", line 50, in wrapper
 result = func(*args, **kwargs)
 File "/home/airflow/data/airflow/.venv/local/lib/python2.7/site-packages/airflow/models.py", line 1584, in run
 session=session)
 File "/home/airflow/data/airflow/.venv/local/lib/python2.7/site-packages/airflow/utils/db.py", line 50, in wrapper
 result = func(*args, **kwargs)
 File "/home/airflow/data/airflow/.venv/local/lib/python2.7/site-packages/airflow/models.py", line 1493, in _run_raw_task
 result = task_copy.execute(context=context)
 File "/home/airflow/data/airflow/.venv/local/lib/python2.7/site-packages/airflow/operators/s3_file_transform_operator.py", line 86, in execute
 source_s3_key_object.get_contents_to_file(f_source)
AttributeError: 's3.Object' object has no attribute 'get_contents_to_file'

I believe the method has been renamed to download_fileobj (http://boto3.readthedocs.io/en/latest/reference/services/s3.html#S3.Object.download_fileobj)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)