[jira] [Commented] (AIRFLOW-3419) S3_hook.select_key is broken on Python3

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-3419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16800662#comment-16800662 ] 

ASF GitHub Bot commented on AIRFLOW-3419:
-----------------------------------------

ashb commented on pull request #4571: [AIRFLOW-3419] S3_hook.select_key is broken on Python3
URL: https://github.com/apache/airflow/pull/4571
 
 
   
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> S3_hook.select_key is broken on Python3
> ---------------------------------------
>
>                 Key: AIRFLOW-3419
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-3419
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: boto3, hooks
>    Affects Versions: 1.10.1
>            Reporter: Maria Rebelka
>            Priority: Major
>             Fix For: 1.10.3
>
>
> Hello,
> Using select_key throws an error:
> {quote}text = S3Hook('aws_conn').select_key(key='my_key',
>                                      bucket_name='my_bucket',
>                                      expression='SELECT * FROM S3Object s',
>                                      expression_type='SQL',
>                                      input_serialization={'JSON': \{'Type': 'DOCUMENT'}},
>                                      output_serialization={'JSON': {}}){quote}
> Traceback (most recent call last):
> {quote}   File "db.py", line 31, in <module>
>     output_serialization={'JSON': {}})
>   File "/usr/local/lib/python3.4/site-packages/airflow/hooks/S3_hook.py", line 262, in select_key
>     for event in response['Payload']
> TypeError: sequence item 0: expected str instance, bytes found{quote}
> Seems that the problem is in this line:
> S3_hook.py, line 262:  return ''.join(event['Records']['Payload']
> which probably should be return ''.join(event['Records']['Payload'].decode('utf-8')
> From example in Amazon blog:
> https://aws.amazon.com/blogs/aws/s3-glacier-select/



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)