You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "ASF subversion and git services (JIRA)" <ji...@apache.org> on 2019/04/02 20:19:03 UTC
[jira] [Commented] (AIRFLOW-3419) S3_hook.select_key is broken on
Python3
[ https://issues.apache.org/jira/browse/AIRFLOW-3419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16808121#comment-16808121 ]
ASF subversion and git services commented on AIRFLOW-3419:
----------------------------------------------------------
Commit 41fd951f9087e8cecc0e4255102c9f551e1acda4 in airflow's branch refs/heads/v1-10-test from Ash Berlin-Taylor
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=41fd951 ]
[AIRFLOW-3419] Fix S3Hook.select_key on Python3 (#4970)
> S3_hook.select_key is broken on Python3
> ---------------------------------------
>
> Key: AIRFLOW-3419
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3419
> Project: Apache Airflow
> Issue Type: Bug
> Components: boto3, hooks
> Affects Versions: 1.10.1
> Reporter: Maria Rebelka
> Priority: Major
> Fix For: 1.10.3
>
>
> Hello,
> Using select_key throws an error:
> {quote}text = S3Hook('aws_conn').select_key(key='my_key',
> bucket_name='my_bucket',
> expression='SELECT * FROM S3Object s',
> expression_type='SQL',
> input_serialization={'JSON': \{'Type': 'DOCUMENT'}},
> output_serialization={'JSON': {}}){quote}
> Traceback (most recent call last):
> {quote} File "db.py", line 31, in <module>
> output_serialization={'JSON': {}})
> File "/usr/local/lib/python3.4/site-packages/airflow/hooks/S3_hook.py", line 262, in select_key
> for event in response['Payload']
> TypeError: sequence item 0: expected str instance, bytes found{quote}
> Seems that the problem is in this line:
> S3_hook.py, line 262: return ''.join(event['Records']['Payload']
> which probably should be return ''.join(event['Records']['Payload'].decode('utf-8')
> From example in Amazon blog:
> https://aws.amazon.com/blogs/aws/s3-glacier-select/
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)