[jira] [Commented] (ARROW-11161) [Python][C++] S3Filesystem: file Content-Type not set correctly?

["Antoine Pitrou (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-11161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17260566#comment-17260566 ] 

Antoine Pitrou commented on ARROW-11161:
----------------------------------------

I had no idea S3 stored a content-type metadata about files. Can you also post a screenshot of the metadata for the boto3-created file?

> [Python][C++] S3Filesystem: file Content-Type not set correctly?
> ----------------------------------------------------------------
>
>                 Key: ARROW-11161
>                 URL: https://issues.apache.org/jira/browse/ARROW-11161
>             Project: Apache Arrow
>          Issue Type: Bug
>    Affects Versions: 2.0.0
>            Reporter: Nicolas Renkamp
>            Priority: Major
>              Labels: filesystem
>         Attachments: Screen Shot 2021-01-07 at 15.23.07.png
>
>
> I am using the Fileystem abstraction to write out html / text files to the local filesystem as well as s3.
> I noticed that when using s3_fs.open_output_stream in combination with file.write(bytes), the object that gets created has a Content-Type of 'application/xml' even tough it's plain text, which is problematic for me.
> Here is a minimal example:
> {code:java}
> import boto3
> BUCKET = "my-bucket"
> path = f"s3://{BUCKET}/pyarrow_encoding.txt"
> s3_fs, output_path = FileSystem.from_uri(path)
> with s3_fs.open_output_stream(path=output_path, compression=None) as f:
>     f.write('hello'.encode('UTF-8'))
> s3 = boto3.client('s3')
> response = s3.get_object(Bucket=BUCKET, Key='pyarrow_encoding.txt')
> print(response['ContentType']) # Output: application/xml
> print(response['Body'].read().decode('UTF-8')) # Output: hello
> s3.put_object(Bucket=BUCKET,
>               Key='boto3_encoding.txt',
>               Body='hello'.encode('UTF-8'))
> response = s3.get_object(Bucket=BUCKET, Key='boto3_encoding.txt')
> print(response['ContentType']) # Output: binary/octet-stream
> print(response['Body'].read().decode('UTF-8')) # Output: hello
> {code}
> I know, that the S3Filesystem implementation of pyarrow might no have mime type inference implemented, but I am wondering, why always 'application/xml' is the resulting Content-Type? Maybe this is hardcoded somewhere?
> Originally, I tried this with '.html' files and also there, the objects on s3 always got the 'application/xml' Content-Type. (Please also see attachment from the s3 console)
>  
> Any help or pointer is appreciated. 
> Thank you,
> Nicolas



--
This message was sent by Atlassian Jira
(v8.3.4#803005)