You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2021/11/12 06:28:00 UTC
[jira] [Updated] (HDDS-5975) S3G: object key name should be encoded
[ https://issues.apache.org/jira/browse/HDDS-5975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ASF GitHub Bot updated HDDS-5975:
---------------------------------
Labels: pull-request-available (was: )
> S3G: object key name should be encoded
> --------------------------------------
>
> Key: HDDS-5975
> URL: https://issues.apache.org/jira/browse/HDDS-5975
> Project: Apache Ozone
> Issue Type: Bug
> Components: S3
> Affects Versions: 1.1.0
> Reporter: Kohei Sugihara
> Assignee: Kohei Sugihara
> Priority: Major
> Labels: pull-request-available
>
> S3G will not work properly for some objects that is includes `+` and ` ` in a key/prefix name. They should be url-encoded.
> h1. Quick reproduce
> `aws s3 rm --recursive` will not work correctly because it confuse `+` and ` `.
> {code:sh}
> # create files and sync them
> % ls special-files -l
> total 0
> -rw-r--r-- 1 ksugihara ksugihara 0 Nov 12 08:39 '&=;:+ ,?'
> -rw-r--r-- 1 ksugihara ksugihara 0 Nov 12 08:36 '<abc>'
> -rw-r--r-- 1 ksugihara ksugihara 0 Nov 12 08:41 '\{^}%'
> -rw-r--r-- 1 ksugihara ksugihara 0 Nov 12 08:41 '\{^}%`]>[~<#|'
> -rw-r--r-- 1 ksugihara ksugihara 0 Nov 12 08:24 'a b c'
> -rw-r--r-- 1 ksugihara ksugihara 0 Nov 12 08:21 a+b+c
> % aws s3 --endpoint HOST sync special-files s3://should-be-deleted
> upload: special-files/&=;:+ ,? to s3://should-be-deleted/&=;:+ ,?
> upload: special-files/<abc> to s3://should-be-deleted/<abc>
> upload: special-files/a b c to s3://should-be-deleted/a b c
> upload: special-files/a+b+c to s3://should-be-deleted/a+b+c
> upload: special-files/\{^}%`]>[~<#| to s3://should-be-deleted/\{^}%`]>[~<#|
> upload: special-files/\{^}% to s3://should-be-deleted/\{^}%
> # two 'a b c' files in this bucket. '+' and ' ' should be identified
> % aws s3 --endpoint HOST ls s3://should-be-deleted/
> 2021-11-12 14:34:29 0 &=;: ,?
> 2021-11-12 14:34:29 0 <abc>
> 2021-11-12 14:34:29 0 \{^}%
> 2021-11-12 14:34:29 0 \{^}%`]>[~<#|
> 2021-11-12 14:34:29 0 a b c
> 2021-11-12 14:34:29 0 a b c
> # let's delete all files in the bucket
> % aws s3 --endpoint HOST rm --recursive s3://should-be-deleted/
> delete: s3://should-be-deleted/&=;: ,?
> delete: s3://should-be-deleted/<abc>
> delete: s3://should-be-deleted/a b c
> delete: s3://should-be-deleted/\{^}%
> delete: s3://should-be-deleted/a b c
> delete: s3://should-be-deleted/\{^}%`]>[~<#|
> # several objects still remain
> % aws s3 --endpoint HOST ls s3://should-be-deleted
> 2021-11-12 14:34:29 0 &=;: ,?
> 2021-11-12 14:34:29 0 a b c
> {code}
> {code:xml}
> <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> <ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
> <Name>should-be-deleted</Name>
> <Prefix></Prefix>
> <Marker></Marker>
> <MaxKeys>1000</MaxKeys>
> <KeyCount>2</KeyCount>
> <Delimiter>/</Delimiter>
> <EncodingType>url</EncodingType>
> <IsTruncated>false</IsTruncated>
> <Contents>
> <Key>&=;:+ ,?</Key>
> <LastModified>2021-11-12T05:34:29.337Z</LastModified>
> <ETag>2021-11-12T05:34:29.337Z</ETag>
> <Size>0</Size>
> <StorageClass>STANDARD</StorageClass>
> </Contents>
> <Contents>
> <Key>a+b+c</Key>
> <LastModified>2021-11-12T05:34:29.610Z</LastModified>
> <ETag>2021-11-12T05:34:29.610Z</ETag>
> <Size>0</Size>
> <StorageClass>STANDARD</StorageClass>
> </Contents>
> </ListBucketResult>
> {code}
> h1. References
> * [https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-keys.html] – a guideline for object key name & special handlings
> * https://issues.apache.org/jira/browse/HDDS-3161 – plus (`+`) is allowed here
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org