[jira] [Created] (HDDS-5975) S3G: object key name should be encoded

["Kohei Sugihara (Jira)" <ji...@apache.org>]

Kohei Sugihara created HDDS-5975:
------------------------------------

             Summary: S3G: object key name should be encoded
                 Key: HDDS-5975
                 URL: https://issues.apache.org/jira/browse/HDDS-5975
             Project: Apache Ozone
          Issue Type: Bug
          Components: S3
    Affects Versions: 1.1.0
            Reporter: Kohei Sugihara
            Assignee: Kohei Sugihara


S3G will not work properly for some objects that is includes `+` and ` ` in a key/prefix name. They should be url-encoded.
h1. Quick reproduce

`aws s3 rm --recursive` will not work correctly because it confuse `+` and ` `.
{code:sh}
# create files and sync them
% ls special-files -l
total 0
-rw-r--r-- 1 ksugihara ksugihara 0 Nov 12 08:39 '&=;:+ ,?'
-rw-r--r-- 1 ksugihara ksugihara 0 Nov 12 08:36 '<abc>'
-rw-r--r-- 1 ksugihara ksugihara 0 Nov 12 08:41 '\{^}%'
-rw-r--r-- 1 ksugihara ksugihara 0 Nov 12 08:41 '\{^}%`]>[~<#|'
-rw-r--r-- 1 ksugihara ksugihara 0 Nov 12 08:24 'a b c'
-rw-r--r-- 1 ksugihara ksugihara 0 Nov 12 08:21  a+b+c

% aws s3 --endpoint HOST sync special-files s3://should-be-deleted
upload: special-files/&=;:+ ,? to s3://should-be-deleted/&=;:+ ,?
upload: special-files/<abc> to s3://should-be-deleted/<abc>
upload: special-files/a b c to s3://should-be-deleted/a b c
upload: special-files/a+b+c to s3://should-be-deleted/a+b+c
upload: special-files/\{^}%`]>[~<#| to s3://should-be-deleted/\{^}%`]>[~<#|
upload: special-files/\{^}% to s3://should-be-deleted/\{^}%

# two 'a b c' files in this bucket. '+' and ' ' should be identified
% aws s3 --endpoint HOST ls s3://should-be-deleted/
2021-11-12 14:34:29          0 &=;:  ,?
2021-11-12 14:34:29          0 <abc>
2021-11-12 14:34:29          0 \{^}%
2021-11-12 14:34:29          0 \{^}%`]>[~<#|
2021-11-12 14:34:29          0 a b c
2021-11-12 14:34:29          0 a b c

# let's delete all files in the bucket
% aws s3 --endpoint HOST rm --recursive s3://should-be-deleted/
delete: s3://should-be-deleted/&=;:  ,?
delete: s3://should-be-deleted/<abc>
delete: s3://should-be-deleted/a b c
delete: s3://should-be-deleted/\{^}%
delete: s3://should-be-deleted/a b c
delete: s3://should-be-deleted/\{^}%`]>[~<#|

# several objects still remain
% aws s3 --endpoint HOST ls s3://should-be-deleted
2021-11-12 14:34:29          0 &=;:  ,?
2021-11-12 14:34:29          0 a b c
{code}
{code:xml}
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Name>should-be-deleted</Name>
  <Prefix></Prefix>
  <Marker></Marker>
  <MaxKeys>1000</MaxKeys>
  <KeyCount>2</KeyCount>
  <Delimiter>/</Delimiter>
  <EncodingType>url</EncodingType>
  <IsTruncated>false</IsTruncated>
  <Contents>
    <Key>&amp;=;:+ ,?</Key>
    <LastModified>2021-11-12T05:34:29.337Z</LastModified>
    <ETag>2021-11-12T05:34:29.337Z</ETag>
    <Size>0</Size>
    <StorageClass>STANDARD</StorageClass>
  </Contents>
  <Contents>
    <Key>a+b+c</Key>
    <LastModified>2021-11-12T05:34:29.610Z</LastModified>
    <ETag>2021-11-12T05:34:29.610Z</ETag>
    <Size>0</Size>
    <StorageClass>STANDARD</StorageClass>
  </Contents>
</ListBucketResult>
{code}
h1. References
 * [https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-keys.html] – a guideline for object key name & special handlings
 * https://issues.apache.org/jira/browse/HDDS-3161 – plus (`+`) is allowed here 

 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org