You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2020/09/14 20:00:00 UTC
[jira] [Work logged] (HADOOP-17261) s3a rename() now requires
s3:deleteObjectVersion permission
[ https://issues.apache.org/jira/browse/HADOOP-17261?focusedWorklogId=484193&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-484193 ]
ASF GitHub Bot logged work on HADOOP-17261:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 14/Sep/20 19:59
Start Date: 14/Sep/20 19:59
Worklog Time Spent: 10m
Work Description: steveloughran opened a new pull request #2303:
URL: https://github.com/apache/hadoop/pull/2303
Stop using the versionId when building the list of files
to delete as the rename progresses
Testing: manual. For anyone wishing to replicate the problem on an unpatched release
1. Have a versioned bucket.
1. Latest version of cloudstore now allows you to create a set of session keys from an IAM role and json file, printing them as env vars and hadoop config options
1. You need S3Guard enabled and the files to have been created individually by a client through S3Guard (i.e. the entries not discovered through listing). Then S3Guard caches the version ID in the DB, which is then used when deleting the object.
1. mkdir src/; touchz src/file1 file2
2. mv src dst
if that works, try renaming back.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 484193)
Remaining Estimate: 0h
Time Spent: 10m
> s3a rename() now requires s3:deleteObjectVersion permission
> -----------------------------------------------------------
>
> Key: HADOOP-17261
> URL: https://issues.apache.org/jira/browse/HADOOP-17261
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.4.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> With the directory marker change (HADOOP-13230) you need the s3:deleteObjectVersion permission in your role, else the operation will fail in the bulk delete, *if S3Guard is in use*
> Root cause
> -if fileStatus has a versionId, we pass that in to the delete KeyVersion pair
> -an unguarded listing doesn't get that versionId, so this is not an issue
> -but if files in a directory were previously created such that S3Guard has their versionId in its tables, that is used in the request
> -which then fails if the caller doesn't have the permission
> Although we say "you need s3:delete*", this is a regression as any IAM role without the permission will have rename fail during delete
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org