You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Eli Collins (JIRA)" <ji...@apache.org> on 2012/08/13 22:06:38 UTC
[jira] [Created] (HADOOP-8690) Shell may remove a file without
going to trash even if skipTrash is not enabled
Eli Collins created HADOOP-8690:
-----------------------------------
Summary: Shell may remove a file without going to trash even if skipTrash is not enabled
Key: HADOOP-8690
URL: https://issues.apache.org/jira/browse/HADOOP-8690
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.0.0-alpha
Reporter: Eli Collins
Priority: Minor
Delete.java contains the following comment:
{noformat}
// TODO: if the user wants the trash to be used but there is any
// problem (ie. creating the trash dir, moving the item to be deleted,
// etc), then the path will just be deleted because moveToTrash returns
// false and it falls thru to fs.delete. this doesn't seem right
{noformat}
If Trash#moveToAppropriateTrash returns false FsShell will delete the path even if skipTrash is not enabled. The comment isn't quite right as some of these failure scenarios result in exceptions not a false return value, and in the case of an exception we don't unconditionally delete the path. TrashPolicy#moveToTrash states that it only returns false if the item is already in the trash or trash is disabled, and the expected behavior for these cases is to just delete the path. However TrashPolicyDefault#moveToTrash also returns false if there's a problem creating the trash directory, so for this case I don't think we should throw an exception rather than return false.
I also question the behavior of just deleting when the item is already in the trash as it may have changed since previously put in the trash and not been checkpointed yet. Seems like in this case we should move it to trash but with a file name suffix.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8690) Shell may remove a file without
going to trash even if skipTrash is not enabled
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13434274#comment-13434274 ]
Eli Collins commented on HADOOP-8690:
-------------------------------------
Looks like there's a bug in the following check in TrashPolicyDefault#moveToTrash and it's falling through to the code below that appends the current time in milliseconds to the file name.
{code}
if (qpath.startsWith(trash.toString())) {
return false; // already in trash
}
{code}
> Shell may remove a file without going to trash even if skipTrash is not enabled
> -------------------------------------------------------------------------------
>
> Key: HADOOP-8690
> URL: https://issues.apache.org/jira/browse/HADOOP-8690
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Priority: Minor
>
> Delete.java contains the following comment:
> {noformat}
> // TODO: if the user wants the trash to be used but there is any
> // problem (ie. creating the trash dir, moving the item to be deleted,
> // etc), then the path will just be deleted because moveToTrash returns
> // false and it falls thru to fs.delete. this doesn't seem right
> {noformat}
> If Trash#moveToAppropriateTrash returns false FsShell will delete the path even if skipTrash is not enabled. The comment isn't quite right as some of these failure scenarios result in exceptions not a false return value, and in the case of an exception we don't unconditionally delete the path. TrashPolicy#moveToTrash states that it only returns false if the item is already in the trash or trash is disabled, and the expected behavior for these cases is to just delete the path. However TrashPolicyDefault#moveToTrash also returns false if there's a problem creating the trash directory, so for this case I think we should throw an exception rather than return false (and delete the path bypassing trash).
> I also question the behavior of just deleting when the item is already in the trash as it may have changed since previously put in the trash and not been checkpointed yet. Seems like in this case we should move it to trash but with a file name suffix.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8690) Shell may remove a file without
going to trash even if skipTrash is not enabled
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HADOOP-8690:
--------------------------------
Description:
Delete.java contains the following comment:
{noformat}
// TODO: if the user wants the trash to be used but there is any
// problem (ie. creating the trash dir, moving the item to be deleted,
// etc), then the path will just be deleted because moveToTrash returns
// false and it falls thru to fs.delete. this doesn't seem right
{noformat}
If Trash#moveToAppropriateTrash returns false FsShell will delete the path even if skipTrash is not enabled. The comment isn't quite right as some of these failure scenarios result in exceptions not a false return value, and in the case of an exception we don't unconditionally delete the path. TrashPolicy#moveToTrash states that it only returns false if the item is already in the trash or trash is disabled, and the expected behavior for these cases is to just delete the path. However TrashPolicyDefault#moveToTrash also returns false if there's a problem creating the trash directory, so for this case I think we should throw an exception rather than return false (and delete the path bypassing trash).
I also question the behavior of just deleting when the item is already in the trash as it may have changed since previously put in the trash and not been checkpointed yet. Seems like in this case we should move it to trash but with a file name suffix.
was:
Delete.java contains the following comment:
{noformat}
// TODO: if the user wants the trash to be used but there is any
// problem (ie. creating the trash dir, moving the item to be deleted,
// etc), then the path will just be deleted because moveToTrash returns
// false and it falls thru to fs.delete. this doesn't seem right
{noformat}
If Trash#moveToAppropriateTrash returns false FsShell will delete the path even if skipTrash is not enabled. The comment isn't quite right as some of these failure scenarios result in exceptions not a false return value, and in the case of an exception we don't unconditionally delete the path. TrashPolicy#moveToTrash states that it only returns false if the item is already in the trash or trash is disabled, and the expected behavior for these cases is to just delete the path. However TrashPolicyDefault#moveToTrash also returns false if there's a problem creating the trash directory, so for this case I don't think we should throw an exception rather than return false.
I also question the behavior of just deleting when the item is already in the trash as it may have changed since previously put in the trash and not been checkpointed yet. Seems like in this case we should move it to trash but with a file name suffix.
> Shell may remove a file without going to trash even if skipTrash is not enabled
> -------------------------------------------------------------------------------
>
> Key: HADOOP-8690
> URL: https://issues.apache.org/jira/browse/HADOOP-8690
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Priority: Minor
>
> Delete.java contains the following comment:
> {noformat}
> // TODO: if the user wants the trash to be used but there is any
> // problem (ie. creating the trash dir, moving the item to be deleted,
> // etc), then the path will just be deleted because moveToTrash returns
> // false and it falls thru to fs.delete. this doesn't seem right
> {noformat}
> If Trash#moveToAppropriateTrash returns false FsShell will delete the path even if skipTrash is not enabled. The comment isn't quite right as some of these failure scenarios result in exceptions not a false return value, and in the case of an exception we don't unconditionally delete the path. TrashPolicy#moveToTrash states that it only returns false if the item is already in the trash or trash is disabled, and the expected behavior for these cases is to just delete the path. However TrashPolicyDefault#moveToTrash also returns false if there's a problem creating the trash directory, so for this case I think we should throw an exception rather than return false (and delete the path bypassing trash).
> I also question the behavior of just deleting when the item is already in the trash as it may have changed since previously put in the trash and not been checkpointed yet. Seems like in this case we should move it to trash but with a file name suffix.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8690) Shell may remove a file without
going to trash even if skipTrash is not enabled
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13433488#comment-13433488 ]
Daryn Sharp commented on HADOOP-8690:
-------------------------------------
I added that comment a long time ago. It looks like the code has changed a lot, but I think I agree with all your current observations.
> Shell may remove a file without going to trash even if skipTrash is not enabled
> -------------------------------------------------------------------------------
>
> Key: HADOOP-8690
> URL: https://issues.apache.org/jira/browse/HADOOP-8690
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Priority: Minor
>
> Delete.java contains the following comment:
> {noformat}
> // TODO: if the user wants the trash to be used but there is any
> // problem (ie. creating the trash dir, moving the item to be deleted,
> // etc), then the path will just be deleted because moveToTrash returns
> // false and it falls thru to fs.delete. this doesn't seem right
> {noformat}
> If Trash#moveToAppropriateTrash returns false FsShell will delete the path even if skipTrash is not enabled. The comment isn't quite right as some of these failure scenarios result in exceptions not a false return value, and in the case of an exception we don't unconditionally delete the path. TrashPolicy#moveToTrash states that it only returns false if the item is already in the trash or trash is disabled, and the expected behavior for these cases is to just delete the path. However TrashPolicyDefault#moveToTrash also returns false if there's a problem creating the trash directory, so for this case I don't think we should throw an exception rather than return false.
> I also question the behavior of just deleting when the item is already in the trash as it may have changed since previously put in the trash and not been checkpointed yet. Seems like in this case we should move it to trash but with a file name suffix.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8690) Shell may remove a file without
going to trash even if skipTrash is not enabled
Posted by "Koji Noguchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13434240#comment-13434240 ]
Koji Noguchi commented on HADOOP-8690:
--------------------------------------
bq. Seems like in this case we should move it to trash but with a file name suffix.
I believe it does that already.
On 0.23,
{noformat}
[knoguchi ~]$ hdfs dfs -touchz abcdef
[knoguchi ~]$ hdfs dfs -rm abcdef
(repeat 3 times)
[knoguchi ~]$ hdfs dfs -ls /user/knoguchi/.Trash/Current/user/knoguchi/abc\*
Found 1 items
-rw-r--r-- 3 knoguchi users 0 2012-08-14 16:30 /user/knoguchi/.Trash/Current/user/knoguchi/abcdef
Found 1 items
-rw-r--r-- 3 knoguchi users 0 2012-08-14 16:31 /user/knoguchi/.Trash/Current/user/knoguchi/abcdef1344961878400
Found 1 items
-rw-r--r-- 3 knoguchi users 0 2012-08-14 16:31 /user/knoguchi/.Trash/Current/user/knoguchi/abcdef1344961912093
{noformat}
> Shell may remove a file without going to trash even if skipTrash is not enabled
> -------------------------------------------------------------------------------
>
> Key: HADOOP-8690
> URL: https://issues.apache.org/jira/browse/HADOOP-8690
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Priority: Minor
>
> Delete.java contains the following comment:
> {noformat}
> // TODO: if the user wants the trash to be used but there is any
> // problem (ie. creating the trash dir, moving the item to be deleted,
> // etc), then the path will just be deleted because moveToTrash returns
> // false and it falls thru to fs.delete. this doesn't seem right
> {noformat}
> If Trash#moveToAppropriateTrash returns false FsShell will delete the path even if skipTrash is not enabled. The comment isn't quite right as some of these failure scenarios result in exceptions not a false return value, and in the case of an exception we don't unconditionally delete the path. TrashPolicy#moveToTrash states that it only returns false if the item is already in the trash or trash is disabled, and the expected behavior for these cases is to just delete the path. However TrashPolicyDefault#moveToTrash also returns false if there's a problem creating the trash directory, so for this case I think we should throw an exception rather than return false (and delete the path bypassing trash).
> I also question the behavior of just deleting when the item is already in the trash as it may have changed since previously put in the trash and not been checkpointed yet. Seems like in this case we should move it to trash but with a file name suffix.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira