You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Ramya R (JIRA)" <ji...@apache.org> on 2008/09/29 08:32:44 UTC
[jira] Created: (HADOOP-4300) When fs.trash.interval is set to
non-zero value, the deleted files and directory which are in .Trash are not
getting removed from there after
When fs.trash.interval is set to non-zero value, the deleted files and directory which are in .Trash are not getting removed from there after <fs.trash.interval>
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
Key: HADOOP-4300
URL: https://issues.apache.org/jira/browse/HADOOP-4300
Project: Hadoop Core
Issue Type: Bug
Components: dfs
Affects Versions: 0.19.0
Reporter: Ramya R
Priority: Minor
Fix For: 0.19.0
Set fs.trash.interval to non zero value(say 1), touch a file (say file.txt) and delete it. The expected behavior would be that file.txt is moved to .Trash and also file.txt is removed from .Trash after 1min. But the observed behavior is that, even though file.txt is being moved to .Trash, it is not removed from .Trash after 1min.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4300) When fs.trash.interval is set to
non-zero value, the deleted files and directory which are in .Trash are not
getting removed from there after
Posted by "Ramya R (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639724#action_12639724 ]
Ramya R commented on HADOOP-4300:
---------------------------------
Sorry, for not being clear. I am using HOD to create a HDFS cluster.
I am not making use of the static external HDFS.
> When fs.trash.interval is set to non-zero value, the deleted files and directory which are in .Trash are not getting removed from there after <fs.trash.interval>
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-4300
> URL: https://issues.apache.org/jira/browse/HADOOP-4300
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Ramya R
> Priority: Blocker
> Fix For: 0.19.0
>
>
> Set fs.trash.interval to non zero value(say 1), touch a file (say file.txt) and delete it. The expected behavior would be that file.txt is moved to .Trash and also file.txt is removed from .Trash after 1min. But the observed behavior is that, even though file.txt is being moved to .Trash, it is not removed from .Trash after 1min.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4300) When fs.trash.interval is set to
non-zero value, the deleted files and directory which are in .Trash are not
getting removed from there after
Posted by "Ramya R (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12635995#action_12635995 ]
Ramya R commented on HADOOP-4300:
---------------------------------
Sorry for the mistake Chris. Koji pointed out the right way to execute it. Here is what I followed:
1) Allocate a cluster through HOD and pass --gridservice-hdfs.cmdline-params=fs.trash.interval=10 to HOD.
2) Append ><property><name>fs.trash.interval</name> <value>10</value> <description>Number of minutes between trash checkpoints. If zero, the trash feature is disabled. </description></property> in hadoop-site.xml under the <cluster_dir>
3) Create some files on HDFS and delete it.
The files are getting deleted and being moved to .Trash but not being removed from .Trash after <fs.trash.interval>
Interestingly, I set up a stand alone mode and the files were removed from .Trash after <fs.trash.interval>. But the same behavior was not seen in distributed mode.
> When fs.trash.interval is set to non-zero value, the deleted files and directory which are in .Trash are not getting removed from there after <fs.trash.interval>
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-4300
> URL: https://issues.apache.org/jira/browse/HADOOP-4300
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Ramya R
> Priority: Minor
> Fix For: 0.19.0
>
>
> Set fs.trash.interval to non zero value(say 1), touch a file (say file.txt) and delete it. The expected behavior would be that file.txt is moved to .Trash and also file.txt is removed from .Trash after 1min. But the observed behavior is that, even though file.txt is being moved to .Trash, it is not removed from .Trash after 1min.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4300) When fs.trash.interval is set to
non-zero value, the deleted files and directory which are in .Trash are not
getting removed from there after
Posted by "Koji Noguchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12635438#action_12635438 ]
Koji Noguchi commented on HADOOP-4300:
--------------------------------------
I haven't tested on 0.19, but we've been using the Trash feature on our clusters for some time .
Could you check if Namenode config also has the non-zero fs.trash.interval?
- Client uses 'fs.trash.interval' to determine whether to delete files or move them to Trash.
- Namenode uses 'fs.trash.interval' for running the expunge thread to clean up the users' Trash directories.
So your files should be deleted between namenode's 'fs.trash.interval to 'fs.trash.interval * 2' period.
> When fs.trash.interval is set to non-zero value, the deleted files and directory which are in .Trash are not getting removed from there after <fs.trash.interval>
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-4300
> URL: https://issues.apache.org/jira/browse/HADOOP-4300
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Ramya R
> Priority: Minor
> Fix For: 0.19.0
>
>
> Set fs.trash.interval to non zero value(say 1), touch a file (say file.txt) and delete it. The expected behavior would be that file.txt is moved to .Trash and also file.txt is removed from .Trash after 1min. But the observed behavior is that, even though file.txt is being moved to .Trash, it is not removed from .Trash after 1min.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4300) When fs.trash.interval is set to
non-zero value, the deleted files and directory which are in .Trash are not
getting removed from there after
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-4300:
--------------------------------
Fix Version/s: (was: 0.19.0)
Please mark it as a blocker for 0.19 if required
> When fs.trash.interval is set to non-zero value, the deleted files and directory which are in .Trash are not getting removed from there after <fs.trash.interval>
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-4300
> URL: https://issues.apache.org/jira/browse/HADOOP-4300
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Ramya R
> Priority: Minor
>
> Set fs.trash.interval to non zero value(say 1), touch a file (say file.txt) and delete it. The expected behavior would be that file.txt is moved to .Trash and also file.txt is removed from .Trash after 1min. But the observed behavior is that, even though file.txt is being moved to .Trash, it is not removed from .Trash after 1min.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4300) When fs.trash.interval is set to
non-zero value, the deleted files and directory which are in .Trash are not
getting removed from there after
Posted by "Sameer Paranjpye (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12638648#action_12638648 ]
Sameer Paranjpye commented on HADOOP-4300:
------------------------------------------
Are you using HOD to create an HDFS cluster? It's not clear from your description that you are.
If you allocate a map/reduce only cluster through HOD set fs.trash.interval for your HOD allocation then that setting will not affect a static external HDFS.
> When fs.trash.interval is set to non-zero value, the deleted files and directory which are in .Trash are not getting removed from there after <fs.trash.interval>
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-4300
> URL: https://issues.apache.org/jira/browse/HADOOP-4300
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Ramya R
> Priority: Blocker
> Fix For: 0.19.0
>
>
> Set fs.trash.interval to non zero value(say 1), touch a file (say file.txt) and delete it. The expected behavior would be that file.txt is moved to .Trash and also file.txt is removed from .Trash after 1min. But the observed behavior is that, even though file.txt is being moved to .Trash, it is not removed from .Trash after 1min.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4300) When fs.trash.interval is set to
non-zero value, the deleted files and directory which are in .Trash are not
getting removed from there after
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12635875#action_12635875 ]
Chris Douglas commented on HADOOP-4300:
---------------------------------------
Wait... just to be clear, _after_ the cluster was allocated, you updated the config file of the running namenode? If so, that's not supported and the parameter needs to be given to HOD prior to allocation.
> When fs.trash.interval is set to non-zero value, the deleted files and directory which are in .Trash are not getting removed from there after <fs.trash.interval>
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-4300
> URL: https://issues.apache.org/jira/browse/HADOOP-4300
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Ramya R
> Priority: Minor
> Fix For: 0.19.0
>
>
> Set fs.trash.interval to non zero value(say 1), touch a file (say file.txt) and delete it. The expected behavior would be that file.txt is moved to .Trash and also file.txt is removed from .Trash after 1min. But the observed behavior is that, even though file.txt is being moved to .Trash, it is not removed from .Trash after 1min.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-4300) When fs.trash.interval is set to
non-zero value, the deleted files and directory which are in .Trash are not
getting removed from there after
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas resolved HADOOP-4300.
-----------------------------------
Resolution: Cannot Reproduce
bq. Interestingly, I set up a stand alone mode and the files were removed from .Trash after <fs.trash.interval>. But the same behavior was not seen in distributed mode.
It is difficult to imagine circumstances where this would be true. I tried to reproduce this on a small, non-HOD cluster using (0.19, r704262) and trash worked as expected. It looks like this issue is with HOD or its configuration.
> When fs.trash.interval is set to non-zero value, the deleted files and directory which are in .Trash are not getting removed from there after <fs.trash.interval>
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-4300
> URL: https://issues.apache.org/jira/browse/HADOOP-4300
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Ramya R
> Priority: Blocker
> Fix For: 0.19.0
>
>
> Set fs.trash.interval to non zero value(say 1), touch a file (say file.txt) and delete it. The expected behavior would be that file.txt is moved to .Trash and also file.txt is removed from .Trash after 1min. But the observed behavior is that, even though file.txt is being moved to .Trash, it is not removed from .Trash after 1min.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4300) When fs.trash.interval is set to
non-zero value, the deleted files and directory which are in .Trash are not
getting removed from there after
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nigel Daley updated HADOOP-4300:
--------------------------------
Priority: Blocker (was: Major)
Fix Version/s: 0.19.0
> When fs.trash.interval is set to non-zero value, the deleted files and directory which are in .Trash are not getting removed from there after <fs.trash.interval>
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-4300
> URL: https://issues.apache.org/jira/browse/HADOOP-4300
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Ramya R
> Priority: Blocker
> Fix For: 0.19.0
>
>
> Set fs.trash.interval to non zero value(say 1), touch a file (say file.txt) and delete it. The expected behavior would be that file.txt is moved to .Trash and also file.txt is removed from .Trash after 1min. But the observed behavior is that, even though file.txt is being moved to .Trash, it is not removed from .Trash after 1min.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4300) When fs.trash.interval is set to
non-zero value, the deleted files and directory which are in .Trash are not
getting removed from there after
Posted by "Ramya R (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12635658#action_12635658 ]
Ramya R commented on HADOOP-4300:
---------------------------------
To explain it in more detail, here is how I carried out the test:
1) Allocate a cluster through HOD
2) Append ><property><name>fs.trash.interval</name> <value>1</value> <description>Number of minutes between trash checkpoints. If zero, the trash feature is disabled. </description></property> in hadoop-site.xml under the <cluster_dir>
3) ssh to the machine where name node is running and append the same above line in <hodring.temp-dir>/<username>.<jobid>.<hostname>.hodring/0-namenode/confdir
4) Create some files on HDFS and delete it.
The files are getting deleted and being moved to .Trash but not being removed from .Trash after <fs.trash.interval>
> When fs.trash.interval is set to non-zero value, the deleted files and directory which are in .Trash are not getting removed from there after <fs.trash.interval>
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-4300
> URL: https://issues.apache.org/jira/browse/HADOOP-4300
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Ramya R
> Priority: Minor
> Fix For: 0.19.0
>
>
> Set fs.trash.interval to non zero value(say 1), touch a file (say file.txt) and delete it. The expected behavior would be that file.txt is moved to .Trash and also file.txt is removed from .Trash after 1min. But the observed behavior is that, even though file.txt is being moved to .Trash, it is not removed from .Trash after 1min.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4300) When fs.trash.interval is set to
non-zero value, the deleted files and directory which are in .Trash are not
getting removed from there after
Posted by "Ramya R (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12637113#action_12637113 ]
Ramya R commented on HADOOP-4300:
---------------------------------
Verified the issue with Milind and confirming it as a blocker for 0.19.
> When fs.trash.interval is set to non-zero value, the deleted files and directory which are in .Trash are not getting removed from there after <fs.trash.interval>
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-4300
> URL: https://issues.apache.org/jira/browse/HADOOP-4300
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Ramya R
> Priority: Minor
>
> Set fs.trash.interval to non zero value(say 1), touch a file (say file.txt) and delete it. The expected behavior would be that file.txt is moved to .Trash and also file.txt is removed from .Trash after 1min. But the observed behavior is that, even though file.txt is being moved to .Trash, it is not removed from .Trash after 1min.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4300) When fs.trash.interval is set to
non-zero value, the deleted files and directory which are in .Trash are not
getting removed from there after
Posted by "Ramya R (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ramya R updated HADOOP-4300:
----------------------------
Priority: Major (was: Minor)
> When fs.trash.interval is set to non-zero value, the deleted files and directory which are in .Trash are not getting removed from there after <fs.trash.interval>
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-4300
> URL: https://issues.apache.org/jira/browse/HADOOP-4300
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Ramya R
>
> Set fs.trash.interval to non zero value(say 1), touch a file (say file.txt) and delete it. The expected behavior would be that file.txt is moved to .Trash and also file.txt is removed from .Trash after 1min. But the observed behavior is that, even though file.txt is being moved to .Trash, it is not removed from .Trash after 1min.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4300) When fs.trash.interval is set to
non-zero value, the deleted files and directory which are in .Trash are not
getting removed from there after
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12635436#action_12635436 ]
Nigel Daley commented on HADOOP-4300:
-------------------------------------
Ramya, how are you setting the interval to 1 minute? in hodrc or hadoop-site.xml?
If this is truly broken, why is this marked "minor"?
> When fs.trash.interval is set to non-zero value, the deleted files and directory which are in .Trash are not getting removed from there after <fs.trash.interval>
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-4300
> URL: https://issues.apache.org/jira/browse/HADOOP-4300
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Ramya R
> Priority: Minor
> Fix For: 0.19.0
>
>
> Set fs.trash.interval to non zero value(say 1), touch a file (say file.txt) and delete it. The expected behavior would be that file.txt is moved to .Trash and also file.txt is removed from .Trash after 1min. But the observed behavior is that, even though file.txt is being moved to .Trash, it is not removed from .Trash after 1min.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.