You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Eli Collins (JIRA)" <ji...@apache.org> on 2012/08/13 20:53:38 UTC
[jira] [Created] (HADOOP-8689) Make trash a server side
configuration option
Eli Collins created HADOOP-8689:
-----------------------------------
Summary: Make trash a server side configuration option
Key: HADOOP-8689
URL: https://issues.apache.org/jira/browse/HADOOP-8689
Project: Hadoop Common
Issue Type: Improvement
Components: fs
Affects Versions: 2.0.0-alpha
Reporter: Eli Collins
Assignee: Eli Collins
Per ATM's suggestion in HADOOP-8598 for v2 let's make {{fs.trash.interval}} configured server side rather than client side. The {{fs.trash.checkpoint.interval}} option is already server side as the emptier runs in the NameNode.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8689) Make trash a server side
configuration option
Posted by "Kihwal Lee (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13442648#comment-13442648 ]
Kihwal Lee commented on HADOOP-8689:
------------------------------------
In HDFS-3856, namenode does System.exit(1), because getServerDefaults() is not allowed on backup nodes.
> Make trash a server side configuration option
> ---------------------------------------------
>
> Key: HADOOP-8689
> URL: https://issues.apache.org/jira/browse/HADOOP-8689
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Fix For: 2.2.0-alpha
>
> Attachments: hadoop-8689.txt, hadoop-8689.txt
>
>
> Per ATM's suggestion in HADOOP-8598 for v2 let's make {{fs.trash.interval}} configured server side. If it is not configured server side then the client side configuration is used. The {{fs.trash.checkpoint.interval}} option is already server side as the emptier runs in the NameNode. Clients may manually run an emptier via hadoop org.apache.hadoop.fs.Trash but it's OK if it uses a separate interval.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8689) Make trash a server side
configuration option
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437145#comment-13437145 ]
Hudson commented on HADOOP-8689:
--------------------------------
Integrated in Hadoop-Common-trunk-Commit #2601 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2601/])
HADOOP-8689. Make trash a server side configuration option. Contributed by Eli Collins (Revision 1374472)
Result = SUCCESS
eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1374472
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FsServerDefaults.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/TrashPolicyDefault.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ftp/FtpConfigKeys.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/local/LocalConfigKeys.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestTrash.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/proto/hdfs.proto
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestHDFSTrash.java
> Make trash a server side configuration option
> ---------------------------------------------
>
> Key: HADOOP-8689
> URL: https://issues.apache.org/jira/browse/HADOOP-8689
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Fix For: 2.2.0-alpha
>
> Attachments: hadoop-8689.txt, hadoop-8689.txt
>
>
> Per ATM's suggestion in HADOOP-8598 for v2 let's make {{fs.trash.interval}} configured server side. If it is not configured server side then the client side configuration is used. The {{fs.trash.checkpoint.interval}} option is already server side as the emptier runs in the NameNode. Clients may manually run an emptier via hadoop org.apache.hadoop.fs.Trash but it's OK if it uses a separate interval.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8689) Make trash a server side
configuration option
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437171#comment-13437171 ]
Hudson commented on HADOOP-8689:
--------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #2630 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2630/])
HADOOP-8689. Make trash a server side configuration option. Contributed by Eli Collins (Revision 1374472)
Result = FAILURE
eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1374472
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FsServerDefaults.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/TrashPolicyDefault.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ftp/FtpConfigKeys.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/local/LocalConfigKeys.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestTrash.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/proto/hdfs.proto
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestHDFSTrash.java
> Make trash a server side configuration option
> ---------------------------------------------
>
> Key: HADOOP-8689
> URL: https://issues.apache.org/jira/browse/HADOOP-8689
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Fix For: 2.2.0-alpha
>
> Attachments: hadoop-8689.txt, hadoop-8689.txt
>
>
> Per ATM's suggestion in HADOOP-8598 for v2 let's make {{fs.trash.interval}} configured server side. If it is not configured server side then the client side configuration is used. The {{fs.trash.checkpoint.interval}} option is already server side as the emptier runs in the NameNode. Clients may manually run an emptier via hadoop org.apache.hadoop.fs.Trash but it's OK if it uses a separate interval.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8689) Make trash a server side
configuration option
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HADOOP-8689:
--------------------------------
Status: Patch Available (was: Open)
> Make trash a server side configuration option
> ---------------------------------------------
>
> Key: HADOOP-8689
> URL: https://issues.apache.org/jira/browse/HADOOP-8689
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hadoop-8689.txt
>
>
> Per ATM's suggestion in HADOOP-8598 for v2 let's make {{fs.trash.interval}} configured server side rather than client side. The {{fs.trash.checkpoint.interval}} option is already server side as the emptier runs in the NameNode.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8689) Make trash a server side
configuration option
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13436055#comment-13436055 ]
Eli Collins commented on HADOOP-8689:
-------------------------------------
Test failure is unrelated.
> Make trash a server side configuration option
> ---------------------------------------------
>
> Key: HADOOP-8689
> URL: https://issues.apache.org/jira/browse/HADOOP-8689
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hadoop-8689.txt
>
>
> Per ATM's suggestion in HADOOP-8598 for v2 let's make {{fs.trash.interval}} configured server side rather than client side. The {{fs.trash.checkpoint.interval}} option is already server side as the emptier runs in the NameNode.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8689) Make trash a server side
configuration option
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437329#comment-13437329 ]
Hudson commented on HADOOP-8689:
--------------------------------
Integrated in Hadoop-Mapreduce-trunk #1170 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1170/])
HADOOP-8689. Make trash a server side configuration option. Contributed by Eli Collins (Revision 1374472)
Result = FAILURE
eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1374472
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FsServerDefaults.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/TrashPolicyDefault.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ftp/FtpConfigKeys.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/local/LocalConfigKeys.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestTrash.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/proto/hdfs.proto
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestHDFSTrash.java
> Make trash a server side configuration option
> ---------------------------------------------
>
> Key: HADOOP-8689
> URL: https://issues.apache.org/jira/browse/HADOOP-8689
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Fix For: 2.2.0-alpha
>
> Attachments: hadoop-8689.txt, hadoop-8689.txt
>
>
> Per ATM's suggestion in HADOOP-8598 for v2 let's make {{fs.trash.interval}} configured server side. If it is not configured server side then the client side configuration is used. The {{fs.trash.checkpoint.interval}} option is already server side as the emptier runs in the NameNode. Clients may manually run an emptier via hadoop org.apache.hadoop.fs.Trash but it's OK if it uses a separate interval.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8689) Make trash a server side
configuration option
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437307#comment-13437307 ]
Hudson commented on HADOOP-8689:
--------------------------------
Integrated in Hadoop-Hdfs-trunk #1138 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1138/])
HADOOP-8689. Make trash a server side configuration option. Contributed by Eli Collins (Revision 1374472)
Result = FAILURE
eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1374472
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FsServerDefaults.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/TrashPolicyDefault.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ftp/FtpConfigKeys.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/local/LocalConfigKeys.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestTrash.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/proto/hdfs.proto
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestHDFSTrash.java
> Make trash a server side configuration option
> ---------------------------------------------
>
> Key: HADOOP-8689
> URL: https://issues.apache.org/jira/browse/HADOOP-8689
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Fix For: 2.2.0-alpha
>
> Attachments: hadoop-8689.txt, hadoop-8689.txt
>
>
> Per ATM's suggestion in HADOOP-8598 for v2 let's make {{fs.trash.interval}} configured server side. If it is not configured server side then the client side configuration is used. The {{fs.trash.checkpoint.interval}} option is already server side as the emptier runs in the NameNode. Clients may manually run an emptier via hadoop org.apache.hadoop.fs.Trash but it's OK if it uses a separate interval.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8689) Make trash a server side
configuration option
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HADOOP-8689:
--------------------------------
Attachment: hadoop-8689.txt
Thanks for the review ATM. Updated patch attached.
#1 I moved the initialization of the checkpoint interval to trash policy creation (where the deletion interval is set) rather than when an emptier object is created so the emptier uses the interval configured by the trash policy that creates the emptier. I think this is more clear, and in practice doesn't change things since the just NN creates an emptier once. As an aside, the emptier should probably only ever be run from the server side (that's mostly true currently) therefore the checkpoint interval is really server side as well (unlike the trash interval shell clients don't run emptiers). I filed HADOOP-8710 to remove the one place where a user might be able to run an emptier (w/o explicitly creating objects themselves).
#2 Fixed, the content of that description was actually incorrect so I update it.
#3 Good suggestion done.
I noticed the log for setting the checkpoint interval if its not set is a warn, made that an info, also it prints values in ms but the message says "seconds" so I fixed that bug as well.
> Make trash a server side configuration option
> ---------------------------------------------
>
> Key: HADOOP-8689
> URL: https://issues.apache.org/jira/browse/HADOOP-8689
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hadoop-8689.txt, hadoop-8689.txt
>
>
> Per ATM's suggestion in HADOOP-8598 for v2 let's make {{fs.trash.interval}} configured server side. If it is not configured server side then the client side configuration is used. The {{fs.trash.checkpoint.interval}} option is already server side as the emptier runs in the NameNode. Clients may manually run an emptier via hadoop org.apache.hadoop.fs.Trash but it's OK if it uses a separate interval.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8689) Make trash a server side
configuration option
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HADOOP-8689:
--------------------------------
Attachment: hadoop-8689.txt
Patch attached. If {{fs.trash.interval}} is enabled server side then the value configured on the server is always used by clients. If it is not enabled server side then the client configuration is still respected for compatibility.
> Make trash a server side configuration option
> ---------------------------------------------
>
> Key: HADOOP-8689
> URL: https://issues.apache.org/jira/browse/HADOOP-8689
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hadoop-8689.txt
>
>
> Per ATM's suggestion in HADOOP-8598 for v2 let's make {{fs.trash.interval}} configured server side rather than client side. The {{fs.trash.checkpoint.interval}} option is already server side as the emptier runs in the NameNode.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8689) Make trash a server side
configuration option
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13436388#comment-13436388 ]
Aaron T. Myers commented on HADOOP-8689:
----------------------------------------
Patch looks pretty good to me. Three small comments:
# Unless I'm missing something, it seems like moving the configuration of the client-side emptier interval (not deletion interval) into TrashPolicyDefault#initialize is an unrelated change, given that it's still configured via the client-side conf only. If I'm right about that, seems like it should be done as a separate change, (or perhaps not done at all; it makes sense to me that the emptier interval would be configured only in the Emptier class.)
# One too many "is": "If zero, trash emptying is is disabled."
# In the tests, I recommend you make a copy of the Configuration object before checking the client side value, to ensure there's no issue with the configuration references in fact referring to the same object between the client and server. (I don't think there is such a problem, but making a copy of the conf object will make the test a little more robust to this sort of issue.)
> Make trash a server side configuration option
> ---------------------------------------------
>
> Key: HADOOP-8689
> URL: https://issues.apache.org/jira/browse/HADOOP-8689
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hadoop-8689.txt
>
>
> Per ATM's suggestion in HADOOP-8598 for v2 let's make {{fs.trash.interval}} configured server side. If it is not configured server side then the client side configuration is used. The {{fs.trash.checkpoint.interval}} option is already server side as the emptier runs in the NameNode. Clients may manually run an emptier via hadoop org.apache.hadoop.fs.Trash but it's OK if it uses a separate interval.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8689) Make trash a server side
configuration option
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13435873#comment-13435873 ]
Hadoop QA commented on HADOOP-8689:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12541195/hadoop-8689.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 2 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.ha.TestZKFailoverController
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1313//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1313//console
This message is automatically generated.
> Make trash a server side configuration option
> ---------------------------------------------
>
> Key: HADOOP-8689
> URL: https://issues.apache.org/jira/browse/HADOOP-8689
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hadoop-8689.txt
>
>
> Per ATM's suggestion in HADOOP-8598 for v2 let's make {{fs.trash.interval}} configured server side rather than client side. The {{fs.trash.checkpoint.interval}} option is already server side as the emptier runs in the NameNode.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8689) Make trash a server side
configuration option
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HADOOP-8689:
--------------------------------
Description: Per ATM's suggestion in HADOOP-8598 for v2 let's make {{fs.trash.interval}} configured server side. If it is not configured server side then the client side configuration is used. The {{fs.trash.checkpoint.interval}} option is already server side as the emptier runs in the NameNode. Clients may manually run an emptier via hadoop org.apache.hadoop.fs.Trash but it's OK if it uses a separate interval. (was: Per ATM's suggestion in HADOOP-8598 for v2 let's make {{fs.trash.interval}} configured server side rather than client side. The {{fs.trash.checkpoint.interval}} option is already server side as the emptier runs in the NameNode.)
> Make trash a server side configuration option
> ---------------------------------------------
>
> Key: HADOOP-8689
> URL: https://issues.apache.org/jira/browse/HADOOP-8689
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hadoop-8689.txt
>
>
> Per ATM's suggestion in HADOOP-8598 for v2 let's make {{fs.trash.interval}} configured server side. If it is not configured server side then the client side configuration is used. The {{fs.trash.checkpoint.interval}} option is already server side as the emptier runs in the NameNode. Clients may manually run an emptier via hadoop org.apache.hadoop.fs.Trash but it's OK if it uses a separate interval.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8689) Make trash a server side
configuration option
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HADOOP-8689:
--------------------------------
Resolution: Fixed
Fix Version/s: 2.2.0-alpha
Target Version/s: (was: 2.2.0-alpha)
Hadoop Flags: Incompatible change,Reviewed (was: Incompatible change)
Status: Resolved (was: Patch Available)
Thanks for the review ATM. I've committed this and merged to branch-2.
> Make trash a server side configuration option
> ---------------------------------------------
>
> Key: HADOOP-8689
> URL: https://issues.apache.org/jira/browse/HADOOP-8689
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Fix For: 2.2.0-alpha
>
> Attachments: hadoop-8689.txt, hadoop-8689.txt
>
>
> Per ATM's suggestion in HADOOP-8598 for v2 let's make {{fs.trash.interval}} configured server side. If it is not configured server side then the client side configuration is used. The {{fs.trash.checkpoint.interval}} option is already server side as the emptier runs in the NameNode. Clients may manually run an emptier via hadoop org.apache.hadoop.fs.Trash but it's OK if it uses a separate interval.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HADOOP-8689) Make trash a server side
configuration option
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy closed HADOOP-8689.
---------------------------------
> Make trash a server side configuration option
> ---------------------------------------------
>
> Key: HADOOP-8689
> URL: https://issues.apache.org/jira/browse/HADOOP-8689
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Fix For: 2.0.2-alpha
>
> Attachments: hadoop-8689.txt, hadoop-8689.txt
>
>
> Per ATM's suggestion in HADOOP-8598 for v2 let's make {{fs.trash.interval}} configured server side. If it is not configured server side then the client side configuration is used. The {{fs.trash.checkpoint.interval}} option is already server side as the emptier runs in the NameNode. Clients may manually run an emptier via hadoop org.apache.hadoop.fs.Trash but it's OK if it uses a separate interval.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8689) Make trash a server side
configuration option
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HADOOP-8689:
--------------------------------
Release Note: If fs.trash.interval is configured on the server then the client's value for this configuration is ignored.
Hadoop Flags: Incompatible change
> Make trash a server side configuration option
> ---------------------------------------------
>
> Key: HADOOP-8689
> URL: https://issues.apache.org/jira/browse/HADOOP-8689
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hadoop-8689.txt, hadoop-8689.txt
>
>
> Per ATM's suggestion in HADOOP-8598 for v2 let's make {{fs.trash.interval}} configured server side. If it is not configured server side then the client side configuration is used. The {{fs.trash.checkpoint.interval}} option is already server side as the emptier runs in the NameNode. Clients may manually run an emptier via hadoop org.apache.hadoop.fs.Trash but it's OK if it uses a separate interval.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8689) Make trash a server side
configuration option
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437143#comment-13437143 ]
Hudson commented on HADOOP-8689:
--------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #2665 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2665/])
HADOOP-8689. Make trash a server side configuration option. Contributed by Eli Collins (Revision 1374472)
Result = SUCCESS
eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1374472
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FsServerDefaults.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/TrashPolicyDefault.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ftp/FtpConfigKeys.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/local/LocalConfigKeys.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestTrash.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/proto/hdfs.proto
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestHDFSTrash.java
> Make trash a server side configuration option
> ---------------------------------------------
>
> Key: HADOOP-8689
> URL: https://issues.apache.org/jira/browse/HADOOP-8689
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Fix For: 2.2.0-alpha
>
> Attachments: hadoop-8689.txt, hadoop-8689.txt
>
>
> Per ATM's suggestion in HADOOP-8598 for v2 let's make {{fs.trash.interval}} configured server side. If it is not configured server side then the client side configuration is used. The {{fs.trash.checkpoint.interval}} option is already server side as the emptier runs in the NameNode. Clients may manually run an emptier via hadoop org.apache.hadoop.fs.Trash but it's OK if it uses a separate interval.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8689) Make trash a server side
configuration option
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13436989#comment-13436989 ]
Eli Collins commented on HADOOP-8689:
-------------------------------------
I tested this on a pseudo cluster run from a tarball, basically running the same tests in the unit test by hand on a real cluster for sanity.
> Make trash a server side configuration option
> ---------------------------------------------
>
> Key: HADOOP-8689
> URL: https://issues.apache.org/jira/browse/HADOOP-8689
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hadoop-8689.txt, hadoop-8689.txt
>
>
> Per ATM's suggestion in HADOOP-8598 for v2 let's make {{fs.trash.interval}} configured server side. If it is not configured server side then the client side configuration is used. The {{fs.trash.checkpoint.interval}} option is already server side as the emptier runs in the NameNode. Clients may manually run an emptier via hadoop org.apache.hadoop.fs.Trash but it's OK if it uses a separate interval.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8689) Make trash a server side
configuration option
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437034#comment-13437034 ]
Aaron T. Myers commented on HADOOP-8689:
----------------------------------------
bq. #1 I moved the initialization of the checkpoint interval to trash policy creation (where the deletion interval is set) rather than when an emptier object is created so the emptier uses the interval configured by the trash policy that creates the emptier. I think this is more clear, and in practice doesn't change things since the just NN creates an emptier once.
Sure, that makes sense to me.
The latest patch looks good to me. +1 pending Jenkins.
> Make trash a server side configuration option
> ---------------------------------------------
>
> Key: HADOOP-8689
> URL: https://issues.apache.org/jira/browse/HADOOP-8689
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hadoop-8689.txt, hadoop-8689.txt
>
>
> Per ATM's suggestion in HADOOP-8598 for v2 let's make {{fs.trash.interval}} configured server side. If it is not configured server side then the client side configuration is used. The {{fs.trash.checkpoint.interval}} option is already server side as the emptier runs in the NameNode. Clients may manually run an emptier via hadoop org.apache.hadoop.fs.Trash but it's OK if it uses a separate interval.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8689) Make trash a server side
configuration option
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437069#comment-13437069 ]
Hadoop QA commented on HADOOP-8689:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12541409/hadoop-8689.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 2 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1325//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1325//console
This message is automatically generated.
> Make trash a server side configuration option
> ---------------------------------------------
>
> Key: HADOOP-8689
> URL: https://issues.apache.org/jira/browse/HADOOP-8689
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hadoop-8689.txt, hadoop-8689.txt
>
>
> Per ATM's suggestion in HADOOP-8598 for v2 let's make {{fs.trash.interval}} configured server side. If it is not configured server side then the client side configuration is used. The {{fs.trash.checkpoint.interval}} option is already server side as the emptier runs in the NameNode. Clients may manually run an emptier via hadoop org.apache.hadoop.fs.Trash but it's OK if it uses a separate interval.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira