You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Jing Zhao (JIRA)" <ji...@apache.org> on 2012/11/11 01:01:12 UTC
[jira] [Created] (HDFS-4175) Extend existing TestSnapshot to
support more complicated directory structure and more modifications on
current files/direcotories
Jing Zhao created HDFS-4175:
-------------------------------
Summary: Extend existing TestSnapshot to support more complicated directory structure and more modifications on current files/direcotories
Key: HDFS-4175
URL: https://issues.apache.org/jira/browse/HDFS-4175
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Jing Zhao
Assignee: Jing Zhao
Currently the TestSnapshot only uses a static simple directory structure for testing snapshot functionalities. We need to test snapshot under more complicated directory structures.
Also the current TestSnapshot only compares snapshots before/after three types of modifications: file creation, deletion, and appending. We need to test snapshots for more file modifications, including file permission change, replication factor change, owner/group change, and sub-directory creation/deletion.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4175) Extend existing TestSnapshot to
support more complicated directory structure and more modifications on
current files/direcotories
Posted by "Jing Zhao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jing Zhao updated HDFS-4175:
----------------------------
Attachment: snapshot-test.001.patch
Initial patch.
> Extend existing TestSnapshot to support more complicated directory structure and more modifications on current files/direcotories
> ---------------------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-4175
> URL: https://issues.apache.org/jira/browse/HDFS-4175
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: data-node, name-node
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Attachments: snapshot-test.001.patch
>
>
> Currently the TestSnapshot only uses a static simple directory structure for testing snapshot functionalities. We need to test snapshot under more complicated directory structures.
> Also the current TestSnapshot only compares snapshots before/after three types of modifications: file creation, deletion, and appending. We need to test snapshots for more file modifications, including file permission change, replication factor change, owner/group change, and sub-directory creation/deletion.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4175) Extend existing TestSnapshot to
support more complicated directory structure and more modifications on
current files/direcotories
Posted by "Jing Zhao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jing Zhao updated HDFS-4175:
----------------------------
Attachment: HDFS-4175.003.patch
Suresh and Colin, thanks for the comments! New patch uploaded to address the comments.
> Extend existing TestSnapshot to support more complicated directory structure and more modifications on current files/direcotories
> ---------------------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-4175
> URL: https://issues.apache.org/jira/browse/HDFS-4175
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: data-node, name-node
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Attachments: HDFS-4175.003.patch, HDFS-4175-snapshot-test.002.patch, snapshot-test.001.patch
>
>
> Currently the TestSnapshot only uses a static simple directory structure for testing snapshot functionalities. We need to test snapshot under more complicated directory structures.
> Also the current TestSnapshot only compares snapshots before/after three types of modifications: file creation, deletion, and appending. We need to test snapshots for more file modifications, including file permission change, replication factor change, owner/group change, and sub-directory creation/deletion.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4175) Extend existing TestSnapshot to
support more complicated directory structure and more modifications on
current files/direcotories
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13499982#comment-13499982 ]
Suresh Srinivas commented on HDFS-4175:
---------------------------------------
This is one complicated test to review. Took me a while. Some comments:
# The TestDirectoryTree and TestDirectoryNode need lot more javadocs to make it easy to understand.
# Move TestDirectoryNode as an inner class of TestDirectoryTree.
# It is better to rename TestDirectoryTree#numOfLevel as height?
# The change in SnapshotTestHelper.java is no longer needed it looks like.
# TestSnapshotDeletion has test for snapshottable directories. Should it be moved to TestSnapshot?
> Extend existing TestSnapshot to support more complicated directory structure and more modifications on current files/direcotories
> ---------------------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-4175
> URL: https://issues.apache.org/jira/browse/HDFS-4175
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: data-node, name-node
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Attachments: HDFS-4175-snapshot-test.002.patch, snapshot-test.001.patch
>
>
> Currently the TestSnapshot only uses a static simple directory structure for testing snapshot functionalities. We need to test snapshot under more complicated directory structures.
> Also the current TestSnapshot only compares snapshots before/after three types of modifications: file creation, deletion, and appending. We need to test snapshots for more file modifications, including file permission change, replication factor change, owner/group change, and sub-directory creation/deletion.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4175) Extend existing TestSnapshot to
support more complicated directory structure and more modifications on
current files/direcotories
Posted by "Jing Zhao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jing Zhao updated HDFS-4175:
----------------------------
Attachment: HDFS-4175-snapshot-test.002.patch
Update the patch with javadoc.
> Extend existing TestSnapshot to support more complicated directory structure and more modifications on current files/direcotories
> ---------------------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-4175
> URL: https://issues.apache.org/jira/browse/HDFS-4175
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: data-node, name-node
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Attachments: HDFS-4175-snapshot-test.002.patch, snapshot-test.001.patch
>
>
> Currently the TestSnapshot only uses a static simple directory structure for testing snapshot functionalities. We need to test snapshot under more complicated directory structures.
> Also the current TestSnapshot only compares snapshots before/after three types of modifications: file creation, deletion, and appending. We need to test snapshots for more file modifications, including file permission change, replication factor change, owner/group change, and sub-directory creation/deletion.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4175) Extend existing TestSnapshot to
support more complicated directory structure and more modifications on
current files/direcotories
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13499158#comment-13499158 ]
Colin Patrick McCabe commented on HDFS-4175:
--------------------------------------------
{code}
> /**
> * The number of iterating times snapshots are created
> */
> public static final int SNAPSHOT_ITERATION_NUMBER = 20;
{code}
Does it make more sense to put this final int inside the testSnapshot function? It's not
used anywhere else.
> Extend existing TestSnapshot to support more complicated directory structure and more modifications on current files/direcotories
> ---------------------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-4175
> URL: https://issues.apache.org/jira/browse/HDFS-4175
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: data-node, name-node
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Attachments: HDFS-4175-snapshot-test.002.patch, snapshot-test.001.patch
>
>
> Currently the TestSnapshot only uses a static simple directory structure for testing snapshot functionalities. We need to test snapshot under more complicated directory structures.
> Also the current TestSnapshot only compares snapshots before/after three types of modifications: file creation, deletion, and appending. We need to test snapshots for more file modifications, including file permission change, replication factor change, owner/group change, and sub-directory creation/deletion.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4175) Extend existing TestSnapshot to
support more complicated directory structure and more modifications on
current files/direcotories
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13499150#comment-13499150 ]
Colin Patrick McCabe commented on HDFS-4175:
--------------------------------------------
Hi Jing,
This one doesn't apply for me with the latest from HDFS-2802. Can you rebase the patch on the latest code?
* I think it makes more sense to rename {{DirCreationDeletion}} to {{DirCreationOrDeletion}}, since it's one or the other (but not both).
* In {{testSnapshottableDirectory}}, {{testDeleteDirectoryWithSnapshot}}, {{testDeleteDirectoryWithSnapshot2}}, etc., it seems like you check for {{RemoteException}} in every case. This seems awfully generic. Any remote exception could cause the test to succeed here. Can you check for the exception that is wrapped in the {{RemoteException}} instead? Or in the edit log code we sometimes do something like e.getMessage().contains(EXPECTED_STRING), to make sure that the failure we're observing is really the one we expect to get in the test, and not something else.
> Extend existing TestSnapshot to support more complicated directory structure and more modifications on current files/direcotories
> ---------------------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-4175
> URL: https://issues.apache.org/jira/browse/HDFS-4175
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: data-node, name-node
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Attachments: HDFS-4175-snapshot-test.002.patch, snapshot-test.001.patch
>
>
> Currently the TestSnapshot only uses a static simple directory structure for testing snapshot functionalities. We need to test snapshot under more complicated directory structures.
> Also the current TestSnapshot only compares snapshots before/after three types of modifications: file creation, deletion, and appending. We need to test snapshots for more file modifications, including file permission change, replication factor change, owner/group change, and sub-directory creation/deletion.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4175) Additional snapshot tests for more
complicated directory structure and modifications.
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501955#comment-13501955 ]
Hudson commented on HDFS-4175:
------------------------------
Integrated in Hadoop-Hdfs-Snapshots-Branch-build #18 (See [https://builds.apache.org/job/Hadoop-Hdfs-Snapshots-Branch-build/18/])
HDFS-4175. Add the jira to CHANGES.HDFS-2802.txt. (Revision 1411690)
Result = FAILURE
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1411690
Files :
* /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/CHANGES.HDFS-2802.txt
> Additional snapshot tests for more complicated directory structure and modifications.
> -------------------------------------------------------------------------------------
>
> Key: HDFS-4175
> URL: https://issues.apache.org/jira/browse/HDFS-4175
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: data-node, name-node
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Fix For: Snapshot (HDFS-2802)
>
> Attachments: HDFS-4175.003.patch, HDFS-4175-snapshot-test.002.patch, snapshot-test.001.patch
>
>
> Currently the TestSnapshot only uses a static simple directory structure for testing snapshot functionalities. We need to test snapshot under more complicated directory structures.
> Also the current TestSnapshot only compares snapshots before/after three types of modifications: file creation, deletion, and appending. We need to test snapshots for more file modifications, including file permission change, replication factor change, owner/group change, and sub-directory creation/deletion.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4175) Additional snapshot tests for more
complicated directory structure and modifications.
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501184#comment-13501184 ]
Hudson commented on HDFS-4175:
------------------------------
Integrated in Hadoop-Hdfs-Snapshots-Branch-build #17 (See [https://builds.apache.org/job/Hadoop-Hdfs-Snapshots-Branch-build/17/])
HDFS-4175. Additional snapshot tests for more complicated directory structure and modifications. Contributed by Jing Zhao. (Revision 1411503)
Result = FAILURE
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1411503
Files :
* /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/SnapshotTestHelper.java
* /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshot.java
* /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotDeletion.java
> Additional snapshot tests for more complicated directory structure and modifications.
> -------------------------------------------------------------------------------------
>
> Key: HDFS-4175
> URL: https://issues.apache.org/jira/browse/HDFS-4175
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: data-node, name-node
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Fix For: Snapshot (HDFS-2802)
>
> Attachments: HDFS-4175.003.patch, HDFS-4175-snapshot-test.002.patch, snapshot-test.001.patch
>
>
> Currently the TestSnapshot only uses a static simple directory structure for testing snapshot functionalities. We need to test snapshot under more complicated directory structures.
> Also the current TestSnapshot only compares snapshots before/after three types of modifications: file creation, deletion, and appending. We need to test snapshots for more file modifications, including file permission change, replication factor change, owner/group change, and sub-directory creation/deletion.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira