You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Tanping Wang (JIRA)" <ji...@apache.org> on 2010/09/23 01:15:33 UTC
[jira] Created: (HDFS-1414) HDFS federation : fix unit test cases
HDFS federation : fix unit test cases
-------------------------------------
Key: HDFS-1414
URL: https://issues.apache.org/jira/browse/HDFS-1414
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Tanping Wang
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1414) HDFS federation : fix unit test cases
Posted by "Tanping Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tanping Wang updated HDFS-1414:
-------------------------------
Attachment: hadoop-14-dfs-dir.tgz
== fix TestDFSUpgradeFromImage
This test case generates its testing data by untar hadoop-14-dfs-dir.tgz. There are ten VERSION files under each storage directories provided by the tar file and they are,
./name2/current/VERSION
./data/data3/current/VERSION
./data/data4/current/VERSION
./data/data8/current/VERSION
./data/data7/current/VERSION
./data/data6/current/VERSION
./data/data2/current/VERSION
./data/data5/current/VERSION
./data/data1/current/VERSION
./name1/current/VERSION
As we introduced clusterID and blockpoolID, we need to add two new entries,
clusterID=test-cid
blockpoolID=test-bpid
into the 10 above listed VERSION files. The tgz file is re-generated accordingly and uploaded. It needs to be checked into
src/test/hdfs/org/apache/hadoop/hdfs/hadoop-14-dfs-dir.tgz
to replace the currently one.
> HDFS federation : fix unit test cases
> -------------------------------------
>
> Key: HDFS-1414
> URL: https://issues.apache.org/jira/browse/HDFS-1414
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Tanping Wang
> Attachments: hadoop-14-dfs-dir.tgz
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1414) HDFS federation : fix unit test cases
Posted by "Tanping Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tanping Wang updated HDFS-1414:
-------------------------------
Attachment: (was: hadoop-14-dfs-dir.tgz)
> HDFS federation : fix unit test cases
> -------------------------------------
>
> Key: HDFS-1414
> URL: https://issues.apache.org/jira/browse/HDFS-1414
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Tanping Wang
> Attachments: HDFS1414-branchHDFS1052.1.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1414) HDFS federation : fix unit test cases
Posted by "Tanping Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12915087#action_12915087 ]
Tanping Wang commented on HDFS-1414:
------------------------------------
I removed the hadoop-14-dfs-dir.tgz that I generated with added VERSION files because after adding clusterID and blockpoolID. We need to generate new FSImages that is used in TestUpgradeFromImage. However as I commented in HADOOP-2797. Since there is test program to generate these FSImages, I need to write one that contains the various categories
* zero length files
* file with replication set higher than number of datanodes
* file with no .crc file
* file with corrupt .crc file
* file with multiple blocks (will need to set dfs.block.size to a small value)
* file with multiple checksum blocks
* empty directory
* all of the above again but with a different io.bytes.per.checksum setting
Except for the very initiative ones such as zero length files or empty directory, I need to spend some time to find out how to make
* file with replication set higher than number of datanodes
* file with no .crc file
* file with corrupt .crc file
* file with multiple blocks (will need to set dfs.block.size to a small value)
* file with multiple checksum blocks
Any how, I think the test should be easier to maintain than how it is now......
> HDFS federation : fix unit test cases
> -------------------------------------
>
> Key: HDFS-1414
> URL: https://issues.apache.org/jira/browse/HDFS-1414
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Tanping Wang
> Attachments: HDFS1414-branchHDFS1052.1.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1414) HDFS federation : fix unit test cases
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12915422#action_12915422 ]
Raghu Angadi commented on HDFS-1414:
------------------------------------
If the existing upgrade test fails, that implies HDFS can not be upgraded from that version. If the upgrade from hadoop-0.14 is supported than we should not need to modify.
First we need decide what the latest version that this HDFS can upgrade from. Then we should create an image with that version (say it is hadoop-0.21)
> HDFS federation : fix unit test cases
> -------------------------------------
>
> Key: HDFS-1414
> URL: https://issues.apache.org/jira/browse/HDFS-1414
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Tanping Wang
> Attachments: HDFS1414-branchHDFS1052.1.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1414) HDFS federation : fix unit test cases
Posted by "Tanping Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12913837#action_12913837 ]
Tanping Wang commented on HDFS-1414:
------------------------------------
This is JIRA is used to track the fixes of unit test cases.
> HDFS federation : fix unit test cases
> -------------------------------------
>
> Key: HDFS-1414
> URL: https://issues.apache.org/jira/browse/HDFS-1414
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Tanping Wang
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1414) HDFS federation : fix unit test cases
Posted by "Tanping Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tanping Wang updated HDFS-1414:
-------------------------------
Attachment: HDFS1414-branchHDFS1052.1.patch
HDFS1414-branchHDFS1052.1.patch
fixes
Teststartup
TestCheckpoint
Add reading clusterID and blockpoolID into loadFSImage
Add writing clusterID and blockpoolID into writeFSImage.
> HDFS federation : fix unit test cases
> -------------------------------------
>
> Key: HDFS-1414
> URL: https://issues.apache.org/jira/browse/HDFS-1414
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Tanping Wang
> Attachments: HDFS1414-branchHDFS1052.1.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.