You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Jing Zhao (JIRA)" <ji...@apache.org> on 2012/09/12 01:28:07 UTC
[jira] [Created] (HDFS-3923) libwebhdfs testing code cleanup
Jing Zhao created HDFS-3923:
-------------------------------
Summary: libwebhdfs testing code cleanup
Key: HDFS-3923
URL: https://issues.apache.org/jira/browse/HDFS-3923
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Jing Zhao
Assignee: Jing Zhao
1. Testing code cleanup for libwebhdfs
1.1 Tests should generate a test-specific filename and should use TMPDIR appropriately.
2. Enabling automate testing
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3923) libwebhdfs testing code cleanup
Posted by "Andy Isaacson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487223#comment-13487223 ]
Andy Isaacson commented on HDFS-3923:
-------------------------------------
The new .002.patch seems to be the same file as the old .001.patch.
> libwebhdfs testing code cleanup
> -------------------------------
>
> Key: HDFS-3923
> URL: https://issues.apache.org/jira/browse/HDFS-3923
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Attachments: HDFS-3923.001.patch, HDFS-3923.002.patch
>
>
> 1. Testing code cleanup for libwebhdfs
> 1.1 Tests should generate a test-specific filename and should use TMPDIR appropriately.
> 2. Enabling automate testing
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3923) libwebhdfs testing code cleanup
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464194#comment-13464194 ]
Colin Patrick McCabe commented on HDFS-3923:
--------------------------------------------
{{nmdGetNameNodeHttpAddress}} looks good to me. I'd really like to see automated testing for this, good work Jing.
> libwebhdfs testing code cleanup
> -------------------------------
>
> Key: HDFS-3923
> URL: https://issues.apache.org/jira/browse/HDFS-3923
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Attachments: HDFS-3923.001.patch
>
>
> 1. Testing code cleanup for libwebhdfs
> 1.1 Tests should generate a test-specific filename and should use TMPDIR appropriately.
> 2. Enabling automate testing
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3923) libwebhdfs testing code cleanup
Posted by "Andy Isaacson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487227#comment-13487227 ]
Andy Isaacson commented on HDFS-3923:
-------------------------------------
Sorry, user error on my part -- the files are different, I managed to mistakenly wget the same file twice.
> libwebhdfs testing code cleanup
> -------------------------------
>
> Key: HDFS-3923
> URL: https://issues.apache.org/jira/browse/HDFS-3923
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Attachments: HDFS-3923.001.patch, HDFS-3923.002.patch
>
>
> 1. Testing code cleanup for libwebhdfs
> 1.1 Tests should generate a test-specific filename and should use TMPDIR appropriately.
> 2. Enabling automate testing
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3923) libwebhdfs testing code cleanup
Posted by "Jing Zhao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jing Zhao updated HDFS-3923:
----------------------------
Attachment: HDFS-3923.002.patch
Updated the patch.
> libwebhdfs testing code cleanup
> -------------------------------
>
> Key: HDFS-3923
> URL: https://issues.apache.org/jira/browse/HDFS-3923
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Attachments: HDFS-3923.001.patch, HDFS-3923.002.patch
>
>
> 1. Testing code cleanup for libwebhdfs
> 1.1 Tests should generate a test-specific filename and should use TMPDIR appropriately.
> 2. Enabling automate testing
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3923) libwebhdfs testing code cleanup
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13453602#comment-13453602 ]
Colin Patrick McCabe commented on HDFS-3923:
--------------------------------------------
Yeah, it will definitely be nice to have automated tests for libwebhdfs. We can extend {{native_mini_dfs}} by adding a function like this:
{code}
int nmdGetNameNodeWebHdfsPort(const struct NativeMiniDfsCluster *cl);
{code}
Then we could create a NativeMiniDfsCluster at the beginning of the libwebhdfs tests, and use that cluster to test against. This way Jenkins can run the unit tests automatically to spot any regressions.
For example, {{test_libhdfs_threaded}} uses this strategy. It should be pretty easy to re-use the {{native_mini_dfs}} library-- just add it to {{target_link_libraries}} for the test.
> libwebhdfs testing code cleanup
> -------------------------------
>
> Key: HDFS-3923
> URL: https://issues.apache.org/jira/browse/HDFS-3923
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Jing Zhao
> Assignee: Jing Zhao
>
> 1. Testing code cleanup for libwebhdfs
> 1.1 Tests should generate a test-specific filename and should use TMPDIR appropriately.
> 2. Enabling automate testing
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3923) libwebhdfs testing code cleanup
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487358#comment-13487358 ]
Suresh Srinivas commented on HDFS-3923:
---------------------------------------
+1 for the patch. I committed it to trunk.
> libwebhdfs testing code cleanup
> -------------------------------
>
> Key: HDFS-3923
> URL: https://issues.apache.org/jira/browse/HDFS-3923
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Attachments: HDFS-3923.001.patch, HDFS-3923.002.patch
>
>
> 1. Testing code cleanup for libwebhdfs
> 1.1 Tests should generate a test-specific filename and should use TMPDIR appropriately.
> 2. Enabling automate testing
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3923) libwebhdfs testing code cleanup
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486065#comment-13486065 ]
Suresh Srinivas commented on HDFS-3923:
---------------------------------------
This patch does not apply to current trunk. It needs to be rebased on latest trunk.
> libwebhdfs testing code cleanup
> -------------------------------
>
> Key: HDFS-3923
> URL: https://issues.apache.org/jira/browse/HDFS-3923
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Attachments: HDFS-3923.001.patch
>
>
> 1. Testing code cleanup for libwebhdfs
> 1.1 Tests should generate a test-specific filename and should use TMPDIR appropriately.
> 2. Enabling automate testing
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3923) libwebhdfs testing code cleanup
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13485841#comment-13485841 ]
Suresh Srinivas commented on HDFS-3923:
---------------------------------------
bq. I'd really like to see automated testing for this, good work Jing.
Colin, I want to make sure I understand your comment correctly. Do you still want more tests or are you happy with what the patch currently has? I take it as the latter - is that right?
> libwebhdfs testing code cleanup
> -------------------------------
>
> Key: HDFS-3923
> URL: https://issues.apache.org/jira/browse/HDFS-3923
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Attachments: HDFS-3923.001.patch
>
>
> 1. Testing code cleanup for libwebhdfs
> 1.1 Tests should generate a test-specific filename and should use TMPDIR appropriately.
> 2. Enabling automate testing
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3923) libwebhdfs testing code cleanup
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486062#comment-13486062 ]
Suresh Srinivas commented on HDFS-3923:
---------------------------------------
Colin, thanks for the clarification.
> libwebhdfs testing code cleanup
> -------------------------------
>
> Key: HDFS-3923
> URL: https://issues.apache.org/jira/browse/HDFS-3923
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Attachments: HDFS-3923.001.patch
>
>
> 1. Testing code cleanup for libwebhdfs
> 1.1 Tests should generate a test-specific filename and should use TMPDIR appropriately.
> 2. Enabling automate testing
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3923) libwebhdfs testing code cleanup
Posted by "Andy Isaacson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487231#comment-13487231 ]
Andy Isaacson commented on HDFS-3923:
-------------------------------------
The .002.patch looks good to me. In the spirit of continuous improvement let's get this checked in ASAP so we can continue to make progress on getting libwebhdfs to a reliable state.
> libwebhdfs testing code cleanup
> -------------------------------
>
> Key: HDFS-3923
> URL: https://issues.apache.org/jira/browse/HDFS-3923
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Attachments: HDFS-3923.001.patch, HDFS-3923.002.patch
>
>
> 1. Testing code cleanup for libwebhdfs
> 1.1 Tests should generate a test-specific filename and should use TMPDIR appropriately.
> 2. Enabling automate testing
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HDFS-3923) libwebhdfs testing code cleanup
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suresh Srinivas resolved HDFS-3923.
-----------------------------------
Resolution: Fixed
Fix Version/s: 2.0.3-alpha
3.0.0
Hadoop Flags: Reviewed
I committed the patch to branch-2 as well.
Thank you Jing for the patch. Thank you Colin and Andy for the review.
> libwebhdfs testing code cleanup
> -------------------------------
>
> Key: HDFS-3923
> URL: https://issues.apache.org/jira/browse/HDFS-3923
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Fix For: 3.0.0, 2.0.3-alpha
>
> Attachments: HDFS-3923.001.patch, HDFS-3923.002.patch
>
>
> 1. Testing code cleanup for libwebhdfs
> 1.1 Tests should generate a test-specific filename and should use TMPDIR appropriately.
> 2. Enabling automate testing
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3923) libwebhdfs testing code cleanup
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486059#comment-13486059 ]
Colin Patrick McCabe commented on HDFS-3923:
--------------------------------------------
Sorry if my comment was unclear. I think this patch is a good stepping stone on the way to having a libwebhdfs unit test that runs as part of Jenkins (similar to how we do libhdfs). I don't have any further suggestions on this one.
> libwebhdfs testing code cleanup
> -------------------------------
>
> Key: HDFS-3923
> URL: https://issues.apache.org/jira/browse/HDFS-3923
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Attachments: HDFS-3923.001.patch
>
>
> 1. Testing code cleanup for libwebhdfs
> 1.1 Tests should generate a test-specific filename and should use TMPDIR appropriately.
> 2. Enabling automate testing
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3923) libwebhdfs testing code cleanup
Posted by "Jing Zhao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jing Zhao updated HDFS-3923:
----------------------------
Attachment: HDFS-3923.001.patch
Initial patch that cleans the test code: test_libwebhdfs_ops.c, test_libwebhdfs_read.c, test_libwebhdfs_write.c, and test_libwebhdfs_threaded.c.
To automate the test, currently I reused the NativeMiniDfsCluster (with some modification), and now the test cases in test_libwebhdfs_ops.c and test_libwebhdfs_threaded.c are automated.
> libwebhdfs testing code cleanup
> -------------------------------
>
> Key: HDFS-3923
> URL: https://issues.apache.org/jira/browse/HDFS-3923
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Attachments: HDFS-3923.001.patch
>
>
> 1. Testing code cleanup for libwebhdfs
> 1.1 Tests should generate a test-specific filename and should use TMPDIR appropriately.
> 2. Enabling automate testing
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira