You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Tom White (JIRA)" <ji...@apache.org> on 2011/03/17 01:04:29 UTC
[jira] Created: (HDFS-1762) Allow TestHDFSCLI to be run against a
cluster
Allow TestHDFSCLI to be run against a cluster
---------------------------------------------
Key: HDFS-1762
URL: https://issues.apache.org/jira/browse/HDFS-1762
Project: Hadoop HDFS
Issue Type: Test
Reporter: Tom White
Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HDFS-1762) Allow TestHDFSCLI to be run against a
cluster
Posted by "Konstantin Boudnik (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik resolved HDFS-1762.
--------------------------------------
Resolution: Fixed
Fix Version/s: 0.22.0
Hadoop Flags: Reviewed
I have committed it.
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
> Key: HDFS-1762
> URL: https://issues.apache.org/jira/browse/HDFS-1762
> Project: Hadoop HDFS
> Issue Type: Test
> Components: build, test
> Affects Versions: 0.22.0
> Reporter: Tom White
> Assignee: Konstantin Boudnik
> Fix For: 0.22.0
>
> Attachments: HDFS-1762-20.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against
a cluster
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016498#comment-13016498 ]
Konstantin Boudnik commented on HDFS-1762:
------------------------------------------
Todd, I was talking about {{testHDFSConf.xml}} file which actually defined test cases executed by TestCLI class. The risk of 'overriding' in this case is pretty much close to 0. I wouldn't dare to suggest to add any hadoop related config files into test artifact.
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
> Key: HDFS-1762
> URL: https://issues.apache.org/jira/browse/HDFS-1762
> Project: Hadoop HDFS
> Issue Type: Test
> Reporter: Tom White
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1762) Allow TestHDFSCLI to be run against a
cluster
Posted by "Konstantin Boudnik (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik updated HDFS-1762:
-------------------------------------
Attachment: (was: HDFS-1762.common.patch)
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
> Key: HDFS-1762
> URL: https://issues.apache.org/jira/browse/HDFS-1762
> Project: Hadoop HDFS
> Issue Type: Test
> Components: build, test
> Affects Versions: 0.22.0
> Reporter: Tom White
> Assignee: Konstantin Boudnik
> Attachments: HDFS-1762-20.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against
a cluster
Posted by "Konstantin Shvachko (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13124307#comment-13124307 ]
Konstantin Shvachko commented on HDFS-1762:
-------------------------------------------
The issues I mentioned before are intended to support host names with "-" (etc.) in them. Some tests (like du:) will fail on my host as it has a dash in it.
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
> Key: HDFS-1762
> URL: https://issues.apache.org/jira/browse/HDFS-1762
> Project: Hadoop HDFS
> Issue Type: Test
> Components: build, test
> Affects Versions: 0.22.0
> Reporter: Tom White
> Assignee: Konstantin Boudnik
> Attachments: HDFS-1762-20.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1762) Allow TestHDFSCLI to be run against a
cluster
Posted by "Konstantin Boudnik (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik updated HDFS-1762:
-------------------------------------
Affects Version/s: 0.22.0
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
> Key: HDFS-1762
> URL: https://issues.apache.org/jira/browse/HDFS-1762
> Project: Hadoop HDFS
> Issue Type: Test
> Affects Versions: 0.22.0
> Reporter: Tom White
> Assignee: Konstantin Boudnik
> Attachments: HDFS-1762-20.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-1762) Allow TestHDFSCLI to be run against a
cluster
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik reassigned HDFS-1762:
----------------------------------------
Assignee: Konstantin Boudnik
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
> Key: HDFS-1762
> URL: https://issues.apache.org/jira/browse/HDFS-1762
> Project: Hadoop HDFS
> Issue Type: Test
> Reporter: Tom White
> Assignee: Konstantin Boudnik
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1762) Allow TestHDFSCLI to be run against a
cluster
Posted by "Konstantin Boudnik (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik updated HDFS-1762:
-------------------------------------
Attachment: HDFS-1762.hdfs.patch
Addressing issue for hostnames with '-'
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
> Key: HDFS-1762
> URL: https://issues.apache.org/jira/browse/HDFS-1762
> Project: Hadoop HDFS
> Issue Type: Test
> Components: build, test
> Affects Versions: 0.22.0
> Reporter: Tom White
> Assignee: Konstantin Boudnik
> Attachments: HDFS-1762-20.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against
a cluster
Posted by "Konstantin Boudnik (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123627#comment-13123627 ]
Konstantin Boudnik commented on HDFS-1762:
------------------------------------------
For complete picture look at the patches associated with HADOOP-7730 and MAPREDUCE-3156
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
> Key: HDFS-1762
> URL: https://issues.apache.org/jira/browse/HDFS-1762
> Project: Hadoop HDFS
> Issue Type: Test
> Components: build, test
> Affects Versions: 0.22.0
> Reporter: Tom White
> Assignee: Konstantin Boudnik
> Attachments: HDFS-1762-20.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1762) Allow TestHDFSCLI to be run against a
cluster
Posted by "Konstantin Boudnik (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik updated HDFS-1762:
-------------------------------------
Attachment: HDFS-1762.hdfs.patch
Correcting some regexps to reflect changes in the output formats.
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
> Key: HDFS-1762
> URL: https://issues.apache.org/jira/browse/HDFS-1762
> Project: Hadoop HDFS
> Issue Type: Test
> Components: build, test
> Affects Versions: 0.22.0
> Reporter: Tom White
> Assignee: Konstantin Boudnik
> Attachments: HDFS-1762-20.patch, HDFS-1762.common.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.mapreduce.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against
a cluster
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13012228#comment-13012228 ]
Konstantin Boudnik commented on HDFS-1762:
------------------------------------------
Also, it seems to be necessary to include appropriate XML files into hadoop-tests.jar because otherwise TestCLI won't work at all.
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
> Key: HDFS-1762
> URL: https://issues.apache.org/jira/browse/HDFS-1762
> Project: Hadoop HDFS
> Issue Type: Test
> Reporter: Tom White
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1762) Allow TestHDFSCLI to be run against a
cluster
Posted by "Konstantin Boudnik (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik updated HDFS-1762:
-------------------------------------
Component/s: test
build
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
> Key: HDFS-1762
> URL: https://issues.apache.org/jira/browse/HDFS-1762
> Project: Hadoop HDFS
> Issue Type: Test
> Components: build, test
> Affects Versions: 0.22.0
> Reporter: Tom White
> Assignee: Konstantin Boudnik
> Attachments: HDFS-1762-20.patch, HDFS-1762.common.patch, HDFS-1762.hdfs.patch, HDFS-1762.mapreduce.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against
a cluster
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016214#comment-13016214 ]
Todd Lipcon commented on HDFS-1762:
-----------------------------------
I may have misread. In the past, over in HBase land, we had an hbase-site.xml included in our hbase-test.jar. This meant that if someone accidentally ended up with this on their classpath, they picked up weird configurations that we meant to only apply in tests.
Putting an XML file that's specific to TestHDFSCLI in the jar seems fine, though.
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
> Key: HDFS-1762
> URL: https://issues.apache.org/jira/browse/HDFS-1762
> Project: Hadoop HDFS
> Issue Type: Test
> Reporter: Tom White
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1762) Allow TestHDFSCLI to be run against a
cluster
Posted by "Konstantin Boudnik (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik updated HDFS-1762:
-------------------------------------
Attachment: HDFS-1762.hdfs.patch
Fixing comparators for 4 cases. Now everything passes with hdfs-0.22 cluster.
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
> Key: HDFS-1762
> URL: https://issues.apache.org/jira/browse/HDFS-1762
> Project: Hadoop HDFS
> Issue Type: Test
> Components: build, test
> Affects Versions: 0.22.0
> Reporter: Tom White
> Assignee: Konstantin Boudnik
> Attachments: HDFS-1762-20.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against
a cluster
Posted by "Konstantin Boudnik (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120658#comment-13120658 ]
Konstantin Boudnik commented on HDFS-1762:
------------------------------------------
Patches above works for local mini-cluster setup without any problem. I will be validating this with a real cluster before committing them.
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
> Key: HDFS-1762
> URL: https://issues.apache.org/jira/browse/HDFS-1762
> Project: Hadoop HDFS
> Issue Type: Test
> Affects Versions: 0.22.0
> Reporter: Tom White
> Assignee: Konstantin Boudnik
> Attachments: HDFS-1762-20.patch, HDFS-1762.common.patch, HDFS-1762.hdfs.patch, HDFS-1762.mapreduce.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against
a cluster
Posted by "Konstantin Shvachko (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13124308#comment-13124308 ]
Konstantin Shvachko commented on HDFS-1762:
-------------------------------------------
(like du: )
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
> Key: HDFS-1762
> URL: https://issues.apache.org/jira/browse/HDFS-1762
> Project: Hadoop HDFS
> Issue Type: Test
> Components: build, test
> Affects Versions: 0.22.0
> Reporter: Tom White
> Assignee: Konstantin Boudnik
> Attachments: HDFS-1762-20.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against
a cluster
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016545#comment-13016545 ]
Todd Lipcon commented on HDFS-1762:
-----------------------------------
Ah, sorry for the misunderstanding. Including {{testHDFSConf.xml}} seems totally fine to me.
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
> Key: HDFS-1762
> URL: https://issues.apache.org/jira/browse/HDFS-1762
> Project: Hadoop HDFS
> Issue Type: Test
> Reporter: Tom White
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against
a cluster
Posted by "Konstantin Shvachko (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123796#comment-13123796 ]
Konstantin Shvachko commented on HDFS-1762:
-------------------------------------------
The patch looks good. One thing I would recommend to update regular expressions for host names and paths in the spirit of HDFS-2232 and MAPREDUCE-2304.
Also TestConfCluster.xml need ASF license header.
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
> Key: HDFS-1762
> URL: https://issues.apache.org/jira/browse/HDFS-1762
> Project: Hadoop HDFS
> Issue Type: Test
> Components: build, test
> Affects Versions: 0.22.0
> Reporter: Tom White
> Assignee: Konstantin Boudnik
> Attachments: HDFS-1762-20.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1762) Allow TestHDFSCLI to be run against a
cluster
Posted by "Konstantin Boudnik (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik updated HDFS-1762:
-------------------------------------
Attachment: HDFS-1762.patch
Added missing license. Also, I'd propose to address any regexp issues separately. The reason for two files was to avoid too wide (e.g. generic) expressions in the first place.
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
> Key: HDFS-1762
> URL: https://issues.apache.org/jira/browse/HDFS-1762
> Project: Hadoop HDFS
> Issue Type: Test
> Components: build, test
> Affects Versions: 0.22.0
> Reporter: Tom White
> Assignee: Konstantin Boudnik
> Attachments: HDFS-1762-20.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1762) Allow TestHDFSCLI to be run against a
cluster
Posted by "Konstantin Boudnik (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik updated HDFS-1762:
-------------------------------------
Attachment: HDFS-1762.mapreduce.patch
HDFS-1762.hdfs.patch
HDFS-1762.common.patch
These are patches for all three subprojects for 0.22 branch. I will create separate tickets in a bit.
They should be mostly good for 0.23 except the ant build part which needs to be re-worked for maven.
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
> Key: HDFS-1762
> URL: https://issues.apache.org/jira/browse/HDFS-1762
> Project: Hadoop HDFS
> Issue Type: Test
> Affects Versions: 0.22.0
> Reporter: Tom White
> Assignee: Konstantin Boudnik
> Attachments: HDFS-1762-20.patch, HDFS-1762.common.patch, HDFS-1762.hdfs.patch, HDFS-1762.mapreduce.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against
a cluster
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13012253#comment-13012253 ]
Todd Lipcon commented on HDFS-1762:
-----------------------------------
I don't think adding the XML files to the test jar is a good idea. We've had that in the past, and since the test jar often ends up on other people's classpaths, and can override conf directories, etc.
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
> Key: HDFS-1762
> URL: https://issues.apache.org/jira/browse/HDFS-1762
> Project: Hadoop HDFS
> Issue Type: Test
> Reporter: Tom White
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1762) Allow TestHDFSCLI to be run against a
cluster
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated HDFS-1762:
----------------------------
Attachment: HDFS-1762-20.patch
Here's a 20-based patch.
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
> Key: HDFS-1762
> URL: https://issues.apache.org/jira/browse/HDFS-1762
> Project: Hadoop HDFS
> Issue Type: Test
> Reporter: Tom White
> Assignee: Konstantin Boudnik
> Attachments: HDFS-1762-20.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1762) Allow TestHDFSCLI to be run against a
cluster
Posted by "Konstantin Boudnik (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik updated HDFS-1762:
-------------------------------------
Attachment: (was: HDFS-1762.mapreduce.patch)
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
> Key: HDFS-1762
> URL: https://issues.apache.org/jira/browse/HDFS-1762
> Project: Hadoop HDFS
> Issue Type: Test
> Components: build, test
> Affects Versions: 0.22.0
> Reporter: Tom White
> Assignee: Konstantin Boudnik
> Attachments: HDFS-1762-20.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HDFS-1762) Allow TestHDFSCLI to be run against a
cluster
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13007750#comment-13007750 ]
Tom White commented on HDFS-1762:
---------------------------------
This could work by looking for system properties that specify the filesystem and jobtracker, dropping back to the mini clusters if neither were specified.
Some changes to the tests are needed too. Absolute paths at / should be avoided, so a prefix should be allowed. It might be a good idea to split the tests that require superuser privileges (e.g. the quota tests) into a separate file.
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
> Key: HDFS-1762
> URL: https://issues.apache.org/jira/browse/HDFS-1762
> Project: Hadoop HDFS
> Issue Type: Test
> Reporter: Tom White
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against
a cluster
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016196#comment-13016196 ]
Konstantin Boudnik commented on HDFS-1762:
------------------------------------------
bq. ... can override conf directories
we are talking about very special kind of file, actually... So, I am not sure how it can override any configs. But if you're saying there were incidents of the sort I'd rather believe you ;)
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
> Key: HDFS-1762
> URL: https://issues.apache.org/jira/browse/HDFS-1762
> Project: Hadoop HDFS
> Issue Type: Test
> Reporter: Tom White
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against
a cluster
Posted by "Roman Shaposhnik (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122924#comment-13122924 ]
Roman Shaposhnik commented on HDFS-1762:
----------------------------------------
@Cos, please commit it to .22 the Bigtop validation has passed.
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
> Key: HDFS-1762
> URL: https://issues.apache.org/jira/browse/HDFS-1762
> Project: Hadoop HDFS
> Issue Type: Test
> Components: build, test
> Affects Versions: 0.22.0
> Reporter: Tom White
> Assignee: Konstantin Boudnik
> Attachments: HDFS-1762-20.patch, HDFS-1762.common.patch, HDFS-1762.hdfs.patch, HDFS-1762.mapreduce.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against
a cluster
Posted by "Konstantin Shvachko (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13124724#comment-13124724 ]
Konstantin Shvachko commented on HDFS-1762:
-------------------------------------------
+1
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
> Key: HDFS-1762
> URL: https://issues.apache.org/jira/browse/HDFS-1762
> Project: Hadoop HDFS
> Issue Type: Test
> Components: build, test
> Affects Versions: 0.22.0
> Reporter: Tom White
> Assignee: Konstantin Boudnik
> Attachments: HDFS-1762-20.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against
a cluster
Posted by "Konstantin Boudnik (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13124311#comment-13124311 ]
Konstantin Boudnik commented on HDFS-1762:
------------------------------------------
makes sense - I will update the patch shortly.
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
> Key: HDFS-1762
> URL: https://issues.apache.org/jira/browse/HDFS-1762
> Project: Hadoop HDFS
> Issue Type: Test
> Components: build, test
> Affects Versions: 0.22.0
> Reporter: Tom White
> Assignee: Konstantin Boudnik
> Attachments: HDFS-1762-20.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against
a cluster
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13125515#comment-13125515 ]
Hudson commented on HDFS-1762:
------------------------------
Integrated in Hadoop-Hdfs-22-branch #97 (See [https://builds.apache.org/job/Hadoop-Hdfs-22-branch/97/])
HDFS-1762. Allow TestHDFSCLI to be run against a cluster. Contributed by Tom White, Konstantin Boudnik.
cos : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1182116
Files :
* /hadoop/common/branches/branch-0.22/hdfs/CHANGES.txt
* /hadoop/common/branches/branch-0.22/hdfs/build.xml
* /hadoop/common/branches/branch-0.22/hdfs/src/test/hdfs/org/apache/hadoop/cli/TestHDFSCLI.java
* /hadoop/common/branches/branch-0.22/hdfs/src/test/hdfs/org/apache/hadoop/cli/testConfCluster.xml
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
> Key: HDFS-1762
> URL: https://issues.apache.org/jira/browse/HDFS-1762
> Project: Hadoop HDFS
> Issue Type: Test
> Components: build, test
> Affects Versions: 0.22.0
> Reporter: Tom White
> Assignee: Konstantin Boudnik
> Fix For: 0.22.0
>
> Attachments: HDFS-1762-20.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira