You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Tom White (JIRA)" <ji...@apache.org> on 2011/03/17 01:04:29 UTC

[jira] Created: (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Allow TestHDFSCLI to be run against a cluster
---------------------------------------------

                 Key: HDFS-1762
                 URL: https://issues.apache.org/jira/browse/HDFS-1762
             Project: Hadoop HDFS
          Issue Type: Test
            Reporter: Tom White


Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Posted by "Konstantin Boudnik (Resolved) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Konstantin Boudnik resolved HDFS-1762.
--------------------------------------

       Resolution: Fixed
    Fix Version/s: 0.22.0
     Hadoop Flags: Reviewed

I have committed it.
                
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
>                 Key: HDFS-1762
>                 URL: https://issues.apache.org/jira/browse/HDFS-1762
>             Project: Hadoop HDFS
>          Issue Type: Test
>          Components: build, test
>    Affects Versions: 0.22.0
>            Reporter: Tom White
>            Assignee: Konstantin Boudnik
>             Fix For: 0.22.0
>
>         Attachments: HDFS-1762-20.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016498#comment-13016498 ] 

Konstantin Boudnik commented on HDFS-1762:
------------------------------------------

Todd, I was talking about {{testHDFSConf.xml}} file which actually defined test cases executed by TestCLI class. The risk of 'overriding' in this case is pretty much close to 0. I wouldn't dare to suggest to add any hadoop related config files into test artifact.

> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
>                 Key: HDFS-1762
>                 URL: https://issues.apache.org/jira/browse/HDFS-1762
>             Project: Hadoop HDFS
>          Issue Type: Test
>            Reporter: Tom White
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Posted by "Konstantin Boudnik (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Konstantin Boudnik updated HDFS-1762:
-------------------------------------

    Attachment:     (was: HDFS-1762.common.patch)
    
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
>                 Key: HDFS-1762
>                 URL: https://issues.apache.org/jira/browse/HDFS-1762
>             Project: Hadoop HDFS
>          Issue Type: Test
>          Components: build, test
>    Affects Versions: 0.22.0
>            Reporter: Tom White
>            Assignee: Konstantin Boudnik
>         Attachments: HDFS-1762-20.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Posted by "Konstantin Shvachko (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13124307#comment-13124307 ] 

Konstantin Shvachko commented on HDFS-1762:
-------------------------------------------

The issues I mentioned before are intended to support host names with "-" (etc.) in them. Some tests (like du:) will fail on my host as it has a dash in it. 
                
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
>                 Key: HDFS-1762
>                 URL: https://issues.apache.org/jira/browse/HDFS-1762
>             Project: Hadoop HDFS
>          Issue Type: Test
>          Components: build, test
>    Affects Versions: 0.22.0
>            Reporter: Tom White
>            Assignee: Konstantin Boudnik
>         Attachments: HDFS-1762-20.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Posted by "Konstantin Boudnik (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Konstantin Boudnik updated HDFS-1762:
-------------------------------------

    Affects Version/s: 0.22.0
    
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
>                 Key: HDFS-1762
>                 URL: https://issues.apache.org/jira/browse/HDFS-1762
>             Project: Hadoop HDFS
>          Issue Type: Test
>    Affects Versions: 0.22.0
>            Reporter: Tom White
>            Assignee: Konstantin Boudnik
>         Attachments: HDFS-1762-20.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Assigned] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Konstantin Boudnik reassigned HDFS-1762:
----------------------------------------

    Assignee: Konstantin Boudnik

> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
>                 Key: HDFS-1762
>                 URL: https://issues.apache.org/jira/browse/HDFS-1762
>             Project: Hadoop HDFS
>          Issue Type: Test
>            Reporter: Tom White
>            Assignee: Konstantin Boudnik
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Posted by "Konstantin Boudnik (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Konstantin Boudnik updated HDFS-1762:
-------------------------------------

    Attachment: HDFS-1762.hdfs.patch

Addressing issue for hostnames with '-' 
                
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
>                 Key: HDFS-1762
>                 URL: https://issues.apache.org/jira/browse/HDFS-1762
>             Project: Hadoop HDFS
>          Issue Type: Test
>          Components: build, test
>    Affects Versions: 0.22.0
>            Reporter: Tom White
>            Assignee: Konstantin Boudnik
>         Attachments: HDFS-1762-20.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Posted by "Konstantin Boudnik (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123627#comment-13123627 ] 

Konstantin Boudnik commented on HDFS-1762:
------------------------------------------

For complete picture look at the patches associated with HADOOP-7730 and MAPREDUCE-3156
                
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
>                 Key: HDFS-1762
>                 URL: https://issues.apache.org/jira/browse/HDFS-1762
>             Project: Hadoop HDFS
>          Issue Type: Test
>          Components: build, test
>    Affects Versions: 0.22.0
>            Reporter: Tom White
>            Assignee: Konstantin Boudnik
>         Attachments: HDFS-1762-20.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Posted by "Konstantin Boudnik (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Konstantin Boudnik updated HDFS-1762:
-------------------------------------

    Attachment: HDFS-1762.hdfs.patch

Correcting some regexps to reflect changes in the output formats.
                
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
>                 Key: HDFS-1762
>                 URL: https://issues.apache.org/jira/browse/HDFS-1762
>             Project: Hadoop HDFS
>          Issue Type: Test
>          Components: build, test
>    Affects Versions: 0.22.0
>            Reporter: Tom White
>            Assignee: Konstantin Boudnik
>         Attachments: HDFS-1762-20.patch, HDFS-1762.common.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.mapreduce.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13012228#comment-13012228 ] 

Konstantin Boudnik commented on HDFS-1762:
------------------------------------------

Also, it seems to be necessary to include appropriate XML files into hadoop-tests.jar because otherwise TestCLI won't work at all.

> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
>                 Key: HDFS-1762
>                 URL: https://issues.apache.org/jira/browse/HDFS-1762
>             Project: Hadoop HDFS
>          Issue Type: Test
>            Reporter: Tom White
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Posted by "Konstantin Boudnik (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Konstantin Boudnik updated HDFS-1762:
-------------------------------------

    Component/s: test
                 build
    
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
>                 Key: HDFS-1762
>                 URL: https://issues.apache.org/jira/browse/HDFS-1762
>             Project: Hadoop HDFS
>          Issue Type: Test
>          Components: build, test
>    Affects Versions: 0.22.0
>            Reporter: Tom White
>            Assignee: Konstantin Boudnik
>         Attachments: HDFS-1762-20.patch, HDFS-1762.common.patch, HDFS-1762.hdfs.patch, HDFS-1762.mapreduce.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016214#comment-13016214 ] 

Todd Lipcon commented on HDFS-1762:
-----------------------------------

I may have misread. In the past, over in HBase land, we had an hbase-site.xml included in our hbase-test.jar. This meant that if someone accidentally ended up with this on their classpath, they picked up weird configurations that we meant to only apply in tests.

Putting an XML file that's specific to TestHDFSCLI in the jar seems fine, though.

> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
>                 Key: HDFS-1762
>                 URL: https://issues.apache.org/jira/browse/HDFS-1762
>             Project: Hadoop HDFS
>          Issue Type: Test
>            Reporter: Tom White
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Posted by "Konstantin Boudnik (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Konstantin Boudnik updated HDFS-1762:
-------------------------------------

    Attachment: HDFS-1762.hdfs.patch

Fixing comparators for 4 cases. Now everything passes with hdfs-0.22 cluster.
                
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
>                 Key: HDFS-1762
>                 URL: https://issues.apache.org/jira/browse/HDFS-1762
>             Project: Hadoop HDFS
>          Issue Type: Test
>          Components: build, test
>    Affects Versions: 0.22.0
>            Reporter: Tom White
>            Assignee: Konstantin Boudnik
>         Attachments: HDFS-1762-20.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Posted by "Konstantin Boudnik (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120658#comment-13120658 ] 

Konstantin Boudnik commented on HDFS-1762:
------------------------------------------

Patches above works for local mini-cluster setup without any problem. I will be validating this with a real cluster before committing them.
                
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
>                 Key: HDFS-1762
>                 URL: https://issues.apache.org/jira/browse/HDFS-1762
>             Project: Hadoop HDFS
>          Issue Type: Test
>    Affects Versions: 0.22.0
>            Reporter: Tom White
>            Assignee: Konstantin Boudnik
>         Attachments: HDFS-1762-20.patch, HDFS-1762.common.patch, HDFS-1762.hdfs.patch, HDFS-1762.mapreduce.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Posted by "Konstantin Shvachko (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13124308#comment-13124308 ] 

Konstantin Shvachko commented on HDFS-1762:
-------------------------------------------

(like du: )
                
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
>                 Key: HDFS-1762
>                 URL: https://issues.apache.org/jira/browse/HDFS-1762
>             Project: Hadoop HDFS
>          Issue Type: Test
>          Components: build, test
>    Affects Versions: 0.22.0
>            Reporter: Tom White
>            Assignee: Konstantin Boudnik
>         Attachments: HDFS-1762-20.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016545#comment-13016545 ] 

Todd Lipcon commented on HDFS-1762:
-----------------------------------

Ah, sorry for the misunderstanding. Including {{testHDFSConf.xml}} seems totally fine to me.

> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
>                 Key: HDFS-1762
>                 URL: https://issues.apache.org/jira/browse/HDFS-1762
>             Project: Hadoop HDFS
>          Issue Type: Test
>            Reporter: Tom White
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Posted by "Konstantin Shvachko (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123796#comment-13123796 ] 

Konstantin Shvachko commented on HDFS-1762:
-------------------------------------------

The patch looks good. One thing I would recommend to update regular expressions for host names and paths in the spirit of HDFS-2232 and MAPREDUCE-2304.
Also TestConfCluster.xml need ASF license header.
                
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
>                 Key: HDFS-1762
>                 URL: https://issues.apache.org/jira/browse/HDFS-1762
>             Project: Hadoop HDFS
>          Issue Type: Test
>          Components: build, test
>    Affects Versions: 0.22.0
>            Reporter: Tom White
>            Assignee: Konstantin Boudnik
>         Attachments: HDFS-1762-20.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Posted by "Konstantin Boudnik (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Konstantin Boudnik updated HDFS-1762:
-------------------------------------

    Attachment: HDFS-1762.patch

Added missing license. Also, I'd propose to address any regexp issues separately. The reason for two files was to avoid too wide (e.g. generic) expressions in the first place. 
                
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
>                 Key: HDFS-1762
>                 URL: https://issues.apache.org/jira/browse/HDFS-1762
>             Project: Hadoop HDFS
>          Issue Type: Test
>          Components: build, test
>    Affects Versions: 0.22.0
>            Reporter: Tom White
>            Assignee: Konstantin Boudnik
>         Attachments: HDFS-1762-20.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Posted by "Konstantin Boudnik (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Konstantin Boudnik updated HDFS-1762:
-------------------------------------

    Attachment: HDFS-1762.mapreduce.patch
                HDFS-1762.hdfs.patch
                HDFS-1762.common.patch

These are patches for all three subprojects for 0.22 branch. I will create separate tickets in a bit.
They should be mostly good for 0.23 except the ant build part which needs to be re-worked for maven.
                
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
>                 Key: HDFS-1762
>                 URL: https://issues.apache.org/jira/browse/HDFS-1762
>             Project: Hadoop HDFS
>          Issue Type: Test
>    Affects Versions: 0.22.0
>            Reporter: Tom White
>            Assignee: Konstantin Boudnik
>         Attachments: HDFS-1762-20.patch, HDFS-1762.common.patch, HDFS-1762.hdfs.patch, HDFS-1762.mapreduce.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13012253#comment-13012253 ] 

Todd Lipcon commented on HDFS-1762:
-----------------------------------

I don't think adding the XML files to the test jar is a good idea. We've had that in the past, and since the test jar often ends up on other people's classpaths, and can override conf directories, etc.

> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
>                 Key: HDFS-1762
>                 URL: https://issues.apache.org/jira/browse/HDFS-1762
>             Project: Hadoop HDFS
>          Issue Type: Test
>            Reporter: Tom White
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Posted by "Tom White (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tom White updated HDFS-1762:
----------------------------

    Attachment: HDFS-1762-20.patch

Here's a 20-based patch.

> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
>                 Key: HDFS-1762
>                 URL: https://issues.apache.org/jira/browse/HDFS-1762
>             Project: Hadoop HDFS
>          Issue Type: Test
>            Reporter: Tom White
>            Assignee: Konstantin Boudnik
>         Attachments: HDFS-1762-20.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Posted by "Konstantin Boudnik (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Konstantin Boudnik updated HDFS-1762:
-------------------------------------

    Attachment:     (was: HDFS-1762.mapreduce.patch)
    
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
>                 Key: HDFS-1762
>                 URL: https://issues.apache.org/jira/browse/HDFS-1762
>             Project: Hadoop HDFS
>          Issue Type: Test
>          Components: build, test
>    Affects Versions: 0.22.0
>            Reporter: Tom White
>            Assignee: Konstantin Boudnik
>         Attachments: HDFS-1762-20.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] Commented: (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Posted by "Tom White (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13007750#comment-13007750 ] 

Tom White commented on HDFS-1762:
---------------------------------

This could work by looking for system properties that specify the filesystem and jobtracker, dropping back to the mini clusters if neither were specified.

Some changes to the tests are needed too. Absolute paths at / should be avoided, so a prefix should be allowed. It might be a good idea to split the tests that require superuser privileges (e.g. the quota tests) into a separate file. 

> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
>                 Key: HDFS-1762
>                 URL: https://issues.apache.org/jira/browse/HDFS-1762
>             Project: Hadoop HDFS
>          Issue Type: Test
>            Reporter: Tom White
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016196#comment-13016196 ] 

Konstantin Boudnik commented on HDFS-1762:
------------------------------------------

bq. ... can override conf directories

we are talking about very special kind of file, actually... So, I am not sure how it can override any configs. But if you're saying there were incidents of the sort I'd rather believe you ;)

> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
>                 Key: HDFS-1762
>                 URL: https://issues.apache.org/jira/browse/HDFS-1762
>             Project: Hadoop HDFS
>          Issue Type: Test
>            Reporter: Tom White
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Posted by "Roman Shaposhnik (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122924#comment-13122924 ] 

Roman Shaposhnik commented on HDFS-1762:
----------------------------------------

@Cos, please commit it to .22 the Bigtop validation has passed.
                
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
>                 Key: HDFS-1762
>                 URL: https://issues.apache.org/jira/browse/HDFS-1762
>             Project: Hadoop HDFS
>          Issue Type: Test
>          Components: build, test
>    Affects Versions: 0.22.0
>            Reporter: Tom White
>            Assignee: Konstantin Boudnik
>         Attachments: HDFS-1762-20.patch, HDFS-1762.common.patch, HDFS-1762.hdfs.patch, HDFS-1762.mapreduce.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Posted by "Konstantin Shvachko (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13124724#comment-13124724 ] 

Konstantin Shvachko commented on HDFS-1762:
-------------------------------------------

+1
                
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
>                 Key: HDFS-1762
>                 URL: https://issues.apache.org/jira/browse/HDFS-1762
>             Project: Hadoop HDFS
>          Issue Type: Test
>          Components: build, test
>    Affects Versions: 0.22.0
>            Reporter: Tom White
>            Assignee: Konstantin Boudnik
>         Attachments: HDFS-1762-20.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Posted by "Konstantin Boudnik (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13124311#comment-13124311 ] 

Konstantin Boudnik commented on HDFS-1762:
------------------------------------------

makes sense - I will update the patch shortly.
                
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
>                 Key: HDFS-1762
>                 URL: https://issues.apache.org/jira/browse/HDFS-1762
>             Project: Hadoop HDFS
>          Issue Type: Test
>          Components: build, test
>    Affects Versions: 0.22.0
>            Reporter: Tom White
>            Assignee: Konstantin Boudnik
>         Attachments: HDFS-1762-20.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster

Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13125515#comment-13125515 ] 

Hudson commented on HDFS-1762:
------------------------------

Integrated in Hadoop-Hdfs-22-branch #97 (See [https://builds.apache.org/job/Hadoop-Hdfs-22-branch/97/])
    HDFS-1762. Allow TestHDFSCLI to be run against a cluster. Contributed by Tom White, Konstantin Boudnik.

cos : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1182116
Files : 
* /hadoop/common/branches/branch-0.22/hdfs/CHANGES.txt
* /hadoop/common/branches/branch-0.22/hdfs/build.xml
* /hadoop/common/branches/branch-0.22/hdfs/src/test/hdfs/org/apache/hadoop/cli/TestHDFSCLI.java
* /hadoop/common/branches/branch-0.22/hdfs/src/test/hdfs/org/apache/hadoop/cli/testConfCluster.xml

                
> Allow TestHDFSCLI to be run against a cluster
> ---------------------------------------------
>
>                 Key: HDFS-1762
>                 URL: https://issues.apache.org/jira/browse/HDFS-1762
>             Project: Hadoop HDFS
>          Issue Type: Test
>          Components: build, test
>    Affects Versions: 0.22.0
>            Reporter: Tom White
>            Assignee: Konstantin Boudnik
>             Fix For: 0.22.0
>
>         Attachments: HDFS-1762-20.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.hdfs.patch, HDFS-1762.patch
>
>
> Currently TestHDFSCLI starts mini clusters to run tests against. It would be useful to be able to support running against arbitrary clusters for testing purposes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira