You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by kamaldeep singh <ka...@gmail.com> on 2015/01/05 07:09:16 UTC
Unit tests on Hadoop Cluster
Hi ,
I'm a beginner to writing unit tests in hadoop.
As per https://wiki.apache.org/hadoop/HowToDevelopUnitTests the
Hadoop Unit tests are all designed to work on a local machine, rather than
a full-scale Hadoop cluster.
However i do see the Hadoop-QA
https://issues.apache.org/jira/secure/ViewProfile.jspa?name=hadoopqa also
runs unit test cases when it validates a patch for any issue like
https://issues.apache.org/jira/browse/YARN-2459?focusedCommentId=14115586&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14115586
So does this mean it runs unit tests only in single node / local setup for
validating any patches ?
I wish to write some unit tests in a HA env thus i need test in a cluster
setup
Thanks,
Kamal
Re: Unit tests on Hadoop Cluster
Posted by Ted Yu <yu...@gmail.com>.
kamaldeep :
Here is list of HA related tests:
http://pastebin.com/uFnNdGwR
FYI
On Mon, Jan 5, 2015 at 8:22 AM, Ravi Prakash <ra...@ymail.com> wrote:
> Hi Kamal!
>
> Thanks for your initiative. Please take a look at MiniDFSCluster /
> MiniJournalCluster / MiniYarnCluster etc. In your unit tests you can
> essentially start a cluster in a single JVM. You can look at
> TestQJMWithFaults.java
>
> HTH
> Ravi
>
>
> On Sunday, January 4, 2015 10:09 PM, kamaldeep singh <
> kamaldeep.ebay@gmail.com> wrote:
>
>
> Hi ,
>
> I'm a beginner to writing unit tests in hadoop.
>
>
> As per https://wiki.apache.org/hadoop/HowToDevelopUnitTests the
> Hadoop Unit tests are all designed to work on a local machine, rather than
> a full-scale Hadoop cluster.
>
> However i do see the Hadoop-QA
> https://issues.apache.org/jira/secure/ViewProfile.jspa?name=hadoopqa also
> runs unit test cases when it validates a patch for any issue like
> https://issues.apache.org/jira/browse/YARN-2459?focusedCommentId=14115586&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14115586
>
> So does this mean it runs unit tests only in single node / local setup for
> validating any patches ?
>
> I wish to write some unit tests in a HA env thus i need test in a cluster
> setup
>
> Thanks,
> Kamal
>
>
>
Re: Unit tests on Hadoop Cluster
Posted by Ted Yu <yu...@gmail.com>.
kamaldeep :
Here is list of HA related tests:
http://pastebin.com/uFnNdGwR
FYI
On Mon, Jan 5, 2015 at 8:22 AM, Ravi Prakash <ra...@ymail.com> wrote:
> Hi Kamal!
>
> Thanks for your initiative. Please take a look at MiniDFSCluster /
> MiniJournalCluster / MiniYarnCluster etc. In your unit tests you can
> essentially start a cluster in a single JVM. You can look at
> TestQJMWithFaults.java
>
> HTH
> Ravi
>
>
> On Sunday, January 4, 2015 10:09 PM, kamaldeep singh <
> kamaldeep.ebay@gmail.com> wrote:
>
>
> Hi ,
>
> I'm a beginner to writing unit tests in hadoop.
>
>
> As per https://wiki.apache.org/hadoop/HowToDevelopUnitTests the
> Hadoop Unit tests are all designed to work on a local machine, rather than
> a full-scale Hadoop cluster.
>
> However i do see the Hadoop-QA
> https://issues.apache.org/jira/secure/ViewProfile.jspa?name=hadoopqa also
> runs unit test cases when it validates a patch for any issue like
> https://issues.apache.org/jira/browse/YARN-2459?focusedCommentId=14115586&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14115586
>
> So does this mean it runs unit tests only in single node / local setup for
> validating any patches ?
>
> I wish to write some unit tests in a HA env thus i need test in a cluster
> setup
>
> Thanks,
> Kamal
>
>
>
Re: Unit tests on Hadoop Cluster
Posted by Ted Yu <yu...@gmail.com>.
kamaldeep :
Here is list of HA related tests:
http://pastebin.com/uFnNdGwR
FYI
On Mon, Jan 5, 2015 at 8:22 AM, Ravi Prakash <ra...@ymail.com> wrote:
> Hi Kamal!
>
> Thanks for your initiative. Please take a look at MiniDFSCluster /
> MiniJournalCluster / MiniYarnCluster etc. In your unit tests you can
> essentially start a cluster in a single JVM. You can look at
> TestQJMWithFaults.java
>
> HTH
> Ravi
>
>
> On Sunday, January 4, 2015 10:09 PM, kamaldeep singh <
> kamaldeep.ebay@gmail.com> wrote:
>
>
> Hi ,
>
> I'm a beginner to writing unit tests in hadoop.
>
>
> As per https://wiki.apache.org/hadoop/HowToDevelopUnitTests the
> Hadoop Unit tests are all designed to work on a local machine, rather than
> a full-scale Hadoop cluster.
>
> However i do see the Hadoop-QA
> https://issues.apache.org/jira/secure/ViewProfile.jspa?name=hadoopqa also
> runs unit test cases when it validates a patch for any issue like
> https://issues.apache.org/jira/browse/YARN-2459?focusedCommentId=14115586&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14115586
>
> So does this mean it runs unit tests only in single node / local setup for
> validating any patches ?
>
> I wish to write some unit tests in a HA env thus i need test in a cluster
> setup
>
> Thanks,
> Kamal
>
>
>
Re: Unit tests on Hadoop Cluster
Posted by Ted Yu <yu...@gmail.com>.
kamaldeep :
Here is list of HA related tests:
http://pastebin.com/uFnNdGwR
FYI
On Mon, Jan 5, 2015 at 8:22 AM, Ravi Prakash <ra...@ymail.com> wrote:
> Hi Kamal!
>
> Thanks for your initiative. Please take a look at MiniDFSCluster /
> MiniJournalCluster / MiniYarnCluster etc. In your unit tests you can
> essentially start a cluster in a single JVM. You can look at
> TestQJMWithFaults.java
>
> HTH
> Ravi
>
>
> On Sunday, January 4, 2015 10:09 PM, kamaldeep singh <
> kamaldeep.ebay@gmail.com> wrote:
>
>
> Hi ,
>
> I'm a beginner to writing unit tests in hadoop.
>
>
> As per https://wiki.apache.org/hadoop/HowToDevelopUnitTests the
> Hadoop Unit tests are all designed to work on a local machine, rather than
> a full-scale Hadoop cluster.
>
> However i do see the Hadoop-QA
> https://issues.apache.org/jira/secure/ViewProfile.jspa?name=hadoopqa also
> runs unit test cases when it validates a patch for any issue like
> https://issues.apache.org/jira/browse/YARN-2459?focusedCommentId=14115586&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14115586
>
> So does this mean it runs unit tests only in single node / local setup for
> validating any patches ?
>
> I wish to write some unit tests in a HA env thus i need test in a cluster
> setup
>
> Thanks,
> Kamal
>
>
>
Re: Unit tests on Hadoop Cluster
Posted by Ravi Prakash <ra...@ymail.com>.
Hi Kamal!
Thanks for your initiative. Please take a look at MiniDFSCluster / MiniJournalCluster / MiniYarnCluster etc. In your unit tests you can essentially start a cluster in a single JVM. You can look at TestQJMWithFaults.java
HTHRavi
On Sunday, January 4, 2015 10:09 PM, kamaldeep singh <ka...@gmail.com> wrote:
Hi ,
I'm a beginner to writing unit tests in hadoop.
As per https://wiki.apache.org/hadoop/HowToDevelopUnitTests the Hadoop Unit tests are all designed to work on a local machine, rather than a full-scale Hadoop cluster.
However i do see the Hadoop-QA https://issues.apache.org/jira/secure/ViewProfile.jspa?name=hadoopqa also runs unit test cases when it validates a patch for any issue like https://issues.apache.org/jira/browse/YARN-2459?focusedCommentId=14115586&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14115586
So does this mean it runs unit tests only in single node / local setup for validating any patches ?
I wish to write some unit tests in a HA env thus i need test in a cluster setup
Thanks,Kamal
Re: Unit tests on Hadoop Cluster
Posted by Ravi Prakash <ra...@ymail.com>.
Hi Kamal!
Thanks for your initiative. Please take a look at MiniDFSCluster / MiniJournalCluster / MiniYarnCluster etc. In your unit tests you can essentially start a cluster in a single JVM. You can look at TestQJMWithFaults.java
HTHRavi
On Sunday, January 4, 2015 10:09 PM, kamaldeep singh <ka...@gmail.com> wrote:
Hi ,
I'm a beginner to writing unit tests in hadoop.
As per https://wiki.apache.org/hadoop/HowToDevelopUnitTests the Hadoop Unit tests are all designed to work on a local machine, rather than a full-scale Hadoop cluster.
However i do see the Hadoop-QA https://issues.apache.org/jira/secure/ViewProfile.jspa?name=hadoopqa also runs unit test cases when it validates a patch for any issue like https://issues.apache.org/jira/browse/YARN-2459?focusedCommentId=14115586&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14115586
So does this mean it runs unit tests only in single node / local setup for validating any patches ?
I wish to write some unit tests in a HA env thus i need test in a cluster setup
Thanks,Kamal
Re: Unit tests on Hadoop Cluster
Posted by Ravi Prakash <ra...@ymail.com>.
Hi Kamal!
Thanks for your initiative. Please take a look at MiniDFSCluster / MiniJournalCluster / MiniYarnCluster etc. In your unit tests you can essentially start a cluster in a single JVM. You can look at TestQJMWithFaults.java
HTHRavi
On Sunday, January 4, 2015 10:09 PM, kamaldeep singh <ka...@gmail.com> wrote:
Hi ,
I'm a beginner to writing unit tests in hadoop.
As per https://wiki.apache.org/hadoop/HowToDevelopUnitTests the Hadoop Unit tests are all designed to work on a local machine, rather than a full-scale Hadoop cluster.
However i do see the Hadoop-QA https://issues.apache.org/jira/secure/ViewProfile.jspa?name=hadoopqa also runs unit test cases when it validates a patch for any issue like https://issues.apache.org/jira/browse/YARN-2459?focusedCommentId=14115586&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14115586
So does this mean it runs unit tests only in single node / local setup for validating any patches ?
I wish to write some unit tests in a HA env thus i need test in a cluster setup
Thanks,Kamal
Re: Unit tests on Hadoop Cluster
Posted by Ravi Prakash <ra...@ymail.com>.
Hi Kamal!
Thanks for your initiative. Please take a look at MiniDFSCluster / MiniJournalCluster / MiniYarnCluster etc. In your unit tests you can essentially start a cluster in a single JVM. You can look at TestQJMWithFaults.java
HTHRavi
On Sunday, January 4, 2015 10:09 PM, kamaldeep singh <ka...@gmail.com> wrote:
Hi ,
I'm a beginner to writing unit tests in hadoop.
As per https://wiki.apache.org/hadoop/HowToDevelopUnitTests the Hadoop Unit tests are all designed to work on a local machine, rather than a full-scale Hadoop cluster.
However i do see the Hadoop-QA https://issues.apache.org/jira/secure/ViewProfile.jspa?name=hadoopqa also runs unit test cases when it validates a patch for any issue like https://issues.apache.org/jira/browse/YARN-2459?focusedCommentId=14115586&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14115586
So does this mean it runs unit tests only in single node / local setup for validating any patches ?
I wish to write some unit tests in a HA env thus i need test in a cluster setup
Thanks,Kamal