You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Stefan Groschupf <sg...@101tec.com> on 2008/03/07 01:25:40 UTC
test suite
Hi there,
I checked out latest svn and did run the test.
The test suite failed at TestTable, are there known problems today?
Also the test suite took 24 min to run on my little poor box - is
there any way to speed this up?
Thanks.
Stefan
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
101tec Inc.
Menlo Park, California, USA
http://www.101tec.com
Re: test suite
Posted by Stefan Groschupf <sg...@101tec.com>.
> There is an issue using local filesystem on windows that we need to
> figure. Will try leaning on our windows user to see if we can get
> more info around the prob.
If it is a problem with unix commands on linux there is a simple
solution for that.
Just install cygwin on a windows box and add the cygwin bin folder
where all the *.exe files are to your classpath.
We did run also in issues using the dfs clients on windows but this
solution fixed that problem.
HTH
Stefan
Re: test suite
Posted by stack <st...@duboce.net>.
You're not missing anything Stefan. I think there is general agreement
that 30 mins is too long to wait on tests (Bryan Duxbury, to his credit,
keeps trying to bring up this fact. He filed HBASE-498 last night).
There is an issue using local filesystem on windows that we need to
figure. Will try leaning on our windows user to see if we can get more
info around the prob. Will post to HBASE-498.
St.Ack
Stefan Groschupf wrote:
> I'm sorry if I ask stupid questions, I'm very new to HBase.
> Waiting 30 min to see if a change breaks something or not is painful.
> I understand why you want to use a hadoop minicluster for integration
> tests but for normal unit tests.
> I would expect that there is an interface for each hbase component
> that the server and the clients implements. Similar to the hadoop
> jobrunner and localJobrunner. Since both components implement this
> interface it is transparent for the test if we use a distributed mini
> cluster or a local non tcp ip implementatipon. Also we could have
> mocks implementing this interface.
> So which puzzle peace I miss that there is the requirement to start up
> this minicluster every time?
>
>
>
> On Mar 6, 2008, at 8:24 PM, stack wrote:
>
>> Stefan Groschupf wrote:
>>> Is there an hudson that build hbase?
>>> Hudson should run the integration test suite with mini cluster and
>>> dfs where ant test for the user should only run local fs and local
>>> map reduce jobrunner.
>> Yeah. We have our own little zone on hudson:
>> http://hudson.zones.apache.org/hudson/job/HBase-Patch/. Needs some
>> work. Currently just runs a build on checkin. Would like to add
>> the patch testing that happens in hadoop core.
>>
>> Starting dfs cluster from ant script is a little awkward.
>> MiniDFSCluster is nice because it chooses unoccupied ports; important
>> on a populated build box. Would need to then pass the chosen dfs
>> port then to tests and mini dfs would have to be in current process
>> if mindfs so can shut it down when done. I tried running in ant
>> context as distinct task and as wrapper around junit task but ran
>> into class loading issues.
>>
>> Could call out to a script that starts minidfs but need to do a
>> little dev so can send it a shutdown when done. Was not enthusiastic
>> since replicating the 'real' dfs start/stop scripts. Alternative
>> would be to start a 'real' dfs using fixed ports. This would involve
>> ant exec'ing. Would need to deal with *nix and windows paths to make
>> sure it worked on both. If I'm not mistaken, even 'local' mode
>> requires ssh'ing to start datanodes even if on localhost.
>>
>> Maybe the best thing would be to just punt on trying to have the
>> start/stop of the dfs be run from ant and just do it as part of the
>> build scripts we run up on the hudson build box?
>>
>> St.Ack
>>
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 101tec Inc.
> Menlo Park, California, USA
> http://www.101tec.com
>
>
RE: test suite
Posted by Jim Kellerman <ji...@powerset.com>.
Sigh. Yes. That test is very problematic.
---
Jim Kellerman, Senior Engineer; Powerset
> -----Original Message-----
> From: Stefan Groschupf [mailto:sg@101tec.com]
> Sent: Friday, March 07, 2008 1:55 PM
> To: hbase-dev@hadoop.apache.org
> Subject: Re: test suite
>
> [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
> [junit] Test org.apache.hadoop.hbase.mapred.TestTableIndex FAILED
> (timeout)
>
>
> Testsuite: org.apache.hadoop.hbase.mapred.TestTableIndex
> Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
>
> Testcase: unknown took 0.05 sec
> Caused an ERROR
> Timeout occurred
> junit.framework.AssertionFailedError: Timeout occurred
>
>
>
> Thats all I have, sounds this familiar?
>
> Thanks.
> Stefan
>
>
>
> On Mar 7, 2008, at 1:22 PM, Jim Kellerman wrote:
>
> > If the test suite doesn't pass, that's a problem. What tests failed?
> >
> > And you can run single tests if you want.
> >
> > Wrt Windows, I use cygwin, but the problem is not unix
> commands, but
> > code in HBase that deletes files and directories. FileSystem.delete
> > behaves differently on unix vs Windows. What we should be doing for
> > directories is use FileUtil.deleteFully
> >
> > ---
> > Jim Kellerman, Senior Engineer; Powerset
> >
> >
> >> -----Original Message-----
> >> From: Stefan Groschupf [mailto:sg@101tec.com]
> >> Sent: Friday, March 07, 2008 1:15 PM
> >> To: hbase-dev@hadoop.apache.org
> >> Subject: Re: test suite
> >>
> >>> Since HBase is targeted to run on hdfs and not the local
> >> file system,
> >>> we want to test it in its "native" environment and not a
> >> mock-up which
> >>> might not have all the ideosynchracies of the "real thing".
> >>
> >> Jim, I agree there should be integration tests that run
> against the
> >> hdfs - that can run on hudon. However there should be unit test
> >> that test code units of hbase not of hadoop. Those should
> not use the
> >> dfs since the idea of unit tests is not test underlaying
> third party
> >> systems.
> >>
> >> Anyhow just a user feedback the test suite as it is today is not
> >> useful for me personal, since first, it never passed on my box and
> >> second it takes 30 min to run - so I never run it
> >> - what is not the idea of having a test suite and never
> run it during
> >> development.
> >>
> >> Just some feedback...
> >>
> >> Stefan
> >>
> >>
> >> No virus found in this incoming message.
> >> Checked by AVG.
> >> Version: 7.5.518 / Virus Database: 269.21.6/1318 - Release
> >> Date: 3/7/2008 2:01 PM
> >>
> >>
> >
> > No virus found in this outgoing message.
> > Checked by AVG.
> > Version: 7.5.518 / Virus Database: 269.21.6/1318 - Release Date:
> > 3/7/2008 2:01 PM
> >
> >
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 101tec Inc.
> Menlo Park, California, USA
> http://www.101tec.com
>
>
>
> No virus found in this incoming message.
> Checked by AVG.
> Version: 7.5.518 / Virus Database: 269.21.6/1318 - Release
> Date: 3/7/2008 2:01 PM
>
>
No virus found in this outgoing message.
Checked by AVG.
Version: 7.5.518 / Virus Database: 269.21.6/1318 - Release Date: 3/7/2008 2:01 PM
Re: test suite
Posted by Stefan Groschupf <sg...@101tec.com>.
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
[junit] Test org.apache.hadoop.hbase.mapred.TestTableIndex FAILED
(timeout)
Testsuite: org.apache.hadoop.hbase.mapred.TestTableIndex
Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
Testcase: unknown took 0.05 sec
Caused an ERROR
Timeout occurred
junit.framework.AssertionFailedError: Timeout occurred
Thats all I have, sounds this familiar?
Thanks.
Stefan
On Mar 7, 2008, at 1:22 PM, Jim Kellerman wrote:
> If the test suite doesn't pass, that's a problem. What tests failed?
>
> And you can run single tests if you want.
>
> Wrt Windows, I use cygwin, but the problem is not unix commands, but
> code in HBase that deletes files and directories. FileSystem.delete
> behaves differently on unix vs Windows. What we should be doing for
> directories is use FileUtil.deleteFully
>
> ---
> Jim Kellerman, Senior Engineer; Powerset
>
>
>> -----Original Message-----
>> From: Stefan Groschupf [mailto:sg@101tec.com]
>> Sent: Friday, March 07, 2008 1:15 PM
>> To: hbase-dev@hadoop.apache.org
>> Subject: Re: test suite
>>
>>> Since HBase is targeted to run on hdfs and not the local
>> file system,
>>> we want to test it in its "native" environment and not a
>> mock-up which
>>> might not have all the ideosynchracies of the "real thing".
>>
>> Jim, I agree there should be integration tests that run
>> against the hdfs - that can run on hudon. However there
>> should be unit test that test code units of hbase not of
>> hadoop. Those should not use the dfs since the idea of unit
>> tests is not test underlaying third party systems.
>>
>> Anyhow just a user feedback the test suite as it is today is
>> not useful for me personal, since first, it never passed on
>> my box and second it takes 30 min to run - so I never run it
>> - what is not the idea of having a test suite and never run
>> it during development.
>>
>> Just some feedback...
>>
>> Stefan
>>
>>
>> No virus found in this incoming message.
>> Checked by AVG.
>> Version: 7.5.518 / Virus Database: 269.21.6/1318 - Release
>> Date: 3/7/2008 2:01 PM
>>
>>
>
> No virus found in this outgoing message.
> Checked by AVG.
> Version: 7.5.518 / Virus Database: 269.21.6/1318 - Release Date:
> 3/7/2008 2:01 PM
>
>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
101tec Inc.
Menlo Park, California, USA
http://www.101tec.com
RE: test suite
Posted by Jim Kellerman <ji...@powerset.com>.
If the test suite doesn't pass, that's a problem. What tests failed?
And you can run single tests if you want.
Wrt Windows, I use cygwin, but the problem is not unix commands, but
code in HBase that deletes files and directories. FileSystem.delete
behaves differently on unix vs Windows. What we should be doing for
directories is use FileUtil.deleteFully
---
Jim Kellerman, Senior Engineer; Powerset
> -----Original Message-----
> From: Stefan Groschupf [mailto:sg@101tec.com]
> Sent: Friday, March 07, 2008 1:15 PM
> To: hbase-dev@hadoop.apache.org
> Subject: Re: test suite
>
> > Since HBase is targeted to run on hdfs and not the local
> file system,
> > we want to test it in its "native" environment and not a
> mock-up which
> > might not have all the ideosynchracies of the "real thing".
>
> Jim, I agree there should be integration tests that run
> against the hdfs - that can run on hudon. However there
> should be unit test that test code units of hbase not of
> hadoop. Those should not use the dfs since the idea of unit
> tests is not test underlaying third party systems.
>
> Anyhow just a user feedback the test suite as it is today is
> not useful for me personal, since first, it never passed on
> my box and second it takes 30 min to run - so I never run it
> - what is not the idea of having a test suite and never run
> it during development.
>
> Just some feedback...
>
> Stefan
>
>
> No virus found in this incoming message.
> Checked by AVG.
> Version: 7.5.518 / Virus Database: 269.21.6/1318 - Release
> Date: 3/7/2008 2:01 PM
>
>
No virus found in this outgoing message.
Checked by AVG.
Version: 7.5.518 / Virus Database: 269.21.6/1318 - Release Date: 3/7/2008 2:01 PM
Re: test suite
Posted by Stefan Groschupf <sg...@101tec.com>.
> Since HBase is targeted to run on hdfs and not the local file system,
> we want to test it in its "native" environment and not a mock-up which
> might not have all the ideosynchracies of the "real thing".
Jim, I agree there should be integration tests that run against the
hdfs - that can run on hudon. However there should be unit test that
test code units of hbase not of hadoop. Those should not use the dfs
since the idea of unit tests is not test underlaying third party
systems.
Anyhow just a user feedback the test suite as it is today is not
useful for me personal, since first, it never passed on my box and
second it takes 30 min to run - so I never run it - what is not the
idea of having a test suite and never run it during development.
Just some feedback...
Stefan
RE: test suite
Posted by Jim Kellerman <ji...@powerset.com>.
Since HBase is targeted to run on hdfs and not the local file system,
we want to test it in its "native" environment and not a mock-up which
might not have all the ideosynchracies of the "real thing".
---
Jim Kellerman, Senior Engineer; Powerset
> -----Original Message-----
> From: Stefan Groschupf [mailto:sg@101tec.com]
> Sent: Thursday, March 06, 2008 10:37 PM
> To: hbase-dev@hadoop.apache.org
> Subject: Re: test suite
>
> I'm sorry if I ask stupid questions, I'm very new to HBase.
> Waiting 30 min to see if a change breaks something or not is painful.
> I understand why you want to use a hadoop minicluster for
> integration tests but for normal unit tests.
> I would expect that there is an interface for each hbase
> component that the server and the clients implements. Similar
> to the hadoop jobrunner and localJobrunner. Since both
> components implement this interface it is transparent for the
> test if we use a distributed mini cluster or a local non tcp
> ip implementatipon. Also we could have mocks implementing
> this interface.
> So which puzzle peace I miss that there is the requirement to
> start up this minicluster every time?
>
>
>
> On Mar 6, 2008, at 8:24 PM, stack wrote:
>
> > Stefan Groschupf wrote:
> >> Is there an hudson that build hbase?
> >> Hudson should run the integration test suite with mini cluster and
> >> dfs where ant test for the user should only run local fs and local
> >> map reduce jobrunner.
> > Yeah. We have our own little zone on hudson:
> http://hudson.zones.apache.org/hudson/job/HBase-Patch/
> > . Needs some work. Currently just runs a build on checkin.
> > Would like to add the patch testing that happens in hadoop core.
> >
> > Starting dfs cluster from ant script is a little awkward.
> > MiniDFSCluster is nice because it chooses unoccupied ports;
> important
> > on a populated build box. Would need to then pass the
> chosen dfs port
> > then to tests and mini dfs would have to be in current process if
> > mindfs so can shut it down when done. I tried running in
> ant context
> > as distinct task and as wrapper around junit task but ran
> into class
> > loading issues.
> >
> > Could call out to a script that starts minidfs but need to
> do a little
> > dev so can send it a shutdown when done. Was not
> enthusiastic since
> > replicating the 'real' dfs start/stop scripts.
> > Alternative would be to start a 'real' dfs using fixed ports. This
> > would involve ant exec'ing. Would need to deal with *nix
> and windows
> > paths to make sure it worked on both. If I'm not mistaken, even
> > 'local' mode requires ssh'ing to start datanodes even if on
> localhost.
> >
> > Maybe the best thing would be to just punt on trying to have the
> > start/stop of the dfs be run from ant and just do it as part of the
> > build scripts we run up on the hudson build box?
> >
> > St.Ack
> >
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 101tec Inc.
> Menlo Park, California, USA
> http://www.101tec.com
>
>
>
> No virus found in this incoming message.
> Checked by AVG.
> Version: 7.5.518 / Virus Database: 269.21.6/1318 - Release
> Date: 3/7/2008 2:01 PM
>
>
No virus found in this outgoing message.
Checked by AVG.
Version: 7.5.518 / Virus Database: 269.21.6/1318 - Release Date: 3/7/2008 2:01 PM
Re: test suite
Posted by Stefan Groschupf <sg...@101tec.com>.
I'm sorry if I ask stupid questions, I'm very new to HBase.
Waiting 30 min to see if a change breaks something or not is painful.
I understand why you want to use a hadoop minicluster for integration
tests but for normal unit tests.
I would expect that there is an interface for each hbase component
that the server and the clients implements. Similar to the hadoop
jobrunner and localJobrunner. Since both components implement this
interface it is transparent for the test if we use a distributed mini
cluster or a local non tcp ip implementatipon. Also we could have
mocks implementing this interface.
So which puzzle peace I miss that there is the requirement to start up
this minicluster every time?
On Mar 6, 2008, at 8:24 PM, stack wrote:
> Stefan Groschupf wrote:
>> Is there an hudson that build hbase?
>> Hudson should run the integration test suite with mini cluster and
>> dfs where ant test for the user should only run local fs and local
>> map reduce jobrunner.
> Yeah. We have our own little zone on hudson: http://hudson.zones.apache.org/hudson/job/HBase-Patch/
> . Needs some work. Currently just runs a build on checkin.
> Would like to add the patch testing that happens in hadoop core.
>
> Starting dfs cluster from ant script is a little awkward.
> MiniDFSCluster is nice because it chooses unoccupied ports;
> important on a populated build box. Would need to then pass the
> chosen dfs port then to tests and mini dfs would have to be in
> current process if mindfs so can shut it down when done. I tried
> running in ant context as distinct task and as wrapper around junit
> task but ran into class loading issues.
>
> Could call out to a script that starts minidfs but need to do a
> little dev so can send it a shutdown when done. Was not
> enthusiastic since replicating the 'real' dfs start/stop scripts.
> Alternative would be to start a 'real' dfs using fixed ports. This
> would involve ant exec'ing. Would need to deal with *nix and
> windows paths to make sure it worked on both. If I'm not mistaken,
> even 'local' mode requires ssh'ing to start datanodes even if on
> localhost.
>
> Maybe the best thing would be to just punt on trying to have the
> start/stop of the dfs be run from ant and just do it as part of the
> build scripts we run up on the hudson build box?
>
> St.Ack
>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
101tec Inc.
Menlo Park, California, USA
http://www.101tec.com
Re: test suite
Posted by stack <st...@duboce.net>.
Stefan Groschupf wrote:
> Is there an hudson that build hbase?
> Hudson should run the integration test suite with mini cluster and dfs
> where ant test for the user should only run local fs and local map
> reduce jobrunner.
Yeah. We have our own little zone on hudson:
http://hudson.zones.apache.org/hudson/job/HBase-Patch/. Needs some
work. Currently just runs a build on checkin. Would like to add the
patch testing that happens in hadoop core.
Starting dfs cluster from ant script is a little awkward.
MiniDFSCluster is nice because it chooses unoccupied ports; important on
a populated build box. Would need to then pass the chosen dfs port then
to tests and mini dfs would have to be in current process if mindfs so
can shut it down when done. I tried running in ant context as distinct
task and as wrapper around junit task but ran into class loading issues.
Could call out to a script that starts minidfs but need to do a little
dev so can send it a shutdown when done. Was not enthusiastic since
replicating the 'real' dfs start/stop scripts. Alternative would be to
start a 'real' dfs using fixed ports. This would involve ant exec'ing.
Would need to deal with *nix and windows paths to make sure it worked on
both. If I'm not mistaken, even 'local' mode requires ssh'ing to start
datanodes even if on localhost.
Maybe the best thing would be to just punt on trying to have the
start/stop of the dfs be run from ant and just do it as part of the
build scripts we run up on the hudson build box?
St.Ack
Re: test suite
Posted by Stefan Groschupf <sg...@101tec.com>.
Is there an hudson that build hbase?
Hudson should run the integration test suite with mini cluster and dfs
where ant test for the user should only run local fs and local map
reduce jobrunner.
On Mar 6, 2008, at 5:23 PM, Bryan Duxbury wrote:
> I also vote for the local file system. However, I understand that
> there are some issues with local filesystem implementation on
> Windows, and there are some bugs that will only manifest when run on
> hdfs, mini or otherwise.
>
> I think the best middle ground is expecting an externally configured
> dfs, local or otherwise, rather than spinning one up for every test.
> This would require an extra step on the part of developers running
> the test suite. Do you think that would make it notably more
> difficult for new developers to get the test suite going?
>
> Whatever we end up doing regarding filesystem spin up, my testing
> shows that save us maybe a minute over the entire suite. It's
> certainly still a worthwhile task to pursue, but won't be a magic
> bullet.
>
> -Bryan
>
> On Mar 6, 2008, at 5:11 PM, Stefan Groschupf wrote:
>
>> jmock2 is pretty useful, I agree.
>> Also we in pig I submitted a patch that uses the hadoop
>> localjobrunner (jobtracker=local) - significant faster than the
>> minicluster.
>> Also in case hbase uses hadoop configuration why not just using the
>> local file system?
>>
>> On Mar 6, 2008, at 4:45 PM, Bryan Duxbury wrote:
>>
>>> I would add that we have an issue open for speeding up the test
>>> suite, HBASE-410. Probably the most promising idea for speeding up
>>> the test suite is converting some of the tests that don't need to
>>> be integration tests into something more like true unit tests, or
>>> specs for that matter. Mock objects used to test hyper-focused
>>> bits of the system would be absolutely ideal, and has the
>>> advantage of being quite fast.
>>>
>>> If you wanted to make some early contributions, anything in this
>>> area would be a FANTASTIC place to put them.
>>>
>>> -Bryan
>>>
>>> On Mar 6, 2008, at 4:28 PM, stack wrote:
>>>
>>>> Stefan Groschupf wrote:
>>>>> Hi there,
>>>>> I checked out latest svn and did run the test.
>>>>>
>>>>> The test suite failed at TestTable, are there known problems
>>>>> today?
>>>>
>>>> Supposed to work. Usually does. New hadoop was checked in last
>>>> night. That might have been responsible. Breaks everytime? If
>>>> so, open JIRA.
>>>>
>>>>> Also the test suite took 24 min to run on my little poor box -
>>>>> is there any way to speed this up?
>>>>
>>>> Yeah. Its a real prob. Test suite starts up little mini dfs's
>>>> and mini hbase clusters and sometimes mini mr clusters. I've
>>>> spent time on figuring how to do weird things like start the dfs
>>>> cluster inside of an ant task but ran into classloading issues
>>>> (joy). Needs to be fixed.
>>>>
>>>> (Our 'unit tests' are more like 'integration tests' -- one
>>>> approache would be to call them so and move them out of the unit
>>>> test suite).
>>>>
>>>> St.Ack
>>>>
>>>>> Thanks.
>>>>> Stefan
>>>>>
>>>>>
>>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>> 101tec Inc.
>>>>> Menlo Park, California, USA
>>>>> http://www.101tec.com
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> 101tec Inc.
>> Menlo Park, California, USA
>> http://www.101tec.com
>>
>>
>
>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
101tec Inc.
Menlo Park, California, USA
http://www.101tec.com
Re: test suite
Posted by Bryan Duxbury <br...@rapleaf.com>.
I also vote for the local file system. However, I understand that
there are some issues with local filesystem implementation on
Windows, and there are some bugs that will only manifest when run on
hdfs, mini or otherwise.
I think the best middle ground is expecting an externally configured
dfs, local or otherwise, rather than spinning one up for every test.
This would require an extra step on the part of developers running
the test suite. Do you think that would make it notably more
difficult for new developers to get the test suite going?
Whatever we end up doing regarding filesystem spin up, my testing
shows that save us maybe a minute over the entire suite. It's
certainly still a worthwhile task to pursue, but won't be a magic
bullet.
-Bryan
On Mar 6, 2008, at 5:11 PM, Stefan Groschupf wrote:
> jmock2 is pretty useful, I agree.
> Also we in pig I submitted a patch that uses the hadoop
> localjobrunner (jobtracker=local) - significant faster than the
> minicluster.
> Also in case hbase uses hadoop configuration why not just using the
> local file system?
>
> On Mar 6, 2008, at 4:45 PM, Bryan Duxbury wrote:
>
>> I would add that we have an issue open for speeding up the test
>> suite, HBASE-410. Probably the most promising idea for speeding up
>> the test suite is converting some of the tests that don't need to
>> be integration tests into something more like true unit tests, or
>> specs for that matter. Mock objects used to test hyper-focused
>> bits of the system would be absolutely ideal, and has the
>> advantage of being quite fast.
>>
>> If you wanted to make some early contributions, anything in this
>> area would be a FANTASTIC place to put them.
>>
>> -Bryan
>>
>> On Mar 6, 2008, at 4:28 PM, stack wrote:
>>
>>> Stefan Groschupf wrote:
>>>> Hi there,
>>>> I checked out latest svn and did run the test.
>>>>
>>>> The test suite failed at TestTable, are there known problems today?
>>>
>>> Supposed to work. Usually does. New hadoop was checked in last
>>> night. That might have been responsible. Breaks everytime? If
>>> so, open JIRA.
>>>
>>>> Also the test suite took 24 min to run on my little poor box -
>>>> is there any way to speed this up?
>>>
>>> Yeah. Its a real prob. Test suite starts up little mini dfs's
>>> and mini hbase clusters and sometimes mini mr clusters. I've
>>> spent time on figuring how to do weird things like start the dfs
>>> cluster inside of an ant task but ran into classloading issues
>>> (joy). Needs to be fixed.
>>>
>>> (Our 'unit tests' are more like 'integration tests' -- one
>>> approache would be to call them so and move them out of the unit
>>> test suite).
>>>
>>> St.Ack
>>>
>>>> Thanks.
>>>> Stefan
>>>>
>>>>
>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>> 101tec Inc.
>>>> Menlo Park, California, USA
>>>> http://www.101tec.com
>>>>
>>>>
>>>
>>
>>
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 101tec Inc.
> Menlo Park, California, USA
> http://www.101tec.com
>
>
Re: test suite
Posted by Stefan Groschupf <sg...@101tec.com>.
jmock2 is pretty useful, I agree.
Also we in pig I submitted a patch that uses the hadoop localjobrunner
(jobtracker=local) - significant faster than the minicluster.
Also in case hbase uses hadoop configuration why not just using the
local file system?
On Mar 6, 2008, at 4:45 PM, Bryan Duxbury wrote:
> I would add that we have an issue open for speeding up the test
> suite, HBASE-410. Probably the most promising idea for speeding up
> the test suite is converting some of the tests that don't need to be
> integration tests into something more like true unit tests, or specs
> for that matter. Mock objects used to test hyper-focused bits of the
> system would be absolutely ideal, and has the advantage of being
> quite fast.
>
> If you wanted to make some early contributions, anything in this
> area would be a FANTASTIC place to put them.
>
> -Bryan
>
> On Mar 6, 2008, at 4:28 PM, stack wrote:
>
>> Stefan Groschupf wrote:
>>> Hi there,
>>> I checked out latest svn and did run the test.
>>>
>>> The test suite failed at TestTable, are there known problems today?
>>
>> Supposed to work. Usually does. New hadoop was checked in last
>> night. That might have been responsible. Breaks everytime? If
>> so, open JIRA.
>>
>>> Also the test suite took 24 min to run on my little poor box - is
>>> there any way to speed this up?
>>
>> Yeah. Its a real prob. Test suite starts up little mini dfs's and
>> mini hbase clusters and sometimes mini mr clusters. I've spent
>> time on figuring how to do weird things like start the dfs cluster
>> inside of an ant task but ran into classloading issues (joy).
>> Needs to be fixed.
>>
>> (Our 'unit tests' are more like 'integration tests' -- one
>> approache would be to call them so and move them out of the unit
>> test suite).
>>
>> St.Ack
>>
>>> Thanks.
>>> Stefan
>>>
>>>
>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>> 101tec Inc.
>>> Menlo Park, California, USA
>>> http://www.101tec.com
>>>
>>>
>>
>
>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
101tec Inc.
Menlo Park, California, USA
http://www.101tec.com
Re: test suite
Posted by Bryan Duxbury <br...@rapleaf.com>.
I would add that we have an issue open for speeding up the test
suite, HBASE-410. Probably the most promising idea for speeding up
the test suite is converting some of the tests that don't need to be
integration tests into something more like true unit tests, or specs
for that matter. Mock objects used to test hyper-focused bits of the
system would be absolutely ideal, and has the advantage of being
quite fast.
If you wanted to make some early contributions, anything in this area
would be a FANTASTIC place to put them.
-Bryan
On Mar 6, 2008, at 4:28 PM, stack wrote:
> Stefan Groschupf wrote:
>> Hi there,
>> I checked out latest svn and did run the test.
>>
>> The test suite failed at TestTable, are there known problems today?
>
> Supposed to work. Usually does. New hadoop was checked in last
> night. That might have been responsible. Breaks everytime? If
> so, open JIRA.
>
>> Also the test suite took 24 min to run on my little poor box - is
>> there any way to speed this up?
>
> Yeah. Its a real prob. Test suite starts up little mini dfs's and
> mini hbase clusters and sometimes mini mr clusters. I've spent
> time on figuring how to do weird things like start the dfs cluster
> inside of an ant task but ran into classloading issues (joy).
> Needs to be fixed.
>
> (Our 'unit tests' are more like 'integration tests' -- one
> approache would be to call them so and move them out of the unit
> test suite).
>
> St.Ack
>
>> Thanks.
>> Stefan
>>
>>
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> 101tec Inc.
>> Menlo Park, California, USA
>> http://www.101tec.com
>>
>>
>
Re: test suite
Posted by stack <st...@duboce.net>.
Stefan Groschupf wrote:
> Hi there,
> I checked out latest svn and did run the test.
>
> The test suite failed at TestTable, are there known problems today?
Supposed to work. Usually does. New hadoop was checked in last night.
That might have been responsible. Breaks everytime? If so, open JIRA.
> Also the test suite took 24 min to run on my little poor box - is
> there any way to speed this up?
Yeah. Its a real prob. Test suite starts up little mini dfs's and mini
hbase clusters and sometimes mini mr clusters. I've spent time on
figuring how to do weird things like start the dfs cluster inside of an
ant task but ran into classloading issues (joy). Needs to be fixed.
(Our 'unit tests' are more like 'integration tests' -- one approache
would be to call them so and move them out of the unit test suite).
St.Ack
> Thanks.
> Stefan
>
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 101tec Inc.
> Menlo Park, California, USA
> http://www.101tec.com
>
>