You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gora.apache.org by Henry Saputra <he...@gmail.com> on 2012/01/29 18:49:58 UTC
Is the total new build with unit test takes about 40 mins?
Hi Guys,
Is it just me or the new build with Maven with unit test takes about 40 minutes?
I am running with MacbookPro 2.5GHz Intel Core Duo and 4GB memory
- Henry
Re: Is the total new build with unit test takes about 40 mins?
Posted by Henry Saputra <he...@gmail.com>.
Ah cool, I will leave it to you then since you already started with the
implementation =)
Thanks Lewis.
- Henry
On Wed, Nov 28, 2012 at 10:30 AM, Lewis John Mcgibbney <
lewis.mcgibbney@gmail.com> wrote:
> Hi Henry,
>
> Yes I did, I also began a branch on my github repos a while back to
> replace the 30 mins code with some newer, cleaner HBase testing
> implementations. E.g. maintain our own HBase cluster and call it once
> as a singleton then keep the cluster up for all HBase tests.
>
> Here is the Jira issue I filed.
>
> https://issues.apache.org/jira/browse/GORA-89
>
> On Wed, Nov 28, 2012 at 4:54 PM, Henry Saputra <he...@gmail.com>
> wrote:
> > HI Lewis,
> >
> > Sorry to bring back old thread but did you get chance to file JIRA for
> > this? I am trying to see if I could try to fix this.
> >
> >
> > Thanks,
> >
> > - Henry
> >
> >
> > On Mon, Jan 30, 2012 at 6:58 AM, Lewis John Mcgibbney <
> > lewis.mcgibbney@gmail.com> wrote:
> >
> >> Hi Henry,
> >>
> >> Aye the trunk build currently takes around this time. Me an Ioannis had
> a
> >> look at it a while ago and I think we came to the conclusion that it was
> >> the HBase module that was taking ages. I think this has something to do
> >> with a new HBase clutser being created after every test, this takes
> time.
> >> It would be great to reduce this time though as it seems a somewhat
> lengthy
> >> amount of time doesn't it.
> >>
> >> Lewis
> >>
> >> On Sun, Jan 29, 2012 at 5:49 PM, Henry Saputra <henry.saputra@gmail.com
> >> >wrote:
> >>
> >> > Hi Guys,
> >> >
> >> > Is it just me or the new build with Maven with unit test takes about
> 40
> >> > minutes?
> >> >
> >> > I am running with MacbookPro 2.5GHz Intel Core Duo and 4GB memory
> >> >
> >> > - Henry
> >> >
> >>
> >>
> >>
> >> --
> >> *Lewis*
> >>
>
>
>
> --
> Lewis
>
Re: Is the total new build with unit test takes about 40 mins?
Posted by Lewis John Mcgibbney <le...@gmail.com>.
Hi Henry,
Yes I did, I also began a branch on my github repos a while back to
replace the 30 mins code with some newer, cleaner HBase testing
implementations. E.g. maintain our own HBase cluster and call it once
as a singleton then keep the cluster up for all HBase tests.
Here is the Jira issue I filed.
https://issues.apache.org/jira/browse/GORA-89
On Wed, Nov 28, 2012 at 4:54 PM, Henry Saputra <he...@gmail.com> wrote:
> HI Lewis,
>
> Sorry to bring back old thread but did you get chance to file JIRA for
> this? I am trying to see if I could try to fix this.
>
>
> Thanks,
>
> - Henry
>
>
> On Mon, Jan 30, 2012 at 6:58 AM, Lewis John Mcgibbney <
> lewis.mcgibbney@gmail.com> wrote:
>
>> Hi Henry,
>>
>> Aye the trunk build currently takes around this time. Me an Ioannis had a
>> look at it a while ago and I think we came to the conclusion that it was
>> the HBase module that was taking ages. I think this has something to do
>> with a new HBase clutser being created after every test, this takes time.
>> It would be great to reduce this time though as it seems a somewhat lengthy
>> amount of time doesn't it.
>>
>> Lewis
>>
>> On Sun, Jan 29, 2012 at 5:49 PM, Henry Saputra <henry.saputra@gmail.com
>> >wrote:
>>
>> > Hi Guys,
>> >
>> > Is it just me or the new build with Maven with unit test takes about 40
>> > minutes?
>> >
>> > I am running with MacbookPro 2.5GHz Intel Core Duo and 4GB memory
>> >
>> > - Henry
>> >
>>
>>
>>
>> --
>> *Lewis*
>>
--
Lewis
Re: Is the total new build with unit test takes about 40 mins?
Posted by Henry Saputra <he...@gmail.com>.
HI Lewis,
Sorry to bring back old thread but did you get chance to file JIRA for
this? I am trying to see if I could try to fix this.
Thanks,
- Henry
On Mon, Jan 30, 2012 at 6:58 AM, Lewis John Mcgibbney <
lewis.mcgibbney@gmail.com> wrote:
> Hi Henry,
>
> Aye the trunk build currently takes around this time. Me an Ioannis had a
> look at it a while ago and I think we came to the conclusion that it was
> the HBase module that was taking ages. I think this has something to do
> with a new HBase clutser being created after every test, this takes time.
> It would be great to reduce this time though as it seems a somewhat lengthy
> amount of time doesn't it.
>
> Lewis
>
> On Sun, Jan 29, 2012 at 5:49 PM, Henry Saputra <henry.saputra@gmail.com
> >wrote:
>
> > Hi Guys,
> >
> > Is it just me or the new build with Maven with unit test takes about 40
> > minutes?
> >
> > I am running with MacbookPro 2.5GHz Intel Core Duo and 4GB memory
> >
> > - Henry
> >
>
>
>
> --
> *Lewis*
>
Re: Is the total new build with unit test takes about 40 mins?
Posted by Lewis John Mcgibbney <le...@gmail.com>.
Thanks for the insight into your work Ioan and of course for the kind words.
As a community, we've a clear road path for Gora as a TLP entity within ASF
so heres to some good development for the future.
Thanks for now
Lewis
On Mon, Jan 30, 2012 at 3:39 PM, Ioan Eugen Stan <st...@gmail.com>wrote:
> Pe 30.01.2012 17:27, Lewis John Mcgibbney a scris:
>
> Yeah this is something I was going to contact you about, but as you said
>> maybe some time in the future.
>>
>> I've opened a ticket for the HBase timing issue.
>>
>> Great to hear from you... What brings you over to Goraland? Do you have
>> some problems or use cases you can share with us? I'm guessing it's HBase
>> specific if there is one.
>>
>> Thanks
>>
>> Lewis
>>
>>
> I'm a long time Goraland troller. I've considered using it as a data
> access layer for my GSoC project - HBase mailbox implementation for Apache
> James. It didn't look good back then (Gora was incubating and unstable API,
> I didn't know James, HBase) and I chose to stay with plain HBase API
> because it was simple and avoided the trouble of another layer of
> indirection. One that I couldn't have managed.
>
> I've stayed on the list from then on and I read some messages from time to
> time. Sometimes I even reply ;).
>
> Congrats on becoming PMC for the project and good luck. I'll be keeping an
> eye on you(re messages) :).
>
>
> Cheers,
> --
> Ioan Eugen Stan
> http://ieugen.blogspot.com
>
--
*Lewis*
Re: Is the total new build with unit test takes about 40 mins?
Posted by Ioan Eugen Stan <st...@gmail.com>.
Pe 30.01.2012 17:27, Lewis John Mcgibbney a scris:
> Yeah this is something I was going to contact you about, but as you said
> maybe some time in the future.
>
> I've opened a ticket for the HBase timing issue.
>
> Great to hear from you... What brings you over to Goraland? Do you have
> some problems or use cases you can share with us? I'm guessing it's HBase
> specific if there is one.
>
> Thanks
>
> Lewis
>
I'm a long time Goraland troller. I've considered using it as a data
access layer for my GSoC project - HBase mailbox implementation for
Apache James. It didn't look good back then (Gora was incubating and
unstable API, I didn't know James, HBase) and I chose to stay with plain
HBase API because it was simple and avoided the trouble of another layer
of indirection. One that I couldn't have managed.
I've stayed on the list from then on and I read some messages from time
to time. Sometimes I even reply ;).
Congrats on becoming PMC for the project and good luck. I'll be keeping
an eye on you(re messages) :).
Cheers,
--
Ioan Eugen Stan
http://ieugen.blogspot.com
Re: Is the total new build with unit test takes about 40 mins?
Posted by Lewis John Mcgibbney <le...@gmail.com>.
Yeah this is something I was going to contact you about, but as you said
maybe some time in the future.
I've opened a ticket for the HBase timing issue.
Great to hear from you... What brings you over to Goraland? Do you have
some problems or use cases you can share with us? I'm guessing it's HBase
specific if there is one.
Thanks
Lewis
On Mon, Jan 30, 2012 at 3:23 PM, Ioan Eugen Stan <st...@gmail.com>wrote:
> Pe 30.01.2012 17:20, Lewis John Mcgibbney a scris:
>
> Hi Ioan,
>>
>> Excellent, thanks for the heads up.
>>
>> There is some work to be done at the moment e.g. site migration etc but
>> however hopefully we can get round to this shortly. I'll open an issue on
>> Jira to remind us.
>>
>> Thanks
>>
>> Lewis
>>
>>
> No problem. I also didn't forget about Debian packaging, but I didn't have
> time for this. Maybe in the near future.
>
> Cheers,
>
> --
> Ioan Eugen Stan
> http://ieugen.blogspot.com
>
--
*Lewis*
Re: Is the total new build with unit test takes about 40 mins?
Posted by Ioan Eugen Stan <st...@gmail.com>.
Pe 30.01.2012 17:20, Lewis John Mcgibbney a scris:
> Hi Ioan,
>
> Excellent, thanks for the heads up.
>
> There is some work to be done at the moment e.g. site migration etc but
> however hopefully we can get round to this shortly. I'll open an issue on
> Jira to remind us.
>
> Thanks
>
> Lewis
>
No problem. I also didn't forget about Debian packaging, but I didn't
have time for this. Maybe in the near future.
Cheers,
--
Ioan Eugen Stan
http://ieugen.blogspot.com
Re: Is the total new build with unit test takes about 40 mins?
Posted by Lewis John Mcgibbney <le...@gmail.com>.
Hi Ioan,
Excellent, thanks for the heads up.
There is some work to be done at the moment e.g. site migration etc but
however hopefully we can get round to this shortly. I'll open an issue on
Jira to remind us.
Thanks
Lewis
On Mon, Jan 30, 2012 at 3:05 PM, Ioan Eugen Stan <st...@gmail.com>wrote:
> 2012/1/30 Lewis John Mcgibbney <le...@gmail.com>:
> > Hi Henry,
> >
> > Aye the trunk build currently takes around this time. Me an Ioannis had a
> > look at it a while ago and I think we came to the conclusion that it was
> > the HBase module that was taking ages. I think this has something to do
> > with a new HBase clutser being created after every test, this takes time.
> > It would be great to reduce this time though as it seems a somewhat
> lengthy
> > amount of time doesn't it.
> >
> > Lewis
>
> Hello Lewis,
>
> I think you can shorten the time by avoiding MiniCluster restarts.
> Just implement the cluster as a singleton and clean up the tables in
> between test by doing a scan and deletes for all rows. It's much
> faster than restarting the cluster.
>
> For code referenece please see my implementation here[1]. The class is
> HBaseClusterSingleton. It needs some refactoring but I think it's
> enough to speed your tests.
>
> If you have trouble while implementing I can take a look.
>
> Cheers,
>
> [1]
> http://svn.apache.org/repos/asf/james/mailbox/trunk/hbase/src/test/java/org/apache/james/mailbox/hbase/
> --
> Ioan Eugen Stan
> http://ieugen.blogspot.com/
>
--
*Lewis*
Re: Is the total new build with unit test takes about 40 mins?
Posted by Ioan Eugen Stan <st...@gmail.com>.
2012/1/30 Lewis John Mcgibbney <le...@gmail.com>:
> Hi Henry,
>
> Aye the trunk build currently takes around this time. Me an Ioannis had a
> look at it a while ago and I think we came to the conclusion that it was
> the HBase module that was taking ages. I think this has something to do
> with a new HBase clutser being created after every test, this takes time.
> It would be great to reduce this time though as it seems a somewhat lengthy
> amount of time doesn't it.
>
> Lewis
Hello Lewis,
I think you can shorten the time by avoiding MiniCluster restarts.
Just implement the cluster as a singleton and clean up the tables in
between test by doing a scan and deletes for all rows. It's much
faster than restarting the cluster.
For code referenece please see my implementation here[1]. The class is
HBaseClusterSingleton. It needs some refactoring but I think it's
enough to speed your tests.
If you have trouble while implementing I can take a look.
Cheers,
[1] http://svn.apache.org/repos/asf/james/mailbox/trunk/hbase/src/test/java/org/apache/james/mailbox/hbase/
--
Ioan Eugen Stan
http://ieugen.blogspot.com/
Re: Is the total new build with unit test takes about 40 mins?
Posted by Lewis John Mcgibbney <le...@gmail.com>.
Hi Henry,
Aye the trunk build currently takes around this time. Me an Ioannis had a
look at it a while ago and I think we came to the conclusion that it was
the HBase module that was taking ages. I think this has something to do
with a new HBase clutser being created after every test, this takes time.
It would be great to reduce this time though as it seems a somewhat lengthy
amount of time doesn't it.
Lewis
On Sun, Jan 29, 2012 at 5:49 PM, Henry Saputra <he...@gmail.com>wrote:
> Hi Guys,
>
> Is it just me or the new build with Maven with unit test takes about 40
> minutes?
>
> I am running with MacbookPro 2.5GHz Intel Core Duo and 4GB memory
>
> - Henry
>
--
*Lewis*