You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by stack <st...@duboce.net> on 2010/01/08 05:30:11 UTC
OK with all that build moves to use ivy resolving dependencies?
Unless there is objection, we'll commit hbase-1433 in the next day or so.
This contribution by Kay Kay adds ivy (http://ant.apache.org/ivy/) to our
build for resolving dependencies. Whereas previous all dependencies were
checked-in under $HBASE_HOME/lib, instead, ivy will pull from remote
internet repositories all dependencies listed/needed by the hbase project on
ant build (It keeps a local cache so pulls a dependency once-only). The
move to ivy realigns our build system with that of our parent hadoop -- it
uses ivy resolving dependencies. Ivy also has strong affinity to Maven and
will make it easier for us publishing hbase builds up to a Maven repository,
e.g. the Apache Maven repository (Our parent project currently does this).
This ok with all?
Thanks,
St.Ack
Re: OK with all that build moves to use ivy resolving dependencies?
Posted by Kay Kay <ka...@gmail.com>.
On 1/11/10 11:08 AM, Andrew Purtell wrote:
> Kay Kay's contributed effort and attention to detail is much appreciated.
>
> - Andy
>
>
Thanks everyone for accepting this. Hope this makes the integration with
other dependent projects a lot more smoother to keep up and sets way for
the publication of mvn artifacts of hbase eventually.
>
> ----- Original Message ----
>
>> From: stack<st...@duboce.net>
>> To: hbase-dev@hadoop.apache.org
>> Sent: Sun, January 10, 2010 4:57:40 PM
>> Subject: Re: OK with all that build moves to use ivy resolving dependencies?
>>
>> I committed hbase-1443. HBase TRUNK build is now ivy based. On update,
>> you'll notice a leaner lib directory and on build, hbase (ivy) will go to
>> the net to pull in hbase dependencies. Kay Kay has written up a useful Ivy
>> Primer, http://wiki.apache.org/hadoop/Hbase/IvyPrimer, that you all might
>> want to take a look at. Stick any questions here.
>>
>> Thanks,
>> St.Ack
>>
>
>
>
>
>
Re: OK with all that build moves to use ivy resolving dependencies?
Posted by Andrew Purtell <ap...@apache.org>.
Kay Kay's contributed effort and attention to detail is much appreciated.
- Andy
----- Original Message ----
> From: stack <st...@duboce.net>
> To: hbase-dev@hadoop.apache.org
> Sent: Sun, January 10, 2010 4:57:40 PM
> Subject: Re: OK with all that build moves to use ivy resolving dependencies?
>
> I committed hbase-1443. HBase TRUNK build is now ivy based. On update,
> you'll notice a leaner lib directory and on build, hbase (ivy) will go to
> the net to pull in hbase dependencies. Kay Kay has written up a useful Ivy
> Primer, http://wiki.apache.org/hadoop/Hbase/IvyPrimer, that you all might
> want to take a look at. Stick any questions here.
>
> Thanks,
> St.Ack
Re: OK with all that build moves to use ivy resolving dependencies?
Posted by Ryan Rawson <ry...@gmail.com>.
Thanks for documenting... sorry for being a jerk.
On Jan 10, 2010 4:58 PM, "stack" <st...@duboce.net> wrote:
I committed hbase-1443. HBase TRUNK build is now ivy based. On update,
you'll notice a leaner lib directory and on build, hbase (ivy) will go to
the net to pull in hbase dependencies. Kay Kay has written up a useful Ivy
Primer, http://wiki.apache.org/hadoop/Hbase/IvyPrimer, that you all might
want to take a look at. Stick any questions here.
Thanks,
St.Ack
On Fri, Jan 8, 2010 at 1:33 PM, stack <st...@duboce.net> wrote: > Good on
you Ryan. > > I made hb...
Re: OK with all that build moves to use ivy resolving dependencies?
Posted by stack <st...@duboce.net>.
I committed hbase-1443. HBase TRUNK build is now ivy based. On update,
you'll notice a leaner lib directory and on build, hbase (ivy) will go to
the net to pull in hbase dependencies. Kay Kay has written up a useful Ivy
Primer, http://wiki.apache.org/hadoop/Hbase/IvyPrimer, that you all might
want to take a look at. Stick any questions here.
Thanks,
St.Ack
On Fri, Jan 8, 2010 at 1:33 PM, stack <st...@duboce.net> wrote:
> Good on you Ryan.
>
> I made hbase-2099 for discussion of moving hbase to maven (There is already
> HBASE-1403 on moving src/contribs out of hbase).
>
> Unless I get a -1 soon, I'll go ahead with the hbase-1433 commit.
>
> St.Ack
>
>
> On Thu, Jan 7, 2010 at 10:54 PM, Ryan Rawson <ry...@gmail.com> wrote:
>
>> I'm ok with the ivy thing, I just wanted to raise the non-disconnected
>> build issue, since moving to ivy is not 'no cost' as proponents would
>> like to paper over. I just wanted to make sure everyone realizes this
>> will hurt productivity of some people.
>>
>> On Thu, Jan 7, 2010 at 10:52 PM, stack <st...@duboce.net> wrote:
>> > On Thu, Jan 7, 2010 at 9:42 PM, Ryan Rawson <ry...@gmail.com> wrote:
>> >
>> >> So given all that, I'd rather do the whole hog, rather than the half
>> >> and half thing. Ie: maven over ivy.
>> >>
>> >
>> >
>> > I'm not opposed to a move to Maven. I've not really given it much
>> thought,
>> > but I think that a separate discussion and undertaking. It would take
>> some
>> > work doing it properly. We would need to take on the maven layout at a
>> > minimum -- move all to src/main/java, etc. -- and we'd need to purge any
>> > deviation from the maven way because the alternative is hours burnt
>> > wandering in the weeds of poorly documented plugin xml configs., or
>> worse,
>> > hours writing custom maven plugins to pull Maven in alternate
>> directions.
>> > For one, our notion of contrib (src/contrib/*), IIRC, does not map to
>> > maven's notion of subprojects. Its been a while but with Maven
>> subprojects
>> > notion, you could not without backflips have the parent project build
>> its
>> > jar and then have subprojects depend on parent.
>> >
>> > If someone wants to take on the Maven work, well and good but for me the
>> Ivy
>> > work is done. Lets commit it. The way it does its dependencies is
>> > Maven-like (you list them in pom for maven, in properties for ivy; both
>> pull
>> > to local caches, etc.). Committing Ivy gets us working with external
>> > repositories, pulling and publishing. So, the ivy commit takes us some
>> of
>> > the ways toward a mavenized hbase while meantime, making hbase build
>> like
>> > its hosting project and its siblings.
>> >
>> > St.Ack
>> >
>>
>
>
Re: OK with all that build moves to use ivy resolving dependencies?
Posted by stack <st...@duboce.net>.
Good on you Ryan.
I made hbase-2099 for discussion of moving hbase to maven (There is already
HBASE-1403 on moving src/contribs out of hbase).
Unless I get a -1 soon, I'll go ahead with the hbase-1433 commit.
St.Ack
On Thu, Jan 7, 2010 at 10:54 PM, Ryan Rawson <ry...@gmail.com> wrote:
> I'm ok with the ivy thing, I just wanted to raise the non-disconnected
> build issue, since moving to ivy is not 'no cost' as proponents would
> like to paper over. I just wanted to make sure everyone realizes this
> will hurt productivity of some people.
>
> On Thu, Jan 7, 2010 at 10:52 PM, stack <st...@duboce.net> wrote:
> > On Thu, Jan 7, 2010 at 9:42 PM, Ryan Rawson <ry...@gmail.com> wrote:
> >
> >> So given all that, I'd rather do the whole hog, rather than the half
> >> and half thing. Ie: maven over ivy.
> >>
> >
> >
> > I'm not opposed to a move to Maven. I've not really given it much
> thought,
> > but I think that a separate discussion and undertaking. It would take
> some
> > work doing it properly. We would need to take on the maven layout at a
> > minimum -- move all to src/main/java, etc. -- and we'd need to purge any
> > deviation from the maven way because the alternative is hours burnt
> > wandering in the weeds of poorly documented plugin xml configs., or
> worse,
> > hours writing custom maven plugins to pull Maven in alternate directions.
> > For one, our notion of contrib (src/contrib/*), IIRC, does not map to
> > maven's notion of subprojects. Its been a while but with Maven
> subprojects
> > notion, you could not without backflips have the parent project build its
> > jar and then have subprojects depend on parent.
> >
> > If someone wants to take on the Maven work, well and good but for me the
> Ivy
> > work is done. Lets commit it. The way it does its dependencies is
> > Maven-like (you list them in pom for maven, in properties for ivy; both
> pull
> > to local caches, etc.). Committing Ivy gets us working with external
> > repositories, pulling and publishing. So, the ivy commit takes us some
> of
> > the ways toward a mavenized hbase while meantime, making hbase build like
> > its hosting project and its siblings.
> >
> > St.Ack
> >
>
Re: OK with all that build moves to use ivy resolving dependencies?
Posted by Ryan Rawson <ry...@gmail.com>.
I'm ok with the ivy thing, I just wanted to raise the non-disconnected
build issue, since moving to ivy is not 'no cost' as proponents would
like to paper over. I just wanted to make sure everyone realizes this
will hurt productivity of some people.
On Thu, Jan 7, 2010 at 10:52 PM, stack <st...@duboce.net> wrote:
> On Thu, Jan 7, 2010 at 9:42 PM, Ryan Rawson <ry...@gmail.com> wrote:
>
>> So given all that, I'd rather do the whole hog, rather than the half
>> and half thing. Ie: maven over ivy.
>>
>
>
> I'm not opposed to a move to Maven. I've not really given it much thought,
> but I think that a separate discussion and undertaking. It would take some
> work doing it properly. We would need to take on the maven layout at a
> minimum -- move all to src/main/java, etc. -- and we'd need to purge any
> deviation from the maven way because the alternative is hours burnt
> wandering in the weeds of poorly documented plugin xml configs., or worse,
> hours writing custom maven plugins to pull Maven in alternate directions.
> For one, our notion of contrib (src/contrib/*), IIRC, does not map to
> maven's notion of subprojects. Its been a while but with Maven subprojects
> notion, you could not without backflips have the parent project build its
> jar and then have subprojects depend on parent.
>
> If someone wants to take on the Maven work, well and good but for me the Ivy
> work is done. Lets commit it. The way it does its dependencies is
> Maven-like (you list them in pom for maven, in properties for ivy; both pull
> to local caches, etc.). Committing Ivy gets us working with external
> repositories, pulling and publishing. So, the ivy commit takes us some of
> the ways toward a mavenized hbase while meantime, making hbase build like
> its hosting project and its siblings.
>
> St.Ack
>
Re: OK with all that build moves to use ivy resolving dependencies?
Posted by stack <st...@duboce.net>.
On Thu, Jan 7, 2010 at 9:42 PM, Ryan Rawson <ry...@gmail.com> wrote:
> So given all that, I'd rather do the whole hog, rather than the half
> and half thing. Ie: maven over ivy.
>
I'm not opposed to a move to Maven. I've not really given it much thought,
but I think that a separate discussion and undertaking. It would take some
work doing it properly. We would need to take on the maven layout at a
minimum -- move all to src/main/java, etc. -- and we'd need to purge any
deviation from the maven way because the alternative is hours burnt
wandering in the weeds of poorly documented plugin xml configs., or worse,
hours writing custom maven plugins to pull Maven in alternate directions.
For one, our notion of contrib (src/contrib/*), IIRC, does not map to
maven's notion of subprojects. Its been a while but with Maven subprojects
notion, you could not without backflips have the parent project build its
jar and then have subprojects depend on parent.
If someone wants to take on the Maven work, well and good but for me the Ivy
work is done. Lets commit it. The way it does its dependencies is
Maven-like (you list them in pom for maven, in properties for ivy; both pull
to local caches, etc.). Committing Ivy gets us working with external
repositories, pulling and publishing. So, the ivy commit takes us some of
the ways toward a mavenized hbase while meantime, making hbase build like
its hosting project and its siblings.
St.Ack
Re: OK with all that build moves to use ivy resolving dependencies?
Posted by Kay Kay <ka...@gmail.com>.
Ryan Rawson wrote:
> I'm not a fan of systems that increase complexity for no major wins.
>
I can bet , somebody else, bringing up dependency management in one form
or another, soon (well- this was already a jira issue, before I started
this thread ).
> After talking to some guys on #hbase, I think we'd be better off going
> to maven.
Can you be more specific, about some of the pros/ cons, that you think
is applicable to hbase.
> It's more complex, but you get more wins out of it. It
> sounds like the contributor just wants maven published artifacts, so
> it's not even 'ivy' as per se.
>
> So given all that, I'd rather do the whole hog, rather than the half
> and half thing. Ie: maven over ivy.
>
My original objective was to upgrade the Lucene version, for indexing
from 2.2 ( pretty old ) to 3.x , where I needed to use some of the
significant changes gone in 2.9 and some API sugar / performance
improvements in 3.0+ .
I was under the assumption of flipping a version in properties and
resubmitting a patch towards it, but later discovered the versioning of
dependencies (deleting + adding jars).
Chose ivy because it was used in core/mr/hdfs and thought that is the
standard dependency management tool in the hadoop (+ family) world.
I have nothing against maven, but chose ivy only because it offers the
best way to reuse investments in build.xml , made over a period of time
and a comparatively easier learning curve (compared to mvn). My $ 0.02 .
> -ryan
>
>
> On Thu, Jan 7, 2010 at 9:35 PM, Andrew Purtell <ap...@apache.org> wrote:
>
>> +0
>>
>>
>>> On Jan 7, 2010 8:30 PM, "stack" wrote:
>>>
>>> Unless there is objection, we'll commit hbase-1433 in the next day or so.
>>> This contribution by Kay Kay adds ivy (http://ant.apache.org/ivy/) to our
>>> build for resolving dependencies. Whereas previous all dependencies were
>>> checked-in under $HBASE_HOME/lib, instead, ivy will pull from remote
>>> internet repositories all dependencies listed/needed by the hbase project on
>>> ant build (It keeps a local cache so pulls a dependency once-only). The
>>> move to ivy realigns our build system with that of our parent hadoop -- it
>>> uses ivy resolving dependencies. Ivy also has strong affinity to Maven and
>>> will make it easier for us publishing hbase builds up to a Maven repository,
>>> e.g. the Apache Maven repository (Our parent project currently does this).
>>>
>>> This ok with all?
>>> Thanks,
>>> St.Ack
>>>
>>
>>
>>
>>
>>
>
>
Re: OK with all that build moves to use ivy resolving dependencies?
Posted by Ryan Rawson <ry...@gmail.com>.
I'm not a fan of systems that increase complexity for no major wins.
After talking to some guys on #hbase, I think we'd be better off going
to maven. It's more complex, but you get more wins out of it. It
sounds like the contributor just wants maven published artifacts, so
it's not even 'ivy' as per se.
So given all that, I'd rather do the whole hog, rather than the half
and half thing. Ie: maven over ivy.
-ryan
On Thu, Jan 7, 2010 at 9:35 PM, Andrew Purtell <ap...@apache.org> wrote:
> +0
>
>> On Jan 7, 2010 8:30 PM, "stack" wrote:
>
>>
>> Unless there is objection, we'll commit hbase-1433 in the next day or so.
>> This contribution by Kay Kay adds ivy (http://ant.apache.org/ivy/) to our
>> build for resolving dependencies. Whereas previous all dependencies were
>> checked-in under $HBASE_HOME/lib, instead, ivy will pull from remote
>> internet repositories all dependencies listed/needed by the hbase project on
>> ant build (It keeps a local cache so pulls a dependency once-only). The
>> move to ivy realigns our build system with that of our parent hadoop -- it
>> uses ivy resolving dependencies. Ivy also has strong affinity to Maven and
>> will make it easier for us publishing hbase builds up to a Maven repository,
>> e.g. the Apache Maven repository (Our parent project currently does this).
>>
>> This ok with all?
>> Thanks,
>> St.Ack
>
>
>
>
>
>
Re: OK with all that build moves to use ivy resolving dependencies?
Posted by Andrew Purtell <ap...@apache.org>.
+0
> On Jan 7, 2010 8:30 PM, "stack" wrote:
>
> Unless there is objection, we'll commit hbase-1433 in the next day or so.
> This contribution by Kay Kay adds ivy (http://ant.apache.org/ivy/) to our
> build for resolving dependencies. Whereas previous all dependencies were
> checked-in under $HBASE_HOME/lib, instead, ivy will pull from remote
> internet repositories all dependencies listed/needed by the hbase project on
> ant build (It keeps a local cache so pulls a dependency once-only). The
> move to ivy realigns our build system with that of our parent hadoop -- it
> uses ivy resolving dependencies. Ivy also has strong affinity to Maven and
> will make it easier for us publishing hbase builds up to a Maven repository,
> e.g. the Apache Maven repository (Our parent project currently does this).
>
> This ok with all?
> Thanks,
> St.Ack
Re: OK with all that build moves to use ivy resolving dependencies?
Posted by Jean-Daniel Cryans <jd...@apache.org>.
+1
J-D
On Jan 7, 2010 8:30 PM, "stack" <st...@duboce.net> wrote:
Unless there is objection, we'll commit hbase-1433 in the next day or so.
This contribution by Kay Kay adds ivy (http://ant.apache.org/ivy/) to our
build for resolving dependencies. Whereas previous all dependencies were
checked-in under $HBASE_HOME/lib, instead, ivy will pull from remote
internet repositories all dependencies listed/needed by the hbase project on
ant build (It keeps a local cache so pulls a dependency once-only). The
move to ivy realigns our build system with that of our parent hadoop -- it
uses ivy resolving dependencies. Ivy also has strong affinity to Maven and
will make it easier for us publishing hbase builds up to a Maven repository,
e.g. the Apache Maven repository (Our parent project currently does this).
This ok with all?
Thanks,
St.Ack