You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by Stepan Mishura <st...@gmail.com> on 2007/04/03 18:04:19 UTC

[general] Managing dependencies: Antlib for Maven 2.0, Ivy or Ant only (i.e. Harmony specific way)?

Hi all,

I'd like to hear opinions about choosing a way for managing library
dependencies.

Currently I'm reviewing proposal for improving build-and-test infra
[1][2]. One of declared features according to the proposal is:
  "Implemented Approach to share External Library Dependencies between different
   Test Suites allows to perform external dependencies fetching in one place by
   one means and to save time and internet traffic."

And the implementation [2] of the proposal contains a set of ANT
macros/targets that does library dependencies management for testing
suites. Briefly, a test suite may have a number of dependencies on
external libraries. It is proposed for suite to use parameters.xml
file to declare them. And the infra provides a common way for
downloading, unpacking and storing them in a 'local libraries
repository'. For example a suite can declare dependency on
'ant-contrib' in the following way:
<parameters>
    <external>
        <ant-contrib
            jar="ant-contrib-1.0b3.jar"
            url="http://kent.dl.sourceforge.net/sourceforge/ant-contrib/ant-contrib-1.0b3-bin.zip"
            md5="c5a75fc28cbc52f09bd43b5506978601"
        />
    </external>
</parameters>

Alexander, could you add more details if my description is incomplete
(or not quite incorrect)?

Taking into account that build-and-test is going to grow I agree with
the proposal [1] (that we need common solution for managing library
dependencies). And I think that the proposal may be a good moment for
reconsidering library dependencies management. Currently Class
library, DRL VM and build-and-test infra use ANT tasks to manage
external dependencies in similar way: check whether a library exists,
if not – download it. Within Harmony we have subproject that also is
aimed to resolve the issue with libraries [3]. Also Maven was
mentioned in several discussions but I don't remember any proposal for
migrating to it.

I have browsed through documentation for Antlib for Maven 2.0[4] and
Ivy[5] and it seems for me that they may help us to find a common
approach for Harmony that can be tested with the build-and-test infra.
And class library with DRL VM will migrate later.

So the question is: are we OK with the current state (no common
approach and tool) and develop and support a set of ANT tasks that
covers needs of the build-and-test infra. Or does it make sense to
look for 'standard' solution (Antlib for Maven 2.0, Ivy or something
else)? I've red through several recommendations of the Web but … I
have no experience with using Maven or Ivy. So it is hard for me to
make deliberate choose.

Thoughts? Recommendations?

[1] http://mail-archives.apache.org/mod_mbox/harmony-dev/200703.mbox/%3ce09a11790703262343m576dee90j5a59d52e14290274@mail.gmail.com%3e
[2] http://issues.apache.org/jira/browse/HARMONY-3501
[3] http://svn.apache.org/viewvc/harmony/enhanced/common_resources/
[4] http://maven.apache.org/ant-tasks.html
[5] http://incubator.apache.org/ivy/index.html

Thanks,
Stepan Mishura
Intel Enterprise Solutions Software Division

Re: [general] Managing dependencies: Antlib for Maven 2.0, Ivy or Ant only (i.e. Harmony specific way)?

Posted by Stepan Mishura <st...@gmail.com>.
On 4/4/07, Alexey Petrenko wrote:
> I'm using Maven in Yoko project... And I likes Ant more :) It looks
> more clear to me how Ant works.
> Probably the only reason for this is much larger experience with ant...
>
> Speaking about Harmony dependencies...
> One of the advantages of the current scheme is flexibility: we can
> download a dependency from anywhere, unpack it or use as is...
> With Maven we will be limited by Maven repositories. As far as I
> know... And moving a project to a new build system is also a big pane.
>
> So I vote for staying with Ant.
>

Sorry for not being clear. I don't propose to move to a new building
system such as Maven. The post was about using a set of Ant tasks[1]
provided by Maven to utilise its dependency manager. If we find that
we are not OK with limitation to Maven repositories only we may
consider using Ivy as alternative[2]. As far as I understood Ivy's
documentation it doesn't have such limitation. Sure we can continue
enhancing/supporting our set of ANT scripts and be independent from
any dependency manager tool. But taking into account growing needs of
build-and-test infra I feel that in the end we may create a tool that
will be very similar to existing tools.

[1] http://maven.apache.org/ant-tasks.html
[2] http://incubator.apache.org/ivy/index.html

Thanks,
Stepan.

> SY, Alexey
>
> 2007/4/3, Stepan Mishura <st...@gmail.com>:
> > Hi all,
> >
> > I'd like to hear opinions about choosing a way for managing library
> > dependencies.
> >
> > Currently I'm reviewing proposal for improving build-and-test infra
> > [1][2]. One of declared features according to the proposal is:
> >  "Implemented Approach to share External Library Dependencies between different
> >   Test Suites allows to perform external dependencies fetching in one place by
> >   one means and to save time and internet traffic."
> >
> > And the implementation [2] of the proposal contains a set of ANT
> > macros/targets that does library dependencies management for testing
> > suites. Briefly, a test suite may have a number of dependencies on
> > external libraries. It is proposed for suite to use parameters.xml
> > file to declare them. And the infra provides a common way for
> > downloading, unpacking and storing them in a 'local libraries
> > repository'. For example a suite can declare dependency on
> > 'ant-contrib' in the following way:
> > <parameters>
> >    <external>
> >        <ant-contrib
> >            jar="ant-contrib-1.0b3.jar"
> >            url="http://kent.dl.sourceforge.net/sourceforge/ant-contrib/ant-contrib-1.0b3-bin.zip"
> >            md5="c5a75fc28cbc52f09bd43b5506978601"
> >        />
> >    </external>
> > </parameters>
> >
> > Alexander, could you add more details if my description is incomplete
> > (or not quite incorrect)?
> >
> > Taking into account that build-and-test is going to grow I agree with
> > the proposal [1] (that we need common solution for managing library
> > dependencies). And I think that the proposal may be a good moment for
> > reconsidering library dependencies management. Currently Class
> > library, DRL VM and build-and-test infra use ANT tasks to manage
> > external dependencies in similar way: check whether a library exists,
> > if not – download it. Within Harmony we have subproject that also is
> > aimed to resolve the issue with libraries [3]. Also Maven was
> > mentioned in several discussions but I don't remember any proposal for
> > migrating to it.
> >
> > I have browsed through documentation for Antlib for Maven 2.0[4] and
> > Ivy[5] and it seems for me that they may help us to find a common
> > approach for Harmony that can be tested with the build-and-test infra.
> > And class library with DRL VM will migrate later.
> >
> > So the question is: are we OK with the current state (no common
> > approach and tool) and develop and support a set of ANT tasks that
> > covers needs of the build-and-test infra. Or does it make sense to
> > look for 'standard' solution (Antlib for Maven 2.0, Ivy or something
> > else)? I've red through several recommendations of the Web but … I
> > have no experience with using Maven or Ivy. So it is hard for me to
> > make deliberate choose.
> >
> > Thoughts? Recommendations?
> >
> > [1] http://mail-archives.apache.org/mod_mbox/harmony-dev/200703.mbox/%3ce09a11790703262343m576dee90j5a59d52e14290274@mail.gmail.com%3e
> > [2] http://issues.apache.org/jira/browse/HARMONY-3501
> > [3] http://svn.apache.org/viewvc/harmony/enhanced/common_resources/
> > [4] http://maven.apache.org/ant-tasks.html
> > [5] http://incubator.apache.org/ivy/index.html
> >
> > Thanks,
> > Stepan Mishura
> > Intel Enterprise Solutions Software Division
> >
>


-- 
Stepan Mishura
Intel Enterprise Solutions Software Division

Re: [general] Managing dependencies: Antlib for Maven 2.0, Ivy or Ant only (i.e. Harmony specific way)?

Posted by Alexey Petrenko <al...@gmail.com>.
I'm using Maven in Yoko project... And I likes Ant more :) It looks
more clear to me how Ant works.
Probably the only reason for this is much larger experience with ant...

Speaking about Harmony dependencies...
One of the advantages of the current scheme is flexibility: we can
download a dependency from anywhere, unpack it or use as is...
With Maven we will be limited by Maven repositories. As far as I
know... And moving a project to a new build system is also a big pane.

So I vote for staying with Ant.

SY, Alexey

2007/4/3, Stepan Mishura <st...@gmail.com>:
> Hi all,
>
> I'd like to hear opinions about choosing a way for managing library
> dependencies.
>
> Currently I'm reviewing proposal for improving build-and-test infra
> [1][2]. One of declared features according to the proposal is:
>  "Implemented Approach to share External Library Dependencies between different
>   Test Suites allows to perform external dependencies fetching in one place by
>   one means and to save time and internet traffic."
>
> And the implementation [2] of the proposal contains a set of ANT
> macros/targets that does library dependencies management for testing
> suites. Briefly, a test suite may have a number of dependencies on
> external libraries. It is proposed for suite to use parameters.xml
> file to declare them. And the infra provides a common way for
> downloading, unpacking and storing them in a 'local libraries
> repository'. For example a suite can declare dependency on
> 'ant-contrib' in the following way:
> <parameters>
>    <external>
>        <ant-contrib
>            jar="ant-contrib-1.0b3.jar"
>            url="http://kent.dl.sourceforge.net/sourceforge/ant-contrib/ant-contrib-1.0b3-bin.zip"
>            md5="c5a75fc28cbc52f09bd43b5506978601"
>        />
>    </external>
> </parameters>
>
> Alexander, could you add more details if my description is incomplete
> (or not quite incorrect)?
>
> Taking into account that build-and-test is going to grow I agree with
> the proposal [1] (that we need common solution for managing library
> dependencies). And I think that the proposal may be a good moment for
> reconsidering library dependencies management. Currently Class
> library, DRL VM and build-and-test infra use ANT tasks to manage
> external dependencies in similar way: check whether a library exists,
> if not – download it. Within Harmony we have subproject that also is
> aimed to resolve the issue with libraries [3]. Also Maven was
> mentioned in several discussions but I don't remember any proposal for
> migrating to it.
>
> I have browsed through documentation for Antlib for Maven 2.0[4] and
> Ivy[5] and it seems for me that they may help us to find a common
> approach for Harmony that can be tested with the build-and-test infra.
> And class library with DRL VM will migrate later.
>
> So the question is: are we OK with the current state (no common
> approach and tool) and develop and support a set of ANT tasks that
> covers needs of the build-and-test infra. Or does it make sense to
> look for 'standard' solution (Antlib for Maven 2.0, Ivy or something
> else)? I've red through several recommendations of the Web but … I
> have no experience with using Maven or Ivy. So it is hard for me to
> make deliberate choose.
>
> Thoughts? Recommendations?
>
> [1] http://mail-archives.apache.org/mod_mbox/harmony-dev/200703.mbox/%3ce09a11790703262343m576dee90j5a59d52e14290274@mail.gmail.com%3e
> [2] http://issues.apache.org/jira/browse/HARMONY-3501
> [3] http://svn.apache.org/viewvc/harmony/enhanced/common_resources/
> [4] http://maven.apache.org/ant-tasks.html
> [5] http://incubator.apache.org/ivy/index.html
>
> Thanks,
> Stepan Mishura
> Intel Enterprise Solutions Software Division
>

Re: [general] Managing dependencies: Antlib for Maven 2.0, Ivy or Ant only (i.e. Harmony specific way)?

Posted by Alexander Kleymenov <kl...@gmail.com>.
Hi Stepan,

The approach to an external dependencies sharing was suggested as an
additional feature of the new BT Infrastructure (BTI) for demonstration
purposes only. It acts as an example of a flexibility, modularity and
extensibility of a new Build and Testing Framework.

The new BTI was designed in such a way that every functional subpart of
the infrastructure is implemented in a different source file. In case of
external dependencies sharing means this source file is
scripts/download.xml. You can substitute it with your implementation.
In this case we can test your approach to implementation of this means.
Now there are several (10 and more) Test Suites integrated under
new BTI and some of them use external dependencies sharing feature of
the framework. So it can be a good real-usage test for you.

What's regarding current implementation suggested in new BTI: it was
made in an implementation independent manner. I.e. you can use different
libraries instead of current one. Even maven or something else. The main
idea of an approach is:

  1. User describes its dependencies in a specified format. The
     main piece of description data are: URL from
     where to get the resource, md5 to made the process more secure,
     and some switches describing how to operate on resource - extract,
     rename, filter and so on.

  2. Framework processes these descriptions and provides the properties
     with a specified format of the name (ext.resource-name.location)
     pointing to the described resource.

In other words it makes the correspondence between input data (URL) and
output data (Local Path to resource). And it is not important what
functionality makes such a correspondence. You can use even hardcoded table
listing the correspondences between "URL" and "Local Path to Resource".
Or, I can suggest using of hardcoded table of "MD5" -> "Path", i.e. MD5
values acting as an primary indexes of resource database :)
It's so cool I've no words :)

So I want to summarize my message with the following:
    - Current implementation provided with new BTI is raw, but can be
      substituted with much more mature (I even have the point for
      improvements)
    - It can be great to have your implementation made under new BTI.

Thank you,
Alexander

Re: [general] Managing dependencies: Antlib for Maven 2.0, Ivy or Ant only (i.e. Harmony specific way)?

Posted by Stepan Mishura <st...@gmail.com>.
Current resolution:
A dependency framework that is based on ANT only was added to
build-and-test infra at r527838. The framework is result of merging
class library approach and newly proposed approach from [1] with some
modifications.
The current implementation is aimed:
-provide unified approach for downloading and storing external resources
-be simple and clear
-use only ANT tasks/types (i.e. get rid of using antcontrib)

Definitely it will be improved. So any suggestions, ideas, help in
testing, criticism are highly appreciated.

Here are some brief details about the current implementation:
each external resource is described in 'parameters.xml' file by the
set of properties:
parameters.external.<resorce_name>.<resource_property> (that was
suggested in [1]). The framework iterates over property set
(parameters.external.*.file). I wasn't able to find "standard way" to
make ANT to iterate over propertyset so I've implemented the first
approach that work for me but I assume there might be more elegant
solution. So it checks if the resource was downloaded or not. If it is
absent it is downloaded to depend/<resource_dir> directory. If there
is md5 check specified the framework verifies it.

[1] http://issues.apache.org/jira/browse/HARMONY-3501

Thanks,
Stepan.

On 4/3/07, Stepan Mishura wrote:
> Hi all,
>
> I'd like to hear opinions about choosing a way for managing library
> dependencies.
>
> Currently I'm reviewing proposal for improving build-and-test infra
> [1][2]. One of declared features according to the proposal is:
>  "Implemented Approach to share External Library Dependencies between different
>   Test Suites allows to perform external dependencies fetching in one place by
>   one means and to save time and internet traffic."
>
> And the implementation [2] of the proposal contains a set of ANT
> macros/targets that does library dependencies management for testing
> suites. Briefly, a test suite may have a number of dependencies on
> external libraries. It is proposed for suite to use parameters.xml
> file to declare them. And the infra provides a common way for
> downloading, unpacking and storing them in a 'local libraries
> repository'. For example a suite can declare dependency on
> 'ant-contrib' in the following way:
> <parameters>
>    <external>
>        <ant-contrib
>            jar="ant-contrib-1.0b3.jar"
>            url="http://kent.dl.sourceforge.net/sourceforge/ant-contrib/ant-contrib-1.0b3-bin.zip"
>            md5="c5a75fc28cbc52f09bd43b5506978601"
>        />
>    </external>
> </parameters>
>
> Alexander, could you add more details if my description is incomplete
> (or not quite incorrect)?
>
> Taking into account that build-and-test is going to grow I agree with
> the proposal [1] (that we need common solution for managing library
> dependencies). And I think that the proposal may be a good moment for
> reconsidering library dependencies management. Currently Class
> library, DRL VM and build-and-test infra use ANT tasks to manage
> external dependencies in similar way: check whether a library exists,
> if not – download it. Within Harmony we have subproject that also is
> aimed to resolve the issue with libraries [3]. Also Maven was
> mentioned in several discussions but I don't remember any proposal for
> migrating to it.
>
> I have browsed through documentation for Antlib for Maven 2.0[4] and
> Ivy[5] and it seems for me that they may help us to find a common
> approach for Harmony that can be tested with the build-and-test infra.
> And class library with DRL VM will migrate later.
>
> So the question is: are we OK with the current state (no common
> approach and tool) and develop and support a set of ANT tasks that
> covers needs of the build-and-test infra. Or does it make sense to
> look for 'standard' solution (Antlib for Maven 2.0, Ivy or something
> else)? I've red through several recommendations of the Web but … I
> have no experience with using Maven or Ivy. So it is hard for me to
> make deliberate choose.
>
> Thoughts? Recommendations?
>
> [1] http://mail-archives.apache.org/mod_mbox/harmony-dev/200703.mbox/%3ce09a11790703262343m576dee90j5a59d52e14290274@mail.gmail.com%3e
> [2] http://issues.apache.org/jira/browse/HARMONY-3501
> [3] http://svn.apache.org/viewvc/harmony/enhanced/common_resources/
> [4] http://maven.apache.org/ant-tasks.html
> [5] http://incubator.apache.org/ivy/index.html
>
> Thanks,
> Stepan Mishura
> Intel Enterprise Solutions Software Division
>