Re: Experiences with transitive dependencies in buildr

[Michael Guymon <mi...@gmail.com>]

I have a new xample Jarfile and Jarfile.lock: 
https://gist.github.com/1734249

I added support so Maven POMs can be used as dependencies along with 
artifact notation.

To make it work, you need to create a Jarfile, such one in the example. 
Then you generate the lock file (Jarfile.lock) by calling:

   LockJar.lock()

With a lock file, you can get an array of resolved jars by calling:

   LockJar.list()

or the resolved jars in the lock file directly into the classpath by 
calling:

   LockJar.load()

The next step is to wire in support for LockJar into buildr. If a lock 
file is detected, automatically run LockJar for tasks like compile and test.

On 12/28/2011 03:02 PM, Rhett Sutphin wrote:
> Hi Michael,
>
> On Dec 28, 2011, at 1:43 PM, Michael Guymon wrote:
>
>> So I have slowly being making progress on a dependency manager for buildr and have put together a very simple implementation based on Bundler using Aether for the transitive resolution. I would love some feed back on what I have pieced together so far.
>>
>> https://github.com/mguymon/lockjar
>>
>> A Jarfile is used to specify the repos and dependencies, an example:
>>
>> repository 'http://repository.jboss.org/nexus/content/groups/public-jboss'
>> jar "com.slackworks:model-citizen:0.1"
>> jar "org.apache.mina:mina-core:2.0.4" do exclude "com.package", "com.another.package:blah:7" end
>>
>> Would produce a Jarfile.lock with absolute paths to the local jars, an example:
>>
>> ---
>>
>> dependencies:
>>   com.slackworks:model-citizen:0.1: /home/zinger/devel/projects/swx/lockjar/tmp/test-repo/com/slackworks/model-citizen/0.1/model-citizen-0.1.jar
>>   org.apache.mina:mina-core:jar:2.0.4: /home/zinger/devel/projects/swx/lockjar/tmp/test-repo/org/apache/mina/mina-core/2.0.4/mina-core-2.0.4.jar
>>   org.slf4j:slf4j-api:jar:1.6.1: /home/zinger/devel/projects/swx/lockjar/tmp/test-repo/org/slf4j/slf4j-api/1.6.1/slf4j-api-1.6.1.jar
>>
>> repositories:
>>   - http://repository.jboss.org/nexus/content/groups/public-jboss
>>
>>
>> The Jarfile.lock is standard yaml, not sure if a custom dsl is needed. Similar to bundler, the Jarfile.lock has to be generated before getting access to the dependencies. Once the Jarfile.lock is generated, Buildr can reuse it instead of checking for transitive dependencies every time.
> This is a very interesting idea. Might I suggest, though, that you have the lockfile contain just the list of dependent artifacts? That way you could check it in and use it to ensure that all developers/CI/etc. are using the same artifact versions. Converting from a buildr artifact name to the local path is not an expensive operation.
>
> Rhett
>
>> Next step is moving the project towards something that integrates nicely with Buildr. Maybe allow the Jarfile to be defined directly in the buildfile?
>>
>> thanks,
>> Michael
>>
>> On 11/04/2011 06:13 AM, Peter Tillotson wrote:
>>> This sounds pretty good - some thoughts inline below
>>>
>>>> Dependency Resolution
>>>>
>>>> * Dependency resolution is optional
>>>> * Resolve Transitive Dependencies for an artifact(s)
>>>> * Create/Rebuild a dependency file
>>>> * Lock dependencies based on the dependency file
>>> Transitive resolution should be capable of producing the same dependencies as maven ( possibly via an interface compatible plugin ) and support the full maven spec.
>>>
>>> I'd add  filtering and blacklist has been useful as well. I always have to package(:war).libs.reject! { |lib| lib.group == 'servlet-api' } this is a reasonable compile dependency that shouldn't get into runtime. If the dependencies file something like the following
>>>
>>> runtime:
>>>     - org.apache.cassandra:cassalndra-all:1.0.1
>>>         exclude:
>>>     - *:servlet-api:*
>>>     - *:junit:*
>>>
>>> That would be pretty handy - I really like the idea of the dependency report effectively acting as the whitelist / blacklist so when the universe changes my build file doesn't necessarily have to :-)
>>>
>>>> Maven interop
>>>>
>>>> * Generate a maven POM based on a Buildr project
>>>> * Create&   deploy maven friendly artifacts based on a Buildr project
>>>>
>>>> Would there be any benefit for having Ivy interoperability as well?
>>> p
>>>
>>> ________________________________
>>> From: Michael Guymon<mi...@gmail.com>
>>> To: dev@buildr.apache.org
>>> Sent: Thursday, 3 November 2011, 23:16
>>> Subject: Re: Experiences with transitive dependencies in buildr
>>>
>>>
>>> So based on the discussion, it sounds like the following solutions would work well:
>>>
>>> Dependency Resolution
>>>
>>> * Dependency resolution is optional
>>> * Resolve Transitive Dependencies for an artifact(s)
>>> * Create/Rebuild a dependency file
>>> * Lock dependencies based on the dependency file
>>>
>>> Maven interop
>>>
>>> * Generate a maven POM based on a Buildr project
>>> * Create&   deploy maven friendly artifacts based on a Buildr project
>>>
>>> Would there be any benefit for having Ivy interoperability as well?
>>>
>>> On 11/02/2011 09:59 PM, Peter Donald wrote:
>>>> Hi,
>>>>
>>>> On Thu, Nov 3, 2011 at 12:26 AM, Chiaming Hsu<ca...@yahoo.com>    wrote:
>>>>> If this becomes part of the core of buildr as Peter suggested, would there be performance impact when not using transitive dependencies?
>>>> I would hope not.
>>>>
>>>> I am not a huge fan of transitive closure across the builds when you
>>>> can't lock it down to a specific set (like Gemfile.lock or what Ivy4r
>>>> supports it seems). I don't object so much to the network chattiness
>>>> but due to the un-reproducible of builds.
>>>>
>>>> Aether would mainly used as the API for interacting with Maven repos
>>>> and for implementing transitive dependencies of maven artifacts.
>>>>
>>>>