Ivy repository and resolver for Amazon's S3 service

[Jeremy Handcock <je...@aperte.org>]

Hi there,

I've recently developed an Ivy repository and resolver that allows you
to store and resolve Ivy artifacts/metadata using Amazon's S3 service
[1].  Is there any interest in having this become part of the Ivy core?
Is there some kind of Ivy extensions project that this might fit better
under?

Thanks,

Jeremy

[1] http://s3.amazonaws.com

Re: Ivy repository and resolver for Amazon's S3 service

[Jeremy Handcock <je...@aperte.org>]

Hi Xavier,

On Tue, May 22, 2007 at 09:15:39AM +0200, Xavier Hanin wrote:
>      Is there any interest in having this become part of the Ivy core?
> 
> 
> This is something that can be discussed indeed. A few questions: -
> does it require any external library, and if so, which license do they
> have?

Right now I'm using the JetS3t library: https://jets3t.dev.java.net.
It's licensed under the Apache License, Version 2.0.  So it's compatible
from the license perspective.  I'm happy to discuss alternatives though
if you want to avoid introducing a dependency on that library.

> - do you have some kind of test?

I have a JUnit test suite for it.

> - do you have some documentation? Is it easy to use? Can you use
> patterns to configure it?

You can configure it like any other resolver in your ivyconf.xml.  You
can indeed use patterns to configure the metadata and artifact paths.
There are a few things that would be really useful but that I don't have
implemented yet:

- encryption/decryption support (useful for organizations storing
  proprietary artifacts)
- compression/decompression support (reduces S3 bandwidth and storage
  costs)
- leverage S3's BitTorrent support (reduces S3 bandwidth costs)

I don't have public documentation or a public demo available right now.
Unfortunately I need to put the code through my company's open source
approval process before I can release it and/or work on it in public,
but if there is interest I can certainly get the ball rolling.  Let me
know what you think.

Jeremy

Re: Ivy repository and resolver for Amazon's S3 service

[Xavier Hanin <xa...@gmail.com>]

On 5/22/07, Jeremy Handcock <je...@aperte.org> wrote:
>
> Hi there,


Hi,

I've recently developed an Ivy repository and resolver that allows you
> to store and resolve Ivy artifacts/metadata using Amazon's S3 service
> [1].


Very interesting. Leveraging the S3 scalability for an Ivy repository is
indeed a very good idea.

 Is there any interest in having this become part of the Ivy core?


This is something that can be discussed indeed. A few questions:
- does it require any external library, and if so, which license do they
have?
- do you have some kind of test?
- do you have some documentation? Is it easy to use? Can you use patterns to
configure it?

Is there some kind of Ivy extensions project that this might fit better
> under?


There is one such project, but it has never actually become an active
project:
http://ivytools.sourceforge.net/

Xavier

Thanks,
>
> Jeremy
>
> [1] http://s3.amazonaws.com
>



-- 
Xavier Hanin - Independent Java Consultant
Manage your dependencies with Ivy!
http://incubator.apache.org/ivy/

Ivy extensions support (Was: Ivy repository and resolver for Amazon's S3 service)

[Gilles Scokart <gs...@gmail.com>]

The mail of Jeremy raise a question: How do we support extension of ivy?

I see different approaches.  More than one is probably valid, and the choice
will certainly have to be done on a case by case basis.  But deciding some
a-priori guidelines could help the future choice.

Here are the approaches that I see:
1. Having a separate project outside apache and maintain the extension
there.
2. Having a 'sandbox' project inside apache and maintain the extension
there.
3. Incorporate the extension into the ivy project itself.
4. Same as 3. But incorporate also the contributor as a committer.


The pros/cons I see for each approach are:
- 1. is nice when there are license issue. It is easer to put in place.
However, the risk of non maintenance is bigger.  Also, it is more difficult
for the developer of those extensions to keep their motivation when their
project is +/- disconnected from the rest of the ivy community.
- I don't know if the approach 2 is possible for a project under incubation.
I guess not.
- The approach 3 requires that the current set of committer have the ability
to maintain the extension.  We should also check that there is no license
issue.
- The approach 4 requires that the contributor is 'trusted' by the
community.

WDYT?

Gilles

> -----Original Message-----
> From: Jeremy Handcock [mailto:jeremy@aperte.org]
> Sent: mardi 22 mai 2007 6:50
> To: ivy-dev@incubator.apache.org
> Subject: Ivy repository and resolver for Amazon's S3 service
> 
> Hi there,
> 
> I've recently developed an Ivy repository and resolver that allows you
> to store and resolve Ivy artifacts/metadata using Amazon's S3 service
> [1].  Is there any interest in having this become part of the Ivy core?
> Is there some kind of Ivy extensions project that this might fit better
> under?
> 
> Thanks,
> 
> Jeremy
> 
> [1] http://s3.amazonaws.com