You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jackrabbit.apache.org by Jorge Ferrer <jo...@gmail.com> on 2007/07/02 14:39:07 UTC
Re: S3 persistence manager
Hi Jukka, Martin,
Has there been any progress on this? I'm also interested in this and
would like to help.
Related to Jerome's suggestions, do you think that the current PM
architecture perform well if using a FileSystem PM to a directory that
is actually a S3 FileSystem [1]?
Regards,
Jorge
[1] http://developer.amazonwebservices.com/connect/thread.jspa?threadID=10271&tstart=30
On 6/9/07, Jerome B <je...@gmail.com> wrote:
>
>
> I think there are projects which access S3 as a WebDAV store, as well as S3
> "filesystems". If there was a S3 Jackrabbit persistence manager being
> developed, I'd be interested as well.
>
> -- jerome
>
>
> Martin Perez wrote:
> >
> > Really glad to read that. Massive distributed storage propositions are
> > being
> > proven like very good solutions for massive scaling. I guess the key
> > challenge in this case is trying to minimize bandwidth usage as much as
> > possible in addition to the traditional storage challenge.
> >
> > Would be really nice to test it when that global data store stuff you are
> > working in is done. If at that time I have some web projec using
> > Jackrabbit
> > (hopefully) I would love to spend some time playing with that. However, It
> > would be even better if someone already has experiences or is already
> > trying
> > to do it and shares his experience.
> >
> > Cheers,
> > Martin
> >
> > On 6/5/07, Jukka Zitting <ju...@gmail.com> wrote:
> >>
> >> Hi,
> >>
> >> On 6/4/07, Martin Perez <mp...@gmail.com> wrote:
> >> > Is there anyone developing something like a Amazon S3 persistence
> >> manager?
> >> >
> >> > If not, it seems to me like a good candidate for a new project ( it is
> >> too
> >> > late for SOC ), or perhaps it is an insane idea?
> >>
> >> I haven't heard of anyone actively working on something like that, but
> >> I agree that it would be a cool idea.
> >>
> >> Actually S3 was floating around in my mind when I wrote the NGP
> >> proposal (http://jackrabbit.apache.org/dev/ngp.html). S3 or another
> >> similar massively distributed storage mechanism would work very well
> >> with the NGP concepts.
> >>
> >> I'm currently working on a related more short-term idea of adding a
> >> global "data store" that would handle all binary properties (and
> >> possibly other data like item bundles as well). The data store concept
> >> is designed to work very well with remote or distributed storage, so
> >> S3 would be an interesting option there as well.
> >>
> >> BR,
> >>
> >> Jukka Zitting
> >>
> >
> >
>
> --
> View this message in context: http://www.nabble.com/S3-persistence-manager-tf3864934.html#a11036005
> Sent from the Jackrabbit - Dev mailing list archive at Nabble.com.
>
>
--
Jorge Ferrer Zarzuela
http://www.jorgeferrer.com
http://www.liferay.com
Re: S3 persistence manager
Posted by Jorge Ferrer <jo...@gmail.com>.
Hi Martin,
Thanks for your answer. I'll take a look at the filesystem
implementation from Hadoop.
Also, I've found jets3t [1] which seem to simplify development with
Java on top of S3. We'll probably use it to build a less-featured
alternative to Jackrabbit for now and will think about implementing a
PM with it later on.
Regards,
Jorge
[1] https://jets3t.dev.java.net/
On 7/2/07, Martin Perez <mp...@gmail.com> wrote:
> I don't think anyone is currently looking at this. It was only a random idea
> that I dropped at the forum.
>
> However, seeing that another Apache project has already a S3 filesystem
> built in perhaps it would be interesting to look at it (
> http://wiki.apache.org/lucene-hadoop/AmazonS3)
>
> Cheers,
> Martin
>
> On 7/2/07, Jorge Ferrer <jo...@gmail.com> wrote:
> >
> > Hi Jukka, Martin,
> >
> > Has there been any progress on this? I'm also interested in this and
> > would like to help.
> >
> > Related to Jerome's suggestions, do you think that the current PM
> > architecture perform well if using a FileSystem PM to a directory that
> > is actually a S3 FileSystem [1]?
> >
> > Regards,
> > Jorge
> >
> > [1]
> > http://developer.amazonwebservices.com/connect/thread.jspa?threadID=10271&tstart=30
> >
> > On 6/9/07, Jerome B <je...@gmail.com> wrote:
> > >
> > >
> > > I think there are projects which access S3 as a WebDAV store, as well
> > as S3
> > > "filesystems". If there was a S3 Jackrabbit persistence manager being
> > > developed, I'd be interested as well.
> > >
> > > -- jerome
> > >
> > >
> > > Martin Perez wrote:
> > > >
> > > > Really glad to read that. Massive distributed storage propositions are
> > > > being
> > > > proven like very good solutions for massive scaling. I guess the key
> > > > challenge in this case is trying to minimize bandwidth usage as much
> > as
> > > > possible in addition to the traditional storage challenge.
> > > >
> > > > Would be really nice to test it when that global data store stuff you
> > are
> > > > working in is done. If at that time I have some web projec using
> > > > Jackrabbit
> > > > (hopefully) I would love to spend some time playing with that.
> > However, It
> > > > would be even better if someone already has experiences or is already
> > > > trying
> > > > to do it and shares his experience.
> > > >
> > > > Cheers,
> > > > Martin
> > > >
> > > > On 6/5/07, Jukka Zitting <ju...@gmail.com> wrote:
> > > >>
> > > >> Hi,
> > > >>
> > > >> On 6/4/07, Martin Perez <mp...@gmail.com> wrote:
> > > >> > Is there anyone developing something like a Amazon S3 persistence
> > > >> manager?
> > > >> >
> > > >> > If not, it seems to me like a good candidate for a new project ( it
> > is
> > > >> too
> > > >> > late for SOC ), or perhaps it is an insane idea?
> > > >>
> > > >> I haven't heard of anyone actively working on something like that,
> > but
> > > >> I agree that it would be a cool idea.
> > > >>
> > > >> Actually S3 was floating around in my mind when I wrote the NGP
> > > >> proposal (http://jackrabbit.apache.org/dev/ngp.html). S3 or another
> > > >> similar massively distributed storage mechanism would work very well
> > > >> with the NGP concepts.
> > > >>
> > > >> I'm currently working on a related more short-term idea of adding a
> > > >> global "data store" that would handle all binary properties (and
> > > >> possibly other data like item bundles as well). The data store
> > concept
> > > >> is designed to work very well with remote or distributed storage, so
> > > >> S3 would be an interesting option there as well.
> > > >>
> > > >> BR,
> > > >>
> > > >> Jukka Zitting
> > > >>
> > > >
> > > >
> > >
> > > --
> > > View this message in context:
> > http://www.nabble.com/S3-persistence-manager-tf3864934.html#a11036005
> > > Sent from the Jackrabbit - Dev mailing list archive at Nabble.com.
> > >
> > >
> >
> >
> > --
> > Jorge Ferrer Zarzuela
> > http://www.jorgeferrer.com
> > http://www.liferay.com
> >
>
--
Jorge Ferrer Zarzuela
http://www.jorgeferrer.com
http://www.liferay.com
Re: S3 persistence manager
Posted by Martin Perez <mp...@gmail.com>.
I don't think anyone is currently looking at this. It was only a random idea
that I dropped at the forum.
However, seeing that another Apache project has already a S3 filesystem
built in perhaps it would be interesting to look at it (
http://wiki.apache.org/lucene-hadoop/AmazonS3)
Cheers,
Martin
On 7/2/07, Jorge Ferrer <jo...@gmail.com> wrote:
>
> Hi Jukka, Martin,
>
> Has there been any progress on this? I'm also interested in this and
> would like to help.
>
> Related to Jerome's suggestions, do you think that the current PM
> architecture perform well if using a FileSystem PM to a directory that
> is actually a S3 FileSystem [1]?
>
> Regards,
> Jorge
>
> [1]
> http://developer.amazonwebservices.com/connect/thread.jspa?threadID=10271&tstart=30
>
> On 6/9/07, Jerome B <je...@gmail.com> wrote:
> >
> >
> > I think there are projects which access S3 as a WebDAV store, as well
> as S3
> > "filesystems". If there was a S3 Jackrabbit persistence manager being
> > developed, I'd be interested as well.
> >
> > -- jerome
> >
> >
> > Martin Perez wrote:
> > >
> > > Really glad to read that. Massive distributed storage propositions are
> > > being
> > > proven like very good solutions for massive scaling. I guess the key
> > > challenge in this case is trying to minimize bandwidth usage as much
> as
> > > possible in addition to the traditional storage challenge.
> > >
> > > Would be really nice to test it when that global data store stuff you
> are
> > > working in is done. If at that time I have some web projec using
> > > Jackrabbit
> > > (hopefully) I would love to spend some time playing with that.
> However, It
> > > would be even better if someone already has experiences or is already
> > > trying
> > > to do it and shares his experience.
> > >
> > > Cheers,
> > > Martin
> > >
> > > On 6/5/07, Jukka Zitting <ju...@gmail.com> wrote:
> > >>
> > >> Hi,
> > >>
> > >> On 6/4/07, Martin Perez <mp...@gmail.com> wrote:
> > >> > Is there anyone developing something like a Amazon S3 persistence
> > >> manager?
> > >> >
> > >> > If not, it seems to me like a good candidate for a new project ( it
> is
> > >> too
> > >> > late for SOC ), or perhaps it is an insane idea?
> > >>
> > >> I haven't heard of anyone actively working on something like that,
> but
> > >> I agree that it would be a cool idea.
> > >>
> > >> Actually S3 was floating around in my mind when I wrote the NGP
> > >> proposal (http://jackrabbit.apache.org/dev/ngp.html). S3 or another
> > >> similar massively distributed storage mechanism would work very well
> > >> with the NGP concepts.
> > >>
> > >> I'm currently working on a related more short-term idea of adding a
> > >> global "data store" that would handle all binary properties (and
> > >> possibly other data like item bundles as well). The data store
> concept
> > >> is designed to work very well with remote or distributed storage, so
> > >> S3 would be an interesting option there as well.
> > >>
> > >> BR,
> > >>
> > >> Jukka Zitting
> > >>
> > >
> > >
> >
> > --
> > View this message in context:
> http://www.nabble.com/S3-persistence-manager-tf3864934.html#a11036005
> > Sent from the Jackrabbit - Dev mailing list archive at Nabble.com.
> >
> >
>
>
> --
> Jorge Ferrer Zarzuela
> http://www.jorgeferrer.com
> http://www.liferay.com
>