mirroring/rsyncing the maven repos on people.apache.org?

[James Strachan <ja...@gmail.com>]

With the hardware issues of people.apache.org lately its been a trying
week working on stuff in the incubator - maven builds have been
failing most of the week. We've also lost a few releases it seems
(XBean 2.7, ServiceMix 3.0 maybe others too) due to the reversal to an
old backup.

Am wondering, is it time to mirror the repos in people.apache.org so
we can insulate ourselves against future failures?

-- 

James
-------
http://radio.weblogs.com/0112098/

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: mirroring/rsyncing the maven repos on people.apache.org?

[robert burrell donkin <ro...@gmail.com>]

On 10/27/06, Justin Erenkrantz <ju...@erenkrantz.com> wrote:
> On 10/27/06, Noel J. Bergman <no...@devtech.com> wrote:
> > That's contrary to what we were originally told by the mirroring folks,
> > e.g., Joshua.  So we told everyone, as we were told, to use cvs(now
> > people).a.o.  If infra is OK with this, we'll change the release docs to
> > reflect it.
>
> In the original concept, I think we were going to allow PPMCs to
> release on their own; however, we've since refined that concept so
> that the IPMC reviews each release.  Since the IPMC is approving the
> release and there aren't *that* many PPMC releases made (i.e. it's not
> a daily occurance),

there are now several releases most weeks. the limiting factor is the
energy of the IPMC to review and approve these releases. once the IPMC
starts to approve releases more promptly and the standard of releases
submitted improves, i expect the frequency of release to increase.

> it makes more sense to use the mirrors rather than
> just slam one server.

the bandwidth used pushing releases out to all the mirrors would be
large. using a subset for incubator releases would address this
concern.

- robert

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: mirroring/rsyncing the maven repos on people.apache.org?

[Justin Erenkrantz <ju...@erenkrantz.com>]

On 10/27/06, Noel J. Bergman <no...@devtech.com> wrote:
> That's contrary to what we were originally told by the mirroring folks,
> e.g., Joshua.  So we told everyone, as we were told, to use cvs(now
> people).a.o.  If infra is OK with this, we'll change the release docs to
> reflect it.

In the original concept, I think we were going to allow PPMCs to
release on their own; however, we've since refined that concept so
that the IPMC reviews each release.  Since the IPMC is approving the
release and there aren't *that* many PPMC releases made (i.e. it's not
a daily occurance), it makes more sense to use the mirrors rather than
just slam one server.  -- justin

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

RE: mirroring/rsyncing the maven repos on people.apache.org?

["Noel J. Bergman" <no...@devtech.com>]

Justin Erenkrantz wrote:

> IMO, if the IPMC votes to approve the release, it should be slapped in
> www.a.o/dist/incubator/ and hence use the same mirror system as
> everyone else.  The mirrors can live with the noise - the IPMC isn't
> approving *that* many releases.

That's contrary to what we were originally told by the mirroring folks,
e.g., Joshua.  So we told everyone, as we were told, to use cvs(now
people).a.o.  If infra is OK with this, we'll change the release docs to
reflect it.

> I have no idea how folks got the idea that people.a.o/dist was ever backed
up

I was never under that illusion.  Don't know why anyone should have been.

	--- Noel



---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: mirroring/rsyncing the maven repos on people.apache.org?

[Justin Erenkrantz <ju...@erenkrantz.com>]

On 10/26/06, robert burrell donkin <ro...@gmail.com> wrote:
> so, i would prefer to see incubator releases stored in a central
> subversion repository and a zone managed by the IPMC used to host them
> for download. perhaps a small number of select incubator-only mirrors
> could be recruited. IMHO this would solve the worries over excessive
> rsync bandwidth and the IPMCs concerns about isolation.

IMO, if the IPMC votes to approve the release, it should be slapped in
www.a.o/dist/incubator/ and hence use the same mirror system as
everyone else.  The mirrors can live with the noise - the IPMC isn't
approving *that* many releases.  I have no idea how folks got the idea
that people.a.o/dist was ever backed up - it was originally created
only for temporary snapshots (and not mirrored anywhere) - so it seems
there was a huge breakdown in communication here.  -- justin

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: mirroring/rsyncing the maven repos on people.apache.org?

[robert burrell donkin <ro...@gmail.com>]

On 10/26/06, James Strachan <ja...@gmail.com> wrote:
> With the hardware issues of people.apache.org lately its been a trying
> week working on stuff in the incubator - maven builds have been
> failing most of the week. We've also lost a few releases it seems
> (XBean 2.7, ServiceMix 3.0 maybe others too) due to the reversal to an
> old backup.
>
> Am wondering, is it time to mirror the repos in people.apache.org so
> we can insulate ourselves against future failures?

infrastructure historically have very real worries about the bandwidth
which would be consumed by lots of short lived releases being pushed
out to the full range of mirrors. the incubator historically has
issues about users becoming confused between full apache releases and
incubator releases. both forces pushed towards installing on
people.apache.org.

i'm strongly in favour of using a special subversion repository to
store releases of all kinds and maven repositories in particular. this
would allow pre-commit scripts to be run to check that the artifacts
complies with the requirements about signatures and checksums. it
would also give a complete history for all artifacts, log who uploaded
which artifacts and allow easy email messages to be sent whenever
changes are made to the repository. access could be limited to release
managers.

so, i would prefer to see incubator releases stored in a central
subversion repository and a zone managed by the IPMC used to host them
for download. perhaps a small number of select incubator-only mirrors
could be recruited. IMHO this would solve the worries over excessive
rsync bandwidth and the IPMCs concerns about isolation.

- robert

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: mirroring/rsyncing the maven repos on people.apache.org?

[robert burrell donkin <ro...@gmail.com>]

On 10/27/06, Guillaume Nodet <gn...@apache.org> wrote:

<snip>

> Could we start by publishing to our own svn repository ?
> http://svn.apache.org/repos/asf/incubator/servicemix/repo ?

-1

we have enough issues with robots using excessive bandwidth whilst
crawling our repository already

any repository used for staging would need to be isolated from the
source repository

- robert

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: mirroring/rsyncing the maven repos on people.apache.org?

[Guillaume Nodet <gn...@apache.org>]

Henri Yandell wrote:
> On 10/26/06, James Strachan <ja...@gmail.com> wrote:
>> With the hardware issues of people.apache.org lately its been a trying
>> week working on stuff in the incubator - maven builds have been
>> failing most of the week. We've also lost a few releases it seems
>> (XBean 2.7, ServiceMix 3.0 maybe others too) due to the reversal to an
>> old backup.
>>
>> Am wondering, is it time to mirror the repos in people.apache.org so
>> we can insulate ourselves against future failures?
>
> We need to do something.
Agreed :)
We have lots of users complaining that they can not find the ServiceMix 3.0
release. 
>
> With regards to snapshots, one hope is to set up a central snapshot
> repository and manage it in the same way the maven central one is
> managed.
>
> With regards to the incubating projects - the options are pretty clear:
Note that it seems the m2 (non - incubating) repository which is currently
located at people.apache.org/repo/m2-ibiblio-rsync-repository is not
backup.   That's why we have also lost the XBean 2.7 release (between the
time it was published and the next rsync).
>
> 1) Rsync them over to the maven central repository or central snapshot
> repository (our choice, presuming the latter is possible).
That would be great.  But I recall that some IPMC members were not
very keen on publishing podling artifacts on ibiblio.
>
> 2) Start managing the infrastructure for a maven repository properly.
> I'm in favour of Robert's SVN ideas - it's how we manage our
> repository at osjava.org and it's never been a problem - so just
> becomes a question of whether it scales. It also lets us learn slowly
> before applying such ideas to an asf repository. There's real value in
> a repository that only has asf vetted items in and has high quality of
> releases into it, but I think we need to be doing a smaller prototype
> first.
SVN is a really good idea to store a maven repository I think.
It may even be used in addition to publishing to the central repo.

I'd like to start publishing a ServiceMix 3.0.1 release, but I really need
to know where is the maven repository that we should use, as we need
to include them in different places in the distribution so that users can
access the maven repo (it would not be needed if they were rsynced on
ibiblio). 
Could we start by publishing to our own svn repository ?
http://svn.apache.org/repos/asf/incubator/servicemix/repo ?

Cheers,
Guillaume Nodet
>
> Hen
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: mirroring/rsyncing the maven repos on people.apache.org?

[Henri Yandell <fl...@gmail.com>]

On 10/26/06, James Strachan <ja...@gmail.com> wrote:
> With the hardware issues of people.apache.org lately its been a trying
> week working on stuff in the incubator - maven builds have been
> failing most of the week. We've also lost a few releases it seems
> (XBean 2.7, ServiceMix 3.0 maybe others too) due to the reversal to an
> old backup.
>
> Am wondering, is it time to mirror the repos in people.apache.org so
> we can insulate ourselves against future failures?

We need to do something.

With regards to snapshots, one hope is to set up a central snapshot
repository and manage it in the same way the maven central one is
managed.

With regards to the incubating projects - the options are pretty clear:

1) Rsync them over to the maven central repository or central snapshot
repository (our choice, presuming the latter is possible).

2) Start managing the infrastructure for a maven repository properly.
I'm in favour of Robert's SVN ideas - it's how we manage our
repository at osjava.org and it's never been a problem - so just
becomes a question of whether it scales. It also lets us learn slowly
before applying such ideas to an asf repository. There's real value in
a repository that only has asf vetted items in and has high quality of
releases into it, but I think we need to be doing a smaller prototype
first.

Hen

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org