You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@openoffice.apache.org by "Marcus (OOo)" <ma...@wtnet.de> on 2011/10/22 01:15:34 UTC

Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Hi Peter,

it seems the end is near and the "download.services.openoffice.org" host 
on Oracle side with our Mirrorbrain instance will be shutdown in one week.

As we cannot buildup a solution that is running, tested and 
long-term-proven in a few days I would like to ask you if we can switch 
to your "openoffice.mirrorbrain.org" instance as long as we have no 
other solution at hand.

In the past it was a very reliable host that we have used when there 
were outages (mostly unplanned as you know) and doesn't resulted in a 
significant higher load on the host.

So, it would help us very much to have a stable download section until 
we have an own solution here at ASF.

Thanks in advance and have a nice, sunny weekend.

Best regards

Marcus

Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by "Marcus (OOo)" <ma...@wtnet.de>.
Am 10/25/2011 12:23 AM, schrieb Donald Whytock:
> On Sun, Oct 23, 2011 at 9:08 PM, Marcus (OOo)<ma...@wtnet.de>  wrote:
>> @List:
>> Has anybody an idea about where to host this service? It doesn't need to be
>> necessarily inside the ASF.
>
> The Pirate Bay?  Lend them some legitimacy? :)
>
> More seriously, on the risk of being off-topic, has torrenting been
> considered for downloads?

Yes, torrents are already used. However, I would like to have something 
that we have under our own controal when we initializue a master server 
to distribute our release files.

Marcus


Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by Robert Burrell Donkin <ro...@gmail.com>.
On Tue, Oct 25, 2011 at 1:04 AM, Dennis E. Hamilton
<de...@acm.org> wrote:
> I read somewhere, and I don't know where, that ASF did not want torrents to be used.

The meaning and force of this statement is hard to judge without a full context

Apache has surprisingly and confusingly little policy, and most of
that should be written down

Apache encourages wide participation on open lists. Conventionally,
opinions expressed on lists are just personal opinions - unless backed
by evidence or clear marking[1]. So this is just my personal opinion
;-)

Robert

[1] wearing a hat :-) http://www.apache.org/foundation/how-it-works.html

Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by Robert Burrell Donkin <ro...@gmail.com>.
On Tue, Oct 25, 2011 at 7:58 PM, Christian Lohmaier
<cl...@openoffice.org> wrote:
> Hi Robert, *,
>
> On Tue, Oct 25, 2011 at 5:05 PM, Robert Burrell Donkin
> <ro...@gmail.com> wrote:
>> On Tue, Oct 25, 2011 at 1:38 PM, Christian Lohmaier
>> <cl...@openoffice.org> wrote:
>>> On Tue, Oct 25, 2011 at 2:15 PM, Robert Burrell Donkin
>>> <ro...@gmail.com> wrote:

<snip>

> If anyone were to be able to create
> convincing gpg singatures of Apache releases, then this...
>
>> Downloading the signature from a trusted server means that
>> such an attacker would need to replace an existing signature on secure
>> hardware without detection.
>
> is moot anyway, the lesser problem to be concerned about.

Every line of defence (weak or strong) that an attacker has to breach
gives more time for defends to respond

> And this btw. is not any different than to download the torrent via https.

Modulo client respect for certificates, yes

>>> So it is not a matter of infrastructure, but a matter of policy.
>>
>> Where's the URL for this policy?
>
> I didn't mean to imply there was a set-in-stone policy already.

(Apache has quite a lot of nomenclature which is often confusing.
Policy is often used as shorthand for Apache policy.  Which is
reasonably set-in-stone.)

> What I meant was that it is up to the project to decide whether torrents are
> used or not, that the technical implementation of using torrents is so
> simple that apache infrastructure is not needed at all. You want
> torrents, you got torrents. You don't want them, you just don't use
> them.

For Apache releases, I think torrents would be do-able with a little
work (I think that seeds would need to be obtained from mirrors but
signatures from Apache)

For OOo legacy releases, my advice is just to show up and ask
Infrastructure to help solve your problem.

Robert

Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by Christian Lohmaier <cl...@openoffice.org>.
Hi Robert, *,

On Tue, Oct 25, 2011 at 5:05 PM, Robert Burrell Donkin
<ro...@gmail.com> wrote:
> On Tue, Oct 25, 2011 at 1:38 PM, Christian Lohmaier
> <cl...@openoffice.org> wrote:
>> On Tue, Oct 25, 2011 at 2:15 PM, Robert Burrell Donkin
>> <ro...@gmail.com> wrote:
>>> On Tue, Oct 25, 2011 at 12:36 PM, Christian Lohmaier
>>> <cl...@openoffice.org> wrote:
>>>> [...]
>>> Better to download the signature over HTTPS but yes, I see no reason
>>> why this approach could not be made to work
>>
>> With signature I meant a real signature (gpg signature), not a md5sum
>> or sha1sum file.
>> When it is a cryptographic signature, it doesn't matter how you
>> download it, as it cannot be faked.
>> (of course the user has to get the proper key, but that's a different issue)
>
> FWIW it's a defense in depth measure[1]
>[...]
> [1] Consider an attacker with some ability to fabricate convincing
> signatures.

Define "convincing signatures". If anyone were to be able to create
convincing gpg singatures of Apache releases, then this...

> Downloading the signature from a trusted server means that
> such an attacker would need to replace an existing signature on secure
> hardware without detection.

is moot anyway, the lesser problem to be concerned about. And this
btw. is not any different than to download the torrent via https.

>> So it is not a matter of infrastructure, but a matter of policy.
>
> Where's the URL for this policy?

I didn't mean to imply there was a set-in-stone policy already. What I
meant was that it is up to the project to decide whether torrents are
used or not, that the technical implementation of using torrents is so
simple that apache infrastructure is not needed at all. You want
torrents, you got torrents. You don't want them, you just don't use
them.

(Of course I don't know whether Apache as a whole has a written policy
or guidelines wrt. using torrents, but I don't think there is one)

ciao
Christian

Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by Robert Burrell Donkin <ro...@gmail.com>.
On Tue, Oct 25, 2011 at 1:38 PM, Christian Lohmaier
<cl...@openoffice.org> wrote:
> Hi Robert, *,
>
> On Tue, Oct 25, 2011 at 2:15 PM, Robert Burrell Donkin
> <ro...@gmail.com> wrote:
>> On Tue, Oct 25, 2011 at 12:36 PM, Christian Lohmaier
>> <cl...@openoffice.org> wrote:
>>> [...]
>>> That doesn't make sense - integrity is assured by bittorrent by
>>> providing sha1sums for each  chunk. And authenticity can be assured
>>> just like it is with regular releases - just include a corresponding
>>> signature file within the torrent.
>>
>> Better to download the signature over HTTPS but yes, I see no reason
>> why this approach could not be made to work
>
> With signature I meant a real signature (gpg signature), not a md5sum
> or sha1sum file.
> When it is a cryptographic signature, it doesn't matter how you
> download it, as it cannot be faked.
> (of course the user has to get the proper key, but that's a different issue)

FWIW it's a defense in depth measure[1]

>>>> I may have dreamed it or I am mixing this up with something else.
>>>
>>> If those were the only reasons, then they were made-up arguments.
>>
>> When engaging with Infrastructure, expect to be challenged and to have
>> to defend any proposal. These lists are open, so expect a range of
>> cluefulness from contributors. The best way to impress the core
>> infrastructure team is for plenty of clueful people from a project to
>> show up and defend the proposal with well research arguments. Giving
>> up and going away is the surest way to lose the argument...
>
> With OOo the tracker network[1] was run independently anyway and not
> hosted on the Oracle or OSUOSL hosted infrastructure. The main tracker
> was Mike's at utwente, and that mirror also was the initial/main seed
> for all the releases. There were other trackers linked together via a
> tracker-hub (backup tracker as well as the hub were provided by
> Harold).
>
> So it is not a matter of infrastructure, but a matter of policy.

Where's the URL for this policy?

Robert

[1] Consider an attacker with some ability to fabricate convincing
signatures. Downloading the signature from a trusted server means that
such an attacker would need to replace an existing signature on secure
hardware without detection. The small increase in traffic is a small
price to pay for this additional defense in depth.

Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by Christian Lohmaier <cl...@openoffice.org>.
Hi Robert, *,

On Tue, Oct 25, 2011 at 2:15 PM, Robert Burrell Donkin
<ro...@gmail.com> wrote:
> On Tue, Oct 25, 2011 at 12:36 PM, Christian Lohmaier
> <cl...@openoffice.org> wrote:
>> [...]
>> That doesn't make sense - integrity is assured by bittorrent by
>> providing sha1sums for each  chunk. And authenticity can be assured
>> just like it is with regular releases - just include a corresponding
>> signature file within the torrent.
>
> Better to download the signature over HTTPS but yes, I see no reason
> why this approach could not be made to work

With signature I meant a real signature (gpg signature), not a md5sum
or sha1sum file.
When it is a cryptographic signature, it doesn't matter how you
download it, as it cannot be faked.
(of course the user has to get the proper key, but that's a different issue)


>>> I may have dreamed it or I am mixing this up with something else.
>>
>> If those were the only reasons, then they were made-up arguments.
>
> When engaging with Infrastructure, expect to be challenged and to have
> to defend any proposal. These lists are open, so expect a range of
> cluefulness from contributors. The best way to impress the core
> infrastructure team is for plenty of clueful people from a project to
> show up and defend the proposal with well research arguments. Giving
> up and going away is the surest way to lose the argument...

With OOo the tracker network[1] was run independently anyway and not
hosted on the Oracle or OSUOSL hosted infrastructure. The main tracker
was Mike's at utwente, and that mirror also was the initial/main seed
for all the releases. There were other trackers linked together via a
tracker-hub (backup tracker as well as the hub were provided by
Harold).

So it is not a matter of infrastructure, but a matter of policy.

There's no need for the mechanism to change in my opinion. (torrents
are generated automatically as soon as they hit the mirror).

So if apache wants to setup their own bt network, they need one
capable machine (in terms of bandwidth) server to be the initial seed,
and one with almost no resources (can be the same machine of course)
to act as tracker.

[1] The trackers are *linked*, not separate, all trackers know about
every peer, so there is
no swarm fragmentation, and you got the fallback in case on of the
trackers is down
(TDF only uses one single tracker, but webseeds (traditional http/ftp
URLs) are included, so even when the tracker is down, the clients can
still use regular mirrors and DHT.)

ciao
Christian

Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by Robert Burrell Donkin <ro...@gmail.com>.
On Tue, Oct 25, 2011 at 12:36 PM, Christian Lohmaier
<cl...@openoffice.org> wrote:
> Hi Dennis, *,
>
> On Tue, Oct 25, 2011 at 2:04 AM, Dennis E. Hamilton
> <de...@acm.org> wrote:
>> I read somewhere, and I don't know where, that ASF did not want torrents to be used.
>> I'm guessing that the issue is related to ensuring the integrity and authenticity of
>> packaged releases.
>
> That doesn't make sense - integrity is assured by bittorrent by
> providing sha1sums for each  chunk. And authenticity can be assured
> just like it is with regular releases - just include a corresponding
> signature file within the torrent.

Better to download the signature over HTTPS but yes, I see no reason
why this approach could not be made to work

>> I may have dreamed it or I am mixing this up with something else.
>
> If those were the only reasons, then they were made-up arguments.

When engaging with Infrastructure, expect to be challenged and to have
to defend any proposal. These lists are open, so expect a range of
cluefulness from contributors. The best way to impress the core
infrastructure team is for plenty of clueful people from a project to
show up and defend the proposal with well research arguments. Giving
up and going away is the surest way to lose the argument...

Robert

Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by Christian Lohmaier <cl...@openoffice.org>.
Hi Dennis, *,

On Tue, Oct 25, 2011 at 2:04 AM, Dennis E. Hamilton
<de...@acm.org> wrote:
> I read somewhere, and I don't know where, that ASF did not want torrents to be used.
> I'm guessing that the issue is related to ensuring the integrity and authenticity of
> packaged releases.

That doesn't make sense - integrity is assured by bittorrent by
providing sha1sums for each  chunk. And authenticity can be assured
just like it is with regular releases - just include a corresponding
signature file within the torrent.

> I may have dreamed it or I am mixing this up with something else.

If those were the only reasons, then they were made-up arguments.

But bittorrent only makes sense for larger files anyway.

ciao
Christian

RE: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by "Dennis E. Hamilton" <de...@acm.org>.
I read somewhere, and I don't know where, that ASF did not want torrents to be used.  I'm guessing that the issue is related to ensuring the integrity and authenticity of packaged releases.  Of course, one cannot prevent folks from torrenting binaries, but it is not something ASF wanted to seed.

I may have dreamed it or I am mixing this up with something else.

Sorry for not looking around more closely, have a household IT emergency.

 - Dennis

-----Original Message-----
From: Daniel Shahaf [mailto:d.s@daniel.shahaf.name] 
Sent: Monday, October 24, 2011 16:06
To: Donald Whytock
Cc: ooo-dev@incubator.apache.org
Subject: Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Donald Whytock wrote on Mon, Oct 24, 2011 at 18:53:05 -0400:
> I didn't see anything about P2P at http://apache.org/dyn/closer.cgi.
> Does anyone know if ASF has any capability for this?

Not to my knowledge.


Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Donald Whytock wrote on Mon, Oct 24, 2011 at 18:53:05 -0400:
> I didn't see anything about P2P at http://apache.org/dyn/closer.cgi.
> Does anyone know if ASF has any capability for this?

Not to my knowledge.

Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by Donald Whytock <dw...@gmail.com>.
On Mon, Oct 24, 2011 at 6:34 PM, Simon Phipps <si...@webmink.com> wrote:
>
> On 25 Oct 2011, at 00:23, Donald Whytock wrote:
>
>> On Sun, Oct 23, 2011 at 9:08 PM, Marcus (OOo) <ma...@wtnet.de> wrote:
>>> @List:
>>> Has anybody an idea about where to host this service? It doesn't need to be
>>> necessarily inside the ASF.
>>
>> The Pirate Bay?  Lend them some legitimacy? :)
>>
>> More seriously, on the risk of being off-topic, has torrenting been
>> considered for downloads?
>
> It's been a feature of OOo for years: http://distribution.openoffice.org/p2p/  and is now also a feature at LibreOffice: http://www.libreoffice.org/download
>
> So I think that's a big "yes" and the question is how we keep it going...

I'd actually meant (and probably should have said) whether it has been
considered by ASF.  Make every mirror a seed and all that.  Or, like
in this case, have people willing to volunteer bits of space on a
short-term basis.

I didn't see anything about P2P at http://apache.org/dyn/closer.cgi.
Does anyone know if ASF has any capability for this?

Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by Simon Phipps <si...@webmink.com>.
On 25 Oct 2011, at 00:23, Donald Whytock wrote:

> On Sun, Oct 23, 2011 at 9:08 PM, Marcus (OOo) <ma...@wtnet.de> wrote:
>> @List:
>> Has anybody an idea about where to host this service? It doesn't need to be
>> necessarily inside the ASF.
> 
> The Pirate Bay?  Lend them some legitimacy? :)
> 
> More seriously, on the risk of being off-topic, has torrenting been
> considered for downloads?

It's been a feature of OOo for years: http://distribution.openoffice.org/p2p/  and is now also a feature at LibreOffice: http://www.libreoffice.org/download

So I think that's a big "yes" and the question is how we keep it going...

S.


Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by Donald Whytock <dw...@gmail.com>.
On Sun, Oct 23, 2011 at 9:08 PM, Marcus (OOo) <ma...@wtnet.de> wrote:
> @List:
> Has anybody an idea about where to host this service? It doesn't need to be
> necessarily inside the ASF.

The Pirate Bay?  Lend them some legitimacy? :)

More seriously, on the risk of being off-topic, has torrenting been
considered for downloads?

Don

Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by "Marcus (OOo)" <ma...@wtnet.de>.
Am 10/28/2011 10:02 AM, schrieb Daniel Shahaf:
> Marcus (OOo) wrote on Fri, Oct 28, 2011 at 01:12:33 +0200:
>> Why OOo is special:
>>
>> - OOo 3.3 consists of ~1.000 files with ~70 GB
>> - in the peak with 300,000 downloads per day
>
> That's the relevant part of your email.  I agree that 70GB per release

Yes, forget the other numbers as only the download part is of course the 
topic-related part.

> is quite an outlier (and will cause issues not only related to
> mirroring).
> When the PPMC wants to ask infra something or wants infra to do
> something, email us on the usual channels.

OK

Marcus

Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Marcus (OOo) wrote on Fri, Oct 28, 2011 at 01:12:33 +0200:
> Why OOo is special:
> 
> - OOo 3.3 consists of ~1.000 files with ~70 GB
> - in the peak with 300,000 downloads per day

That's the relevant part of your email.  I agree that 70GB per release
is quite an outlier (and will cause issues not only related to
mirroring).

When the PPMC wants to ask infra something or wants infra to do
something, email us on the usual channels.

Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by "Marcus (OOo)" <ma...@wtnet.de>.
Am 10/28/2011 03:38 AM, schrieb Gavin McDonald:
>
>
>> -----Original Message-----
>> From: Marcus (OOo) [mailto:marcus.mail@wtnet.de]
>> Sent: Friday, 28 October 2011 9:13 AM
>> To: ooo-dev@incubator.apache.org
>> Cc: Peter Pöml
>> Subject: Re: Shutdown of the "download.services.openoffice.org" host and
>> its Mirrorbrain instance
>>
>> Am 10/27/2011 11:36 AM, schrieb Gavin McDonald:
>>>
>>>
>>>> -----Original Message-----
>>>> From: Ross Gardler [mailto:rgardler@opendirective.com]
>>>> Sent: Thursday, 27 October 2011 10:48 AM
>>>> To: ooo-dev@incubator.apache.org
>>>> Cc: Peter Pöml
>>>> Subject: Re: Shutdown of the "download.services.openoffice.org" host
>>>> and its Mirrorbrain instance
>>>>
>>>> Sent from my mobile device, please forgive errors and brevity.
>>>> On Oct 27, 2011 1:22 AM, "Marcus (OOo)"<ma...@wtnet.de>
>> wrote:
>>>>>
>>>>> Am 10/27/2011 02:03 AM, schrieb Ross Gardler:
>>>>>
>>>>>> Sent from my mobile device, please forgive errors and brevity.
>>>>>> On Oct 27, 2011 12:37 AM, "Marcus (OOo)"<ma...@wtnet.de>
>>>> wrote:
>>>>>>>
>>>>>>> Am 10/26/2011 11:57 PM, schrieb Peter Pöml:
>>>>>>
>>>>>> ....
>>>>>>
>>>>>>>> Setting up MirrorBrain would be one way, but it would require
>>>>>>>> replication additional configuration (for instance, download
>>>> statistics)
>>>>>>>> that we have built on the current download server
>>>>>>>> (download.services.openoffice.org).
>>>>>>>>
>>>>>>>> Another way would be to simply have a virtual machine, where we
>>>>>>>> move
>>>> the
>>>>>>>> current server to. That would cause the least effort, I guess.
>>>>>>>
>>>>>>> +1 this should be really our goal for now ...
>>>>>>
>>>>>> ...
>>>>>>
>>>>>>>> Starting from scratch would mean to lose a lot of the previos
>>>>>>>> work
>>>>>>>> -- and I really mean lots, which I dare to judge because I spent
>>>>>>>> a lot of time with download.services.openoffice.org.
>>>>>>>>
>>>>>>>> On the other hand, having MirrorBrain at the core of the ASF's
>>>>>>>> mirror system could be interesting for other projects, too. I
>>>>>>>> know closer.cgi but I'm sure that MirrorBrain could serve the ASF
>> well.
>>>>>>>> (Well possible as an addition, rather than a replacement, for a
>>>>>>>> soft transition.) That might outweigh the pain of creating OOo's
>>>>>>>> download service from scratch in a different environment.
>>>>>>>
>>>>>>> ... and this the long term goal.
>>>>>>>
>>>>>>> The ASF can really benefit from this way of downloading software.
>>>>>>
>>>>>> Currently you have to choose a mirror, then change to the directory
>>>>>> structure where the respective binary is located and finally
>>>>>> download it.
>>>>>>>
>>>>>> This is not correct. I have no idea if the ASF can benefit from
>>>> MirrorBrain
>>>>>> or not, but if your justification for such a statement is based on
>>>>>> the above
>>>>>> erroneous analysis of the current mirror system then I have my
>>> concerns.
>>>> I
>>>>>
>>>>> OK, thats what I've done to come to my point:
>>>>> - browse to "http://www.apache.org/"
>>>>> - click on "Download" top right
>>>>> - choose a mirror
>>>>> - change to the dir structure to your file
>>>>> - now download
>>>>>
>>>>> Is there an easier way to get software?
>>>>
>>>> E.g. http://httpd.apache.org/download.cgi#apache22
>>>>
>>>> It auto selects the nearest mirror and provides appropriate download
>>>> links
>>> - a
>>>> single click. Want it pretty with big blue buttons? It's just HTML
>>>>
>>>> This was discussed early in the AOOo podlings existence.see archives
>>>> for.more.
>>>
>>> I would like us to move on from this Topic, I have said before and I
>>> will say it again, ASF Infra will NOT support mirrorbrain.
>>>
>>> We have one mirror system and it deals with 200 or so projects.
>>> OpenOffice project will use the current ASF supported mirror system. End
>> of story.
>>
>> If you see this like black and white then I can do this, too:
>>
>> If we don't get the support that AOO needs then we can outsource the
>> download easily. EOD
>>
>> (yes, this wasn't meant seriously)
>>
>> It's sad that you brush it simply aside without to see the chance to at
> least try
>> to improve things.
>
> Excuse me? I am doing quite a lot already thank you.
> You do not our infrastructure or how it works, I do.

And we are all very thankful for this. This wasn't meant as offense. 
Sorry if you have understood it this way.

>> Why OOo is special:
>>
>> - OOo 3.3 consists of ~1.000 files with ~70 GB
>
> Correct.
>
>> - in the peak with 300,000 downloads per day
>
> That’s fine. The mirrors will handle that.

OK, forget the other numbers as of course only this is important for the 
topic.

>> Even when we won't get back the download numbers in the middle run due
>> to the long silence of the last months and reduce the number by lets say
> 50%
>> we would be still by far the largest ASF project.
>
> Doesn’t give you the right to try and bully me or the rest of the infra team
> into
> giving you want you want. In fact, it will have the opposite effect.

Stop. I haven't bully you and never the entire infra team. Again, sorry 
if you have understood it this way.

>> But you are right ...
>>
>>> That said, let us get these other migrations done and we can see
>>> how/if the previous releases can be integrated.
>>
>> ... currently we have indeed other more important road works on the plan.
>>
>
> And we are working on them.
>
> The downloads will be sorted don’t worry, but if I can help it, it will be
> using the same
> system as all the other projects. If I am wrong about that, I will fess up
> if it happens.

Marcus

RE: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by Gavin McDonald <ga...@16degrees.com.au>.

> -----Original Message-----
> From: Marcus (OOo) [mailto:marcus.mail@wtnet.de]
> Sent: Friday, 28 October 2011 9:13 AM
> To: ooo-dev@incubator.apache.org
> Cc: Peter Pöml
> Subject: Re: Shutdown of the "download.services.openoffice.org" host and
> its Mirrorbrain instance
> 
> Am 10/27/2011 11:36 AM, schrieb Gavin McDonald:
> >
> >
> >> -----Original Message-----
> >> From: Ross Gardler [mailto:rgardler@opendirective.com]
> >> Sent: Thursday, 27 October 2011 10:48 AM
> >> To: ooo-dev@incubator.apache.org
> >> Cc: Peter Pöml
> >> Subject: Re: Shutdown of the "download.services.openoffice.org" host
> >> and its Mirrorbrain instance
> >>
> >> Sent from my mobile device, please forgive errors and brevity.
> >> On Oct 27, 2011 1:22 AM, "Marcus (OOo)"<ma...@wtnet.de>
> wrote:
> >>>
> >>> Am 10/27/2011 02:03 AM, schrieb Ross Gardler:
> >>>
> >>>> Sent from my mobile device, please forgive errors and brevity.
> >>>> On Oct 27, 2011 12:37 AM, "Marcus (OOo)"<ma...@wtnet.de>
> >> wrote:
> >>>>>
> >>>>>
> >>>>> Am 10/26/2011 11:57 PM, schrieb Peter Pöml:
> >>>>>
> >>>>>>
> >>>>
> >>>> ....
> >>>>
> >>>>>>
> >>>>>> Setting up MirrorBrain would be one way, but it would require
> >>>>>> replication additional configuration (for instance, download
> >> statistics)
> >>>>>> that we have built on the current download server
> >>>>>> (download.services.openoffice.org).
> >>>>>>
> >>>>>> Another way would be to simply have a virtual machine, where we
> >>>>>> move
> >> the
> >>>>>> current server to. That would cause the least effort, I guess.
> >>>>>
> >>>>>
> >>>>>
> >>>>> +1 this should be really our goal for now ...
> >>>>
> >>>>
> >>>> ...
> >>>>
> >>>>>
> >>>>>
> >>>>>> Starting from scratch would mean to lose a lot of the previos
> >>>>>> work
> >>>>>> -- and I really mean lots, which I dare to judge because I spent
> >>>>>> a lot of time with download.services.openoffice.org.
> >>>>>>
> >>>>>> On the other hand, having MirrorBrain at the core of the ASF's
> >>>>>> mirror system could be interesting for other projects, too. I
> >>>>>> know closer.cgi but I'm sure that MirrorBrain could serve the ASF
> well.
> >>>>>> (Well possible as an addition, rather than a replacement, for a
> >>>>>> soft transition.) That might outweigh the pain of creating OOo's
> >>>>>> download service from scratch in a different environment.
> >>>>>
> >>>>>
> >>>>>
> >>>>> ... and this the long term goal.
> >>>>>
> >>>>> The ASF can really benefit from this way of downloading software.
> >>>>
> >>>> Currently you have to choose a mirror, then change to the directory
> >>>> structure where the respective binary is located and finally
> >>>> download
> > it.
> >>>>>
> >>>>>
> >>>>
> >>>> This is not correct. I have no idea if the ASF can benefit from
> >> MirrorBrain
> >>>> or not, but if your justification for such a statement is based on
> >>>> the
> >> above
> >>>> erroneous analysis of the current mirror system then I have my
> > concerns.
> >> I
> >>>
> >>>
> >>> OK, thats what I've done to come to my point:
> >>> - browse to "http://www.apache.org/"
> >>> - click on "Download" top right
> >>> - choose a mirror
> >>> - change to the dir structure to your file
> >>> - now download
> >>>
> >>> Is there an easier way to get software?
> >>
> >> E.g. http://httpd.apache.org/download.cgi#apache22
> >>
> >> It auto selects the nearest mirror and provides appropriate download
> >> links
> > - a
> >> single click. Want it pretty with big blue buttons? It's just HTML
> >>
> >> This was discussed early in the AOOo podlings existence.see archives
> >> for.more.
> >
> > I would like us to move on from this Topic, I have said before and I
> > will say it again, ASF Infra will NOT support mirrorbrain.
> >
> > We have one mirror system and it deals with 200 or so projects.
> > OpenOffice project will use the current ASF supported mirror system. End
> of story.
> 
> If you see this like black and white then I can do this, too:
> 
> If we don't get the support that AOO needs then we can outsource the
> download easily. EOD
> 
> (yes, this wasn't meant seriously)
> 
> It's sad that you brush it simply aside without to see the chance to at
least try
> to improve things.

Excuse me? I am doing quite a lot already thank you.
You do not our infrastructure or how it works, I do.

> 
> Why OOo is special:
> 
> - over 70 committers (more than 10x of other projects, right ?)

Wrong

Subversion 66 
hadoop 64 
commons 103 
lucene 61 
cocoon 81
Geronimo 64

Shall I go on, 10x other projects where on earth did you pull that gem from?



> - several millons of code lines

LOC does not make a project special.
Perhaps as things improve with efficiency those will come down.

> - over 70 supported languages

Other project support other languages too.

> - 4 supported CPU architectures

Some projects support more than that, your point is?

> - 4 supported operating systems

Some projects support more than that, your point is?

> - OOo 3.3 consists of ~1.000 files with ~70 GB

Correct.

> - in the peak with 300,000 downloads per day

That’s fine. The mirrors will handle that.

> 
> With these short facts IMHO everybody should be able to see that this is
not
> just another project and that it needs special treatment when needed.

sorry, you already are getting special treatment in other ways, but if every
Tom Dick
and Sally from this project got exactly what they asked for, they would also
be
paying for more servers, more bandwidth and more people to look after it
all.

Does everything in this projects life involve having a tantrum when someone
has to 
say no to something? 

> 
> Even when we won't get back the download numbers in the middle run due
> to the long silence of the last months and reduce the number by lets say
50%
> we would be still by far the largest ASF project.

Doesn’t give you the right to try and bully me or the rest of the infra team
into 
giving you want you want. In fact, it will have the opposite effect.

> 
> But you are right ...
> 
> > That said, let us get these other migrations done and we can see
> > how/if the previous releases can be integrated.
> 
> ... currently we have indeed other more important road works on the plan.
> 

And we are working on them.

The downloads will be sorted don’t worry, but if I can help it, it will be
using the same
system as all the other projects. If I am wrong about that, I will fess up
if it happens.

Gav...

> Marcus



Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by "Marcus (OOo)" <ma...@wtnet.de>.
Am 10/27/2011 11:36 AM, schrieb Gavin McDonald:
>
>
>> -----Original Message-----
>> From: Ross Gardler [mailto:rgardler@opendirective.com]
>> Sent: Thursday, 27 October 2011 10:48 AM
>> To: ooo-dev@incubator.apache.org
>> Cc: Peter Pöml
>> Subject: Re: Shutdown of the "download.services.openoffice.org" host and
>> its Mirrorbrain instance
>>
>> Sent from my mobile device, please forgive errors and brevity.
>> On Oct 27, 2011 1:22 AM, "Marcus (OOo)"<ma...@wtnet.de>  wrote:
>>>
>>> Am 10/27/2011 02:03 AM, schrieb Ross Gardler:
>>>
>>>> Sent from my mobile device, please forgive errors and brevity.
>>>> On Oct 27, 2011 12:37 AM, "Marcus (OOo)"<ma...@wtnet.de>
>> wrote:
>>>>>
>>>>>
>>>>> Am 10/26/2011 11:57 PM, schrieb Peter Pöml:
>>>>>
>>>>>>
>>>>
>>>> ....
>>>>
>>>>>>
>>>>>> Setting up MirrorBrain would be one way, but it would require
>>>>>> replication additional configuration (for instance, download
>> statistics)
>>>>>> that we have built on the current download server
>>>>>> (download.services.openoffice.org).
>>>>>>
>>>>>> Another way would be to simply have a virtual machine, where we
>>>>>> move
>> the
>>>>>> current server to. That would cause the least effort, I guess.
>>>>>
>>>>>
>>>>>
>>>>> +1 this should be really our goal for now ...
>>>>
>>>>
>>>> ...
>>>>
>>>>>
>>>>>
>>>>>> Starting from scratch would mean to lose a lot of the previos work
>>>>>> -- and I really mean lots, which I dare to judge because I spent a
>>>>>> lot of time with download.services.openoffice.org.
>>>>>>
>>>>>> On the other hand, having MirrorBrain at the core of the ASF's
>>>>>> mirror system could be interesting for other projects, too. I know
>>>>>> closer.cgi but I'm sure that MirrorBrain could serve the ASF well.
>>>>>> (Well possible as an addition, rather than a replacement, for a
>>>>>> soft transition.) That might outweigh the pain of creating OOo's
>>>>>> download service from scratch in a different environment.
>>>>>
>>>>>
>>>>>
>>>>> ... and this the long term goal.
>>>>>
>>>>> The ASF can really benefit from this way of downloading software.
>>>>
>>>> Currently you have to choose a mirror, then change to the directory
>>>> structure where the respective binary is located and finally download
> it.
>>>>>
>>>>>
>>>>
>>>> This is not correct. I have no idea if the ASF can benefit from
>> MirrorBrain
>>>> or not, but if your justification for such a statement is based on
>>>> the
>> above
>>>> erroneous analysis of the current mirror system then I have my
> concerns.
>> I
>>>
>>>
>>> OK, thats what I've done to come to my point:
>>> - browse to "http://www.apache.org/"
>>> - click on "Download" top right
>>> - choose a mirror
>>> - change to the dir structure to your file
>>> - now download
>>>
>>> Is there an easier way to get software?
>>
>> E.g. http://httpd.apache.org/download.cgi#apache22
>>
>> It auto selects the nearest mirror and provides appropriate download links
> - a
>> single click. Want it pretty with big blue buttons? It's just HTML
>>
>> This was discussed early in the AOOo podlings existence.see archives
>> for.more.
>
> I would like us to move on from this Topic, I have said before and I will
> say it again, ASF Infra will NOT support mirrorbrain.
>
> We have one mirror system and it deals with 200 or so projects. OpenOffice
> project will use the current ASF supported mirror system. End of story.

If you see this like black and white then I can do this, too:

If we don't get the support that AOO needs then we can outsource the 
download easily. EOD

(yes, this wasn't meant seriously)

It's sad that you brush it simply aside without to see the chance to at 
least try to improve things.

Why OOo is special:

- over 70 committers (more than 10x of other projects, right ?)
- several millons of code lines
- over 70 supported languages
- 4 supported CPU architectures
- 4 supported operating systems
- OOo 3.3 consists of ~1.000 files with ~70 GB
- in the peak with 300,000 downloads per day

With these short facts IMHO everybody should be able to see that this is 
not just another project and that it needs special treatment when needed.

Even when we won't get back the download numbers in the middle run due 
to the long silence of the last months and reduce the number by lets say 
50% we would be still by far the largest ASF project.

But you are right ...

> That said, let us get these other migrations done and we can see how/if the
> previous releases can be integrated.

... currently we have indeed other more important road works on the plan.

Marcus


RE: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by Gavin McDonald <ga...@16degrees.com.au>.

> -----Original Message-----
> From: Ross Gardler [mailto:rgardler@opendirective.com]
> Sent: Thursday, 27 October 2011 10:48 AM
> To: ooo-dev@incubator.apache.org
> Cc: Peter Pöml
> Subject: Re: Shutdown of the "download.services.openoffice.org" host and
> its Mirrorbrain instance
> 
> Sent from my mobile device, please forgive errors and brevity.
> On Oct 27, 2011 1:22 AM, "Marcus (OOo)" <ma...@wtnet.de> wrote:
> >
> > Am 10/27/2011 02:03 AM, schrieb Ross Gardler:
> >
> >> Sent from my mobile device, please forgive errors and brevity.
> >> On Oct 27, 2011 12:37 AM, "Marcus (OOo)"<ma...@wtnet.de>
> wrote:
> >>>
> >>>
> >>> Am 10/26/2011 11:57 PM, schrieb Peter Pöml:
> >>>
> >>>>
> >>
> >> ....
> >>
> >>>>
> >>>> Setting up MirrorBrain would be one way, but it would require
> >>>> replication additional configuration (for instance, download
> statistics)
> >>>> that we have built on the current download server
> >>>> (download.services.openoffice.org).
> >>>>
> >>>> Another way would be to simply have a virtual machine, where we
> >>>> move
> the
> >>>> current server to. That would cause the least effort, I guess.
> >>>
> >>>
> >>>
> >>> +1 this should be really our goal for now ...
> >>
> >>
> >> ...
> >>
> >>>
> >>>
> >>>> Starting from scratch would mean to lose a lot of the previos work
> >>>> -- and I really mean lots, which I dare to judge because I spent a
> >>>> lot of time with download.services.openoffice.org.
> >>>>
> >>>> On the other hand, having MirrorBrain at the core of the ASF's
> >>>> mirror system could be interesting for other projects, too. I know
> >>>> closer.cgi but I'm sure that MirrorBrain could serve the ASF well.
> >>>> (Well possible as an addition, rather than a replacement, for a
> >>>> soft transition.) That might outweigh the pain of creating OOo's
> >>>> download service from scratch in a different environment.
> >>>
> >>>
> >>>
> >>> ... and this the long term goal.
> >>>
> >>> The ASF can really benefit from this way of downloading software.
> >>
> >> Currently you have to choose a mirror, then change to the directory
> >> structure where the respective binary is located and finally download
it.
> >>>
> >>>
> >>
> >> This is not correct. I have no idea if the ASF can benefit from
> MirrorBrain
> >> or not, but if your justification for such a statement is based on
> >> the
> above
> >> erroneous analysis of the current mirror system then I have my
concerns.
> I
> >
> >
> > OK, thats what I've done to come to my point:
> > - browse to "http://www.apache.org/"
> > - click on "Download" top right
> > - choose a mirror
> > - change to the dir structure to your file
> > - now download
> >
> > Is there an easier way to get software?
> 
> E.g. http://httpd.apache.org/download.cgi#apache22
> 
> It auto selects the nearest mirror and provides appropriate download links
- a
> single click. Want it pretty with big blue buttons? It's just HTML
> 
> This was discussed early in the AOOo podlings existence.see archives
> for.more.

I would like us to move on from this Topic, I have said before and I will
say it again,
ASF Infra will NOT support mirrorbrain.

We have one mirror system and it deals with 200 or so projects. OpenOffice
project will
use the current ASF supported mirror system. End of story.

That said, let us get these other migrations done and we can see how/if the
previous
releases can be integrated.

Gav...



Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by Ross Gardler <rg...@opendirective.com>.
Sent from my mobile device, please forgive errors and brevity.
On Oct 27, 2011 1:22 AM, "Marcus (OOo)" <ma...@wtnet.de> wrote:
>
> Am 10/27/2011 02:03 AM, schrieb Ross Gardler:
>
>> Sent from my mobile device, please forgive errors and brevity.
>> On Oct 27, 2011 12:37 AM, "Marcus (OOo)"<ma...@wtnet.de>  wrote:
>>>
>>>
>>> Am 10/26/2011 11:57 PM, schrieb Peter Pöml:
>>>
>>>>
>>
>> ....
>>
>>>>
>>>> Setting up MirrorBrain would be one way, but it would require
>>>> replication additional configuration (for instance, download
statistics)
>>>> that we have built on the current download server
>>>> (download.services.openoffice.org).
>>>>
>>>> Another way would be to simply have a virtual machine, where we move
the
>>>> current server to. That would cause the least effort, I guess.
>>>
>>>
>>>
>>> +1 this should be really our goal for now ...
>>
>>
>> ...
>>
>>>
>>>
>>>> Starting from scratch would mean to lose a lot of the previos work --
>>>> and I really mean lots, which I dare to judge because I spent a lot of
>>>> time with download.services.openoffice.org.
>>>>
>>>> On the other hand, having MirrorBrain at the core of the ASF's mirror
>>>> system could be interesting for other projects, too. I know closer.cgi
>>>> but I'm sure that MirrorBrain could serve the ASF well. (Well possible
>>>> as an addition, rather than a replacement, for a soft transition.)
>>>> That might outweigh the pain of creating OOo's download service from
>>>> scratch in a different environment.
>>>
>>>
>>>
>>> ... and this the long term goal.
>>>
>>> The ASF can really benefit from this way of downloading software.
>>
>> Currently you have to choose a mirror, then change to the directory
>> structure where the respective binary is located and finally download it.
>>>
>>>
>>
>> This is not correct. I have no idea if the ASF can benefit from
MirrorBrain
>> or not, but if your justification for such a statement is based on the
above
>> erroneous analysis of the current mirror system then I have my concerns.
I
>
>
> OK, thats what I've done to come to my point:
> - browse to "http://www.apache.org/"
> - click on "Download" top right
> - choose a mirror
> - change to the dir structure to your file
> - now download
>
> Is there an easier way to get software?

E.g. http://httpd.apache.org/download.cgi#apache22

It auto selects the nearest mirror and provides appropriate download links -
a single click. Want it pretty with big blue buttons? It's just HTML

This was discussed early in the AOOo podlings existence.see archives
for.more.

Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by "Marcus (OOo)" <ma...@wtnet.de>.
Am 10/27/2011 02:03 AM, schrieb Ross Gardler:
> Sent from my mobile device, please forgive errors and brevity.
> On Oct 27, 2011 12:37 AM, "Marcus (OOo)"<ma...@wtnet.de>  wrote:
>>
>> Am 10/26/2011 11:57 PM, schrieb Peter Pöml:
>>
>>>
>
> ....
>
>>>
>>> Setting up MirrorBrain would be one way, but it would require
>>> replication additional configuration (for instance, download statistics)
>>> that we have built on the current download server
>>> (download.services.openoffice.org).
>>>
>>> Another way would be to simply have a virtual machine, where we move the
>>> current server to. That would cause the least effort, I guess.
>>
>>
>> +1 this should be really our goal for now ...
>
> ...
>
>>
>>
>>> Starting from scratch would mean to lose a lot of the previos work --
>>> and I really mean lots, which I dare to judge because I spent a lot of
>>> time with download.services.openoffice.org.
>>>
>>> On the other hand, having MirrorBrain at the core of the ASF's mirror
>>> system could be interesting for other projects, too. I know closer.cgi
>>> but I'm sure that MirrorBrain could serve the ASF well. (Well possible
>>> as an addition, rather than a replacement, for a soft transition.)
>>> That might outweigh the pain of creating OOo's download service from
>>> scratch in a different environment.
>>
>>
>> ... and this the long term goal.
>>
>> The ASF can really benefit from this way of downloading software.
> Currently you have to choose a mirror, then change to the directory
> structure where the respective binary is located and finally download it.
>>
>
> This is not correct. I have no idea if the ASF can benefit from MirrorBrain
> or not, but if your justification for such a statement is based on the above
> erroneous analysis of the current mirror system then I have my concerns. I

OK, thats what I've done to come to my point:
- browse to "http://www.apache.org/"
- click on "Download" top right
- choose a mirror
- change to the dir structure to your file
- now download

Is there an easier way to get software?

> don't want to discuss it now since you indicate this is a long term goal
> (search this lists archives if you want to read more on the ASF mirror
> system before bringing this to the infra team).

Of course, first we need to clean our own home before our village is 
next. ;-)

Marcus

Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by Ross Gardler <rg...@opendirective.com>.
Sent from my mobile device, please forgive errors and brevity.
On Oct 27, 2011 12:37 AM, "Marcus (OOo)" <ma...@wtnet.de> wrote:
>
> Am 10/26/2011 11:57 PM, schrieb Peter Pöml:
>
>>

....

>>
>> Setting up MirrorBrain would be one way, but it would require
>> replication additional configuration (for instance, download statistics)
>> that we have built on the current download server
>> (download.services.openoffice.org).
>>
>> Another way would be to simply have a virtual machine, where we move the
>> current server to. That would cause the least effort, I guess.
>
>
> +1 this should be really our goal for now ...

...

>
>
>> Starting from scratch would mean to lose a lot of the previos work --
>> and I really mean lots, which I dare to judge because I spent a lot of
>> time with download.services.openoffice.org.
>>
>> On the other hand, having MirrorBrain at the core of the ASF's mirror
>> system could be interesting for other projects, too. I know closer.cgi
>> but I'm sure that MirrorBrain could serve the ASF well. (Well possible
>> as an addition, rather than a replacement, for a soft transition.)
>> That might outweigh the pain of creating OOo's download service from
>> scratch in a different environment.
>
>
> ... and this the long term goal.
>
> The ASF can really benefit from this way of downloading software.
Currently you have to choose a mirror, then change to the directory
structure where the respective binary is located and finally download it.
>

This is not correct. I have no idea if the ASF can benefit from MirrorBrain
or not, but if your justification for such a statement is based on the above
erroneous analysis of the current mirror system then I have my concerns. I
don't want to discuss it now since you indicate this is a long term goal
(search this lists archives if you want to read more on the ASF mirror
system before bringing this to the infra team).

Ross

Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by "Marcus (OOo)" <ma...@wtnet.de>.
Am 10/26/2011 11:57 PM, schrieb Peter Pöml:
> Hi,
>
> On Wed, Oct 26, 2011 at 01:43:20 -0700, Dave Fisher wrote:
>> Are there requirements for more than what is on the mirrorbrain page?
>
> The page should list all essential requirements.
>
>> http://mirrorbrain.org/requirements/
>>
>> Extracting the list gives:
>>
>> Apache HTTPD server
>> mod_mirrorbrain
>> mod_form
>> mod_geoip / libGeoIP
>> PostgreSQL (or Apache DBD API compatible driver)
>
> While the DBD framework would allow to use other databases, MirrorBrain
> depends on features that only PostgreSQL offers. So this is a must, I'm
> sorry ;-)
>
>> mod_asn ?
>
> Highly useful when there are many mirrors, and when downloads are large.
> Both is the case with OOo. Thus, we have used mod_asn for OOo downloads
> so far and I would really recommend to continue using it.
>
>> If we are on ASF hardware then we won't want to serve a local copy:
>>
>> So, "Alternatively, a pseudo file tree can created locally. In that case, MirrorBrain needs to be configured to never deliver files directly."
>>
>> http://mirrorbrain.org/archive/mirrorbrain/0045.html
>
> MirrorBrain is more than a file server or redirector. It automates
> the creation of checksums, hashes, torrents, metalinks and serves
> cryptographic signatures. Files need to be present locally to exploit
> these features. These features are useful for OOo because downloads are
> generally large, which requires reliable transfer.
>
>>>> @List:
>>>> Has anybody an idea about where to host this service? It doesn't need to
>>>> be necessarily inside the ASF.
>>>
>>> Really nobody?
>>
>> If the mirrorbrain instructions are that simple and can be easily supplemented with detail.
>
> Setting up MirrorBrain would be one way, but it would require
> replication additional configuration (for instance, download statistics)
> that we have built on the current download server
> (download.services.openoffice.org).
>
> Another way would be to simply have a virtual machine, where we move the
> current server to. That would cause the least effort, I guess.

+1 this should be really our goal for now ...

> Starting from scratch would mean to lose a lot of the previos work --
> and I really mean lots, which I dare to judge because I spent a lot of
> time with download.services.openoffice.org.
>
> On the other hand, having MirrorBrain at the core of the ASF's mirror
> system could be interesting for other projects, too. I know closer.cgi
> but I'm sure that MirrorBrain could serve the ASF well. (Well possible
> as an addition, rather than a replacement, for a soft transition.)
> That might outweigh the pain of creating OOo's download service from
> scratch in a different environment.

... and this the long term goal.

The ASF can really benefit from this way of downloading software. 
Currently you have to choose a mirror, then change to the directory 
structure where the respective binary is located and finally download it.

OK, the target audience the ASF currently has is developer-oriented, so 
this is not too difficult for them. ;-) However, it could be indeed easier.

But I try to think more as an enduser. And these people need to have a 
simple solution. That means a link on a webpage where they click on and 
the download of the file is starting. IMHO we have already a very simple 
solution you can see on "http://download.openoffice.org" and I would be 
happy to help to buildup a similar thing for the entire ASF.

@Donald:
Of course we can keep the current download behavior to satisfy also 
users that needs to download from a specific mirror server.

Marcus



> Are you involved in maintaining ASF infrastructure, or know people who
> are?
>
>> Like what database is currently being used on download.s.oo.o? and which version?
>
> Uhm, PostgreSQL 8.4.4. (But any 8.3/8.4 version should do.)
>
> Thanks,
> Peter

Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by "Marcus (OOo)" <ma...@wtnet.de>.
Am 10/27/2011 12:43 AM, schrieb Donald Whytock:
> On Wed, Oct 26, 2011 at 5:57 PM, Peter Pöml<pe...@poeml.de>  wrote:
>> The page should list all essential requirements.
>>
>>> http://mirrorbrain.org/requirements/
>
> Does MirrorBrain allow downloaders to specify the mirror they want to
> use?  I didn't see that as a feature.
>
> This is necessary for people operating behind firewalls that block
> websites based on type.  For example, my company's firewall doesn't
> allow access to many filesharing sites and certain types of
> businesses, but does allow access to universities.  This means when I
> get an update for an Apache product I often need to manually select a
> university mirror.

I would say not directly but it's possible:

Example:
If you want to download the OOo 3.4 Beta you normally would download 
from here for your favorite OS and language:
http://download.openoffice.org/all_beta.html

When the automatically chosen mirror is not suitable, then you can use 
the "Details" webpage that MirrorBrain creates for every file to look 
for other mirrors. E.g.:

http://download.services.openoffice.org/files/extended/3.4beta/

http://download.services.openoffice.org/files/extended/3.4beta/OOo-Dev_3.4beta_20110411_Linux_x86-64_install-rpm_en-US.tar.gz.mirrorlist

It's not that obvious but IMHO simple enough.

But I guess even this could be more simplified. ;-)

Marcus

Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by Donald Whytock <dw...@gmail.com>.
On Wed, Oct 26, 2011 at 5:57 PM, Peter Pöml <pe...@poeml.de> wrote:
> The page should list all essential requirements.
>
>> http://mirrorbrain.org/requirements/

Does MirrorBrain allow downloaders to specify the mirror they want to
use?  I didn't see that as a feature.

This is necessary for people operating behind firewalls that block
websites based on type.  For example, my company's firewall doesn't
allow access to many filesharing sites and certain types of
businesses, but does allow access to universities.  This means when I
get an update for an Apache product I often need to manually select a
university mirror.

Don

Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by Peter Pöml <pe...@poeml.de>.
Hi,

On Wed, Oct 26, 2011 at 01:43:20 -0700, Dave Fisher wrote:
> Are there requirements for more than what is on the mirrorbrain page?

The page should list all essential requirements.

> http://mirrorbrain.org/requirements/
> 
> Extracting the list gives:
> 
> Apache HTTPD server
> mod_mirrorbrain
> mod_form
> mod_geoip / libGeoIP
> PostgreSQL (or Apache DBD API compatible driver)

While the DBD framework would allow to use other databases, MirrorBrain
depends on features that only PostgreSQL offers. So this is a must, I'm
sorry ;-)

> mod_asn ?

Highly useful when there are many mirrors, and when downloads are large.
Both is the case with OOo. Thus, we have used mod_asn for OOo downloads
so far and I would really recommend to continue using it.

> If we are on ASF hardware then we won't want to serve a local copy:
> 
> So, "Alternatively, a pseudo file tree can created locally. In that case, MirrorBrain needs to be configured to never deliver files directly."
> 
> http://mirrorbrain.org/archive/mirrorbrain/0045.html

MirrorBrain is more than a file server or redirector. It automates
the creation of checksums, hashes, torrents, metalinks and serves
cryptographic signatures. Files need to be present locally to exploit
these features. These features are useful for OOo because downloads are
generally large, which requires reliable transfer. 

> >> @List:
> >> Has anybody an idea about where to host this service? It doesn't need to
> >> be necessarily inside the ASF.
> > 
> > Really nobody?
> 
> If the mirrorbrain instructions are that simple and can be easily supplemented with detail.

Setting up MirrorBrain would be one way, but it would require
replication additional configuration (for instance, download statistics)
that we have built on the current download server
(download.services.openoffice.org). 

Another way would be to simply have a virtual machine, where we move the
current server to. That would cause the least effort, I guess. 

Starting from scratch would mean to lose a lot of the previos work --
and I really mean lots, which I dare to judge because I spent a lot of
time with download.services.openoffice.org.

On the other hand, having MirrorBrain at the core of the ASF's mirror
system could be interesting for other projects, too. I know closer.cgi
but I'm sure that MirrorBrain could serve the ASF well. (Well possible
as an addition, rather than a replacement, for a soft transition.)
That might outweigh the pain of creating OOo's download service from
scratch in a different environment. 

Are you involved in maintaining ASF infrastructure, or know people who
are?

> Like what database is currently being used on download.s.oo.o? and which version?

Uhm, PostgreSQL 8.4.4. (But any 8.3/8.4 version should do.)

Thanks,
Peter

Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by Dave Fisher <da...@comcast.net>.
On Oct 26, 2011, at 12:11 PM, Marcus (OOo) wrote:

> Am 10/24/2011 03:08 AM, schrieb Marcus (OOo):
>> Am 10/23/2011 11:50 PM, schrieb Peter Pöml:
>>> On Sat, Oct 22, 2011 at 01:15:34 +0200, Marcus (OOo) wrote:
>> 
>> Hi Peter,
>> 
>>>> it seems the end is near and the "download.services.openoffice.org"
>>>> host on Oracle side with our Mirrorbrain instance will be shutdown
>>>> in one week.
>>> 
>>> Okay... Little question, wouldn't it make sense, at this point, to take
>>> the download.s.o.o VM and move it somewhere else? I don't have the disk
>>> space myself, but maybe somebody else has? Perhaps the ASF can provide
>>> the space? (Other than disk space, the requirements are pretty modest.)
>>> Then the entire download service could continue to run, with very low
>>> effort, and without the need to build everything from scratch for now.
>> 
>> The problem is that the ASF do not want to host and provide services of
>> special software for single projects. I can understand this as even the
>> ASF infra is a team of volunteers and their time is limited as it is for
>> all others.

They are building a lot special for AOOo. THANKS!

So, it depends. If we define a stack for services and get support then it is a lot like the MWiki and Forums. Infrastructure will help secure the servers, but the project will need to provide sysadmins.

I've heard that "Jails" are available.

>> 
>> Furthermore, I don't know details about the VM setup and where it's
>> detailed located. Do you?
>> 
>> However, I like your idea. It requires indeed only a bit diskspace and
>> internet access. The maintainance could be done by us, the project
>> members. If you would share a bit of your knowledge then I could
>> takeover the admin role. ;-)
> 
> @Peter:
> Can you tell me how to get to the VM? Or at least which important config files to save?

Are there requirements for more than what is on the mirrorbrain page?

http://mirrorbrain.org/requirements/

Extracting the list gives:

Apache HTTPD server
mod_mirrorbrain
mod_form
mod_geoip / libGeoIP
PostgreSQL (or Apache DBD API compatible driver)

mod_asn ?

	• Python and some modules: psycopg2, sqlobject, cmdln.
	• Perl and some modules: Config::IniFiles, libwww::perl, DBD::Pg, Digest::MD4, Date::Parse

If we are on ASF hardware then we won't want to serve a local copy:

So, "Alternatively, a pseudo file tree can created locally. In that case, MirrorBrain needs to be configured to never deliver files directly."

http://mirrorbrain.org/archive/mirrorbrain/0045.html

>> @List:
>> Has anybody an idea about where to host this service? It doesn't need to
>> be necessarily inside the ASF.
> 
> Really nobody?

If the mirrorbrain instructions are that simple and can be easily supplemented with detail.

Like what database is currently being used on download.s.oo.o? and which version?


> 
>>>> As we cannot buildup a solution that is running, tested and
>>>> long-term-proven in a few days I would like to ask you if we can
>>>> switch to your "openoffice.mirrorbrain.org" instance as long as we
>>>> have no other solution at hand.
>>>> 
>>>> In the past it was a very reliable host that we have used when there
>>>> were outages (mostly unplanned as you know) and doesn't resulted in
>>>> a significant higher load on the host.
>>>> 
>>>> So, it would help us very much to have a stable download section
>>>> until we have an own solution here at ASF.
>>> 
>>> Yes, please use my host for now!
>> 
>> Great, thanks a lot.
>> 
>>>> Thanks in advance and have a nice, sunny weekend.
>>> 
>>> Thanks a lot. Same to you all,
> 
> Marcus

Regards,
Dave


Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by "Marcus (OOo)" <ma...@wtnet.de>.
Am 10/24/2011 03:08 AM, schrieb Marcus (OOo):
> Am 10/23/2011 11:50 PM, schrieb Peter Pöml:
>> On Sat, Oct 22, 2011 at 01:15:34 +0200, Marcus (OOo) wrote:
>
> Hi Peter,
>
>>> it seems the end is near and the "download.services.openoffice.org"
>>> host on Oracle side with our Mirrorbrain instance will be shutdown
>>> in one week.
>>
>> Okay... Little question, wouldn't it make sense, at this point, to take
>> the download.s.o.o VM and move it somewhere else? I don't have the disk
>> space myself, but maybe somebody else has? Perhaps the ASF can provide
>> the space? (Other than disk space, the requirements are pretty modest.)
>> Then the entire download service could continue to run, with very low
>> effort, and without the need to build everything from scratch for now.
>
> The problem is that the ASF do not want to host and provide services of
> special software for single projects. I can understand this as even the
> ASF infra is a team of volunteers and their time is limited as it is for
> all others.
>
> Furthermore, I don't know details about the VM setup and where it's
> detailed located. Do you?
>
> However, I like your idea. It requires indeed only a bit diskspace and
> internet access. The maintainance could be done by us, the project
> members. If you would share a bit of your knowledge then I could
> takeover the admin role. ;-)

@Peter:
Can you tell me how to get to the VM? Or at least which important config 
files to save?

> @List:
> Has anybody an idea about where to host this service? It doesn't need to
> be necessarily inside the ASF.

Really nobody?

>>> As we cannot buildup a solution that is running, tested and
>>> long-term-proven in a few days I would like to ask you if we can
>>> switch to your "openoffice.mirrorbrain.org" instance as long as we
>>> have no other solution at hand.
>>>
>>> In the past it was a very reliable host that we have used when there
>>> were outages (mostly unplanned as you know) and doesn't resulted in
>>> a significant higher load on the host.
>>>
>>> So, it would help us very much to have a stable download section
>>> until we have an own solution here at ASF.
>>
>> Yes, please use my host for now!
>
> Great, thanks a lot.
>
>>> Thanks in advance and have a nice, sunny weekend.
>>
>> Thanks a lot. Same to you all,

Marcus

Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by "Marcus (OOo)" <ma...@wtnet.de>.
Am 10/25/2011 02:02 PM, schrieb Robert Burrell Donkin:
> On Mon, Oct 24, 2011 at 2:08 AM, Marcus (OOo)<ma...@wtnet.de>  wrote:
>
> <snip>
>
>> The problem is that the ASF do not want to host and provide services of
>> special software for single projects. I can understand this as even the ASF
>> infra is a team of volunteers and their time is limited as it is for all
>> others.
>
> I think this is a little open to misinterpretation. Hopefully a Mentor
> will jump in but (until they do) I'll do my best to explain a little
> bit more about the way infrastructure works here at Apache...
>
> The infrastructure team at Apache is an independent, volunteer-led
> self-organising community of experts. Apache delegates infrastructure
> to this community, and provides resources to sustain their work[1].
> When asking infrastructure for help, it's essential to remember this
> and engage with them as peers with special expertise. Anyone arriving
> with a solution or a request for a new service must expect to be
> challenged to defend and refine their choice of solution.

Thanks for the explaination. I thought it would be a kind of just 
another project. But it's not.

> To move back to the particular, this is a migration issue. A valuable
> service is about to be closed and needs to be migrated. Whether this
> is right long term solution is open to debate but accepting a service
> for a temporary period doesn't raise the issues that committing to
> provide a similar service for all projects forever would. Please
> explain the problem to infrastructure and ask for their help to find a
> solution.

I know a service like MirrorBrain could be a high value of any Apache 
project that provides file downloads, so it should be helpful for every 
project.

As it is able keep the overview of all possible mirrors that offers 
Apache aoftware there needs to be only a single instance and not per 
project.

Let's see what to get until Friday. Otherwise he have to start again.

Marcus



> [1] The team has a budget and some flexibility to bring additional
> resources - included hired help - when needed. Apache has adequate
> financial resources but is culturally resistant to committing to
> additional spending without good reason. Apache values independence.
> Dependency on funding risks that independence.

Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by Robert Burrell Donkin <ro...@gmail.com>.
On Mon, Oct 24, 2011 at 2:08 AM, Marcus (OOo) <ma...@wtnet.de> wrote:

<snip>

> The problem is that the ASF do not want to host and provide services of
> special software for single projects. I can understand this as even the ASF
> infra is a team of volunteers and their time is limited as it is for all
> others.

I think this is a little open to misinterpretation. Hopefully a Mentor
will jump in but (until they do) I'll do my best to explain a little
bit more about the way infrastructure works here at Apache...

The infrastructure team at Apache is an independent, volunteer-led
self-organising community of experts. Apache delegates infrastructure
to this community, and provides resources to sustain their work[1].
When asking infrastructure for help, it's essential to remember this
and engage with them as peers with special expertise. Anyone arriving
with a solution or a request for a new service must expect to be
challenged to defend and refine their choice of solution.


To move back to the particular, this is a migration issue. A valuable
service is about to be closed and needs to be migrated. Whether this
is right long term solution is open to debate but accepting a service
for a temporary period doesn't raise the issues that committing to
provide a similar service for all projects forever would. Please
explain the problem to infrastructure and ask for their help to find a
solution.

Robert

[1] The team has a budget and some flexibility to bring additional
resources - included hired help - when needed. Apache has adequate
financial resources but is culturally resistant to committing to
additional spending without good reason. Apache values independence.
Dependency on funding risks that independence.

Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by "Marcus (OOo)" <ma...@wtnet.de>.
Am 10/23/2011 11:50 PM, schrieb Peter Pöml:
> On Sat, Oct 22, 2011 at 01:15:34 +0200, Marcus (OOo) wrote:

Hi Peter,

>> it seems the end is near and the "download.services.openoffice.org"
>> host on Oracle side with our Mirrorbrain instance will be shutdown
>> in one week.
>
> Okay... Little question, wouldn't it make sense, at this point, to take
> the download.s.o.o VM and move it somewhere else? I don't have the disk
> space myself, but maybe somebody else has? Perhaps the ASF can provide
> the space? (Other than disk space, the requirements are pretty modest.)
> Then the entire download service could continue to run, with very low
> effort, and without the need to build everything from scratch for now.

The problem is that the ASF do not want to host and provide services of 
special software for single projects. I can understand this as even the 
ASF infra is a team of volunteers and their time is limited as it is for 
all others.

Furthermore, I don't know details about the VM setup and where it's 
detailed located. Do you?

However, I like your idea. It requires indeed only a bit diskspace and 
internet access. The maintainance could be done by us, the project 
members. If you would share a bit of your knowledge then I could 
takeover the admin role. ;-)

@List:
Has anybody an idea about where to host this service? It doesn't need to 
be necessarily inside the ASF.

>> As we cannot buildup a solution that is running, tested and
>> long-term-proven in a few days I would like to ask you if we can
>> switch to your "openoffice.mirrorbrain.org" instance as long as we
>> have no other solution at hand.
>>
>> In the past it was a very reliable host that we have used when there
>> were outages (mostly unplanned as you know) and doesn't resulted in
>> a significant higher load on the host.
>>
>> So, it would help us very much to have a stable download section
>> until we have an own solution here at ASF.
>
> Yes, please use my host for now!

Great, thanks a lot.

>> Thanks in advance and have a nice, sunny weekend.
>
> Thanks a lot. Same to you all,

Marcus

Re: Shutdown of the "download.services.openoffice.org" host and its Mirrorbrain instance

Posted by Peter Pöml <pe...@poeml.de>.
Hi Marcus, 
hi list,

On Sat, Oct 22, 2011 at 01:15:34 +0200, Marcus (OOo) wrote:
> Hi Peter,
> 
> it seems the end is near and the "download.services.openoffice.org"
> host on Oracle side with our Mirrorbrain instance will be shutdown
> in one week.

Okay... Little question, wouldn't it make sense, at this point, to take
the download.s.o.o VM and move it somewhere else? I don't have the disk
space myself, but maybe somebody else has? Perhaps the ASF can provide
the space? (Other than disk space, the requirements are pretty modest.)
Then the entire download service could continue to run, with very low
effort, and without the need to build everything from scratch for now.

> As we cannot buildup a solution that is running, tested and
> long-term-proven in a few days I would like to ask you if we can
> switch to your "openoffice.mirrorbrain.org" instance as long as we
> have no other solution at hand.
> 
> In the past it was a very reliable host that we have used when there
> were outages (mostly unplanned as you know) and doesn't resulted in
> a significant higher load on the host.
> 
> So, it would help us very much to have a stable download section
> until we have an own solution here at ASF.

Yes, please use my host for now!

> Thanks in advance and have a nice, sunny weekend.

Thanks a lot. Same to you all,
Peter