You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@uima.apache.org by Marshall Schor <ms...@schor.com> on 2007/12/22 02:46:50 UTC
Where to put large generated things for our website to access
Here are some examples of large files (or large collections of files
The api java docs;
The api java docs as a zip file
The 4 books in html format
It seems good to put them in people.a.o/www/incubator.a.o/uima/downloads.
It seems bad to put them in SVN (because there's no need for
"versioning" these - they're generated, and they are big, taking up SVN
space).
Our current strategy is hybrid:
1) for current release only: api java docs and the api java docs.zip
file are put in people.a.o/www/incubator.a.o/uima/downloads, and are
*not* kept in SVN.
2) for current release only: the 4 books in html format are put in SVN
and copied to people.a.o/www/incubator.a.o/uima/downloads with the svn
update command.
I see, in fact that for release 2.2.0, we managed to put the books in
html format into SVN twice - once under
2.2.0-incubating/docs/html, and once under
2.2.0-incubating/html
and of course, on the website, it shows up twice also... not good.
Is there any automated process for getting the files installed on
people.a.o/www/incubator.a.o/uima/downloads? (Has anyone done any
scripts for this)?
Does everyone agree that it's best to keep these out of SVN, and to put
them in the web server spot on people.a.o/www/incubator.a.o/uima/downloads?
===================
The mirrored distribution spot contains, in addition to /binaries and
/source, a /docs directory with the the following:
<release>/apiDocs.zip , plus the 3 "signing" files [asc, md5, sha1]
<release>/api /.... <-- unzipped set of javaDoc html files, no
signing files
<release>/html/..... <-- set of 4 books as html files, no signing files
<release>/pdf/..... <-- set of 4 books as pdfs, no signing files
I think everything that's put onto the mirroring system is supposed to
be signed, because Apache doesn't "control" what goes on at the mirrors
(e.g., they could be "hacked").
Currently, our download page is slient about the existance of these.
I think we should delete these on the mirroring distribution system.
Assuming we followed the top part of this note, we would have everything
(except the pdf form of the 4 books) on the UIMA website, directly (not
going thru a mirror).
Other opinions?
-Marshall
Re: Where to put large generated things for our website to access
Posted by Marshall Schor <ms...@schor.com>.
One more consideration:
Some people use a URL to javadocs as part of their javadoc build
process. For that, we might want to consider if we want to support
people doing that - and if so, we probably need to keep the javadocs for
each version, "forever".
The dist system supports this, via archiving + redirects.
-Marshall
Marshall Schor wrote:
> Here are some examples of large files (or large collections of files
>
> The api java docs;
> The api java docs as a zip file
> The 4 books in html format
>
> It seems good to put them in people.a.o/www/incubator.a.o/uima/downloads.
>
> It seems bad to put them in SVN (because there's no need for
> "versioning" these - they're generated, and they are big, taking up SVN
> space).
>
> Our current strategy is hybrid:
>
> 1) for current release only: api java docs and the api java docs.zip
> file are put in people.a.o/www/incubator.a.o/uima/downloads, and are
> *not* kept in SVN.
> 2) for current release only: the 4 books in html format are put in SVN
> and copied to people.a.o/www/incubator.a.o/uima/downloads with the svn
> update command.
>
> I see, in fact that for release 2.2.0, we managed to put the books in
> html format into SVN twice - once under
> 2.2.0-incubating/docs/html, and once under
> 2.2.0-incubating/html
> and of course, on the website, it shows up twice also... not good.
>
> Is there any automated process for getting the files installed on
> people.a.o/www/incubator.a.o/uima/downloads? (Has anyone done any
> scripts for this)?
>
> Does everyone agree that it's best to keep these out of SVN, and to put
> them in the web server spot on people.a.o/www/incubator.a.o/uima/downloads?
>
> ===================
>
> The mirrored distribution spot contains, in addition to /binaries and
> /source, a /docs directory with the the following:
> <release>/apiDocs.zip , plus the 3 "signing" files [asc, md5, sha1]
> <release>/api /.... <-- unzipped set of javaDoc html files, no
> signing files
> <release>/html/..... <-- set of 4 books as html files, no signing files
> <release>/pdf/..... <-- set of 4 books as pdfs, no signing files
>
> I think everything that's put onto the mirroring system is supposed to
> be signed, because Apache doesn't "control" what goes on at the mirrors
> (e.g., they could be "hacked").
>
> Currently, our download page is slient about the existance of these.
>
> I think we should delete these on the mirroring distribution system.
> Assuming we followed the top part of this note, we would have everything
> (except the pdf form of the 4 books) on the UIMA website, directly (not
> going thru a mirror).
>
> Other opinions?
>
> -Marshall
>
>
>
>
Re: Where to put large generated things for our website to access
Posted by Robert Burrell Donkin <ro...@gmail.com>.
On Jan 2, 2008 4:05 PM, Marshall Schor <ms...@schor.com> wrote:
<snip>
> Since we want our docs pages to refer not only to the "current release"
> but also previous releases, it would be good to figure out a fairly
> automatic system for this. (That was a virtue of the /dist/ - archive
> system - we could point the previous releases doc links to a directory
> containing all the releases, and wouldn't need to update this link for
> subsequent releases).
>
> The other thing to do is to figure out how to keep the archived things
> on our web-site. It was suggested that we should do this like we handle
> the web-site checkout.
> Probably the straight-forward thing to do is to have a special directory
> where all the docs we want to refer to live, have the archive link point
> there, and have special links that refer to the "current version".
probably better to use a symbolic link rather than a .htaccess
> As I recall, the web-site, itself, is replicated to other servers (since
> after you update it , you have to wait a while for it to appear).
yep
> I have to confess that this seems quite wasteful of disk resources
> (double+ copies of things like javadocs - one in SVN, one on
> people.apache.org in our web-site place, and maybe (several?) additional
> copies on web-servers used for incubator.apache.org/uima).
disc's cheap :-)
bandwidth and server load are usually the most important factors
AIUI having lots of duplicates allows low downtime. even if minotaur
takes a tumble, ajax is able to step up almost immediately.
> But Robert Donkin suggested this was the best way.
i talked it over the the infra team and their strong recommendation
was to store in svn and then checkout onto the website
- robert
Re: Where to put large generated things for our website to access
Posted by Marshall Schor <ms...@schor.com>.
Michael Baessler wrote:
> Marshall Schor wrote:
>> Michael Baessler wrote:
>>
>>> Robert Burrell Donkin wrote:
>>>
>>>> subversion really is the way to go for release documentation
>>>>
>>> OK, so as far as I understand we go with the documentation the same
>>> way as with the previous Apache UIMA releases. We check in the
>>> documentation
>>> to SVN and provide a download similar to release 2.2.0-incubating:
>>>
>>> http://incubator.apache.org/uima/downloads/releaseDocs/2.2.0-incubating/docs/html/index.html
>>>
>>>
>>>
>>> The JavaDocs will also go to SVN in both versions, HTML and zip.
>>>
>>> I can do the necessary changes, if all agree on that.
>>>
>>> -- Michael
>>>
>>>
>>>
>>>
>> +1.
>>
>> Also - remove the docs from /dist/incubator/uima
>>
>> -Marshall
>>
> Should we really remove the documentation from there. I think other
> projects also have the documentation there. So I think we should
> provide it too, maybe as one package to download.?
>
> -- Michael
This one is a judgement call - I can see arguments on both sides.
We've heard that the "rsync" mechanism handles small numbers of large
files better than large numbers of small files - so putting only the 1
archive file to download seems a better fit, if we do this.
Putting them in /dist/ means they will be mirrored, and archived. So we
will have dual archiving (one in SVN, and one in the archive spot).
The mirroring would be useful *if* we expected a large load on the
apache servers for downloading these. I think this will not be the
case. Most of the time I use these to send people links to specific
sections of the docs; for that it would be annoying if when they clicked
the link, they were asked to pick a mirror.
Considering all of this - I'm slightly in favor of keeping the docs just
in SVN, and not on the /dist/ mirroring system.
------------
Some more things to think about
------------
Since we want our docs pages to refer not only to the "current release"
but also previous releases, it would be good to figure out a fairly
automatic system for this. (That was a virtue of the /dist/ - archive
system - we could point the previous releases doc links to a directory
containing all the releases, and wouldn't need to update this link for
subsequent releases).
The other thing to do is to figure out how to keep the archived things
on our web-site. It was suggested that we should do this like we handle
the web-site checkout.
Probably the straight-forward thing to do is to have a special directory
where all the docs we want to refer to live, have the archive link point
there, and have special links that refer to the "current version".
As I recall, the web-site, itself, is replicated to other servers (since
after you update it , you have to wait a while for it to appear).
I have to confess that this seems quite wasteful of disk resources
(double+ copies of things like javadocs - one in SVN, one on
people.apache.org in our web-site place, and maybe (several?) additional
copies on web-servers used for incubator.apache.org/uima).
But Robert Donkin suggested this was the best way.
-Marshall
Re: Where to put large generated things for our website to access
Posted by Michael Baessler <mb...@michael-baessler.de>.
Marshall Schor wrote:
> Michael Baessler wrote:
>
>> Robert Burrell Donkin wrote:
>>
>>> subversion really is the way to go for release documentation
>>>
>>>
>> OK, so as far as I understand we go with the documentation the same
>> way as with the previous Apache UIMA releases. We check in the
>> documentation
>> to SVN and provide a download similar to release 2.2.0-incubating:
>>
>> http://incubator.apache.org/uima/downloads/releaseDocs/2.2.0-incubating/docs/html/index.html
>>
>>
>> The JavaDocs will also go to SVN in both versions, HTML and zip.
>>
>> I can do the necessary changes, if all agree on that.
>>
>> -- Michael
>>
>>
>>
>>
> +1.
>
> Also - remove the docs from /dist/incubator/uima
>
> -Marshall
>
Should we really remove the documentation from there. I think other
projects also have the documentation there. So I think we should provide
it too, maybe as one package to download.?
-- Michael
Re: Where to put large generated things for our website to access
Posted by Michael Baessler <mb...@michael-baessler.de>.
Marshall Schor wrote:
> Michael Baessler wrote:
>
>> Robert Burrell Donkin wrote:
>>
>>> subversion really is the way to go for release documentation
>>>
>>>
>> OK, so as far as I understand we go with the documentation the same
>> way as with the previous Apache UIMA releases. We check in the
>> documentation
>> to SVN and provide a download similar to release 2.2.0-incubating:
>>
>> http://incubator.apache.org/uima/downloads/releaseDocs/2.2.0-incubating/docs/html/index.html
>>
>>
>> The JavaDocs will also go to SVN in both versions, HTML and zip.
>>
>> I can do the necessary changes, if all agree on that.
>>
>> -- Michael
>>
>>
>>
>>
> +1.
>
> Also - remove the docs from /dist/incubator/uima
>
> -Marshall
>
Changes are done. Please review.
-- Michael
Re: Where to put large generated things for our website to access
Posted by Michael Baessler <mb...@michael-baessler.de>.
Marshall Schor wrote:
> Michael Baessler wrote:
>
>> Robert Burrell Donkin wrote:
>>
>>> subversion really is the way to go for release documentation
>>>
>>>
>> OK, so as far as I understand we go with the documentation the same
>> way as with the previous Apache UIMA releases. We check in the
>> documentation
>> to SVN and provide a download similar to release 2.2.0-incubating:
>>
>> http://incubator.apache.org/uima/downloads/releaseDocs/2.2.0-incubating/docs/html/index.html
>>
>>
>> The JavaDocs will also go to SVN in both versions, HTML and zip.
>>
>> I can do the necessary changes, if all agree on that.
>>
>> -- Michael
>>
>>
>>
>>
> +1.
>
> Also - remove the docs from /dist/incubator/uima
>
> -Marshall
>
Docs deleted from /dist/incubator/uima
-- Michael
Re: Where to put large generated things for our website to access
Posted by Marshall Schor <ms...@schor.com>.
Michael Baessler wrote:
> Robert Burrell Donkin wrote:
>> subversion really is the way to go for release documentation
>>
> OK, so as far as I understand we go with the documentation the same
> way as with the previous Apache UIMA releases. We check in the
> documentation
> to SVN and provide a download similar to release 2.2.0-incubating:
>
> http://incubator.apache.org/uima/downloads/releaseDocs/2.2.0-incubating/docs/html/index.html
>
>
> The JavaDocs will also go to SVN in both versions, HTML and zip.
>
> I can do the necessary changes, if all agree on that.
>
> -- Michael
>
>
>
+1.
Also - remove the docs from /dist/incubator/uima
-Marshall
Re: Where to put large generated things for our website to access
Posted by Michael Baessler <mb...@michael-baessler.de>.
Robert Burrell Donkin wrote:
> subversion really is the way to go for release documentation
>
OK, so as far as I understand we go with the documentation the same way
as with the previous Apache UIMA releases. We check in the documentation
to SVN and provide a download similar to release 2.2.0-incubating:
http://incubator.apache.org/uima/downloads/releaseDocs/2.2.0-incubating/docs/html/index.html
The JavaDocs will also go to SVN in both versions, HTML and zip.
I can do the necessary changes, if all agree on that.
-- Michael
Re: Where to put large generated things for our website to access
Posted by Robert Burrell Donkin <ro...@gmail.com>.
On Dec 31, 2007 3:43 AM, Marshall Schor <ms...@schor.com> wrote:
>
> Robert Burrell Donkin wrote:
> > (apologies for not jumping in promptly)
> >
> > On Dec 24, 2007 3:48 AM, Marshall Schor <ms...@schor.com> wrote:
> >
> >> I updated our website download page and documentation page. I made the
> >> download page work with mirrors, and changed the format for accessing
> >> previous archived files to follow the common practice on other sites,
> >> referring to the archive.apache.org site.
> >>
> >> I made our documentation page refer to apache.org/dist/incubator/uima
> >> for the doc files - and didn't put any of these into our SVN for our
> >> website.
> >>
> >
> > after feeling a little uncertain about this, i asked the
> > intrastructure team who gave some good arguments for storing docs in
> > dist:
> >
> > 1. rsync is good for large files but struggles with lots of small files
> > 2. mirrored documentation is not supported so push all that content to
> > the mirrors is wasteful
> > 3. released documentation should have an unchanging URL. when a
> > release is archived, the documentation URL would need to change (a
> > redirect would help people but not all robots).
> >
> > having release documentation permanently stored and archived is a good
> > idea but it's strongly recommended that subversion is used. the zip'd
> > archive is fine where it is but it would be better for the contents of
> > the folders to be committed to subversion and then checked out to an
> > appropriate place on the website.
> >
> > - robert
> >
> I felt uncertain about all of this, too. It seems to me that the right
> way to do this would be to have something like w.a.o/dist-not-mirrored/
> ... etc. where the same "archive" mechanism could be used as is used for
> /dist/, but which doesn't do mirroring. Has this come up before in
> discussions - a way to have things that are not to be mirrored, but
> which would reasonably be "archived"? You might say that the docs don't
> need to be archived (because they can always be extracted from an
> archived "release" zip/tar), but I find having at least some older
> versions of the docs quite useful in helping users running on a specific
> level - I can say things like "see xxx on page yyy" and know it matches
> their documentation.
agreed. IMHO retaining documentation is best practice.
quite a number of other projects do this eg httpd, most
(ex-jakarta-)commons libraries
httpd stores generated release documentation in subversion. commons
just manually transferred the documentation onto minotaur. IIRC it was
a lot of hassle recovering unarchived release documentation so i now
recommend subversion.
> It seems inefficient to store large generated things in SVN, such as the
> javadocs (these are large numbers of small files) -- but I would be
> happy to learn if I'm worrying about this unnecessarily.
subversion is a very different from CVS under the hood. apache uses a
file system backing subversion. this configuration copes very well
with large numbers of files. apache runs a single repository
containing every revision of every apache project past and present.
so, in that context even a well documented project (such as httpd)
will only have a relatively small number of documentation files.
subversion is archived. the file system storage copes fine with lots
of files. it is recommended that the documentation is checked out from
subversion (rather than using the subversion URLs directly). website
URLs are mirrored (and worldwide mirroring will be rolled out in 2008)
whereas svn.apache.org is not (currently). it is possible that once
the next version of subversion is released, read-only mirroring of
http://svn.apache.org will be rolled out.
> I can see an argument against something like w.a.o/dist-not-mirrored/ -
> avoiding creating even more "infrastructure stuff".
apache is moving away from dist towards a formal release staging
(rather than just sync'ing). this will not only improve security but
will also allow scripts to enforce policy. i expect this to happen
sometime in 2008. another directory would just be another problem to
be dealt with.
subversion really is the way to go for release documentation
- robert
Re: Where to put large generated things for our website to access
Posted by Marshall Schor <ms...@schor.com>.
Robert Burrell Donkin wrote:
> (apologies for not jumping in promptly)
>
> On Dec 24, 2007 3:48 AM, Marshall Schor <ms...@schor.com> wrote:
>
>> I updated our website download page and documentation page. I made the
>> download page work with mirrors, and changed the format for accessing
>> previous archived files to follow the common practice on other sites,
>> referring to the archive.apache.org site.
>>
>> I made our documentation page refer to apache.org/dist/incubator/uima
>> for the doc files - and didn't put any of these into our SVN for our
>> website.
>>
>
> after feeling a little uncertain about this, i asked the
> intrastructure team who gave some good arguments for storing docs in
> dist:
>
> 1. rsync is good for large files but struggles with lots of small files
> 2. mirrored documentation is not supported so push all that content to
> the mirrors is wasteful
> 3. released documentation should have an unchanging URL. when a
> release is archived, the documentation URL would need to change (a
> redirect would help people but not all robots).
>
> having release documentation permanently stored and archived is a good
> idea but it's strongly recommended that subversion is used. the zip'd
> archive is fine where it is but it would be better for the contents of
> the folders to be committed to subversion and then checked out to an
> appropriate place on the website.
>
> - robert
>
I felt uncertain about all of this, too. It seems to me that the right
way to do this would be to have something like w.a.o/dist-not-mirrored/
... etc. where the same "archive" mechanism could be used as is used for
/dist/, but which doesn't do mirroring. Has this come up before in
discussions - a way to have things that are not to be mirrored, but
which would reasonably be "archived"? You might say that the docs don't
need to be archived (because they can always be extracted from an
archived "release" zip/tar), but I find having at least some older
versions of the docs quite useful in helping users running on a specific
level - I can say things like "see xxx on page yyy" and know it matches
their documentation.
It seems inefficient to store large generated things in SVN, such as the
javadocs (these are large numbers of small files) -- but I would be
happy to learn if I'm worrying about this unnecessarily.
I can see an argument against something like w.a.o/dist-not-mirrored/ -
avoiding creating even more "infrastructure stuff".
Other opinions / options?
-Marshall
Re: Where to put large generated things for our website to access
Posted by Robert Burrell Donkin <ro...@gmail.com>.
(apologies for not jumping in promptly)
On Dec 24, 2007 3:48 AM, Marshall Schor <ms...@schor.com> wrote:
> I updated our website download page and documentation page. I made the
> download page work with mirrors, and changed the format for accessing
> previous archived files to follow the common practice on other sites,
> referring to the archive.apache.org site.
>
> I made our documentation page refer to apache.org/dist/incubator/uima
> for the doc files - and didn't put any of these into our SVN for our
> website.
after feeling a little uncertain about this, i asked the
intrastructure team who gave some good arguments for storing docs in
dist:
1. rsync is good for large files but struggles with lots of small files
2. mirrored documentation is not supported so push all that content to
the mirrors is wasteful
3. released documentation should have an unchanging URL. when a
release is archived, the documentation URL would need to change (a
redirect would help people but not all robots).
having release documentation permanently stored and archived is a good
idea but it's strongly recommended that subversion is used. the zip'd
archive is fine where it is but it would be better for the contents of
the folders to be committed to subversion and then checked out to an
appropriate place on the website.
- robert
Re: Where to put large generated things for our website to access
Posted by Marshall Schor <ms...@schor.com>.
I updated our website download page and documentation page. I made the
download page work with mirrors, and changed the format for accessing
previous archived files to follow the common practice on other sites,
referring to the archive.apache.org site.
I made our documentation page refer to apache.org/dist/incubator/uima
for the doc files - and didn't put any of these into our SVN for our
website.
I also followed common practice and put our Release notes into the
apache.org/dist/i/u at the top level (I changed the name to add the
suffix of the release version). This allows (via the archive system)
for these things to be always available.
I added the Eclipse update site to a.o/d/i/u
The only thing not done as of yet is setting up a .htaccess file in this
directory, and adding HEADER.html and README.html files to make
directory listing more "customized". I'm not going to tackle this right
now - if anyone else wants to take a crack, OK with me.
-Marshall
Michael Baessler wrote:
> Fine with me.
>
> -- Michael
>
> Marshall Schor wrote:
>> One further thought: A lot of projects put the RELEASE NOTES for
>> particular releases at the "top level" of the dist/ - where the file
>> name includes the release: for example:
>> ANT: RELEASE-NOTES-1.7.0.html
>> HTTPD: CHANGES_2.2.6
>>
>> Since these will get archived, and redirects can be done for them, their
>> URLs can be permanent.
>>
>> To be consistent with this practice, I would like to put our release
>> notes for display from our web site in a.o/dist/...
>>
>> They don't need to be signed, because only archives need that; also,
>> other projects don't sign these kinds of things. Any objections?
>>
>> -Marshall
>>
>>
>> Marshall Schor wrote:
>>
>>> Here's an argument for keeping the big things we point to from our
>>> website, like the javaDocs and the 4 books in html form, on the
>>> a.o/dist/incubator/uima site:
>>>
>>> It is automatically archived. And, when it's "deleted" from the
>>> mirror, a redirect is put in to the archive spot.
>>>
>>> This would be ideal for being able to have older versions kept with
>>> permanent static URLs.
>>>
>>> So - upon further reflection - I think I'm changing my mind on this,
>>> and
>>> am now in favor of keeping these on the mirroring system.
>>> We can avoid having people who are on our web site and want to view the
>>> documentation, having to check signatures by not using the mirroring
>>> system, but just pointing them to the main a.o/dist location. I'm not
>>> sure if this would be OK in terms of protocol for load balancing, but
>>> I'll set the doc page up this way for now, and we can change it if we
>>> need to.
>>> -Marshall
>>>
>>> Michael Baessler wrote:
>>>
>>>> Fine with me to delete the HTML documentations (manual and javadoc) on
>>>> the mirror. I thought we can use it and link them from our website.
>>>>
>>>> As far as I know, there is no script to upload the documentation. I
>>>> did it manually.
>>>>
>>>> -- Michael
>>>>
>>>> Marshall Schor wrote:
>>>>
>>>>> Here are some examples of large files (or large collections of files
>>>>>
>>>>> The api java docs;
>>>>> The api java docs as a zip file
>>>>> The 4 books in html format
>>>>>
>>>>> It seems good to put them in
>>>>> people.a.o/www/incubator.a.o/uima/downloads.
>>>>>
>>>>> It seems bad to put them in SVN (because there's no need for
>>>>> "versioning" these - they're generated, and they are big, taking
>>>>> up SVN
>>>>> space).
>>>>>
>>>>> Our current strategy is hybrid:
>>>>>
>>>>> 1) for current release only: api java docs and the api java docs.zip
>>>>> file are put in people.a.o/www/incubator.a.o/uima/downloads, and are
>>>>> *not* kept in SVN.
>>>>> 2) for current release only: the 4 books in html format are put in
>>>>> SVN
>>>>> and copied to people.a.o/www/incubator.a.o/uima/downloads with the
>>>>> svn
>>>>> update command.
>>>>>
>>>>> I see, in fact that for release 2.2.0, we managed to put the books in
>>>>> html format into SVN twice - once under
>>>>> 2.2.0-incubating/docs/html, and once under
>>>>> 2.2.0-incubating/html
>>>>> and of course, on the website, it shows up twice also... not good.
>>>>>
>>>>> Is there any automated process for getting the files installed on
>>>>> people.a.o/www/incubator.a.o/uima/downloads? (Has anyone done any
>>>>> scripts for this)?
>>>>>
>>>>> Does everyone agree that it's best to keep these out of SVN, and
>>>>> to put
>>>>> them in the web server spot on
>>>>> people.a.o/www/incubator.a.o/uima/downloads?
>>>>>
>>>>> ===================
>>>>>
>>>>> The mirrored distribution spot contains, in addition to /binaries and
>>>>> /source, a /docs directory with the the following:
>>>>> <release>/apiDocs.zip , plus the 3 "signing" files [asc, md5, sha1]
>>>>> <release>/api /.... <-- unzipped set of javaDoc html files, no
>>>>> signing files
>>>>> <release>/html/..... <-- set of 4 books as html files, no signing
>>>>> files
>>>>> <release>/pdf/..... <-- set of 4 books as pdfs, no signing files
>>>>>
>>>>> I think everything that's put onto the mirroring system is
>>>>> supposed to
>>>>> be signed, because Apache doesn't "control" what goes on at the
>>>>> mirrors
>>>>> (e.g., they could be "hacked").
>>>>>
>>>>> Currently, our download page is slient about the existance of these.
>>>>>
>>>>> I think we should delete these on the mirroring distribution system.
>>>>> Assuming we followed the top part of this note, we would have
>>>>> everything
>>>>> (except the pdf form of the 4 books) on the UIMA website, directly
>>>>> (not
>>>>> going thru a mirror).
>>>>>
>>>>> Other opinions?
>>>>>
>>>>> -Marshall
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>>
>
>
>
Re: Where to put large generated things for our website to access
Posted by Michael Baessler <mb...@michael-baessler.de>.
Fine with me.
-- Michael
Marshall Schor wrote:
> One further thought: A lot of projects put the RELEASE NOTES for
> particular releases at the "top level" of the dist/ - where the file
> name includes the release: for example:
> ANT: RELEASE-NOTES-1.7.0.html
> HTTPD: CHANGES_2.2.6
>
> Since these will get archived, and redirects can be done for them, their
> URLs can be permanent.
>
> To be consistent with this practice, I would like to put our release
> notes for display from our web site in a.o/dist/...
>
> They don't need to be signed, because only archives need that; also,
> other projects don't sign these kinds of things. Any objections?
>
> -Marshall
>
>
> Marshall Schor wrote:
>
>> Here's an argument for keeping the big things we point to from our
>> website, like the javaDocs and the 4 books in html form, on the
>> a.o/dist/incubator/uima site:
>>
>> It is automatically archived. And, when it's "deleted" from the
>> mirror, a redirect is put in to the archive spot.
>>
>> This would be ideal for being able to have older versions kept with
>> permanent static URLs.
>>
>> So - upon further reflection - I think I'm changing my mind on this, and
>> am now in favor of keeping these on the mirroring system.
>>
>> We can avoid having people who are on our web site and want to view the
>> documentation, having to check signatures by not using the mirroring
>> system, but just pointing them to the main a.o/dist location. I'm not
>> sure if this would be OK in terms of protocol for load balancing, but
>> I'll set the doc page up this way for now, and we can change it if we
>> need to.
>>
>> -Marshall
>>
>> Michael Baessler wrote:
>>
>>
>>> Fine with me to delete the HTML documentations (manual and javadoc) on
>>> the mirror. I thought we can use it and link them from our website.
>>>
>>> As far as I know, there is no script to upload the documentation. I
>>> did it manually.
>>>
>>> -- Michael
>>>
>>> Marshall Schor wrote:
>>>
>>>
>>>> Here are some examples of large files (or large collections of files
>>>>
>>>> The api java docs;
>>>> The api java docs as a zip file
>>>> The 4 books in html format
>>>>
>>>> It seems good to put them in
>>>> people.a.o/www/incubator.a.o/uima/downloads.
>>>>
>>>> It seems bad to put them in SVN (because there's no need for
>>>> "versioning" these - they're generated, and they are big, taking up SVN
>>>> space).
>>>>
>>>> Our current strategy is hybrid:
>>>>
>>>> 1) for current release only: api java docs and the api java docs.zip
>>>> file are put in people.a.o/www/incubator.a.o/uima/downloads, and are
>>>> *not* kept in SVN.
>>>> 2) for current release only: the 4 books in html format are put in SVN
>>>> and copied to people.a.o/www/incubator.a.o/uima/downloads with the svn
>>>> update command.
>>>>
>>>> I see, in fact that for release 2.2.0, we managed to put the books in
>>>> html format into SVN twice - once under
>>>> 2.2.0-incubating/docs/html, and once under
>>>> 2.2.0-incubating/html
>>>> and of course, on the website, it shows up twice also... not good.
>>>>
>>>> Is there any automated process for getting the files installed on
>>>> people.a.o/www/incubator.a.o/uima/downloads? (Has anyone done any
>>>> scripts for this)?
>>>>
>>>> Does everyone agree that it's best to keep these out of SVN, and to put
>>>> them in the web server spot on
>>>> people.a.o/www/incubator.a.o/uima/downloads?
>>>>
>>>> ===================
>>>>
>>>> The mirrored distribution spot contains, in addition to /binaries and
>>>> /source, a /docs directory with the the following:
>>>> <release>/apiDocs.zip , plus the 3 "signing" files [asc, md5, sha1]
>>>> <release>/api /.... <-- unzipped set of javaDoc html files, no
>>>> signing files
>>>> <release>/html/..... <-- set of 4 books as html files, no signing
>>>> files
>>>> <release>/pdf/..... <-- set of 4 books as pdfs, no signing files
>>>>
>>>> I think everything that's put onto the mirroring system is supposed to
>>>> be signed, because Apache doesn't "control" what goes on at the mirrors
>>>> (e.g., they could be "hacked").
>>>>
>>>> Currently, our download page is slient about the existance of these.
>>>>
>>>> I think we should delete these on the mirroring distribution system.
>>>> Assuming we followed the top part of this note, we would have everything
>>>> (except the pdf form of the 4 books) on the UIMA website, directly (not
>>>> going thru a mirror).
>>>>
>>>> Other opinions?
>>>>
>>>> -Marshall
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>>
>
>
Re: Where to put large generated things for our website to access
Posted by Marshall Schor <ms...@schor.com>.
One further thought: A lot of projects put the RELEASE NOTES for
particular releases at the "top level" of the dist/ - where the file
name includes the release: for example:
ANT: RELEASE-NOTES-1.7.0.html
HTTPD: CHANGES_2.2.6
Since these will get archived, and redirects can be done for them, their
URLs can be permanent.
To be consistent with this practice, I would like to put our release
notes for display from our web site in a.o/dist/...
They don't need to be signed, because only archives need that; also,
other projects don't sign these kinds of things. Any objections?
-Marshall
Marshall Schor wrote:
> Here's an argument for keeping the big things we point to from our
> website, like the javaDocs and the 4 books in html form, on the
> a.o/dist/incubator/uima site:
>
> It is automatically archived. And, when it's "deleted" from the
> mirror, a redirect is put in to the archive spot.
>
> This would be ideal for being able to have older versions kept with
> permanent static URLs.
>
> So - upon further reflection - I think I'm changing my mind on this, and
> am now in favor of keeping these on the mirroring system.
>
> We can avoid having people who are on our web site and want to view the
> documentation, having to check signatures by not using the mirroring
> system, but just pointing them to the main a.o/dist location. I'm not
> sure if this would be OK in terms of protocol for load balancing, but
> I'll set the doc page up this way for now, and we can change it if we
> need to.
>
> -Marshall
>
> Michael Baessler wrote:
>
>> Fine with me to delete the HTML documentations (manual and javadoc) on
>> the mirror. I thought we can use it and link them from our website.
>>
>> As far as I know, there is no script to upload the documentation. I
>> did it manually.
>>
>> -- Michael
>>
>> Marshall Schor wrote:
>>
>>> Here are some examples of large files (or large collections of files
>>>
>>> The api java docs;
>>> The api java docs as a zip file
>>> The 4 books in html format
>>>
>>> It seems good to put them in
>>> people.a.o/www/incubator.a.o/uima/downloads.
>>>
>>> It seems bad to put them in SVN (because there's no need for
>>> "versioning" these - they're generated, and they are big, taking up SVN
>>> space).
>>>
>>> Our current strategy is hybrid:
>>>
>>> 1) for current release only: api java docs and the api java docs.zip
>>> file are put in people.a.o/www/incubator.a.o/uima/downloads, and are
>>> *not* kept in SVN.
>>> 2) for current release only: the 4 books in html format are put in SVN
>>> and copied to people.a.o/www/incubator.a.o/uima/downloads with the svn
>>> update command.
>>>
>>> I see, in fact that for release 2.2.0, we managed to put the books in
>>> html format into SVN twice - once under
>>> 2.2.0-incubating/docs/html, and once under
>>> 2.2.0-incubating/html
>>> and of course, on the website, it shows up twice also... not good.
>>>
>>> Is there any automated process for getting the files installed on
>>> people.a.o/www/incubator.a.o/uima/downloads? (Has anyone done any
>>> scripts for this)?
>>>
>>> Does everyone agree that it's best to keep these out of SVN, and to put
>>> them in the web server spot on
>>> people.a.o/www/incubator.a.o/uima/downloads?
>>>
>>> ===================
>>>
>>> The mirrored distribution spot contains, in addition to /binaries and
>>> /source, a /docs directory with the the following:
>>> <release>/apiDocs.zip , plus the 3 "signing" files [asc, md5, sha1]
>>> <release>/api /.... <-- unzipped set of javaDoc html files, no
>>> signing files
>>> <release>/html/..... <-- set of 4 books as html files, no signing
>>> files
>>> <release>/pdf/..... <-- set of 4 books as pdfs, no signing files
>>>
>>> I think everything that's put onto the mirroring system is supposed to
>>> be signed, because Apache doesn't "control" what goes on at the mirrors
>>> (e.g., they could be "hacked").
>>>
>>> Currently, our download page is slient about the existance of these.
>>>
>>> I think we should delete these on the mirroring distribution system.
>>> Assuming we followed the top part of this note, we would have everything
>>> (except the pdf form of the 4 books) on the UIMA website, directly (not
>>> going thru a mirror).
>>>
>>> Other opinions?
>>>
>>> -Marshall
>>>
>>>
>>>
>>
>>
>
>
>
>
Re: Where to put large generated things for our website to access
Posted by Marshall Schor <ms...@schor.com>.
Here's an argument for keeping the big things we point to from our
website, like the javaDocs and the 4 books in html form, on the
a.o/dist/incubator/uima site:
It is automatically archived. And, when it's "deleted" from the
mirror, a redirect is put in to the archive spot.
This would be ideal for being able to have older versions kept with
permanent static URLs.
So - upon further reflection - I think I'm changing my mind on this, and
am now in favor of keeping these on the mirroring system.
We can avoid having people who are on our web site and want to view the
documentation, having to check signatures by not using the mirroring
system, but just pointing them to the main a.o/dist location. I'm not
sure if this would be OK in terms of protocol for load balancing, but
I'll set the doc page up this way for now, and we can change it if we
need to.
-Marshall
Michael Baessler wrote:
> Fine with me to delete the HTML documentations (manual and javadoc) on
> the mirror. I thought we can use it and link them from our website.
>
> As far as I know, there is no script to upload the documentation. I
> did it manually.
>
> -- Michael
>
> Marshall Schor wrote:
>> Here are some examples of large files (or large collections of files
>>
>> The api java docs;
>> The api java docs as a zip file
>> The 4 books in html format
>>
>> It seems good to put them in
>> people.a.o/www/incubator.a.o/uima/downloads.
>>
>> It seems bad to put them in SVN (because there's no need for
>> "versioning" these - they're generated, and they are big, taking up SVN
>> space).
>>
>> Our current strategy is hybrid:
>>
>> 1) for current release only: api java docs and the api java docs.zip
>> file are put in people.a.o/www/incubator.a.o/uima/downloads, and are
>> *not* kept in SVN.
>> 2) for current release only: the 4 books in html format are put in SVN
>> and copied to people.a.o/www/incubator.a.o/uima/downloads with the svn
>> update command.
>>
>> I see, in fact that for release 2.2.0, we managed to put the books in
>> html format into SVN twice - once under
>> 2.2.0-incubating/docs/html, and once under
>> 2.2.0-incubating/html
>> and of course, on the website, it shows up twice also... not good.
>>
>> Is there any automated process for getting the files installed on
>> people.a.o/www/incubator.a.o/uima/downloads? (Has anyone done any
>> scripts for this)?
>>
>> Does everyone agree that it's best to keep these out of SVN, and to put
>> them in the web server spot on
>> people.a.o/www/incubator.a.o/uima/downloads?
>>
>> ===================
>>
>> The mirrored distribution spot contains, in addition to /binaries and
>> /source, a /docs directory with the the following:
>> <release>/apiDocs.zip , plus the 3 "signing" files [asc, md5, sha1]
>> <release>/api /.... <-- unzipped set of javaDoc html files, no
>> signing files
>> <release>/html/..... <-- set of 4 books as html files, no signing
>> files
>> <release>/pdf/..... <-- set of 4 books as pdfs, no signing files
>>
>> I think everything that's put onto the mirroring system is supposed to
>> be signed, because Apache doesn't "control" what goes on at the mirrors
>> (e.g., they could be "hacked").
>>
>> Currently, our download page is slient about the existance of these.
>>
>> I think we should delete these on the mirroring distribution system.
>> Assuming we followed the top part of this note, we would have everything
>> (except the pdf form of the 4 books) on the UIMA website, directly (not
>> going thru a mirror).
>>
>> Other opinions?
>>
>> -Marshall
>>
>>
>
>
>
Re: Where to put large generated things for our website to access
Posted by Michael Baessler <mb...@michael-baessler.de>.
Fine with me to delete the HTML documentations (manual and javadoc) on
the mirror. I thought we can use it and link them from our website.
As far as I know, there is no script to upload the documentation. I did
it manually.
-- Michael
Marshall Schor wrote:
> Here are some examples of large files (or large collections of files
>
> The api java docs;
> The api java docs as a zip file
> The 4 books in html format
>
> It seems good to put them in people.a.o/www/incubator.a.o/uima/downloads.
>
> It seems bad to put them in SVN (because there's no need for
> "versioning" these - they're generated, and they are big, taking up SVN
> space).
>
> Our current strategy is hybrid:
>
> 1) for current release only: api java docs and the api java docs.zip
> file are put in people.a.o/www/incubator.a.o/uima/downloads, and are
> *not* kept in SVN.
> 2) for current release only: the 4 books in html format are put in SVN
> and copied to people.a.o/www/incubator.a.o/uima/downloads with the svn
> update command.
>
> I see, in fact that for release 2.2.0, we managed to put the books in
> html format into SVN twice - once under
> 2.2.0-incubating/docs/html, and once under
> 2.2.0-incubating/html
> and of course, on the website, it shows up twice also... not good.
>
> Is there any automated process for getting the files installed on
> people.a.o/www/incubator.a.o/uima/downloads? (Has anyone done any
> scripts for this)?
>
> Does everyone agree that it's best to keep these out of SVN, and to put
> them in the web server spot on people.a.o/www/incubator.a.o/uima/downloads?
>
> ===================
>
> The mirrored distribution spot contains, in addition to /binaries and
> /source, a /docs directory with the the following:
> <release>/apiDocs.zip , plus the 3 "signing" files [asc, md5, sha1]
> <release>/api /.... <-- unzipped set of javaDoc html files, no
> signing files
> <release>/html/..... <-- set of 4 books as html files, no signing files
> <release>/pdf/..... <-- set of 4 books as pdfs, no signing files
>
> I think everything that's put onto the mirroring system is supposed to
> be signed, because Apache doesn't "control" what goes on at the mirrors
> (e.g., they could be "hacked").
>
> Currently, our download page is slient about the existance of these.
>
> I think we should delete these on the mirroring distribution system.
> Assuming we followed the top part of this note, we would have everything
> (except the pdf form of the 4 books) on the UIMA website, directly (not
> going thru a mirror).
>
> Other opinions?
>
> -Marshall
>
>