[jira] [Commented] (INFRA-10859) Allow rsync to list entire contents of dist area

["Henk Penning (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/INFRA-10859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15033369#comment-15033369 ] 

Henk Penning commented on INFRA-10859:
--------------------------------------

Hi Daniel,

I agree ; it is very inconvenient ; on mino, /dist/ is now out of date.
It would be easy to (for now) keep mino:/dist/ in sync with rsync.us.apache.org/apache-dist/.
I've asked infra, but the request was ignored.

I think the proposed workflow is :
-- checkout all of dist
-- .... compute ....
-- sftp results to home.a.o or into svn
That is inconvenient and inefficient ; it hinders people who want to make better tooling.

Perhaps it is possible to have a "mino for members" (instead of a "mino for committers").

I wouldn't worry about the .revision files ; to my knowledge, they serve no purpose.
It is just debug info for the svnpubsub (or whatever) stuff.
IMHO, they should be removed from (excluded for) the mirrors, or be removed altogether.

Regards,

Henk Penning

> Allow rsync to list entire contents of dist area
> ------------------------------------------------
>
>                 Key: INFRA-10859
>                 URL: https://issues.apache.org/jira/browse/INFRA-10859
>             Project: Infrastructure
>          Issue Type: Improvement
>          Components: Mirrors, Website
>         Environment: ASF mirror hosts
>            Reporter: Sebb
>
> The ASF mirror hosts has an rsync module which allows most of the contents of the /www/www.apache.org/dist area to be listed. However some files such as hashes, sigs, and the KEYS files are intentionally excluded from the module.
> It would be useful to be able to list all the contents of the directory structure.
> It is obviously possible to scrape the web pages, but that is time-consuming, fragile and uses more resources. Likewise, using SVN to list the folder structure is time consuming, and it does not show the actual contents which will generally be different (e.g. the .revision files are not in SVN)
> There is a module that allows listing all the files on minotaur, but it looks like the module has either not been configured or it disallows listing on the ASF mirror hosts.
> Note: the incubator clutch tool currently relies on being able to list the incubator files, including the KEYS and sig files. These don't appear in the standard rsync listings, so the code relies on using find as a cron job on minotaur.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)