Proposal for reducing directory checkouts by using svn ls

[sebb <se...@gmail.com>]

Update repositories.yml to use depth: empty for the relevant directory trees
The regular update cronjob will thus ensure that the directory has got the
current revision.

When a listing is needed, Whimsy checks the revision of the checkout and
the revision in the listing file.

If there is a mismatch, recreate the listing.
On failure, retain the previous listing.
[This is what would have happened if svn update had failed]

Since 'svn list' is pretty fast, I doubt it will be necessary for the
cronjob to create the listings, but that could be arranged if necessary.

Comments?

S.

Re: Proposal for reducing directory checkouts by using svn ls

[Sam Ruby <ru...@intertwingly.net>]

On Tue, Nov 26, 2019 at 12:23 PM sebb <se...@gmail.com> wrote:
>
> Update repositories.yml to use depth: empty for the relevant directory trees
> The regular update cronjob will thus ensure that the directory has got the
> current revision.
>
> When a listing is needed, Whimsy checks the revision of the checkout and
> the revision in the listing file.
>
> If there is a mismatch, recreate the listing.
> On failure, retain the previous listing.
> [This is what would have happened if svn update had failed]
>
> Since 'svn list' is pretty fast, I doubt it will be necessary for the
> cronjob to create the listings, but that could be arranged if necessary.
>
> Comments?

I'd suggest going with the cron job.  Perhaps instead of a depth
option, put a type: list in the repository.yml, and capture the output
somewhere in the /srv/svn directory (perhaps /srv/svn/iclas.list for
iclas).  Note: the cron job can update /srv/svn and /srv/git, but not
other directories (like /srv/cache).

Generally, I'm adverse to assuming that network operations will
succeed.  If you do the svn list on the fly, it may fail due to a
transient error.  Much of what I learned on the original whimsy_vm was
to avoid doing this.  Perhaps, I'm overcompensating.

What's nice about the cron job is that if it fails, there always will
be a next time, and the code can continue with a slightly more stale
listing.

As an added benefit, checking against a list on disk will be
noticeably faster than waiting for the svn list.  A few seconds may
not sound like a lot, but it affects the cadence of the user of the
secretary workbench.

> S.

- Sam Ruby