You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Greg Stein <gs...@gmail.com> on 2020/07/13 11:19:58 UTC
Nightly builds not getting auto-purged
Hey all,
Not sure who is keeping an eye on the buildbot config, but Infra has been
looking at the buildmaster and found that we (svn) have about 14G of
nightlies laying around since 2015 (!!)
Re: Nightly builds not getting auto-purged
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Greg Stein wrote on Mon, 13 Jul 2020 09:39 -0500:
> On Mon, Jul 13, 2020 at 8:49 AM Daniel Shahaf <d....@daniel.shahaf.name>
> wrote:
> > (Don't know where your 50MB figure is from; on dist/ it's 25MB/release.)
> >
>
> $ ls -laF r1784492/
> total 51276
> drwxr-xr-x 2 buildmaster buildmaster 4096 Feb 27 2017 ./
> drwxr-xr-x 293 buildmaster buildmaster 12288 Feb 27 2017 ../
> -rwxr-xr-x 1 buildmaster buildmaster 15559956 Feb 27 2017 subversion-nightly.tar.bz2*
> -rwxr-xr-x 1 buildmaster buildmaster 40 Feb 27 2017 subversion-nightly.tar.bz2.sha1*
> -rwxr-xr-x 1 buildmaster buildmaster 18534901 Feb 27 2017 subversion-nightly.tar.gz*
> -rwxr-xr-x 1 buildmaster buildmaster 40 Feb 27 2017 subversion-nightly.tar.gz.sha1*
> -rwxr-xr-x 1 buildmaster buildmaster 18360378 Feb 27 2017 subversion-nightly.zip*
> -rwxr-xr-x 1 buildmaster buildmaster 40 Feb 27 2017 subversion-nightly.zip.sha1*
> -rwxr-xr-x 1 buildmaster buildmaster 13087 Feb 27 2017 svn_version.h.dist-nightly*
% rsync -r rsync.apache.org::apache-dist/subversion/ | fgrep "1.14.0."
-rw-rw-r-- 8,497,531 2020/05/27 07:33:22 subversion-1.14.0.tar.bz2
-rw-rw-r-- 11,519,871 2020/05/27 07:33:22 subversion-1.14.0.tar.gz
-rw-rw-r-- 11,306,039 2020/05/27 07:33:22 subversion-1.14.0.zip
That's odd…
Cheers,
Daniel
Re: Nightly builds not getting auto-purged
Posted by Greg Stein <gs...@gmail.com>.
On Mon, Jul 13, 2020 at 8:49 AM Daniel Shahaf <d....@daniel.shahaf.name>
wrote:
>...
> Also, why seven? I'd rather keep as many builds as needed to go back
>
I just threw out a number, to start the conversation.
> to before the branching point of the latest stable release. It's not
> going to be that much disk space, is it? Assuming we discard .bz2 files
> to save some disk space ("Let them use gzip!"):
>
> 20MB/nightly * 1 nightly/month * 24 months[minor line release period] =
> 0.5GB
>
As I said: the space is likely not going to be an issue. The primary
question is "what do we want to retain?"
> (Don't know where your 50MB figure is from; on dist/ it's 25MB/release.)
>
$ ls -laF r1784492/
total 51276
drwxr-xr-x 2 buildmaster buildmaster 4096 Feb 27 2017 ./
drwxr-xr-x 293 buildmaster buildmaster 12288 Feb 27 2017 ../
-rwxr-xr-x 1 buildmaster buildmaster 15559956 Feb 27 2017
subversion-nightly.tar.bz2*
-rwxr-xr-x 1 buildmaster buildmaster 40 Feb 27 2017
subversion-nightly.tar.bz2.sha1*
-rwxr-xr-x 1 buildmaster buildmaster 18534901 Feb 27 2017
subversion-nightly.tar.gz*
-rwxr-xr-x 1 buildmaster buildmaster 40 Feb 27 2017
subversion-nightly.tar.gz.sha1*
-rwxr-xr-x 1 buildmaster buildmaster 18360378 Feb 27 2017
subversion-nightly.zip*
-rwxr-xr-x 1 buildmaster buildmaster 40 Feb 27 2017
subversion-nightly.zip.sha1*
-rwxr-xr-x 1 buildmaster buildmaster 13087 Feb 27 2017
svn_version.h.dist-nightly*
Mark notes else-thread that this hasn't worked in a long while. So maybe we
simply disable it, until there is a call for nightly builds?
Note: distinct from build/test and report build success/failure. ... this
is about storing builds for people to access/test.
Cheers,
-g
Re: Nightly builds not getting auto-purged
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Greg Stein wrote on Mon, 13 Jul 2020 08:00 -0500:
> On Mon, Jul 13, 2020 at 6:34 AM Mark Phippard <ma...@gmail.com> wrote:
> > I'll lead with this stake-in-the-ground:
> > ** retain one week of nightly builds
> >
> >
> > +1 a week seems reasonable
> >
> Then we still have the "how many to retain?" question. I do like Mark's
> idea of maybe defining it as "7 builds" rather than "7 nights".
No reason we have to keep the most recent builds. We might as well
keep, say, the most recent N builds provided that no two of them are
from the same calendar month:
build()
all_builds = sorted(map(os.stat, glob.glob('*.gz')), key=operator.attrgetter('st_mtime'))
month_of = lambda x: x.st_mtime // (60 * 60 * 24 * 31)
if len(all_builds) > 7 and month_of(all_builds[-1]) == month_of(all_builds[-2]):
unlink(all_builds[-2])
all_builds[-2:-1] = []
if len(all_builds) > 7:
unlink(all_builds[0])
This should be more useful for bisecting.
Also, why seven? I'd rather keep as many builds as needed to go back
to before the branching point of the latest stable release. It's not
going to be that much disk space, is it? Assuming we discard .bz2 files
to save some disk space ("Let them use gzip!"):
20MB/nightly * 1 nightly/month * 24 months[minor line release period] = 0.5GB
(Don't know where your 50MB figure is from; on dist/ it's 25MB/release.)
Cheers,
Daniel
(Yes, I know there's datetime.somethingorother(st_mtime).strftime(),
but ENOTIME to look up what it's called…)
Re: Nightly builds not getting auto-purged
Posted by Mark Phippard <ma...@gmail.com>.
On Mon, Jul 13, 2020 at 9:00 AM Greg Stein <gs...@gmail.com> wrote:
> On Mon, Jul 13, 2020 at 6:34 AM Mark Phippard <ma...@gmail.com> wrote:
>
>> [ sigh; now I sent it unfinished ]
>>
>> On Jul 13, 2020, at 7:31 AM, Mark Phippard <ma...@gmail.com> wrote:
>>
>> On Jul 13, 2020, at 7:28 AM, Greg Stein <gs...@gmail.com> wrote:
>>
>> [sigh; sent unfinished]
>>
>> On Mon, Jul 13, 2020 at 6:19 AM Greg Stein <gs...@gmail.com> wrote:
>>
>>> Hey all,
>>>
>>> Not sure who is keeping an eye on the buildbot config, but Infra has
>>> been looking at the buildmaster and found that we (svn) have about 14G of
>>> nightlies laying around since 2015 (!!)
>>>
>>
>> https://ci.apache.org/projects/subversion/nightlies/index.html
>> https://ci.apache.org/projects/subversion/nightlies/dist/
>>
>> It seems that we likely only need (say) one week's worth of nightlies
>> (each is about 50M). What is the appropriate number?
>>
>> Is there a buildbot config option that is missing, that is avoiding a
>> purge of these files? Or does something "behind the scenes" need to take
>> responsibility?
>>
>> I believe Infra is likely to start *enforcing* some duration and overall
>> disk usage constraints. This likely won't impact us, as long as we purge
>> old builds according to our desired retention. So I think the more
>> important concern for our community is the question of "how many nightlies
>> do we want to retain?" ... I suspect whatever answer will be fine, but
>> finding an accepted consensus would be helpful.
>>
>> I'll lead with this stake-in-the-ground:
>> ** retain one week of nightly builds
>>
>>
>> +1 a week seems reasonable
>>
>> Do we only make a new nightly if the /trunk HEAD has advanced since the
>> last one? If so, then are you suggesting we keep the last 7 nightlies?
>>
>>
>> Did you happen to check if we even have a current nightly? ISTR several
>> months ago C-Mike had a question about building the tarball so he could try
>> out the Python 3 bindings where someone pointed out we post nightlies and
>> he could use that ... but then it turned out that the process had not been
>> working for a few years.
>>
>> Maybe I mis-remember or maybe it was fixed. But it could be that none of
>> them have any value (they are all old) and we are not even making these
>> anymore.
>>
>
> So I was just looking at the *build date* ... but Mark is right: it's
> totally broken. It has been building the same thing for a loooooong time.
>
> Can anybody volunteer to pick this up, and correct it? Get it building
> HEAD rather than $oldcrap ?
>
> Then we still have the "how many to retain?" question. I do like Mark's
> idea of maybe defining it as "7 builds" rather than "7 nights".
>
>
Given that the job has not worked for a while, I would say we just disable
it. Delete all of the nightlies that exist currently and then if someone
wants to make the job work again we discuss how the quantity could be
better managed. Of course, I do not know how to do any of this so it is
just an opinion. If we are using too much space I would just delete all but
the last one which should at least keep our UI from breaking.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
Re: Nightly builds not getting auto-purged
Posted by Greg Stein <gs...@gmail.com>.
On Mon, Jul 13, 2020 at 6:34 AM Mark Phippard <ma...@gmail.com> wrote:
> [ sigh; now I sent it unfinished ]
>
> On Jul 13, 2020, at 7:31 AM, Mark Phippard <ma...@gmail.com> wrote:
>
> On Jul 13, 2020, at 7:28 AM, Greg Stein <gs...@gmail.com> wrote:
>
> [sigh; sent unfinished]
>
> On Mon, Jul 13, 2020 at 6:19 AM Greg Stein <gs...@gmail.com> wrote:
>
>> Hey all,
>>
>> Not sure who is keeping an eye on the buildbot config, but Infra has been
>> looking at the buildmaster and found that we (svn) have about 14G of
>> nightlies laying around since 2015 (!!)
>>
>
> https://ci.apache.org/projects/subversion/nightlies/index.html
> https://ci.apache.org/projects/subversion/nightlies/dist/
>
> It seems that we likely only need (say) one week's worth of nightlies
> (each is about 50M). What is the appropriate number?
>
> Is there a buildbot config option that is missing, that is avoiding a
> purge of these files? Or does something "behind the scenes" need to take
> responsibility?
>
> I believe Infra is likely to start *enforcing* some duration and overall
> disk usage constraints. This likely won't impact us, as long as we purge
> old builds according to our desired retention. So I think the more
> important concern for our community is the question of "how many nightlies
> do we want to retain?" ... I suspect whatever answer will be fine, but
> finding an accepted consensus would be helpful.
>
> I'll lead with this stake-in-the-ground:
> ** retain one week of nightly builds
>
>
> +1 a week seems reasonable
>
> Do we only make a new nightly if the /trunk HEAD has advanced since the
> last one? If so, then are you suggesting we keep the last 7 nightlies?
>
>
> Did you happen to check if we even have a current nightly? ISTR several
> months ago C-Mike had a question about building the tarball so he could try
> out the Python 3 bindings where someone pointed out we post nightlies and
> he could use that ... but then it turned out that the process had not been
> working for a few years.
>
> Maybe I mis-remember or maybe it was fixed. But it could be that none of
> them have any value (they are all old) and we are not even making these
> anymore.
>
So I was just looking at the *build date* ... but Mark is right: it's
totally broken. It has been building the same thing for a loooooong time.
Can anybody volunteer to pick this up, and correct it? Get it building HEAD
rather than $oldcrap ?
Then we still have the "how many to retain?" question. I do like Mark's
idea of maybe defining it as "7 builds" rather than "7 nights".
Cheers,
-g
Re: Nightly builds not getting auto-purged
Posted by Mark Phippard <ma...@gmail.com>.
[ sigh; now I sent it unfinished ]
> On Jul 13, 2020, at 7:31 AM, Mark Phippard <ma...@gmail.com> wrote:
>
>
>>> On Jul 13, 2020, at 7:28 AM, Greg Stein <gs...@gmail.com> wrote:
>>>
>>
>> [sigh; sent unfinished]
>>
>>> On Mon, Jul 13, 2020 at 6:19 AM Greg Stein <gs...@gmail.com> wrote:
>>
>>> Hey all,
>>>
>>> Not sure who is keeping an eye on the buildbot config, but Infra has been looking at the buildmaster and found that we (svn) have about 14G of nightlies laying around since 2015 (!!)
>>
>> https://ci.apache.org/projects/subversion/nightlies/index.html
>> https://ci.apache.org/projects/subversion/nightlies/dist/
>>
>> It seems that we likely only need (say) one week's worth of nightlies (each is about 50M). What is the appropriate number?
>>
>> Is there a buildbot config option that is missing, that is avoiding a purge of these files? Or does something "behind the scenes" need to take responsibility?
>>
>> I believe Infra is likely to start *enforcing* some duration and overall disk usage constraints. This likely won't impact us, as long as we purge old builds according to our desired retention. So I think the more important concern for our community is the question of "how many nightlies do we want to retain?" ... I suspect whatever answer will be fine, but finding an accepted consensus would be helpful.
>>
>> I'll lead with this stake-in-the-ground:
>> ** retain one week of nightly builds
>>
>
> +1 a week seems reasonable
>
> Do we only make a new nightly if the /trunk HEAD has advanced since the last one? If so, then are you suggesting we keep the last 7 nightlies?
Did you happen to check if we even have a current nightly? ISTR several months ago C-Mike had a question about building the tarball so he could try out the Python 3 bindings where someone pointed out we post nightlies and he could use that ... but then it turned out that the process had not been working for a few years.
Maybe I mis-remember or maybe it was fixed. But it could be that none of them have any value (they are all old) and we are not even making these anymore.
Mark
Re: Nightly builds not getting auto-purged
Posted by Mark Phippard <ma...@gmail.com>.
> On Jul 13, 2020, at 7:28 AM, Greg Stein <gs...@gmail.com> wrote:
>
>
> [sigh; sent unfinished]
>
>> On Mon, Jul 13, 2020 at 6:19 AM Greg Stein <gs...@gmail.com> wrote:
>
>> Hey all,
>>
>> Not sure who is keeping an eye on the buildbot config, but Infra has been looking at the buildmaster and found that we (svn) have about 14G of nightlies laying around since 2015 (!!)
>
> https://ci.apache.org/projects/subversion/nightlies/index.html
> https://ci.apache.org/projects/subversion/nightlies/dist/
>
> It seems that we likely only need (say) one week's worth of nightlies (each is about 50M). What is the appropriate number?
>
> Is there a buildbot config option that is missing, that is avoiding a purge of these files? Or does something "behind the scenes" need to take responsibility?
>
> I believe Infra is likely to start *enforcing* some duration and overall disk usage constraints. This likely won't impact us, as long as we purge old builds according to our desired retention. So I think the more important concern for our community is the question of "how many nightlies do we want to retain?" ... I suspect whatever answer will be fine, but finding an accepted consensus would be helpful.
>
> I'll lead with this stake-in-the-ground:
> ** retain one week of nightly builds
>
+1 a week seems reasonable
Do we only make a new nightly if the /trunk HEAD has advanced since the last one? If so, then are you suggesting we keep the last 7 nightlies?
Mark
Re: Nightly builds not getting auto-purged
Posted by Greg Stein <gs...@gmail.com>.
[sigh; sent unfinished]
On Mon, Jul 13, 2020 at 6:19 AM Greg Stein <gs...@gmail.com> wrote:
> Hey all,
>
> Not sure who is keeping an eye on the buildbot config, but Infra has been
> looking at the buildmaster and found that we (svn) have about 14G of
> nightlies laying around since 2015 (!!)
>
https://ci.apache.org/projects/subversion/nightlies/index.html
https://ci.apache.org/projects/subversion/nightlies/dist/
It seems that we likely only need (say) one week's worth of nightlies (each
is about 50M). What is the appropriate number?
Is there a buildbot config option that is missing, that is avoiding a purge
of these files? Or does something "behind the scenes" need to take
responsibility?
I believe Infra is likely to start *enforcing* some duration and overall
disk usage constraints. This likely won't impact us, as long as we purge
old builds according to our desired retention. So I think the more
important concern for our community is the question of "how many nightlies
do we want to retain?" ... I suspect whatever answer will be fine, but
finding an accepted consensus would be helpful.
I'll lead with this stake-in-the-ground:
** retain one week of nightly builds
Cheers,
-g