You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@apache.org by Gavin McDonald <gm...@apache.org> on 2022/04/20 07:27:28 UTC

ci-builds all 3.6TB disk is full!

Hi All,

Seems we need to do another cull of projects storing way too much data.

Below are everyone above 1GB. Just FYI, 1GB is fine, likely 50GB is fine,
but above
that, its just too much. I will be removing 1TB of data from wherever I can
get it.

Please, look after your jobs, and your fellow projects by limiting what you
keep.

1.6T    Tomee
451G    Kafka
303G    james
176G    carbondata
129G    Jackrabbit
71G     Brooklyn
64G     Sling
64G     Netbeans
60G     Ranger
38G     AsterixDB
33G     OODT
29G     Tika
27G     Syncope
24G     Atlas
20G     IoTDB
18G     CXF
16G     POI
11G     Solr
11G     Mesos
8.7G    Royale
7.8G    Lucene
7.6G    MyFaces
7.6G    Directory
6.4G    OpenJPA
6.0G    ManifoldCF
5.9G    ActiveMQ
5.7G    Logging
5.6G    Archiva
5.5G    UIMA
5.3G    ctakes
4.7G    Heron
4.6G    Jena
4.0G    OpenOffice
3.8G    Cloudstack
3.4G    Shiro
2.5G    Qpid
2.1G    JSPWiki
2.1G    JMeter
2.0G    JClouds
1.8G    Santuario
1.8G    OpenMeetings
1.8G    Camel
1.7G    Karaf
1.7G    HttpComponents
1.7G    Ant
1.5G    Tapestry
1.5G    Commons
1.3G    DeltaSpike
1.2G    Rya
1.2G    Aries
1.2G    Accumulo
1.1G    PDFBox

-- 

*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team

Fwd: ci-builds all 3.6TB disk is full!

Posted by Josh Fischer <jo...@joshfischer.io>.
Does some one want clean up what our ci builds are storing?

---------- Forwarded message ---------
From: Gavin McDonald <gm...@apache.org>
Date: Wed, Apr 20, 2022 at 2:27 AM
Subject: ci-builds all 3.6TB disk is full!
To: builds <bu...@apache.org>


Hi All,

Seems we need to do another cull of projects storing way too much data.

Below are everyone above 1GB. Just FYI, 1GB is fine, likely 50GB is fine,
but above
that, its just too much. I will be removing 1TB of data from wherever I can
get it.

Please, look after your jobs, and your fellow projects by limiting what you
keep.

1.6T    Tomee
451G    Kafka
303G    james
176G    carbondata
129G    Jackrabbit
71G     Brooklyn
64G     Sling
64G     Netbeans
60G     Ranger
38G     AsterixDB
33G     OODT
29G     Tika
27G     Syncope
24G     Atlas
20G     IoTDB
18G     CXF
16G     POI
11G     Solr
11G     Mesos
8.7G    Royale
7.8G    Lucene
7.6G    MyFaces
7.6G    Directory
6.4G    OpenJPA
6.0G    ManifoldCF
5.9G    ActiveMQ
5.7G    Logging
5.6G    Archiva
5.5G    UIMA
5.3G    ctakes
4.7G    Heron
4.6G    Jena
4.0G    OpenOffice
3.8G    Cloudstack
3.4G    Shiro
2.5G    Qpid
2.1G    JSPWiki
2.1G    JMeter
2.0G    JClouds
1.8G    Santuario
1.8G    OpenMeetings
1.8G    Camel
1.7G    Karaf
1.7G    HttpComponents
1.7G    Ant
1.5G    Tapestry
1.5G    Commons
1.3G    DeltaSpike
1.2G    Rya
1.2G    Aries
1.2G    Accumulo
1.1G    PDFBox

-- 

*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team
-- 
Sent from A Mobile Device

Re: ci-builds all 3.6TB disk is full!

Posted by Gavin McDonald <gm...@apache.org>.
Hi Richard.

Thanks for keeping on top of this, note that if there are artifacts to
keep, you can use the send files over ssh to copy them
over to nightlies.apache.org

On Wed, May 18, 2022 at 9:38 AM Richard Eckart de Castilho <re...@apache.org>
wrote:

> Hi,
>
> > On 20. Apr 2022, at 09:27, Gavin McDonald <gm...@apache.org> wrote:
> >
> > 5.5G    UIMA
>
> although I guess we are still in a good range, I'll look into updating our
> Jenins pipeline template in such a way that the workspace is cleaned up by
> default after the build. I hope that will minimize disk usage.
>
>   https://issues.apache.org/jira/browse/UIMA-6457
>
> Best,
>
> -- Richard



-- 

*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team

Re: ci-builds all 3.6TB disk is full!

Posted by Richard Eckart de Castilho <re...@apache.org>.
Hi,

> On 20. Apr 2022, at 09:27, Gavin McDonald <gm...@apache.org> wrote:
> 
> 5.5G    UIMA

although I guess we are still in a good range, I'll look into updating our Jenins pipeline template in such a way that the workspace is cleaned up by default after the build. I hope that will minimize disk usage.

  https://issues.apache.org/jira/browse/UIMA-6457

Best,

-- Richard

Re: ci-builds all 3.6TB disk is full!

Posted by Jonathan Gallimore <jo...@gmail.com>.
Looks like I missed Richard's email yesterday and he's already taken care
of it. Thanks Richard, and sorry for the noise.

Jon

On Thu, Apr 21, 2022 at 4:30 PM Jonathan Gallimore <
jonathan.gallimore@gmail.com> wrote:

> Looks like we're top of the list, and not in a good way... :/
>
> I'm happy to help, but not sure where to start. Does anyone have any
> pointers?
>
> Jon
>
> ---------- Forwarded message ---------
> From: Gavin McDonald <gm...@apache.org>
> Date: Wed, Apr 20, 2022 at 8:27 AM
> Subject: ci-builds all 3.6TB disk is full!
> To: builds <bu...@apache.org>
>
>
> Hi All,
>
> Seems we need to do another cull of projects storing way too much data.
>
> Below are everyone above 1GB. Just FYI, 1GB is fine, likely 50GB is fine,
> but above
> that, its just too much. I will be removing 1TB of data from wherever I can
> get it.
>
> Please, look after your jobs, and your fellow projects by limiting what you
> keep.
>
> 1.6T    Tomee
> 451G    Kafka
> 303G    james
> 176G    carbondata
> 129G    Jackrabbit
> 71G     Brooklyn
> 64G     Sling
> 64G     Netbeans
> 60G     Ranger
> 38G     AsterixDB
> 33G     OODT
> 29G     Tika
> 27G     Syncope
> 24G     Atlas
> 20G     IoTDB
> 18G     CXF
> 16G     POI
> 11G     Solr
> 11G     Mesos
> 8.7G    Royale
> 7.8G    Lucene
> 7.6G    MyFaces
> 7.6G    Directory
> 6.4G    OpenJPA
> 6.0G    ManifoldCF
> 5.9G    ActiveMQ
> 5.7G    Logging
> 5.6G    Archiva
> 5.5G    UIMA
> 5.3G    ctakes
> 4.7G    Heron
> 4.6G    Jena
> 4.0G    OpenOffice
> 3.8G    Cloudstack
> 3.4G    Shiro
> 2.5G    Qpid
> 2.1G    JSPWiki
> 2.1G    JMeter
> 2.0G    JClouds
> 1.8G    Santuario
> 1.8G    OpenMeetings
> 1.8G    Camel
> 1.7G    Karaf
> 1.7G    HttpComponents
> 1.7G    Ant
> 1.5G    Tapestry
> 1.5G    Commons
> 1.3G    DeltaSpike
> 1.2G    Rya
> 1.2G    Aries
> 1.2G    Accumulo
> 1.1G    PDFBox
>
> --
>
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team
>

Fwd: ci-builds all 3.6TB disk is full!

Posted by Jonathan Gallimore <jo...@gmail.com>.
Looks like we're top of the list, and not in a good way... :/

I'm happy to help, but not sure where to start. Does anyone have any
pointers?

Jon

---------- Forwarded message ---------
From: Gavin McDonald <gm...@apache.org>
Date: Wed, Apr 20, 2022 at 8:27 AM
Subject: ci-builds all 3.6TB disk is full!
To: builds <bu...@apache.org>


Hi All,

Seems we need to do another cull of projects storing way too much data.

Below are everyone above 1GB. Just FYI, 1GB is fine, likely 50GB is fine,
but above
that, its just too much. I will be removing 1TB of data from wherever I can
get it.

Please, look after your jobs, and your fellow projects by limiting what you
keep.

1.6T    Tomee
451G    Kafka
303G    james
176G    carbondata
129G    Jackrabbit
71G     Brooklyn
64G     Sling
64G     Netbeans
60G     Ranger
38G     AsterixDB
33G     OODT
29G     Tika
27G     Syncope
24G     Atlas
20G     IoTDB
18G     CXF
16G     POI
11G     Solr
11G     Mesos
8.7G    Royale
7.8G    Lucene
7.6G    MyFaces
7.6G    Directory
6.4G    OpenJPA
6.0G    ManifoldCF
5.9G    ActiveMQ
5.7G    Logging
5.6G    Archiva
5.5G    UIMA
5.3G    ctakes
4.7G    Heron
4.6G    Jena
4.0G    OpenOffice
3.8G    Cloudstack
3.4G    Shiro
2.5G    Qpid
2.1G    JSPWiki
2.1G    JMeter
2.0G    JClouds
1.8G    Santuario
1.8G    OpenMeetings
1.8G    Camel
1.7G    Karaf
1.7G    HttpComponents
1.7G    Ant
1.5G    Tapestry
1.5G    Commons
1.3G    DeltaSpike
1.2G    Rya
1.2G    Aries
1.2G    Accumulo
1.1G    PDFBox

-- 

*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team

Re: ci-builds all 3.6TB disk is full!

Posted by Richard Zowalla <rz...@apache.org>.
Thanks for the listing. Retention policies are now in place for our
daily deploy jobs, so we shouldn't produce this huge amount of data
over time now.

If the retention kicks in, we should be fine.

Gruß
Richard


Am Mittwoch, dem 20.04.2022 um 09:41 +0200 schrieb Gavin McDonald:
> Thanks Richard,
> 
> On Wed, Apr 20, 2022 at 9:37 AM Richard Zowalla <rz...@apache.org>
> wrote:
> > Hi,
> > 
> > seems we are the "top" consumers with 1,6TB disk usage on the CI
> > infrastructure.
> > 
> > I looked at some of our jobs and found, that there is no retention
> > policy in place (for some of them). I added a policy similar to
> > what we
> > had in the past for newly created jobs. Looks like the retention
> > policy
> > is not copied then cloning jobs.
> > 
> > In addition, I asked Gavin, if he can provide a "du" listing for
> > our
> > jobs, so we can better dig into this issue.
> 
> Here is your listing:
> 
> 834G master-deploy
> 
> of which:
> 
> 445G org.apache.tomee$apache-tomee
> 111G org.apache.tomee$tomee-embedded
> 60G org.apache.tomee$openejb-standalone
> 44G org.apache.tomee$tomee-plume-webapp
> 39G org.apache.tomee$tomee-plus-webapp
> 36G org.apache.tomee$tomee-microprofile-webapp
> 26G org.apache.tomee$tomee-webapp
> 25G org.apache.tomee$openejb-lite
> 5.4G org.apache.tomee$tomee-webaccess
> 5.1G org.apache.tomee$taglibs-shade
> 4.4G org.apache.tomee$openejb-provisionning
> 4.0G org.apache.tomee$openejb-itests-app
> 3.4G org.apache.tomee$openejb-ssh
> 3.3G org.apache.tomee$arquillian-tomee-moviefun-example
> 3.1G org.apache.tomee$cxf-shade
> 2.3G org.apache.tomee$openejb-core
> 
> 597G jakarta-deploy
> 
> of which:
> 
> 354G org.apache.tomee$apache-tomee
> 73G org.apache.tomee$tomee-plume-webapp
> 66G org.apache.tomee$tomee-plus-webapp
> 60G org.apache.tomee$tomee-microprofile-webapp
> 42G org.apache.tomee$tomee-webprofile-webapp
> 4.3G org.apache.tomee$jakartaee-api
> 65M org.apache.tomee$tomee-project
> 44M org.apache.tomee$tomee
> 38M org.apache.tomee$transform
> 20M org.apache.tomee.jakarta$apache-tomee
> 7.2M org.apache.tomee.bom$jaxb-runtime
> 6.5M org.apache.tomee.bom$boms
> 
> 63G tomee-8.x-deploy
> 25G jakarta-wip-9.x-deploy
> 23G master-build-full
> 7.0G site-publish
> 4.2G tomee-8.x-sanity-checks
> 3.1G tomee-7.0.x
> 2.5G master-sanity-checks
> 2.1G pull-request
> 2.0G tomee-8.x-build-full
> 2.0G TOMEE-3872
> 1.7G master-pull-request
> 1.1G tomee-8.x-owasp-check
> 1.1G master-owasp-check
> 1.1G master-build-quick
> 945M tomee-8.x-build-quick
> 35M tomee-jakartaee-api-master
> 27M tomee-patch-plugin-deploy
> 428K clean-repo
> 256K tomee-jenkins-pipelines
> 
> > Gruß
> > Richard
> > 
> > -------- Ursprüngliche Nachricht --------
> > Von: Gavin McDonald <gm...@apache.org>
> > Antwort an: builds@apache.org, gmcdonald@apache.org
> > An: builds <bu...@apache.org>
> > Betreff: ci-builds all 3.6TB disk is full!
> > Datum: Wed, 20 Apr 2022 09:27:28 +0200
> > 
> > > Hi All,
> > > 
> > > Seems we need to do another cull of projects storing way too much
> > > data.
> > > 
> > > Below are everyone above 1GB. Just FYI, 1GB is fine, likely 50GB
> > is
> > > fine,
> > > but above
> > > that, its just too much. I will be removing 1TB of data from
> > wherever
> > > I can
> > > get it.
> > > 
> > > Please, look after your jobs, and your fellow projects by
> > limiting
> > > what you
> > > keep.
> > > 
> > > 1.6T    Tomee
> > > 451G    Kafka
> > > 303G    james
> > > 176G    carbondata
> > > 129G    Jackrabbit
> > > 71G     Brooklyn
> > > 64G     Sling
> > > 64G     Netbeans
> > > 60G     Ranger
> > > 38G     AsterixDB
> > > 33G     OODT
> > > 29G     Tika
> > > 27G     Syncope
> > > 24G     Atlas
> > > 20G     IoTDB
> > > 18G     CXF
> > > 16G     POI
> > > 11G     Solr
> > > 11G     Mesos
> > > 8.7G    Royale
> > > 7.8G    Lucene
> > > 7.6G    MyFaces
> > > 7.6G    Directory
> > > 6.4G    OpenJPA
> > > 6.0G    ManifoldCF
> > > 5.9G    ActiveMQ
> > > 5.7G    Logging
> > > 5.6G    Archiva
> > > 5.5G    UIMA
> > > 5.3G    ctakes
> > > 4.7G    Heron
> > > 4.6G    Jena
> > > 4.0G    OpenOffice
> > > 3.8G    Cloudstack
> > > 3.4G    Shiro
> > > 2.5G    Qpid
> > > 2.1G    JSPWiki
> > > 2.1G    JMeter
> > > 2.0G    JClouds
> > > 1.8G    Santuario
> > > 1.8G    OpenMeetings
> > > 1.8G    Camel
> > > 1.7G    Karaf
> > > 1.7G    HttpComponents
> > > 1.7G    Ant
> > > 1.5G    Tapestry
> > > 1.5G    Commons
> > > 1.3G    DeltaSpike
> > > 1.2G    Rya
> > > 1.2G    Aries
> > > 1.2G    Accumulo
> > > 1.1G    PDFBox
> > > 
> > > -- 
> > > 
> > > *Gavin McDonald*
> > > Systems Administrator
> > > ASF Infrastructure Team
> > 
> 
> 

Re: ci-builds all 3.6TB disk is full!

Posted by Gavin McDonald <gm...@apache.org>.
Thanks Richard,

On Wed, Apr 20, 2022 at 9:37 AM Richard Zowalla <rz...@apache.org> wrote:

> Hi,
>
> seems we are the "top" consumers with 1,6TB disk usage on the CI
> infrastructure.
>
> I looked at some of our jobs and found, that there is no retention
> policy in place (for some of them). I added a policy similar to what we
> had in the past for newly created jobs. Looks like the retention policy
> is not copied then cloning jobs.
>
> In addition, I asked Gavin, if he can provide a "du" listing for our
> jobs, so we can better dig into this issue.
>

Here is your listing:

834G master-deploy

of which:

445G org.apache.tomee$apache-tomee
111G org.apache.tomee$tomee-embedded
60G org.apache.tomee$openejb-standalone
44G org.apache.tomee$tomee-plume-webapp
39G org.apache.tomee$tomee-plus-webapp
36G org.apache.tomee$tomee-microprofile-webapp
26G org.apache.tomee$tomee-webapp
25G org.apache.tomee$openejb-lite
5.4G org.apache.tomee$tomee-webaccess
5.1G org.apache.tomee$taglibs-shade
4.4G org.apache.tomee$openejb-provisionning
4.0G org.apache.tomee$openejb-itests-app
3.4G org.apache.tomee$openejb-ssh
3.3G org.apache.tomee$arquillian-tomee-moviefun-example
3.1G org.apache.tomee$cxf-shade
2.3G org.apache.tomee$openejb-core

597G jakarta-deploy

of which:

354G org.apache.tomee$apache-tomee
73G org.apache.tomee$tomee-plume-webapp
66G org.apache.tomee$tomee-plus-webapp
60G org.apache.tomee$tomee-microprofile-webapp
42G org.apache.tomee$tomee-webprofile-webapp
4.3G org.apache.tomee$jakartaee-api
65M org.apache.tomee$tomee-project
44M org.apache.tomee$tomee
38M org.apache.tomee$transform
20M org.apache.tomee.jakarta$apache-tomee
7.2M org.apache.tomee.bom$jaxb-runtime
6.5M org.apache.tomee.bom$boms

63G tomee-8.x-deploy
25G jakarta-wip-9.x-deploy
23G master-build-full
7.0G site-publish
4.2G tomee-8.x-sanity-checks
3.1G tomee-7.0.x
2.5G master-sanity-checks
2.1G pull-request
2.0G tomee-8.x-build-full
2.0G TOMEE-3872
1.7G master-pull-request
1.1G tomee-8.x-owasp-check
1.1G master-owasp-check
1.1G master-build-quick
945M tomee-8.x-build-quick
35M tomee-jakartaee-api-master
27M tomee-patch-plugin-deploy
428K clean-repo
256K tomee-jenkins-pipelines


> Gruß
> Richard
>
> -------- Ursprüngliche Nachricht --------
> Von: Gavin McDonald <gm...@apache.org>
> Antwort an: builds@apache.org, gmcdonald@apache.org
> An: builds <bu...@apache.org>
> Betreff: ci-builds all 3.6TB disk is full!
> Datum: Wed, 20 Apr 2022 09:27:28 +0200
>
> > Hi All,
> >
> > Seems we need to do another cull of projects storing way too much
> > data.
> >
> > Below are everyone above 1GB. Just FYI, 1GB is fine, likely 50GB is
> > fine,
> > but above
> > that, its just too much. I will be removing 1TB of data from wherever
> > I can
> > get it.
> >
> > Please, look after your jobs, and your fellow projects by limiting
> > what you
> > keep.
> >
> > 1.6T    Tomee
> > 451G    Kafka
> > 303G    james
> > 176G    carbondata
> > 129G    Jackrabbit
> > 71G     Brooklyn
> > 64G     Sling
> > 64G     Netbeans
> > 60G     Ranger
> > 38G     AsterixDB
> > 33G     OODT
> > 29G     Tika
> > 27G     Syncope
> > 24G     Atlas
> > 20G     IoTDB
> > 18G     CXF
> > 16G     POI
> > 11G     Solr
> > 11G     Mesos
> > 8.7G    Royale
> > 7.8G    Lucene
> > 7.6G    MyFaces
> > 7.6G    Directory
> > 6.4G    OpenJPA
> > 6.0G    ManifoldCF
> > 5.9G    ActiveMQ
> > 5.7G    Logging
> > 5.6G    Archiva
> > 5.5G    UIMA
> > 5.3G    ctakes
> > 4.7G    Heron
> > 4.6G    Jena
> > 4.0G    OpenOffice
> > 3.8G    Cloudstack
> > 3.4G    Shiro
> > 2.5G    Qpid
> > 2.1G    JSPWiki
> > 2.1G    JMeter
> > 2.0G    JClouds
> > 1.8G    Santuario
> > 1.8G    OpenMeetings
> > 1.8G    Camel
> > 1.7G    Karaf
> > 1.7G    HttpComponents
> > 1.7G    Ant
> > 1.5G    Tapestry
> > 1.5G    Commons
> > 1.3G    DeltaSpike
> > 1.2G    Rya
> > 1.2G    Aries
> > 1.2G    Accumulo
> > 1.1G    PDFBox
> >
> > --
> >
> > *Gavin McDonald*
> > Systems Administrator
> > ASF Infrastructure Team
>
>

-- 

*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team

Fwd: ci-builds all 3.6TB disk is full!

Posted by Richard Zowalla <rz...@apache.org>.
Hi,

seems we are the "top" consumers with 1,6TB disk usage on the CI
infrastructure.

I looked at some of our jobs and found, that there is no retention
policy in place (for some of them). I added a policy similar to what we
had in the past for newly created jobs. Looks like the retention policy
is not copied then cloning jobs.

In addition, I asked Gavin, if he can provide a "du" listing for our
jobs, so we can better dig into this issue.

Gruß
Richard

-------- Ursprüngliche Nachricht --------
Von: Gavin McDonald <gm...@apache.org>
Antwort an: builds@apache.org, gmcdonald@apache.org
An: builds <bu...@apache.org>
Betreff: ci-builds all 3.6TB disk is full!
Datum: Wed, 20 Apr 2022 09:27:28 +0200

> Hi All,
> 
> Seems we need to do another cull of projects storing way too much
> data.
> 
> Below are everyone above 1GB. Just FYI, 1GB is fine, likely 50GB is
> fine,
> but above
> that, its just too much. I will be removing 1TB of data from wherever
> I can
> get it.
> 
> Please, look after your jobs, and your fellow projects by limiting
> what you
> keep.
> 
> 1.6T    Tomee
> 451G    Kafka
> 303G    james
> 176G    carbondata
> 129G    Jackrabbit
> 71G     Brooklyn
> 64G     Sling
> 64G     Netbeans
> 60G     Ranger
> 38G     AsterixDB
> 33G     OODT
> 29G     Tika
> 27G     Syncope
> 24G     Atlas
> 20G     IoTDB
> 18G     CXF
> 16G     POI
> 11G     Solr
> 11G     Mesos
> 8.7G    Royale
> 7.8G    Lucene
> 7.6G    MyFaces
> 7.6G    Directory
> 6.4G    OpenJPA
> 6.0G    ManifoldCF
> 5.9G    ActiveMQ
> 5.7G    Logging
> 5.6G    Archiva
> 5.5G    UIMA
> 5.3G    ctakes
> 4.7G    Heron
> 4.6G    Jena
> 4.0G    OpenOffice
> 3.8G    Cloudstack
> 3.4G    Shiro
> 2.5G    Qpid
> 2.1G    JSPWiki
> 2.1G    JMeter
> 2.0G    JClouds
> 1.8G    Santuario
> 1.8G    OpenMeetings
> 1.8G    Camel
> 1.7G    Karaf
> 1.7G    HttpComponents
> 1.7G    Ant
> 1.5G    Tapestry
> 1.5G    Commons
> 1.3G    DeltaSpike
> 1.2G    Rya
> 1.2G    Aries
> 1.2G    Accumulo
> 1.1G    PDFBox
> 
> -- 
> 
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team


Re: ci-builds all 3.6TB disk is full!

Posted by Dominik Psenner <dp...@gmail.com>.
Before blindly shooting away storage, I propose to look at what is actually
stored. For example, there could be log4net builds in need of storage. I
assume that log4net is still building on jenkins.

On Wed, 20 Apr 2022 at 20:15, Gary Gregory <ga...@gmail.com> wrote:

> I guess you can ask Gavin ;-)
>
> Gary
>
> On Wed, Apr 20, 2022, 09:12 Volkan Yazıcı <vo...@yazi.ci> wrote:
>
> > We haven't been using Jenkins for months. I think we can safely remove
> all
> > `target` folders and whatever cache there is. But I am curious, how is
> this
> > number exactly obtained?
> >
> > On Wed, Apr 20, 2022 at 3:04 PM Gary Gregory <ga...@gmail.com>
> > wrote:
> >
> > > Hi All,
> > > Looks like our Looging project is using too much space, almost 6 GB in
> > the
> > > CI system. Does anyone here know how to clean this up and prevent it
> from
> > > happening?
> > >
> > > Gary
> > >
> > > ---------- Forwarded message ---------
> > > From: Gavin McDonald <gm...@apache.org>
> > > Date: Wed, Apr 20, 2022, 03:27
> > > Subject: ci-builds all 3.6TB disk is full!
> > > To: builds <bu...@apache.org>
> > >
> > >
> > > Hi All,
> > >
> > > Seems we need to do another cull of projects storing way too much data.
> > >
> > > Below are everyone above 1GB. Just FYI, 1GB is fine, likely 50GB is
> fine,
> > > but above
> > > that, its just too much. I will be removing 1TB of data from wherever I
> > can
> > > get it.
> > >
> > > Please, look after your jobs, and your fellow projects by limiting what
> > you
> > > keep.
> > >
> > > 1.6T    Tomee
> > > 451G    Kafka
> > > 303G    james
> > > 176G    carbondata
> > > 129G    Jackrabbit
> > > 71G     Brooklyn
> > > 64G     Sling
> > > 64G     Netbeans
> > > 60G     Ranger
> > > 38G     AsterixDB
> > > 33G     OODT
> > > 29G     Tika
> > > 27G     Syncope
> > > 24G     Atlas
> > > 20G     IoTDB
> > > 18G     CXF
> > > 16G     POI
> > > 11G     Solr
> > > 11G     Mesos
> > > 8.7G    Royale
> > > 7.8G    Lucene
> > > 7.6G    MyFaces
> > > 7.6G    Directory
> > > 6.4G    OpenJPA
> > > 6.0G    ManifoldCF
> > > 5.9G    ActiveMQ
> > > 5.7G    Logging
> > > 5.6G    Archiva
> > > 5.5G    UIMA
> > > 5.3G    ctakes
> > > 4.7G    Heron
> > > 4.6G    Jena
> > > 4.0G    OpenOffice
> > > 3.8G    Cloudstack
> > > 3.4G    Shiro
> > > 2.5G    Qpid
> > > 2.1G    JSPWiki
> > > 2.1G    JMeter
> > > 2.0G    JClouds
> > > 1.8G    Santuario
> > > 1.8G    OpenMeetings
> > > 1.8G    Camel
> > > 1.7G    Karaf
> > > 1.7G    HttpComponents
> > > 1.7G    Ant
> > > 1.5G    Tapestry
> > > 1.5G    Commons
> > > 1.3G    DeltaSpike
> > > 1.2G    Rya
> > > 1.2G    Aries
> > > 1.2G    Accumulo
> > > 1.1G    PDFBox
> > >
> > > --
> > >
> > > *Gavin McDonald*
> > > Systems Administrator
> > > ASF Infrastructure Team
> > >
> >
>


-- 
Dominik Psenner

Re: ci-builds all 3.6TB disk is full!

Posted by Gary Gregory <ga...@gmail.com>.
I guess you can ask Gavin ;-)

Gary

On Wed, Apr 20, 2022, 09:12 Volkan Yazıcı <vo...@yazi.ci> wrote:

> We haven't been using Jenkins for months. I think we can safely remove all
> `target` folders and whatever cache there is. But I am curious, how is this
> number exactly obtained?
>
> On Wed, Apr 20, 2022 at 3:04 PM Gary Gregory <ga...@gmail.com>
> wrote:
>
> > Hi All,
> > Looks like our Looging project is using too much space, almost 6 GB in
> the
> > CI system. Does anyone here know how to clean this up and prevent it from
> > happening?
> >
> > Gary
> >
> > ---------- Forwarded message ---------
> > From: Gavin McDonald <gm...@apache.org>
> > Date: Wed, Apr 20, 2022, 03:27
> > Subject: ci-builds all 3.6TB disk is full!
> > To: builds <bu...@apache.org>
> >
> >
> > Hi All,
> >
> > Seems we need to do another cull of projects storing way too much data.
> >
> > Below are everyone above 1GB. Just FYI, 1GB is fine, likely 50GB is fine,
> > but above
> > that, its just too much. I will be removing 1TB of data from wherever I
> can
> > get it.
> >
> > Please, look after your jobs, and your fellow projects by limiting what
> you
> > keep.
> >
> > 1.6T    Tomee
> > 451G    Kafka
> > 303G    james
> > 176G    carbondata
> > 129G    Jackrabbit
> > 71G     Brooklyn
> > 64G     Sling
> > 64G     Netbeans
> > 60G     Ranger
> > 38G     AsterixDB
> > 33G     OODT
> > 29G     Tika
> > 27G     Syncope
> > 24G     Atlas
> > 20G     IoTDB
> > 18G     CXF
> > 16G     POI
> > 11G     Solr
> > 11G     Mesos
> > 8.7G    Royale
> > 7.8G    Lucene
> > 7.6G    MyFaces
> > 7.6G    Directory
> > 6.4G    OpenJPA
> > 6.0G    ManifoldCF
> > 5.9G    ActiveMQ
> > 5.7G    Logging
> > 5.6G    Archiva
> > 5.5G    UIMA
> > 5.3G    ctakes
> > 4.7G    Heron
> > 4.6G    Jena
> > 4.0G    OpenOffice
> > 3.8G    Cloudstack
> > 3.4G    Shiro
> > 2.5G    Qpid
> > 2.1G    JSPWiki
> > 2.1G    JMeter
> > 2.0G    JClouds
> > 1.8G    Santuario
> > 1.8G    OpenMeetings
> > 1.8G    Camel
> > 1.7G    Karaf
> > 1.7G    HttpComponents
> > 1.7G    Ant
> > 1.5G    Tapestry
> > 1.5G    Commons
> > 1.3G    DeltaSpike
> > 1.2G    Rya
> > 1.2G    Aries
> > 1.2G    Accumulo
> > 1.1G    PDFBox
> >
> > --
> >
> > *Gavin McDonald*
> > Systems Administrator
> > ASF Infrastructure Team
> >
>

Re: ci-builds all 3.6TB disk is full!

Posted by Volkan Yazıcı <vo...@yazi.ci>.
We haven't been using Jenkins for months. I think we can safely remove all
`target` folders and whatever cache there is. But I am curious, how is this
number exactly obtained?

On Wed, Apr 20, 2022 at 3:04 PM Gary Gregory <ga...@gmail.com> wrote:

> Hi All,
> Looks like our Looging project is using too much space, almost 6 GB in the
> CI system. Does anyone here know how to clean this up and prevent it from
> happening?
>
> Gary
>
> ---------- Forwarded message ---------
> From: Gavin McDonald <gm...@apache.org>
> Date: Wed, Apr 20, 2022, 03:27
> Subject: ci-builds all 3.6TB disk is full!
> To: builds <bu...@apache.org>
>
>
> Hi All,
>
> Seems we need to do another cull of projects storing way too much data.
>
> Below are everyone above 1GB. Just FYI, 1GB is fine, likely 50GB is fine,
> but above
> that, its just too much. I will be removing 1TB of data from wherever I can
> get it.
>
> Please, look after your jobs, and your fellow projects by limiting what you
> keep.
>
> 1.6T    Tomee
> 451G    Kafka
> 303G    james
> 176G    carbondata
> 129G    Jackrabbit
> 71G     Brooklyn
> 64G     Sling
> 64G     Netbeans
> 60G     Ranger
> 38G     AsterixDB
> 33G     OODT
> 29G     Tika
> 27G     Syncope
> 24G     Atlas
> 20G     IoTDB
> 18G     CXF
> 16G     POI
> 11G     Solr
> 11G     Mesos
> 8.7G    Royale
> 7.8G    Lucene
> 7.6G    MyFaces
> 7.6G    Directory
> 6.4G    OpenJPA
> 6.0G    ManifoldCF
> 5.9G    ActiveMQ
> 5.7G    Logging
> 5.6G    Archiva
> 5.5G    UIMA
> 5.3G    ctakes
> 4.7G    Heron
> 4.6G    Jena
> 4.0G    OpenOffice
> 3.8G    Cloudstack
> 3.4G    Shiro
> 2.5G    Qpid
> 2.1G    JSPWiki
> 2.1G    JMeter
> 2.0G    JClouds
> 1.8G    Santuario
> 1.8G    OpenMeetings
> 1.8G    Camel
> 1.7G    Karaf
> 1.7G    HttpComponents
> 1.7G    Ant
> 1.5G    Tapestry
> 1.5G    Commons
> 1.3G    DeltaSpike
> 1.2G    Rya
> 1.2G    Aries
> 1.2G    Accumulo
> 1.1G    PDFBox
>
> --
>
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team
>

Fwd: ci-builds all 3.6TB disk is full!

Posted by Gary Gregory <ga...@gmail.com>.
Hi All,
Looks like our Looging project is using too much space, almost 6 GB in the
CI system. Does anyone here know how to clean this up and prevent it from
happening?

Gary

---------- Forwarded message ---------
From: Gavin McDonald <gm...@apache.org>
Date: Wed, Apr 20, 2022, 03:27
Subject: ci-builds all 3.6TB disk is full!
To: builds <bu...@apache.org>


Hi All,

Seems we need to do another cull of projects storing way too much data.

Below are everyone above 1GB. Just FYI, 1GB is fine, likely 50GB is fine,
but above
that, its just too much. I will be removing 1TB of data from wherever I can
get it.

Please, look after your jobs, and your fellow projects by limiting what you
keep.

1.6T    Tomee
451G    Kafka
303G    james
176G    carbondata
129G    Jackrabbit
71G     Brooklyn
64G     Sling
64G     Netbeans
60G     Ranger
38G     AsterixDB
33G     OODT
29G     Tika
27G     Syncope
24G     Atlas
20G     IoTDB
18G     CXF
16G     POI
11G     Solr
11G     Mesos
8.7G    Royale
7.8G    Lucene
7.6G    MyFaces
7.6G    Directory
6.4G    OpenJPA
6.0G    ManifoldCF
5.9G    ActiveMQ
5.7G    Logging
5.6G    Archiva
5.5G    UIMA
5.3G    ctakes
4.7G    Heron
4.6G    Jena
4.0G    OpenOffice
3.8G    Cloudstack
3.4G    Shiro
2.5G    Qpid
2.1G    JSPWiki
2.1G    JMeter
2.0G    JClouds
1.8G    Santuario
1.8G    OpenMeetings
1.8G    Camel
1.7G    Karaf
1.7G    HttpComponents
1.7G    Ant
1.5G    Tapestry
1.5G    Commons
1.3G    DeltaSpike
1.2G    Rya
1.2G    Aries
1.2G    Accumulo
1.1G    PDFBox

-- 

*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team

Re: ci-builds all 3.6TB disk is full!

Posted by Richard Zowalla <rz...@apache.org>.
Hi,

would be nice to get a "du" for TomEE too...

Gruß
Richard

Am Mittwoch, dem 20.04.2022 um 10:32 +0300 schrieb Robert Munteanu:
> Hi,
> 
> On Wed, 2022-04-20 at 09:27 +0200, Gavin McDonald wrote:
> > 64G     Sling
> 
> We have large a number of jobs at
> 
>   https://ci-builds.apache.org/job/Sling/job/modules/
> 
> Would it be possible to get a listing of the top jobs in terms of
> disk
> usage?
> 
> Thanks,
> Robert


Re: ci-builds all 3.6TB disk is full!

Posted by Maxim Solodovnik <so...@gmail.com>.
/3 days/7 days/

sorry :((

On Wed, 20 Apr 2022 at 14:51, Maxim Solodovnik <so...@gmail.com> wrote:
>
> Hello Gavin,
>
> We have usual settings for all OpenMeetings builds (keep no more than
> 3 builds 3 days maximum)
> Can it be the workspace?
> Would it be possible to delete workspace older than 24 hours or so?
>
> On Wed, 20 Apr 2022 at 14:37, Gavin McDonald <gm...@apache.org> wrote:
> >
> > On Wed, Apr 20, 2022 at 9:32 AM Robert Munteanu <ro...@apache.org> wrote:
> >
> > > Hi,
> > >
> > > On Wed, 2022-04-20 at 09:27 +0200, Gavin McDonald wrote:
> > > > 64G     Sling
> > >
> > > We have large a number of jobs at
> > >
> > >   https://ci-builds.apache.org/job/Sling/job/modules/
> > >
> > > Would it be possible to get a listing of the top jobs in terms of disk
> > > usage?
> > >
> >
> > Sure listing those above 100mb
> >
> > 12G sling-org-apache-sling-app-cms
> > 8.7G sling-org-apache-sling-event
> > 8.0G sling-org-apache-sling-starter
> > 4.0G sling-org-apache-sling-i18n
> > 3.1G sling-org.27mv5jk8bpa0.ermanager
> > 2.7G sling-org-apache-sling-kickstart
> > 2.5G sling-org-.2trf7ri5q1.pconverter
> > 2.1G sling-org-.h4ujb8c6o.istribution
> > 1.9G sling-org.g3gsbplapvfe.ssmanager
> > 1.4G sling-slingstart-archetype
> > 1.4G sling-samples
> > 1.2G sling-whiteboard
> > 1.2G sling-org-ap.m2la5a.graphql-core
> > 992M sling-org-a.gddpna15.ntentloader
> > 836M sling-org-ap.gnsshn.jcr-repoinit
> > 800M sling-org.jgd2nlcpj3om.ion-tests
> > 765M sling-org-ap.gp64fk.ql-launchpad
> > 627M sling-org-apache-sling-clam
> > 621M sling-launchpad-debian
> > 516M sling-org-ap.ouauhi.ting-paxexam
> > 451M sling-launch.sd2cha.ne-archetype
> > 440M sling-launchpad-webapp-archetype
> > 415M sling-org-.asq13p2ok4.-thymeleaf
> > 409M sling-org-a.bgbpb0tf.pad-testing
> > 396M sling-org-apache-sling-pipes
> > 352M sling-org.lvh8f4qpt2j6.ations-it
> > 292M sling-org-ap.ph58pv.ripting-core
> > 290M sling-org.6hgkkg8ahnv8.ion-tests
> > 280M sling-org-ap.5t5tsb.g-auth-saml2
> > 274M sling-org-apache-sling-xss
> > 244M sling-org-apache-sling-auth-form
> > 227M sling-org.mulim9fse96o.-mock-oak
> > 227M sling-org.7t9t992amjh.on-journal
> > 223M sling-site
> > 201M sling-org.7pqsj447mf2.freemarker
> > 174M sling-org-ap.ti6qvk.iscovery-oak
> > 166M sling-org-a.i6boopr4.ts-resolver
> > 131M sling-org-a.kte37g7.ure-launcher
> > 116M sling-org.g2nc69q0mip2.n-content
> > 113M sling-org-apache-sling-rewriter
> > 109M sling-org-ap.j9tkfn.r-oak-server
> > 107M sling-org-ap.ofrcs2.scovery-impl
> > 104M sling-org-a.na9bmk2.urceresolver
> > 104M sling-org-a.26ndcff.pting-groovy
> > 101M sling-org-ap.4p135t.g-junit-core
> >
> >
> > > Thanks,
> > > Robert
> > >
> >
> >
> > --
> >
> > *Gavin McDonald*
> > Systems Administrator
> > ASF Infrastructure Team
>
>
>
> --
> Best regards,
> Maxim



-- 
Best regards,
Maxim

Re: ci-builds all 3.6TB disk is full!

Posted by Maxim Solodovnik <so...@gmail.com>.
Hello Gavin,

We have usual settings for all OpenMeetings builds (keep no more than
3 builds 3 days maximum)
Can it be the workspace?
Would it be possible to delete workspace older than 24 hours or so?

On Wed, 20 Apr 2022 at 14:37, Gavin McDonald <gm...@apache.org> wrote:
>
> On Wed, Apr 20, 2022 at 9:32 AM Robert Munteanu <ro...@apache.org> wrote:
>
> > Hi,
> >
> > On Wed, 2022-04-20 at 09:27 +0200, Gavin McDonald wrote:
> > > 64G     Sling
> >
> > We have large a number of jobs at
> >
> >   https://ci-builds.apache.org/job/Sling/job/modules/
> >
> > Would it be possible to get a listing of the top jobs in terms of disk
> > usage?
> >
>
> Sure listing those above 100mb
>
> 12G sling-org-apache-sling-app-cms
> 8.7G sling-org-apache-sling-event
> 8.0G sling-org-apache-sling-starter
> 4.0G sling-org-apache-sling-i18n
> 3.1G sling-org.27mv5jk8bpa0.ermanager
> 2.7G sling-org-apache-sling-kickstart
> 2.5G sling-org-.2trf7ri5q1.pconverter
> 2.1G sling-org-.h4ujb8c6o.istribution
> 1.9G sling-org.g3gsbplapvfe.ssmanager
> 1.4G sling-slingstart-archetype
> 1.4G sling-samples
> 1.2G sling-whiteboard
> 1.2G sling-org-ap.m2la5a.graphql-core
> 992M sling-org-a.gddpna15.ntentloader
> 836M sling-org-ap.gnsshn.jcr-repoinit
> 800M sling-org.jgd2nlcpj3om.ion-tests
> 765M sling-org-ap.gp64fk.ql-launchpad
> 627M sling-org-apache-sling-clam
> 621M sling-launchpad-debian
> 516M sling-org-ap.ouauhi.ting-paxexam
> 451M sling-launch.sd2cha.ne-archetype
> 440M sling-launchpad-webapp-archetype
> 415M sling-org-.asq13p2ok4.-thymeleaf
> 409M sling-org-a.bgbpb0tf.pad-testing
> 396M sling-org-apache-sling-pipes
> 352M sling-org.lvh8f4qpt2j6.ations-it
> 292M sling-org-ap.ph58pv.ripting-core
> 290M sling-org.6hgkkg8ahnv8.ion-tests
> 280M sling-org-ap.5t5tsb.g-auth-saml2
> 274M sling-org-apache-sling-xss
> 244M sling-org-apache-sling-auth-form
> 227M sling-org.mulim9fse96o.-mock-oak
> 227M sling-org.7t9t992amjh.on-journal
> 223M sling-site
> 201M sling-org.7pqsj447mf2.freemarker
> 174M sling-org-ap.ti6qvk.iscovery-oak
> 166M sling-org-a.i6boopr4.ts-resolver
> 131M sling-org-a.kte37g7.ure-launcher
> 116M sling-org.g2nc69q0mip2.n-content
> 113M sling-org-apache-sling-rewriter
> 109M sling-org-ap.j9tkfn.r-oak-server
> 107M sling-org-ap.ofrcs2.scovery-impl
> 104M sling-org-a.na9bmk2.urceresolver
> 104M sling-org-a.26ndcff.pting-groovy
> 101M sling-org-ap.4p135t.g-junit-core
>
>
> > Thanks,
> > Robert
> >
>
>
> --
>
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team



-- 
Best regards,
Maxim

Re: ci-builds all 3.6TB disk is full!

Posted by Robert Munteanu <ro...@apache.org>.
On Wed, 2022-04-20 at 14:24 +0200, Gavin McDonald wrote:
> Hi Robert
> 
> On Wed, Apr 20, 2022 at 2:10 PM Robert Munteanu <ro...@apache.org>
> wrote:
> 
> > On Wed, 2022-04-20 at 09:37 +0200, Gavin McDonald wrote:
> > > On Wed, Apr 20, 2022 at 9:32 AM Robert Munteanu
> > > <ro...@apache.org>
> > > wrote:
> > > 
> > > > Hi,
> > > > 
> > > > On Wed, 2022-04-20 at 09:27 +0200, Gavin McDonald wrote:
> > > > > 64G     Sling
> > > > 
> > > > We have large a number of jobs at
> > > > 
> > > >    https://ci-builds.apache.org/job/Sling/job/modules/
> > > > 
> > > > Would it be possible to get a listing of the top jobs in terms
> > > > of
> > > > disk
> > > > usage?
> > > > 
> > > 
> > > Sure listing those above 100mb
> > > 
> > > 12G sling-org-apache-sling-app-cms
> > 
> > I'm looking at this one since it's the largest and I am not sure
> > where
> > the large disk usage is coming from. What exactly is retained by
> > Jenkins?
> > 
> > We do have a retention policy of 10 builds set globally for Sling
> > builds, it can be seen for instance at [1], but maybe that is too
> > much
> > if we have too many active branches.
> > 
> > There are multiple things we can tweak, but I'd like to see the
> > data
> > 
> 
> /home/jenkins/jenkins-home/jobs/Sling/jobs/modules/jobs/sling-org-
> apache-sling-app-cms/branches
> # du -sh * | sort -rh
> 6.2G master - of which there are 11 jobs stored at 635M each
> 
> 635M PR-14
> 635M PR-13
> 635M dependabot.o0l8t0g19.imist-1-2-6
> 635M dependabot.d9qujnh19.imist-1-2-6
> 626M sitemap-support
> 598M PR-12
> 596M dklco-preview-img.ln1s45
> 397M forms-enhancements
> 169M SLING-8917-ldap
> 163M SLING-8947-grid-missing-fields
> 162M SLING-8930-i18n-not-reloading
> 162M SLING-8919-fix-error-dialog
> 3.9M SLING-11104
> 2.2M SLING-8913-m.h8mc0v.stance-types
> 1.7M PR-11
> 1.7M graphql-support
> 1.2M dklco-form-framework.7q4mpe
> 1012K commons-thumbnails
> 860K dklco-conf-cleanup.seumnn
> 692K provisioning-support

That is helpful, thank you. I have adjusted the global 'Orphaned Items
Strategy' for the Sling modules to only retain old jobs for 3 days.

I think that the old limit of 15 jobs was causing PR builds to stick
around too long and contributed to the increase in disk usage.

This has triggered a rescan of the Sling modules on GitHub [1] which
has started cleaning up modules. However, it's going slowly due to
GitHub API rate limits, so it may be worth checking out the disk usage
for Sling in an hour or so.

Thanks,
Robert

[1]:
https://ci-builds.apache.org/job/Sling/job/modules/computation/console

Re: ci-builds all 3.6TB disk is full!

Posted by Gavin McDonald <gm...@apache.org>.
Hi Robert

On Wed, Apr 20, 2022 at 2:10 PM Robert Munteanu <ro...@apache.org> wrote:

> On Wed, 2022-04-20 at 09:37 +0200, Gavin McDonald wrote:
> > On Wed, Apr 20, 2022 at 9:32 AM Robert Munteanu <ro...@apache.org>
> > wrote:
> >
> > > Hi,
> > >
> > > On Wed, 2022-04-20 at 09:27 +0200, Gavin McDonald wrote:
> > > > 64G     Sling
> > >
> > > We have large a number of jobs at
> > >
> > >    https://ci-builds.apache.org/job/Sling/job/modules/
> > >
> > > Would it be possible to get a listing of the top jobs in terms of
> > > disk
> > > usage?
> > >
> >
> > Sure listing those above 100mb
> >
> > 12G sling-org-apache-sling-app-cms
>
> I'm looking at this one since it's the largest and I am not sure where
> the large disk usage is coming from. What exactly is retained by
> Jenkins?
>
> We do have a retention policy of 10 builds set globally for Sling
> builds, it can be seen for instance at [1], but maybe that is too much
> if we have too many active branches.
>
> There are multiple things we can tweak, but I'd like to see the data
>

/home/jenkins/jenkins-home/jobs/Sling/jobs/modules/jobs/sling-org-apache-sling-app-cms/branches
# du -sh * | sort -rh
6.2G master - of which there are 11 jobs stored at 635M each

635M PR-14
635M PR-13
635M dependabot.o0l8t0g19.imist-1-2-6
635M dependabot.d9qujnh19.imist-1-2-6
626M sitemap-support
598M PR-12
596M dklco-preview-img.ln1s45
397M forms-enhancements
169M SLING-8917-ldap
163M SLING-8947-grid-missing-fields
162M SLING-8930-i18n-not-reloading
162M SLING-8919-fix-error-dialog
3.9M SLING-11104
2.2M SLING-8913-m.h8mc0v.stance-types
1.7M PR-11
1.7M graphql-support
1.2M dklco-form-framework.7q4mpe
1012K commons-thumbnails
860K dklco-conf-cleanup.seumnn
692K provisioning-support



> first.
>
> Thanks,
> Robert
>
> [1]:
>
> https://ci-builds.apache.org/job/Sling/job/modules/job/sling-org-apache-sling-app-cms/job/master/configure
>


-- 

*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team

Re: ci-builds all 3.6TB disk is full!

Posted by Robert Munteanu <ro...@apache.org>.
On Wed, 2022-04-20 at 09:37 +0200, Gavin McDonald wrote:
> On Wed, Apr 20, 2022 at 9:32 AM Robert Munteanu <ro...@apache.org>
> wrote:
> 
> > Hi,
> > 
> > On Wed, 2022-04-20 at 09:27 +0200, Gavin McDonald wrote:
> > > 64G     Sling
> > 
> > We have large a number of jobs at
> > 
> >    https://ci-builds.apache.org/job/Sling/job/modules/
> > 
> > Would it be possible to get a listing of the top jobs in terms of
> > disk
> > usage?
> > 
> 
> Sure listing those above 100mb
> 
> 12G sling-org-apache-sling-app-cms

I'm looking at this one since it's the largest and I am not sure where
the large disk usage is coming from. What exactly is retained by
Jenkins?

We do have a retention policy of 10 builds set globally for Sling
builds, it can be seen for instance at [1], but maybe that is too much
if we have too many active branches.

There are multiple things we can tweak, but I'd like to see the data
first.

Thanks,
Robert

[1]:
https://ci-builds.apache.org/job/Sling/job/modules/job/sling-org-apache-sling-app-cms/job/master/configure

Re: ci-builds all 3.6TB disk is full!

Posted by Gavin McDonald <gm...@apache.org>.
On Wed, Apr 20, 2022 at 9:32 AM Robert Munteanu <ro...@apache.org> wrote:

> Hi,
>
> On Wed, 2022-04-20 at 09:27 +0200, Gavin McDonald wrote:
> > 64G     Sling
>
> We have large a number of jobs at
>
>   https://ci-builds.apache.org/job/Sling/job/modules/
>
> Would it be possible to get a listing of the top jobs in terms of disk
> usage?
>

Sure listing those above 100mb

12G sling-org-apache-sling-app-cms
8.7G sling-org-apache-sling-event
8.0G sling-org-apache-sling-starter
4.0G sling-org-apache-sling-i18n
3.1G sling-org.27mv5jk8bpa0.ermanager
2.7G sling-org-apache-sling-kickstart
2.5G sling-org-.2trf7ri5q1.pconverter
2.1G sling-org-.h4ujb8c6o.istribution
1.9G sling-org.g3gsbplapvfe.ssmanager
1.4G sling-slingstart-archetype
1.4G sling-samples
1.2G sling-whiteboard
1.2G sling-org-ap.m2la5a.graphql-core
992M sling-org-a.gddpna15.ntentloader
836M sling-org-ap.gnsshn.jcr-repoinit
800M sling-org.jgd2nlcpj3om.ion-tests
765M sling-org-ap.gp64fk.ql-launchpad
627M sling-org-apache-sling-clam
621M sling-launchpad-debian
516M sling-org-ap.ouauhi.ting-paxexam
451M sling-launch.sd2cha.ne-archetype
440M sling-launchpad-webapp-archetype
415M sling-org-.asq13p2ok4.-thymeleaf
409M sling-org-a.bgbpb0tf.pad-testing
396M sling-org-apache-sling-pipes
352M sling-org.lvh8f4qpt2j6.ations-it
292M sling-org-ap.ph58pv.ripting-core
290M sling-org.6hgkkg8ahnv8.ion-tests
280M sling-org-ap.5t5tsb.g-auth-saml2
274M sling-org-apache-sling-xss
244M sling-org-apache-sling-auth-form
227M sling-org.mulim9fse96o.-mock-oak
227M sling-org.7t9t992amjh.on-journal
223M sling-site
201M sling-org.7pqsj447mf2.freemarker
174M sling-org-ap.ti6qvk.iscovery-oak
166M sling-org-a.i6boopr4.ts-resolver
131M sling-org-a.kte37g7.ure-launcher
116M sling-org.g2nc69q0mip2.n-content
113M sling-org-apache-sling-rewriter
109M sling-org-ap.j9tkfn.r-oak-server
107M sling-org-ap.ofrcs2.scovery-impl
104M sling-org-a.na9bmk2.urceresolver
104M sling-org-a.26ndcff.pting-groovy
101M sling-org-ap.4p135t.g-junit-core


> Thanks,
> Robert
>


-- 

*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team

Re: ci-builds all 3.6TB disk is full!

Posted by Robert Munteanu <ro...@apache.org>.
Hi,

On Wed, 2022-04-20 at 09:27 +0200, Gavin McDonald wrote:
> 64G     Sling

We have large a number of jobs at

  https://ci-builds.apache.org/job/Sling/job/modules/

Would it be possible to get a listing of the top jobs in terms of disk
usage?

Thanks,
Robert