[jira] [Work logged] (BEAM-9865) Clean up jenkins workspaces for successful jobs

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-9865?focusedWorklogId=518163&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-518163 ]

ASF GitHub Bot logged work on BEAM-9865:
----------------------------------------

                Author: ASF GitHub Bot
            Created on: 30/Nov/20 22:47
            Start Date: 30/Nov/20 22:47
    Worklog Time Spent: 10m 
      Work Description: kennknowles commented on pull request #12326:
URL: https://github.com/apache/beam/pull/12326#issuecomment-736101571


   Since you pointed me here, the problem is that there are important transparent caches in the workspace, such as the maven dep downloads. It is slow and wasteful for all parties to hit maven central on every build, but more importantly it is flaky.
   
   TBH "pure" caches could/should probably live outside the workspace. I think the Gradle dep cache must be, or we'd see huge flakiness of Gradle jobs. We can clone `--depth 1` to eliminate the costs of the repo history. But I kind of agree with the simple approach of wiping out the workspace to keep it simple and not depend on details of what happens in the workspace. The issue is we need to get the replacement in place first. I think setting up the `.m2` to be outside the workspace is the move in this case.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 518163)
    Time Spent: 3.5h  (was: 3h 20m)

> Clean up jenkins workspaces for successful jobs
> -----------------------------------------------
>
>                 Key: BEAM-9865
>                 URL: https://issues.apache.org/jira/browse/BEAM-9865
>             Project: Beam
>          Issue Type: Bug
>          Components: build-system
>            Reporter: Udi Meiri
>            Priority: P3
>          Time Spent: 3.5h
>  Remaining Estimate: 0h
>
> Two recent bugs (and many more in the past) citing lack of disk space:
> https://issues.apache.org/jira/browse/BEAM-9854
> https://issues.apache.org/jira/browse/BEAM-9462
> There are around 150 workspaces on each Jenkins machine: apache-beam-jenkins-1..15.
> Total size:
> 1: 175G
> 7: 158G
> 8: 173G
> The majority of jobs use a clone of the Beam read/write files under src/, which is wiped out at the start of the job (wipeOutWorkspace()), so there is really no point in keeping workspace files around after the job has completed successfully.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)