You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by Chris Lambertus <cm...@apache.org> on 2018/09/30 22:57:23 UTC
Re: No space on H21 (Was: Fwd: Build failed in Jenkins:
guacamole-client-coverity #73)
The main culprit on H31 appears to be pulsar ws-cleanup jobs consuming almost 40% of the disk. (The problem on H21 was something docker-related.)
This is the second project I have found (Brooklyn was the first) which is leaving ws-cleanup directories all over the workspace and filling up the disks. Each one of these was 500+ MB. Pulsar, please address ASAP.
drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:19 pulsar_precommit_cpp@2_ws-cleanup_1535737112280
drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:18 pulsar_precommit_cpp@2_ws-cleanup_1536174576053
drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 06:04 pulsar_precommit_cpp@2_ws-cleanup_1537292379178
drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 22:45 pulsar_precommit_cpp@2_ws-cleanup_1537397120862
drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45 pulsar_precommit_cpp@2_ws-cleanup_1537403378279
drwxr-xr-x 55 jenkins jenkins 4096 Sep 21 23:31 pulsar_precommit_cpp@2_ws-cleanup_1537572701829
drwxr-xr-x 54 jenkins jenkins 4096 Sep 26 16:04 pulsar_precommit_cpp@2_ws-cleanup_1537977879341
drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 22:23 pulsar_precommit_cpp@2_ws-cleanup_1538087035027
drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:14 pulsar_precommit_cpp_ws-cleanup_1535699830230
drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:16 pulsar_precommit_cpp_ws-cleanup_1535735511423
drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:13 pulsar_precommit_cpp_ws-cleanup_1535786064397
drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:18 pulsar_precommit_cpp_ws-cleanup_1536096963586
drwxr-xr-x 53 jenkins jenkins 4096 Sep 21 06:47 pulsar_precommit_cpp_ws-cleanup_1536114938910
drwxr-xr-x 53 jenkins jenkins 4096 Sep 21 06:45 pulsar_precommit_cpp_ws-cleanup_1536173165909
drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:38 pulsar_precommit_cpp_ws-cleanup_1536175584418
drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:50 pulsar_precommit_cpp_ws-cleanup_1536180707506
drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:38 pulsar_precommit_cpp_ws-cleanup_1536182927940
drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:51 pulsar_precommit_cpp_ws-cleanup_1536185193716
drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:50 pulsar_precommit_cpp_ws-cleanup_1536273709029
drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 05:59 pulsar_precommit_cpp_ws-cleanup_1536316964794
drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:06 pulsar_precommit_cpp_ws-cleanup_1536675650893
drwxr-xr-x 53 jenkins jenkins 4096 Sep 28 06:08 pulsar_precommit_cpp_ws-cleanup_1536879616962
drwxr-xr-x 54 jenkins jenkins 4096 Sep 30 19:53 pulsar_precommit_cpp_ws-cleanup_1536891056968
drwxr-xr-x 54 jenkins jenkins 4096 Sep 30 19:47 pulsar_precommit_cpp_ws-cleanup_1536966466683
drwxr-xr-x 54 jenkins jenkins 4096 Sep 18 00:43 pulsar_precommit_cpp_ws-cleanup_1537216387387
drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 00:45 pulsar_precommit_cpp_ws-cleanup_1537292309413
drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 00:45 pulsar_precommit_cpp_ws-cleanup_1537308526251
drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45 pulsar_precommit_cpp_ws-cleanup_1537331780420
drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45 pulsar_precommit_cpp_ws-cleanup_1537379886715
drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46 pulsar_precommit_cpp_ws-cleanup_1537386197893
drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46 pulsar_precommit_cpp_ws-cleanup_1537397073464
drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46 pulsar_precommit_cpp_ws-cleanup_1537403247575
drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:43 pulsar_precommit_cpp_ws-cleanup_1537413954888
drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45 pulsar_precommit_cpp_ws-cleanup_1537417322151
drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45 pulsar_precommit_cpp_ws-cleanup_1537421381129
drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:44 pulsar_precommit_cpp_ws-cleanup_1537469348852
drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45 pulsar_precommit_cpp_ws-cleanup_1537489399468
drwxr-xr-x 54 jenkins jenkins 4096 Sep 22 00:44 pulsar_precommit_cpp_ws-cleanup_1537566528660
drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 23:12 pulsar_precommit_cpp_ws-cleanup_1537571549978
drwxr-xr-x 55 jenkins jenkins 4096 Sep 22 00:44 pulsar_precommit_cpp_ws-cleanup_1537573605258
drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 00:43 pulsar_precommit_cpp_ws-cleanup_1537855939828
drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 11:18 pulsar_precommit_cpp_ws-cleanup_1537874305575
drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 12:41 pulsar_precommit_cpp_ws-cleanup_1537879296622
drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 01:39 pulsar_precommit_cpp_ws-cleanup_1537925955967
drwxr-xr-x 54 jenkins jenkins 4096 Sep 26 13:17 pulsar_precommit_cpp_ws-cleanup_1537967826838
drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 15:55 pulsar_precommit_cpp_ws-cleanup_1537977326736
drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 17:34 pulsar_precommit_cpp_ws-cleanup_1537983268177
drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 22:04 pulsar_precommit_cpp_ws-cleanup_1538085846793
drwxr-xr-x 55 jenkins jenkins 4096 Sep 28 00:20 pulsar_precommit_cpp_ws-cleanup_1538094024821
drwxr-xr-x 55 jenkins jenkins 4096 Sep 28 17:42 pulsar_precommit_cpp_ws-cleanup_1538156543343
drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:02 pulsar_precommit_integrationtests_ws-cleanup_1536298125841
drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:00 pulsar_precommit_integrationtests_ws-cleanup_1536968128660
drwxr-xr-x 54 jenkins jenkins 4096 Sep 18 00:43 pulsar_precommit_integrationtests_ws-cleanup_1537218541598
drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45 pulsar_precommit_integrationtests_ws-cleanup_1537331703324
drwxr-xr-x 54 jenkins jenkins 4096 Sep 22 00:43 pulsar_precommit_integrationtests_ws-cleanup_1537553748859
drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 00:43 pulsar_precommit_integrationtests_ws-cleanup_1537730402937
drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 07:48 pulsar_precommit_integrationtests_ws-cleanup_1537775336586
drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 23:30 pulsar_precommit_integrationtests_ws-cleanup_1537831851711
drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 23:08 pulsar_precommit_integrationtests_ws-cleanup_1537916928485
drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 15:21 pulsar_precommit_integrationtests_ws-cleanup_1538061702243
-Chris
ASF Infra
> On Sep 30, 2018, at 3:12 PM, Chris Thistlethwaite <ct...@outlook.com> wrote:
>
> I've been working on H21 and H31. They both had a some docker issues where the process was locked and I couldn't do anything with a few containers that were over 8 days old.
>
> Thought we might be good this morning so I threw them back into rotation. Guess not, so I'll take them back out.
>
> -Chris T.
> ________________________________
> From: Mike Jumper <mj...@apache.org>
> Sent: Sunday, September 30, 2018 6:03:51 PM
> To: builds@apache.org
> Subject: No space on H21 (Was: Fwd: Build failed in Jenkins: guacamole-client-coverity #73)
>
> FYI - we're currently seeing disk space issues on H21.
>
> - Mike
>
>
> ---------- Forwarded message ---------
> From: Apache Jenkins Server <je...@builds.apache.org>
> Date: Sun, Sep 30, 2018, 14:37
> Subject: Build failed in Jenkins: guacamole-client-coverity #73
> To: <de...@guacamole.apache.org>
>
>
> See <
> https://builds.apache.org/job/guacamole-client-coverity/73/display/redirect>
>
> ------------------------------------------
> Started by user mjumper
> [EnvInject] - Loading node environment variables.
> Building remotely on H21 (ubuntu xenial) in workspace <
> https://builds.apache.org/job/guacamole-client-coverity/ws/>
> [WS-CLEANUP] Deleting project workspace...
> Cloning the remote Git repository
> Cloning repository
> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>> git init <
> https://builds.apache.org/job/guacamole-client-coverity/ws/guacamole-client>
> # timeout=10
> Fetching upstream changes from
> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>> git --version # timeout=10
>> git fetch --tags --progress
> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> +refs/heads/*:refs/remotes/origin/*
> ERROR: Error cloning remote repo 'origin'
> hudson.plugins.git.GitException: Command "git fetch --tags --progress
> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> +refs/heads/*:refs/remotes/origin/*" returned status code 128:
> stdout:
> stderr: remote: Counting objects: 62166, done.
> remote: Compressing objects: 0% (1/16096) remote: Compressing
> objects: 1% (161/16096) remote: Compressing objects: 2%
> (322/16096) remote: Compressing objects: 3% (483/16096)
> remote: Compressing objects: 4% (644/16096) remote:
> Compressing objects: 5% (805/16096) remote: Compressing
> objects: 6% (966/16096) remote: Compressing objects: 7%
> (1127/16096) remote: Compressing objects: 8% (1288/16096)
> remote: Compressing objects: 9% (1449/16096) remote:
> Compressing objects: 10% (1610/16096) remote: Compressing
> objects: 11% (1771/16096) remote: Compressing objects: 12%
> (1932/16096) remote: Compressing objects: 13% (2093/16096)
> remote: Compressing objects: 14% (2254/16096) remote:
> Compressing objects: 15% (2415/16096) remote: Compressing
> objects: 16% (2576/16096) remote: Compressing objects: 17%
> (2737/16096) remote: Compressing objects: 18% (2898/16096)
> remote: Compressing objects: 19% (3059/16096) remote:
> Compressing objects: 20% (3220/16096) remote: Compressing
> objects: 21% (3381/16096) remote: Compressing objects: 22%
> (3542/16096) remote: Compressing objects: 23% (3703/16096)
> remote: Compressing objects: 24% (3864/16096) remote:
> Compressing objects: 25% (4024/16096) remote: Compressing
> objects: 26% (4185/16096) remote: Compressing objects: 27%
> (4346/16096) remote: Compressing objects: 28% (4507/16096)
> remote: Compressing objects: 29% (4668/16096) remote:
> Compressing objects: 30% (4829/16096) remote: Compressing
> objects: 31% (4990/16096) remote: Compressing objects: 32%
> (5151/16096) remote: Compressing objects: 33% (5312/16096)
> remote: Compressing objects: 34% (5473/16096) remote:
> Compressing objects: 35% (5634/16096) remote: Compressing
> objects: 36% (5795/16096) remote: Compressing objects: 37%
> (5956/16096) remote: Compressing objects: 38% (6117/16096)
> remote: Compressing objects: 39% (6278/16096) remote:
> Compressing objects: 40% (6439/16096) remote: Compressing
> objects: 41% (6600/16096) remote: Compressing objects: 42%
> (6761/16096) remote: Compressing objects: 43% (6922/16096)
> remote: Compressing objects: 44% (7083/16096) remote:
> Compressing objects: 45% (7244/16096) remote: Compressing
> objects: 46% (7405/16096) remote: Compressing objects: 47%
> (7566/16096) remote: Compressing objects: 48% (7727/16096)
> remote: Compressing objects: 49% (7888/16096) remote:
> Compressing objects: 50% (8048/16096) remote: Compressing
> objects: 51% (8209/16096) remote: Compressing objects: 52%
> (8370/16096) remote: Compressing objects: 53% (8531/16096)
> remote: Compressing objects: 54% (8692/16096) remote:
> Compressing objects: 55% (8853/16096) remote: Compressing
> objects: 56% (9014/16096) remote: Compressing objects: 57%
> (9175/16096) remote: Compressing objects: 58% (9336/16096)
> remote: Compressing objects: 59% (9497/16096) remote:
> Compressing objects: 60% (9658/16096) remote: Compressing
> objects: 61% (9819/16096) remote: Compressing objects: 62%
> (9980/16096) remote: Compressing objects: 63% (10141/16096)
> remote: Compressing objects: 64% (10302/16096) remote:
> Compressing objects: 65% (10463/16096) remote: Compressing
> objects: 66% (10624/16096) remote: Compressing objects: 67%
> (10785/16096) remote: Compressing objects: 68% (10946/16096)
> remote: Compressing objects: 69% (11107/16096) remote:
> Compressing objects: 70% (11268/16096) remote: Compressing
> objects: 71% (11429/16096) remote: Compressing objects: 72%
> (11590/16096) remote: Compressing objects: 73% (11751/16096)
> remote: Compressing objects: 74% (11912/16096) remote:
> Compressing objects: 75% (12072/16096) remote: Compressing
> objects: 76% (12233/16096) remote: Compressing objects: 77%
> (12394/16096) remote: Compressing objects: 78% (12555/16096)
> remote: Compressing objects: 79% (12716/16096) remote:
> Compressing objects: 80% (12877/16096) remote: Compressing
> objects: 81% (13038/16096) remote: Compressing objects: 82%
> (13199/16096) remote: Compressing objects: 83% (13360/16096)
> remote: Compressing objects: 84% (13521/16096) remote:
> Compressing objects: 85% (13682/16096) remote: Compressing
> objects: 86% (13843/16096) remote: Compressing objects: 87%
> (14004/16096) remote: Compressing objects: 88% (14165/16096)
> remote: Compressing objects: 89% (14326/16096) remote:
> Compressing objects: 90% (14487/16096) remote: Compressing
> objects: 91% (14648/16096) remote: Compressing objects: 92%
> (14809/16096) remote: Compressing objects: 93% (14970/16096)
> remote: Compressing objects: 94% (15131/16096) remote:
> Compressing objects: 95% (15292/16096) remote: Compressing
> objects: 96% (15453/16096) remote: Compressing objects: 97%
> (15614/16096) remote: Compressing objects: 98% (15775/16096)
> remote: Compressing objects: 99% (15936/16096) remote:
> Compressing objects: 100% (16096/16096) remote: Compressing
> objects: 100% (16096/16096), done.
> Receiving objects: 0% (1/62166) Receiving objects: 1% (622/62166)
> Receiving objects: 2% (1244/62166) Receiving objects: 3%
> (1865/62166) Receiving objects: 4% (2487/62166) Receiving objects:
> 5% (3109/62166) Receiving objects: 6% (3730/62166) Receiving
> objects: 7% (4352/62166) Receiving objects: 8% (4974/62166)
> Receiving objects: 9% (5595/62166) Receiving objects: 10%
> (6217/62166) Receiving objects: 11% (6839/62166) Receiving objects:
> 12% (7460/62166) Receiving objects: 13% (8082/62166) Receiving
> objects: 14% (8704/62166) Receiving objects: 15% (9325/62166)
> Receiving objects: 16% (9947/62166) Receiving objects: 17%
> (10569/62166) Receiving objects: 18% (11190/62166) Receiving objects:
> 19% (11812/62166) Receiving objects: 20% (12434/62166) Receiving
> objects: 21% (13055/62166) Receiving objects: 22% (13677/62166)
> fatal: write error: No space left on device
> fatal: index-pack failed
>
> at
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
> at
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1721)
> at
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:72)
> at
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:405)
> at
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:614)
> at
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
> at
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
> at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> at hudson.remoting.Request$2.run(Request.java:369)
> at
> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call
> to H21
> at
> hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> at
> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
> at hudson.remoting.Channel.call(Channel.java:955)
> at
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
> at sun.reflect.GeneratedMethodAccessor918.invoke(Unknown
> Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
> at com.sun.proxy.$Proxy142.execute(Unknown Source)
> at
> hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1146)
> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> at hudson.scm.SCM.checkout(SCM.java:504)
> at
> hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> at
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> at
> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> at
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> at hudson.model.Run.execute(Run.java:1794)
> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> at
> hudson.model.ResourceController.execute(ResourceController.java:97)
> at hudson.model.Executor.run(Executor.java:429)
> ERROR: Error cloning remote repo 'origin'
> Retrying after 10 seconds
>> git rev-parse --is-inside-work-tree # timeout=10
> Fetching changes from the remote Git repository
>> git config remote.origin.url
> https://git-wip-us.apache.org/repos/asf/guacamole-client.git # timeout=10
> Fetching upstream changes from
> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>> git --version # timeout=10
>> git fetch --tags --progress
> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> +refs/heads/*:refs/remotes/origin/*
> ERROR: Error fetching remote repo 'origin'
> hudson.plugins.git.GitException: Failed to fetch from
> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
> at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> at hudson.scm.SCM.checkout(SCM.java:504)
> at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> at
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> at
> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> at
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> at hudson.model.Run.execute(Run.java:1794)
> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> at
> hudson.model.ResourceController.execute(ResourceController.java:97)
> at hudson.model.Executor.run(Executor.java:429)
> Caused by: hudson.plugins.git.GitException: Command "git fetch --tags
> --progress https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> +refs/heads/*:refs/remotes/origin/*" returned status code 128:
> stdout:
> stderr: remote: Counting objects: 62166, done.
> remote: Compressing objects: 0% (1/16096) remote: Compressing
> objects: 1% (161/16096) remote: Compressing objects: 2%
> (322/16096) remote: Compressing objects: 3% (483/16096)
> remote: Compressing objects: 4% (644/16096) remote:
> Compressing objects: 5% (805/16096) remote: Compressing
> objects: 6% (966/16096) remote: Compressing objects: 7%
> (1127/16096) remote: Compressing objects: 8% (1288/16096)
> remote: Compressing objects: 9% (1449/16096) remote:
> Compressing objects: 10% (1610/16096) remote: Compressing
> objects: 11% (1771/16096) remote: Compressing objects: 12%
> (1932/16096) remote: Compressing objects: 13% (2093/16096)
> remote: Compressing objects: 14% (2254/16096) remote:
> Compressing objects: 15% (2415/16096) remote: Compressing
> objects: 16% (2576/16096) remote: Compressing objects: 17%
> (2737/16096) remote: Compressing objects: 18% (2898/16096)
> remote: Compressing objects: 19% (3059/16096) remote:
> Compressing objects: 20% (3220/16096) remote: Compressing
> objects: 21% (3381/16096) remote: Compressing objects: 22%
> (3542/16096) remote: Compressing objects: 23% (3703/16096)
> remote: Compressing objects: 24% (3864/16096) remote:
> Compressing objects: 25% (4024/16096) remote: Compressing
> objects: 26% (4185/16096) remote: Compressing objects: 27%
> (4346/16096) remote: Compressing objects: 28% (4507/16096)
> remote: Compressing objects: 29% (4668/16096) remote:
> Compressing objects: 30% (4829/16096) remote: Compressing
> objects: 31% (4990/16096) remote: Compressing objects: 32%
> (5151/16096) remote: Compressing objects: 33% (5312/16096)
> remote: Compressing objects: 34% (5473/16096) remote:
> Compressing objects: 35% (5634/16096) remote: Compressing
> objects: 36% (5795/16096) remote: Compressing objects: 37%
> (5956/16096) remote: Compressing objects: 38% (6117/16096)
> remote: Compressing objects: 39% (6278/16096) remote:
> Compressing objects: 40% (6439/16096) remote: Compressing
> objects: 41% (6600/16096) remote: Compressing objects: 42%
> (6761/16096) remote: Compressing objects: 43% (6922/16096)
> remote: Compressing objects: 44% (7083/16096) remote:
> Compressing objects: 45% (7244/16096) remote: Compressing
> objects: 46% (7405/16096) remote: Compressing objects: 47%
> (7566/16096) remote: Compressing objects: 48% (7727/16096)
> remote: Compressing objects: 49% (7888/16096) remote:
> Compressing objects: 50% (8048/16096) remote: Compressing
> objects: 51% (8209/16096) remote: Compressing objects: 52%
> (8370/16096) remote: Compressing objects: 53% (8531/16096)
> remote: Compressing objects: 54% (8692/16096) remote:
> Compressing objects: 55% (8853/16096) remote: Compressing
> objects: 56% (9014/16096) remote: Compressing objects: 57%
> (9175/16096) remote: Compressing objects: 58% (9336/16096)
> remote: Compressing objects: 59% (9497/16096) remote:
> Compressing objects: 60% (9658/16096) remote: Compressing
> objects: 61% (9819/16096) remote: Compressing objects: 62%
> (9980/16096) remote: Compressing objects: 63% (10141/16096)
> remote: Compressing objects: 64% (10302/16096) remote:
> Compressing objects: 65% (10463/16096) remote: Compressing
> objects: 66% (10624/16096) remote: Compressing objects: 67%
> (10785/16096) remote: Compressing objects: 68% (10946/16096)
> remote: Compressing objects: 69% (11107/16096) remote:
> Compressing objects: 70% (11268/16096) remote: Compressing
> objects: 71% (11429/16096) remote: Compressing objects: 72%
> (11590/16096) remote: Compressing objects: 73% (11751/16096)
> remote: Compressing objects: 74% (11912/16096) remote:
> Compressing objects: 75% (12072/16096) remote: Compressing
> objects: 76% (12233/16096) remote: Compressing objects: 77%
> (12394/16096) remote: Compressing objects: 78% (12555/16096)
> remote: Compressing objects: 79% (12716/16096) remote:
> Compressing objects: 80% (12877/16096) remote: Compressing
> objects: 81% (13038/16096) remote: Compressing objects: 82%
> (13199/16096) remote: Compressing objects: 83% (13360/16096)
> remote: Compressing objects: 84% (13521/16096) remote:
> Compressing objects: 85% (13682/16096) remote: Compressing
> objects: 86% (13843/16096) remote: Compressing objects: 87%
> (14004/16096) remote: Compressing objects: 88% (14165/16096)
> remote: Compressing objects: 89% (14326/16096) remote:
> Compressing objects: 90% (14487/16096) remote: Compressing
> objects: 91% (14648/16096) remote: Compressing objects: 92%
> (14809/16096) remote: Compressing objects: 93% (14970/16096)
> remote: Compressing objects: 94% (15131/16096) remote:
> Compressing objects: 95% (15292/16096) remote: Compressing
> objects: 96% (15453/16096) remote: Compressing objects: 97%
> (15614/16096) remote: Compressing objects: 98% (15775/16096)
> remote: Compressing objects: 99% (15936/16096) remote:
> Compressing objects: 100% (16096/16096) remote: Compressing
> objects: 100% (16096/16096), done.
> Receiving objects: 0% (1/62166) fatal: write error: No space left on
> device
> fatal: index-pack failed
>
> at
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
> at
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1721)
> at
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:72)
> at
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:405)
> at
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
> at
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
> at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> at hudson.remoting.Request$2.run(Request.java:369)
> at
> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call
> to H21
> at
> hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> at
> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
> at hudson.remoting.Channel.call(Channel.java:955)
> at
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
> at sun.reflect.GeneratedMethodAccessor918.invoke(Unknown
> Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
> at com.sun.proxy.$Proxy119.execute(Unknown Source)
> at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:886)
> at
> hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> at hudson.scm.SCM.checkout(SCM.java:504)
> at
> hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> at
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> at
> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> at
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> at hudson.model.Run.execute(Run.java:1794)
> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> at
> hudson.model.ResourceController.execute(ResourceController.java:97)
> at hudson.model.Executor.run(Executor.java:429)
> ERROR: Error fetching remote repo 'origin'
> Retrying after 10 seconds
>> git rev-parse --is-inside-work-tree # timeout=10
> Fetching changes from the remote Git repository
>> git config remote.origin.url
> https://git-wip-us.apache.org/repos/asf/guacamole-client.git # timeout=10
> ERROR: Error fetching remote repo 'origin'
> hudson.plugins.git.GitException: Failed to fetch from
> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
> at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> at hudson.scm.SCM.checkout(SCM.java:504)
> at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> at
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> at
> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> at
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> at hudson.model.Run.execute(Run.java:1794)
> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> at
> hudson.model.ResourceController.execute(ResourceController.java:97)
> at hudson.model.Executor.run(Executor.java:429)
> Caused by: hudson.plugins.git.GitException: Command "git config
> remote.origin.url
> https://git-wip-us.apache.org/repos/asf/guacamole-client.git" returned
> status code 4:
> stdout:
> stderr: error: failed to write new configuration file <
> https://builds.apache.org/job/guacamole-client-coverity/ws/guacamole-client/.git/config.lock
>>
>
> at
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
> at
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1970)
> at
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1966)
> at
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1597)
> at
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1609)
> at
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.setRemoteUrl(CliGitAPIImpl.java:1243)
> at hudson.plugins.git.GitAPI.setRemoteUrl(GitAPI.java:160)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:929)
> at
> hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:903)
> at
> hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:855)
> at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> at hudson.remoting.Request$2.run(Request.java:369)
> at
> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call
> to H21
> at
> hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> at
> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
> at hudson.remoting.Channel.call(Channel.java:955)
> at
> hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:283)
> at com.sun.proxy.$Proxy118.setRemoteUrl(Unknown Source)
> at
> org.jenkinsci.plugins.gitclient.RemoteGitImpl.setRemoteUrl(RemoteGitImpl.java:295)
> at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:876)
> at
> hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> at hudson.scm.SCM.checkout(SCM.java:504)
> at
> hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> at
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> at
> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> at
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> at hudson.model.Run.execute(Run.java:1794)
> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> at
> hudson.model.ResourceController.execute(ResourceController.java:97)
> at hudson.model.Executor.run(Executor.java:429)
> ERROR: Error fetching remote repo 'origin'
Re: No space on H21 (Was: Fwd: Build failed in Jenkins:
guacamole-client-coverity #73)
Posted by Mike Jumper <mj...@apache.org>.
Checking the Jenkins instance I have deployed locally, there's a
"Preventive Node Monitoring" section with a "Free Space" option
under:
Manage Jenkins -> Manage Nodes -> Configure
The help text for that reads:
"This option configures the amount of minimum amount of free disk
space desired for an agent's proper operation, such as "1.5GB",
"100KB", etc. If an agent is found to have less free disk space than
this amount, it will be marked offline."
- Mike
On Sun, Sep 30, 2018 at 9:23 PM, Greg Stein <gs...@gmail.com> wrote:
>
> Oh, and I'm not trying to avoid your question about auto-removal. I don't
> have that answer, and will leave it for others.
>
> On Sun, Sep 30, 2018 at 11:22 PM Greg Stein <gs...@gmail.com> wrote:
>
> > We're refining our monitoring and some cron jobs to try and keep the disk
> > space pruned. But when space balloons due to poor cleanup, there is only so
> > much we can do.
> >
> > The short answer is that the multitude of groups at the Foundation using
> > these shared resources need to be cognizant of that fact, and cooperate on
> > ways to keep them available for all. We've seen disk space problems, too
> > many jobs, too-long jobs, etc.
> >
> > Infra will keep working and improving our management of the build nodes.
> > In the meantime, please keep reporting problems, and we'll do our best to
> > keep the system up for y'all.
> >
> > Cheers,
> > Greg
> > InfraAdmin, ASF
> >
> >
> > On Sun, Sep 30, 2018 at 10:54 PM Mike Jumper <mj...@apache.org> wrote:
> >
> >> Is there any way Jenkins could be configured to take nodes offline
> >> automatically when free space is below a certain threshold? Granted
> >> space-hungry builds and proper cleanup remain a problem, but at least
> >> there
> >> wouldn't be a random chance of jobs failing due to being assigned to a
> >> full
> >> node.
> >>
> >> - Mike
> >>
> >>
> >> On Sun, Sep 30, 2018 at 3:57 PM, Chris Lambertus <cm...@apache.org> wrote:
> >>
> >> > The main culprit on H31 appears to be pulsar ws-cleanup jobs consuming
> >> > almost 40% of the disk. (The problem on H21 was something
> >> docker-related.)
> >> >
> >> >
> >> > This is the second project I have found (Brooklyn was the first) which
> >> is
> >> > leaving ws-cleanup directories all over the workspace and filling up the
> >> > disks. Each one of these was 500+ MB. Pulsar, please address ASAP.
> >> >
> >> >
> >> >
> >> >
> >> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:19
> >> > pulsar_precommit_cpp@2_ws-cleanup_1535737112280
> >> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:18
> >> > pulsar_precommit_cpp@2_ws-cleanup_1536174576053
> >> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 06:04
> >> > pulsar_precommit_cpp@2_ws-cleanup_1537292379178
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 22:45
> >> > pulsar_precommit_cpp@2_ws-cleanup_1537397120862
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> >> > pulsar_precommit_cpp@2_ws-cleanup_1537403378279
> >> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 21 23:31
> >> > pulsar_precommit_cpp@2_ws-cleanup_1537572701829
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 26 16:04
> >> > pulsar_precommit_cpp@2_ws-cleanup_1537977879341
> >> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 22:23
> >> > pulsar_precommit_cpp@2_ws-cleanup_1538087035027
> >> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:14
> >> > pulsar_precommit_cpp_ws-cleanup_1535699830230
> >> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:16
> >> > pulsar_precommit_cpp_ws-cleanup_1535735511423
> >> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:13
> >> > pulsar_precommit_cpp_ws-cleanup_1535786064397
> >> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:18
> >> > pulsar_precommit_cpp_ws-cleanup_1536096963586
> >> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 21 06:47
> >> > pulsar_precommit_cpp_ws-cleanup_1536114938910
> >> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 21 06:45
> >> > pulsar_precommit_cpp_ws-cleanup_1536173165909
> >> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:38
> >> > pulsar_precommit_cpp_ws-cleanup_1536175584418
> >> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:50
> >> > pulsar_precommit_cpp_ws-cleanup_1536180707506
> >> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:38
> >> > pulsar_precommit_cpp_ws-cleanup_1536182927940
> >> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:51
> >> > pulsar_precommit_cpp_ws-cleanup_1536185193716
> >> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:50
> >> > pulsar_precommit_cpp_ws-cleanup_1536273709029
> >> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 05:59
> >> > pulsar_precommit_cpp_ws-cleanup_1536316964794
> >> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:06
> >> > pulsar_precommit_cpp_ws-cleanup_1536675650893
> >> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 28 06:08
> >> > pulsar_precommit_cpp_ws-cleanup_1536879616962
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 30 19:53
> >> > pulsar_precommit_cpp_ws-cleanup_1536891056968
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 30 19:47
> >> > pulsar_precommit_cpp_ws-cleanup_1536966466683
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 18 00:43
> >> > pulsar_precommit_cpp_ws-cleanup_1537216387387
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 00:45
> >> > pulsar_precommit_cpp_ws-cleanup_1537292309413
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 00:45
> >> > pulsar_precommit_cpp_ws-cleanup_1537308526251
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> >> > pulsar_precommit_cpp_ws-cleanup_1537331780420
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> >> > pulsar_precommit_cpp_ws-cleanup_1537379886715
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
> >> > pulsar_precommit_cpp_ws-cleanup_1537386197893
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
> >> > pulsar_precommit_cpp_ws-cleanup_1537397073464
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
> >> > pulsar_precommit_cpp_ws-cleanup_1537403247575
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:43
> >> > pulsar_precommit_cpp_ws-cleanup_1537413954888
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
> >> > pulsar_precommit_cpp_ws-cleanup_1537417322151
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
> >> > pulsar_precommit_cpp_ws-cleanup_1537421381129
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:44
> >> > pulsar_precommit_cpp_ws-cleanup_1537469348852
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
> >> > pulsar_precommit_cpp_ws-cleanup_1537489399468
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 22 00:44
> >> > pulsar_precommit_cpp_ws-cleanup_1537566528660
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 23:12
> >> > pulsar_precommit_cpp_ws-cleanup_1537571549978
> >> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 22 00:44
> >> > pulsar_precommit_cpp_ws-cleanup_1537573605258
> >> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 00:43
> >> > pulsar_precommit_cpp_ws-cleanup_1537855939828
> >> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 11:18
> >> > pulsar_precommit_cpp_ws-cleanup_1537874305575
> >> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 12:41
> >> > pulsar_precommit_cpp_ws-cleanup_1537879296622
> >> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 01:39
> >> > pulsar_precommit_cpp_ws-cleanup_1537925955967
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 26 13:17
> >> > pulsar_precommit_cpp_ws-cleanup_1537967826838
> >> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 15:55
> >> > pulsar_precommit_cpp_ws-cleanup_1537977326736
> >> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 17:34
> >> > pulsar_precommit_cpp_ws-cleanup_1537983268177
> >> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 22:04
> >> > pulsar_precommit_cpp_ws-cleanup_1538085846793
> >> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 28 00:20
> >> > pulsar_precommit_cpp_ws-cleanup_1538094024821
> >> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 28 17:42
> >> > pulsar_precommit_cpp_ws-cleanup_1538156543343
> >> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:02
> >> > pulsar_precommit_integrationtests_ws-cleanup_1536298125841
> >> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:00
> >> > pulsar_precommit_integrationtests_ws-cleanup_1536968128660
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 18 00:43
> >> > pulsar_precommit_integrationtests_ws-cleanup_1537218541598
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> >> > pulsar_precommit_integrationtests_ws-cleanup_1537331703324
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 22 00:43
> >> > pulsar_precommit_integrationtests_ws-cleanup_1537553748859
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 00:43
> >> > pulsar_precommit_integrationtests_ws-cleanup_1537730402937
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 07:48
> >> > pulsar_precommit_integrationtests_ws-cleanup_1537775336586
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 23:30
> >> > pulsar_precommit_integrationtests_ws-cleanup_1537831851711
> >> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 23:08
> >> > pulsar_precommit_integrationtests_ws-cleanup_1537916928485
> >> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 15:21
> >> > pulsar_precommit_integrationtests_ws-cleanup_1538061702243
> >> >
> >> >
> >> > -Chris
> >> > ASF Infra
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > > On Sep 30, 2018, at 3:12 PM, Chris Thistlethwaite <
> >> > cthistlego@outlook.com> wrote:
> >> > >
> >> > > I've been working on H21 and H31. They both had a some docker issues
> >> > where the process was locked and I couldn't do anything with a few
> >> > containers that were over 8 days old.
> >> > >
> >> > > Thought we might be good this morning so I threw them back into
> >> > rotation. Guess not, so I'll take them back out.
> >> > >
> >> > > -Chris T.
> >> > > ________________________________
> >> > > From: Mike Jumper <mj...@apache.org>
> >> > > Sent: Sunday, September 30, 2018 6:03:51 PM
> >> > > To: builds@apache.org
> >> > > Subject: No space on H21 (Was: Fwd: Build failed in Jenkins:
> >> > guacamole-client-coverity #73)
> >> > >
> >> > > FYI - we're currently seeing disk space issues on H21.
> >> > >
> >> > > - Mike
> >> > >
> >> > >
> >> > > ---------- Forwarded message ---------
> >> > > From: Apache Jenkins Server <je...@builds.apache.org>
> >> > > Date: Sun, Sep 30, 2018, 14:37
> >> > > Subject: Build failed in Jenkins: guacamole-client-coverity #73
> >> > > To: <de...@guacamole.apache.org>
> >> > >
> >> > >
> >> > > See <
> >> > > https://builds.apache.org/job/guacamole-client-coverity/73/d
> >> > isplay/redirect>
> >> > >
> >> > > ------------------------------------------
> >> > > Started by user mjumper
> >> > > [EnvInject] - Loading node environment variables.
> >> > > Building remotely on H21 (ubuntu xenial) in workspace <
> >> > > https://builds.apache.org/job/guacamole-client-coverity/ws/>
> >> > > [WS-CLEANUP] Deleting project workspace...
> >> > > Cloning the remote Git repository
> >> > > Cloning repository
> >> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> >> > >> git init <
> >> > > https://builds.apache.org/job/guacamole-client-coverity/ws/g
> >> > uacamole-client>
> >> > > # timeout=10
> >> > > Fetching upstream changes from
> >> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> >> > >> git --version # timeout=10
> >> > >> git fetch --tags --progress
> >> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> >> > > +refs/heads/*:refs/remotes/origin/*
> >> > > ERROR: Error cloning remote repo 'origin'
> >> > > hudson.plugins.git.GitException: Command "git fetch --tags --progress
> >> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> >> > > +refs/heads/*:refs/remotes/origin/*" returned status code 128:
> >> > > stdout:
> >> > > stderr: remote: Counting objects: 62166, done.
> >> > > remote: Compressing objects: 0% (1/16096) remote:
> >> Compressing
> >> > > objects: 1% (161/16096) remote: Compressing objects: 2%
> >> > > (322/16096) remote: Compressing objects: 3% (483/16096)
> >> > > remote: Compressing objects: 4% (644/16096) remote:
> >> > > Compressing objects: 5% (805/16096) remote: Compressing
> >> > > objects: 6% (966/16096) remote: Compressing objects: 7%
> >> > > (1127/16096) remote: Compressing objects: 8% (1288/16096)
> >> > > remote: Compressing objects: 9% (1449/16096) remote:
> >> > > Compressing objects: 10% (1610/16096) remote: Compressing
> >> > > objects: 11% (1771/16096) remote: Compressing objects: 12%
> >> > > (1932/16096) remote: Compressing objects: 13% (2093/16096)
> >> > > remote: Compressing objects: 14% (2254/16096) remote:
> >> > > Compressing objects: 15% (2415/16096) remote: Compressing
> >> > > objects: 16% (2576/16096) remote: Compressing objects: 17%
> >> > > (2737/16096) remote: Compressing objects: 18% (2898/16096)
> >> > > remote: Compressing objects: 19% (3059/16096) remote:
> >> > > Compressing objects: 20% (3220/16096) remote: Compressing
> >> > > objects: 21% (3381/16096) remote: Compressing objects: 22%
> >> > > (3542/16096) remote: Compressing objects: 23% (3703/16096)
> >> > > remote: Compressing objects: 24% (3864/16096) remote:
> >> > > Compressing objects: 25% (4024/16096) remote: Compressing
> >> > > objects: 26% (4185/16096) remote: Compressing objects: 27%
> >> > > (4346/16096) remote: Compressing objects: 28% (4507/16096)
> >> > > remote: Compressing objects: 29% (4668/16096) remote:
> >> > > Compressing objects: 30% (4829/16096) remote: Compressing
> >> > > objects: 31% (4990/16096) remote: Compressing objects: 32%
> >> > > (5151/16096) remote: Compressing objects: 33% (5312/16096)
> >> > > remote: Compressing objects: 34% (5473/16096) remote:
> >> > > Compressing objects: 35% (5634/16096) remote: Compressing
> >> > > objects: 36% (5795/16096) remote: Compressing objects: 37%
> >> > > (5956/16096) remote: Compressing objects: 38% (6117/16096)
> >> > > remote: Compressing objects: 39% (6278/16096) remote:
> >> > > Compressing objects: 40% (6439/16096) remote: Compressing
> >> > > objects: 41% (6600/16096) remote: Compressing objects: 42%
> >> > > (6761/16096) remote: Compressing objects: 43% (6922/16096)
> >> > > remote: Compressing objects: 44% (7083/16096) remote:
> >> > > Compressing objects: 45% (7244/16096) remote: Compressing
> >> > > objects: 46% (7405/16096) remote: Compressing objects: 47%
> >> > > (7566/16096) remote: Compressing objects: 48% (7727/16096)
> >> > > remote: Compressing objects: 49% (7888/16096) remote:
> >> > > Compressing objects: 50% (8048/16096) remote: Compressing
> >> > > objects: 51% (8209/16096) remote: Compressing objects: 52%
> >> > > (8370/16096) remote: Compressing objects: 53% (8531/16096)
> >> > > remote: Compressing objects: 54% (8692/16096) remote:
> >> > > Compressing objects: 55% (8853/16096) remote: Compressing
> >> > > objects: 56% (9014/16096) remote: Compressing objects: 57%
> >> > > (9175/16096) remote: Compressing objects: 58% (9336/16096)
> >> > > remote: Compressing objects: 59% (9497/16096) remote:
> >> > > Compressing objects: 60% (9658/16096) remote: Compressing
> >> > > objects: 61% (9819/16096) remote: Compressing objects: 62%
> >> > > (9980/16096) remote: Compressing objects: 63% (10141/16096)
> >> > > remote: Compressing objects: 64% (10302/16096) remote:
> >> > > Compressing objects: 65% (10463/16096) remote: Compressing
> >> > > objects: 66% (10624/16096) remote: Compressing objects:
> >> 67%
> >> > > (10785/16096) remote: Compressing objects: 68%
> >> (10946/16096)
> >> > > remote: Compressing objects: 69% (11107/16096)
> >> remote:
> >> > > Compressing objects: 70% (11268/16096) remote: Compressing
> >> > > objects: 71% (11429/16096) remote: Compressing objects:
> >> 72%
> >> > > (11590/16096) remote: Compressing objects: 73%
> >> (11751/16096)
> >> > > remote: Compressing objects: 74% (11912/16096)
> >> remote:
> >> > > Compressing objects: 75% (12072/16096) remote: Compressing
> >> > > objects: 76% (12233/16096) remote: Compressing objects:
> >> 77%
> >> > > (12394/16096) remote: Compressing objects: 78%
> >> (12555/16096)
> >> > > remote: Compressing objects: 79% (12716/16096)
> >> remote:
> >> > > Compressing objects: 80% (12877/16096) remote: Compressing
> >> > > objects: 81% (13038/16096) remote: Compressing objects:
> >> 82%
> >> > > (13199/16096) remote: Compressing objects: 83%
> >> (13360/16096)
> >> > > remote: Compressing objects: 84% (13521/16096)
> >> remote:
> >> > > Compressing objects: 85% (13682/16096) remote: Compressing
> >> > > objects: 86% (13843/16096) remote: Compressing objects:
> >> 87%
> >> > > (14004/16096) remote: Compressing objects: 88%
> >> (14165/16096)
> >> > > remote: Compressing objects: 89% (14326/16096)
> >> remote:
> >> > > Compressing objects: 90% (14487/16096) remote: Compressing
> >> > > objects: 91% (14648/16096) remote: Compressing objects:
> >> 92%
> >> > > (14809/16096) remote: Compressing objects: 93%
> >> (14970/16096)
> >> > > remote: Compressing objects: 94% (15131/16096)
> >> remote:
> >> > > Compressing objects: 95% (15292/16096) remote: Compressing
> >> > > objects: 96% (15453/16096) remote: Compressing objects:
> >> 97%
> >> > > (15614/16096) remote: Compressing objects: 98%
> >> (15775/16096)
> >> > > remote: Compressing objects: 99% (15936/16096)
> >> remote:
> >> > > Compressing objects: 100% (16096/16096) remote: Compressing
> >> > > objects: 100% (16096/16096), done.
> >> > > Receiving objects: 0% (1/62166) Receiving objects: 1%
> >> (622/62166)
> >> > > Receiving objects: 2% (1244/62166) Receiving objects: 3%
> >> > > (1865/62166) Receiving objects: 4% (2487/62166) Receiving
> >> objects:
> >> > > 5% (3109/62166) Receiving objects: 6% (3730/62166) Receiving
> >> > > objects: 7% (4352/62166) Receiving objects: 8% (4974/62166)
> >> > > Receiving objects: 9% (5595/62166) Receiving objects: 10%
> >> > > (6217/62166) Receiving objects: 11% (6839/62166) Receiving
> >> objects:
> >> > > 12% (7460/62166) Receiving objects: 13% (8082/62166) Receiving
> >> > > objects: 14% (8704/62166) Receiving objects: 15% (9325/62166)
> >> > > Receiving objects: 16% (9947/62166) Receiving objects: 17%
> >> > > (10569/62166) Receiving objects: 18% (11190/62166) Receiving
> >> > objects:
> >> > > 19% (11812/62166) Receiving objects: 20% (12434/62166) Receiving
> >> > > objects: 21% (13055/62166) Receiving objects: 22% (13677/62166)
> >> > > fatal: write error: No space left on device
> >> > > fatal: index-pack failed
> >> > >
> >> > > at
> >> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandI
> >> > n(CliGitAPIImpl.java:2002)
> >> > > at
> >> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandW
> >> > ithCredentials(CliGitAPIImpl.java:1721)
> >> > > at
> >> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(
> >> > CliGitAPIImpl.java:72)
> >> > > at
> >> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(
> >> > CliGitAPIImpl.java:405)
> >> > > at
> >> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(
> >> > CliGitAPIImpl.java:614)
> >> > > at
> >> > > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
> >> > ionHandler$1.call(RemoteGitImpl.java:153)
> >> > > at
> >> > > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
> >> > ionHandler$1.call(RemoteGitImpl.java:146)
> >> > > at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> >> > > at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> >> > > at hudson.remoting.Request$2.run(Request.java:369)
> >> > > at
> >> > > hudson.remoting.InterceptingExecutorService$1.call(Intercept
> >> > ingExecutorService.java:72)
> >> > > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> > > at
> >> > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> >> > Executor.java:1142)
> >> > > at
> >> > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> >> > lExecutor.java:617)
> >> > > at java.lang.Thread.run(Thread.java:748)
> >> > > Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
> >> > call
> >> > > to H21
> >> > > at
> >> > > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> >> > > at
> >> > > hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserR
> >> > equest.java:357)
> >> > > at hudson.remoting.Channel.call(Channel.java:955)
> >> > > at
> >> > > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
> >> > ionHandler.execute(RemoteGitImpl.java:146)
> >> > > at
> >> sun.reflect.GeneratedMethodAccessor918.invoke(Unknown
> >> > > Source)
> >> > > at
> >> > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
> >> > thodAccessorImpl.java:43)
> >> > > at java.lang.reflect.Method.invoke(Method.java:498)
> >> > > at
> >> > > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
> >> > ionHandler.invoke(RemoteGitImpl.java:132)
> >> > > at com.sun.proxy.$Proxy142.execute(Unknown Source)
> >> > > at
> >> > > hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1146)
> >> > > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> >> > > at hudson.scm.SCM.checkout(SCM.java:504)
> >> > > at
> >> > > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> >> > > at
> >> > > hudson.model.AbstractBuild$AbstractBuildExecution.defaultChe
> >> > ckout(AbstractBuild.java:574)
> >> > > at
> >> > > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> >> > > at
> >> > > hudson.model.AbstractBuild$AbstractBuildExecution.run(Abstra
> >> > ctBuild.java:499)
> >> > > at hudson.model.Run.execute(Run.java:1794)
> >> > > at hudson.model.FreeStyleBuild.ru
> >> > n(FreeStyleBuild.java:43)
> >> > > at
> >> > > hudson.model.ResourceController.execute(ResourceController.java:97)
> >> > > at hudson.model.Executor.run(Executor.java:429)
> >> > > ERROR: Error cloning remote repo 'origin'
> >> > > Retrying after 10 seconds
> >> > >> git rev-parse --is-inside-work-tree # timeout=10
> >> > > Fetching changes from the remote Git repository
> >> > >> git config remote.origin.url
> >> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git #
> >> > timeout=10
> >> > > Fetching upstream changes from
> >> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> >> > >> git --version # timeout=10
> >> > >> git fetch --tags --progress
> >> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> >> > > +refs/heads/*:refs/remotes/origin/*
> >> > > ERROR: Error fetching remote repo 'origin'
> >> > > hudson.plugins.git.GitException: Failed to fetch from
> >> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> >> > > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
> >> > > at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> >> > > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> >> > > at hudson.scm.SCM.checkout(SCM.java:504)
> >> > > at hudson.model.AbstractProject.checkout(AbstractProject.java:1
> >> > 208)
> >> > > at
> >> > > hudson.model.AbstractBuild$AbstractBuildExecution.defaultChe
> >> > ckout(AbstractBuild.java:574)
> >> > > at
> >> > > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> >> > > at
> >> > > hudson.model.AbstractBuild$AbstractBuildExecution.run(Abstra
> >> > ctBuild.java:499)
> >> > > at hudson.model.Run.execute(Run.java:1794)
> >> > > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> >> > > at
> >> > > hudson.model.ResourceController.execute(ResourceController.java:97)
> >> > > at hudson.model.Executor.run(Executor.java:429)
> >> > > Caused by: hudson.plugins.git.GitException: Command "git fetch --tags
> >> > > --progress
> >> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> >> > > +refs/heads/*:refs/remotes/origin/*" returned status code 128:
> >> > > stdout:
> >> > > stderr: remote: Counting objects: 62166, done.
> >> > > remote: Compressing objects: 0% (1/16096) remote:
> >> Compressing
> >> > > objects: 1% (161/16096) remote: Compressing objects: 2%
> >> > > (322/16096) remote: Compressing objects: 3% (483/16096)
> >> > > remote: Compressing objects: 4% (644/16096) remote:
> >> > > Compressing objects: 5% (805/16096) remote: Compressing
> >> > > objects: 6% (966/16096) remote: Compressing objects: 7%
> >> > > (1127/16096) remote: Compressing objects: 8% (1288/16096)
> >> > > remote: Compressing objects: 9% (1449/16096) remote:
> >> > > Compressing objects: 10% (1610/16096) remote: Compressing
> >> > > objects: 11% (1771/16096) remote: Compressing objects: 12%
> >> > > (1932/16096) remote: Compressing objects: 13% (2093/16096)
> >> > > remote: Compressing objects: 14% (2254/16096) remote:
> >> > > Compressing objects: 15% (2415/16096) remote: Compressing
> >> > > objects: 16% (2576/16096) remote: Compressing objects: 17%
> >> > > (2737/16096) remote: Compressing objects: 18% (2898/16096)
> >> > > remote: Compressing objects: 19% (3059/16096) remote:
> >> > > Compressing objects: 20% (3220/16096) remote: Compressing
> >> > > objects: 21% (3381/16096) remote: Compressing objects: 22%
> >> > > (3542/16096) remote: Compressing objects: 23% (3703/16096)
> >> > > remote: Compressing objects: 24% (3864/16096) remote:
> >> > > Compressing objects: 25% (4024/16096) remote: Compressing
> >> > > objects: 26% (4185/16096) remote: Compressing objects: 27%
> >> > > (4346/16096) remote: Compressing objects: 28% (4507/16096)
> >> > > remote: Compressing objects: 29% (4668/16096) remote:
> >> > > Compressing objects: 30% (4829/16096) remote: Compressing
> >> > > objects: 31% (4990/16096) remote: Compressing objects: 32%
> >> > > (5151/16096) remote: Compressing objects: 33% (5312/16096)
> >> > > remote: Compressing objects: 34% (5473/16096) remote:
> >> > > Compressing objects: 35% (5634/16096) remote: Compressing
> >> > > objects: 36% (5795/16096) remote: Compressing objects: 37%
> >> > > (5956/16096) remote: Compressing objects: 38% (6117/16096)
> >> > > remote: Compressing objects: 39% (6278/16096) remote:
> >> > > Compressing objects: 40% (6439/16096) remote: Compressing
> >> > > objects: 41% (6600/16096) remote: Compressing objects: 42%
> >> > > (6761/16096) remote: Compressing objects: 43% (6922/16096)
> >> > > remote: Compressing objects: 44% (7083/16096) remote:
> >> > > Compressing objects: 45% (7244/16096) remote: Compressing
> >> > > objects: 46% (7405/16096) remote: Compressing objects: 47%
> >> > > (7566/16096) remote: Compressing objects: 48% (7727/16096)
> >> > > remote: Compressing objects: 49% (7888/16096) remote:
> >> > > Compressing objects: 50% (8048/16096) remote: Compressing
> >> > > objects: 51% (8209/16096) remote: Compressing objects: 52%
> >> > > (8370/16096) remote: Compressing objects: 53% (8531/16096)
> >> > > remote: Compressing objects: 54% (8692/16096) remote:
> >> > > Compressing objects: 55% (8853/16096) remote: Compressing
> >> > > objects: 56% (9014/16096) remote: Compressing objects: 57%
> >> > > (9175/16096) remote: Compressing objects: 58% (9336/16096)
> >> > > remote: Compressing objects: 59% (9497/16096) remote:
> >> > > Compressing objects: 60% (9658/16096) remote: Compressing
> >> > > objects: 61% (9819/16096) remote: Compressing objects: 62%
> >> > > (9980/16096) remote: Compressing objects: 63% (10141/16096)
> >> > > remote: Compressing objects: 64% (10302/16096) remote:
> >> > > Compressing objects: 65% (10463/16096) remote: Compressing
> >> > > objects: 66% (10624/16096) remote: Compressing objects:
> >> 67%
> >> > > (10785/16096) remote: Compressing objects: 68%
> >> (10946/16096)
> >> > > remote: Compressing objects: 69% (11107/16096)
> >> remote:
> >> > > Compressing objects: 70% (11268/16096) remote: Compressing
> >> > > objects: 71% (11429/16096) remote: Compressing objects:
> >> 72%
> >> > > (11590/16096) remote: Compressing objects: 73%
> >> (11751/16096)
> >> > > remote: Compressing objects: 74% (11912/16096)
> >> remote:
> >> > > Compressing objects: 75% (12072/16096) remote: Compressing
> >> > > objects: 76% (12233/16096) remote: Compressing objects:
> >> 77%
> >> > > (12394/16096) remote: Compressing objects: 78%
> >> (12555/16096)
> >> > > remote: Compressing objects: 79% (12716/16096)
> >> remote:
> >> > > Compressing objects: 80% (12877/16096) remote: Compressing
> >> > > objects: 81% (13038/16096) remote: Compressing objects:
> >> 82%
> >> > > (13199/16096) remote: Compressing objects: 83%
> >> (13360/16096)
> >> > > remote: Compressing objects: 84% (13521/16096)
> >> remote:
> >> > > Compressing objects: 85% (13682/16096) remote: Compressing
> >> > > objects: 86% (13843/16096) remote: Compressing objects:
> >> 87%
> >> > > (14004/16096) remote: Compressing objects: 88%
> >> (14165/16096)
> >> > > remote: Compressing objects: 89% (14326/16096)
> >> remote:
> >> > > Compressing objects: 90% (14487/16096) remote: Compressing
> >> > > objects: 91% (14648/16096) remote: Compressing objects:
> >> 92%
> >> > > (14809/16096) remote: Compressing objects: 93%
> >> (14970/16096)
> >> > > remote: Compressing objects: 94% (15131/16096)
> >> remote:
> >> > > Compressing objects: 95% (15292/16096) remote: Compressing
> >> > > objects: 96% (15453/16096) remote: Compressing objects:
> >> 97%
> >> > > (15614/16096) remote: Compressing objects: 98%
> >> (15775/16096)
> >> > > remote: Compressing objects: 99% (15936/16096)
> >> remote:
> >> > > Compressing objects: 100% (16096/16096) remote: Compressing
> >> > > objects: 100% (16096/16096), done.
> >> > > Receiving objects: 0% (1/62166) fatal: write error: No space left
> >> on
> >> > > device
> >> > > fatal: index-pack failed
> >> > >
> >> > > at
> >> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandI
> >> > n(CliGitAPIImpl.java:2002)
> >> > > at
> >> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandW
> >> > ithCredentials(CliGitAPIImpl.java:1721)
> >> > > at
> >> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(
> >> > CliGitAPIImpl.java:72)
> >> > > at
> >> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(
> >> > CliGitAPIImpl.java:405)
> >> > > at
> >> > > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
> >> > ionHandler$1.call(RemoteGitImpl.java:153)
> >> > > at
> >> > > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
> >> > ionHandler$1.call(RemoteGitImpl.java:146)
> >> > > at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> >> > > at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> >> > > at hudson.remoting.Request$2.run(Request.java:369)
> >> > > at
> >> > > hudson.remoting.InterceptingExecutorService$1.call(Intercept
> >> > ingExecutorService.java:72)
> >> > > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> > > at
> >> > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> >> > Executor.java:1142)
> >> > > at
> >> > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> >> > lExecutor.java:617)
> >> > > at java.lang.Thread.run(Thread.java:748)
> >> > > Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
> >> > call
> >> > > to H21
> >> > > at
> >> > > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> >> > > at
> >> > > hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserR
> >> > equest.java:357)
> >> > > at hudson.remoting.Channel.call(Channel.java:955)
> >> > > at
> >> > > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
> >> > ionHandler.execute(RemoteGitImpl.java:146)
> >> > > at
> >> sun.reflect.GeneratedMethodAccessor918.invoke(Unknown
> >> > > Source)
> >> > > at
> >> > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
> >> > thodAccessorImpl.java:43)
> >> > > at java.lang.reflect.Method.invoke(Method.java:498)
> >> > > at
> >> > > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
> >> > ionHandler.invoke(RemoteGitImpl.java:132)
> >> > > at com.sun.proxy.$Proxy119.execute(Unknown Source)
> >> > > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:886)
> >> > > at
> >> > > hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> >> > > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> >> > > at hudson.scm.SCM.checkout(SCM.java:504)
> >> > > at
> >> > > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> >> > > at
> >> > > hudson.model.AbstractBuild$AbstractBuildExecution.defaultChe
> >> > ckout(AbstractBuild.java:574)
> >> > > at
> >> > > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> >> > > at
> >> > > hudson.model.AbstractBuild$AbstractBuildExecution.run(Abstra
> >> > ctBuild.java:499)
> >> > > at hudson.model.Run.execute(Run.java:1794)
> >> > > at hudson.model.FreeStyleBuild.ru
> >> > n(FreeStyleBuild.java:43)
> >> > > at
> >> > > hudson.model.ResourceController.execute(ResourceController.java:97)
> >> > > at hudson.model.Executor.run(Executor.java:429)
> >> > > ERROR: Error fetching remote repo 'origin'
> >> > > Retrying after 10 seconds
> >> > >> git rev-parse --is-inside-work-tree # timeout=10
> >> > > Fetching changes from the remote Git repository
> >> > >> git config remote.origin.url
> >> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git #
> >> > timeout=10
> >> > > ERROR: Error fetching remote repo 'origin'
> >> > > hudson.plugins.git.GitException: Failed to fetch from
> >> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> >> > > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
> >> > > at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> >> > > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> >> > > at hudson.scm.SCM.checkout(SCM.java:504)
> >> > > at hudson.model.AbstractProject.checkout(AbstractProject.java:1
> >> > 208)
> >> > > at
> >> > > hudson.model.AbstractBuild$AbstractBuildExecution.defaultChe
> >> > ckout(AbstractBuild.java:574)
> >> > > at
> >> > > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> >> > > at
> >> > > hudson.model.AbstractBuild$AbstractBuildExecution.run(Abstra
> >> > ctBuild.java:499)
> >> > > at hudson.model.Run.execute(Run.java:1794)
> >> > > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> >> > > at
> >> > > hudson.model.ResourceController.execute(ResourceController.java:97)
> >> > > at hudson.model.Executor.run(Executor.java:429)
> >> > > Caused by: hudson.plugins.git.GitException: Command "git config
> >> > > remote.origin.url
> >> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git"
> >> returned
> >> > > status code 4:
> >> > > stdout:
> >> > > stderr: error: failed to write new configuration file <
> >> > > https://builds.apache.org/job/guacamole-client-coverity/ws/g
> >> > uacamole-client/.git/config.lock
> >> > >>
> >> > >
> >> > > at
> >> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandI
> >> > n(CliGitAPIImpl.java:2002)
> >> > > at
> >> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandI
> >> > n(CliGitAPIImpl.java:1970)
> >> > > at
> >> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandI
> >> > n(CliGitAPIImpl.java:1966)
> >> > > at
> >> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(
> >> > CliGitAPIImpl.java:1597)
> >> > > at
> >> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(
> >> > CliGitAPIImpl.java:1609)
> >> > > at
> >> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.setRemoteUrl(
> >> > CliGitAPIImpl.java:1243)
> >> > > at hudson.plugins.git.GitAPI.setRemoteUrl(GitAPI.java:160)
> >> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> > > at
> >> > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
> >> > ssorImpl.java:62)
> >> > > at
> >> > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
> >> > thodAccessorImpl.java:43)
> >> > > at java.lang.reflect.Method.invoke(Method.java:498)
> >> > > at
> >> > > hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(R
> >> > emoteInvocationHandler.java:929)
> >> > > at
> >> > > hudson.remoting.RemoteInvocationHandler$RPCRequest.call(Remo
> >> > teInvocationHandler.java:903)
> >> > > at
> >> > > hudson.remoting.RemoteInvocationHandler$RPCRequest.call(Remo
> >> > teInvocationHandler.java:855)
> >> > > at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> >> > > at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> >> > > at hudson.remoting.Request$2.run(Request.java:369)
> >> > > at
> >> > > hudson.remoting.InterceptingExecutorService$1.call(Intercept
> >> > ingExecutorService.java:72)
> >> > > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> > > at
> >> > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> >> > Executor.java:1142)
> >> > > at
> >> > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> >> > lExecutor.java:617)
> >> > > at java.lang.Thread.run(Thread.java:748)
> >> > > Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
> >> > call
> >> > > to H21
> >> > > at
> >> > > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> >> > > at
> >> > > hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserR
> >> > equest.java:357)
> >> > > at hudson.remoting.Channel.call(Channel.java:955)
> >> > > at
> >> > > hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocat
> >> > ionHandler.java:283)
> >> > > at com.sun.proxy.$Proxy118.setRemoteUrl(Unknown Source)
> >> > > at
> >> > > org.jenkinsci.plugins.gitclient.RemoteGitImpl.setRemoteUrl(
> >> > RemoteGitImpl.java:295)
> >> > > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:876)
> >> > > at
> >> > > hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> >> > > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> >> > > at hudson.scm.SCM.checkout(SCM.java:504)
> >> > > at
> >> > > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> >> > > at
> >> > > hudson.model.AbstractBuild$AbstractBuildExecution.defaultChe
> >> > ckout(AbstractBuild.java:574)
> >> > > at
> >> > > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> >> > > at
> >> > > hudson.model.AbstractBuild$AbstractBuildExecution.run(Abstra
> >> > ctBuild.java:499)
> >> > > at hudson.model.Run.execute(Run.java:1794)
> >> > > at hudson.model.FreeStyleBuild.ru
> >> > n(FreeStyleBuild.java:43)
> >> > > at
> >> > > hudson.model.ResourceController.execute(ResourceController.java:97)
> >> > > at hudson.model.Executor.run(Executor.java:429)
> >> > > ERROR: Error fetching remote repo 'origin'
> >> >
> >> >
> >>
> >
Re: No space on H21 (Was: Fwd: Build failed in Jenkins:
guacamole-client-coverity #73)
Posted by Greg Stein <gs...@gmail.com>.
Oh, and I'm not trying to avoid your question about auto-removal. I don't
have that answer, and will leave it for others.
On Sun, Sep 30, 2018 at 11:22 PM Greg Stein <gs...@gmail.com> wrote:
> We're refining our monitoring and some cron jobs to try and keep the disk
> space pruned. But when space balloons due to poor cleanup, there is only so
> much we can do.
>
> The short answer is that the multitude of groups at the Foundation using
> these shared resources need to be cognizant of that fact, and cooperate on
> ways to keep them available for all. We've seen disk space problems, too
> many jobs, too-long jobs, etc.
>
> Infra will keep working and improving our management of the build nodes.
> In the meantime, please keep reporting problems, and we'll do our best to
> keep the system up for y'all.
>
> Cheers,
> Greg
> InfraAdmin, ASF
>
>
> On Sun, Sep 30, 2018 at 10:54 PM Mike Jumper <mj...@apache.org> wrote:
>
>> Is there any way Jenkins could be configured to take nodes offline
>> automatically when free space is below a certain threshold? Granted
>> space-hungry builds and proper cleanup remain a problem, but at least
>> there
>> wouldn't be a random chance of jobs failing due to being assigned to a
>> full
>> node.
>>
>> - Mike
>>
>>
>> On Sun, Sep 30, 2018 at 3:57 PM, Chris Lambertus <cm...@apache.org> wrote:
>>
>> > The main culprit on H31 appears to be pulsar ws-cleanup jobs consuming
>> > almost 40% of the disk. (The problem on H21 was something
>> docker-related.)
>> >
>> >
>> > This is the second project I have found (Brooklyn was the first) which
>> is
>> > leaving ws-cleanup directories all over the workspace and filling up the
>> > disks. Each one of these was 500+ MB. Pulsar, please address ASAP.
>> >
>> >
>> >
>> >
>> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:19
>> > pulsar_precommit_cpp@2_ws-cleanup_1535737112280
>> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:18
>> > pulsar_precommit_cpp@2_ws-cleanup_1536174576053
>> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 06:04
>> > pulsar_precommit_cpp@2_ws-cleanup_1537292379178
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 22:45
>> > pulsar_precommit_cpp@2_ws-cleanup_1537397120862
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
>> > pulsar_precommit_cpp@2_ws-cleanup_1537403378279
>> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 21 23:31
>> > pulsar_precommit_cpp@2_ws-cleanup_1537572701829
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 26 16:04
>> > pulsar_precommit_cpp@2_ws-cleanup_1537977879341
>> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 22:23
>> > pulsar_precommit_cpp@2_ws-cleanup_1538087035027
>> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:14
>> > pulsar_precommit_cpp_ws-cleanup_1535699830230
>> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:16
>> > pulsar_precommit_cpp_ws-cleanup_1535735511423
>> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:13
>> > pulsar_precommit_cpp_ws-cleanup_1535786064397
>> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:18
>> > pulsar_precommit_cpp_ws-cleanup_1536096963586
>> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 21 06:47
>> > pulsar_precommit_cpp_ws-cleanup_1536114938910
>> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 21 06:45
>> > pulsar_precommit_cpp_ws-cleanup_1536173165909
>> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:38
>> > pulsar_precommit_cpp_ws-cleanup_1536175584418
>> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:50
>> > pulsar_precommit_cpp_ws-cleanup_1536180707506
>> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:38
>> > pulsar_precommit_cpp_ws-cleanup_1536182927940
>> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:51
>> > pulsar_precommit_cpp_ws-cleanup_1536185193716
>> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:50
>> > pulsar_precommit_cpp_ws-cleanup_1536273709029
>> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 05:59
>> > pulsar_precommit_cpp_ws-cleanup_1536316964794
>> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:06
>> > pulsar_precommit_cpp_ws-cleanup_1536675650893
>> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 28 06:08
>> > pulsar_precommit_cpp_ws-cleanup_1536879616962
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 30 19:53
>> > pulsar_precommit_cpp_ws-cleanup_1536891056968
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 30 19:47
>> > pulsar_precommit_cpp_ws-cleanup_1536966466683
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 18 00:43
>> > pulsar_precommit_cpp_ws-cleanup_1537216387387
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 00:45
>> > pulsar_precommit_cpp_ws-cleanup_1537292309413
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 00:45
>> > pulsar_precommit_cpp_ws-cleanup_1537308526251
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
>> > pulsar_precommit_cpp_ws-cleanup_1537331780420
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
>> > pulsar_precommit_cpp_ws-cleanup_1537379886715
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
>> > pulsar_precommit_cpp_ws-cleanup_1537386197893
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
>> > pulsar_precommit_cpp_ws-cleanup_1537397073464
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
>> > pulsar_precommit_cpp_ws-cleanup_1537403247575
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:43
>> > pulsar_precommit_cpp_ws-cleanup_1537413954888
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
>> > pulsar_precommit_cpp_ws-cleanup_1537417322151
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
>> > pulsar_precommit_cpp_ws-cleanup_1537421381129
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:44
>> > pulsar_precommit_cpp_ws-cleanup_1537469348852
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
>> > pulsar_precommit_cpp_ws-cleanup_1537489399468
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 22 00:44
>> > pulsar_precommit_cpp_ws-cleanup_1537566528660
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 23:12
>> > pulsar_precommit_cpp_ws-cleanup_1537571549978
>> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 22 00:44
>> > pulsar_precommit_cpp_ws-cleanup_1537573605258
>> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 00:43
>> > pulsar_precommit_cpp_ws-cleanup_1537855939828
>> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 11:18
>> > pulsar_precommit_cpp_ws-cleanup_1537874305575
>> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 12:41
>> > pulsar_precommit_cpp_ws-cleanup_1537879296622
>> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 01:39
>> > pulsar_precommit_cpp_ws-cleanup_1537925955967
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 26 13:17
>> > pulsar_precommit_cpp_ws-cleanup_1537967826838
>> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 15:55
>> > pulsar_precommit_cpp_ws-cleanup_1537977326736
>> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 17:34
>> > pulsar_precommit_cpp_ws-cleanup_1537983268177
>> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 22:04
>> > pulsar_precommit_cpp_ws-cleanup_1538085846793
>> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 28 00:20
>> > pulsar_precommit_cpp_ws-cleanup_1538094024821
>> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 28 17:42
>> > pulsar_precommit_cpp_ws-cleanup_1538156543343
>> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:02
>> > pulsar_precommit_integrationtests_ws-cleanup_1536298125841
>> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:00
>> > pulsar_precommit_integrationtests_ws-cleanup_1536968128660
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 18 00:43
>> > pulsar_precommit_integrationtests_ws-cleanup_1537218541598
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
>> > pulsar_precommit_integrationtests_ws-cleanup_1537331703324
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 22 00:43
>> > pulsar_precommit_integrationtests_ws-cleanup_1537553748859
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 00:43
>> > pulsar_precommit_integrationtests_ws-cleanup_1537730402937
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 07:48
>> > pulsar_precommit_integrationtests_ws-cleanup_1537775336586
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 23:30
>> > pulsar_precommit_integrationtests_ws-cleanup_1537831851711
>> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 23:08
>> > pulsar_precommit_integrationtests_ws-cleanup_1537916928485
>> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 15:21
>> > pulsar_precommit_integrationtests_ws-cleanup_1538061702243
>> >
>> >
>> > -Chris
>> > ASF Infra
>> >
>> >
>> >
>> >
>> >
>> > > On Sep 30, 2018, at 3:12 PM, Chris Thistlethwaite <
>> > cthistlego@outlook.com> wrote:
>> > >
>> > > I've been working on H21 and H31. They both had a some docker issues
>> > where the process was locked and I couldn't do anything with a few
>> > containers that were over 8 days old.
>> > >
>> > > Thought we might be good this morning so I threw them back into
>> > rotation. Guess not, so I'll take them back out.
>> > >
>> > > -Chris T.
>> > > ________________________________
>> > > From: Mike Jumper <mj...@apache.org>
>> > > Sent: Sunday, September 30, 2018 6:03:51 PM
>> > > To: builds@apache.org
>> > > Subject: No space on H21 (Was: Fwd: Build failed in Jenkins:
>> > guacamole-client-coverity #73)
>> > >
>> > > FYI - we're currently seeing disk space issues on H21.
>> > >
>> > > - Mike
>> > >
>> > >
>> > > ---------- Forwarded message ---------
>> > > From: Apache Jenkins Server <je...@builds.apache.org>
>> > > Date: Sun, Sep 30, 2018, 14:37
>> > > Subject: Build failed in Jenkins: guacamole-client-coverity #73
>> > > To: <de...@guacamole.apache.org>
>> > >
>> > >
>> > > See <
>> > > https://builds.apache.org/job/guacamole-client-coverity/73/d
>> > isplay/redirect>
>> > >
>> > > ------------------------------------------
>> > > Started by user mjumper
>> > > [EnvInject] - Loading node environment variables.
>> > > Building remotely on H21 (ubuntu xenial) in workspace <
>> > > https://builds.apache.org/job/guacamole-client-coverity/ws/>
>> > > [WS-CLEANUP] Deleting project workspace...
>> > > Cloning the remote Git repository
>> > > Cloning repository
>> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>> > >> git init <
>> > > https://builds.apache.org/job/guacamole-client-coverity/ws/g
>> > uacamole-client>
>> > > # timeout=10
>> > > Fetching upstream changes from
>> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>> > >> git --version # timeout=10
>> > >> git fetch --tags --progress
>> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>> > > +refs/heads/*:refs/remotes/origin/*
>> > > ERROR: Error cloning remote repo 'origin'
>> > > hudson.plugins.git.GitException: Command "git fetch --tags --progress
>> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>> > > +refs/heads/*:refs/remotes/origin/*" returned status code 128:
>> > > stdout:
>> > > stderr: remote: Counting objects: 62166, done.
>> > > remote: Compressing objects: 0% (1/16096) remote:
>> Compressing
>> > > objects: 1% (161/16096) remote: Compressing objects: 2%
>> > > (322/16096) remote: Compressing objects: 3% (483/16096)
>> > > remote: Compressing objects: 4% (644/16096) remote:
>> > > Compressing objects: 5% (805/16096) remote: Compressing
>> > > objects: 6% (966/16096) remote: Compressing objects: 7%
>> > > (1127/16096) remote: Compressing objects: 8% (1288/16096)
>> > > remote: Compressing objects: 9% (1449/16096) remote:
>> > > Compressing objects: 10% (1610/16096) remote: Compressing
>> > > objects: 11% (1771/16096) remote: Compressing objects: 12%
>> > > (1932/16096) remote: Compressing objects: 13% (2093/16096)
>> > > remote: Compressing objects: 14% (2254/16096) remote:
>> > > Compressing objects: 15% (2415/16096) remote: Compressing
>> > > objects: 16% (2576/16096) remote: Compressing objects: 17%
>> > > (2737/16096) remote: Compressing objects: 18% (2898/16096)
>> > > remote: Compressing objects: 19% (3059/16096) remote:
>> > > Compressing objects: 20% (3220/16096) remote: Compressing
>> > > objects: 21% (3381/16096) remote: Compressing objects: 22%
>> > > (3542/16096) remote: Compressing objects: 23% (3703/16096)
>> > > remote: Compressing objects: 24% (3864/16096) remote:
>> > > Compressing objects: 25% (4024/16096) remote: Compressing
>> > > objects: 26% (4185/16096) remote: Compressing objects: 27%
>> > > (4346/16096) remote: Compressing objects: 28% (4507/16096)
>> > > remote: Compressing objects: 29% (4668/16096) remote:
>> > > Compressing objects: 30% (4829/16096) remote: Compressing
>> > > objects: 31% (4990/16096) remote: Compressing objects: 32%
>> > > (5151/16096) remote: Compressing objects: 33% (5312/16096)
>> > > remote: Compressing objects: 34% (5473/16096) remote:
>> > > Compressing objects: 35% (5634/16096) remote: Compressing
>> > > objects: 36% (5795/16096) remote: Compressing objects: 37%
>> > > (5956/16096) remote: Compressing objects: 38% (6117/16096)
>> > > remote: Compressing objects: 39% (6278/16096) remote:
>> > > Compressing objects: 40% (6439/16096) remote: Compressing
>> > > objects: 41% (6600/16096) remote: Compressing objects: 42%
>> > > (6761/16096) remote: Compressing objects: 43% (6922/16096)
>> > > remote: Compressing objects: 44% (7083/16096) remote:
>> > > Compressing objects: 45% (7244/16096) remote: Compressing
>> > > objects: 46% (7405/16096) remote: Compressing objects: 47%
>> > > (7566/16096) remote: Compressing objects: 48% (7727/16096)
>> > > remote: Compressing objects: 49% (7888/16096) remote:
>> > > Compressing objects: 50% (8048/16096) remote: Compressing
>> > > objects: 51% (8209/16096) remote: Compressing objects: 52%
>> > > (8370/16096) remote: Compressing objects: 53% (8531/16096)
>> > > remote: Compressing objects: 54% (8692/16096) remote:
>> > > Compressing objects: 55% (8853/16096) remote: Compressing
>> > > objects: 56% (9014/16096) remote: Compressing objects: 57%
>> > > (9175/16096) remote: Compressing objects: 58% (9336/16096)
>> > > remote: Compressing objects: 59% (9497/16096) remote:
>> > > Compressing objects: 60% (9658/16096) remote: Compressing
>> > > objects: 61% (9819/16096) remote: Compressing objects: 62%
>> > > (9980/16096) remote: Compressing objects: 63% (10141/16096)
>> > > remote: Compressing objects: 64% (10302/16096) remote:
>> > > Compressing objects: 65% (10463/16096) remote: Compressing
>> > > objects: 66% (10624/16096) remote: Compressing objects:
>> 67%
>> > > (10785/16096) remote: Compressing objects: 68%
>> (10946/16096)
>> > > remote: Compressing objects: 69% (11107/16096)
>> remote:
>> > > Compressing objects: 70% (11268/16096) remote: Compressing
>> > > objects: 71% (11429/16096) remote: Compressing objects:
>> 72%
>> > > (11590/16096) remote: Compressing objects: 73%
>> (11751/16096)
>> > > remote: Compressing objects: 74% (11912/16096)
>> remote:
>> > > Compressing objects: 75% (12072/16096) remote: Compressing
>> > > objects: 76% (12233/16096) remote: Compressing objects:
>> 77%
>> > > (12394/16096) remote: Compressing objects: 78%
>> (12555/16096)
>> > > remote: Compressing objects: 79% (12716/16096)
>> remote:
>> > > Compressing objects: 80% (12877/16096) remote: Compressing
>> > > objects: 81% (13038/16096) remote: Compressing objects:
>> 82%
>> > > (13199/16096) remote: Compressing objects: 83%
>> (13360/16096)
>> > > remote: Compressing objects: 84% (13521/16096)
>> remote:
>> > > Compressing objects: 85% (13682/16096) remote: Compressing
>> > > objects: 86% (13843/16096) remote: Compressing objects:
>> 87%
>> > > (14004/16096) remote: Compressing objects: 88%
>> (14165/16096)
>> > > remote: Compressing objects: 89% (14326/16096)
>> remote:
>> > > Compressing objects: 90% (14487/16096) remote: Compressing
>> > > objects: 91% (14648/16096) remote: Compressing objects:
>> 92%
>> > > (14809/16096) remote: Compressing objects: 93%
>> (14970/16096)
>> > > remote: Compressing objects: 94% (15131/16096)
>> remote:
>> > > Compressing objects: 95% (15292/16096) remote: Compressing
>> > > objects: 96% (15453/16096) remote: Compressing objects:
>> 97%
>> > > (15614/16096) remote: Compressing objects: 98%
>> (15775/16096)
>> > > remote: Compressing objects: 99% (15936/16096)
>> remote:
>> > > Compressing objects: 100% (16096/16096) remote: Compressing
>> > > objects: 100% (16096/16096), done.
>> > > Receiving objects: 0% (1/62166) Receiving objects: 1%
>> (622/62166)
>> > > Receiving objects: 2% (1244/62166) Receiving objects: 3%
>> > > (1865/62166) Receiving objects: 4% (2487/62166) Receiving
>> objects:
>> > > 5% (3109/62166) Receiving objects: 6% (3730/62166) Receiving
>> > > objects: 7% (4352/62166) Receiving objects: 8% (4974/62166)
>> > > Receiving objects: 9% (5595/62166) Receiving objects: 10%
>> > > (6217/62166) Receiving objects: 11% (6839/62166) Receiving
>> objects:
>> > > 12% (7460/62166) Receiving objects: 13% (8082/62166) Receiving
>> > > objects: 14% (8704/62166) Receiving objects: 15% (9325/62166)
>> > > Receiving objects: 16% (9947/62166) Receiving objects: 17%
>> > > (10569/62166) Receiving objects: 18% (11190/62166) Receiving
>> > objects:
>> > > 19% (11812/62166) Receiving objects: 20% (12434/62166) Receiving
>> > > objects: 21% (13055/62166) Receiving objects: 22% (13677/62166)
>> > > fatal: write error: No space left on device
>> > > fatal: index-pack failed
>> > >
>> > > at
>> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandI
>> > n(CliGitAPIImpl.java:2002)
>> > > at
>> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandW
>> > ithCredentials(CliGitAPIImpl.java:1721)
>> > > at
>> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(
>> > CliGitAPIImpl.java:72)
>> > > at
>> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(
>> > CliGitAPIImpl.java:405)
>> > > at
>> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(
>> > CliGitAPIImpl.java:614)
>> > > at
>> > > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
>> > ionHandler$1.call(RemoteGitImpl.java:153)
>> > > at
>> > > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
>> > ionHandler$1.call(RemoteGitImpl.java:146)
>> > > at hudson.remoting.UserRequest.perform(UserRequest.java:212)
>> > > at hudson.remoting.UserRequest.perform(UserRequest.java:54)
>> > > at hudson.remoting.Request$2.run(Request.java:369)
>> > > at
>> > > hudson.remoting.InterceptingExecutorService$1.call(Intercept
>> > ingExecutorService.java:72)
>> > > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > > at
>> > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>> > Executor.java:1142)
>> > > at
>> > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>> > lExecutor.java:617)
>> > > at java.lang.Thread.run(Thread.java:748)
>> > > Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
>> > call
>> > > to H21
>> > > at
>> > > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
>> > > at
>> > > hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserR
>> > equest.java:357)
>> > > at hudson.remoting.Channel.call(Channel.java:955)
>> > > at
>> > > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
>> > ionHandler.execute(RemoteGitImpl.java:146)
>> > > at
>> sun.reflect.GeneratedMethodAccessor918.invoke(Unknown
>> > > Source)
>> > > at
>> > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>> > thodAccessorImpl.java:43)
>> > > at java.lang.reflect.Method.invoke(Method.java:498)
>> > > at
>> > > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
>> > ionHandler.invoke(RemoteGitImpl.java:132)
>> > > at com.sun.proxy.$Proxy142.execute(Unknown Source)
>> > > at
>> > > hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1146)
>> > > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
>> > > at hudson.scm.SCM.checkout(SCM.java:504)
>> > > at
>> > > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>> > > at
>> > > hudson.model.AbstractBuild$AbstractBuildExecution.defaultChe
>> > ckout(AbstractBuild.java:574)
>> > > at
>> > > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>> > > at
>> > > hudson.model.AbstractBuild$AbstractBuildExecution.run(Abstra
>> > ctBuild.java:499)
>> > > at hudson.model.Run.execute(Run.java:1794)
>> > > at hudson.model.FreeStyleBuild.ru
>> > n(FreeStyleBuild.java:43)
>> > > at
>> > > hudson.model.ResourceController.execute(ResourceController.java:97)
>> > > at hudson.model.Executor.run(Executor.java:429)
>> > > ERROR: Error cloning remote repo 'origin'
>> > > Retrying after 10 seconds
>> > >> git rev-parse --is-inside-work-tree # timeout=10
>> > > Fetching changes from the remote Git repository
>> > >> git config remote.origin.url
>> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git #
>> > timeout=10
>> > > Fetching upstream changes from
>> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>> > >> git --version # timeout=10
>> > >> git fetch --tags --progress
>> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>> > > +refs/heads/*:refs/remotes/origin/*
>> > > ERROR: Error fetching remote repo 'origin'
>> > > hudson.plugins.git.GitException: Failed to fetch from
>> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>> > > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
>> > > at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
>> > > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
>> > > at hudson.scm.SCM.checkout(SCM.java:504)
>> > > at hudson.model.AbstractProject.checkout(AbstractProject.java:1
>> > 208)
>> > > at
>> > > hudson.model.AbstractBuild$AbstractBuildExecution.defaultChe
>> > ckout(AbstractBuild.java:574)
>> > > at
>> > > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>> > > at
>> > > hudson.model.AbstractBuild$AbstractBuildExecution.run(Abstra
>> > ctBuild.java:499)
>> > > at hudson.model.Run.execute(Run.java:1794)
>> > > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>> > > at
>> > > hudson.model.ResourceController.execute(ResourceController.java:97)
>> > > at hudson.model.Executor.run(Executor.java:429)
>> > > Caused by: hudson.plugins.git.GitException: Command "git fetch --tags
>> > > --progress
>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>> > > +refs/heads/*:refs/remotes/origin/*" returned status code 128:
>> > > stdout:
>> > > stderr: remote: Counting objects: 62166, done.
>> > > remote: Compressing objects: 0% (1/16096) remote:
>> Compressing
>> > > objects: 1% (161/16096) remote: Compressing objects: 2%
>> > > (322/16096) remote: Compressing objects: 3% (483/16096)
>> > > remote: Compressing objects: 4% (644/16096) remote:
>> > > Compressing objects: 5% (805/16096) remote: Compressing
>> > > objects: 6% (966/16096) remote: Compressing objects: 7%
>> > > (1127/16096) remote: Compressing objects: 8% (1288/16096)
>> > > remote: Compressing objects: 9% (1449/16096) remote:
>> > > Compressing objects: 10% (1610/16096) remote: Compressing
>> > > objects: 11% (1771/16096) remote: Compressing objects: 12%
>> > > (1932/16096) remote: Compressing objects: 13% (2093/16096)
>> > > remote: Compressing objects: 14% (2254/16096) remote:
>> > > Compressing objects: 15% (2415/16096) remote: Compressing
>> > > objects: 16% (2576/16096) remote: Compressing objects: 17%
>> > > (2737/16096) remote: Compressing objects: 18% (2898/16096)
>> > > remote: Compressing objects: 19% (3059/16096) remote:
>> > > Compressing objects: 20% (3220/16096) remote: Compressing
>> > > objects: 21% (3381/16096) remote: Compressing objects: 22%
>> > > (3542/16096) remote: Compressing objects: 23% (3703/16096)
>> > > remote: Compressing objects: 24% (3864/16096) remote:
>> > > Compressing objects: 25% (4024/16096) remote: Compressing
>> > > objects: 26% (4185/16096) remote: Compressing objects: 27%
>> > > (4346/16096) remote: Compressing objects: 28% (4507/16096)
>> > > remote: Compressing objects: 29% (4668/16096) remote:
>> > > Compressing objects: 30% (4829/16096) remote: Compressing
>> > > objects: 31% (4990/16096) remote: Compressing objects: 32%
>> > > (5151/16096) remote: Compressing objects: 33% (5312/16096)
>> > > remote: Compressing objects: 34% (5473/16096) remote:
>> > > Compressing objects: 35% (5634/16096) remote: Compressing
>> > > objects: 36% (5795/16096) remote: Compressing objects: 37%
>> > > (5956/16096) remote: Compressing objects: 38% (6117/16096)
>> > > remote: Compressing objects: 39% (6278/16096) remote:
>> > > Compressing objects: 40% (6439/16096) remote: Compressing
>> > > objects: 41% (6600/16096) remote: Compressing objects: 42%
>> > > (6761/16096) remote: Compressing objects: 43% (6922/16096)
>> > > remote: Compressing objects: 44% (7083/16096) remote:
>> > > Compressing objects: 45% (7244/16096) remote: Compressing
>> > > objects: 46% (7405/16096) remote: Compressing objects: 47%
>> > > (7566/16096) remote: Compressing objects: 48% (7727/16096)
>> > > remote: Compressing objects: 49% (7888/16096) remote:
>> > > Compressing objects: 50% (8048/16096) remote: Compressing
>> > > objects: 51% (8209/16096) remote: Compressing objects: 52%
>> > > (8370/16096) remote: Compressing objects: 53% (8531/16096)
>> > > remote: Compressing objects: 54% (8692/16096) remote:
>> > > Compressing objects: 55% (8853/16096) remote: Compressing
>> > > objects: 56% (9014/16096) remote: Compressing objects: 57%
>> > > (9175/16096) remote: Compressing objects: 58% (9336/16096)
>> > > remote: Compressing objects: 59% (9497/16096) remote:
>> > > Compressing objects: 60% (9658/16096) remote: Compressing
>> > > objects: 61% (9819/16096) remote: Compressing objects: 62%
>> > > (9980/16096) remote: Compressing objects: 63% (10141/16096)
>> > > remote: Compressing objects: 64% (10302/16096) remote:
>> > > Compressing objects: 65% (10463/16096) remote: Compressing
>> > > objects: 66% (10624/16096) remote: Compressing objects:
>> 67%
>> > > (10785/16096) remote: Compressing objects: 68%
>> (10946/16096)
>> > > remote: Compressing objects: 69% (11107/16096)
>> remote:
>> > > Compressing objects: 70% (11268/16096) remote: Compressing
>> > > objects: 71% (11429/16096) remote: Compressing objects:
>> 72%
>> > > (11590/16096) remote: Compressing objects: 73%
>> (11751/16096)
>> > > remote: Compressing objects: 74% (11912/16096)
>> remote:
>> > > Compressing objects: 75% (12072/16096) remote: Compressing
>> > > objects: 76% (12233/16096) remote: Compressing objects:
>> 77%
>> > > (12394/16096) remote: Compressing objects: 78%
>> (12555/16096)
>> > > remote: Compressing objects: 79% (12716/16096)
>> remote:
>> > > Compressing objects: 80% (12877/16096) remote: Compressing
>> > > objects: 81% (13038/16096) remote: Compressing objects:
>> 82%
>> > > (13199/16096) remote: Compressing objects: 83%
>> (13360/16096)
>> > > remote: Compressing objects: 84% (13521/16096)
>> remote:
>> > > Compressing objects: 85% (13682/16096) remote: Compressing
>> > > objects: 86% (13843/16096) remote: Compressing objects:
>> 87%
>> > > (14004/16096) remote: Compressing objects: 88%
>> (14165/16096)
>> > > remote: Compressing objects: 89% (14326/16096)
>> remote:
>> > > Compressing objects: 90% (14487/16096) remote: Compressing
>> > > objects: 91% (14648/16096) remote: Compressing objects:
>> 92%
>> > > (14809/16096) remote: Compressing objects: 93%
>> (14970/16096)
>> > > remote: Compressing objects: 94% (15131/16096)
>> remote:
>> > > Compressing objects: 95% (15292/16096) remote: Compressing
>> > > objects: 96% (15453/16096) remote: Compressing objects:
>> 97%
>> > > (15614/16096) remote: Compressing objects: 98%
>> (15775/16096)
>> > > remote: Compressing objects: 99% (15936/16096)
>> remote:
>> > > Compressing objects: 100% (16096/16096) remote: Compressing
>> > > objects: 100% (16096/16096), done.
>> > > Receiving objects: 0% (1/62166) fatal: write error: No space left
>> on
>> > > device
>> > > fatal: index-pack failed
>> > >
>> > > at
>> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandI
>> > n(CliGitAPIImpl.java:2002)
>> > > at
>> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandW
>> > ithCredentials(CliGitAPIImpl.java:1721)
>> > > at
>> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(
>> > CliGitAPIImpl.java:72)
>> > > at
>> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(
>> > CliGitAPIImpl.java:405)
>> > > at
>> > > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
>> > ionHandler$1.call(RemoteGitImpl.java:153)
>> > > at
>> > > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
>> > ionHandler$1.call(RemoteGitImpl.java:146)
>> > > at hudson.remoting.UserRequest.perform(UserRequest.java:212)
>> > > at hudson.remoting.UserRequest.perform(UserRequest.java:54)
>> > > at hudson.remoting.Request$2.run(Request.java:369)
>> > > at
>> > > hudson.remoting.InterceptingExecutorService$1.call(Intercept
>> > ingExecutorService.java:72)
>> > > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > > at
>> > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>> > Executor.java:1142)
>> > > at
>> > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>> > lExecutor.java:617)
>> > > at java.lang.Thread.run(Thread.java:748)
>> > > Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
>> > call
>> > > to H21
>> > > at
>> > > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
>> > > at
>> > > hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserR
>> > equest.java:357)
>> > > at hudson.remoting.Channel.call(Channel.java:955)
>> > > at
>> > > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
>> > ionHandler.execute(RemoteGitImpl.java:146)
>> > > at
>> sun.reflect.GeneratedMethodAccessor918.invoke(Unknown
>> > > Source)
>> > > at
>> > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>> > thodAccessorImpl.java:43)
>> > > at java.lang.reflect.Method.invoke(Method.java:498)
>> > > at
>> > > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
>> > ionHandler.invoke(RemoteGitImpl.java:132)
>> > > at com.sun.proxy.$Proxy119.execute(Unknown Source)
>> > > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:886)
>> > > at
>> > > hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
>> > > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
>> > > at hudson.scm.SCM.checkout(SCM.java:504)
>> > > at
>> > > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>> > > at
>> > > hudson.model.AbstractBuild$AbstractBuildExecution.defaultChe
>> > ckout(AbstractBuild.java:574)
>> > > at
>> > > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>> > > at
>> > > hudson.model.AbstractBuild$AbstractBuildExecution.run(Abstra
>> > ctBuild.java:499)
>> > > at hudson.model.Run.execute(Run.java:1794)
>> > > at hudson.model.FreeStyleBuild.ru
>> > n(FreeStyleBuild.java:43)
>> > > at
>> > > hudson.model.ResourceController.execute(ResourceController.java:97)
>> > > at hudson.model.Executor.run(Executor.java:429)
>> > > ERROR: Error fetching remote repo 'origin'
>> > > Retrying after 10 seconds
>> > >> git rev-parse --is-inside-work-tree # timeout=10
>> > > Fetching changes from the remote Git repository
>> > >> git config remote.origin.url
>> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git #
>> > timeout=10
>> > > ERROR: Error fetching remote repo 'origin'
>> > > hudson.plugins.git.GitException: Failed to fetch from
>> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>> > > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
>> > > at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
>> > > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
>> > > at hudson.scm.SCM.checkout(SCM.java:504)
>> > > at hudson.model.AbstractProject.checkout(AbstractProject.java:1
>> > 208)
>> > > at
>> > > hudson.model.AbstractBuild$AbstractBuildExecution.defaultChe
>> > ckout(AbstractBuild.java:574)
>> > > at
>> > > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>> > > at
>> > > hudson.model.AbstractBuild$AbstractBuildExecution.run(Abstra
>> > ctBuild.java:499)
>> > > at hudson.model.Run.execute(Run.java:1794)
>> > > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>> > > at
>> > > hudson.model.ResourceController.execute(ResourceController.java:97)
>> > > at hudson.model.Executor.run(Executor.java:429)
>> > > Caused by: hudson.plugins.git.GitException: Command "git config
>> > > remote.origin.url
>> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git"
>> returned
>> > > status code 4:
>> > > stdout:
>> > > stderr: error: failed to write new configuration file <
>> > > https://builds.apache.org/job/guacamole-client-coverity/ws/g
>> > uacamole-client/.git/config.lock
>> > >>
>> > >
>> > > at
>> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandI
>> > n(CliGitAPIImpl.java:2002)
>> > > at
>> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandI
>> > n(CliGitAPIImpl.java:1970)
>> > > at
>> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandI
>> > n(CliGitAPIImpl.java:1966)
>> > > at
>> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(
>> > CliGitAPIImpl.java:1597)
>> > > at
>> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(
>> > CliGitAPIImpl.java:1609)
>> > > at
>> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.setRemoteUrl(
>> > CliGitAPIImpl.java:1243)
>> > > at hudson.plugins.git.GitAPI.setRemoteUrl(GitAPI.java:160)
>> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> > > at
>> > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
>> > ssorImpl.java:62)
>> > > at
>> > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>> > thodAccessorImpl.java:43)
>> > > at java.lang.reflect.Method.invoke(Method.java:498)
>> > > at
>> > > hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(R
>> > emoteInvocationHandler.java:929)
>> > > at
>> > > hudson.remoting.RemoteInvocationHandler$RPCRequest.call(Remo
>> > teInvocationHandler.java:903)
>> > > at
>> > > hudson.remoting.RemoteInvocationHandler$RPCRequest.call(Remo
>> > teInvocationHandler.java:855)
>> > > at hudson.remoting.UserRequest.perform(UserRequest.java:212)
>> > > at hudson.remoting.UserRequest.perform(UserRequest.java:54)
>> > > at hudson.remoting.Request$2.run(Request.java:369)
>> > > at
>> > > hudson.remoting.InterceptingExecutorService$1.call(Intercept
>> > ingExecutorService.java:72)
>> > > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > > at
>> > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>> > Executor.java:1142)
>> > > at
>> > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>> > lExecutor.java:617)
>> > > at java.lang.Thread.run(Thread.java:748)
>> > > Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
>> > call
>> > > to H21
>> > > at
>> > > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
>> > > at
>> > > hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserR
>> > equest.java:357)
>> > > at hudson.remoting.Channel.call(Channel.java:955)
>> > > at
>> > > hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocat
>> > ionHandler.java:283)
>> > > at com.sun.proxy.$Proxy118.setRemoteUrl(Unknown Source)
>> > > at
>> > > org.jenkinsci.plugins.gitclient.RemoteGitImpl.setRemoteUrl(
>> > RemoteGitImpl.java:295)
>> > > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:876)
>> > > at
>> > > hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
>> > > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
>> > > at hudson.scm.SCM.checkout(SCM.java:504)
>> > > at
>> > > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>> > > at
>> > > hudson.model.AbstractBuild$AbstractBuildExecution.defaultChe
>> > ckout(AbstractBuild.java:574)
>> > > at
>> > > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>> > > at
>> > > hudson.model.AbstractBuild$AbstractBuildExecution.run(Abstra
>> > ctBuild.java:499)
>> > > at hudson.model.Run.execute(Run.java:1794)
>> > > at hudson.model.FreeStyleBuild.ru
>> > n(FreeStyleBuild.java:43)
>> > > at
>> > > hudson.model.ResourceController.execute(ResourceController.java:97)
>> > > at hudson.model.Executor.run(Executor.java:429)
>> > > ERROR: Error fetching remote repo 'origin'
>> >
>> >
>>
>
Re: No space on H21 (Was: Fwd: Build failed in Jenkins:
guacamole-client-coverity #73)
Posted by Greg Stein <gs...@gmail.com>.
We're refining our monitoring and some cron jobs to try and keep the disk
space pruned. But when space balloons due to poor cleanup, there is only so
much we can do.
The short answer is that the multitude of groups at the Foundation using
these shared resources need to be cognizant of that fact, and cooperate on
ways to keep them available for all. We've seen disk space problems, too
many jobs, too-long jobs, etc.
Infra will keep working and improving our management of the build nodes. In
the meantime, please keep reporting problems, and we'll do our best to keep
the system up for y'all.
Cheers,
Greg
InfraAdmin, ASF
On Sun, Sep 30, 2018 at 10:54 PM Mike Jumper <mj...@apache.org> wrote:
> Is there any way Jenkins could be configured to take nodes offline
> automatically when free space is below a certain threshold? Granted
> space-hungry builds and proper cleanup remain a problem, but at least there
> wouldn't be a random chance of jobs failing due to being assigned to a full
> node.
>
> - Mike
>
>
> On Sun, Sep 30, 2018 at 3:57 PM, Chris Lambertus <cm...@apache.org> wrote:
>
> > The main culprit on H31 appears to be pulsar ws-cleanup jobs consuming
> > almost 40% of the disk. (The problem on H21 was something
> docker-related.)
> >
> >
> > This is the second project I have found (Brooklyn was the first) which is
> > leaving ws-cleanup directories all over the workspace and filling up the
> > disks. Each one of these was 500+ MB. Pulsar, please address ASAP.
> >
> >
> >
> >
> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:19
> > pulsar_precommit_cpp@2_ws-cleanup_1535737112280
> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:18
> > pulsar_precommit_cpp@2_ws-cleanup_1536174576053
> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 06:04
> > pulsar_precommit_cpp@2_ws-cleanup_1537292379178
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 22:45
> > pulsar_precommit_cpp@2_ws-cleanup_1537397120862
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> > pulsar_precommit_cpp@2_ws-cleanup_1537403378279
> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 21 23:31
> > pulsar_precommit_cpp@2_ws-cleanup_1537572701829
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 26 16:04
> > pulsar_precommit_cpp@2_ws-cleanup_1537977879341
> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 22:23
> > pulsar_precommit_cpp@2_ws-cleanup_1538087035027
> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:14
> > pulsar_precommit_cpp_ws-cleanup_1535699830230
> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:16
> > pulsar_precommit_cpp_ws-cleanup_1535735511423
> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:13
> > pulsar_precommit_cpp_ws-cleanup_1535786064397
> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:18
> > pulsar_precommit_cpp_ws-cleanup_1536096963586
> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 21 06:47
> > pulsar_precommit_cpp_ws-cleanup_1536114938910
> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 21 06:45
> > pulsar_precommit_cpp_ws-cleanup_1536173165909
> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:38
> > pulsar_precommit_cpp_ws-cleanup_1536175584418
> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:50
> > pulsar_precommit_cpp_ws-cleanup_1536180707506
> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:38
> > pulsar_precommit_cpp_ws-cleanup_1536182927940
> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:51
> > pulsar_precommit_cpp_ws-cleanup_1536185193716
> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:50
> > pulsar_precommit_cpp_ws-cleanup_1536273709029
> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 05:59
> > pulsar_precommit_cpp_ws-cleanup_1536316964794
> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:06
> > pulsar_precommit_cpp_ws-cleanup_1536675650893
> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 28 06:08
> > pulsar_precommit_cpp_ws-cleanup_1536879616962
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 30 19:53
> > pulsar_precommit_cpp_ws-cleanup_1536891056968
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 30 19:47
> > pulsar_precommit_cpp_ws-cleanup_1536966466683
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 18 00:43
> > pulsar_precommit_cpp_ws-cleanup_1537216387387
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 00:45
> > pulsar_precommit_cpp_ws-cleanup_1537292309413
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 00:45
> > pulsar_precommit_cpp_ws-cleanup_1537308526251
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> > pulsar_precommit_cpp_ws-cleanup_1537331780420
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> > pulsar_precommit_cpp_ws-cleanup_1537379886715
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
> > pulsar_precommit_cpp_ws-cleanup_1537386197893
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
> > pulsar_precommit_cpp_ws-cleanup_1537397073464
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
> > pulsar_precommit_cpp_ws-cleanup_1537403247575
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:43
> > pulsar_precommit_cpp_ws-cleanup_1537413954888
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
> > pulsar_precommit_cpp_ws-cleanup_1537417322151
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
> > pulsar_precommit_cpp_ws-cleanup_1537421381129
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:44
> > pulsar_precommit_cpp_ws-cleanup_1537469348852
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
> > pulsar_precommit_cpp_ws-cleanup_1537489399468
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 22 00:44
> > pulsar_precommit_cpp_ws-cleanup_1537566528660
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 23:12
> > pulsar_precommit_cpp_ws-cleanup_1537571549978
> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 22 00:44
> > pulsar_precommit_cpp_ws-cleanup_1537573605258
> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 00:43
> > pulsar_precommit_cpp_ws-cleanup_1537855939828
> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 11:18
> > pulsar_precommit_cpp_ws-cleanup_1537874305575
> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 12:41
> > pulsar_precommit_cpp_ws-cleanup_1537879296622
> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 01:39
> > pulsar_precommit_cpp_ws-cleanup_1537925955967
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 26 13:17
> > pulsar_precommit_cpp_ws-cleanup_1537967826838
> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 15:55
> > pulsar_precommit_cpp_ws-cleanup_1537977326736
> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 17:34
> > pulsar_precommit_cpp_ws-cleanup_1537983268177
> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 22:04
> > pulsar_precommit_cpp_ws-cleanup_1538085846793
> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 28 00:20
> > pulsar_precommit_cpp_ws-cleanup_1538094024821
> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 28 17:42
> > pulsar_precommit_cpp_ws-cleanup_1538156543343
> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:02
> > pulsar_precommit_integrationtests_ws-cleanup_1536298125841
> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:00
> > pulsar_precommit_integrationtests_ws-cleanup_1536968128660
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 18 00:43
> > pulsar_precommit_integrationtests_ws-cleanup_1537218541598
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> > pulsar_precommit_integrationtests_ws-cleanup_1537331703324
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 22 00:43
> > pulsar_precommit_integrationtests_ws-cleanup_1537553748859
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 00:43
> > pulsar_precommit_integrationtests_ws-cleanup_1537730402937
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 07:48
> > pulsar_precommit_integrationtests_ws-cleanup_1537775336586
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 23:30
> > pulsar_precommit_integrationtests_ws-cleanup_1537831851711
> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 23:08
> > pulsar_precommit_integrationtests_ws-cleanup_1537916928485
> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 15:21
> > pulsar_precommit_integrationtests_ws-cleanup_1538061702243
> >
> >
> > -Chris
> > ASF Infra
> >
> >
> >
> >
> >
> > > On Sep 30, 2018, at 3:12 PM, Chris Thistlethwaite <
> > cthistlego@outlook.com> wrote:
> > >
> > > I've been working on H21 and H31. They both had a some docker issues
> > where the process was locked and I couldn't do anything with a few
> > containers that were over 8 days old.
> > >
> > > Thought we might be good this morning so I threw them back into
> > rotation. Guess not, so I'll take them back out.
> > >
> > > -Chris T.
> > > ________________________________
> > > From: Mike Jumper <mj...@apache.org>
> > > Sent: Sunday, September 30, 2018 6:03:51 PM
> > > To: builds@apache.org
> > > Subject: No space on H21 (Was: Fwd: Build failed in Jenkins:
> > guacamole-client-coverity #73)
> > >
> > > FYI - we're currently seeing disk space issues on H21.
> > >
> > > - Mike
> > >
> > >
> > > ---------- Forwarded message ---------
> > > From: Apache Jenkins Server <je...@builds.apache.org>
> > > Date: Sun, Sep 30, 2018, 14:37
> > > Subject: Build failed in Jenkins: guacamole-client-coverity #73
> > > To: <de...@guacamole.apache.org>
> > >
> > >
> > > See <
> > > https://builds.apache.org/job/guacamole-client-coverity/73/d
> > isplay/redirect>
> > >
> > > ------------------------------------------
> > > Started by user mjumper
> > > [EnvInject] - Loading node environment variables.
> > > Building remotely on H21 (ubuntu xenial) in workspace <
> > > https://builds.apache.org/job/guacamole-client-coverity/ws/>
> > > [WS-CLEANUP] Deleting project workspace...
> > > Cloning the remote Git repository
> > > Cloning repository
> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > >> git init <
> > > https://builds.apache.org/job/guacamole-client-coverity/ws/g
> > uacamole-client>
> > > # timeout=10
> > > Fetching upstream changes from
> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > >> git --version # timeout=10
> > >> git fetch --tags --progress
> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > > +refs/heads/*:refs/remotes/origin/*
> > > ERROR: Error cloning remote repo 'origin'
> > > hudson.plugins.git.GitException: Command "git fetch --tags --progress
> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > > +refs/heads/*:refs/remotes/origin/*" returned status code 128:
> > > stdout:
> > > stderr: remote: Counting objects: 62166, done.
> > > remote: Compressing objects: 0% (1/16096) remote:
> Compressing
> > > objects: 1% (161/16096) remote: Compressing objects: 2%
> > > (322/16096) remote: Compressing objects: 3% (483/16096)
> > > remote: Compressing objects: 4% (644/16096) remote:
> > > Compressing objects: 5% (805/16096) remote: Compressing
> > > objects: 6% (966/16096) remote: Compressing objects: 7%
> > > (1127/16096) remote: Compressing objects: 8% (1288/16096)
> > > remote: Compressing objects: 9% (1449/16096) remote:
> > > Compressing objects: 10% (1610/16096) remote: Compressing
> > > objects: 11% (1771/16096) remote: Compressing objects: 12%
> > > (1932/16096) remote: Compressing objects: 13% (2093/16096)
> > > remote: Compressing objects: 14% (2254/16096) remote:
> > > Compressing objects: 15% (2415/16096) remote: Compressing
> > > objects: 16% (2576/16096) remote: Compressing objects: 17%
> > > (2737/16096) remote: Compressing objects: 18% (2898/16096)
> > > remote: Compressing objects: 19% (3059/16096) remote:
> > > Compressing objects: 20% (3220/16096) remote: Compressing
> > > objects: 21% (3381/16096) remote: Compressing objects: 22%
> > > (3542/16096) remote: Compressing objects: 23% (3703/16096)
> > > remote: Compressing objects: 24% (3864/16096) remote:
> > > Compressing objects: 25% (4024/16096) remote: Compressing
> > > objects: 26% (4185/16096) remote: Compressing objects: 27%
> > > (4346/16096) remote: Compressing objects: 28% (4507/16096)
> > > remote: Compressing objects: 29% (4668/16096) remote:
> > > Compressing objects: 30% (4829/16096) remote: Compressing
> > > objects: 31% (4990/16096) remote: Compressing objects: 32%
> > > (5151/16096) remote: Compressing objects: 33% (5312/16096)
> > > remote: Compressing objects: 34% (5473/16096) remote:
> > > Compressing objects: 35% (5634/16096) remote: Compressing
> > > objects: 36% (5795/16096) remote: Compressing objects: 37%
> > > (5956/16096) remote: Compressing objects: 38% (6117/16096)
> > > remote: Compressing objects: 39% (6278/16096) remote:
> > > Compressing objects: 40% (6439/16096) remote: Compressing
> > > objects: 41% (6600/16096) remote: Compressing objects: 42%
> > > (6761/16096) remote: Compressing objects: 43% (6922/16096)
> > > remote: Compressing objects: 44% (7083/16096) remote:
> > > Compressing objects: 45% (7244/16096) remote: Compressing
> > > objects: 46% (7405/16096) remote: Compressing objects: 47%
> > > (7566/16096) remote: Compressing objects: 48% (7727/16096)
> > > remote: Compressing objects: 49% (7888/16096) remote:
> > > Compressing objects: 50% (8048/16096) remote: Compressing
> > > objects: 51% (8209/16096) remote: Compressing objects: 52%
> > > (8370/16096) remote: Compressing objects: 53% (8531/16096)
> > > remote: Compressing objects: 54% (8692/16096) remote:
> > > Compressing objects: 55% (8853/16096) remote: Compressing
> > > objects: 56% (9014/16096) remote: Compressing objects: 57%
> > > (9175/16096) remote: Compressing objects: 58% (9336/16096)
> > > remote: Compressing objects: 59% (9497/16096) remote:
> > > Compressing objects: 60% (9658/16096) remote: Compressing
> > > objects: 61% (9819/16096) remote: Compressing objects: 62%
> > > (9980/16096) remote: Compressing objects: 63% (10141/16096)
> > > remote: Compressing objects: 64% (10302/16096) remote:
> > > Compressing objects: 65% (10463/16096) remote: Compressing
> > > objects: 66% (10624/16096) remote: Compressing objects: 67%
> > > (10785/16096) remote: Compressing objects: 68% (10946/16096)
> > > remote: Compressing objects: 69% (11107/16096) remote:
> > > Compressing objects: 70% (11268/16096) remote: Compressing
> > > objects: 71% (11429/16096) remote: Compressing objects: 72%
> > > (11590/16096) remote: Compressing objects: 73% (11751/16096)
> > > remote: Compressing objects: 74% (11912/16096) remote:
> > > Compressing objects: 75% (12072/16096) remote: Compressing
> > > objects: 76% (12233/16096) remote: Compressing objects: 77%
> > > (12394/16096) remote: Compressing objects: 78% (12555/16096)
> > > remote: Compressing objects: 79% (12716/16096) remote:
> > > Compressing objects: 80% (12877/16096) remote: Compressing
> > > objects: 81% (13038/16096) remote: Compressing objects: 82%
> > > (13199/16096) remote: Compressing objects: 83% (13360/16096)
> > > remote: Compressing objects: 84% (13521/16096) remote:
> > > Compressing objects: 85% (13682/16096) remote: Compressing
> > > objects: 86% (13843/16096) remote: Compressing objects: 87%
> > > (14004/16096) remote: Compressing objects: 88% (14165/16096)
> > > remote: Compressing objects: 89% (14326/16096) remote:
> > > Compressing objects: 90% (14487/16096) remote: Compressing
> > > objects: 91% (14648/16096) remote: Compressing objects: 92%
> > > (14809/16096) remote: Compressing objects: 93% (14970/16096)
> > > remote: Compressing objects: 94% (15131/16096) remote:
> > > Compressing objects: 95% (15292/16096) remote: Compressing
> > > objects: 96% (15453/16096) remote: Compressing objects: 97%
> > > (15614/16096) remote: Compressing objects: 98% (15775/16096)
> > > remote: Compressing objects: 99% (15936/16096) remote:
> > > Compressing objects: 100% (16096/16096) remote: Compressing
> > > objects: 100% (16096/16096), done.
> > > Receiving objects: 0% (1/62166) Receiving objects: 1% (622/62166)
> > > Receiving objects: 2% (1244/62166) Receiving objects: 3%
> > > (1865/62166) Receiving objects: 4% (2487/62166) Receiving
> objects:
> > > 5% (3109/62166) Receiving objects: 6% (3730/62166) Receiving
> > > objects: 7% (4352/62166) Receiving objects: 8% (4974/62166)
> > > Receiving objects: 9% (5595/62166) Receiving objects: 10%
> > > (6217/62166) Receiving objects: 11% (6839/62166) Receiving
> objects:
> > > 12% (7460/62166) Receiving objects: 13% (8082/62166) Receiving
> > > objects: 14% (8704/62166) Receiving objects: 15% (9325/62166)
> > > Receiving objects: 16% (9947/62166) Receiving objects: 17%
> > > (10569/62166) Receiving objects: 18% (11190/62166) Receiving
> > objects:
> > > 19% (11812/62166) Receiving objects: 20% (12434/62166) Receiving
> > > objects: 21% (13055/62166) Receiving objects: 22% (13677/62166)
> > > fatal: write error: No space left on device
> > > fatal: index-pack failed
> > >
> > > at
> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandI
> > n(CliGitAPIImpl.java:2002)
> > > at
> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandW
> > ithCredentials(CliGitAPIImpl.java:1721)
> > > at
> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(
> > CliGitAPIImpl.java:72)
> > > at
> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(
> > CliGitAPIImpl.java:405)
> > > at
> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(
> > CliGitAPIImpl.java:614)
> > > at
> > > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
> > ionHandler$1.call(RemoteGitImpl.java:153)
> > > at
> > > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
> > ionHandler$1.call(RemoteGitImpl.java:146)
> > > at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> > > at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> > > at hudson.remoting.Request$2.run(Request.java:369)
> > > at
> > > hudson.remoting.InterceptingExecutorService$1.call(Intercept
> > ingExecutorService.java:72)
> > > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > > at
> > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> > Executor.java:1142)
> > > at
> > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> > lExecutor.java:617)
> > > at java.lang.Thread.run(Thread.java:748)
> > > Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
> > call
> > > to H21
> > > at
> > > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> > > at
> > > hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserR
> > equest.java:357)
> > > at hudson.remoting.Channel.call(Channel.java:955)
> > > at
> > > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
> > ionHandler.execute(RemoteGitImpl.java:146)
> > > at sun.reflect.GeneratedMethodAccessor918.invoke(Unknown
> > > Source)
> > > at
> > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
> > thodAccessorImpl.java:43)
> > > at java.lang.reflect.Method.invoke(Method.java:498)
> > > at
> > > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
> > ionHandler.invoke(RemoteGitImpl.java:132)
> > > at com.sun.proxy.$Proxy142.execute(Unknown Source)
> > > at
> > > hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1146)
> > > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> > > at hudson.scm.SCM.checkout(SCM.java:504)
> > > at
> > > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> > > at
> > > hudson.model.AbstractBuild$AbstractBuildExecution.defaultChe
> > ckout(AbstractBuild.java:574)
> > > at
> > > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> > > at
> > > hudson.model.AbstractBuild$AbstractBuildExecution.run(Abstra
> > ctBuild.java:499)
> > > at hudson.model.Run.execute(Run.java:1794)
> > > at hudson.model.FreeStyleBuild.ru
> > n(FreeStyleBuild.java:43)
> > > at
> > > hudson.model.ResourceController.execute(ResourceController.java:97)
> > > at hudson.model.Executor.run(Executor.java:429)
> > > ERROR: Error cloning remote repo 'origin'
> > > Retrying after 10 seconds
> > >> git rev-parse --is-inside-work-tree # timeout=10
> > > Fetching changes from the remote Git repository
> > >> git config remote.origin.url
> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git #
> > timeout=10
> > > Fetching upstream changes from
> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > >> git --version # timeout=10
> > >> git fetch --tags --progress
> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > > +refs/heads/*:refs/remotes/origin/*
> > > ERROR: Error fetching remote repo 'origin'
> > > hudson.plugins.git.GitException: Failed to fetch from
> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
> > > at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> > > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> > > at hudson.scm.SCM.checkout(SCM.java:504)
> > > at hudson.model.AbstractProject.checkout(AbstractProject.java:1
> > 208)
> > > at
> > > hudson.model.AbstractBuild$AbstractBuildExecution.defaultChe
> > ckout(AbstractBuild.java:574)
> > > at
> > > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> > > at
> > > hudson.model.AbstractBuild$AbstractBuildExecution.run(Abstra
> > ctBuild.java:499)
> > > at hudson.model.Run.execute(Run.java:1794)
> > > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> > > at
> > > hudson.model.ResourceController.execute(ResourceController.java:97)
> > > at hudson.model.Executor.run(Executor.java:429)
> > > Caused by: hudson.plugins.git.GitException: Command "git fetch --tags
> > > --progress
> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > > +refs/heads/*:refs/remotes/origin/*" returned status code 128:
> > > stdout:
> > > stderr: remote: Counting objects: 62166, done.
> > > remote: Compressing objects: 0% (1/16096) remote:
> Compressing
> > > objects: 1% (161/16096) remote: Compressing objects: 2%
> > > (322/16096) remote: Compressing objects: 3% (483/16096)
> > > remote: Compressing objects: 4% (644/16096) remote:
> > > Compressing objects: 5% (805/16096) remote: Compressing
> > > objects: 6% (966/16096) remote: Compressing objects: 7%
> > > (1127/16096) remote: Compressing objects: 8% (1288/16096)
> > > remote: Compressing objects: 9% (1449/16096) remote:
> > > Compressing objects: 10% (1610/16096) remote: Compressing
> > > objects: 11% (1771/16096) remote: Compressing objects: 12%
> > > (1932/16096) remote: Compressing objects: 13% (2093/16096)
> > > remote: Compressing objects: 14% (2254/16096) remote:
> > > Compressing objects: 15% (2415/16096) remote: Compressing
> > > objects: 16% (2576/16096) remote: Compressing objects: 17%
> > > (2737/16096) remote: Compressing objects: 18% (2898/16096)
> > > remote: Compressing objects: 19% (3059/16096) remote:
> > > Compressing objects: 20% (3220/16096) remote: Compressing
> > > objects: 21% (3381/16096) remote: Compressing objects: 22%
> > > (3542/16096) remote: Compressing objects: 23% (3703/16096)
> > > remote: Compressing objects: 24% (3864/16096) remote:
> > > Compressing objects: 25% (4024/16096) remote: Compressing
> > > objects: 26% (4185/16096) remote: Compressing objects: 27%
> > > (4346/16096) remote: Compressing objects: 28% (4507/16096)
> > > remote: Compressing objects: 29% (4668/16096) remote:
> > > Compressing objects: 30% (4829/16096) remote: Compressing
> > > objects: 31% (4990/16096) remote: Compressing objects: 32%
> > > (5151/16096) remote: Compressing objects: 33% (5312/16096)
> > > remote: Compressing objects: 34% (5473/16096) remote:
> > > Compressing objects: 35% (5634/16096) remote: Compressing
> > > objects: 36% (5795/16096) remote: Compressing objects: 37%
> > > (5956/16096) remote: Compressing objects: 38% (6117/16096)
> > > remote: Compressing objects: 39% (6278/16096) remote:
> > > Compressing objects: 40% (6439/16096) remote: Compressing
> > > objects: 41% (6600/16096) remote: Compressing objects: 42%
> > > (6761/16096) remote: Compressing objects: 43% (6922/16096)
> > > remote: Compressing objects: 44% (7083/16096) remote:
> > > Compressing objects: 45% (7244/16096) remote: Compressing
> > > objects: 46% (7405/16096) remote: Compressing objects: 47%
> > > (7566/16096) remote: Compressing objects: 48% (7727/16096)
> > > remote: Compressing objects: 49% (7888/16096) remote:
> > > Compressing objects: 50% (8048/16096) remote: Compressing
> > > objects: 51% (8209/16096) remote: Compressing objects: 52%
> > > (8370/16096) remote: Compressing objects: 53% (8531/16096)
> > > remote: Compressing objects: 54% (8692/16096) remote:
> > > Compressing objects: 55% (8853/16096) remote: Compressing
> > > objects: 56% (9014/16096) remote: Compressing objects: 57%
> > > (9175/16096) remote: Compressing objects: 58% (9336/16096)
> > > remote: Compressing objects: 59% (9497/16096) remote:
> > > Compressing objects: 60% (9658/16096) remote: Compressing
> > > objects: 61% (9819/16096) remote: Compressing objects: 62%
> > > (9980/16096) remote: Compressing objects: 63% (10141/16096)
> > > remote: Compressing objects: 64% (10302/16096) remote:
> > > Compressing objects: 65% (10463/16096) remote: Compressing
> > > objects: 66% (10624/16096) remote: Compressing objects: 67%
> > > (10785/16096) remote: Compressing objects: 68% (10946/16096)
> > > remote: Compressing objects: 69% (11107/16096) remote:
> > > Compressing objects: 70% (11268/16096) remote: Compressing
> > > objects: 71% (11429/16096) remote: Compressing objects: 72%
> > > (11590/16096) remote: Compressing objects: 73% (11751/16096)
> > > remote: Compressing objects: 74% (11912/16096) remote:
> > > Compressing objects: 75% (12072/16096) remote: Compressing
> > > objects: 76% (12233/16096) remote: Compressing objects: 77%
> > > (12394/16096) remote: Compressing objects: 78% (12555/16096)
> > > remote: Compressing objects: 79% (12716/16096) remote:
> > > Compressing objects: 80% (12877/16096) remote: Compressing
> > > objects: 81% (13038/16096) remote: Compressing objects: 82%
> > > (13199/16096) remote: Compressing objects: 83% (13360/16096)
> > > remote: Compressing objects: 84% (13521/16096) remote:
> > > Compressing objects: 85% (13682/16096) remote: Compressing
> > > objects: 86% (13843/16096) remote: Compressing objects: 87%
> > > (14004/16096) remote: Compressing objects: 88% (14165/16096)
> > > remote: Compressing objects: 89% (14326/16096) remote:
> > > Compressing objects: 90% (14487/16096) remote: Compressing
> > > objects: 91% (14648/16096) remote: Compressing objects: 92%
> > > (14809/16096) remote: Compressing objects: 93% (14970/16096)
> > > remote: Compressing objects: 94% (15131/16096) remote:
> > > Compressing objects: 95% (15292/16096) remote: Compressing
> > > objects: 96% (15453/16096) remote: Compressing objects: 97%
> > > (15614/16096) remote: Compressing objects: 98% (15775/16096)
> > > remote: Compressing objects: 99% (15936/16096) remote:
> > > Compressing objects: 100% (16096/16096) remote: Compressing
> > > objects: 100% (16096/16096), done.
> > > Receiving objects: 0% (1/62166) fatal: write error: No space left
> on
> > > device
> > > fatal: index-pack failed
> > >
> > > at
> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandI
> > n(CliGitAPIImpl.java:2002)
> > > at
> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandW
> > ithCredentials(CliGitAPIImpl.java:1721)
> > > at
> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(
> > CliGitAPIImpl.java:72)
> > > at
> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(
> > CliGitAPIImpl.java:405)
> > > at
> > > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
> > ionHandler$1.call(RemoteGitImpl.java:153)
> > > at
> > > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
> > ionHandler$1.call(RemoteGitImpl.java:146)
> > > at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> > > at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> > > at hudson.remoting.Request$2.run(Request.java:369)
> > > at
> > > hudson.remoting.InterceptingExecutorService$1.call(Intercept
> > ingExecutorService.java:72)
> > > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > > at
> > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> > Executor.java:1142)
> > > at
> > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> > lExecutor.java:617)
> > > at java.lang.Thread.run(Thread.java:748)
> > > Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
> > call
> > > to H21
> > > at
> > > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> > > at
> > > hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserR
> > equest.java:357)
> > > at hudson.remoting.Channel.call(Channel.java:955)
> > > at
> > > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
> > ionHandler.execute(RemoteGitImpl.java:146)
> > > at sun.reflect.GeneratedMethodAccessor918.invoke(Unknown
> > > Source)
> > > at
> > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
> > thodAccessorImpl.java:43)
> > > at java.lang.reflect.Method.invoke(Method.java:498)
> > > at
> > > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
> > ionHandler.invoke(RemoteGitImpl.java:132)
> > > at com.sun.proxy.$Proxy119.execute(Unknown Source)
> > > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:886)
> > > at
> > > hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> > > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> > > at hudson.scm.SCM.checkout(SCM.java:504)
> > > at
> > > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> > > at
> > > hudson.model.AbstractBuild$AbstractBuildExecution.defaultChe
> > ckout(AbstractBuild.java:574)
> > > at
> > > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> > > at
> > > hudson.model.AbstractBuild$AbstractBuildExecution.run(Abstra
> > ctBuild.java:499)
> > > at hudson.model.Run.execute(Run.java:1794)
> > > at hudson.model.FreeStyleBuild.ru
> > n(FreeStyleBuild.java:43)
> > > at
> > > hudson.model.ResourceController.execute(ResourceController.java:97)
> > > at hudson.model.Executor.run(Executor.java:429)
> > > ERROR: Error fetching remote repo 'origin'
> > > Retrying after 10 seconds
> > >> git rev-parse --is-inside-work-tree # timeout=10
> > > Fetching changes from the remote Git repository
> > >> git config remote.origin.url
> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git #
> > timeout=10
> > > ERROR: Error fetching remote repo 'origin'
> > > hudson.plugins.git.GitException: Failed to fetch from
> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
> > > at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> > > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> > > at hudson.scm.SCM.checkout(SCM.java:504)
> > > at hudson.model.AbstractProject.checkout(AbstractProject.java:1
> > 208)
> > > at
> > > hudson.model.AbstractBuild$AbstractBuildExecution.defaultChe
> > ckout(AbstractBuild.java:574)
> > > at
> > > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> > > at
> > > hudson.model.AbstractBuild$AbstractBuildExecution.run(Abstra
> > ctBuild.java:499)
> > > at hudson.model.Run.execute(Run.java:1794)
> > > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> > > at
> > > hudson.model.ResourceController.execute(ResourceController.java:97)
> > > at hudson.model.Executor.run(Executor.java:429)
> > > Caused by: hudson.plugins.git.GitException: Command "git config
> > > remote.origin.url
> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git" returned
> > > status code 4:
> > > stdout:
> > > stderr: error: failed to write new configuration file <
> > > https://builds.apache.org/job/guacamole-client-coverity/ws/g
> > uacamole-client/.git/config.lock
> > >>
> > >
> > > at
> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandI
> > n(CliGitAPIImpl.java:2002)
> > > at
> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandI
> > n(CliGitAPIImpl.java:1970)
> > > at
> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandI
> > n(CliGitAPIImpl.java:1966)
> > > at
> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(
> > CliGitAPIImpl.java:1597)
> > > at
> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(
> > CliGitAPIImpl.java:1609)
> > > at
> > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.setRemoteUrl(
> > CliGitAPIImpl.java:1243)
> > > at hudson.plugins.git.GitAPI.setRemoteUrl(GitAPI.java:160)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > at
> > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
> > ssorImpl.java:62)
> > > at
> > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
> > thodAccessorImpl.java:43)
> > > at java.lang.reflect.Method.invoke(Method.java:498)
> > > at
> > > hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(R
> > emoteInvocationHandler.java:929)
> > > at
> > > hudson.remoting.RemoteInvocationHandler$RPCRequest.call(Remo
> > teInvocationHandler.java:903)
> > > at
> > > hudson.remoting.RemoteInvocationHandler$RPCRequest.call(Remo
> > teInvocationHandler.java:855)
> > > at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> > > at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> > > at hudson.remoting.Request$2.run(Request.java:369)
> > > at
> > > hudson.remoting.InterceptingExecutorService$1.call(Intercept
> > ingExecutorService.java:72)
> > > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > > at
> > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> > Executor.java:1142)
> > > at
> > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> > lExecutor.java:617)
> > > at java.lang.Thread.run(Thread.java:748)
> > > Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
> > call
> > > to H21
> > > at
> > > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> > > at
> > > hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserR
> > equest.java:357)
> > > at hudson.remoting.Channel.call(Channel.java:955)
> > > at
> > > hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocat
> > ionHandler.java:283)
> > > at com.sun.proxy.$Proxy118.setRemoteUrl(Unknown Source)
> > > at
> > > org.jenkinsci.plugins.gitclient.RemoteGitImpl.setRemoteUrl(
> > RemoteGitImpl.java:295)
> > > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:876)
> > > at
> > > hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> > > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> > > at hudson.scm.SCM.checkout(SCM.java:504)
> > > at
> > > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> > > at
> > > hudson.model.AbstractBuild$AbstractBuildExecution.defaultChe
> > ckout(AbstractBuild.java:574)
> > > at
> > > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> > > at
> > > hudson.model.AbstractBuild$AbstractBuildExecution.run(Abstra
> > ctBuild.java:499)
> > > at hudson.model.Run.execute(Run.java:1794)
> > > at hudson.model.FreeStyleBuild.ru
> > n(FreeStyleBuild.java:43)
> > > at
> > > hudson.model.ResourceController.execute(ResourceController.java:97)
> > > at hudson.model.Executor.run(Executor.java:429)
> > > ERROR: Error fetching remote repo 'origin'
> >
> >
>
Re: No space on H21 (Was: Fwd: Build failed in Jenkins:
guacamole-client-coverity #73)
Posted by Mike Jumper <mj...@apache.org>.
Is there any way Jenkins could be configured to take nodes offline
automatically when free space is below a certain threshold? Granted
space-hungry builds and proper cleanup remain a problem, but at least there
wouldn't be a random chance of jobs failing due to being assigned to a full
node.
- Mike
On Sun, Sep 30, 2018 at 3:57 PM, Chris Lambertus <cm...@apache.org> wrote:
> The main culprit on H31 appears to be pulsar ws-cleanup jobs consuming
> almost 40% of the disk. (The problem on H21 was something docker-related.)
>
>
> This is the second project I have found (Brooklyn was the first) which is
> leaving ws-cleanup directories all over the workspace and filling up the
> disks. Each one of these was 500+ MB. Pulsar, please address ASAP.
>
>
>
>
> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:19
> pulsar_precommit_cpp@2_ws-cleanup_1535737112280
> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:18
> pulsar_precommit_cpp@2_ws-cleanup_1536174576053
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 06:04
> pulsar_precommit_cpp@2_ws-cleanup_1537292379178
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 22:45
> pulsar_precommit_cpp@2_ws-cleanup_1537397120862
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> pulsar_precommit_cpp@2_ws-cleanup_1537403378279
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 21 23:31
> pulsar_precommit_cpp@2_ws-cleanup_1537572701829
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 26 16:04
> pulsar_precommit_cpp@2_ws-cleanup_1537977879341
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 22:23
> pulsar_precommit_cpp@2_ws-cleanup_1538087035027
> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:14
> pulsar_precommit_cpp_ws-cleanup_1535699830230
> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:16
> pulsar_precommit_cpp_ws-cleanup_1535735511423
> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:13
> pulsar_precommit_cpp_ws-cleanup_1535786064397
> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:18
> pulsar_precommit_cpp_ws-cleanup_1536096963586
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 21 06:47
> pulsar_precommit_cpp_ws-cleanup_1536114938910
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 21 06:45
> pulsar_precommit_cpp_ws-cleanup_1536173165909
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:38
> pulsar_precommit_cpp_ws-cleanup_1536175584418
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:50
> pulsar_precommit_cpp_ws-cleanup_1536180707506
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:38
> pulsar_precommit_cpp_ws-cleanup_1536182927940
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:51
> pulsar_precommit_cpp_ws-cleanup_1536185193716
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:50
> pulsar_precommit_cpp_ws-cleanup_1536273709029
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 05:59
> pulsar_precommit_cpp_ws-cleanup_1536316964794
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:06
> pulsar_precommit_cpp_ws-cleanup_1536675650893
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 28 06:08
> pulsar_precommit_cpp_ws-cleanup_1536879616962
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 30 19:53
> pulsar_precommit_cpp_ws-cleanup_1536891056968
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 30 19:47
> pulsar_precommit_cpp_ws-cleanup_1536966466683
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 18 00:43
> pulsar_precommit_cpp_ws-cleanup_1537216387387
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 00:45
> pulsar_precommit_cpp_ws-cleanup_1537292309413
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 00:45
> pulsar_precommit_cpp_ws-cleanup_1537308526251
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> pulsar_precommit_cpp_ws-cleanup_1537331780420
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> pulsar_precommit_cpp_ws-cleanup_1537379886715
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
> pulsar_precommit_cpp_ws-cleanup_1537386197893
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
> pulsar_precommit_cpp_ws-cleanup_1537397073464
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
> pulsar_precommit_cpp_ws-cleanup_1537403247575
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:43
> pulsar_precommit_cpp_ws-cleanup_1537413954888
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
> pulsar_precommit_cpp_ws-cleanup_1537417322151
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
> pulsar_precommit_cpp_ws-cleanup_1537421381129
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:44
> pulsar_precommit_cpp_ws-cleanup_1537469348852
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
> pulsar_precommit_cpp_ws-cleanup_1537489399468
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 22 00:44
> pulsar_precommit_cpp_ws-cleanup_1537566528660
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 23:12
> pulsar_precommit_cpp_ws-cleanup_1537571549978
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 22 00:44
> pulsar_precommit_cpp_ws-cleanup_1537573605258
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 00:43
> pulsar_precommit_cpp_ws-cleanup_1537855939828
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 11:18
> pulsar_precommit_cpp_ws-cleanup_1537874305575
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 12:41
> pulsar_precommit_cpp_ws-cleanup_1537879296622
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 01:39
> pulsar_precommit_cpp_ws-cleanup_1537925955967
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 26 13:17
> pulsar_precommit_cpp_ws-cleanup_1537967826838
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 15:55
> pulsar_precommit_cpp_ws-cleanup_1537977326736
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 17:34
> pulsar_precommit_cpp_ws-cleanup_1537983268177
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 22:04
> pulsar_precommit_cpp_ws-cleanup_1538085846793
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 28 00:20
> pulsar_precommit_cpp_ws-cleanup_1538094024821
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 28 17:42
> pulsar_precommit_cpp_ws-cleanup_1538156543343
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:02
> pulsar_precommit_integrationtests_ws-cleanup_1536298125841
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:00
> pulsar_precommit_integrationtests_ws-cleanup_1536968128660
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 18 00:43
> pulsar_precommit_integrationtests_ws-cleanup_1537218541598
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> pulsar_precommit_integrationtests_ws-cleanup_1537331703324
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 22 00:43
> pulsar_precommit_integrationtests_ws-cleanup_1537553748859
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 00:43
> pulsar_precommit_integrationtests_ws-cleanup_1537730402937
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 07:48
> pulsar_precommit_integrationtests_ws-cleanup_1537775336586
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 23:30
> pulsar_precommit_integrationtests_ws-cleanup_1537831851711
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 23:08
> pulsar_precommit_integrationtests_ws-cleanup_1537916928485
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 15:21
> pulsar_precommit_integrationtests_ws-cleanup_1538061702243
>
>
> -Chris
> ASF Infra
>
>
>
>
>
> > On Sep 30, 2018, at 3:12 PM, Chris Thistlethwaite <
> cthistlego@outlook.com> wrote:
> >
> > I've been working on H21 and H31. They both had a some docker issues
> where the process was locked and I couldn't do anything with a few
> containers that were over 8 days old.
> >
> > Thought we might be good this morning so I threw them back into
> rotation. Guess not, so I'll take them back out.
> >
> > -Chris T.
> > ________________________________
> > From: Mike Jumper <mj...@apache.org>
> > Sent: Sunday, September 30, 2018 6:03:51 PM
> > To: builds@apache.org
> > Subject: No space on H21 (Was: Fwd: Build failed in Jenkins:
> guacamole-client-coverity #73)
> >
> > FYI - we're currently seeing disk space issues on H21.
> >
> > - Mike
> >
> >
> > ---------- Forwarded message ---------
> > From: Apache Jenkins Server <je...@builds.apache.org>
> > Date: Sun, Sep 30, 2018, 14:37
> > Subject: Build failed in Jenkins: guacamole-client-coverity #73
> > To: <de...@guacamole.apache.org>
> >
> >
> > See <
> > https://builds.apache.org/job/guacamole-client-coverity/73/d
> isplay/redirect>
> >
> > ------------------------------------------
> > Started by user mjumper
> > [EnvInject] - Loading node environment variables.
> > Building remotely on H21 (ubuntu xenial) in workspace <
> > https://builds.apache.org/job/guacamole-client-coverity/ws/>
> > [WS-CLEANUP] Deleting project workspace...
> > Cloning the remote Git repository
> > Cloning repository
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> >> git init <
> > https://builds.apache.org/job/guacamole-client-coverity/ws/g
> uacamole-client>
> > # timeout=10
> > Fetching upstream changes from
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> >> git --version # timeout=10
> >> git fetch --tags --progress
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > +refs/heads/*:refs/remotes/origin/*
> > ERROR: Error cloning remote repo 'origin'
> > hudson.plugins.git.GitException: Command "git fetch --tags --progress
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > +refs/heads/*:refs/remotes/origin/*" returned status code 128:
> > stdout:
> > stderr: remote: Counting objects: 62166, done.
> > remote: Compressing objects: 0% (1/16096) remote: Compressing
> > objects: 1% (161/16096) remote: Compressing objects: 2%
> > (322/16096) remote: Compressing objects: 3% (483/16096)
> > remote: Compressing objects: 4% (644/16096) remote:
> > Compressing objects: 5% (805/16096) remote: Compressing
> > objects: 6% (966/16096) remote: Compressing objects: 7%
> > (1127/16096) remote: Compressing objects: 8% (1288/16096)
> > remote: Compressing objects: 9% (1449/16096) remote:
> > Compressing objects: 10% (1610/16096) remote: Compressing
> > objects: 11% (1771/16096) remote: Compressing objects: 12%
> > (1932/16096) remote: Compressing objects: 13% (2093/16096)
> > remote: Compressing objects: 14% (2254/16096) remote:
> > Compressing objects: 15% (2415/16096) remote: Compressing
> > objects: 16% (2576/16096) remote: Compressing objects: 17%
> > (2737/16096) remote: Compressing objects: 18% (2898/16096)
> > remote: Compressing objects: 19% (3059/16096) remote:
> > Compressing objects: 20% (3220/16096) remote: Compressing
> > objects: 21% (3381/16096) remote: Compressing objects: 22%
> > (3542/16096) remote: Compressing objects: 23% (3703/16096)
> > remote: Compressing objects: 24% (3864/16096) remote:
> > Compressing objects: 25% (4024/16096) remote: Compressing
> > objects: 26% (4185/16096) remote: Compressing objects: 27%
> > (4346/16096) remote: Compressing objects: 28% (4507/16096)
> > remote: Compressing objects: 29% (4668/16096) remote:
> > Compressing objects: 30% (4829/16096) remote: Compressing
> > objects: 31% (4990/16096) remote: Compressing objects: 32%
> > (5151/16096) remote: Compressing objects: 33% (5312/16096)
> > remote: Compressing objects: 34% (5473/16096) remote:
> > Compressing objects: 35% (5634/16096) remote: Compressing
> > objects: 36% (5795/16096) remote: Compressing objects: 37%
> > (5956/16096) remote: Compressing objects: 38% (6117/16096)
> > remote: Compressing objects: 39% (6278/16096) remote:
> > Compressing objects: 40% (6439/16096) remote: Compressing
> > objects: 41% (6600/16096) remote: Compressing objects: 42%
> > (6761/16096) remote: Compressing objects: 43% (6922/16096)
> > remote: Compressing objects: 44% (7083/16096) remote:
> > Compressing objects: 45% (7244/16096) remote: Compressing
> > objects: 46% (7405/16096) remote: Compressing objects: 47%
> > (7566/16096) remote: Compressing objects: 48% (7727/16096)
> > remote: Compressing objects: 49% (7888/16096) remote:
> > Compressing objects: 50% (8048/16096) remote: Compressing
> > objects: 51% (8209/16096) remote: Compressing objects: 52%
> > (8370/16096) remote: Compressing objects: 53% (8531/16096)
> > remote: Compressing objects: 54% (8692/16096) remote:
> > Compressing objects: 55% (8853/16096) remote: Compressing
> > objects: 56% (9014/16096) remote: Compressing objects: 57%
> > (9175/16096) remote: Compressing objects: 58% (9336/16096)
> > remote: Compressing objects: 59% (9497/16096) remote:
> > Compressing objects: 60% (9658/16096) remote: Compressing
> > objects: 61% (9819/16096) remote: Compressing objects: 62%
> > (9980/16096) remote: Compressing objects: 63% (10141/16096)
> > remote: Compressing objects: 64% (10302/16096) remote:
> > Compressing objects: 65% (10463/16096) remote: Compressing
> > objects: 66% (10624/16096) remote: Compressing objects: 67%
> > (10785/16096) remote: Compressing objects: 68% (10946/16096)
> > remote: Compressing objects: 69% (11107/16096) remote:
> > Compressing objects: 70% (11268/16096) remote: Compressing
> > objects: 71% (11429/16096) remote: Compressing objects: 72%
> > (11590/16096) remote: Compressing objects: 73% (11751/16096)
> > remote: Compressing objects: 74% (11912/16096) remote:
> > Compressing objects: 75% (12072/16096) remote: Compressing
> > objects: 76% (12233/16096) remote: Compressing objects: 77%
> > (12394/16096) remote: Compressing objects: 78% (12555/16096)
> > remote: Compressing objects: 79% (12716/16096) remote:
> > Compressing objects: 80% (12877/16096) remote: Compressing
> > objects: 81% (13038/16096) remote: Compressing objects: 82%
> > (13199/16096) remote: Compressing objects: 83% (13360/16096)
> > remote: Compressing objects: 84% (13521/16096) remote:
> > Compressing objects: 85% (13682/16096) remote: Compressing
> > objects: 86% (13843/16096) remote: Compressing objects: 87%
> > (14004/16096) remote: Compressing objects: 88% (14165/16096)
> > remote: Compressing objects: 89% (14326/16096) remote:
> > Compressing objects: 90% (14487/16096) remote: Compressing
> > objects: 91% (14648/16096) remote: Compressing objects: 92%
> > (14809/16096) remote: Compressing objects: 93% (14970/16096)
> > remote: Compressing objects: 94% (15131/16096) remote:
> > Compressing objects: 95% (15292/16096) remote: Compressing
> > objects: 96% (15453/16096) remote: Compressing objects: 97%
> > (15614/16096) remote: Compressing objects: 98% (15775/16096)
> > remote: Compressing objects: 99% (15936/16096) remote:
> > Compressing objects: 100% (16096/16096) remote: Compressing
> > objects: 100% (16096/16096), done.
> > Receiving objects: 0% (1/62166) Receiving objects: 1% (622/62166)
> > Receiving objects: 2% (1244/62166) Receiving objects: 3%
> > (1865/62166) Receiving objects: 4% (2487/62166) Receiving objects:
> > 5% (3109/62166) Receiving objects: 6% (3730/62166) Receiving
> > objects: 7% (4352/62166) Receiving objects: 8% (4974/62166)
> > Receiving objects: 9% (5595/62166) Receiving objects: 10%
> > (6217/62166) Receiving objects: 11% (6839/62166) Receiving objects:
> > 12% (7460/62166) Receiving objects: 13% (8082/62166) Receiving
> > objects: 14% (8704/62166) Receiving objects: 15% (9325/62166)
> > Receiving objects: 16% (9947/62166) Receiving objects: 17%
> > (10569/62166) Receiving objects: 18% (11190/62166) Receiving
> objects:
> > 19% (11812/62166) Receiving objects: 20% (12434/62166) Receiving
> > objects: 21% (13055/62166) Receiving objects: 22% (13677/62166)
> > fatal: write error: No space left on device
> > fatal: index-pack failed
> >
> > at
> > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandI
> n(CliGitAPIImpl.java:2002)
> > at
> > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandW
> ithCredentials(CliGitAPIImpl.java:1721)
> > at
> > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(
> CliGitAPIImpl.java:72)
> > at
> > org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(
> CliGitAPIImpl.java:405)
> > at
> > org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(
> CliGitAPIImpl.java:614)
> > at
> > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
> ionHandler$1.call(RemoteGitImpl.java:153)
> > at
> > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
> ionHandler$1.call(RemoteGitImpl.java:146)
> > at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> > at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> > at hudson.remoting.Request$2.run(Request.java:369)
> > at
> > hudson.remoting.InterceptingExecutorService$1.call(Intercept
> ingExecutorService.java:72)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> Executor.java:1142)
> > at
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> lExecutor.java:617)
> > at java.lang.Thread.run(Thread.java:748)
> > Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
> call
> > to H21
> > at
> > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> > at
> > hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserR
> equest.java:357)
> > at hudson.remoting.Channel.call(Channel.java:955)
> > at
> > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
> ionHandler.execute(RemoteGitImpl.java:146)
> > at sun.reflect.GeneratedMethodAccessor918.invoke(Unknown
> > Source)
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
> thodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:498)
> > at
> > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
> ionHandler.invoke(RemoteGitImpl.java:132)
> > at com.sun.proxy.$Proxy142.execute(Unknown Source)
> > at
> > hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1146)
> > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> > at hudson.scm.SCM.checkout(SCM.java:504)
> > at
> > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> > at
> > hudson.model.AbstractBuild$AbstractBuildExecution.defaultChe
> ckout(AbstractBuild.java:574)
> > at
> > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> > at
> > hudson.model.AbstractBuild$AbstractBuildExecution.run(Abstra
> ctBuild.java:499)
> > at hudson.model.Run.execute(Run.java:1794)
> > at hudson.model.FreeStyleBuild.ru
> n(FreeStyleBuild.java:43)
> > at
> > hudson.model.ResourceController.execute(ResourceController.java:97)
> > at hudson.model.Executor.run(Executor.java:429)
> > ERROR: Error cloning remote repo 'origin'
> > Retrying after 10 seconds
> >> git rev-parse --is-inside-work-tree # timeout=10
> > Fetching changes from the remote Git repository
> >> git config remote.origin.url
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git #
> timeout=10
> > Fetching upstream changes from
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> >> git --version # timeout=10
> >> git fetch --tags --progress
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > +refs/heads/*:refs/remotes/origin/*
> > ERROR: Error fetching remote repo 'origin'
> > hudson.plugins.git.GitException: Failed to fetch from
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
> > at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> > at hudson.scm.SCM.checkout(SCM.java:504)
> > at hudson.model.AbstractProject.checkout(AbstractProject.java:1
> 208)
> > at
> > hudson.model.AbstractBuild$AbstractBuildExecution.defaultChe
> ckout(AbstractBuild.java:574)
> > at
> > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> > at
> > hudson.model.AbstractBuild$AbstractBuildExecution.run(Abstra
> ctBuild.java:499)
> > at hudson.model.Run.execute(Run.java:1794)
> > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> > at
> > hudson.model.ResourceController.execute(ResourceController.java:97)
> > at hudson.model.Executor.run(Executor.java:429)
> > Caused by: hudson.plugins.git.GitException: Command "git fetch --tags
> > --progress https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > +refs/heads/*:refs/remotes/origin/*" returned status code 128:
> > stdout:
> > stderr: remote: Counting objects: 62166, done.
> > remote: Compressing objects: 0% (1/16096) remote: Compressing
> > objects: 1% (161/16096) remote: Compressing objects: 2%
> > (322/16096) remote: Compressing objects: 3% (483/16096)
> > remote: Compressing objects: 4% (644/16096) remote:
> > Compressing objects: 5% (805/16096) remote: Compressing
> > objects: 6% (966/16096) remote: Compressing objects: 7%
> > (1127/16096) remote: Compressing objects: 8% (1288/16096)
> > remote: Compressing objects: 9% (1449/16096) remote:
> > Compressing objects: 10% (1610/16096) remote: Compressing
> > objects: 11% (1771/16096) remote: Compressing objects: 12%
> > (1932/16096) remote: Compressing objects: 13% (2093/16096)
> > remote: Compressing objects: 14% (2254/16096) remote:
> > Compressing objects: 15% (2415/16096) remote: Compressing
> > objects: 16% (2576/16096) remote: Compressing objects: 17%
> > (2737/16096) remote: Compressing objects: 18% (2898/16096)
> > remote: Compressing objects: 19% (3059/16096) remote:
> > Compressing objects: 20% (3220/16096) remote: Compressing
> > objects: 21% (3381/16096) remote: Compressing objects: 22%
> > (3542/16096) remote: Compressing objects: 23% (3703/16096)
> > remote: Compressing objects: 24% (3864/16096) remote:
> > Compressing objects: 25% (4024/16096) remote: Compressing
> > objects: 26% (4185/16096) remote: Compressing objects: 27%
> > (4346/16096) remote: Compressing objects: 28% (4507/16096)
> > remote: Compressing objects: 29% (4668/16096) remote:
> > Compressing objects: 30% (4829/16096) remote: Compressing
> > objects: 31% (4990/16096) remote: Compressing objects: 32%
> > (5151/16096) remote: Compressing objects: 33% (5312/16096)
> > remote: Compressing objects: 34% (5473/16096) remote:
> > Compressing objects: 35% (5634/16096) remote: Compressing
> > objects: 36% (5795/16096) remote: Compressing objects: 37%
> > (5956/16096) remote: Compressing objects: 38% (6117/16096)
> > remote: Compressing objects: 39% (6278/16096) remote:
> > Compressing objects: 40% (6439/16096) remote: Compressing
> > objects: 41% (6600/16096) remote: Compressing objects: 42%
> > (6761/16096) remote: Compressing objects: 43% (6922/16096)
> > remote: Compressing objects: 44% (7083/16096) remote:
> > Compressing objects: 45% (7244/16096) remote: Compressing
> > objects: 46% (7405/16096) remote: Compressing objects: 47%
> > (7566/16096) remote: Compressing objects: 48% (7727/16096)
> > remote: Compressing objects: 49% (7888/16096) remote:
> > Compressing objects: 50% (8048/16096) remote: Compressing
> > objects: 51% (8209/16096) remote: Compressing objects: 52%
> > (8370/16096) remote: Compressing objects: 53% (8531/16096)
> > remote: Compressing objects: 54% (8692/16096) remote:
> > Compressing objects: 55% (8853/16096) remote: Compressing
> > objects: 56% (9014/16096) remote: Compressing objects: 57%
> > (9175/16096) remote: Compressing objects: 58% (9336/16096)
> > remote: Compressing objects: 59% (9497/16096) remote:
> > Compressing objects: 60% (9658/16096) remote: Compressing
> > objects: 61% (9819/16096) remote: Compressing objects: 62%
> > (9980/16096) remote: Compressing objects: 63% (10141/16096)
> > remote: Compressing objects: 64% (10302/16096) remote:
> > Compressing objects: 65% (10463/16096) remote: Compressing
> > objects: 66% (10624/16096) remote: Compressing objects: 67%
> > (10785/16096) remote: Compressing objects: 68% (10946/16096)
> > remote: Compressing objects: 69% (11107/16096) remote:
> > Compressing objects: 70% (11268/16096) remote: Compressing
> > objects: 71% (11429/16096) remote: Compressing objects: 72%
> > (11590/16096) remote: Compressing objects: 73% (11751/16096)
> > remote: Compressing objects: 74% (11912/16096) remote:
> > Compressing objects: 75% (12072/16096) remote: Compressing
> > objects: 76% (12233/16096) remote: Compressing objects: 77%
> > (12394/16096) remote: Compressing objects: 78% (12555/16096)
> > remote: Compressing objects: 79% (12716/16096) remote:
> > Compressing objects: 80% (12877/16096) remote: Compressing
> > objects: 81% (13038/16096) remote: Compressing objects: 82%
> > (13199/16096) remote: Compressing objects: 83% (13360/16096)
> > remote: Compressing objects: 84% (13521/16096) remote:
> > Compressing objects: 85% (13682/16096) remote: Compressing
> > objects: 86% (13843/16096) remote: Compressing objects: 87%
> > (14004/16096) remote: Compressing objects: 88% (14165/16096)
> > remote: Compressing objects: 89% (14326/16096) remote:
> > Compressing objects: 90% (14487/16096) remote: Compressing
> > objects: 91% (14648/16096) remote: Compressing objects: 92%
> > (14809/16096) remote: Compressing objects: 93% (14970/16096)
> > remote: Compressing objects: 94% (15131/16096) remote:
> > Compressing objects: 95% (15292/16096) remote: Compressing
> > objects: 96% (15453/16096) remote: Compressing objects: 97%
> > (15614/16096) remote: Compressing objects: 98% (15775/16096)
> > remote: Compressing objects: 99% (15936/16096) remote:
> > Compressing objects: 100% (16096/16096) remote: Compressing
> > objects: 100% (16096/16096), done.
> > Receiving objects: 0% (1/62166) fatal: write error: No space left on
> > device
> > fatal: index-pack failed
> >
> > at
> > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandI
> n(CliGitAPIImpl.java:2002)
> > at
> > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandW
> ithCredentials(CliGitAPIImpl.java:1721)
> > at
> > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(
> CliGitAPIImpl.java:72)
> > at
> > org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(
> CliGitAPIImpl.java:405)
> > at
> > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
> ionHandler$1.call(RemoteGitImpl.java:153)
> > at
> > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
> ionHandler$1.call(RemoteGitImpl.java:146)
> > at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> > at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> > at hudson.remoting.Request$2.run(Request.java:369)
> > at
> > hudson.remoting.InterceptingExecutorService$1.call(Intercept
> ingExecutorService.java:72)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> Executor.java:1142)
> > at
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> lExecutor.java:617)
> > at java.lang.Thread.run(Thread.java:748)
> > Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
> call
> > to H21
> > at
> > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> > at
> > hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserR
> equest.java:357)
> > at hudson.remoting.Channel.call(Channel.java:955)
> > at
> > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
> ionHandler.execute(RemoteGitImpl.java:146)
> > at sun.reflect.GeneratedMethodAccessor918.invoke(Unknown
> > Source)
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
> thodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:498)
> > at
> > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocat
> ionHandler.invoke(RemoteGitImpl.java:132)
> > at com.sun.proxy.$Proxy119.execute(Unknown Source)
> > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:886)
> > at
> > hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> > at hudson.scm.SCM.checkout(SCM.java:504)
> > at
> > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> > at
> > hudson.model.AbstractBuild$AbstractBuildExecution.defaultChe
> ckout(AbstractBuild.java:574)
> > at
> > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> > at
> > hudson.model.AbstractBuild$AbstractBuildExecution.run(Abstra
> ctBuild.java:499)
> > at hudson.model.Run.execute(Run.java:1794)
> > at hudson.model.FreeStyleBuild.ru
> n(FreeStyleBuild.java:43)
> > at
> > hudson.model.ResourceController.execute(ResourceController.java:97)
> > at hudson.model.Executor.run(Executor.java:429)
> > ERROR: Error fetching remote repo 'origin'
> > Retrying after 10 seconds
> >> git rev-parse --is-inside-work-tree # timeout=10
> > Fetching changes from the remote Git repository
> >> git config remote.origin.url
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git #
> timeout=10
> > ERROR: Error fetching remote repo 'origin'
> > hudson.plugins.git.GitException: Failed to fetch from
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
> > at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> > at hudson.scm.SCM.checkout(SCM.java:504)
> > at hudson.model.AbstractProject.checkout(AbstractProject.java:1
> 208)
> > at
> > hudson.model.AbstractBuild$AbstractBuildExecution.defaultChe
> ckout(AbstractBuild.java:574)
> > at
> > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> > at
> > hudson.model.AbstractBuild$AbstractBuildExecution.run(Abstra
> ctBuild.java:499)
> > at hudson.model.Run.execute(Run.java:1794)
> > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> > at
> > hudson.model.ResourceController.execute(ResourceController.java:97)
> > at hudson.model.Executor.run(Executor.java:429)
> > Caused by: hudson.plugins.git.GitException: Command "git config
> > remote.origin.url
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git" returned
> > status code 4:
> > stdout:
> > stderr: error: failed to write new configuration file <
> > https://builds.apache.org/job/guacamole-client-coverity/ws/g
> uacamole-client/.git/config.lock
> >>
> >
> > at
> > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandI
> n(CliGitAPIImpl.java:2002)
> > at
> > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandI
> n(CliGitAPIImpl.java:1970)
> > at
> > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandI
> n(CliGitAPIImpl.java:1966)
> > at
> > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(
> CliGitAPIImpl.java:1597)
> > at
> > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(
> CliGitAPIImpl.java:1609)
> > at
> > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.setRemoteUrl(
> CliGitAPIImpl.java:1243)
> > at hudson.plugins.git.GitAPI.setRemoteUrl(GitAPI.java:160)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
> ssorImpl.java:62)
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
> thodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:498)
> > at
> > hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(R
> emoteInvocationHandler.java:929)
> > at
> > hudson.remoting.RemoteInvocationHandler$RPCRequest.call(Remo
> teInvocationHandler.java:903)
> > at
> > hudson.remoting.RemoteInvocationHandler$RPCRequest.call(Remo
> teInvocationHandler.java:855)
> > at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> > at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> > at hudson.remoting.Request$2.run(Request.java:369)
> > at
> > hudson.remoting.InterceptingExecutorService$1.call(Intercept
> ingExecutorService.java:72)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> Executor.java:1142)
> > at
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> lExecutor.java:617)
> > at java.lang.Thread.run(Thread.java:748)
> > Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
> call
> > to H21
> > at
> > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> > at
> > hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserR
> equest.java:357)
> > at hudson.remoting.Channel.call(Channel.java:955)
> > at
> > hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocat
> ionHandler.java:283)
> > at com.sun.proxy.$Proxy118.setRemoteUrl(Unknown Source)
> > at
> > org.jenkinsci.plugins.gitclient.RemoteGitImpl.setRemoteUrl(
> RemoteGitImpl.java:295)
> > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:876)
> > at
> > hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> > at hudson.scm.SCM.checkout(SCM.java:504)
> > at
> > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> > at
> > hudson.model.AbstractBuild$AbstractBuildExecution.defaultChe
> ckout(AbstractBuild.java:574)
> > at
> > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> > at
> > hudson.model.AbstractBuild$AbstractBuildExecution.run(Abstra
> ctBuild.java:499)
> > at hudson.model.Run.execute(Run.java:1794)
> > at hudson.model.FreeStyleBuild.ru
> n(FreeStyleBuild.java:43)
> > at
> > hudson.model.ResourceController.execute(ResourceController.java:97)
> > at hudson.model.Executor.run(Executor.java:429)
> > ERROR: Error fetching remote repo 'origin'
>
>
Re: No space on H21 (Was: Fwd: Build failed in Jenkins:
guacamole-client-coverity #73)
Posted by Chris Lambertus <cm...@apache.org>.
Resolved by https://github.com/apache/infrastructure-puppet/pull/1181 <https://github.com/apache/infrastructure-puppet/pull/1181>
> On Oct 2, 2018, at 11:22 PM, Maxim Solodovnik <so...@gmail.com> wrote:
>
> It seems there is no space on ubuntu-eu2
> <https://builds.apache.org/computer/ubuntu-eu2>
> (git clone fails :((( )
>
> On Tue, 2 Oct 2018 at 02:29, Chris Lambertus <cm...@apache.org> wrote:
>
>> Well, I had thought it was something specific to the build, since it only
>> happens for a couple projects, but upon further investigation, there
>> appears to be an open issue related to this:
>>
>> https://issues.jenkins-ci.org/browse/JENKINS-53579
>>
>> I will need to look into this further. In the meantime, Infra will need to
>> implement a cronjob that purges the errant ws-cleanup folders.
>>
>> -Chris
>>
>>
>>
>>
>> On Oct 1, 2018, at 11:30 AM, Matteo Merli <ma...@gmail.com> wrote:
>>
>> Chris, are these `ws-cleanup` jobs something that is created internally by
>> Jenkins? The job we have is `pulsar_precommit_cpp`.
>>
>> Is there any setting in Jenkins that could explain that these files are not
>> removed?
>>
>> On https://builds.apache.org/job/pulsar_precommit_cpp/configure , we
>> already have "Days to keep builds == 1".
>> For "Days to keep artifacts" is left blank. Should that be set to 1 day
>> too?
>>
>> Any other hint at things we can try to ensure builds are cleaned up
>> promptly?
>>
>> Thanks,
>> Matteo
>>
>> On Sun, Sep 30, 2018 at 3:57 PM Chris Lambertus <cm...@apache.org> wrote:
>>
>> The main culprit on H31 appears to be pulsar ws-cleanup jobs consuming
>> almost 40% of the disk. (The problem on H21 was something docker-related.)
>>
>>
>> This is the second project I have found (Brooklyn was the first) which is
>> leaving ws-cleanup directories all over the workspace and filling up the
>> disks. Each one of these was 500+ MB. Pulsar, please address ASAP.
>>
>>
>>
>>
>> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:19
>> pulsar_precommit_cpp@2_ws-cleanup_1535737112280
>> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:18
>> pulsar_precommit_cpp@2_ws-cleanup_1536174576053
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 06:04
>> pulsar_precommit_cpp@2_ws-cleanup_1537292379178
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 22:45
>> pulsar_precommit_cpp@2_ws-cleanup_1537397120862
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
>> pulsar_precommit_cpp@2_ws-cleanup_1537403378279
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 21 23:31
>> pulsar_precommit_cpp@2_ws-cleanup_1537572701829
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 26 16:04
>> pulsar_precommit_cpp@2_ws-cleanup_1537977879341
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 22:23
>> pulsar_precommit_cpp@2_ws-cleanup_1538087035027
>> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:14
>> pulsar_precommit_cpp_ws-cleanup_1535699830230
>> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:16
>> pulsar_precommit_cpp_ws-cleanup_1535735511423
>> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:13
>> pulsar_precommit_cpp_ws-cleanup_1535786064397
>> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:18
>> pulsar_precommit_cpp_ws-cleanup_1536096963586
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 21 06:47
>> pulsar_precommit_cpp_ws-cleanup_1536114938910
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 21 06:45
>> pulsar_precommit_cpp_ws-cleanup_1536173165909
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:38
>> pulsar_precommit_cpp_ws-cleanup_1536175584418
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:50
>> pulsar_precommit_cpp_ws-cleanup_1536180707506
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:38
>> pulsar_precommit_cpp_ws-cleanup_1536182927940
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:51
>> pulsar_precommit_cpp_ws-cleanup_1536185193716
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:50
>> pulsar_precommit_cpp_ws-cleanup_1536273709029
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 05:59
>> pulsar_precommit_cpp_ws-cleanup_1536316964794
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:06
>> pulsar_precommit_cpp_ws-cleanup_1536675650893
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 28 06:08
>> pulsar_precommit_cpp_ws-cleanup_1536879616962
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 30 19:53
>> pulsar_precommit_cpp_ws-cleanup_1536891056968
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 30 19:47
>> pulsar_precommit_cpp_ws-cleanup_1536966466683
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 18 00:43
>> pulsar_precommit_cpp_ws-cleanup_1537216387387
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 00:45
>> pulsar_precommit_cpp_ws-cleanup_1537292309413
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 00:45
>> pulsar_precommit_cpp_ws-cleanup_1537308526251
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
>> pulsar_precommit_cpp_ws-cleanup_1537331780420
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
>> pulsar_precommit_cpp_ws-cleanup_1537379886715
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
>> pulsar_precommit_cpp_ws-cleanup_1537386197893
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
>> pulsar_precommit_cpp_ws-cleanup_1537397073464
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
>> pulsar_precommit_cpp_ws-cleanup_1537403247575
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:43
>> pulsar_precommit_cpp_ws-cleanup_1537413954888
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
>> pulsar_precommit_cpp_ws-cleanup_1537417322151
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
>> pulsar_precommit_cpp_ws-cleanup_1537421381129
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:44
>> pulsar_precommit_cpp_ws-cleanup_1537469348852
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
>> pulsar_precommit_cpp_ws-cleanup_1537489399468
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 22 00:44
>> pulsar_precommit_cpp_ws-cleanup_1537566528660
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 23:12
>> pulsar_precommit_cpp_ws-cleanup_1537571549978
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 22 00:44
>> pulsar_precommit_cpp_ws-cleanup_1537573605258
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 00:43
>> pulsar_precommit_cpp_ws-cleanup_1537855939828
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 11:18
>> pulsar_precommit_cpp_ws-cleanup_1537874305575
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 12:41
>> pulsar_precommit_cpp_ws-cleanup_1537879296622
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 01:39
>> pulsar_precommit_cpp_ws-cleanup_1537925955967
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 26 13:17
>> pulsar_precommit_cpp_ws-cleanup_1537967826838
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 15:55
>> pulsar_precommit_cpp_ws-cleanup_1537977326736
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 17:34
>> pulsar_precommit_cpp_ws-cleanup_1537983268177
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 22:04
>> pulsar_precommit_cpp_ws-cleanup_1538085846793
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 28 00:20
>> pulsar_precommit_cpp_ws-cleanup_1538094024821
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 28 17:42
>> pulsar_precommit_cpp_ws-cleanup_1538156543343
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:02
>> pulsar_precommit_integrationtests_ws-cleanup_1536298125841
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:00
>> pulsar_precommit_integrationtests_ws-cleanup_1536968128660
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 18 00:43
>> pulsar_precommit_integrationtests_ws-cleanup_1537218541598
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
>> pulsar_precommit_integrationtests_ws-cleanup_1537331703324
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 22 00:43
>> pulsar_precommit_integrationtests_ws-cleanup_1537553748859
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 00:43
>> pulsar_precommit_integrationtests_ws-cleanup_1537730402937
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 07:48
>> pulsar_precommit_integrationtests_ws-cleanup_1537775336586
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 23:30
>> pulsar_precommit_integrationtests_ws-cleanup_1537831851711
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 23:08
>> pulsar_precommit_integrationtests_ws-cleanup_1537916928485
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 15:21
>> pulsar_precommit_integrationtests_ws-cleanup_1538061702243
>>
>>
>> -Chris
>> ASF Infra
>>
>>
>>
>>
>>
>> On Sep 30, 2018, at 3:12 PM, Chris Thistlethwaite <
>>
>> cthistlego@outlook.com> wrote:
>>
>>
>> I've been working on H21 and H31. They both had a some docker issues
>>
>> where the process was locked and I couldn't do anything with a few
>> containers that were over 8 days old.
>>
>>
>> Thought we might be good this morning so I threw them back into
>>
>> rotation. Guess not, so I'll take them back out.
>>
>>
>> -Chris T.
>> ________________________________
>> From: Mike Jumper <mj...@apache.org>
>> Sent: Sunday, September 30, 2018 6:03:51 PM
>> To: builds@apache.org
>> Subject: No space on H21 (Was: Fwd: Build failed in Jenkins:
>>
>> guacamole-client-coverity #73)
>>
>>
>> FYI - we're currently seeing disk space issues on H21.
>>
>> - Mike
>>
>>
>> ---------- Forwarded message ---------
>> From: Apache Jenkins Server <je...@builds.apache.org>
>> Date: Sun, Sep 30, 2018, 14:37
>> Subject: Build failed in Jenkins: guacamole-client-coverity #73
>> To: <de...@guacamole.apache.org>
>>
>>
>> See <
>>
>> https://builds.apache.org/job/guacamole-client-coverity/73/display/redirect
>>
>>
>>
>> ------------------------------------------
>> Started by user mjumper
>> [EnvInject] - Loading node environment variables.
>> Building remotely on H21 (ubuntu xenial) in workspace <
>> https://builds.apache.org/job/guacamole-client-coverity/ws/>
>> [WS-CLEANUP] Deleting project workspace...
>> Cloning the remote Git repository
>> Cloning repository
>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>>
>> git init <
>>
>>
>> https://builds.apache.org/job/guacamole-client-coverity/ws/guacamole-client
>>
>>
>> # timeout=10
>> Fetching upstream changes from
>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>>
>> git --version # timeout=10
>> git fetch --tags --progress
>>
>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>> +refs/heads/*:refs/remotes/origin/*
>> ERROR: Error cloning remote repo 'origin'
>> hudson.plugins.git.GitException: Command "git fetch --tags --progress
>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>> +refs/heads/*:refs/remotes/origin/*" returned status code 128:
>> stdout:
>> stderr: remote: Counting objects: 62166, done.
>> remote: Compressing objects: 0% (1/16096) remote: Compressing
>> objects: 1% (161/16096) remote: Compressing objects: 2%
>> (322/16096) remote: Compressing objects: 3% (483/16096)
>> remote: Compressing objects: 4% (644/16096) remote:
>> Compressing objects: 5% (805/16096) remote: Compressing
>> objects: 6% (966/16096) remote: Compressing objects: 7%
>> (1127/16096) remote: Compressing objects: 8% (1288/16096)
>> remote: Compressing objects: 9% (1449/16096) remote:
>> Compressing objects: 10% (1610/16096) remote: Compressing
>> objects: 11% (1771/16096) remote: Compressing objects: 12%
>> (1932/16096) remote: Compressing objects: 13% (2093/16096)
>> remote: Compressing objects: 14% (2254/16096) remote:
>> Compressing objects: 15% (2415/16096) remote: Compressing
>> objects: 16% (2576/16096) remote: Compressing objects: 17%
>> (2737/16096) remote: Compressing objects: 18% (2898/16096)
>> remote: Compressing objects: 19% (3059/16096) remote:
>> Compressing objects: 20% (3220/16096) remote: Compressing
>> objects: 21% (3381/16096) remote: Compressing objects: 22%
>> (3542/16096) remote: Compressing objects: 23% (3703/16096)
>> remote: Compressing objects: 24% (3864/16096) remote:
>> Compressing objects: 25% (4024/16096) remote: Compressing
>> objects: 26% (4185/16096) remote: Compressing objects: 27%
>> (4346/16096) remote: Compressing objects: 28% (4507/16096)
>> remote: Compressing objects: 29% (4668/16096) remote:
>> Compressing objects: 30% (4829/16096) remote: Compressing
>> objects: 31% (4990/16096) remote: Compressing objects: 32%
>> (5151/16096) remote: Compressing objects: 33% (5312/16096)
>> remote: Compressing objects: 34% (5473/16096) remote:
>> Compressing objects: 35% (5634/16096) remote: Compressing
>> objects: 36% (5795/16096) remote: Compressing objects: 37%
>> (5956/16096) remote: Compressing objects: 38% (6117/16096)
>> remote: Compressing objects: 39% (6278/16096) remote:
>> Compressing objects: 40% (6439/16096) remote: Compressing
>> objects: 41% (6600/16096) remote: Compressing objects: 42%
>> (6761/16096) remote: Compressing objects: 43% (6922/16096)
>> remote: Compressing objects: 44% (7083/16096) remote:
>> Compressing objects: 45% (7244/16096) remote: Compressing
>> objects: 46% (7405/16096) remote: Compressing objects: 47%
>> (7566/16096) remote: Compressing objects: 48% (7727/16096)
>> remote: Compressing objects: 49% (7888/16096) remote:
>> Compressing objects: 50% (8048/16096) remote: Compressing
>> objects: 51% (8209/16096) remote: Compressing objects: 52%
>> (8370/16096) remote: Compressing objects: 53% (8531/16096)
>> remote: Compressing objects: 54% (8692/16096) remote:
>> Compressing objects: 55% (8853/16096) remote: Compressing
>> objects: 56% (9014/16096) remote: Compressing objects: 57%
>> (9175/16096) remote: Compressing objects: 58% (9336/16096)
>> remote: Compressing objects: 59% (9497/16096) remote:
>> Compressing objects: 60% (9658/16096) remote: Compressing
>> objects: 61% (9819/16096) remote: Compressing objects: 62%
>> (9980/16096) remote: Compressing objects: 63% (10141/16096)
>> remote: Compressing objects: 64% (10302/16096) remote:
>> Compressing objects: 65% (10463/16096) remote: Compressing
>> objects: 66% (10624/16096) remote: Compressing objects: 67%
>> (10785/16096) remote: Compressing objects: 68% (10946/16096)
>> remote: Compressing objects: 69% (11107/16096) remote:
>> Compressing objects: 70% (11268/16096) remote: Compressing
>> objects: 71% (11429/16096) remote: Compressing objects: 72%
>> (11590/16096) remote: Compressing objects: 73% (11751/16096)
>> remote: Compressing objects: 74% (11912/16096) remote:
>> Compressing objects: 75% (12072/16096) remote: Compressing
>> objects: 76% (12233/16096) remote: Compressing objects: 77%
>> (12394/16096) remote: Compressing objects: 78% (12555/16096)
>> remote: Compressing objects: 79% (12716/16096) remote:
>> Compressing objects: 80% (12877/16096) remote: Compressing
>> objects: 81% (13038/16096) remote: Compressing objects: 82%
>> (13199/16096) remote: Compressing objects: 83% (13360/16096)
>> remote: Compressing objects: 84% (13521/16096) remote:
>> Compressing objects: 85% (13682/16096) remote: Compressing
>> objects: 86% (13843/16096) remote: Compressing objects: 87%
>> (14004/16096) remote: Compressing objects: 88% (14165/16096)
>> remote: Compressing objects: 89% (14326/16096) remote:
>> Compressing objects: 90% (14487/16096) remote: Compressing
>> objects: 91% (14648/16096) remote: Compressing objects: 92%
>> (14809/16096) remote: Compressing objects: 93% (14970/16096)
>> remote: Compressing objects: 94% (15131/16096) remote:
>> Compressing objects: 95% (15292/16096) remote: Compressing
>> objects: 96% (15453/16096) remote: Compressing objects: 97%
>> (15614/16096) remote: Compressing objects: 98% (15775/16096)
>> remote: Compressing objects: 99% (15936/16096) remote:
>> Compressing objects: 100% (16096/16096) remote: Compressing
>> objects: 100% (16096/16096), done.
>> Receiving objects: 0% (1/62166) Receiving objects: 1% (622/62166)
>> Receiving objects: 2% (1244/62166) Receiving objects: 3%
>> (1865/62166) Receiving objects: 4% (2487/62166) Receiving objects:
>> 5% (3109/62166) Receiving objects: 6% (3730/62166) Receiving
>> objects: 7% (4352/62166) Receiving objects: 8% (4974/62166)
>> Receiving objects: 9% (5595/62166) Receiving objects: 10%
>> (6217/62166) Receiving objects: 11% (6839/62166) Receiving objects:
>> 12% (7460/62166) Receiving objects: 13% (8082/62166) Receiving
>> objects: 14% (8704/62166) Receiving objects: 15% (9325/62166)
>> Receiving objects: 16% (9947/62166) Receiving objects: 17%
>> (10569/62166) Receiving objects: 18% (11190/62166) Receiving
>>
>> objects:
>>
>> 19% (11812/62166) Receiving objects: 20% (12434/62166) Receiving
>> objects: 21% (13055/62166) Receiving objects: 22% (13677/62166)
>> fatal: write error: No space left on device
>> fatal: index-pack failed
>>
>> at
>>
>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
>>
>> at
>>
>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1721)
>>
>> at
>>
>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:72)
>>
>> at
>>
>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:405)
>>
>> at
>>
>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:614)
>>
>> at
>>
>>
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
>>
>> at
>>
>>
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
>>
>> at hudson.remoting.UserRequest.perform(UserRequest.java:212)
>> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
>> at hudson.remoting.Request$2.run(Request.java:369)
>> at
>>
>>
>> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
>>
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>>
>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>
>> at
>>
>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>
>> at java.lang.Thread.run(Thread.java:748)
>> Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
>>
>> call
>>
>> to H21
>> at
>> hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
>> at
>>
>>
>> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
>>
>> at hudson.remoting.Channel.call(Channel.java:955)
>> at
>>
>>
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
>>
>> at sun.reflect.GeneratedMethodAccessor918.invoke(Unknown
>> Source)
>> at
>>
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at
>>
>>
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
>>
>> at com.sun.proxy.$Proxy142.execute(Unknown Source)
>> at
>> hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1146)
>> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
>> at hudson.scm.SCM.checkout(SCM.java:504)
>> at
>> hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>> at
>>
>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>>
>> at
>> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>> at
>>
>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>>
>> at hudson.model.Run.execute(Run.java:1794)
>> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>> at
>> hudson.model.ResourceController.execute(ResourceController.java:97)
>> at hudson.model.Executor.run(Executor.java:429)
>> ERROR: Error cloning remote repo 'origin'
>> Retrying after 10 seconds
>>
>> git rev-parse --is-inside-work-tree # timeout=10
>>
>> Fetching changes from the remote Git repository
>>
>> git config remote.origin.url
>>
>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git #
>>
>> timeout=10
>>
>> Fetching upstream changes from
>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>>
>> git --version # timeout=10
>> git fetch --tags --progress
>>
>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>> +refs/heads/*:refs/remotes/origin/*
>> ERROR: Error fetching remote repo 'origin'
>> hudson.plugins.git.GitException: Failed to fetch from
>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>> at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
>> at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
>> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
>> at hudson.scm.SCM.checkout(SCM.java:504)
>> at
>>
>> hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>>
>> at
>>
>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>>
>> at
>> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>> at
>>
>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>>
>> at hudson.model.Run.execute(Run.java:1794)
>> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>> at
>> hudson.model.ResourceController.execute(ResourceController.java:97)
>> at hudson.model.Executor.run(Executor.java:429)
>> Caused by: hudson.plugins.git.GitException: Command "git fetch --tags
>> --progress https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>> +refs/heads/*:refs/remotes/origin/*" returned status code 128:
>> stdout:
>> stderr: remote: Counting objects: 62166, done.
>> remote: Compressing objects: 0% (1/16096) remote: Compressing
>> objects: 1% (161/16096) remote: Compressing objects: 2%
>> (322/16096) remote: Compressing objects: 3% (483/16096)
>> remote: Compressing objects: 4% (644/16096) remote:
>> Compressing objects: 5% (805/16096) remote: Compressing
>> objects: 6% (966/16096) remote: Compressing objects: 7%
>> (1127/16096) remote: Compressing objects: 8% (1288/16096)
>> remote: Compressing objects: 9% (1449/16096) remote:
>> Compressing objects: 10% (1610/16096) remote: Compressing
>> objects: 11% (1771/16096) remote: Compressing objects: 12%
>> (1932/16096) remote: Compressing objects: 13% (2093/16096)
>> remote: Compressing objects: 14% (2254/16096) remote:
>> Compressing objects: 15% (2415/16096) remote: Compressing
>> objects: 16% (2576/16096) remote: Compressing objects: 17%
>> (2737/16096) remote: Compressing objects: 18% (2898/16096)
>> remote: Compressing objects: 19% (3059/16096) remote:
>> Compressing objects: 20% (3220/16096) remote: Compressing
>> objects: 21% (3381/16096) remote: Compressing objects: 22%
>> (3542/16096) remote: Compressing objects: 23% (3703/16096)
>> remote: Compressing objects: 24% (3864/16096) remote:
>> Compressing objects: 25% (4024/16096) remote: Compressing
>> objects: 26% (4185/16096) remote: Compressing objects: 27%
>> (4346/16096) remote: Compressing objects: 28% (4507/16096)
>> remote: Compressing objects: 29% (4668/16096) remote:
>> Compressing objects: 30% (4829/16096) remote: Compressing
>> objects: 31% (4990/16096) remote: Compressing objects: 32%
>> (5151/16096) remote: Compressing objects: 33% (5312/16096)
>> remote: Compressing objects: 34% (5473/16096) remote:
>> Compressing objects: 35% (5634/16096) remote: Compressing
>> objects: 36% (5795/16096) remote: Compressing objects: 37%
>> (5956/16096) remote: Compressing objects: 38% (6117/16096)
>> remote: Compressing objects: 39% (6278/16096) remote:
>> Compressing objects: 40% (6439/16096) remote: Compressing
>> objects: 41% (6600/16096) remote: Compressing objects: 42%
>> (6761/16096) remote: Compressing objects: 43% (6922/16096)
>> remote: Compressing objects: 44% (7083/16096) remote:
>> Compressing objects: 45% (7244/16096) remote: Compressing
>> objects: 46% (7405/16096) remote: Compressing objects: 47%
>> (7566/16096) remote: Compressing objects: 48% (7727/16096)
>> remote: Compressing objects: 49% (7888/16096) remote:
>> Compressing objects: 50% (8048/16096) remote: Compressing
>> objects: 51% (8209/16096) remote: Compressing objects: 52%
>> (8370/16096) remote: Compressing objects: 53% (8531/16096)
>> remote: Compressing objects: 54% (8692/16096) remote:
>> Compressing objects: 55% (8853/16096) remote: Compressing
>> objects: 56% (9014/16096) remote: Compressing objects: 57%
>> (9175/16096) remote: Compressing objects: 58% (9336/16096)
>> remote: Compressing objects: 59% (9497/16096) remote:
>> Compressing objects: 60% (9658/16096) remote: Compressing
>> objects: 61% (9819/16096) remote: Compressing objects: 62%
>> (9980/16096) remote: Compressing objects: 63% (10141/16096)
>> remote: Compressing objects: 64% (10302/16096) remote:
>> Compressing objects: 65% (10463/16096) remote: Compressing
>> objects: 66% (10624/16096) remote: Compressing objects: 67%
>> (10785/16096) remote: Compressing objects: 68% (10946/16096)
>> remote: Compressing objects: 69% (11107/16096) remote:
>> Compressing objects: 70% (11268/16096) remote: Compressing
>> objects: 71% (11429/16096) remote: Compressing objects: 72%
>> (11590/16096) remote: Compressing objects: 73% (11751/16096)
>> remote: Compressing objects: 74% (11912/16096) remote:
>> Compressing objects: 75% (12072/16096) remote: Compressing
>> objects: 76% (12233/16096) remote: Compressing objects: 77%
>> (12394/16096) remote: Compressing objects: 78% (12555/16096)
>> remote: Compressing objects: 79% (12716/16096) remote:
>> Compressing objects: 80% (12877/16096) remote: Compressing
>> objects: 81% (13038/16096) remote: Compressing objects: 82%
>> (13199/16096) remote: Compressing objects: 83% (13360/16096)
>> remote: Compressing objects: 84% (13521/16096) remote:
>> Compressing objects: 85% (13682/16096) remote: Compressing
>> objects: 86% (13843/16096) remote: Compressing objects: 87%
>> (14004/16096) remote: Compressing objects: 88% (14165/16096)
>> remote: Compressing objects: 89% (14326/16096) remote:
>> Compressing objects: 90% (14487/16096) remote: Compressing
>> objects: 91% (14648/16096) remote: Compressing objects: 92%
>> (14809/16096) remote: Compressing objects: 93% (14970/16096)
>> remote: Compressing objects: 94% (15131/16096) remote:
>> Compressing objects: 95% (15292/16096) remote: Compressing
>> objects: 96% (15453/16096) remote: Compressing objects: 97%
>> (15614/16096) remote: Compressing objects: 98% (15775/16096)
>> remote: Compressing objects: 99% (15936/16096) remote:
>> Compressing objects: 100% (16096/16096) remote: Compressing
>> objects: 100% (16096/16096), done.
>> Receiving objects: 0% (1/62166) fatal: write error: No space left on
>> device
>> fatal: index-pack failed
>>
>> at
>>
>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
>>
>> at
>>
>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1721)
>>
>> at
>>
>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:72)
>>
>> at
>>
>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:405)
>>
>> at
>>
>>
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
>>
>> at
>>
>>
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
>>
>> at hudson.remoting.UserRequest.perform(UserRequest.java:212)
>> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
>> at hudson.remoting.Request$2.run(Request.java:369)
>> at
>>
>>
>> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
>>
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>>
>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>
>> at
>>
>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>
>> at java.lang.Thread.run(Thread.java:748)
>> Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
>>
>> call
>>
>> to H21
>> at
>> hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
>> at
>>
>>
>> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
>>
>> at hudson.remoting.Channel.call(Channel.java:955)
>> at
>>
>>
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
>>
>> at sun.reflect.GeneratedMethodAccessor918.invoke(Unknown
>> Source)
>> at
>>
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at
>>
>>
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
>>
>> at com.sun.proxy.$Proxy119.execute(Unknown Source)
>> at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:886)
>> at
>> hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
>> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
>> at hudson.scm.SCM.checkout(SCM.java:504)
>> at
>> hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>> at
>>
>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>>
>> at
>> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>> at
>>
>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>>
>> at hudson.model.Run.execute(Run.java:1794)
>> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>> at
>> hudson.model.ResourceController.execute(ResourceController.java:97)
>> at hudson.model.Executor.run(Executor.java:429)
>> ERROR: Error fetching remote repo 'origin'
>> Retrying after 10 seconds
>>
>> git rev-parse --is-inside-work-tree # timeout=10
>>
>> Fetching changes from the remote Git repository
>>
>> git config remote.origin.url
>>
>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git #
>>
>> timeout=10
>>
>> ERROR: Error fetching remote repo 'origin'
>> hudson.plugins.git.GitException: Failed to fetch from
>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>> at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
>> at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
>> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
>> at hudson.scm.SCM.checkout(SCM.java:504)
>> at
>>
>> hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>>
>> at
>>
>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>>
>> at
>> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>> at
>>
>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>>
>> at hudson.model.Run.execute(Run.java:1794)
>> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>> at
>> hudson.model.ResourceController.execute(ResourceController.java:97)
>> at hudson.model.Executor.run(Executor.java:429)
>> Caused by: hudson.plugins.git.GitException: Command "git config
>> remote.origin.url
>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git" returned
>> status code 4:
>> stdout:
>> stderr: error: failed to write new configuration file <
>>
>>
>> https://builds.apache.org/job/guacamole-client-coverity/ws/guacamole-client/.git/config.lock
>>
>>
>>
>> at
>>
>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
>>
>> at
>>
>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1970)
>>
>> at
>>
>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1966)
>>
>> at
>>
>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1597)
>>
>> at
>>
>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1609)
>>
>> at
>>
>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.setRemoteUrl(CliGitAPIImpl.java:1243)
>>
>> at hudson.plugins.git.GitAPI.setRemoteUrl(GitAPI.java:160)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>>
>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>
>> at
>>
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at
>>
>>
>> hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:929)
>>
>> at
>>
>>
>> hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:903)
>>
>> at
>>
>>
>> hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:855)
>>
>> at hudson.remoting.UserRequest.perform(UserRequest.java:212)
>> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
>> at hudson.remoting.Request$2.run(Request.java:369)
>> at
>>
>>
>> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
>>
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>>
>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>
>> at
>>
>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>
>> at java.lang.Thread.run(Thread.java:748)
>> Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
>>
>> call
>>
>> to H21
>> at
>> hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
>> at
>>
>>
>> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
>>
>> at hudson.remoting.Channel.call(Channel.java:955)
>> at
>>
>>
>> hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:283)
>>
>> at com.sun.proxy.$Proxy118.setRemoteUrl(Unknown Source)
>> at
>>
>>
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl.setRemoteUrl(RemoteGitImpl.java:295)
>>
>> at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:876)
>> at
>> hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
>> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
>> at hudson.scm.SCM.checkout(SCM.java:504)
>> at
>> hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>> at
>>
>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>>
>> at
>> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>> at
>>
>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>>
>> at hudson.model.Run.execute(Run.java:1794)
>> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>> at
>> hudson.model.ResourceController.execute(ResourceController.java:97)
>> at hudson.model.Executor.run(Executor.java:429)
>> ERROR: Error fetching remote repo 'origin'
>>
>>
>> --
>>
>> Matteo Merli
>> <mm...@apache.org>
>>
>>
>>
>
> --
> WBR
> Maxim aka solomax
Re: No space on H21 (Was: Fwd: Build failed in Jenkins:
guacamole-client-coverity #73)
Posted by Maxim Solodovnik <so...@gmail.com>.
It seems there is no space on ubuntu-eu2
<https://builds.apache.org/computer/ubuntu-eu2>
(git clone fails :((( )
On Tue, 2 Oct 2018 at 02:29, Chris Lambertus <cm...@apache.org> wrote:
> Well, I had thought it was something specific to the build, since it only
> happens for a couple projects, but upon further investigation, there
> appears to be an open issue related to this:
>
> https://issues.jenkins-ci.org/browse/JENKINS-53579
>
> I will need to look into this further. In the meantime, Infra will need to
> implement a cronjob that purges the errant ws-cleanup folders.
>
> -Chris
>
>
>
>
> On Oct 1, 2018, at 11:30 AM, Matteo Merli <ma...@gmail.com> wrote:
>
> Chris, are these `ws-cleanup` jobs something that is created internally by
> Jenkins? The job we have is `pulsar_precommit_cpp`.
>
> Is there any setting in Jenkins that could explain that these files are not
> removed?
>
> On https://builds.apache.org/job/pulsar_precommit_cpp/configure , we
> already have "Days to keep builds == 1".
> For "Days to keep artifacts" is left blank. Should that be set to 1 day
> too?
>
> Any other hint at things we can try to ensure builds are cleaned up
> promptly?
>
> Thanks,
> Matteo
>
> On Sun, Sep 30, 2018 at 3:57 PM Chris Lambertus <cm...@apache.org> wrote:
>
> The main culprit on H31 appears to be pulsar ws-cleanup jobs consuming
> almost 40% of the disk. (The problem on H21 was something docker-related.)
>
>
> This is the second project I have found (Brooklyn was the first) which is
> leaving ws-cleanup directories all over the workspace and filling up the
> disks. Each one of these was 500+ MB. Pulsar, please address ASAP.
>
>
>
>
> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:19
> pulsar_precommit_cpp@2_ws-cleanup_1535737112280
> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:18
> pulsar_precommit_cpp@2_ws-cleanup_1536174576053
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 06:04
> pulsar_precommit_cpp@2_ws-cleanup_1537292379178
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 22:45
> pulsar_precommit_cpp@2_ws-cleanup_1537397120862
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> pulsar_precommit_cpp@2_ws-cleanup_1537403378279
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 21 23:31
> pulsar_precommit_cpp@2_ws-cleanup_1537572701829
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 26 16:04
> pulsar_precommit_cpp@2_ws-cleanup_1537977879341
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 22:23
> pulsar_precommit_cpp@2_ws-cleanup_1538087035027
> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:14
> pulsar_precommit_cpp_ws-cleanup_1535699830230
> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:16
> pulsar_precommit_cpp_ws-cleanup_1535735511423
> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:13
> pulsar_precommit_cpp_ws-cleanup_1535786064397
> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:18
> pulsar_precommit_cpp_ws-cleanup_1536096963586
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 21 06:47
> pulsar_precommit_cpp_ws-cleanup_1536114938910
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 21 06:45
> pulsar_precommit_cpp_ws-cleanup_1536173165909
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:38
> pulsar_precommit_cpp_ws-cleanup_1536175584418
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:50
> pulsar_precommit_cpp_ws-cleanup_1536180707506
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:38
> pulsar_precommit_cpp_ws-cleanup_1536182927940
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:51
> pulsar_precommit_cpp_ws-cleanup_1536185193716
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:50
> pulsar_precommit_cpp_ws-cleanup_1536273709029
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 05:59
> pulsar_precommit_cpp_ws-cleanup_1536316964794
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:06
> pulsar_precommit_cpp_ws-cleanup_1536675650893
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 28 06:08
> pulsar_precommit_cpp_ws-cleanup_1536879616962
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 30 19:53
> pulsar_precommit_cpp_ws-cleanup_1536891056968
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 30 19:47
> pulsar_precommit_cpp_ws-cleanup_1536966466683
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 18 00:43
> pulsar_precommit_cpp_ws-cleanup_1537216387387
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 00:45
> pulsar_precommit_cpp_ws-cleanup_1537292309413
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 00:45
> pulsar_precommit_cpp_ws-cleanup_1537308526251
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> pulsar_precommit_cpp_ws-cleanup_1537331780420
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> pulsar_precommit_cpp_ws-cleanup_1537379886715
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
> pulsar_precommit_cpp_ws-cleanup_1537386197893
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
> pulsar_precommit_cpp_ws-cleanup_1537397073464
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
> pulsar_precommit_cpp_ws-cleanup_1537403247575
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:43
> pulsar_precommit_cpp_ws-cleanup_1537413954888
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
> pulsar_precommit_cpp_ws-cleanup_1537417322151
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
> pulsar_precommit_cpp_ws-cleanup_1537421381129
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:44
> pulsar_precommit_cpp_ws-cleanup_1537469348852
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
> pulsar_precommit_cpp_ws-cleanup_1537489399468
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 22 00:44
> pulsar_precommit_cpp_ws-cleanup_1537566528660
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 23:12
> pulsar_precommit_cpp_ws-cleanup_1537571549978
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 22 00:44
> pulsar_precommit_cpp_ws-cleanup_1537573605258
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 00:43
> pulsar_precommit_cpp_ws-cleanup_1537855939828
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 11:18
> pulsar_precommit_cpp_ws-cleanup_1537874305575
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 12:41
> pulsar_precommit_cpp_ws-cleanup_1537879296622
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 01:39
> pulsar_precommit_cpp_ws-cleanup_1537925955967
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 26 13:17
> pulsar_precommit_cpp_ws-cleanup_1537967826838
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 15:55
> pulsar_precommit_cpp_ws-cleanup_1537977326736
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 17:34
> pulsar_precommit_cpp_ws-cleanup_1537983268177
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 22:04
> pulsar_precommit_cpp_ws-cleanup_1538085846793
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 28 00:20
> pulsar_precommit_cpp_ws-cleanup_1538094024821
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 28 17:42
> pulsar_precommit_cpp_ws-cleanup_1538156543343
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:02
> pulsar_precommit_integrationtests_ws-cleanup_1536298125841
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:00
> pulsar_precommit_integrationtests_ws-cleanup_1536968128660
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 18 00:43
> pulsar_precommit_integrationtests_ws-cleanup_1537218541598
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> pulsar_precommit_integrationtests_ws-cleanup_1537331703324
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 22 00:43
> pulsar_precommit_integrationtests_ws-cleanup_1537553748859
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 00:43
> pulsar_precommit_integrationtests_ws-cleanup_1537730402937
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 07:48
> pulsar_precommit_integrationtests_ws-cleanup_1537775336586
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 23:30
> pulsar_precommit_integrationtests_ws-cleanup_1537831851711
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 23:08
> pulsar_precommit_integrationtests_ws-cleanup_1537916928485
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 15:21
> pulsar_precommit_integrationtests_ws-cleanup_1538061702243
>
>
> -Chris
> ASF Infra
>
>
>
>
>
> On Sep 30, 2018, at 3:12 PM, Chris Thistlethwaite <
>
> cthistlego@outlook.com> wrote:
>
>
> I've been working on H21 and H31. They both had a some docker issues
>
> where the process was locked and I couldn't do anything with a few
> containers that were over 8 days old.
>
>
> Thought we might be good this morning so I threw them back into
>
> rotation. Guess not, so I'll take them back out.
>
>
> -Chris T.
> ________________________________
> From: Mike Jumper <mj...@apache.org>
> Sent: Sunday, September 30, 2018 6:03:51 PM
> To: builds@apache.org
> Subject: No space on H21 (Was: Fwd: Build failed in Jenkins:
>
> guacamole-client-coverity #73)
>
>
> FYI - we're currently seeing disk space issues on H21.
>
> - Mike
>
>
> ---------- Forwarded message ---------
> From: Apache Jenkins Server <je...@builds.apache.org>
> Date: Sun, Sep 30, 2018, 14:37
> Subject: Build failed in Jenkins: guacamole-client-coverity #73
> To: <de...@guacamole.apache.org>
>
>
> See <
>
> https://builds.apache.org/job/guacamole-client-coverity/73/display/redirect
>
>
>
> ------------------------------------------
> Started by user mjumper
> [EnvInject] - Loading node environment variables.
> Building remotely on H21 (ubuntu xenial) in workspace <
> https://builds.apache.org/job/guacamole-client-coverity/ws/>
> [WS-CLEANUP] Deleting project workspace...
> Cloning the remote Git repository
> Cloning repository
> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>
> git init <
>
>
> https://builds.apache.org/job/guacamole-client-coverity/ws/guacamole-client
>
>
> # timeout=10
> Fetching upstream changes from
> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>
> git --version # timeout=10
> git fetch --tags --progress
>
> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> +refs/heads/*:refs/remotes/origin/*
> ERROR: Error cloning remote repo 'origin'
> hudson.plugins.git.GitException: Command "git fetch --tags --progress
> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> +refs/heads/*:refs/remotes/origin/*" returned status code 128:
> stdout:
> stderr: remote: Counting objects: 62166, done.
> remote: Compressing objects: 0% (1/16096) remote: Compressing
> objects: 1% (161/16096) remote: Compressing objects: 2%
> (322/16096) remote: Compressing objects: 3% (483/16096)
> remote: Compressing objects: 4% (644/16096) remote:
> Compressing objects: 5% (805/16096) remote: Compressing
> objects: 6% (966/16096) remote: Compressing objects: 7%
> (1127/16096) remote: Compressing objects: 8% (1288/16096)
> remote: Compressing objects: 9% (1449/16096) remote:
> Compressing objects: 10% (1610/16096) remote: Compressing
> objects: 11% (1771/16096) remote: Compressing objects: 12%
> (1932/16096) remote: Compressing objects: 13% (2093/16096)
> remote: Compressing objects: 14% (2254/16096) remote:
> Compressing objects: 15% (2415/16096) remote: Compressing
> objects: 16% (2576/16096) remote: Compressing objects: 17%
> (2737/16096) remote: Compressing objects: 18% (2898/16096)
> remote: Compressing objects: 19% (3059/16096) remote:
> Compressing objects: 20% (3220/16096) remote: Compressing
> objects: 21% (3381/16096) remote: Compressing objects: 22%
> (3542/16096) remote: Compressing objects: 23% (3703/16096)
> remote: Compressing objects: 24% (3864/16096) remote:
> Compressing objects: 25% (4024/16096) remote: Compressing
> objects: 26% (4185/16096) remote: Compressing objects: 27%
> (4346/16096) remote: Compressing objects: 28% (4507/16096)
> remote: Compressing objects: 29% (4668/16096) remote:
> Compressing objects: 30% (4829/16096) remote: Compressing
> objects: 31% (4990/16096) remote: Compressing objects: 32%
> (5151/16096) remote: Compressing objects: 33% (5312/16096)
> remote: Compressing objects: 34% (5473/16096) remote:
> Compressing objects: 35% (5634/16096) remote: Compressing
> objects: 36% (5795/16096) remote: Compressing objects: 37%
> (5956/16096) remote: Compressing objects: 38% (6117/16096)
> remote: Compressing objects: 39% (6278/16096) remote:
> Compressing objects: 40% (6439/16096) remote: Compressing
> objects: 41% (6600/16096) remote: Compressing objects: 42%
> (6761/16096) remote: Compressing objects: 43% (6922/16096)
> remote: Compressing objects: 44% (7083/16096) remote:
> Compressing objects: 45% (7244/16096) remote: Compressing
> objects: 46% (7405/16096) remote: Compressing objects: 47%
> (7566/16096) remote: Compressing objects: 48% (7727/16096)
> remote: Compressing objects: 49% (7888/16096) remote:
> Compressing objects: 50% (8048/16096) remote: Compressing
> objects: 51% (8209/16096) remote: Compressing objects: 52%
> (8370/16096) remote: Compressing objects: 53% (8531/16096)
> remote: Compressing objects: 54% (8692/16096) remote:
> Compressing objects: 55% (8853/16096) remote: Compressing
> objects: 56% (9014/16096) remote: Compressing objects: 57%
> (9175/16096) remote: Compressing objects: 58% (9336/16096)
> remote: Compressing objects: 59% (9497/16096) remote:
> Compressing objects: 60% (9658/16096) remote: Compressing
> objects: 61% (9819/16096) remote: Compressing objects: 62%
> (9980/16096) remote: Compressing objects: 63% (10141/16096)
> remote: Compressing objects: 64% (10302/16096) remote:
> Compressing objects: 65% (10463/16096) remote: Compressing
> objects: 66% (10624/16096) remote: Compressing objects: 67%
> (10785/16096) remote: Compressing objects: 68% (10946/16096)
> remote: Compressing objects: 69% (11107/16096) remote:
> Compressing objects: 70% (11268/16096) remote: Compressing
> objects: 71% (11429/16096) remote: Compressing objects: 72%
> (11590/16096) remote: Compressing objects: 73% (11751/16096)
> remote: Compressing objects: 74% (11912/16096) remote:
> Compressing objects: 75% (12072/16096) remote: Compressing
> objects: 76% (12233/16096) remote: Compressing objects: 77%
> (12394/16096) remote: Compressing objects: 78% (12555/16096)
> remote: Compressing objects: 79% (12716/16096) remote:
> Compressing objects: 80% (12877/16096) remote: Compressing
> objects: 81% (13038/16096) remote: Compressing objects: 82%
> (13199/16096) remote: Compressing objects: 83% (13360/16096)
> remote: Compressing objects: 84% (13521/16096) remote:
> Compressing objects: 85% (13682/16096) remote: Compressing
> objects: 86% (13843/16096) remote: Compressing objects: 87%
> (14004/16096) remote: Compressing objects: 88% (14165/16096)
> remote: Compressing objects: 89% (14326/16096) remote:
> Compressing objects: 90% (14487/16096) remote: Compressing
> objects: 91% (14648/16096) remote: Compressing objects: 92%
> (14809/16096) remote: Compressing objects: 93% (14970/16096)
> remote: Compressing objects: 94% (15131/16096) remote:
> Compressing objects: 95% (15292/16096) remote: Compressing
> objects: 96% (15453/16096) remote: Compressing objects: 97%
> (15614/16096) remote: Compressing objects: 98% (15775/16096)
> remote: Compressing objects: 99% (15936/16096) remote:
> Compressing objects: 100% (16096/16096) remote: Compressing
> objects: 100% (16096/16096), done.
> Receiving objects: 0% (1/62166) Receiving objects: 1% (622/62166)
> Receiving objects: 2% (1244/62166) Receiving objects: 3%
> (1865/62166) Receiving objects: 4% (2487/62166) Receiving objects:
> 5% (3109/62166) Receiving objects: 6% (3730/62166) Receiving
> objects: 7% (4352/62166) Receiving objects: 8% (4974/62166)
> Receiving objects: 9% (5595/62166) Receiving objects: 10%
> (6217/62166) Receiving objects: 11% (6839/62166) Receiving objects:
> 12% (7460/62166) Receiving objects: 13% (8082/62166) Receiving
> objects: 14% (8704/62166) Receiving objects: 15% (9325/62166)
> Receiving objects: 16% (9947/62166) Receiving objects: 17%
> (10569/62166) Receiving objects: 18% (11190/62166) Receiving
>
> objects:
>
> 19% (11812/62166) Receiving objects: 20% (12434/62166) Receiving
> objects: 21% (13055/62166) Receiving objects: 22% (13677/62166)
> fatal: write error: No space left on device
> fatal: index-pack failed
>
> at
>
>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
>
> at
>
>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1721)
>
> at
>
>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:72)
>
> at
>
>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:405)
>
> at
>
>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:614)
>
> at
>
>
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
>
> at
>
>
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
>
> at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> at hudson.remoting.Request$2.run(Request.java:369)
> at
>
>
> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
>
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>
> at
>
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>
> at java.lang.Thread.run(Thread.java:748)
> Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
>
> call
>
> to H21
> at
> hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> at
>
>
> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
>
> at hudson.remoting.Channel.call(Channel.java:955)
> at
>
>
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
>
> at sun.reflect.GeneratedMethodAccessor918.invoke(Unknown
> Source)
> at
>
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
>
>
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
>
> at com.sun.proxy.$Proxy142.execute(Unknown Source)
> at
> hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1146)
> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> at hudson.scm.SCM.checkout(SCM.java:504)
> at
> hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> at
>
>
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>
> at
> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> at
>
>
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>
> at hudson.model.Run.execute(Run.java:1794)
> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> at
> hudson.model.ResourceController.execute(ResourceController.java:97)
> at hudson.model.Executor.run(Executor.java:429)
> ERROR: Error cloning remote repo 'origin'
> Retrying after 10 seconds
>
> git rev-parse --is-inside-work-tree # timeout=10
>
> Fetching changes from the remote Git repository
>
> git config remote.origin.url
>
> https://git-wip-us.apache.org/repos/asf/guacamole-client.git #
>
> timeout=10
>
> Fetching upstream changes from
> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>
> git --version # timeout=10
> git fetch --tags --progress
>
> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> +refs/heads/*:refs/remotes/origin/*
> ERROR: Error fetching remote repo 'origin'
> hudson.plugins.git.GitException: Failed to fetch from
> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
> at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> at hudson.scm.SCM.checkout(SCM.java:504)
> at
>
> hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>
> at
>
>
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>
> at
> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> at
>
>
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>
> at hudson.model.Run.execute(Run.java:1794)
> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> at
> hudson.model.ResourceController.execute(ResourceController.java:97)
> at hudson.model.Executor.run(Executor.java:429)
> Caused by: hudson.plugins.git.GitException: Command "git fetch --tags
> --progress https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> +refs/heads/*:refs/remotes/origin/*" returned status code 128:
> stdout:
> stderr: remote: Counting objects: 62166, done.
> remote: Compressing objects: 0% (1/16096) remote: Compressing
> objects: 1% (161/16096) remote: Compressing objects: 2%
> (322/16096) remote: Compressing objects: 3% (483/16096)
> remote: Compressing objects: 4% (644/16096) remote:
> Compressing objects: 5% (805/16096) remote: Compressing
> objects: 6% (966/16096) remote: Compressing objects: 7%
> (1127/16096) remote: Compressing objects: 8% (1288/16096)
> remote: Compressing objects: 9% (1449/16096) remote:
> Compressing objects: 10% (1610/16096) remote: Compressing
> objects: 11% (1771/16096) remote: Compressing objects: 12%
> (1932/16096) remote: Compressing objects: 13% (2093/16096)
> remote: Compressing objects: 14% (2254/16096) remote:
> Compressing objects: 15% (2415/16096) remote: Compressing
> objects: 16% (2576/16096) remote: Compressing objects: 17%
> (2737/16096) remote: Compressing objects: 18% (2898/16096)
> remote: Compressing objects: 19% (3059/16096) remote:
> Compressing objects: 20% (3220/16096) remote: Compressing
> objects: 21% (3381/16096) remote: Compressing objects: 22%
> (3542/16096) remote: Compressing objects: 23% (3703/16096)
> remote: Compressing objects: 24% (3864/16096) remote:
> Compressing objects: 25% (4024/16096) remote: Compressing
> objects: 26% (4185/16096) remote: Compressing objects: 27%
> (4346/16096) remote: Compressing objects: 28% (4507/16096)
> remote: Compressing objects: 29% (4668/16096) remote:
> Compressing objects: 30% (4829/16096) remote: Compressing
> objects: 31% (4990/16096) remote: Compressing objects: 32%
> (5151/16096) remote: Compressing objects: 33% (5312/16096)
> remote: Compressing objects: 34% (5473/16096) remote:
> Compressing objects: 35% (5634/16096) remote: Compressing
> objects: 36% (5795/16096) remote: Compressing objects: 37%
> (5956/16096) remote: Compressing objects: 38% (6117/16096)
> remote: Compressing objects: 39% (6278/16096) remote:
> Compressing objects: 40% (6439/16096) remote: Compressing
> objects: 41% (6600/16096) remote: Compressing objects: 42%
> (6761/16096) remote: Compressing objects: 43% (6922/16096)
> remote: Compressing objects: 44% (7083/16096) remote:
> Compressing objects: 45% (7244/16096) remote: Compressing
> objects: 46% (7405/16096) remote: Compressing objects: 47%
> (7566/16096) remote: Compressing objects: 48% (7727/16096)
> remote: Compressing objects: 49% (7888/16096) remote:
> Compressing objects: 50% (8048/16096) remote: Compressing
> objects: 51% (8209/16096) remote: Compressing objects: 52%
> (8370/16096) remote: Compressing objects: 53% (8531/16096)
> remote: Compressing objects: 54% (8692/16096) remote:
> Compressing objects: 55% (8853/16096) remote: Compressing
> objects: 56% (9014/16096) remote: Compressing objects: 57%
> (9175/16096) remote: Compressing objects: 58% (9336/16096)
> remote: Compressing objects: 59% (9497/16096) remote:
> Compressing objects: 60% (9658/16096) remote: Compressing
> objects: 61% (9819/16096) remote: Compressing objects: 62%
> (9980/16096) remote: Compressing objects: 63% (10141/16096)
> remote: Compressing objects: 64% (10302/16096) remote:
> Compressing objects: 65% (10463/16096) remote: Compressing
> objects: 66% (10624/16096) remote: Compressing objects: 67%
> (10785/16096) remote: Compressing objects: 68% (10946/16096)
> remote: Compressing objects: 69% (11107/16096) remote:
> Compressing objects: 70% (11268/16096) remote: Compressing
> objects: 71% (11429/16096) remote: Compressing objects: 72%
> (11590/16096) remote: Compressing objects: 73% (11751/16096)
> remote: Compressing objects: 74% (11912/16096) remote:
> Compressing objects: 75% (12072/16096) remote: Compressing
> objects: 76% (12233/16096) remote: Compressing objects: 77%
> (12394/16096) remote: Compressing objects: 78% (12555/16096)
> remote: Compressing objects: 79% (12716/16096) remote:
> Compressing objects: 80% (12877/16096) remote: Compressing
> objects: 81% (13038/16096) remote: Compressing objects: 82%
> (13199/16096) remote: Compressing objects: 83% (13360/16096)
> remote: Compressing objects: 84% (13521/16096) remote:
> Compressing objects: 85% (13682/16096) remote: Compressing
> objects: 86% (13843/16096) remote: Compressing objects: 87%
> (14004/16096) remote: Compressing objects: 88% (14165/16096)
> remote: Compressing objects: 89% (14326/16096) remote:
> Compressing objects: 90% (14487/16096) remote: Compressing
> objects: 91% (14648/16096) remote: Compressing objects: 92%
> (14809/16096) remote: Compressing objects: 93% (14970/16096)
> remote: Compressing objects: 94% (15131/16096) remote:
> Compressing objects: 95% (15292/16096) remote: Compressing
> objects: 96% (15453/16096) remote: Compressing objects: 97%
> (15614/16096) remote: Compressing objects: 98% (15775/16096)
> remote: Compressing objects: 99% (15936/16096) remote:
> Compressing objects: 100% (16096/16096) remote: Compressing
> objects: 100% (16096/16096), done.
> Receiving objects: 0% (1/62166) fatal: write error: No space left on
> device
> fatal: index-pack failed
>
> at
>
>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
>
> at
>
>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1721)
>
> at
>
>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:72)
>
> at
>
>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:405)
>
> at
>
>
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
>
> at
>
>
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
>
> at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> at hudson.remoting.Request$2.run(Request.java:369)
> at
>
>
> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
>
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>
> at
>
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>
> at java.lang.Thread.run(Thread.java:748)
> Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
>
> call
>
> to H21
> at
> hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> at
>
>
> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
>
> at hudson.remoting.Channel.call(Channel.java:955)
> at
>
>
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
>
> at sun.reflect.GeneratedMethodAccessor918.invoke(Unknown
> Source)
> at
>
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
>
>
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
>
> at com.sun.proxy.$Proxy119.execute(Unknown Source)
> at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:886)
> at
> hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> at hudson.scm.SCM.checkout(SCM.java:504)
> at
> hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> at
>
>
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>
> at
> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> at
>
>
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>
> at hudson.model.Run.execute(Run.java:1794)
> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> at
> hudson.model.ResourceController.execute(ResourceController.java:97)
> at hudson.model.Executor.run(Executor.java:429)
> ERROR: Error fetching remote repo 'origin'
> Retrying after 10 seconds
>
> git rev-parse --is-inside-work-tree # timeout=10
>
> Fetching changes from the remote Git repository
>
> git config remote.origin.url
>
> https://git-wip-us.apache.org/repos/asf/guacamole-client.git #
>
> timeout=10
>
> ERROR: Error fetching remote repo 'origin'
> hudson.plugins.git.GitException: Failed to fetch from
> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
> at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> at hudson.scm.SCM.checkout(SCM.java:504)
> at
>
> hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>
> at
>
>
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>
> at
> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> at
>
>
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>
> at hudson.model.Run.execute(Run.java:1794)
> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> at
> hudson.model.ResourceController.execute(ResourceController.java:97)
> at hudson.model.Executor.run(Executor.java:429)
> Caused by: hudson.plugins.git.GitException: Command "git config
> remote.origin.url
> https://git-wip-us.apache.org/repos/asf/guacamole-client.git" returned
> status code 4:
> stdout:
> stderr: error: failed to write new configuration file <
>
>
> https://builds.apache.org/job/guacamole-client-coverity/ws/guacamole-client/.git/config.lock
>
>
>
> at
>
>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
>
> at
>
>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1970)
>
> at
>
>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1966)
>
> at
>
>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1597)
>
> at
>
>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1609)
>
> at
>
>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.setRemoteUrl(CliGitAPIImpl.java:1243)
>
> at hudson.plugins.git.GitAPI.setRemoteUrl(GitAPI.java:160)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>
> at
>
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
>
>
> hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:929)
>
> at
>
>
> hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:903)
>
> at
>
>
> hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:855)
>
> at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> at hudson.remoting.Request$2.run(Request.java:369)
> at
>
>
> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
>
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>
> at
>
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>
> at java.lang.Thread.run(Thread.java:748)
> Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
>
> call
>
> to H21
> at
> hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> at
>
>
> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
>
> at hudson.remoting.Channel.call(Channel.java:955)
> at
>
>
> hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:283)
>
> at com.sun.proxy.$Proxy118.setRemoteUrl(Unknown Source)
> at
>
>
> org.jenkinsci.plugins.gitclient.RemoteGitImpl.setRemoteUrl(RemoteGitImpl.java:295)
>
> at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:876)
> at
> hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> at hudson.scm.SCM.checkout(SCM.java:504)
> at
> hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> at
>
>
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>
> at
> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> at
>
>
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>
> at hudson.model.Run.execute(Run.java:1794)
> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> at
> hudson.model.ResourceController.execute(ResourceController.java:97)
> at hudson.model.Executor.run(Executor.java:429)
> ERROR: Error fetching remote repo 'origin'
>
>
> --
>
> Matteo Merli
> <mm...@apache.org>
>
>
>
--
WBR
Maxim aka solomax
Re: No space on H21 (Was: Fwd: Build failed in Jenkins:
guacamole-client-coverity #73)
Posted by Chris Lambertus <cm...@apache.org>.
Well, I had thought it was something specific to the build, since it only happens for a couple projects, but upon further investigation, there appears to be an open issue related to this:
https://issues.jenkins-ci.org/browse/JENKINS-53579 <https://issues.jenkins-ci.org/browse/JENKINS-53579>
I will need to look into this further. In the meantime, Infra will need to implement a cronjob that purges the errant ws-cleanup folders.
-Chris
> On Oct 1, 2018, at 11:30 AM, Matteo Merli <ma...@gmail.com> wrote:
>
> Chris, are these `ws-cleanup` jobs something that is created internally by
> Jenkins? The job we have is `pulsar_precommit_cpp`.
>
> Is there any setting in Jenkins that could explain that these files are not
> removed?
>
> On https://builds.apache.org/job/pulsar_precommit_cpp/configure , we
> already have "Days to keep builds == 1".
> For "Days to keep artifacts" is left blank. Should that be set to 1 day
> too?
>
> Any other hint at things we can try to ensure builds are cleaned up
> promptly?
>
> Thanks,
> Matteo
>
> On Sun, Sep 30, 2018 at 3:57 PM Chris Lambertus <cm...@apache.org> wrote:
>
>> The main culprit on H31 appears to be pulsar ws-cleanup jobs consuming
>> almost 40% of the disk. (The problem on H21 was something docker-related.)
>>
>>
>> This is the second project I have found (Brooklyn was the first) which is
>> leaving ws-cleanup directories all over the workspace and filling up the
>> disks. Each one of these was 500+ MB. Pulsar, please address ASAP.
>>
>>
>>
>>
>> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:19
>> pulsar_precommit_cpp@2_ws-cleanup_1535737112280
>> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:18
>> pulsar_precommit_cpp@2_ws-cleanup_1536174576053
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 06:04
>> pulsar_precommit_cpp@2_ws-cleanup_1537292379178
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 22:45
>> pulsar_precommit_cpp@2_ws-cleanup_1537397120862
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
>> pulsar_precommit_cpp@2_ws-cleanup_1537403378279
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 21 23:31
>> pulsar_precommit_cpp@2_ws-cleanup_1537572701829
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 26 16:04
>> pulsar_precommit_cpp@2_ws-cleanup_1537977879341
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 22:23
>> pulsar_precommit_cpp@2_ws-cleanup_1538087035027
>> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:14
>> pulsar_precommit_cpp_ws-cleanup_1535699830230
>> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:16
>> pulsar_precommit_cpp_ws-cleanup_1535735511423
>> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:13
>> pulsar_precommit_cpp_ws-cleanup_1535786064397
>> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:18
>> pulsar_precommit_cpp_ws-cleanup_1536096963586
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 21 06:47
>> pulsar_precommit_cpp_ws-cleanup_1536114938910
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 21 06:45
>> pulsar_precommit_cpp_ws-cleanup_1536173165909
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:38
>> pulsar_precommit_cpp_ws-cleanup_1536175584418
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:50
>> pulsar_precommit_cpp_ws-cleanup_1536180707506
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:38
>> pulsar_precommit_cpp_ws-cleanup_1536182927940
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:51
>> pulsar_precommit_cpp_ws-cleanup_1536185193716
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:50
>> pulsar_precommit_cpp_ws-cleanup_1536273709029
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 05:59
>> pulsar_precommit_cpp_ws-cleanup_1536316964794
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:06
>> pulsar_precommit_cpp_ws-cleanup_1536675650893
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 28 06:08
>> pulsar_precommit_cpp_ws-cleanup_1536879616962
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 30 19:53
>> pulsar_precommit_cpp_ws-cleanup_1536891056968
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 30 19:47
>> pulsar_precommit_cpp_ws-cleanup_1536966466683
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 18 00:43
>> pulsar_precommit_cpp_ws-cleanup_1537216387387
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 00:45
>> pulsar_precommit_cpp_ws-cleanup_1537292309413
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 00:45
>> pulsar_precommit_cpp_ws-cleanup_1537308526251
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
>> pulsar_precommit_cpp_ws-cleanup_1537331780420
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
>> pulsar_precommit_cpp_ws-cleanup_1537379886715
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
>> pulsar_precommit_cpp_ws-cleanup_1537386197893
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
>> pulsar_precommit_cpp_ws-cleanup_1537397073464
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
>> pulsar_precommit_cpp_ws-cleanup_1537403247575
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:43
>> pulsar_precommit_cpp_ws-cleanup_1537413954888
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
>> pulsar_precommit_cpp_ws-cleanup_1537417322151
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
>> pulsar_precommit_cpp_ws-cleanup_1537421381129
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:44
>> pulsar_precommit_cpp_ws-cleanup_1537469348852
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
>> pulsar_precommit_cpp_ws-cleanup_1537489399468
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 22 00:44
>> pulsar_precommit_cpp_ws-cleanup_1537566528660
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 23:12
>> pulsar_precommit_cpp_ws-cleanup_1537571549978
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 22 00:44
>> pulsar_precommit_cpp_ws-cleanup_1537573605258
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 00:43
>> pulsar_precommit_cpp_ws-cleanup_1537855939828
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 11:18
>> pulsar_precommit_cpp_ws-cleanup_1537874305575
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 12:41
>> pulsar_precommit_cpp_ws-cleanup_1537879296622
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 01:39
>> pulsar_precommit_cpp_ws-cleanup_1537925955967
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 26 13:17
>> pulsar_precommit_cpp_ws-cleanup_1537967826838
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 15:55
>> pulsar_precommit_cpp_ws-cleanup_1537977326736
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 17:34
>> pulsar_precommit_cpp_ws-cleanup_1537983268177
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 22:04
>> pulsar_precommit_cpp_ws-cleanup_1538085846793
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 28 00:20
>> pulsar_precommit_cpp_ws-cleanup_1538094024821
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 28 17:42
>> pulsar_precommit_cpp_ws-cleanup_1538156543343
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:02
>> pulsar_precommit_integrationtests_ws-cleanup_1536298125841
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:00
>> pulsar_precommit_integrationtests_ws-cleanup_1536968128660
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 18 00:43
>> pulsar_precommit_integrationtests_ws-cleanup_1537218541598
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
>> pulsar_precommit_integrationtests_ws-cleanup_1537331703324
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 22 00:43
>> pulsar_precommit_integrationtests_ws-cleanup_1537553748859
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 00:43
>> pulsar_precommit_integrationtests_ws-cleanup_1537730402937
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 07:48
>> pulsar_precommit_integrationtests_ws-cleanup_1537775336586
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 23:30
>> pulsar_precommit_integrationtests_ws-cleanup_1537831851711
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 23:08
>> pulsar_precommit_integrationtests_ws-cleanup_1537916928485
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 15:21
>> pulsar_precommit_integrationtests_ws-cleanup_1538061702243
>>
>>
>> -Chris
>> ASF Infra
>>
>>
>>
>>
>>
>>> On Sep 30, 2018, at 3:12 PM, Chris Thistlethwaite <
>> cthistlego@outlook.com> wrote:
>>>
>>> I've been working on H21 and H31. They both had a some docker issues
>> where the process was locked and I couldn't do anything with a few
>> containers that were over 8 days old.
>>>
>>> Thought we might be good this morning so I threw them back into
>> rotation. Guess not, so I'll take them back out.
>>>
>>> -Chris T.
>>> ________________________________
>>> From: Mike Jumper <mj...@apache.org>
>>> Sent: Sunday, September 30, 2018 6:03:51 PM
>>> To: builds@apache.org
>>> Subject: No space on H21 (Was: Fwd: Build failed in Jenkins:
>> guacamole-client-coverity #73)
>>>
>>> FYI - we're currently seeing disk space issues on H21.
>>>
>>> - Mike
>>>
>>>
>>> ---------- Forwarded message ---------
>>> From: Apache Jenkins Server <je...@builds.apache.org>
>>> Date: Sun, Sep 30, 2018, 14:37
>>> Subject: Build failed in Jenkins: guacamole-client-coverity #73
>>> To: <de...@guacamole.apache.org>
>>>
>>>
>>> See <
>>>
>> https://builds.apache.org/job/guacamole-client-coverity/73/display/redirect
>>>
>>>
>>> ------------------------------------------
>>> Started by user mjumper
>>> [EnvInject] - Loading node environment variables.
>>> Building remotely on H21 (ubuntu xenial) in workspace <
>>> https://builds.apache.org/job/guacamole-client-coverity/ws/>
>>> [WS-CLEANUP] Deleting project workspace...
>>> Cloning the remote Git repository
>>> Cloning repository
>>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>>>> git init <
>>>
>> https://builds.apache.org/job/guacamole-client-coverity/ws/guacamole-client
>>>
>>> # timeout=10
>>> Fetching upstream changes from
>>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>>>> git --version # timeout=10
>>>> git fetch --tags --progress
>>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>>> +refs/heads/*:refs/remotes/origin/*
>>> ERROR: Error cloning remote repo 'origin'
>>> hudson.plugins.git.GitException: Command "git fetch --tags --progress
>>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>>> +refs/heads/*:refs/remotes/origin/*" returned status code 128:
>>> stdout:
>>> stderr: remote: Counting objects: 62166, done.
>>> remote: Compressing objects: 0% (1/16096) remote: Compressing
>>> objects: 1% (161/16096) remote: Compressing objects: 2%
>>> (322/16096) remote: Compressing objects: 3% (483/16096)
>>> remote: Compressing objects: 4% (644/16096) remote:
>>> Compressing objects: 5% (805/16096) remote: Compressing
>>> objects: 6% (966/16096) remote: Compressing objects: 7%
>>> (1127/16096) remote: Compressing objects: 8% (1288/16096)
>>> remote: Compressing objects: 9% (1449/16096) remote:
>>> Compressing objects: 10% (1610/16096) remote: Compressing
>>> objects: 11% (1771/16096) remote: Compressing objects: 12%
>>> (1932/16096) remote: Compressing objects: 13% (2093/16096)
>>> remote: Compressing objects: 14% (2254/16096) remote:
>>> Compressing objects: 15% (2415/16096) remote: Compressing
>>> objects: 16% (2576/16096) remote: Compressing objects: 17%
>>> (2737/16096) remote: Compressing objects: 18% (2898/16096)
>>> remote: Compressing objects: 19% (3059/16096) remote:
>>> Compressing objects: 20% (3220/16096) remote: Compressing
>>> objects: 21% (3381/16096) remote: Compressing objects: 22%
>>> (3542/16096) remote: Compressing objects: 23% (3703/16096)
>>> remote: Compressing objects: 24% (3864/16096) remote:
>>> Compressing objects: 25% (4024/16096) remote: Compressing
>>> objects: 26% (4185/16096) remote: Compressing objects: 27%
>>> (4346/16096) remote: Compressing objects: 28% (4507/16096)
>>> remote: Compressing objects: 29% (4668/16096) remote:
>>> Compressing objects: 30% (4829/16096) remote: Compressing
>>> objects: 31% (4990/16096) remote: Compressing objects: 32%
>>> (5151/16096) remote: Compressing objects: 33% (5312/16096)
>>> remote: Compressing objects: 34% (5473/16096) remote:
>>> Compressing objects: 35% (5634/16096) remote: Compressing
>>> objects: 36% (5795/16096) remote: Compressing objects: 37%
>>> (5956/16096) remote: Compressing objects: 38% (6117/16096)
>>> remote: Compressing objects: 39% (6278/16096) remote:
>>> Compressing objects: 40% (6439/16096) remote: Compressing
>>> objects: 41% (6600/16096) remote: Compressing objects: 42%
>>> (6761/16096) remote: Compressing objects: 43% (6922/16096)
>>> remote: Compressing objects: 44% (7083/16096) remote:
>>> Compressing objects: 45% (7244/16096) remote: Compressing
>>> objects: 46% (7405/16096) remote: Compressing objects: 47%
>>> (7566/16096) remote: Compressing objects: 48% (7727/16096)
>>> remote: Compressing objects: 49% (7888/16096) remote:
>>> Compressing objects: 50% (8048/16096) remote: Compressing
>>> objects: 51% (8209/16096) remote: Compressing objects: 52%
>>> (8370/16096) remote: Compressing objects: 53% (8531/16096)
>>> remote: Compressing objects: 54% (8692/16096) remote:
>>> Compressing objects: 55% (8853/16096) remote: Compressing
>>> objects: 56% (9014/16096) remote: Compressing objects: 57%
>>> (9175/16096) remote: Compressing objects: 58% (9336/16096)
>>> remote: Compressing objects: 59% (9497/16096) remote:
>>> Compressing objects: 60% (9658/16096) remote: Compressing
>>> objects: 61% (9819/16096) remote: Compressing objects: 62%
>>> (9980/16096) remote: Compressing objects: 63% (10141/16096)
>>> remote: Compressing objects: 64% (10302/16096) remote:
>>> Compressing objects: 65% (10463/16096) remote: Compressing
>>> objects: 66% (10624/16096) remote: Compressing objects: 67%
>>> (10785/16096) remote: Compressing objects: 68% (10946/16096)
>>> remote: Compressing objects: 69% (11107/16096) remote:
>>> Compressing objects: 70% (11268/16096) remote: Compressing
>>> objects: 71% (11429/16096) remote: Compressing objects: 72%
>>> (11590/16096) remote: Compressing objects: 73% (11751/16096)
>>> remote: Compressing objects: 74% (11912/16096) remote:
>>> Compressing objects: 75% (12072/16096) remote: Compressing
>>> objects: 76% (12233/16096) remote: Compressing objects: 77%
>>> (12394/16096) remote: Compressing objects: 78% (12555/16096)
>>> remote: Compressing objects: 79% (12716/16096) remote:
>>> Compressing objects: 80% (12877/16096) remote: Compressing
>>> objects: 81% (13038/16096) remote: Compressing objects: 82%
>>> (13199/16096) remote: Compressing objects: 83% (13360/16096)
>>> remote: Compressing objects: 84% (13521/16096) remote:
>>> Compressing objects: 85% (13682/16096) remote: Compressing
>>> objects: 86% (13843/16096) remote: Compressing objects: 87%
>>> (14004/16096) remote: Compressing objects: 88% (14165/16096)
>>> remote: Compressing objects: 89% (14326/16096) remote:
>>> Compressing objects: 90% (14487/16096) remote: Compressing
>>> objects: 91% (14648/16096) remote: Compressing objects: 92%
>>> (14809/16096) remote: Compressing objects: 93% (14970/16096)
>>> remote: Compressing objects: 94% (15131/16096) remote:
>>> Compressing objects: 95% (15292/16096) remote: Compressing
>>> objects: 96% (15453/16096) remote: Compressing objects: 97%
>>> (15614/16096) remote: Compressing objects: 98% (15775/16096)
>>> remote: Compressing objects: 99% (15936/16096) remote:
>>> Compressing objects: 100% (16096/16096) remote: Compressing
>>> objects: 100% (16096/16096), done.
>>> Receiving objects: 0% (1/62166) Receiving objects: 1% (622/62166)
>>> Receiving objects: 2% (1244/62166) Receiving objects: 3%
>>> (1865/62166) Receiving objects: 4% (2487/62166) Receiving objects:
>>> 5% (3109/62166) Receiving objects: 6% (3730/62166) Receiving
>>> objects: 7% (4352/62166) Receiving objects: 8% (4974/62166)
>>> Receiving objects: 9% (5595/62166) Receiving objects: 10%
>>> (6217/62166) Receiving objects: 11% (6839/62166) Receiving objects:
>>> 12% (7460/62166) Receiving objects: 13% (8082/62166) Receiving
>>> objects: 14% (8704/62166) Receiving objects: 15% (9325/62166)
>>> Receiving objects: 16% (9947/62166) Receiving objects: 17%
>>> (10569/62166) Receiving objects: 18% (11190/62166) Receiving
>> objects:
>>> 19% (11812/62166) Receiving objects: 20% (12434/62166) Receiving
>>> objects: 21% (13055/62166) Receiving objects: 22% (13677/62166)
>>> fatal: write error: No space left on device
>>> fatal: index-pack failed
>>>
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1721)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:72)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:405)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:614)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
>>> at hudson.remoting.UserRequest.perform(UserRequest.java:212)
>>> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
>>> at hudson.remoting.Request$2.run(Request.java:369)
>>> at
>>>
>> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> at
>>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> at
>>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> at java.lang.Thread.run(Thread.java:748)
>>> Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
>> call
>>> to H21
>>> at
>>> hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
>>> at
>>>
>> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
>>> at hudson.remoting.Channel.call(Channel.java:955)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
>>> at sun.reflect.GeneratedMethodAccessor918.invoke(Unknown
>>> Source)
>>> at
>>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
>>> at com.sun.proxy.$Proxy142.execute(Unknown Source)
>>> at
>>> hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1146)
>>> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
>>> at hudson.scm.SCM.checkout(SCM.java:504)
>>> at
>>> hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>>> at
>>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>>> at
>>> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>>> at
>>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>>> at hudson.model.Run.execute(Run.java:1794)
>>> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>>> at
>>> hudson.model.ResourceController.execute(ResourceController.java:97)
>>> at hudson.model.Executor.run(Executor.java:429)
>>> ERROR: Error cloning remote repo 'origin'
>>> Retrying after 10 seconds
>>>> git rev-parse --is-inside-work-tree # timeout=10
>>> Fetching changes from the remote Git repository
>>>> git config remote.origin.url
>>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git #
>> timeout=10
>>> Fetching upstream changes from
>>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>>>> git --version # timeout=10
>>>> git fetch --tags --progress
>>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>>> +refs/heads/*:refs/remotes/origin/*
>>> ERROR: Error fetching remote repo 'origin'
>>> hudson.plugins.git.GitException: Failed to fetch from
>>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>>> at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
>>> at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
>>> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
>>> at hudson.scm.SCM.checkout(SCM.java:504)
>>> at
>> hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>>> at
>>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>>> at
>>> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>>> at
>>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>>> at hudson.model.Run.execute(Run.java:1794)
>>> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>>> at
>>> hudson.model.ResourceController.execute(ResourceController.java:97)
>>> at hudson.model.Executor.run(Executor.java:429)
>>> Caused by: hudson.plugins.git.GitException: Command "git fetch --tags
>>> --progress https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>>> +refs/heads/*:refs/remotes/origin/*" returned status code 128:
>>> stdout:
>>> stderr: remote: Counting objects: 62166, done.
>>> remote: Compressing objects: 0% (1/16096) remote: Compressing
>>> objects: 1% (161/16096) remote: Compressing objects: 2%
>>> (322/16096) remote: Compressing objects: 3% (483/16096)
>>> remote: Compressing objects: 4% (644/16096) remote:
>>> Compressing objects: 5% (805/16096) remote: Compressing
>>> objects: 6% (966/16096) remote: Compressing objects: 7%
>>> (1127/16096) remote: Compressing objects: 8% (1288/16096)
>>> remote: Compressing objects: 9% (1449/16096) remote:
>>> Compressing objects: 10% (1610/16096) remote: Compressing
>>> objects: 11% (1771/16096) remote: Compressing objects: 12%
>>> (1932/16096) remote: Compressing objects: 13% (2093/16096)
>>> remote: Compressing objects: 14% (2254/16096) remote:
>>> Compressing objects: 15% (2415/16096) remote: Compressing
>>> objects: 16% (2576/16096) remote: Compressing objects: 17%
>>> (2737/16096) remote: Compressing objects: 18% (2898/16096)
>>> remote: Compressing objects: 19% (3059/16096) remote:
>>> Compressing objects: 20% (3220/16096) remote: Compressing
>>> objects: 21% (3381/16096) remote: Compressing objects: 22%
>>> (3542/16096) remote: Compressing objects: 23% (3703/16096)
>>> remote: Compressing objects: 24% (3864/16096) remote:
>>> Compressing objects: 25% (4024/16096) remote: Compressing
>>> objects: 26% (4185/16096) remote: Compressing objects: 27%
>>> (4346/16096) remote: Compressing objects: 28% (4507/16096)
>>> remote: Compressing objects: 29% (4668/16096) remote:
>>> Compressing objects: 30% (4829/16096) remote: Compressing
>>> objects: 31% (4990/16096) remote: Compressing objects: 32%
>>> (5151/16096) remote: Compressing objects: 33% (5312/16096)
>>> remote: Compressing objects: 34% (5473/16096) remote:
>>> Compressing objects: 35% (5634/16096) remote: Compressing
>>> objects: 36% (5795/16096) remote: Compressing objects: 37%
>>> (5956/16096) remote: Compressing objects: 38% (6117/16096)
>>> remote: Compressing objects: 39% (6278/16096) remote:
>>> Compressing objects: 40% (6439/16096) remote: Compressing
>>> objects: 41% (6600/16096) remote: Compressing objects: 42%
>>> (6761/16096) remote: Compressing objects: 43% (6922/16096)
>>> remote: Compressing objects: 44% (7083/16096) remote:
>>> Compressing objects: 45% (7244/16096) remote: Compressing
>>> objects: 46% (7405/16096) remote: Compressing objects: 47%
>>> (7566/16096) remote: Compressing objects: 48% (7727/16096)
>>> remote: Compressing objects: 49% (7888/16096) remote:
>>> Compressing objects: 50% (8048/16096) remote: Compressing
>>> objects: 51% (8209/16096) remote: Compressing objects: 52%
>>> (8370/16096) remote: Compressing objects: 53% (8531/16096)
>>> remote: Compressing objects: 54% (8692/16096) remote:
>>> Compressing objects: 55% (8853/16096) remote: Compressing
>>> objects: 56% (9014/16096) remote: Compressing objects: 57%
>>> (9175/16096) remote: Compressing objects: 58% (9336/16096)
>>> remote: Compressing objects: 59% (9497/16096) remote:
>>> Compressing objects: 60% (9658/16096) remote: Compressing
>>> objects: 61% (9819/16096) remote: Compressing objects: 62%
>>> (9980/16096) remote: Compressing objects: 63% (10141/16096)
>>> remote: Compressing objects: 64% (10302/16096) remote:
>>> Compressing objects: 65% (10463/16096) remote: Compressing
>>> objects: 66% (10624/16096) remote: Compressing objects: 67%
>>> (10785/16096) remote: Compressing objects: 68% (10946/16096)
>>> remote: Compressing objects: 69% (11107/16096) remote:
>>> Compressing objects: 70% (11268/16096) remote: Compressing
>>> objects: 71% (11429/16096) remote: Compressing objects: 72%
>>> (11590/16096) remote: Compressing objects: 73% (11751/16096)
>>> remote: Compressing objects: 74% (11912/16096) remote:
>>> Compressing objects: 75% (12072/16096) remote: Compressing
>>> objects: 76% (12233/16096) remote: Compressing objects: 77%
>>> (12394/16096) remote: Compressing objects: 78% (12555/16096)
>>> remote: Compressing objects: 79% (12716/16096) remote:
>>> Compressing objects: 80% (12877/16096) remote: Compressing
>>> objects: 81% (13038/16096) remote: Compressing objects: 82%
>>> (13199/16096) remote: Compressing objects: 83% (13360/16096)
>>> remote: Compressing objects: 84% (13521/16096) remote:
>>> Compressing objects: 85% (13682/16096) remote: Compressing
>>> objects: 86% (13843/16096) remote: Compressing objects: 87%
>>> (14004/16096) remote: Compressing objects: 88% (14165/16096)
>>> remote: Compressing objects: 89% (14326/16096) remote:
>>> Compressing objects: 90% (14487/16096) remote: Compressing
>>> objects: 91% (14648/16096) remote: Compressing objects: 92%
>>> (14809/16096) remote: Compressing objects: 93% (14970/16096)
>>> remote: Compressing objects: 94% (15131/16096) remote:
>>> Compressing objects: 95% (15292/16096) remote: Compressing
>>> objects: 96% (15453/16096) remote: Compressing objects: 97%
>>> (15614/16096) remote: Compressing objects: 98% (15775/16096)
>>> remote: Compressing objects: 99% (15936/16096) remote:
>>> Compressing objects: 100% (16096/16096) remote: Compressing
>>> objects: 100% (16096/16096), done.
>>> Receiving objects: 0% (1/62166) fatal: write error: No space left on
>>> device
>>> fatal: index-pack failed
>>>
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1721)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:72)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:405)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
>>> at hudson.remoting.UserRequest.perform(UserRequest.java:212)
>>> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
>>> at hudson.remoting.Request$2.run(Request.java:369)
>>> at
>>>
>> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> at
>>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> at
>>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> at java.lang.Thread.run(Thread.java:748)
>>> Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
>> call
>>> to H21
>>> at
>>> hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
>>> at
>>>
>> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
>>> at hudson.remoting.Channel.call(Channel.java:955)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
>>> at sun.reflect.GeneratedMethodAccessor918.invoke(Unknown
>>> Source)
>>> at
>>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
>>> at com.sun.proxy.$Proxy119.execute(Unknown Source)
>>> at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:886)
>>> at
>>> hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
>>> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
>>> at hudson.scm.SCM.checkout(SCM.java:504)
>>> at
>>> hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>>> at
>>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>>> at
>>> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>>> at
>>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>>> at hudson.model.Run.execute(Run.java:1794)
>>> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>>> at
>>> hudson.model.ResourceController.execute(ResourceController.java:97)
>>> at hudson.model.Executor.run(Executor.java:429)
>>> ERROR: Error fetching remote repo 'origin'
>>> Retrying after 10 seconds
>>>> git rev-parse --is-inside-work-tree # timeout=10
>>> Fetching changes from the remote Git repository
>>>> git config remote.origin.url
>>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git #
>> timeout=10
>>> ERROR: Error fetching remote repo 'origin'
>>> hudson.plugins.git.GitException: Failed to fetch from
>>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>>> at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
>>> at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
>>> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
>>> at hudson.scm.SCM.checkout(SCM.java:504)
>>> at
>> hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>>> at
>>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>>> at
>>> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>>> at
>>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>>> at hudson.model.Run.execute(Run.java:1794)
>>> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>>> at
>>> hudson.model.ResourceController.execute(ResourceController.java:97)
>>> at hudson.model.Executor.run(Executor.java:429)
>>> Caused by: hudson.plugins.git.GitException: Command "git config
>>> remote.origin.url
>>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git" returned
>>> status code 4:
>>> stdout:
>>> stderr: error: failed to write new configuration file <
>>>
>> https://builds.apache.org/job/guacamole-client-coverity/ws/guacamole-client/.git/config.lock
>>>>
>>>
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1970)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1966)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1597)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1609)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.setRemoteUrl(CliGitAPIImpl.java:1243)
>>> at hudson.plugins.git.GitAPI.setRemoteUrl(GitAPI.java:160)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>> at
>>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>> at
>>>
>> hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:929)
>>> at
>>>
>> hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:903)
>>> at
>>>
>> hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:855)
>>> at hudson.remoting.UserRequest.perform(UserRequest.java:212)
>>> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
>>> at hudson.remoting.Request$2.run(Request.java:369)
>>> at
>>>
>> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> at
>>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> at
>>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> at java.lang.Thread.run(Thread.java:748)
>>> Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
>> call
>>> to H21
>>> at
>>> hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
>>> at
>>>
>> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
>>> at hudson.remoting.Channel.call(Channel.java:955)
>>> at
>>>
>> hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:283)
>>> at com.sun.proxy.$Proxy118.setRemoteUrl(Unknown Source)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl.setRemoteUrl(RemoteGitImpl.java:295)
>>> at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:876)
>>> at
>>> hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
>>> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
>>> at hudson.scm.SCM.checkout(SCM.java:504)
>>> at
>>> hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>>> at
>>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>>> at
>>> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>>> at
>>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>>> at hudson.model.Run.execute(Run.java:1794)
>>> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>>> at
>>> hudson.model.ResourceController.execute(ResourceController.java:97)
>>> at hudson.model.Executor.run(Executor.java:429)
>>> ERROR: Error fetching remote repo 'origin'
>>
>> --
> Matteo Merli
> <mm...@apache.org>
Re: No space on H21 (Was: Fwd: Build failed in Jenkins:
guacamole-client-coverity #73)
Posted by Chris Lambertus <cm...@apache.org>.
Well, I had thought it was something specific to the build, since it only happens for a couple projects, but upon further investigation, there appears to be an open issue related to this:
https://issues.jenkins-ci.org/browse/JENKINS-53579 <https://issues.jenkins-ci.org/browse/JENKINS-53579>
I will need to look into this further. In the meantime, Infra will need to implement a cronjob that purges the errant ws-cleanup folders.
-Chris
> On Oct 1, 2018, at 11:30 AM, Matteo Merli <ma...@gmail.com> wrote:
>
> Chris, are these `ws-cleanup` jobs something that is created internally by
> Jenkins? The job we have is `pulsar_precommit_cpp`.
>
> Is there any setting in Jenkins that could explain that these files are not
> removed?
>
> On https://builds.apache.org/job/pulsar_precommit_cpp/configure , we
> already have "Days to keep builds == 1".
> For "Days to keep artifacts" is left blank. Should that be set to 1 day
> too?
>
> Any other hint at things we can try to ensure builds are cleaned up
> promptly?
>
> Thanks,
> Matteo
>
> On Sun, Sep 30, 2018 at 3:57 PM Chris Lambertus <cm...@apache.org> wrote:
>
>> The main culprit on H31 appears to be pulsar ws-cleanup jobs consuming
>> almost 40% of the disk. (The problem on H21 was something docker-related.)
>>
>>
>> This is the second project I have found (Brooklyn was the first) which is
>> leaving ws-cleanup directories all over the workspace and filling up the
>> disks. Each one of these was 500+ MB. Pulsar, please address ASAP.
>>
>>
>>
>>
>> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:19
>> pulsar_precommit_cpp@2_ws-cleanup_1535737112280
>> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:18
>> pulsar_precommit_cpp@2_ws-cleanup_1536174576053
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 06:04
>> pulsar_precommit_cpp@2_ws-cleanup_1537292379178
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 22:45
>> pulsar_precommit_cpp@2_ws-cleanup_1537397120862
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
>> pulsar_precommit_cpp@2_ws-cleanup_1537403378279
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 21 23:31
>> pulsar_precommit_cpp@2_ws-cleanup_1537572701829
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 26 16:04
>> pulsar_precommit_cpp@2_ws-cleanup_1537977879341
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 22:23
>> pulsar_precommit_cpp@2_ws-cleanup_1538087035027
>> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:14
>> pulsar_precommit_cpp_ws-cleanup_1535699830230
>> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:16
>> pulsar_precommit_cpp_ws-cleanup_1535735511423
>> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:13
>> pulsar_precommit_cpp_ws-cleanup_1535786064397
>> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:18
>> pulsar_precommit_cpp_ws-cleanup_1536096963586
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 21 06:47
>> pulsar_precommit_cpp_ws-cleanup_1536114938910
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 21 06:45
>> pulsar_precommit_cpp_ws-cleanup_1536173165909
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:38
>> pulsar_precommit_cpp_ws-cleanup_1536175584418
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:50
>> pulsar_precommit_cpp_ws-cleanup_1536180707506
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:38
>> pulsar_precommit_cpp_ws-cleanup_1536182927940
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:51
>> pulsar_precommit_cpp_ws-cleanup_1536185193716
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:50
>> pulsar_precommit_cpp_ws-cleanup_1536273709029
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 05:59
>> pulsar_precommit_cpp_ws-cleanup_1536316964794
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:06
>> pulsar_precommit_cpp_ws-cleanup_1536675650893
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 28 06:08
>> pulsar_precommit_cpp_ws-cleanup_1536879616962
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 30 19:53
>> pulsar_precommit_cpp_ws-cleanup_1536891056968
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 30 19:47
>> pulsar_precommit_cpp_ws-cleanup_1536966466683
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 18 00:43
>> pulsar_precommit_cpp_ws-cleanup_1537216387387
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 00:45
>> pulsar_precommit_cpp_ws-cleanup_1537292309413
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 00:45
>> pulsar_precommit_cpp_ws-cleanup_1537308526251
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
>> pulsar_precommit_cpp_ws-cleanup_1537331780420
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
>> pulsar_precommit_cpp_ws-cleanup_1537379886715
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
>> pulsar_precommit_cpp_ws-cleanup_1537386197893
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
>> pulsar_precommit_cpp_ws-cleanup_1537397073464
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
>> pulsar_precommit_cpp_ws-cleanup_1537403247575
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:43
>> pulsar_precommit_cpp_ws-cleanup_1537413954888
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
>> pulsar_precommit_cpp_ws-cleanup_1537417322151
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
>> pulsar_precommit_cpp_ws-cleanup_1537421381129
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:44
>> pulsar_precommit_cpp_ws-cleanup_1537469348852
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
>> pulsar_precommit_cpp_ws-cleanup_1537489399468
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 22 00:44
>> pulsar_precommit_cpp_ws-cleanup_1537566528660
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 23:12
>> pulsar_precommit_cpp_ws-cleanup_1537571549978
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 22 00:44
>> pulsar_precommit_cpp_ws-cleanup_1537573605258
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 00:43
>> pulsar_precommit_cpp_ws-cleanup_1537855939828
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 11:18
>> pulsar_precommit_cpp_ws-cleanup_1537874305575
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 12:41
>> pulsar_precommit_cpp_ws-cleanup_1537879296622
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 01:39
>> pulsar_precommit_cpp_ws-cleanup_1537925955967
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 26 13:17
>> pulsar_precommit_cpp_ws-cleanup_1537967826838
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 15:55
>> pulsar_precommit_cpp_ws-cleanup_1537977326736
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 17:34
>> pulsar_precommit_cpp_ws-cleanup_1537983268177
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 22:04
>> pulsar_precommit_cpp_ws-cleanup_1538085846793
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 28 00:20
>> pulsar_precommit_cpp_ws-cleanup_1538094024821
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 28 17:42
>> pulsar_precommit_cpp_ws-cleanup_1538156543343
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:02
>> pulsar_precommit_integrationtests_ws-cleanup_1536298125841
>> drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:00
>> pulsar_precommit_integrationtests_ws-cleanup_1536968128660
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 18 00:43
>> pulsar_precommit_integrationtests_ws-cleanup_1537218541598
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
>> pulsar_precommit_integrationtests_ws-cleanup_1537331703324
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 22 00:43
>> pulsar_precommit_integrationtests_ws-cleanup_1537553748859
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 00:43
>> pulsar_precommit_integrationtests_ws-cleanup_1537730402937
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 07:48
>> pulsar_precommit_integrationtests_ws-cleanup_1537775336586
>> drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 23:30
>> pulsar_precommit_integrationtests_ws-cleanup_1537831851711
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 23:08
>> pulsar_precommit_integrationtests_ws-cleanup_1537916928485
>> drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 15:21
>> pulsar_precommit_integrationtests_ws-cleanup_1538061702243
>>
>>
>> -Chris
>> ASF Infra
>>
>>
>>
>>
>>
>>> On Sep 30, 2018, at 3:12 PM, Chris Thistlethwaite <
>> cthistlego@outlook.com> wrote:
>>>
>>> I've been working on H21 and H31. They both had a some docker issues
>> where the process was locked and I couldn't do anything with a few
>> containers that were over 8 days old.
>>>
>>> Thought we might be good this morning so I threw them back into
>> rotation. Guess not, so I'll take them back out.
>>>
>>> -Chris T.
>>> ________________________________
>>> From: Mike Jumper <mj...@apache.org>
>>> Sent: Sunday, September 30, 2018 6:03:51 PM
>>> To: builds@apache.org
>>> Subject: No space on H21 (Was: Fwd: Build failed in Jenkins:
>> guacamole-client-coverity #73)
>>>
>>> FYI - we're currently seeing disk space issues on H21.
>>>
>>> - Mike
>>>
>>>
>>> ---------- Forwarded message ---------
>>> From: Apache Jenkins Server <je...@builds.apache.org>
>>> Date: Sun, Sep 30, 2018, 14:37
>>> Subject: Build failed in Jenkins: guacamole-client-coverity #73
>>> To: <de...@guacamole.apache.org>
>>>
>>>
>>> See <
>>>
>> https://builds.apache.org/job/guacamole-client-coverity/73/display/redirect
>>>
>>>
>>> ------------------------------------------
>>> Started by user mjumper
>>> [EnvInject] - Loading node environment variables.
>>> Building remotely on H21 (ubuntu xenial) in workspace <
>>> https://builds.apache.org/job/guacamole-client-coverity/ws/>
>>> [WS-CLEANUP] Deleting project workspace...
>>> Cloning the remote Git repository
>>> Cloning repository
>>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>>>> git init <
>>>
>> https://builds.apache.org/job/guacamole-client-coverity/ws/guacamole-client
>>>
>>> # timeout=10
>>> Fetching upstream changes from
>>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>>>> git --version # timeout=10
>>>> git fetch --tags --progress
>>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>>> +refs/heads/*:refs/remotes/origin/*
>>> ERROR: Error cloning remote repo 'origin'
>>> hudson.plugins.git.GitException: Command "git fetch --tags --progress
>>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>>> +refs/heads/*:refs/remotes/origin/*" returned status code 128:
>>> stdout:
>>> stderr: remote: Counting objects: 62166, done.
>>> remote: Compressing objects: 0% (1/16096) remote: Compressing
>>> objects: 1% (161/16096) remote: Compressing objects: 2%
>>> (322/16096) remote: Compressing objects: 3% (483/16096)
>>> remote: Compressing objects: 4% (644/16096) remote:
>>> Compressing objects: 5% (805/16096) remote: Compressing
>>> objects: 6% (966/16096) remote: Compressing objects: 7%
>>> (1127/16096) remote: Compressing objects: 8% (1288/16096)
>>> remote: Compressing objects: 9% (1449/16096) remote:
>>> Compressing objects: 10% (1610/16096) remote: Compressing
>>> objects: 11% (1771/16096) remote: Compressing objects: 12%
>>> (1932/16096) remote: Compressing objects: 13% (2093/16096)
>>> remote: Compressing objects: 14% (2254/16096) remote:
>>> Compressing objects: 15% (2415/16096) remote: Compressing
>>> objects: 16% (2576/16096) remote: Compressing objects: 17%
>>> (2737/16096) remote: Compressing objects: 18% (2898/16096)
>>> remote: Compressing objects: 19% (3059/16096) remote:
>>> Compressing objects: 20% (3220/16096) remote: Compressing
>>> objects: 21% (3381/16096) remote: Compressing objects: 22%
>>> (3542/16096) remote: Compressing objects: 23% (3703/16096)
>>> remote: Compressing objects: 24% (3864/16096) remote:
>>> Compressing objects: 25% (4024/16096) remote: Compressing
>>> objects: 26% (4185/16096) remote: Compressing objects: 27%
>>> (4346/16096) remote: Compressing objects: 28% (4507/16096)
>>> remote: Compressing objects: 29% (4668/16096) remote:
>>> Compressing objects: 30% (4829/16096) remote: Compressing
>>> objects: 31% (4990/16096) remote: Compressing objects: 32%
>>> (5151/16096) remote: Compressing objects: 33% (5312/16096)
>>> remote: Compressing objects: 34% (5473/16096) remote:
>>> Compressing objects: 35% (5634/16096) remote: Compressing
>>> objects: 36% (5795/16096) remote: Compressing objects: 37%
>>> (5956/16096) remote: Compressing objects: 38% (6117/16096)
>>> remote: Compressing objects: 39% (6278/16096) remote:
>>> Compressing objects: 40% (6439/16096) remote: Compressing
>>> objects: 41% (6600/16096) remote: Compressing objects: 42%
>>> (6761/16096) remote: Compressing objects: 43% (6922/16096)
>>> remote: Compressing objects: 44% (7083/16096) remote:
>>> Compressing objects: 45% (7244/16096) remote: Compressing
>>> objects: 46% (7405/16096) remote: Compressing objects: 47%
>>> (7566/16096) remote: Compressing objects: 48% (7727/16096)
>>> remote: Compressing objects: 49% (7888/16096) remote:
>>> Compressing objects: 50% (8048/16096) remote: Compressing
>>> objects: 51% (8209/16096) remote: Compressing objects: 52%
>>> (8370/16096) remote: Compressing objects: 53% (8531/16096)
>>> remote: Compressing objects: 54% (8692/16096) remote:
>>> Compressing objects: 55% (8853/16096) remote: Compressing
>>> objects: 56% (9014/16096) remote: Compressing objects: 57%
>>> (9175/16096) remote: Compressing objects: 58% (9336/16096)
>>> remote: Compressing objects: 59% (9497/16096) remote:
>>> Compressing objects: 60% (9658/16096) remote: Compressing
>>> objects: 61% (9819/16096) remote: Compressing objects: 62%
>>> (9980/16096) remote: Compressing objects: 63% (10141/16096)
>>> remote: Compressing objects: 64% (10302/16096) remote:
>>> Compressing objects: 65% (10463/16096) remote: Compressing
>>> objects: 66% (10624/16096) remote: Compressing objects: 67%
>>> (10785/16096) remote: Compressing objects: 68% (10946/16096)
>>> remote: Compressing objects: 69% (11107/16096) remote:
>>> Compressing objects: 70% (11268/16096) remote: Compressing
>>> objects: 71% (11429/16096) remote: Compressing objects: 72%
>>> (11590/16096) remote: Compressing objects: 73% (11751/16096)
>>> remote: Compressing objects: 74% (11912/16096) remote:
>>> Compressing objects: 75% (12072/16096) remote: Compressing
>>> objects: 76% (12233/16096) remote: Compressing objects: 77%
>>> (12394/16096) remote: Compressing objects: 78% (12555/16096)
>>> remote: Compressing objects: 79% (12716/16096) remote:
>>> Compressing objects: 80% (12877/16096) remote: Compressing
>>> objects: 81% (13038/16096) remote: Compressing objects: 82%
>>> (13199/16096) remote: Compressing objects: 83% (13360/16096)
>>> remote: Compressing objects: 84% (13521/16096) remote:
>>> Compressing objects: 85% (13682/16096) remote: Compressing
>>> objects: 86% (13843/16096) remote: Compressing objects: 87%
>>> (14004/16096) remote: Compressing objects: 88% (14165/16096)
>>> remote: Compressing objects: 89% (14326/16096) remote:
>>> Compressing objects: 90% (14487/16096) remote: Compressing
>>> objects: 91% (14648/16096) remote: Compressing objects: 92%
>>> (14809/16096) remote: Compressing objects: 93% (14970/16096)
>>> remote: Compressing objects: 94% (15131/16096) remote:
>>> Compressing objects: 95% (15292/16096) remote: Compressing
>>> objects: 96% (15453/16096) remote: Compressing objects: 97%
>>> (15614/16096) remote: Compressing objects: 98% (15775/16096)
>>> remote: Compressing objects: 99% (15936/16096) remote:
>>> Compressing objects: 100% (16096/16096) remote: Compressing
>>> objects: 100% (16096/16096), done.
>>> Receiving objects: 0% (1/62166) Receiving objects: 1% (622/62166)
>>> Receiving objects: 2% (1244/62166) Receiving objects: 3%
>>> (1865/62166) Receiving objects: 4% (2487/62166) Receiving objects:
>>> 5% (3109/62166) Receiving objects: 6% (3730/62166) Receiving
>>> objects: 7% (4352/62166) Receiving objects: 8% (4974/62166)
>>> Receiving objects: 9% (5595/62166) Receiving objects: 10%
>>> (6217/62166) Receiving objects: 11% (6839/62166) Receiving objects:
>>> 12% (7460/62166) Receiving objects: 13% (8082/62166) Receiving
>>> objects: 14% (8704/62166) Receiving objects: 15% (9325/62166)
>>> Receiving objects: 16% (9947/62166) Receiving objects: 17%
>>> (10569/62166) Receiving objects: 18% (11190/62166) Receiving
>> objects:
>>> 19% (11812/62166) Receiving objects: 20% (12434/62166) Receiving
>>> objects: 21% (13055/62166) Receiving objects: 22% (13677/62166)
>>> fatal: write error: No space left on device
>>> fatal: index-pack failed
>>>
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1721)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:72)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:405)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:614)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
>>> at hudson.remoting.UserRequest.perform(UserRequest.java:212)
>>> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
>>> at hudson.remoting.Request$2.run(Request.java:369)
>>> at
>>>
>> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> at
>>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> at
>>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> at java.lang.Thread.run(Thread.java:748)
>>> Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
>> call
>>> to H21
>>> at
>>> hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
>>> at
>>>
>> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
>>> at hudson.remoting.Channel.call(Channel.java:955)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
>>> at sun.reflect.GeneratedMethodAccessor918.invoke(Unknown
>>> Source)
>>> at
>>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
>>> at com.sun.proxy.$Proxy142.execute(Unknown Source)
>>> at
>>> hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1146)
>>> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
>>> at hudson.scm.SCM.checkout(SCM.java:504)
>>> at
>>> hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>>> at
>>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>>> at
>>> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>>> at
>>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>>> at hudson.model.Run.execute(Run.java:1794)
>>> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>>> at
>>> hudson.model.ResourceController.execute(ResourceController.java:97)
>>> at hudson.model.Executor.run(Executor.java:429)
>>> ERROR: Error cloning remote repo 'origin'
>>> Retrying after 10 seconds
>>>> git rev-parse --is-inside-work-tree # timeout=10
>>> Fetching changes from the remote Git repository
>>>> git config remote.origin.url
>>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git #
>> timeout=10
>>> Fetching upstream changes from
>>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>>>> git --version # timeout=10
>>>> git fetch --tags --progress
>>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>>> +refs/heads/*:refs/remotes/origin/*
>>> ERROR: Error fetching remote repo 'origin'
>>> hudson.plugins.git.GitException: Failed to fetch from
>>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>>> at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
>>> at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
>>> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
>>> at hudson.scm.SCM.checkout(SCM.java:504)
>>> at
>> hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>>> at
>>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>>> at
>>> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>>> at
>>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>>> at hudson.model.Run.execute(Run.java:1794)
>>> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>>> at
>>> hudson.model.ResourceController.execute(ResourceController.java:97)
>>> at hudson.model.Executor.run(Executor.java:429)
>>> Caused by: hudson.plugins.git.GitException: Command "git fetch --tags
>>> --progress https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>>> +refs/heads/*:refs/remotes/origin/*" returned status code 128:
>>> stdout:
>>> stderr: remote: Counting objects: 62166, done.
>>> remote: Compressing objects: 0% (1/16096) remote: Compressing
>>> objects: 1% (161/16096) remote: Compressing objects: 2%
>>> (322/16096) remote: Compressing objects: 3% (483/16096)
>>> remote: Compressing objects: 4% (644/16096) remote:
>>> Compressing objects: 5% (805/16096) remote: Compressing
>>> objects: 6% (966/16096) remote: Compressing objects: 7%
>>> (1127/16096) remote: Compressing objects: 8% (1288/16096)
>>> remote: Compressing objects: 9% (1449/16096) remote:
>>> Compressing objects: 10% (1610/16096) remote: Compressing
>>> objects: 11% (1771/16096) remote: Compressing objects: 12%
>>> (1932/16096) remote: Compressing objects: 13% (2093/16096)
>>> remote: Compressing objects: 14% (2254/16096) remote:
>>> Compressing objects: 15% (2415/16096) remote: Compressing
>>> objects: 16% (2576/16096) remote: Compressing objects: 17%
>>> (2737/16096) remote: Compressing objects: 18% (2898/16096)
>>> remote: Compressing objects: 19% (3059/16096) remote:
>>> Compressing objects: 20% (3220/16096) remote: Compressing
>>> objects: 21% (3381/16096) remote: Compressing objects: 22%
>>> (3542/16096) remote: Compressing objects: 23% (3703/16096)
>>> remote: Compressing objects: 24% (3864/16096) remote:
>>> Compressing objects: 25% (4024/16096) remote: Compressing
>>> objects: 26% (4185/16096) remote: Compressing objects: 27%
>>> (4346/16096) remote: Compressing objects: 28% (4507/16096)
>>> remote: Compressing objects: 29% (4668/16096) remote:
>>> Compressing objects: 30% (4829/16096) remote: Compressing
>>> objects: 31% (4990/16096) remote: Compressing objects: 32%
>>> (5151/16096) remote: Compressing objects: 33% (5312/16096)
>>> remote: Compressing objects: 34% (5473/16096) remote:
>>> Compressing objects: 35% (5634/16096) remote: Compressing
>>> objects: 36% (5795/16096) remote: Compressing objects: 37%
>>> (5956/16096) remote: Compressing objects: 38% (6117/16096)
>>> remote: Compressing objects: 39% (6278/16096) remote:
>>> Compressing objects: 40% (6439/16096) remote: Compressing
>>> objects: 41% (6600/16096) remote: Compressing objects: 42%
>>> (6761/16096) remote: Compressing objects: 43% (6922/16096)
>>> remote: Compressing objects: 44% (7083/16096) remote:
>>> Compressing objects: 45% (7244/16096) remote: Compressing
>>> objects: 46% (7405/16096) remote: Compressing objects: 47%
>>> (7566/16096) remote: Compressing objects: 48% (7727/16096)
>>> remote: Compressing objects: 49% (7888/16096) remote:
>>> Compressing objects: 50% (8048/16096) remote: Compressing
>>> objects: 51% (8209/16096) remote: Compressing objects: 52%
>>> (8370/16096) remote: Compressing objects: 53% (8531/16096)
>>> remote: Compressing objects: 54% (8692/16096) remote:
>>> Compressing objects: 55% (8853/16096) remote: Compressing
>>> objects: 56% (9014/16096) remote: Compressing objects: 57%
>>> (9175/16096) remote: Compressing objects: 58% (9336/16096)
>>> remote: Compressing objects: 59% (9497/16096) remote:
>>> Compressing objects: 60% (9658/16096) remote: Compressing
>>> objects: 61% (9819/16096) remote: Compressing objects: 62%
>>> (9980/16096) remote: Compressing objects: 63% (10141/16096)
>>> remote: Compressing objects: 64% (10302/16096) remote:
>>> Compressing objects: 65% (10463/16096) remote: Compressing
>>> objects: 66% (10624/16096) remote: Compressing objects: 67%
>>> (10785/16096) remote: Compressing objects: 68% (10946/16096)
>>> remote: Compressing objects: 69% (11107/16096) remote:
>>> Compressing objects: 70% (11268/16096) remote: Compressing
>>> objects: 71% (11429/16096) remote: Compressing objects: 72%
>>> (11590/16096) remote: Compressing objects: 73% (11751/16096)
>>> remote: Compressing objects: 74% (11912/16096) remote:
>>> Compressing objects: 75% (12072/16096) remote: Compressing
>>> objects: 76% (12233/16096) remote: Compressing objects: 77%
>>> (12394/16096) remote: Compressing objects: 78% (12555/16096)
>>> remote: Compressing objects: 79% (12716/16096) remote:
>>> Compressing objects: 80% (12877/16096) remote: Compressing
>>> objects: 81% (13038/16096) remote: Compressing objects: 82%
>>> (13199/16096) remote: Compressing objects: 83% (13360/16096)
>>> remote: Compressing objects: 84% (13521/16096) remote:
>>> Compressing objects: 85% (13682/16096) remote: Compressing
>>> objects: 86% (13843/16096) remote: Compressing objects: 87%
>>> (14004/16096) remote: Compressing objects: 88% (14165/16096)
>>> remote: Compressing objects: 89% (14326/16096) remote:
>>> Compressing objects: 90% (14487/16096) remote: Compressing
>>> objects: 91% (14648/16096) remote: Compressing objects: 92%
>>> (14809/16096) remote: Compressing objects: 93% (14970/16096)
>>> remote: Compressing objects: 94% (15131/16096) remote:
>>> Compressing objects: 95% (15292/16096) remote: Compressing
>>> objects: 96% (15453/16096) remote: Compressing objects: 97%
>>> (15614/16096) remote: Compressing objects: 98% (15775/16096)
>>> remote: Compressing objects: 99% (15936/16096) remote:
>>> Compressing objects: 100% (16096/16096) remote: Compressing
>>> objects: 100% (16096/16096), done.
>>> Receiving objects: 0% (1/62166) fatal: write error: No space left on
>>> device
>>> fatal: index-pack failed
>>>
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1721)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:72)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:405)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
>>> at hudson.remoting.UserRequest.perform(UserRequest.java:212)
>>> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
>>> at hudson.remoting.Request$2.run(Request.java:369)
>>> at
>>>
>> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> at
>>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> at
>>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> at java.lang.Thread.run(Thread.java:748)
>>> Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
>> call
>>> to H21
>>> at
>>> hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
>>> at
>>>
>> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
>>> at hudson.remoting.Channel.call(Channel.java:955)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
>>> at sun.reflect.GeneratedMethodAccessor918.invoke(Unknown
>>> Source)
>>> at
>>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
>>> at com.sun.proxy.$Proxy119.execute(Unknown Source)
>>> at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:886)
>>> at
>>> hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
>>> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
>>> at hudson.scm.SCM.checkout(SCM.java:504)
>>> at
>>> hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>>> at
>>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>>> at
>>> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>>> at
>>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>>> at hudson.model.Run.execute(Run.java:1794)
>>> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>>> at
>>> hudson.model.ResourceController.execute(ResourceController.java:97)
>>> at hudson.model.Executor.run(Executor.java:429)
>>> ERROR: Error fetching remote repo 'origin'
>>> Retrying after 10 seconds
>>>> git rev-parse --is-inside-work-tree # timeout=10
>>> Fetching changes from the remote Git repository
>>>> git config remote.origin.url
>>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git #
>> timeout=10
>>> ERROR: Error fetching remote repo 'origin'
>>> hudson.plugins.git.GitException: Failed to fetch from
>>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>>> at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
>>> at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
>>> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
>>> at hudson.scm.SCM.checkout(SCM.java:504)
>>> at
>> hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>>> at
>>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>>> at
>>> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>>> at
>>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>>> at hudson.model.Run.execute(Run.java:1794)
>>> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>>> at
>>> hudson.model.ResourceController.execute(ResourceController.java:97)
>>> at hudson.model.Executor.run(Executor.java:429)
>>> Caused by: hudson.plugins.git.GitException: Command "git config
>>> remote.origin.url
>>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git" returned
>>> status code 4:
>>> stdout:
>>> stderr: error: failed to write new configuration file <
>>>
>> https://builds.apache.org/job/guacamole-client-coverity/ws/guacamole-client/.git/config.lock
>>>>
>>>
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1970)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1966)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1597)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1609)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.setRemoteUrl(CliGitAPIImpl.java:1243)
>>> at hudson.plugins.git.GitAPI.setRemoteUrl(GitAPI.java:160)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>> at
>>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>> at
>>>
>> hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:929)
>>> at
>>>
>> hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:903)
>>> at
>>>
>> hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:855)
>>> at hudson.remoting.UserRequest.perform(UserRequest.java:212)
>>> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
>>> at hudson.remoting.Request$2.run(Request.java:369)
>>> at
>>>
>> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> at
>>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> at
>>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> at java.lang.Thread.run(Thread.java:748)
>>> Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
>> call
>>> to H21
>>> at
>>> hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
>>> at
>>>
>> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
>>> at hudson.remoting.Channel.call(Channel.java:955)
>>> at
>>>
>> hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:283)
>>> at com.sun.proxy.$Proxy118.setRemoteUrl(Unknown Source)
>>> at
>>>
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl.setRemoteUrl(RemoteGitImpl.java:295)
>>> at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:876)
>>> at
>>> hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
>>> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
>>> at hudson.scm.SCM.checkout(SCM.java:504)
>>> at
>>> hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>>> at
>>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>>> at
>>> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>>> at
>>>
>> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>>> at hudson.model.Run.execute(Run.java:1794)
>>> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>>> at
>>> hudson.model.ResourceController.execute(ResourceController.java:97)
>>> at hudson.model.Executor.run(Executor.java:429)
>>> ERROR: Error fetching remote repo 'origin'
>>
>> --
> Matteo Merli
> <mm...@apache.org>
Re: No space on H21 (Was: Fwd: Build failed in Jenkins:
guacamole-client-coverity #73)
Posted by Matteo Merli <ma...@gmail.com>.
Good point. We can try to make sure everything gets deleted when the build
is done.
On Mon, Oct 1, 2018 at 11:45 AM Ivan Kelly <iv...@apache.org> wrote:
> Could it be a problem with the cpp job creating everything as root so
> then jenkins isn't able to clean up?
>
> -Ivan
>
> On Mon, Oct 1, 2018 at 8:34 PM, Joe Francis <jo...@oath.com.invalid> wrote:
> > We should cleanup all the bk/zk dirs created by unit tests. Almost 16gig
> of
> > junk
> >
> > On Mon, Oct 1, 2018, 11:31 AM Matteo Merli <ma...@gmail.com>
> wrote:
> >
> >> Chris, are these `ws-cleanup` jobs something that is created internally
> by
> >> Jenkins? The job we have is `pulsar_precommit_cpp`.
> >>
> >> Is there any setting in Jenkins that could explain that these files are
> not
> >> removed?
> >>
> >> On https://builds.apache.org/job/pulsar_precommit_cpp/configure , we
> >> already have "Days to keep builds == 1".
> >> For "Days to keep artifacts" is left blank. Should that be set to 1 day
> >> too?
> >>
> >> Any other hint at things we can try to ensure builds are cleaned up
> >> promptly?
> >>
> >> Thanks,
> >> Matteo
> >>
> >> On Sun, Sep 30, 2018 at 3:57 PM Chris Lambertus <cm...@apache.org> wrote:
> >>
> >> > The main culprit on H31 appears to be pulsar ws-cleanup jobs consuming
> >> > almost 40% of the disk. (The problem on H21 was something
> >> docker-related.)
> >> >
> >> >
> >> > This is the second project I have found (Brooklyn was the first)
> which is
> >> > leaving ws-cleanup directories all over the workspace and filling up
> the
> >> > disks. Each one of these was 500+ MB. Pulsar, please address ASAP.
> >> >
> >> >
> >> >
> >> >
> >> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:19
> >> > pulsar_precommit_cpp@2_ws-cleanup_1535737112280
> >> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:18
> >> > pulsar_precommit_cpp@2_ws-cleanup_1536174576053
> >> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 06:04
> >> > pulsar_precommit_cpp@2_ws-cleanup_1537292379178
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 22:45
> >> > pulsar_precommit_cpp@2_ws-cleanup_1537397120862
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> >> > pulsar_precommit_cpp@2_ws-cleanup_1537403378279
> >> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 21 23:31
> >> > pulsar_precommit_cpp@2_ws-cleanup_1537572701829
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 26 16:04
> >> > pulsar_precommit_cpp@2_ws-cleanup_1537977879341
> >> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 22:23
> >> > pulsar_precommit_cpp@2_ws-cleanup_1538087035027
> >> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:14
> >> > pulsar_precommit_cpp_ws-cleanup_1535699830230
> >> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:16
> >> > pulsar_precommit_cpp_ws-cleanup_1535735511423
> >> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:13
> >> > pulsar_precommit_cpp_ws-cleanup_1535786064397
> >> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:18
> >> > pulsar_precommit_cpp_ws-cleanup_1536096963586
> >> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 21 06:47
> >> > pulsar_precommit_cpp_ws-cleanup_1536114938910
> >> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 21 06:45
> >> > pulsar_precommit_cpp_ws-cleanup_1536173165909
> >> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:38
> >> > pulsar_precommit_cpp_ws-cleanup_1536175584418
> >> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:50
> >> > pulsar_precommit_cpp_ws-cleanup_1536180707506
> >> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:38
> >> > pulsar_precommit_cpp_ws-cleanup_1536182927940
> >> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:51
> >> > pulsar_precommit_cpp_ws-cleanup_1536185193716
> >> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:50
> >> > pulsar_precommit_cpp_ws-cleanup_1536273709029
> >> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 05:59
> >> > pulsar_precommit_cpp_ws-cleanup_1536316964794
> >> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:06
> >> > pulsar_precommit_cpp_ws-cleanup_1536675650893
> >> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 28 06:08
> >> > pulsar_precommit_cpp_ws-cleanup_1536879616962
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 30 19:53
> >> > pulsar_precommit_cpp_ws-cleanup_1536891056968
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 30 19:47
> >> > pulsar_precommit_cpp_ws-cleanup_1536966466683
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 18 00:43
> >> > pulsar_precommit_cpp_ws-cleanup_1537216387387
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 00:45
> >> > pulsar_precommit_cpp_ws-cleanup_1537292309413
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 00:45
> >> > pulsar_precommit_cpp_ws-cleanup_1537308526251
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> >> > pulsar_precommit_cpp_ws-cleanup_1537331780420
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> >> > pulsar_precommit_cpp_ws-cleanup_1537379886715
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
> >> > pulsar_precommit_cpp_ws-cleanup_1537386197893
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
> >> > pulsar_precommit_cpp_ws-cleanup_1537397073464
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
> >> > pulsar_precommit_cpp_ws-cleanup_1537403247575
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:43
> >> > pulsar_precommit_cpp_ws-cleanup_1537413954888
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
> >> > pulsar_precommit_cpp_ws-cleanup_1537417322151
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
> >> > pulsar_precommit_cpp_ws-cleanup_1537421381129
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:44
> >> > pulsar_precommit_cpp_ws-cleanup_1537469348852
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
> >> > pulsar_precommit_cpp_ws-cleanup_1537489399468
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 22 00:44
> >> > pulsar_precommit_cpp_ws-cleanup_1537566528660
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 23:12
> >> > pulsar_precommit_cpp_ws-cleanup_1537571549978
> >> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 22 00:44
> >> > pulsar_precommit_cpp_ws-cleanup_1537573605258
> >> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 00:43
> >> > pulsar_precommit_cpp_ws-cleanup_1537855939828
> >> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 11:18
> >> > pulsar_precommit_cpp_ws-cleanup_1537874305575
> >> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 12:41
> >> > pulsar_precommit_cpp_ws-cleanup_1537879296622
> >> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 01:39
> >> > pulsar_precommit_cpp_ws-cleanup_1537925955967
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 26 13:17
> >> > pulsar_precommit_cpp_ws-cleanup_1537967826838
> >> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 15:55
> >> > pulsar_precommit_cpp_ws-cleanup_1537977326736
> >> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 17:34
> >> > pulsar_precommit_cpp_ws-cleanup_1537983268177
> >> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 22:04
> >> > pulsar_precommit_cpp_ws-cleanup_1538085846793
> >> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 28 00:20
> >> > pulsar_precommit_cpp_ws-cleanup_1538094024821
> >> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 28 17:42
> >> > pulsar_precommit_cpp_ws-cleanup_1538156543343
> >> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:02
> >> > pulsar_precommit_integrationtests_ws-cleanup_1536298125841
> >> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:00
> >> > pulsar_precommit_integrationtests_ws-cleanup_1536968128660
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 18 00:43
> >> > pulsar_precommit_integrationtests_ws-cleanup_1537218541598
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> >> > pulsar_precommit_integrationtests_ws-cleanup_1537331703324
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 22 00:43
> >> > pulsar_precommit_integrationtests_ws-cleanup_1537553748859
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 00:43
> >> > pulsar_precommit_integrationtests_ws-cleanup_1537730402937
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 07:48
> >> > pulsar_precommit_integrationtests_ws-cleanup_1537775336586
> >> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 23:30
> >> > pulsar_precommit_integrationtests_ws-cleanup_1537831851711
> >> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 23:08
> >> > pulsar_precommit_integrationtests_ws-cleanup_1537916928485
> >> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 15:21
> >> > pulsar_precommit_integrationtests_ws-cleanup_1538061702243
> >> >
> >> >
> >> > -Chris
> >> > ASF Infra
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > > On Sep 30, 2018, at 3:12 PM, Chris Thistlethwaite <
> >> > cthistlego@outlook.com> wrote:
> >> > >
> >> > > I've been working on H21 and H31. They both had a some docker issues
> >> > where the process was locked and I couldn't do anything with a few
> >> > containers that were over 8 days old.
> >> > >
> >> > > Thought we might be good this morning so I threw them back into
> >> > rotation. Guess not, so I'll take them back out.
> >> > >
> >> > > -Chris T.
> >> > > ________________________________
> >> > > From: Mike Jumper <mj...@apache.org>
> >> > > Sent: Sunday, September 30, 2018 6:03:51 PM
> >> > > To: builds@apache.org
> >> > > Subject: No space on H21 (Was: Fwd: Build failed in Jenkins:
> >> > guacamole-client-coverity #73)
> >> > >
> >> > > FYI - we're currently seeing disk space issues on H21.
> >> > >
> >> > > - Mike
> >> > >
> >> > >
> >> > > ---------- Forwarded message ---------
> >> > > From: Apache Jenkins Server <je...@builds.apache.org>
> >> > > Date: Sun, Sep 30, 2018, 14:37
> >> > > Subject: Build failed in Jenkins: guacamole-client-coverity #73
> >> > > To: <de...@guacamole.apache.org>
> >> > >
> >> > >
> >> > > See <
> >> > >
> >> >
> >>
> https://builds.apache.org/job/guacamole-client-coverity/73/display/redirect
> >> > >
> >> > >
> >> > > ------------------------------------------
> >> > > Started by user mjumper
> >> > > [EnvInject] - Loading node environment variables.
> >> > > Building remotely on H21 (ubuntu xenial) in workspace <
> >> > > https://builds.apache.org/job/guacamole-client-coverity/ws/>
> >> > > [WS-CLEANUP] Deleting project workspace...
> >> > > Cloning the remote Git repository
> >> > > Cloning repository
> >> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> >> > >> git init <
> >> > >
> >> >
> >>
> https://builds.apache.org/job/guacamole-client-coverity/ws/guacamole-client
> >> > >
> >> > > # timeout=10
> >> > > Fetching upstream changes from
> >> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> >> > >> git --version # timeout=10
> >> > >> git fetch --tags --progress
> >> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> >> > > +refs/heads/*:refs/remotes/origin/*
> >> > > ERROR: Error cloning remote repo 'origin'
> >> > > hudson.plugins.git.GitException: Command "git fetch --tags
> --progress
> >> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> >> > > +refs/heads/*:refs/remotes/origin/*" returned status code 128:
> >> > > stdout:
> >> > > stderr: remote: Counting objects: 62166, done.
> >> > > remote: Compressing objects: 0% (1/16096) remote:
> >> Compressing
> >> > > objects: 1% (161/16096) remote: Compressing objects:
> 2%
> >> > > (322/16096) remote: Compressing objects: 3% (483/16096)
> >> > > remote: Compressing objects: 4% (644/16096) remote:
> >> > > Compressing objects: 5% (805/16096) remote: Compressing
> >> > > objects: 6% (966/16096) remote: Compressing objects:
> 7%
> >> > > (1127/16096) remote: Compressing objects: 8%
> (1288/16096)
> >> > > remote: Compressing objects: 9% (1449/16096) remote:
> >> > > Compressing objects: 10% (1610/16096) remote: Compressing
> >> > > objects: 11% (1771/16096) remote: Compressing objects:
> 12%
> >> > > (1932/16096) remote: Compressing objects: 13%
> (2093/16096)
> >> > > remote: Compressing objects: 14% (2254/16096) remote:
> >> > > Compressing objects: 15% (2415/16096) remote: Compressing
> >> > > objects: 16% (2576/16096) remote: Compressing objects:
> 17%
> >> > > (2737/16096) remote: Compressing objects: 18%
> (2898/16096)
> >> > > remote: Compressing objects: 19% (3059/16096) remote:
> >> > > Compressing objects: 20% (3220/16096) remote: Compressing
> >> > > objects: 21% (3381/16096) remote: Compressing objects:
> 22%
> >> > > (3542/16096) remote: Compressing objects: 23%
> (3703/16096)
> >> > > remote: Compressing objects: 24% (3864/16096) remote:
> >> > > Compressing objects: 25% (4024/16096) remote: Compressing
> >> > > objects: 26% (4185/16096) remote: Compressing objects:
> 27%
> >> > > (4346/16096) remote: Compressing objects: 28%
> (4507/16096)
> >> > > remote: Compressing objects: 29% (4668/16096) remote:
> >> > > Compressing objects: 30% (4829/16096) remote: Compressing
> >> > > objects: 31% (4990/16096) remote: Compressing objects:
> 32%
> >> > > (5151/16096) remote: Compressing objects: 33%
> (5312/16096)
> >> > > remote: Compressing objects: 34% (5473/16096) remote:
> >> > > Compressing objects: 35% (5634/16096) remote: Compressing
> >> > > objects: 36% (5795/16096) remote: Compressing objects:
> 37%
> >> > > (5956/16096) remote: Compressing objects: 38%
> (6117/16096)
> >> > > remote: Compressing objects: 39% (6278/16096) remote:
> >> > > Compressing objects: 40% (6439/16096) remote: Compressing
> >> > > objects: 41% (6600/16096) remote: Compressing objects:
> 42%
> >> > > (6761/16096) remote: Compressing objects: 43%
> (6922/16096)
> >> > > remote: Compressing objects: 44% (7083/16096) remote:
> >> > > Compressing objects: 45% (7244/16096) remote: Compressing
> >> > > objects: 46% (7405/16096) remote: Compressing objects:
> 47%
> >> > > (7566/16096) remote: Compressing objects: 48%
> (7727/16096)
> >> > > remote: Compressing objects: 49% (7888/16096) remote:
> >> > > Compressing objects: 50% (8048/16096) remote: Compressing
> >> > > objects: 51% (8209/16096) remote: Compressing objects:
> 52%
> >> > > (8370/16096) remote: Compressing objects: 53%
> (8531/16096)
> >> > > remote: Compressing objects: 54% (8692/16096) remote:
> >> > > Compressing objects: 55% (8853/16096) remote: Compressing
> >> > > objects: 56% (9014/16096) remote: Compressing objects:
> 57%
> >> > > (9175/16096) remote: Compressing objects: 58%
> (9336/16096)
> >> > > remote: Compressing objects: 59% (9497/16096) remote:
> >> > > Compressing objects: 60% (9658/16096) remote: Compressing
> >> > > objects: 61% (9819/16096) remote: Compressing objects:
> 62%
> >> > > (9980/16096) remote: Compressing objects: 63%
> (10141/16096)
> >> > > remote: Compressing objects: 64% (10302/16096)
> remote:
> >> > > Compressing objects: 65% (10463/16096) remote:
> Compressing
> >> > > objects: 66% (10624/16096) remote: Compressing objects:
> 67%
> >> > > (10785/16096) remote: Compressing objects: 68%
> (10946/16096)
> >> > > remote: Compressing objects: 69% (11107/16096)
> remote:
> >> > > Compressing objects: 70% (11268/16096) remote:
> Compressing
> >> > > objects: 71% (11429/16096) remote: Compressing objects:
> 72%
> >> > > (11590/16096) remote: Compressing objects: 73%
> (11751/16096)
> >> > > remote: Compressing objects: 74% (11912/16096)
> remote:
> >> > > Compressing objects: 75% (12072/16096) remote:
> Compressing
> >> > > objects: 76% (12233/16096) remote: Compressing objects:
> 77%
> >> > > (12394/16096) remote: Compressing objects: 78%
> (12555/16096)
> >> > > remote: Compressing objects: 79% (12716/16096)
> remote:
> >> > > Compressing objects: 80% (12877/16096) remote:
> Compressing
> >> > > objects: 81% (13038/16096) remote: Compressing objects:
> 82%
> >> > > (13199/16096) remote: Compressing objects: 83%
> (13360/16096)
> >> > > remote: Compressing objects: 84% (13521/16096)
> remote:
> >> > > Compressing objects: 85% (13682/16096) remote:
> Compressing
> >> > > objects: 86% (13843/16096) remote: Compressing objects:
> 87%
> >> > > (14004/16096) remote: Compressing objects: 88%
> (14165/16096)
> >> > > remote: Compressing objects: 89% (14326/16096)
> remote:
> >> > > Compressing objects: 90% (14487/16096) remote:
> Compressing
> >> > > objects: 91% (14648/16096) remote: Compressing objects:
> 92%
> >> > > (14809/16096) remote: Compressing objects: 93%
> (14970/16096)
> >> > > remote: Compressing objects: 94% (15131/16096)
> remote:
> >> > > Compressing objects: 95% (15292/16096) remote:
> Compressing
> >> > > objects: 96% (15453/16096) remote: Compressing objects:
> 97%
> >> > > (15614/16096) remote: Compressing objects: 98%
> (15775/16096)
> >> > > remote: Compressing objects: 99% (15936/16096)
> remote:
> >> > > Compressing objects: 100% (16096/16096) remote:
> Compressing
> >> > > objects: 100% (16096/16096), done.
> >> > > Receiving objects: 0% (1/62166) Receiving objects: 1%
> (622/62166)
> >> > > Receiving objects: 2% (1244/62166) Receiving objects: 3%
> >> > > (1865/62166) Receiving objects: 4% (2487/62166) Receiving
> >> objects:
> >> > > 5% (3109/62166) Receiving objects: 6% (3730/62166) Receiving
> >> > > objects: 7% (4352/62166) Receiving objects: 8% (4974/62166)
> >> > > Receiving objects: 9% (5595/62166) Receiving objects: 10%
> >> > > (6217/62166) Receiving objects: 11% (6839/62166) Receiving
> >> objects:
> >> > > 12% (7460/62166) Receiving objects: 13% (8082/62166) Receiving
> >> > > objects: 14% (8704/62166) Receiving objects: 15% (9325/62166)
> >> > > Receiving objects: 16% (9947/62166) Receiving objects: 17%
> >> > > (10569/62166) Receiving objects: 18% (11190/62166) Receiving
> >> > objects:
> >> > > 19% (11812/62166) Receiving objects: 20% (12434/62166)
> Receiving
> >> > > objects: 21% (13055/62166) Receiving objects: 22% (13677/62166)
> >> > > fatal: write error: No space left on device
> >> > > fatal: index-pack failed
> >> > >
> >> > > at
> >> > >
> >> >
> >>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
> >> > > at
> >> > >
> >> >
> >>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1721)
> >> > > at
> >> > >
> >> >
> >>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:72)
> >> > > at
> >> > >
> >> >
> >>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:405)
> >> > > at
> >> > >
> >> >
> >>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:614)
> >> > > at
> >> > >
> >> >
> >>
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
> >> > > at
> >> > >
> >> >
> >>
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
> >> > > at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> >> > > at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> >> > > at hudson.remoting.Request$2.run(Request.java:369)
> >> > > at
> >> > >
> >> >
> >>
> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> >> > > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> > > at
> >> > >
> >> >
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >> > > at
> >> > >
> >> >
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >> > > at java.lang.Thread.run(Thread.java:748)
> >> > > Suppressed: hudson.remoting.Channel$CallSiteStackTrace:
> Remote
> >> > call
> >> > > to H21
> >> > > at
> >> > > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> >> > > at
> >> > >
> >> >
> >>
> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
> >> > > at hudson.remoting.Channel.call(Channel.java:955)
> >> > > at
> >> > >
> >> >
> >>
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
> >> > > at
> sun.reflect.GeneratedMethodAccessor918.invoke(Unknown
> >> > > Source)
> >> > > at
> >> > >
> >> >
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> > > at java.lang.reflect.Method.invoke(Method.java:498)
> >> > > at
> >> > >
> >> >
> >>
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
> >> > > at com.sun.proxy.$Proxy142.execute(Unknown Source)
> >> > > at
> >> > > hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1146)
> >> > > at
> hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> >> > > at hudson.scm.SCM.checkout(SCM.java:504)
> >> > > at
> >> > > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> >> > > at
> >> > >
> >> >
> >>
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> >> > > at
> >> > >
> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> >> > > at
> >> > >
> >> >
> >>
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> >> > > at hudson.model.Run.execute(Run.java:1794)
> >> > > at
> >> hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> >> > > at
> >> > > hudson.model.ResourceController.execute(ResourceController.java:97)
> >> > > at hudson.model.Executor.run(Executor.java:429)
> >> > > ERROR: Error cloning remote repo 'origin'
> >> > > Retrying after 10 seconds
> >> > >> git rev-parse --is-inside-work-tree # timeout=10
> >> > > Fetching changes from the remote Git repository
> >> > >> git config remote.origin.url
> >> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git #
> >> > timeout=10
> >> > > Fetching upstream changes from
> >> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> >> > >> git --version # timeout=10
> >> > >> git fetch --tags --progress
> >> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> >> > > +refs/heads/*:refs/remotes/origin/*
> >> > > ERROR: Error fetching remote repo 'origin'
> >> > > hudson.plugins.git.GitException: Failed to fetch from
> >> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> >> > > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
> >> > > at
> hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> >> > > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> >> > > at hudson.scm.SCM.checkout(SCM.java:504)
> >> > > at
> >> > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> >> > > at
> >> > >
> >> >
> >>
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> >> > > at
> >> > >
> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> >> > > at
> >> > >
> >> >
> >>
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> >> > > at hudson.model.Run.execute(Run.java:1794)
> >> > > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> >> > > at
> >> > > hudson.model.ResourceController.execute(ResourceController.java:97)
> >> > > at hudson.model.Executor.run(Executor.java:429)
> >> > > Caused by: hudson.plugins.git.GitException: Command "git fetch
> --tags
> >> > > --progress
> >> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> >> > > +refs/heads/*:refs/remotes/origin/*" returned status code 128:
> >> > > stdout:
> >> > > stderr: remote: Counting objects: 62166, done.
> >> > > remote: Compressing objects: 0% (1/16096) remote:
> >> Compressing
> >> > > objects: 1% (161/16096) remote: Compressing objects:
> 2%
> >> > > (322/16096) remote: Compressing objects: 3% (483/16096)
> >> > > remote: Compressing objects: 4% (644/16096) remote:
> >> > > Compressing objects: 5% (805/16096) remote: Compressing
> >> > > objects: 6% (966/16096) remote: Compressing objects:
> 7%
> >> > > (1127/16096) remote: Compressing objects: 8%
> (1288/16096)
> >> > > remote: Compressing objects: 9% (1449/16096) remote:
> >> > > Compressing objects: 10% (1610/16096) remote: Compressing
> >> > > objects: 11% (1771/16096) remote: Compressing objects:
> 12%
> >> > > (1932/16096) remote: Compressing objects: 13%
> (2093/16096)
> >> > > remote: Compressing objects: 14% (2254/16096) remote:
> >> > > Compressing objects: 15% (2415/16096) remote: Compressing
> >> > > objects: 16% (2576/16096) remote: Compressing objects:
> 17%
> >> > > (2737/16096) remote: Compressing objects: 18%
> (2898/16096)
> >> > > remote: Compressing objects: 19% (3059/16096) remote:
> >> > > Compressing objects: 20% (3220/16096) remote: Compressing
> >> > > objects: 21% (3381/16096) remote: Compressing objects:
> 22%
> >> > > (3542/16096) remote: Compressing objects: 23%
> (3703/16096)
> >> > > remote: Compressing objects: 24% (3864/16096) remote:
> >> > > Compressing objects: 25% (4024/16096) remote: Compressing
> >> > > objects: 26% (4185/16096) remote: Compressing objects:
> 27%
> >> > > (4346/16096) remote: Compressing objects: 28%
> (4507/16096)
> >> > > remote: Compressing objects: 29% (4668/16096) remote:
> >> > > Compressing objects: 30% (4829/16096) remote: Compressing
> >> > > objects: 31% (4990/16096) remote: Compressing objects:
> 32%
> >> > > (5151/16096) remote: Compressing objects: 33%
> (5312/16096)
> >> > > remote: Compressing objects: 34% (5473/16096) remote:
> >> > > Compressing objects: 35% (5634/16096) remote: Compressing
> >> > > objects: 36% (5795/16096) remote: Compressing objects:
> 37%
> >> > > (5956/16096) remote: Compressing objects: 38%
> (6117/16096)
> >> > > remote: Compressing objects: 39% (6278/16096) remote:
> >> > > Compressing objects: 40% (6439/16096) remote: Compressing
> >> > > objects: 41% (6600/16096) remote: Compressing objects:
> 42%
> >> > > (6761/16096) remote: Compressing objects: 43%
> (6922/16096)
> >> > > remote: Compressing objects: 44% (7083/16096) remote:
> >> > > Compressing objects: 45% (7244/16096) remote: Compressing
> >> > > objects: 46% (7405/16096) remote: Compressing objects:
> 47%
> >> > > (7566/16096) remote: Compressing objects: 48%
> (7727/16096)
> >> > > remote: Compressing objects: 49% (7888/16096) remote:
> >> > > Compressing objects: 50% (8048/16096) remote: Compressing
> >> > > objects: 51% (8209/16096) remote: Compressing objects:
> 52%
> >> > > (8370/16096) remote: Compressing objects: 53%
> (8531/16096)
> >> > > remote: Compressing objects: 54% (8692/16096) remote:
> >> > > Compressing objects: 55% (8853/16096) remote: Compressing
> >> > > objects: 56% (9014/16096) remote: Compressing objects:
> 57%
> >> > > (9175/16096) remote: Compressing objects: 58%
> (9336/16096)
> >> > > remote: Compressing objects: 59% (9497/16096) remote:
> >> > > Compressing objects: 60% (9658/16096) remote: Compressing
> >> > > objects: 61% (9819/16096) remote: Compressing objects:
> 62%
> >> > > (9980/16096) remote: Compressing objects: 63%
> (10141/16096)
> >> > > remote: Compressing objects: 64% (10302/16096)
> remote:
> >> > > Compressing objects: 65% (10463/16096) remote:
> Compressing
> >> > > objects: 66% (10624/16096) remote: Compressing objects:
> 67%
> >> > > (10785/16096) remote: Compressing objects: 68%
> (10946/16096)
> >> > > remote: Compressing objects: 69% (11107/16096)
> remote:
> >> > > Compressing objects: 70% (11268/16096) remote:
> Compressing
> >> > > objects: 71% (11429/16096) remote: Compressing objects:
> 72%
> >> > > (11590/16096) remote: Compressing objects: 73%
> (11751/16096)
> >> > > remote: Compressing objects: 74% (11912/16096)
> remote:
> >> > > Compressing objects: 75% (12072/16096) remote:
> Compressing
> >> > > objects: 76% (12233/16096) remote: Compressing objects:
> 77%
> >> > > (12394/16096) remote: Compressing objects: 78%
> (12555/16096)
> >> > > remote: Compressing objects: 79% (12716/16096)
> remote:
> >> > > Compressing objects: 80% (12877/16096) remote:
> Compressing
> >> > > objects: 81% (13038/16096) remote: Compressing objects:
> 82%
> >> > > (13199/16096) remote: Compressing objects: 83%
> (13360/16096)
> >> > > remote: Compressing objects: 84% (13521/16096)
> remote:
> >> > > Compressing objects: 85% (13682/16096) remote:
> Compressing
> >> > > objects: 86% (13843/16096) remote: Compressing objects:
> 87%
> >> > > (14004/16096) remote: Compressing objects: 88%
> (14165/16096)
> >> > > remote: Compressing objects: 89% (14326/16096)
> remote:
> >> > > Compressing objects: 90% (14487/16096) remote:
> Compressing
> >> > > objects: 91% (14648/16096) remote: Compressing objects:
> 92%
> >> > > (14809/16096) remote: Compressing objects: 93%
> (14970/16096)
> >> > > remote: Compressing objects: 94% (15131/16096)
> remote:
> >> > > Compressing objects: 95% (15292/16096) remote:
> Compressing
> >> > > objects: 96% (15453/16096) remote: Compressing objects:
> 97%
> >> > > (15614/16096) remote: Compressing objects: 98%
> (15775/16096)
> >> > > remote: Compressing objects: 99% (15936/16096)
> remote:
> >> > > Compressing objects: 100% (16096/16096) remote:
> Compressing
> >> > > objects: 100% (16096/16096), done.
> >> > > Receiving objects: 0% (1/62166) fatal: write error: No space
> left
> >> on
> >> > > device
> >> > > fatal: index-pack failed
> >> > >
> >> > > at
> >> > >
> >> >
> >>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
> >> > > at
> >> > >
> >> >
> >>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1721)
> >> > > at
> >> > >
> >> >
> >>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:72)
> >> > > at
> >> > >
> >> >
> >>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:405)
> >> > > at
> >> > >
> >> >
> >>
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
> >> > > at
> >> > >
> >> >
> >>
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
> >> > > at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> >> > > at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> >> > > at hudson.remoting.Request$2.run(Request.java:369)
> >> > > at
> >> > >
> >> >
> >>
> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> >> > > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> > > at
> >> > >
> >> >
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >> > > at
> >> > >
> >> >
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >> > > at java.lang.Thread.run(Thread.java:748)
> >> > > Suppressed: hudson.remoting.Channel$CallSiteStackTrace:
> Remote
> >> > call
> >> > > to H21
> >> > > at
> >> > > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> >> > > at
> >> > >
> >> >
> >>
> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
> >> > > at hudson.remoting.Channel.call(Channel.java:955)
> >> > > at
> >> > >
> >> >
> >>
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
> >> > > at
> sun.reflect.GeneratedMethodAccessor918.invoke(Unknown
> >> > > Source)
> >> > > at
> >> > >
> >> >
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> > > at java.lang.reflect.Method.invoke(Method.java:498)
> >> > > at
> >> > >
> >> >
> >>
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
> >> > > at com.sun.proxy.$Proxy119.execute(Unknown Source)
> >> > > at
> hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:886)
> >> > > at
> >> > > hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> >> > > at
> hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> >> > > at hudson.scm.SCM.checkout(SCM.java:504)
> >> > > at
> >> > > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> >> > > at
> >> > >
> >> >
> >>
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> >> > > at
> >> > >
> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> >> > > at
> >> > >
> >> >
> >>
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> >> > > at hudson.model.Run.execute(Run.java:1794)
> >> > > at
> >> hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> >> > > at
> >> > > hudson.model.ResourceController.execute(ResourceController.java:97)
> >> > > at hudson.model.Executor.run(Executor.java:429)
> >> > > ERROR: Error fetching remote repo 'origin'
> >> > > Retrying after 10 seconds
> >> > >> git rev-parse --is-inside-work-tree # timeout=10
> >> > > Fetching changes from the remote Git repository
> >> > >> git config remote.origin.url
> >> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git #
> >> > timeout=10
> >> > > ERROR: Error fetching remote repo 'origin'
> >> > > hudson.plugins.git.GitException: Failed to fetch from
> >> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> >> > > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
> >> > > at
> hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> >> > > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> >> > > at hudson.scm.SCM.checkout(SCM.java:504)
> >> > > at
> >> > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> >> > > at
> >> > >
> >> >
> >>
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> >> > > at
> >> > >
> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> >> > > at
> >> > >
> >> >
> >>
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> >> > > at hudson.model.Run.execute(Run.java:1794)
> >> > > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> >> > > at
> >> > > hudson.model.ResourceController.execute(ResourceController.java:97)
> >> > > at hudson.model.Executor.run(Executor.java:429)
> >> > > Caused by: hudson.plugins.git.GitException: Command "git config
> >> > > remote.origin.url
> >> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git"
> returned
> >> > > status code 4:
> >> > > stdout:
> >> > > stderr: error: failed to write new configuration file <
> >> > >
> >> >
> >>
> https://builds.apache.org/job/guacamole-client-coverity/ws/guacamole-client/.git/config.lock
> >> > >>
> >> > >
> >> > > at
> >> > >
> >> >
> >>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
> >> > > at
> >> > >
> >> >
> >>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1970)
> >> > > at
> >> > >
> >> >
> >>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1966)
> >> > > at
> >> > >
> >> >
> >>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1597)
> >> > > at
> >> > >
> >> >
> >>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1609)
> >> > > at
> >> > >
> >> >
> >>
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.setRemoteUrl(CliGitAPIImpl.java:1243)
> >> > > at hudson.plugins.git.GitAPI.setRemoteUrl(GitAPI.java:160)
> >> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> >> > > at
> >> > >
> >> >
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> >> > > at
> >> > >
> >> >
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> > > at java.lang.reflect.Method.invoke(Method.java:498)
> >> > > at
> >> > >
> >> >
> >>
> hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:929)
> >> > > at
> >> > >
> >> >
> >>
> hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:903)
> >> > > at
> >> > >
> >> >
> >>
> hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:855)
> >> > > at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> >> > > at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> >> > > at hudson.remoting.Request$2.run(Request.java:369)
> >> > > at
> >> > >
> >> >
> >>
> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> >> > > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> > > at
> >> > >
> >> >
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >> > > at
> >> > >
> >> >
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >> > > at java.lang.Thread.run(Thread.java:748)
> >> > > Suppressed: hudson.remoting.Channel$CallSiteStackTrace:
> Remote
> >> > call
> >> > > to H21
> >> > > at
> >> > > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> >> > > at
> >> > >
> >> >
> >>
> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
> >> > > at hudson.remoting.Channel.call(Channel.java:955)
> >> > > at
> >> > >
> >> >
> >>
> hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:283)
> >> > > at com.sun.proxy.$Proxy118.setRemoteUrl(Unknown
> Source)
> >> > > at
> >> > >
> >> >
> >>
> org.jenkinsci.plugins.gitclient.RemoteGitImpl.setRemoteUrl(RemoteGitImpl.java:295)
> >> > > at
> hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:876)
> >> > > at
> >> > > hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> >> > > at
> hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> >> > > at hudson.scm.SCM.checkout(SCM.java:504)
> >> > > at
> >> > > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> >> > > at
> >> > >
> >> >
> >>
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> >> > > at
> >> > >
> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> >> > > at
> >> > >
> >> >
> >>
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> >> > > at hudson.model.Run.execute(Run.java:1794)
> >> > > at
> >> hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> >> > > at
> >> > > hudson.model.ResourceController.execute(ResourceController.java:97)
> >> > > at hudson.model.Executor.run(Executor.java:429)
> >> > > ERROR: Error fetching remote repo 'origin'
> >> >
> >> > --
> >> Matteo Merli
> >> <mm...@apache.org>
> >>
>
--
Matteo Merli
<mm...@apache.org>
Re: No space on H21 (Was: Fwd: Build failed in Jenkins:
guacamole-client-coverity #73)
Posted by Ivan Kelly <iv...@apache.org>.
Could it be a problem with the cpp job creating everything as root so
then jenkins isn't able to clean up?
-Ivan
On Mon, Oct 1, 2018 at 8:34 PM, Joe Francis <jo...@oath.com.invalid> wrote:
> We should cleanup all the bk/zk dirs created by unit tests. Almost 16gig of
> junk
>
> On Mon, Oct 1, 2018, 11:31 AM Matteo Merli <ma...@gmail.com> wrote:
>
>> Chris, are these `ws-cleanup` jobs something that is created internally by
>> Jenkins? The job we have is `pulsar_precommit_cpp`.
>>
>> Is there any setting in Jenkins that could explain that these files are not
>> removed?
>>
>> On https://builds.apache.org/job/pulsar_precommit_cpp/configure , we
>> already have "Days to keep builds == 1".
>> For "Days to keep artifacts" is left blank. Should that be set to 1 day
>> too?
>>
>> Any other hint at things we can try to ensure builds are cleaned up
>> promptly?
>>
>> Thanks,
>> Matteo
>>
>> On Sun, Sep 30, 2018 at 3:57 PM Chris Lambertus <cm...@apache.org> wrote:
>>
>> > The main culprit on H31 appears to be pulsar ws-cleanup jobs consuming
>> > almost 40% of the disk. (The problem on H21 was something
>> docker-related.)
>> >
>> >
>> > This is the second project I have found (Brooklyn was the first) which is
>> > leaving ws-cleanup directories all over the workspace and filling up the
>> > disks. Each one of these was 500+ MB. Pulsar, please address ASAP.
>> >
>> >
>> >
>> >
>> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:19
>> > pulsar_precommit_cpp@2_ws-cleanup_1535737112280
>> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:18
>> > pulsar_precommit_cpp@2_ws-cleanup_1536174576053
>> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 06:04
>> > pulsar_precommit_cpp@2_ws-cleanup_1537292379178
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 22:45
>> > pulsar_precommit_cpp@2_ws-cleanup_1537397120862
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
>> > pulsar_precommit_cpp@2_ws-cleanup_1537403378279
>> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 21 23:31
>> > pulsar_precommit_cpp@2_ws-cleanup_1537572701829
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 26 16:04
>> > pulsar_precommit_cpp@2_ws-cleanup_1537977879341
>> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 22:23
>> > pulsar_precommit_cpp@2_ws-cleanup_1538087035027
>> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:14
>> > pulsar_precommit_cpp_ws-cleanup_1535699830230
>> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:16
>> > pulsar_precommit_cpp_ws-cleanup_1535735511423
>> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:13
>> > pulsar_precommit_cpp_ws-cleanup_1535786064397
>> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:18
>> > pulsar_precommit_cpp_ws-cleanup_1536096963586
>> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 21 06:47
>> > pulsar_precommit_cpp_ws-cleanup_1536114938910
>> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 21 06:45
>> > pulsar_precommit_cpp_ws-cleanup_1536173165909
>> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:38
>> > pulsar_precommit_cpp_ws-cleanup_1536175584418
>> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:50
>> > pulsar_precommit_cpp_ws-cleanup_1536180707506
>> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:38
>> > pulsar_precommit_cpp_ws-cleanup_1536182927940
>> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:51
>> > pulsar_precommit_cpp_ws-cleanup_1536185193716
>> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:50
>> > pulsar_precommit_cpp_ws-cleanup_1536273709029
>> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 05:59
>> > pulsar_precommit_cpp_ws-cleanup_1536316964794
>> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:06
>> > pulsar_precommit_cpp_ws-cleanup_1536675650893
>> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 28 06:08
>> > pulsar_precommit_cpp_ws-cleanup_1536879616962
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 30 19:53
>> > pulsar_precommit_cpp_ws-cleanup_1536891056968
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 30 19:47
>> > pulsar_precommit_cpp_ws-cleanup_1536966466683
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 18 00:43
>> > pulsar_precommit_cpp_ws-cleanup_1537216387387
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 00:45
>> > pulsar_precommit_cpp_ws-cleanup_1537292309413
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 00:45
>> > pulsar_precommit_cpp_ws-cleanup_1537308526251
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
>> > pulsar_precommit_cpp_ws-cleanup_1537331780420
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
>> > pulsar_precommit_cpp_ws-cleanup_1537379886715
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
>> > pulsar_precommit_cpp_ws-cleanup_1537386197893
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
>> > pulsar_precommit_cpp_ws-cleanup_1537397073464
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
>> > pulsar_precommit_cpp_ws-cleanup_1537403247575
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:43
>> > pulsar_precommit_cpp_ws-cleanup_1537413954888
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
>> > pulsar_precommit_cpp_ws-cleanup_1537417322151
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
>> > pulsar_precommit_cpp_ws-cleanup_1537421381129
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:44
>> > pulsar_precommit_cpp_ws-cleanup_1537469348852
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
>> > pulsar_precommit_cpp_ws-cleanup_1537489399468
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 22 00:44
>> > pulsar_precommit_cpp_ws-cleanup_1537566528660
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 23:12
>> > pulsar_precommit_cpp_ws-cleanup_1537571549978
>> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 22 00:44
>> > pulsar_precommit_cpp_ws-cleanup_1537573605258
>> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 00:43
>> > pulsar_precommit_cpp_ws-cleanup_1537855939828
>> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 11:18
>> > pulsar_precommit_cpp_ws-cleanup_1537874305575
>> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 12:41
>> > pulsar_precommit_cpp_ws-cleanup_1537879296622
>> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 01:39
>> > pulsar_precommit_cpp_ws-cleanup_1537925955967
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 26 13:17
>> > pulsar_precommit_cpp_ws-cleanup_1537967826838
>> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 15:55
>> > pulsar_precommit_cpp_ws-cleanup_1537977326736
>> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 17:34
>> > pulsar_precommit_cpp_ws-cleanup_1537983268177
>> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 22:04
>> > pulsar_precommit_cpp_ws-cleanup_1538085846793
>> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 28 00:20
>> > pulsar_precommit_cpp_ws-cleanup_1538094024821
>> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 28 17:42
>> > pulsar_precommit_cpp_ws-cleanup_1538156543343
>> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:02
>> > pulsar_precommit_integrationtests_ws-cleanup_1536298125841
>> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:00
>> > pulsar_precommit_integrationtests_ws-cleanup_1536968128660
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 18 00:43
>> > pulsar_precommit_integrationtests_ws-cleanup_1537218541598
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
>> > pulsar_precommit_integrationtests_ws-cleanup_1537331703324
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 22 00:43
>> > pulsar_precommit_integrationtests_ws-cleanup_1537553748859
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 00:43
>> > pulsar_precommit_integrationtests_ws-cleanup_1537730402937
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 07:48
>> > pulsar_precommit_integrationtests_ws-cleanup_1537775336586
>> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 23:30
>> > pulsar_precommit_integrationtests_ws-cleanup_1537831851711
>> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 23:08
>> > pulsar_precommit_integrationtests_ws-cleanup_1537916928485
>> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 15:21
>> > pulsar_precommit_integrationtests_ws-cleanup_1538061702243
>> >
>> >
>> > -Chris
>> > ASF Infra
>> >
>> >
>> >
>> >
>> >
>> > > On Sep 30, 2018, at 3:12 PM, Chris Thistlethwaite <
>> > cthistlego@outlook.com> wrote:
>> > >
>> > > I've been working on H21 and H31. They both had a some docker issues
>> > where the process was locked and I couldn't do anything with a few
>> > containers that were over 8 days old.
>> > >
>> > > Thought we might be good this morning so I threw them back into
>> > rotation. Guess not, so I'll take them back out.
>> > >
>> > > -Chris T.
>> > > ________________________________
>> > > From: Mike Jumper <mj...@apache.org>
>> > > Sent: Sunday, September 30, 2018 6:03:51 PM
>> > > To: builds@apache.org
>> > > Subject: No space on H21 (Was: Fwd: Build failed in Jenkins:
>> > guacamole-client-coverity #73)
>> > >
>> > > FYI - we're currently seeing disk space issues on H21.
>> > >
>> > > - Mike
>> > >
>> > >
>> > > ---------- Forwarded message ---------
>> > > From: Apache Jenkins Server <je...@builds.apache.org>
>> > > Date: Sun, Sep 30, 2018, 14:37
>> > > Subject: Build failed in Jenkins: guacamole-client-coverity #73
>> > > To: <de...@guacamole.apache.org>
>> > >
>> > >
>> > > See <
>> > >
>> >
>> https://builds.apache.org/job/guacamole-client-coverity/73/display/redirect
>> > >
>> > >
>> > > ------------------------------------------
>> > > Started by user mjumper
>> > > [EnvInject] - Loading node environment variables.
>> > > Building remotely on H21 (ubuntu xenial) in workspace <
>> > > https://builds.apache.org/job/guacamole-client-coverity/ws/>
>> > > [WS-CLEANUP] Deleting project workspace...
>> > > Cloning the remote Git repository
>> > > Cloning repository
>> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>> > >> git init <
>> > >
>> >
>> https://builds.apache.org/job/guacamole-client-coverity/ws/guacamole-client
>> > >
>> > > # timeout=10
>> > > Fetching upstream changes from
>> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>> > >> git --version # timeout=10
>> > >> git fetch --tags --progress
>> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>> > > +refs/heads/*:refs/remotes/origin/*
>> > > ERROR: Error cloning remote repo 'origin'
>> > > hudson.plugins.git.GitException: Command "git fetch --tags --progress
>> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>> > > +refs/heads/*:refs/remotes/origin/*" returned status code 128:
>> > > stdout:
>> > > stderr: remote: Counting objects: 62166, done.
>> > > remote: Compressing objects: 0% (1/16096) remote:
>> Compressing
>> > > objects: 1% (161/16096) remote: Compressing objects: 2%
>> > > (322/16096) remote: Compressing objects: 3% (483/16096)
>> > > remote: Compressing objects: 4% (644/16096) remote:
>> > > Compressing objects: 5% (805/16096) remote: Compressing
>> > > objects: 6% (966/16096) remote: Compressing objects: 7%
>> > > (1127/16096) remote: Compressing objects: 8% (1288/16096)
>> > > remote: Compressing objects: 9% (1449/16096) remote:
>> > > Compressing objects: 10% (1610/16096) remote: Compressing
>> > > objects: 11% (1771/16096) remote: Compressing objects: 12%
>> > > (1932/16096) remote: Compressing objects: 13% (2093/16096)
>> > > remote: Compressing objects: 14% (2254/16096) remote:
>> > > Compressing objects: 15% (2415/16096) remote: Compressing
>> > > objects: 16% (2576/16096) remote: Compressing objects: 17%
>> > > (2737/16096) remote: Compressing objects: 18% (2898/16096)
>> > > remote: Compressing objects: 19% (3059/16096) remote:
>> > > Compressing objects: 20% (3220/16096) remote: Compressing
>> > > objects: 21% (3381/16096) remote: Compressing objects: 22%
>> > > (3542/16096) remote: Compressing objects: 23% (3703/16096)
>> > > remote: Compressing objects: 24% (3864/16096) remote:
>> > > Compressing objects: 25% (4024/16096) remote: Compressing
>> > > objects: 26% (4185/16096) remote: Compressing objects: 27%
>> > > (4346/16096) remote: Compressing objects: 28% (4507/16096)
>> > > remote: Compressing objects: 29% (4668/16096) remote:
>> > > Compressing objects: 30% (4829/16096) remote: Compressing
>> > > objects: 31% (4990/16096) remote: Compressing objects: 32%
>> > > (5151/16096) remote: Compressing objects: 33% (5312/16096)
>> > > remote: Compressing objects: 34% (5473/16096) remote:
>> > > Compressing objects: 35% (5634/16096) remote: Compressing
>> > > objects: 36% (5795/16096) remote: Compressing objects: 37%
>> > > (5956/16096) remote: Compressing objects: 38% (6117/16096)
>> > > remote: Compressing objects: 39% (6278/16096) remote:
>> > > Compressing objects: 40% (6439/16096) remote: Compressing
>> > > objects: 41% (6600/16096) remote: Compressing objects: 42%
>> > > (6761/16096) remote: Compressing objects: 43% (6922/16096)
>> > > remote: Compressing objects: 44% (7083/16096) remote:
>> > > Compressing objects: 45% (7244/16096) remote: Compressing
>> > > objects: 46% (7405/16096) remote: Compressing objects: 47%
>> > > (7566/16096) remote: Compressing objects: 48% (7727/16096)
>> > > remote: Compressing objects: 49% (7888/16096) remote:
>> > > Compressing objects: 50% (8048/16096) remote: Compressing
>> > > objects: 51% (8209/16096) remote: Compressing objects: 52%
>> > > (8370/16096) remote: Compressing objects: 53% (8531/16096)
>> > > remote: Compressing objects: 54% (8692/16096) remote:
>> > > Compressing objects: 55% (8853/16096) remote: Compressing
>> > > objects: 56% (9014/16096) remote: Compressing objects: 57%
>> > > (9175/16096) remote: Compressing objects: 58% (9336/16096)
>> > > remote: Compressing objects: 59% (9497/16096) remote:
>> > > Compressing objects: 60% (9658/16096) remote: Compressing
>> > > objects: 61% (9819/16096) remote: Compressing objects: 62%
>> > > (9980/16096) remote: Compressing objects: 63% (10141/16096)
>> > > remote: Compressing objects: 64% (10302/16096) remote:
>> > > Compressing objects: 65% (10463/16096) remote: Compressing
>> > > objects: 66% (10624/16096) remote: Compressing objects: 67%
>> > > (10785/16096) remote: Compressing objects: 68% (10946/16096)
>> > > remote: Compressing objects: 69% (11107/16096) remote:
>> > > Compressing objects: 70% (11268/16096) remote: Compressing
>> > > objects: 71% (11429/16096) remote: Compressing objects: 72%
>> > > (11590/16096) remote: Compressing objects: 73% (11751/16096)
>> > > remote: Compressing objects: 74% (11912/16096) remote:
>> > > Compressing objects: 75% (12072/16096) remote: Compressing
>> > > objects: 76% (12233/16096) remote: Compressing objects: 77%
>> > > (12394/16096) remote: Compressing objects: 78% (12555/16096)
>> > > remote: Compressing objects: 79% (12716/16096) remote:
>> > > Compressing objects: 80% (12877/16096) remote: Compressing
>> > > objects: 81% (13038/16096) remote: Compressing objects: 82%
>> > > (13199/16096) remote: Compressing objects: 83% (13360/16096)
>> > > remote: Compressing objects: 84% (13521/16096) remote:
>> > > Compressing objects: 85% (13682/16096) remote: Compressing
>> > > objects: 86% (13843/16096) remote: Compressing objects: 87%
>> > > (14004/16096) remote: Compressing objects: 88% (14165/16096)
>> > > remote: Compressing objects: 89% (14326/16096) remote:
>> > > Compressing objects: 90% (14487/16096) remote: Compressing
>> > > objects: 91% (14648/16096) remote: Compressing objects: 92%
>> > > (14809/16096) remote: Compressing objects: 93% (14970/16096)
>> > > remote: Compressing objects: 94% (15131/16096) remote:
>> > > Compressing objects: 95% (15292/16096) remote: Compressing
>> > > objects: 96% (15453/16096) remote: Compressing objects: 97%
>> > > (15614/16096) remote: Compressing objects: 98% (15775/16096)
>> > > remote: Compressing objects: 99% (15936/16096) remote:
>> > > Compressing objects: 100% (16096/16096) remote: Compressing
>> > > objects: 100% (16096/16096), done.
>> > > Receiving objects: 0% (1/62166) Receiving objects: 1% (622/62166)
>> > > Receiving objects: 2% (1244/62166) Receiving objects: 3%
>> > > (1865/62166) Receiving objects: 4% (2487/62166) Receiving
>> objects:
>> > > 5% (3109/62166) Receiving objects: 6% (3730/62166) Receiving
>> > > objects: 7% (4352/62166) Receiving objects: 8% (4974/62166)
>> > > Receiving objects: 9% (5595/62166) Receiving objects: 10%
>> > > (6217/62166) Receiving objects: 11% (6839/62166) Receiving
>> objects:
>> > > 12% (7460/62166) Receiving objects: 13% (8082/62166) Receiving
>> > > objects: 14% (8704/62166) Receiving objects: 15% (9325/62166)
>> > > Receiving objects: 16% (9947/62166) Receiving objects: 17%
>> > > (10569/62166) Receiving objects: 18% (11190/62166) Receiving
>> > objects:
>> > > 19% (11812/62166) Receiving objects: 20% (12434/62166) Receiving
>> > > objects: 21% (13055/62166) Receiving objects: 22% (13677/62166)
>> > > fatal: write error: No space left on device
>> > > fatal: index-pack failed
>> > >
>> > > at
>> > >
>> >
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
>> > > at
>> > >
>> >
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1721)
>> > > at
>> > >
>> >
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:72)
>> > > at
>> > >
>> >
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:405)
>> > > at
>> > >
>> >
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:614)
>> > > at
>> > >
>> >
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
>> > > at
>> > >
>> >
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
>> > > at hudson.remoting.UserRequest.perform(UserRequest.java:212)
>> > > at hudson.remoting.UserRequest.perform(UserRequest.java:54)
>> > > at hudson.remoting.Request$2.run(Request.java:369)
>> > > at
>> > >
>> >
>> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
>> > > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > > at
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > > at
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > > at java.lang.Thread.run(Thread.java:748)
>> > > Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
>> > call
>> > > to H21
>> > > at
>> > > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
>> > > at
>> > >
>> >
>> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
>> > > at hudson.remoting.Channel.call(Channel.java:955)
>> > > at
>> > >
>> >
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
>> > > at sun.reflect.GeneratedMethodAccessor918.invoke(Unknown
>> > > Source)
>> > > at
>> > >
>> >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> > > at java.lang.reflect.Method.invoke(Method.java:498)
>> > > at
>> > >
>> >
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
>> > > at com.sun.proxy.$Proxy142.execute(Unknown Source)
>> > > at
>> > > hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1146)
>> > > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
>> > > at hudson.scm.SCM.checkout(SCM.java:504)
>> > > at
>> > > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>> > > at
>> > >
>> >
>> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>> > > at
>> > > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>> > > at
>> > >
>> >
>> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>> > > at hudson.model.Run.execute(Run.java:1794)
>> > > at
>> hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>> > > at
>> > > hudson.model.ResourceController.execute(ResourceController.java:97)
>> > > at hudson.model.Executor.run(Executor.java:429)
>> > > ERROR: Error cloning remote repo 'origin'
>> > > Retrying after 10 seconds
>> > >> git rev-parse --is-inside-work-tree # timeout=10
>> > > Fetching changes from the remote Git repository
>> > >> git config remote.origin.url
>> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git #
>> > timeout=10
>> > > Fetching upstream changes from
>> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>> > >> git --version # timeout=10
>> > >> git fetch --tags --progress
>> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>> > > +refs/heads/*:refs/remotes/origin/*
>> > > ERROR: Error fetching remote repo 'origin'
>> > > hudson.plugins.git.GitException: Failed to fetch from
>> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>> > > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
>> > > at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
>> > > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
>> > > at hudson.scm.SCM.checkout(SCM.java:504)
>> > > at
>> > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>> > > at
>> > >
>> >
>> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>> > > at
>> > > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>> > > at
>> > >
>> >
>> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>> > > at hudson.model.Run.execute(Run.java:1794)
>> > > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>> > > at
>> > > hudson.model.ResourceController.execute(ResourceController.java:97)
>> > > at hudson.model.Executor.run(Executor.java:429)
>> > > Caused by: hudson.plugins.git.GitException: Command "git fetch --tags
>> > > --progress
>> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>> > > +refs/heads/*:refs/remotes/origin/*" returned status code 128:
>> > > stdout:
>> > > stderr: remote: Counting objects: 62166, done.
>> > > remote: Compressing objects: 0% (1/16096) remote:
>> Compressing
>> > > objects: 1% (161/16096) remote: Compressing objects: 2%
>> > > (322/16096) remote: Compressing objects: 3% (483/16096)
>> > > remote: Compressing objects: 4% (644/16096) remote:
>> > > Compressing objects: 5% (805/16096) remote: Compressing
>> > > objects: 6% (966/16096) remote: Compressing objects: 7%
>> > > (1127/16096) remote: Compressing objects: 8% (1288/16096)
>> > > remote: Compressing objects: 9% (1449/16096) remote:
>> > > Compressing objects: 10% (1610/16096) remote: Compressing
>> > > objects: 11% (1771/16096) remote: Compressing objects: 12%
>> > > (1932/16096) remote: Compressing objects: 13% (2093/16096)
>> > > remote: Compressing objects: 14% (2254/16096) remote:
>> > > Compressing objects: 15% (2415/16096) remote: Compressing
>> > > objects: 16% (2576/16096) remote: Compressing objects: 17%
>> > > (2737/16096) remote: Compressing objects: 18% (2898/16096)
>> > > remote: Compressing objects: 19% (3059/16096) remote:
>> > > Compressing objects: 20% (3220/16096) remote: Compressing
>> > > objects: 21% (3381/16096) remote: Compressing objects: 22%
>> > > (3542/16096) remote: Compressing objects: 23% (3703/16096)
>> > > remote: Compressing objects: 24% (3864/16096) remote:
>> > > Compressing objects: 25% (4024/16096) remote: Compressing
>> > > objects: 26% (4185/16096) remote: Compressing objects: 27%
>> > > (4346/16096) remote: Compressing objects: 28% (4507/16096)
>> > > remote: Compressing objects: 29% (4668/16096) remote:
>> > > Compressing objects: 30% (4829/16096) remote: Compressing
>> > > objects: 31% (4990/16096) remote: Compressing objects: 32%
>> > > (5151/16096) remote: Compressing objects: 33% (5312/16096)
>> > > remote: Compressing objects: 34% (5473/16096) remote:
>> > > Compressing objects: 35% (5634/16096) remote: Compressing
>> > > objects: 36% (5795/16096) remote: Compressing objects: 37%
>> > > (5956/16096) remote: Compressing objects: 38% (6117/16096)
>> > > remote: Compressing objects: 39% (6278/16096) remote:
>> > > Compressing objects: 40% (6439/16096) remote: Compressing
>> > > objects: 41% (6600/16096) remote: Compressing objects: 42%
>> > > (6761/16096) remote: Compressing objects: 43% (6922/16096)
>> > > remote: Compressing objects: 44% (7083/16096) remote:
>> > > Compressing objects: 45% (7244/16096) remote: Compressing
>> > > objects: 46% (7405/16096) remote: Compressing objects: 47%
>> > > (7566/16096) remote: Compressing objects: 48% (7727/16096)
>> > > remote: Compressing objects: 49% (7888/16096) remote:
>> > > Compressing objects: 50% (8048/16096) remote: Compressing
>> > > objects: 51% (8209/16096) remote: Compressing objects: 52%
>> > > (8370/16096) remote: Compressing objects: 53% (8531/16096)
>> > > remote: Compressing objects: 54% (8692/16096) remote:
>> > > Compressing objects: 55% (8853/16096) remote: Compressing
>> > > objects: 56% (9014/16096) remote: Compressing objects: 57%
>> > > (9175/16096) remote: Compressing objects: 58% (9336/16096)
>> > > remote: Compressing objects: 59% (9497/16096) remote:
>> > > Compressing objects: 60% (9658/16096) remote: Compressing
>> > > objects: 61% (9819/16096) remote: Compressing objects: 62%
>> > > (9980/16096) remote: Compressing objects: 63% (10141/16096)
>> > > remote: Compressing objects: 64% (10302/16096) remote:
>> > > Compressing objects: 65% (10463/16096) remote: Compressing
>> > > objects: 66% (10624/16096) remote: Compressing objects: 67%
>> > > (10785/16096) remote: Compressing objects: 68% (10946/16096)
>> > > remote: Compressing objects: 69% (11107/16096) remote:
>> > > Compressing objects: 70% (11268/16096) remote: Compressing
>> > > objects: 71% (11429/16096) remote: Compressing objects: 72%
>> > > (11590/16096) remote: Compressing objects: 73% (11751/16096)
>> > > remote: Compressing objects: 74% (11912/16096) remote:
>> > > Compressing objects: 75% (12072/16096) remote: Compressing
>> > > objects: 76% (12233/16096) remote: Compressing objects: 77%
>> > > (12394/16096) remote: Compressing objects: 78% (12555/16096)
>> > > remote: Compressing objects: 79% (12716/16096) remote:
>> > > Compressing objects: 80% (12877/16096) remote: Compressing
>> > > objects: 81% (13038/16096) remote: Compressing objects: 82%
>> > > (13199/16096) remote: Compressing objects: 83% (13360/16096)
>> > > remote: Compressing objects: 84% (13521/16096) remote:
>> > > Compressing objects: 85% (13682/16096) remote: Compressing
>> > > objects: 86% (13843/16096) remote: Compressing objects: 87%
>> > > (14004/16096) remote: Compressing objects: 88% (14165/16096)
>> > > remote: Compressing objects: 89% (14326/16096) remote:
>> > > Compressing objects: 90% (14487/16096) remote: Compressing
>> > > objects: 91% (14648/16096) remote: Compressing objects: 92%
>> > > (14809/16096) remote: Compressing objects: 93% (14970/16096)
>> > > remote: Compressing objects: 94% (15131/16096) remote:
>> > > Compressing objects: 95% (15292/16096) remote: Compressing
>> > > objects: 96% (15453/16096) remote: Compressing objects: 97%
>> > > (15614/16096) remote: Compressing objects: 98% (15775/16096)
>> > > remote: Compressing objects: 99% (15936/16096) remote:
>> > > Compressing objects: 100% (16096/16096) remote: Compressing
>> > > objects: 100% (16096/16096), done.
>> > > Receiving objects: 0% (1/62166) fatal: write error: No space left
>> on
>> > > device
>> > > fatal: index-pack failed
>> > >
>> > > at
>> > >
>> >
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
>> > > at
>> > >
>> >
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1721)
>> > > at
>> > >
>> >
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:72)
>> > > at
>> > >
>> >
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:405)
>> > > at
>> > >
>> >
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
>> > > at
>> > >
>> >
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
>> > > at hudson.remoting.UserRequest.perform(UserRequest.java:212)
>> > > at hudson.remoting.UserRequest.perform(UserRequest.java:54)
>> > > at hudson.remoting.Request$2.run(Request.java:369)
>> > > at
>> > >
>> >
>> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
>> > > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > > at
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > > at
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > > at java.lang.Thread.run(Thread.java:748)
>> > > Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
>> > call
>> > > to H21
>> > > at
>> > > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
>> > > at
>> > >
>> >
>> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
>> > > at hudson.remoting.Channel.call(Channel.java:955)
>> > > at
>> > >
>> >
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
>> > > at sun.reflect.GeneratedMethodAccessor918.invoke(Unknown
>> > > Source)
>> > > at
>> > >
>> >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> > > at java.lang.reflect.Method.invoke(Method.java:498)
>> > > at
>> > >
>> >
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
>> > > at com.sun.proxy.$Proxy119.execute(Unknown Source)
>> > > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:886)
>> > > at
>> > > hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
>> > > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
>> > > at hudson.scm.SCM.checkout(SCM.java:504)
>> > > at
>> > > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>> > > at
>> > >
>> >
>> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>> > > at
>> > > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>> > > at
>> > >
>> >
>> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>> > > at hudson.model.Run.execute(Run.java:1794)
>> > > at
>> hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>> > > at
>> > > hudson.model.ResourceController.execute(ResourceController.java:97)
>> > > at hudson.model.Executor.run(Executor.java:429)
>> > > ERROR: Error fetching remote repo 'origin'
>> > > Retrying after 10 seconds
>> > >> git rev-parse --is-inside-work-tree # timeout=10
>> > > Fetching changes from the remote Git repository
>> > >> git config remote.origin.url
>> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git #
>> > timeout=10
>> > > ERROR: Error fetching remote repo 'origin'
>> > > hudson.plugins.git.GitException: Failed to fetch from
>> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
>> > > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
>> > > at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
>> > > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
>> > > at hudson.scm.SCM.checkout(SCM.java:504)
>> > > at
>> > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>> > > at
>> > >
>> >
>> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>> > > at
>> > > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>> > > at
>> > >
>> >
>> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>> > > at hudson.model.Run.execute(Run.java:1794)
>> > > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>> > > at
>> > > hudson.model.ResourceController.execute(ResourceController.java:97)
>> > > at hudson.model.Executor.run(Executor.java:429)
>> > > Caused by: hudson.plugins.git.GitException: Command "git config
>> > > remote.origin.url
>> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git" returned
>> > > status code 4:
>> > > stdout:
>> > > stderr: error: failed to write new configuration file <
>> > >
>> >
>> https://builds.apache.org/job/guacamole-client-coverity/ws/guacamole-client/.git/config.lock
>> > >>
>> > >
>> > > at
>> > >
>> >
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
>> > > at
>> > >
>> >
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1970)
>> > > at
>> > >
>> >
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1966)
>> > > at
>> > >
>> >
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1597)
>> > > at
>> > >
>> >
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1609)
>> > > at
>> > >
>> >
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.setRemoteUrl(CliGitAPIImpl.java:1243)
>> > > at hudson.plugins.git.GitAPI.setRemoteUrl(GitAPI.java:160)
>> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> > > at
>> > >
>> >
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>> > > at
>> > >
>> >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> > > at java.lang.reflect.Method.invoke(Method.java:498)
>> > > at
>> > >
>> >
>> hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:929)
>> > > at
>> > >
>> >
>> hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:903)
>> > > at
>> > >
>> >
>> hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:855)
>> > > at hudson.remoting.UserRequest.perform(UserRequest.java:212)
>> > > at hudson.remoting.UserRequest.perform(UserRequest.java:54)
>> > > at hudson.remoting.Request$2.run(Request.java:369)
>> > > at
>> > >
>> >
>> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
>> > > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > > at
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > > at
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > > at java.lang.Thread.run(Thread.java:748)
>> > > Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
>> > call
>> > > to H21
>> > > at
>> > > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
>> > > at
>> > >
>> >
>> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
>> > > at hudson.remoting.Channel.call(Channel.java:955)
>> > > at
>> > >
>> >
>> hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:283)
>> > > at com.sun.proxy.$Proxy118.setRemoteUrl(Unknown Source)
>> > > at
>> > >
>> >
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl.setRemoteUrl(RemoteGitImpl.java:295)
>> > > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:876)
>> > > at
>> > > hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
>> > > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
>> > > at hudson.scm.SCM.checkout(SCM.java:504)
>> > > at
>> > > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
>> > > at
>> > >
>> >
>> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>> > > at
>> > > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>> > > at
>> > >
>> >
>> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>> > > at hudson.model.Run.execute(Run.java:1794)
>> > > at
>> hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
>> > > at
>> > > hudson.model.ResourceController.execute(ResourceController.java:97)
>> > > at hudson.model.Executor.run(Executor.java:429)
>> > > ERROR: Error fetching remote repo 'origin'
>> >
>> > --
>> Matteo Merli
>> <mm...@apache.org>
>>
Re: No space on H21 (Was: Fwd: Build failed in Jenkins:
guacamole-client-coverity #73)
Posted by Joe Francis <jo...@oath.com.INVALID>.
We should cleanup all the bk/zk dirs created by unit tests. Almost 16gig of
junk
On Mon, Oct 1, 2018, 11:31 AM Matteo Merli <ma...@gmail.com> wrote:
> Chris, are these `ws-cleanup` jobs something that is created internally by
> Jenkins? The job we have is `pulsar_precommit_cpp`.
>
> Is there any setting in Jenkins that could explain that these files are not
> removed?
>
> On https://builds.apache.org/job/pulsar_precommit_cpp/configure , we
> already have "Days to keep builds == 1".
> For "Days to keep artifacts" is left blank. Should that be set to 1 day
> too?
>
> Any other hint at things we can try to ensure builds are cleaned up
> promptly?
>
> Thanks,
> Matteo
>
> On Sun, Sep 30, 2018 at 3:57 PM Chris Lambertus <cm...@apache.org> wrote:
>
> > The main culprit on H31 appears to be pulsar ws-cleanup jobs consuming
> > almost 40% of the disk. (The problem on H21 was something
> docker-related.)
> >
> >
> > This is the second project I have found (Brooklyn was the first) which is
> > leaving ws-cleanup directories all over the workspace and filling up the
> > disks. Each one of these was 500+ MB. Pulsar, please address ASAP.
> >
> >
> >
> >
> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:19
> > pulsar_precommit_cpp@2_ws-cleanup_1535737112280
> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:18
> > pulsar_precommit_cpp@2_ws-cleanup_1536174576053
> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 06:04
> > pulsar_precommit_cpp@2_ws-cleanup_1537292379178
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 22:45
> > pulsar_precommit_cpp@2_ws-cleanup_1537397120862
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> > pulsar_precommit_cpp@2_ws-cleanup_1537403378279
> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 21 23:31
> > pulsar_precommit_cpp@2_ws-cleanup_1537572701829
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 26 16:04
> > pulsar_precommit_cpp@2_ws-cleanup_1537977879341
> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 22:23
> > pulsar_precommit_cpp@2_ws-cleanup_1538087035027
> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:14
> > pulsar_precommit_cpp_ws-cleanup_1535699830230
> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:16
> > pulsar_precommit_cpp_ws-cleanup_1535735511423
> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:13
> > pulsar_precommit_cpp_ws-cleanup_1535786064397
> > drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:18
> > pulsar_precommit_cpp_ws-cleanup_1536096963586
> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 21 06:47
> > pulsar_precommit_cpp_ws-cleanup_1536114938910
> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 21 06:45
> > pulsar_precommit_cpp_ws-cleanup_1536173165909
> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:38
> > pulsar_precommit_cpp_ws-cleanup_1536175584418
> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:50
> > pulsar_precommit_cpp_ws-cleanup_1536180707506
> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:38
> > pulsar_precommit_cpp_ws-cleanup_1536182927940
> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:51
> > pulsar_precommit_cpp_ws-cleanup_1536185193716
> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:50
> > pulsar_precommit_cpp_ws-cleanup_1536273709029
> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 05:59
> > pulsar_precommit_cpp_ws-cleanup_1536316964794
> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:06
> > pulsar_precommit_cpp_ws-cleanup_1536675650893
> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 28 06:08
> > pulsar_precommit_cpp_ws-cleanup_1536879616962
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 30 19:53
> > pulsar_precommit_cpp_ws-cleanup_1536891056968
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 30 19:47
> > pulsar_precommit_cpp_ws-cleanup_1536966466683
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 18 00:43
> > pulsar_precommit_cpp_ws-cleanup_1537216387387
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 00:45
> > pulsar_precommit_cpp_ws-cleanup_1537292309413
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 00:45
> > pulsar_precommit_cpp_ws-cleanup_1537308526251
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> > pulsar_precommit_cpp_ws-cleanup_1537331780420
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> > pulsar_precommit_cpp_ws-cleanup_1537379886715
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
> > pulsar_precommit_cpp_ws-cleanup_1537386197893
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
> > pulsar_precommit_cpp_ws-cleanup_1537397073464
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
> > pulsar_precommit_cpp_ws-cleanup_1537403247575
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:43
> > pulsar_precommit_cpp_ws-cleanup_1537413954888
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
> > pulsar_precommit_cpp_ws-cleanup_1537417322151
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
> > pulsar_precommit_cpp_ws-cleanup_1537421381129
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:44
> > pulsar_precommit_cpp_ws-cleanup_1537469348852
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
> > pulsar_precommit_cpp_ws-cleanup_1537489399468
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 22 00:44
> > pulsar_precommit_cpp_ws-cleanup_1537566528660
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 23:12
> > pulsar_precommit_cpp_ws-cleanup_1537571549978
> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 22 00:44
> > pulsar_precommit_cpp_ws-cleanup_1537573605258
> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 00:43
> > pulsar_precommit_cpp_ws-cleanup_1537855939828
> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 11:18
> > pulsar_precommit_cpp_ws-cleanup_1537874305575
> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 12:41
> > pulsar_precommit_cpp_ws-cleanup_1537879296622
> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 01:39
> > pulsar_precommit_cpp_ws-cleanup_1537925955967
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 26 13:17
> > pulsar_precommit_cpp_ws-cleanup_1537967826838
> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 15:55
> > pulsar_precommit_cpp_ws-cleanup_1537977326736
> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 17:34
> > pulsar_precommit_cpp_ws-cleanup_1537983268177
> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 22:04
> > pulsar_precommit_cpp_ws-cleanup_1538085846793
> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 28 00:20
> > pulsar_precommit_cpp_ws-cleanup_1538094024821
> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 28 17:42
> > pulsar_precommit_cpp_ws-cleanup_1538156543343
> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:02
> > pulsar_precommit_integrationtests_ws-cleanup_1536298125841
> > drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:00
> > pulsar_precommit_integrationtests_ws-cleanup_1536968128660
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 18 00:43
> > pulsar_precommit_integrationtests_ws-cleanup_1537218541598
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> > pulsar_precommit_integrationtests_ws-cleanup_1537331703324
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 22 00:43
> > pulsar_precommit_integrationtests_ws-cleanup_1537553748859
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 00:43
> > pulsar_precommit_integrationtests_ws-cleanup_1537730402937
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 07:48
> > pulsar_precommit_integrationtests_ws-cleanup_1537775336586
> > drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 23:30
> > pulsar_precommit_integrationtests_ws-cleanup_1537831851711
> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 23:08
> > pulsar_precommit_integrationtests_ws-cleanup_1537916928485
> > drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 15:21
> > pulsar_precommit_integrationtests_ws-cleanup_1538061702243
> >
> >
> > -Chris
> > ASF Infra
> >
> >
> >
> >
> >
> > > On Sep 30, 2018, at 3:12 PM, Chris Thistlethwaite <
> > cthistlego@outlook.com> wrote:
> > >
> > > I've been working on H21 and H31. They both had a some docker issues
> > where the process was locked and I couldn't do anything with a few
> > containers that were over 8 days old.
> > >
> > > Thought we might be good this morning so I threw them back into
> > rotation. Guess not, so I'll take them back out.
> > >
> > > -Chris T.
> > > ________________________________
> > > From: Mike Jumper <mj...@apache.org>
> > > Sent: Sunday, September 30, 2018 6:03:51 PM
> > > To: builds@apache.org
> > > Subject: No space on H21 (Was: Fwd: Build failed in Jenkins:
> > guacamole-client-coverity #73)
> > >
> > > FYI - we're currently seeing disk space issues on H21.
> > >
> > > - Mike
> > >
> > >
> > > ---------- Forwarded message ---------
> > > From: Apache Jenkins Server <je...@builds.apache.org>
> > > Date: Sun, Sep 30, 2018, 14:37
> > > Subject: Build failed in Jenkins: guacamole-client-coverity #73
> > > To: <de...@guacamole.apache.org>
> > >
> > >
> > > See <
> > >
> >
> https://builds.apache.org/job/guacamole-client-coverity/73/display/redirect
> > >
> > >
> > > ------------------------------------------
> > > Started by user mjumper
> > > [EnvInject] - Loading node environment variables.
> > > Building remotely on H21 (ubuntu xenial) in workspace <
> > > https://builds.apache.org/job/guacamole-client-coverity/ws/>
> > > [WS-CLEANUP] Deleting project workspace...
> > > Cloning the remote Git repository
> > > Cloning repository
> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > >> git init <
> > >
> >
> https://builds.apache.org/job/guacamole-client-coverity/ws/guacamole-client
> > >
> > > # timeout=10
> > > Fetching upstream changes from
> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > >> git --version # timeout=10
> > >> git fetch --tags --progress
> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > > +refs/heads/*:refs/remotes/origin/*
> > > ERROR: Error cloning remote repo 'origin'
> > > hudson.plugins.git.GitException: Command "git fetch --tags --progress
> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > > +refs/heads/*:refs/remotes/origin/*" returned status code 128:
> > > stdout:
> > > stderr: remote: Counting objects: 62166, done.
> > > remote: Compressing objects: 0% (1/16096) remote:
> Compressing
> > > objects: 1% (161/16096) remote: Compressing objects: 2%
> > > (322/16096) remote: Compressing objects: 3% (483/16096)
> > > remote: Compressing objects: 4% (644/16096) remote:
> > > Compressing objects: 5% (805/16096) remote: Compressing
> > > objects: 6% (966/16096) remote: Compressing objects: 7%
> > > (1127/16096) remote: Compressing objects: 8% (1288/16096)
> > > remote: Compressing objects: 9% (1449/16096) remote:
> > > Compressing objects: 10% (1610/16096) remote: Compressing
> > > objects: 11% (1771/16096) remote: Compressing objects: 12%
> > > (1932/16096) remote: Compressing objects: 13% (2093/16096)
> > > remote: Compressing objects: 14% (2254/16096) remote:
> > > Compressing objects: 15% (2415/16096) remote: Compressing
> > > objects: 16% (2576/16096) remote: Compressing objects: 17%
> > > (2737/16096) remote: Compressing objects: 18% (2898/16096)
> > > remote: Compressing objects: 19% (3059/16096) remote:
> > > Compressing objects: 20% (3220/16096) remote: Compressing
> > > objects: 21% (3381/16096) remote: Compressing objects: 22%
> > > (3542/16096) remote: Compressing objects: 23% (3703/16096)
> > > remote: Compressing objects: 24% (3864/16096) remote:
> > > Compressing objects: 25% (4024/16096) remote: Compressing
> > > objects: 26% (4185/16096) remote: Compressing objects: 27%
> > > (4346/16096) remote: Compressing objects: 28% (4507/16096)
> > > remote: Compressing objects: 29% (4668/16096) remote:
> > > Compressing objects: 30% (4829/16096) remote: Compressing
> > > objects: 31% (4990/16096) remote: Compressing objects: 32%
> > > (5151/16096) remote: Compressing objects: 33% (5312/16096)
> > > remote: Compressing objects: 34% (5473/16096) remote:
> > > Compressing objects: 35% (5634/16096) remote: Compressing
> > > objects: 36% (5795/16096) remote: Compressing objects: 37%
> > > (5956/16096) remote: Compressing objects: 38% (6117/16096)
> > > remote: Compressing objects: 39% (6278/16096) remote:
> > > Compressing objects: 40% (6439/16096) remote: Compressing
> > > objects: 41% (6600/16096) remote: Compressing objects: 42%
> > > (6761/16096) remote: Compressing objects: 43% (6922/16096)
> > > remote: Compressing objects: 44% (7083/16096) remote:
> > > Compressing objects: 45% (7244/16096) remote: Compressing
> > > objects: 46% (7405/16096) remote: Compressing objects: 47%
> > > (7566/16096) remote: Compressing objects: 48% (7727/16096)
> > > remote: Compressing objects: 49% (7888/16096) remote:
> > > Compressing objects: 50% (8048/16096) remote: Compressing
> > > objects: 51% (8209/16096) remote: Compressing objects: 52%
> > > (8370/16096) remote: Compressing objects: 53% (8531/16096)
> > > remote: Compressing objects: 54% (8692/16096) remote:
> > > Compressing objects: 55% (8853/16096) remote: Compressing
> > > objects: 56% (9014/16096) remote: Compressing objects: 57%
> > > (9175/16096) remote: Compressing objects: 58% (9336/16096)
> > > remote: Compressing objects: 59% (9497/16096) remote:
> > > Compressing objects: 60% (9658/16096) remote: Compressing
> > > objects: 61% (9819/16096) remote: Compressing objects: 62%
> > > (9980/16096) remote: Compressing objects: 63% (10141/16096)
> > > remote: Compressing objects: 64% (10302/16096) remote:
> > > Compressing objects: 65% (10463/16096) remote: Compressing
> > > objects: 66% (10624/16096) remote: Compressing objects: 67%
> > > (10785/16096) remote: Compressing objects: 68% (10946/16096)
> > > remote: Compressing objects: 69% (11107/16096) remote:
> > > Compressing objects: 70% (11268/16096) remote: Compressing
> > > objects: 71% (11429/16096) remote: Compressing objects: 72%
> > > (11590/16096) remote: Compressing objects: 73% (11751/16096)
> > > remote: Compressing objects: 74% (11912/16096) remote:
> > > Compressing objects: 75% (12072/16096) remote: Compressing
> > > objects: 76% (12233/16096) remote: Compressing objects: 77%
> > > (12394/16096) remote: Compressing objects: 78% (12555/16096)
> > > remote: Compressing objects: 79% (12716/16096) remote:
> > > Compressing objects: 80% (12877/16096) remote: Compressing
> > > objects: 81% (13038/16096) remote: Compressing objects: 82%
> > > (13199/16096) remote: Compressing objects: 83% (13360/16096)
> > > remote: Compressing objects: 84% (13521/16096) remote:
> > > Compressing objects: 85% (13682/16096) remote: Compressing
> > > objects: 86% (13843/16096) remote: Compressing objects: 87%
> > > (14004/16096) remote: Compressing objects: 88% (14165/16096)
> > > remote: Compressing objects: 89% (14326/16096) remote:
> > > Compressing objects: 90% (14487/16096) remote: Compressing
> > > objects: 91% (14648/16096) remote: Compressing objects: 92%
> > > (14809/16096) remote: Compressing objects: 93% (14970/16096)
> > > remote: Compressing objects: 94% (15131/16096) remote:
> > > Compressing objects: 95% (15292/16096) remote: Compressing
> > > objects: 96% (15453/16096) remote: Compressing objects: 97%
> > > (15614/16096) remote: Compressing objects: 98% (15775/16096)
> > > remote: Compressing objects: 99% (15936/16096) remote:
> > > Compressing objects: 100% (16096/16096) remote: Compressing
> > > objects: 100% (16096/16096), done.
> > > Receiving objects: 0% (1/62166) Receiving objects: 1% (622/62166)
> > > Receiving objects: 2% (1244/62166) Receiving objects: 3%
> > > (1865/62166) Receiving objects: 4% (2487/62166) Receiving
> objects:
> > > 5% (3109/62166) Receiving objects: 6% (3730/62166) Receiving
> > > objects: 7% (4352/62166) Receiving objects: 8% (4974/62166)
> > > Receiving objects: 9% (5595/62166) Receiving objects: 10%
> > > (6217/62166) Receiving objects: 11% (6839/62166) Receiving
> objects:
> > > 12% (7460/62166) Receiving objects: 13% (8082/62166) Receiving
> > > objects: 14% (8704/62166) Receiving objects: 15% (9325/62166)
> > > Receiving objects: 16% (9947/62166) Receiving objects: 17%
> > > (10569/62166) Receiving objects: 18% (11190/62166) Receiving
> > objects:
> > > 19% (11812/62166) Receiving objects: 20% (12434/62166) Receiving
> > > objects: 21% (13055/62166) Receiving objects: 22% (13677/62166)
> > > fatal: write error: No space left on device
> > > fatal: index-pack failed
> > >
> > > at
> > >
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
> > > at
> > >
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1721)
> > > at
> > >
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:72)
> > > at
> > >
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:405)
> > > at
> > >
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:614)
> > > at
> > >
> >
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
> > > at
> > >
> >
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
> > > at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> > > at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> > > at hudson.remoting.Request$2.run(Request.java:369)
> > > at
> > >
> >
> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> > > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > > at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > > at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > > at java.lang.Thread.run(Thread.java:748)
> > > Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
> > call
> > > to H21
> > > at
> > > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> > > at
> > >
> >
> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
> > > at hudson.remoting.Channel.call(Channel.java:955)
> > > at
> > >
> >
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
> > > at sun.reflect.GeneratedMethodAccessor918.invoke(Unknown
> > > Source)
> > > at
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > at java.lang.reflect.Method.invoke(Method.java:498)
> > > at
> > >
> >
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
> > > at com.sun.proxy.$Proxy142.execute(Unknown Source)
> > > at
> > > hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1146)
> > > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> > > at hudson.scm.SCM.checkout(SCM.java:504)
> > > at
> > > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> > > at
> > >
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> > > at
> > > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> > > at
> > >
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> > > at hudson.model.Run.execute(Run.java:1794)
> > > at
> hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> > > at
> > > hudson.model.ResourceController.execute(ResourceController.java:97)
> > > at hudson.model.Executor.run(Executor.java:429)
> > > ERROR: Error cloning remote repo 'origin'
> > > Retrying after 10 seconds
> > >> git rev-parse --is-inside-work-tree # timeout=10
> > > Fetching changes from the remote Git repository
> > >> git config remote.origin.url
> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git #
> > timeout=10
> > > Fetching upstream changes from
> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > >> git --version # timeout=10
> > >> git fetch --tags --progress
> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > > +refs/heads/*:refs/remotes/origin/*
> > > ERROR: Error fetching remote repo 'origin'
> > > hudson.plugins.git.GitException: Failed to fetch from
> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
> > > at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> > > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> > > at hudson.scm.SCM.checkout(SCM.java:504)
> > > at
> > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> > > at
> > >
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> > > at
> > > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> > > at
> > >
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> > > at hudson.model.Run.execute(Run.java:1794)
> > > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> > > at
> > > hudson.model.ResourceController.execute(ResourceController.java:97)
> > > at hudson.model.Executor.run(Executor.java:429)
> > > Caused by: hudson.plugins.git.GitException: Command "git fetch --tags
> > > --progress
> https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > > +refs/heads/*:refs/remotes/origin/*" returned status code 128:
> > > stdout:
> > > stderr: remote: Counting objects: 62166, done.
> > > remote: Compressing objects: 0% (1/16096) remote:
> Compressing
> > > objects: 1% (161/16096) remote: Compressing objects: 2%
> > > (322/16096) remote: Compressing objects: 3% (483/16096)
> > > remote: Compressing objects: 4% (644/16096) remote:
> > > Compressing objects: 5% (805/16096) remote: Compressing
> > > objects: 6% (966/16096) remote: Compressing objects: 7%
> > > (1127/16096) remote: Compressing objects: 8% (1288/16096)
> > > remote: Compressing objects: 9% (1449/16096) remote:
> > > Compressing objects: 10% (1610/16096) remote: Compressing
> > > objects: 11% (1771/16096) remote: Compressing objects: 12%
> > > (1932/16096) remote: Compressing objects: 13% (2093/16096)
> > > remote: Compressing objects: 14% (2254/16096) remote:
> > > Compressing objects: 15% (2415/16096) remote: Compressing
> > > objects: 16% (2576/16096) remote: Compressing objects: 17%
> > > (2737/16096) remote: Compressing objects: 18% (2898/16096)
> > > remote: Compressing objects: 19% (3059/16096) remote:
> > > Compressing objects: 20% (3220/16096) remote: Compressing
> > > objects: 21% (3381/16096) remote: Compressing objects: 22%
> > > (3542/16096) remote: Compressing objects: 23% (3703/16096)
> > > remote: Compressing objects: 24% (3864/16096) remote:
> > > Compressing objects: 25% (4024/16096) remote: Compressing
> > > objects: 26% (4185/16096) remote: Compressing objects: 27%
> > > (4346/16096) remote: Compressing objects: 28% (4507/16096)
> > > remote: Compressing objects: 29% (4668/16096) remote:
> > > Compressing objects: 30% (4829/16096) remote: Compressing
> > > objects: 31% (4990/16096) remote: Compressing objects: 32%
> > > (5151/16096) remote: Compressing objects: 33% (5312/16096)
> > > remote: Compressing objects: 34% (5473/16096) remote:
> > > Compressing objects: 35% (5634/16096) remote: Compressing
> > > objects: 36% (5795/16096) remote: Compressing objects: 37%
> > > (5956/16096) remote: Compressing objects: 38% (6117/16096)
> > > remote: Compressing objects: 39% (6278/16096) remote:
> > > Compressing objects: 40% (6439/16096) remote: Compressing
> > > objects: 41% (6600/16096) remote: Compressing objects: 42%
> > > (6761/16096) remote: Compressing objects: 43% (6922/16096)
> > > remote: Compressing objects: 44% (7083/16096) remote:
> > > Compressing objects: 45% (7244/16096) remote: Compressing
> > > objects: 46% (7405/16096) remote: Compressing objects: 47%
> > > (7566/16096) remote: Compressing objects: 48% (7727/16096)
> > > remote: Compressing objects: 49% (7888/16096) remote:
> > > Compressing objects: 50% (8048/16096) remote: Compressing
> > > objects: 51% (8209/16096) remote: Compressing objects: 52%
> > > (8370/16096) remote: Compressing objects: 53% (8531/16096)
> > > remote: Compressing objects: 54% (8692/16096) remote:
> > > Compressing objects: 55% (8853/16096) remote: Compressing
> > > objects: 56% (9014/16096) remote: Compressing objects: 57%
> > > (9175/16096) remote: Compressing objects: 58% (9336/16096)
> > > remote: Compressing objects: 59% (9497/16096) remote:
> > > Compressing objects: 60% (9658/16096) remote: Compressing
> > > objects: 61% (9819/16096) remote: Compressing objects: 62%
> > > (9980/16096) remote: Compressing objects: 63% (10141/16096)
> > > remote: Compressing objects: 64% (10302/16096) remote:
> > > Compressing objects: 65% (10463/16096) remote: Compressing
> > > objects: 66% (10624/16096) remote: Compressing objects: 67%
> > > (10785/16096) remote: Compressing objects: 68% (10946/16096)
> > > remote: Compressing objects: 69% (11107/16096) remote:
> > > Compressing objects: 70% (11268/16096) remote: Compressing
> > > objects: 71% (11429/16096) remote: Compressing objects: 72%
> > > (11590/16096) remote: Compressing objects: 73% (11751/16096)
> > > remote: Compressing objects: 74% (11912/16096) remote:
> > > Compressing objects: 75% (12072/16096) remote: Compressing
> > > objects: 76% (12233/16096) remote: Compressing objects: 77%
> > > (12394/16096) remote: Compressing objects: 78% (12555/16096)
> > > remote: Compressing objects: 79% (12716/16096) remote:
> > > Compressing objects: 80% (12877/16096) remote: Compressing
> > > objects: 81% (13038/16096) remote: Compressing objects: 82%
> > > (13199/16096) remote: Compressing objects: 83% (13360/16096)
> > > remote: Compressing objects: 84% (13521/16096) remote:
> > > Compressing objects: 85% (13682/16096) remote: Compressing
> > > objects: 86% (13843/16096) remote: Compressing objects: 87%
> > > (14004/16096) remote: Compressing objects: 88% (14165/16096)
> > > remote: Compressing objects: 89% (14326/16096) remote:
> > > Compressing objects: 90% (14487/16096) remote: Compressing
> > > objects: 91% (14648/16096) remote: Compressing objects: 92%
> > > (14809/16096) remote: Compressing objects: 93% (14970/16096)
> > > remote: Compressing objects: 94% (15131/16096) remote:
> > > Compressing objects: 95% (15292/16096) remote: Compressing
> > > objects: 96% (15453/16096) remote: Compressing objects: 97%
> > > (15614/16096) remote: Compressing objects: 98% (15775/16096)
> > > remote: Compressing objects: 99% (15936/16096) remote:
> > > Compressing objects: 100% (16096/16096) remote: Compressing
> > > objects: 100% (16096/16096), done.
> > > Receiving objects: 0% (1/62166) fatal: write error: No space left
> on
> > > device
> > > fatal: index-pack failed
> > >
> > > at
> > >
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
> > > at
> > >
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1721)
> > > at
> > >
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:72)
> > > at
> > >
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:405)
> > > at
> > >
> >
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
> > > at
> > >
> >
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
> > > at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> > > at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> > > at hudson.remoting.Request$2.run(Request.java:369)
> > > at
> > >
> >
> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> > > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > > at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > > at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > > at java.lang.Thread.run(Thread.java:748)
> > > Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
> > call
> > > to H21
> > > at
> > > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> > > at
> > >
> >
> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
> > > at hudson.remoting.Channel.call(Channel.java:955)
> > > at
> > >
> >
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
> > > at sun.reflect.GeneratedMethodAccessor918.invoke(Unknown
> > > Source)
> > > at
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > at java.lang.reflect.Method.invoke(Method.java:498)
> > > at
> > >
> >
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
> > > at com.sun.proxy.$Proxy119.execute(Unknown Source)
> > > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:886)
> > > at
> > > hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> > > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> > > at hudson.scm.SCM.checkout(SCM.java:504)
> > > at
> > > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> > > at
> > >
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> > > at
> > > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> > > at
> > >
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> > > at hudson.model.Run.execute(Run.java:1794)
> > > at
> hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> > > at
> > > hudson.model.ResourceController.execute(ResourceController.java:97)
> > > at hudson.model.Executor.run(Executor.java:429)
> > > ERROR: Error fetching remote repo 'origin'
> > > Retrying after 10 seconds
> > >> git rev-parse --is-inside-work-tree # timeout=10
> > > Fetching changes from the remote Git repository
> > >> git config remote.origin.url
> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git #
> > timeout=10
> > > ERROR: Error fetching remote repo 'origin'
> > > hudson.plugins.git.GitException: Failed to fetch from
> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
> > > at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> > > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> > > at hudson.scm.SCM.checkout(SCM.java:504)
> > > at
> > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> > > at
> > >
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> > > at
> > > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> > > at
> > >
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> > > at hudson.model.Run.execute(Run.java:1794)
> > > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> > > at
> > > hudson.model.ResourceController.execute(ResourceController.java:97)
> > > at hudson.model.Executor.run(Executor.java:429)
> > > Caused by: hudson.plugins.git.GitException: Command "git config
> > > remote.origin.url
> > > https://git-wip-us.apache.org/repos/asf/guacamole-client.git" returned
> > > status code 4:
> > > stdout:
> > > stderr: error: failed to write new configuration file <
> > >
> >
> https://builds.apache.org/job/guacamole-client-coverity/ws/guacamole-client/.git/config.lock
> > >>
> > >
> > > at
> > >
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
> > > at
> > >
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1970)
> > > at
> > >
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1966)
> > > at
> > >
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1597)
> > > at
> > >
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1609)
> > > at
> > >
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.setRemoteUrl(CliGitAPIImpl.java:1243)
> > > at hudson.plugins.git.GitAPI.setRemoteUrl(GitAPI.java:160)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > at
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > > at
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > at java.lang.reflect.Method.invoke(Method.java:498)
> > > at
> > >
> >
> hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:929)
> > > at
> > >
> >
> hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:903)
> > > at
> > >
> >
> hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:855)
> > > at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> > > at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> > > at hudson.remoting.Request$2.run(Request.java:369)
> > > at
> > >
> >
> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> > > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > > at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > > at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > > at java.lang.Thread.run(Thread.java:748)
> > > Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
> > call
> > > to H21
> > > at
> > > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> > > at
> > >
> >
> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
> > > at hudson.remoting.Channel.call(Channel.java:955)
> > > at
> > >
> >
> hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:283)
> > > at com.sun.proxy.$Proxy118.setRemoteUrl(Unknown Source)
> > > at
> > >
> >
> org.jenkinsci.plugins.gitclient.RemoteGitImpl.setRemoteUrl(RemoteGitImpl.java:295)
> > > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:876)
> > > at
> > > hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> > > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> > > at hudson.scm.SCM.checkout(SCM.java:504)
> > > at
> > > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> > > at
> > >
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> > > at
> > > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> > > at
> > >
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> > > at hudson.model.Run.execute(Run.java:1794)
> > > at
> hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> > > at
> > > hudson.model.ResourceController.execute(ResourceController.java:97)
> > > at hudson.model.Executor.run(Executor.java:429)
> > > ERROR: Error fetching remote repo 'origin'
> >
> > --
> Matteo Merli
> <mm...@apache.org>
>
Re: No space on H21 (Was: Fwd: Build failed in Jenkins:
guacamole-client-coverity #73)
Posted by Matteo Merli <ma...@gmail.com>.
Chris, are these `ws-cleanup` jobs something that is created internally by
Jenkins? The job we have is `pulsar_precommit_cpp`.
Is there any setting in Jenkins that could explain that these files are not
removed?
On https://builds.apache.org/job/pulsar_precommit_cpp/configure , we
already have "Days to keep builds == 1".
For "Days to keep artifacts" is left blank. Should that be set to 1 day
too?
Any other hint at things we can try to ensure builds are cleaned up
promptly?
Thanks,
Matteo
On Sun, Sep 30, 2018 at 3:57 PM Chris Lambertus <cm...@apache.org> wrote:
> The main culprit on H31 appears to be pulsar ws-cleanup jobs consuming
> almost 40% of the disk. (The problem on H21 was something docker-related.)
>
>
> This is the second project I have found (Brooklyn was the first) which is
> leaving ws-cleanup directories all over the workspace and filling up the
> disks. Each one of these was 500+ MB. Pulsar, please address ASAP.
>
>
>
>
> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:19
> pulsar_precommit_cpp@2_ws-cleanup_1535737112280
> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:18
> pulsar_precommit_cpp@2_ws-cleanup_1536174576053
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 06:04
> pulsar_precommit_cpp@2_ws-cleanup_1537292379178
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 22:45
> pulsar_precommit_cpp@2_ws-cleanup_1537397120862
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> pulsar_precommit_cpp@2_ws-cleanup_1537403378279
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 21 23:31
> pulsar_precommit_cpp@2_ws-cleanup_1537572701829
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 26 16:04
> pulsar_precommit_cpp@2_ws-cleanup_1537977879341
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 22:23
> pulsar_precommit_cpp@2_ws-cleanup_1538087035027
> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:14
> pulsar_precommit_cpp_ws-cleanup_1535699830230
> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:16
> pulsar_precommit_cpp_ws-cleanup_1535735511423
> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:13
> pulsar_precommit_cpp_ws-cleanup_1535786064397
> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:18
> pulsar_precommit_cpp_ws-cleanup_1536096963586
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 21 06:47
> pulsar_precommit_cpp_ws-cleanup_1536114938910
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 21 06:45
> pulsar_precommit_cpp_ws-cleanup_1536173165909
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:38
> pulsar_precommit_cpp_ws-cleanup_1536175584418
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:50
> pulsar_precommit_cpp_ws-cleanup_1536180707506
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:38
> pulsar_precommit_cpp_ws-cleanup_1536182927940
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:51
> pulsar_precommit_cpp_ws-cleanup_1536185193716
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:50
> pulsar_precommit_cpp_ws-cleanup_1536273709029
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 05:59
> pulsar_precommit_cpp_ws-cleanup_1536316964794
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:06
> pulsar_precommit_cpp_ws-cleanup_1536675650893
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 28 06:08
> pulsar_precommit_cpp_ws-cleanup_1536879616962
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 30 19:53
> pulsar_precommit_cpp_ws-cleanup_1536891056968
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 30 19:47
> pulsar_precommit_cpp_ws-cleanup_1536966466683
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 18 00:43
> pulsar_precommit_cpp_ws-cleanup_1537216387387
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 00:45
> pulsar_precommit_cpp_ws-cleanup_1537292309413
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 00:45
> pulsar_precommit_cpp_ws-cleanup_1537308526251
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> pulsar_precommit_cpp_ws-cleanup_1537331780420
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> pulsar_precommit_cpp_ws-cleanup_1537379886715
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
> pulsar_precommit_cpp_ws-cleanup_1537386197893
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
> pulsar_precommit_cpp_ws-cleanup_1537397073464
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
> pulsar_precommit_cpp_ws-cleanup_1537403247575
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:43
> pulsar_precommit_cpp_ws-cleanup_1537413954888
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
> pulsar_precommit_cpp_ws-cleanup_1537417322151
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
> pulsar_precommit_cpp_ws-cleanup_1537421381129
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:44
> pulsar_precommit_cpp_ws-cleanup_1537469348852
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
> pulsar_precommit_cpp_ws-cleanup_1537489399468
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 22 00:44
> pulsar_precommit_cpp_ws-cleanup_1537566528660
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 23:12
> pulsar_precommit_cpp_ws-cleanup_1537571549978
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 22 00:44
> pulsar_precommit_cpp_ws-cleanup_1537573605258
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 00:43
> pulsar_precommit_cpp_ws-cleanup_1537855939828
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 11:18
> pulsar_precommit_cpp_ws-cleanup_1537874305575
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 12:41
> pulsar_precommit_cpp_ws-cleanup_1537879296622
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 01:39
> pulsar_precommit_cpp_ws-cleanup_1537925955967
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 26 13:17
> pulsar_precommit_cpp_ws-cleanup_1537967826838
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 15:55
> pulsar_precommit_cpp_ws-cleanup_1537977326736
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 17:34
> pulsar_precommit_cpp_ws-cleanup_1537983268177
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 22:04
> pulsar_precommit_cpp_ws-cleanup_1538085846793
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 28 00:20
> pulsar_precommit_cpp_ws-cleanup_1538094024821
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 28 17:42
> pulsar_precommit_cpp_ws-cleanup_1538156543343
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:02
> pulsar_precommit_integrationtests_ws-cleanup_1536298125841
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:00
> pulsar_precommit_integrationtests_ws-cleanup_1536968128660
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 18 00:43
> pulsar_precommit_integrationtests_ws-cleanup_1537218541598
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> pulsar_precommit_integrationtests_ws-cleanup_1537331703324
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 22 00:43
> pulsar_precommit_integrationtests_ws-cleanup_1537553748859
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 00:43
> pulsar_precommit_integrationtests_ws-cleanup_1537730402937
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 07:48
> pulsar_precommit_integrationtests_ws-cleanup_1537775336586
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 23:30
> pulsar_precommit_integrationtests_ws-cleanup_1537831851711
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 23:08
> pulsar_precommit_integrationtests_ws-cleanup_1537916928485
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 15:21
> pulsar_precommit_integrationtests_ws-cleanup_1538061702243
>
>
> -Chris
> ASF Infra
>
>
>
>
>
> > On Sep 30, 2018, at 3:12 PM, Chris Thistlethwaite <
> cthistlego@outlook.com> wrote:
> >
> > I've been working on H21 and H31. They both had a some docker issues
> where the process was locked and I couldn't do anything with a few
> containers that were over 8 days old.
> >
> > Thought we might be good this morning so I threw them back into
> rotation. Guess not, so I'll take them back out.
> >
> > -Chris T.
> > ________________________________
> > From: Mike Jumper <mj...@apache.org>
> > Sent: Sunday, September 30, 2018 6:03:51 PM
> > To: builds@apache.org
> > Subject: No space on H21 (Was: Fwd: Build failed in Jenkins:
> guacamole-client-coverity #73)
> >
> > FYI - we're currently seeing disk space issues on H21.
> >
> > - Mike
> >
> >
> > ---------- Forwarded message ---------
> > From: Apache Jenkins Server <je...@builds.apache.org>
> > Date: Sun, Sep 30, 2018, 14:37
> > Subject: Build failed in Jenkins: guacamole-client-coverity #73
> > To: <de...@guacamole.apache.org>
> >
> >
> > See <
> >
> https://builds.apache.org/job/guacamole-client-coverity/73/display/redirect
> >
> >
> > ------------------------------------------
> > Started by user mjumper
> > [EnvInject] - Loading node environment variables.
> > Building remotely on H21 (ubuntu xenial) in workspace <
> > https://builds.apache.org/job/guacamole-client-coverity/ws/>
> > [WS-CLEANUP] Deleting project workspace...
> > Cloning the remote Git repository
> > Cloning repository
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> >> git init <
> >
> https://builds.apache.org/job/guacamole-client-coverity/ws/guacamole-client
> >
> > # timeout=10
> > Fetching upstream changes from
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> >> git --version # timeout=10
> >> git fetch --tags --progress
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > +refs/heads/*:refs/remotes/origin/*
> > ERROR: Error cloning remote repo 'origin'
> > hudson.plugins.git.GitException: Command "git fetch --tags --progress
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > +refs/heads/*:refs/remotes/origin/*" returned status code 128:
> > stdout:
> > stderr: remote: Counting objects: 62166, done.
> > remote: Compressing objects: 0% (1/16096) remote: Compressing
> > objects: 1% (161/16096) remote: Compressing objects: 2%
> > (322/16096) remote: Compressing objects: 3% (483/16096)
> > remote: Compressing objects: 4% (644/16096) remote:
> > Compressing objects: 5% (805/16096) remote: Compressing
> > objects: 6% (966/16096) remote: Compressing objects: 7%
> > (1127/16096) remote: Compressing objects: 8% (1288/16096)
> > remote: Compressing objects: 9% (1449/16096) remote:
> > Compressing objects: 10% (1610/16096) remote: Compressing
> > objects: 11% (1771/16096) remote: Compressing objects: 12%
> > (1932/16096) remote: Compressing objects: 13% (2093/16096)
> > remote: Compressing objects: 14% (2254/16096) remote:
> > Compressing objects: 15% (2415/16096) remote: Compressing
> > objects: 16% (2576/16096) remote: Compressing objects: 17%
> > (2737/16096) remote: Compressing objects: 18% (2898/16096)
> > remote: Compressing objects: 19% (3059/16096) remote:
> > Compressing objects: 20% (3220/16096) remote: Compressing
> > objects: 21% (3381/16096) remote: Compressing objects: 22%
> > (3542/16096) remote: Compressing objects: 23% (3703/16096)
> > remote: Compressing objects: 24% (3864/16096) remote:
> > Compressing objects: 25% (4024/16096) remote: Compressing
> > objects: 26% (4185/16096) remote: Compressing objects: 27%
> > (4346/16096) remote: Compressing objects: 28% (4507/16096)
> > remote: Compressing objects: 29% (4668/16096) remote:
> > Compressing objects: 30% (4829/16096) remote: Compressing
> > objects: 31% (4990/16096) remote: Compressing objects: 32%
> > (5151/16096) remote: Compressing objects: 33% (5312/16096)
> > remote: Compressing objects: 34% (5473/16096) remote:
> > Compressing objects: 35% (5634/16096) remote: Compressing
> > objects: 36% (5795/16096) remote: Compressing objects: 37%
> > (5956/16096) remote: Compressing objects: 38% (6117/16096)
> > remote: Compressing objects: 39% (6278/16096) remote:
> > Compressing objects: 40% (6439/16096) remote: Compressing
> > objects: 41% (6600/16096) remote: Compressing objects: 42%
> > (6761/16096) remote: Compressing objects: 43% (6922/16096)
> > remote: Compressing objects: 44% (7083/16096) remote:
> > Compressing objects: 45% (7244/16096) remote: Compressing
> > objects: 46% (7405/16096) remote: Compressing objects: 47%
> > (7566/16096) remote: Compressing objects: 48% (7727/16096)
> > remote: Compressing objects: 49% (7888/16096) remote:
> > Compressing objects: 50% (8048/16096) remote: Compressing
> > objects: 51% (8209/16096) remote: Compressing objects: 52%
> > (8370/16096) remote: Compressing objects: 53% (8531/16096)
> > remote: Compressing objects: 54% (8692/16096) remote:
> > Compressing objects: 55% (8853/16096) remote: Compressing
> > objects: 56% (9014/16096) remote: Compressing objects: 57%
> > (9175/16096) remote: Compressing objects: 58% (9336/16096)
> > remote: Compressing objects: 59% (9497/16096) remote:
> > Compressing objects: 60% (9658/16096) remote: Compressing
> > objects: 61% (9819/16096) remote: Compressing objects: 62%
> > (9980/16096) remote: Compressing objects: 63% (10141/16096)
> > remote: Compressing objects: 64% (10302/16096) remote:
> > Compressing objects: 65% (10463/16096) remote: Compressing
> > objects: 66% (10624/16096) remote: Compressing objects: 67%
> > (10785/16096) remote: Compressing objects: 68% (10946/16096)
> > remote: Compressing objects: 69% (11107/16096) remote:
> > Compressing objects: 70% (11268/16096) remote: Compressing
> > objects: 71% (11429/16096) remote: Compressing objects: 72%
> > (11590/16096) remote: Compressing objects: 73% (11751/16096)
> > remote: Compressing objects: 74% (11912/16096) remote:
> > Compressing objects: 75% (12072/16096) remote: Compressing
> > objects: 76% (12233/16096) remote: Compressing objects: 77%
> > (12394/16096) remote: Compressing objects: 78% (12555/16096)
> > remote: Compressing objects: 79% (12716/16096) remote:
> > Compressing objects: 80% (12877/16096) remote: Compressing
> > objects: 81% (13038/16096) remote: Compressing objects: 82%
> > (13199/16096) remote: Compressing objects: 83% (13360/16096)
> > remote: Compressing objects: 84% (13521/16096) remote:
> > Compressing objects: 85% (13682/16096) remote: Compressing
> > objects: 86% (13843/16096) remote: Compressing objects: 87%
> > (14004/16096) remote: Compressing objects: 88% (14165/16096)
> > remote: Compressing objects: 89% (14326/16096) remote:
> > Compressing objects: 90% (14487/16096) remote: Compressing
> > objects: 91% (14648/16096) remote: Compressing objects: 92%
> > (14809/16096) remote: Compressing objects: 93% (14970/16096)
> > remote: Compressing objects: 94% (15131/16096) remote:
> > Compressing objects: 95% (15292/16096) remote: Compressing
> > objects: 96% (15453/16096) remote: Compressing objects: 97%
> > (15614/16096) remote: Compressing objects: 98% (15775/16096)
> > remote: Compressing objects: 99% (15936/16096) remote:
> > Compressing objects: 100% (16096/16096) remote: Compressing
> > objects: 100% (16096/16096), done.
> > Receiving objects: 0% (1/62166) Receiving objects: 1% (622/62166)
> > Receiving objects: 2% (1244/62166) Receiving objects: 3%
> > (1865/62166) Receiving objects: 4% (2487/62166) Receiving objects:
> > 5% (3109/62166) Receiving objects: 6% (3730/62166) Receiving
> > objects: 7% (4352/62166) Receiving objects: 8% (4974/62166)
> > Receiving objects: 9% (5595/62166) Receiving objects: 10%
> > (6217/62166) Receiving objects: 11% (6839/62166) Receiving objects:
> > 12% (7460/62166) Receiving objects: 13% (8082/62166) Receiving
> > objects: 14% (8704/62166) Receiving objects: 15% (9325/62166)
> > Receiving objects: 16% (9947/62166) Receiving objects: 17%
> > (10569/62166) Receiving objects: 18% (11190/62166) Receiving
> objects:
> > 19% (11812/62166) Receiving objects: 20% (12434/62166) Receiving
> > objects: 21% (13055/62166) Receiving objects: 22% (13677/62166)
> > fatal: write error: No space left on device
> > fatal: index-pack failed
> >
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1721)
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:72)
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:405)
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:614)
> > at
> >
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
> > at
> >
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
> > at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> > at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> > at hudson.remoting.Request$2.run(Request.java:369)
> > at
> >
> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > at java.lang.Thread.run(Thread.java:748)
> > Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
> call
> > to H21
> > at
> > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> > at
> >
> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
> > at hudson.remoting.Channel.call(Channel.java:955)
> > at
> >
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
> > at sun.reflect.GeneratedMethodAccessor918.invoke(Unknown
> > Source)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:498)
> > at
> >
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
> > at com.sun.proxy.$Proxy142.execute(Unknown Source)
> > at
> > hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1146)
> > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> > at hudson.scm.SCM.checkout(SCM.java:504)
> > at
> > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> > at
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> > at
> > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> > at
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> > at hudson.model.Run.execute(Run.java:1794)
> > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> > at
> > hudson.model.ResourceController.execute(ResourceController.java:97)
> > at hudson.model.Executor.run(Executor.java:429)
> > ERROR: Error cloning remote repo 'origin'
> > Retrying after 10 seconds
> >> git rev-parse --is-inside-work-tree # timeout=10
> > Fetching changes from the remote Git repository
> >> git config remote.origin.url
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git #
> timeout=10
> > Fetching upstream changes from
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> >> git --version # timeout=10
> >> git fetch --tags --progress
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > +refs/heads/*:refs/remotes/origin/*
> > ERROR: Error fetching remote repo 'origin'
> > hudson.plugins.git.GitException: Failed to fetch from
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
> > at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> > at hudson.scm.SCM.checkout(SCM.java:504)
> > at
> hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> > at
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> > at
> > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> > at
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> > at hudson.model.Run.execute(Run.java:1794)
> > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> > at
> > hudson.model.ResourceController.execute(ResourceController.java:97)
> > at hudson.model.Executor.run(Executor.java:429)
> > Caused by: hudson.plugins.git.GitException: Command "git fetch --tags
> > --progress https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > +refs/heads/*:refs/remotes/origin/*" returned status code 128:
> > stdout:
> > stderr: remote: Counting objects: 62166, done.
> > remote: Compressing objects: 0% (1/16096) remote: Compressing
> > objects: 1% (161/16096) remote: Compressing objects: 2%
> > (322/16096) remote: Compressing objects: 3% (483/16096)
> > remote: Compressing objects: 4% (644/16096) remote:
> > Compressing objects: 5% (805/16096) remote: Compressing
> > objects: 6% (966/16096) remote: Compressing objects: 7%
> > (1127/16096) remote: Compressing objects: 8% (1288/16096)
> > remote: Compressing objects: 9% (1449/16096) remote:
> > Compressing objects: 10% (1610/16096) remote: Compressing
> > objects: 11% (1771/16096) remote: Compressing objects: 12%
> > (1932/16096) remote: Compressing objects: 13% (2093/16096)
> > remote: Compressing objects: 14% (2254/16096) remote:
> > Compressing objects: 15% (2415/16096) remote: Compressing
> > objects: 16% (2576/16096) remote: Compressing objects: 17%
> > (2737/16096) remote: Compressing objects: 18% (2898/16096)
> > remote: Compressing objects: 19% (3059/16096) remote:
> > Compressing objects: 20% (3220/16096) remote: Compressing
> > objects: 21% (3381/16096) remote: Compressing objects: 22%
> > (3542/16096) remote: Compressing objects: 23% (3703/16096)
> > remote: Compressing objects: 24% (3864/16096) remote:
> > Compressing objects: 25% (4024/16096) remote: Compressing
> > objects: 26% (4185/16096) remote: Compressing objects: 27%
> > (4346/16096) remote: Compressing objects: 28% (4507/16096)
> > remote: Compressing objects: 29% (4668/16096) remote:
> > Compressing objects: 30% (4829/16096) remote: Compressing
> > objects: 31% (4990/16096) remote: Compressing objects: 32%
> > (5151/16096) remote: Compressing objects: 33% (5312/16096)
> > remote: Compressing objects: 34% (5473/16096) remote:
> > Compressing objects: 35% (5634/16096) remote: Compressing
> > objects: 36% (5795/16096) remote: Compressing objects: 37%
> > (5956/16096) remote: Compressing objects: 38% (6117/16096)
> > remote: Compressing objects: 39% (6278/16096) remote:
> > Compressing objects: 40% (6439/16096) remote: Compressing
> > objects: 41% (6600/16096) remote: Compressing objects: 42%
> > (6761/16096) remote: Compressing objects: 43% (6922/16096)
> > remote: Compressing objects: 44% (7083/16096) remote:
> > Compressing objects: 45% (7244/16096) remote: Compressing
> > objects: 46% (7405/16096) remote: Compressing objects: 47%
> > (7566/16096) remote: Compressing objects: 48% (7727/16096)
> > remote: Compressing objects: 49% (7888/16096) remote:
> > Compressing objects: 50% (8048/16096) remote: Compressing
> > objects: 51% (8209/16096) remote: Compressing objects: 52%
> > (8370/16096) remote: Compressing objects: 53% (8531/16096)
> > remote: Compressing objects: 54% (8692/16096) remote:
> > Compressing objects: 55% (8853/16096) remote: Compressing
> > objects: 56% (9014/16096) remote: Compressing objects: 57%
> > (9175/16096) remote: Compressing objects: 58% (9336/16096)
> > remote: Compressing objects: 59% (9497/16096) remote:
> > Compressing objects: 60% (9658/16096) remote: Compressing
> > objects: 61% (9819/16096) remote: Compressing objects: 62%
> > (9980/16096) remote: Compressing objects: 63% (10141/16096)
> > remote: Compressing objects: 64% (10302/16096) remote:
> > Compressing objects: 65% (10463/16096) remote: Compressing
> > objects: 66% (10624/16096) remote: Compressing objects: 67%
> > (10785/16096) remote: Compressing objects: 68% (10946/16096)
> > remote: Compressing objects: 69% (11107/16096) remote:
> > Compressing objects: 70% (11268/16096) remote: Compressing
> > objects: 71% (11429/16096) remote: Compressing objects: 72%
> > (11590/16096) remote: Compressing objects: 73% (11751/16096)
> > remote: Compressing objects: 74% (11912/16096) remote:
> > Compressing objects: 75% (12072/16096) remote: Compressing
> > objects: 76% (12233/16096) remote: Compressing objects: 77%
> > (12394/16096) remote: Compressing objects: 78% (12555/16096)
> > remote: Compressing objects: 79% (12716/16096) remote:
> > Compressing objects: 80% (12877/16096) remote: Compressing
> > objects: 81% (13038/16096) remote: Compressing objects: 82%
> > (13199/16096) remote: Compressing objects: 83% (13360/16096)
> > remote: Compressing objects: 84% (13521/16096) remote:
> > Compressing objects: 85% (13682/16096) remote: Compressing
> > objects: 86% (13843/16096) remote: Compressing objects: 87%
> > (14004/16096) remote: Compressing objects: 88% (14165/16096)
> > remote: Compressing objects: 89% (14326/16096) remote:
> > Compressing objects: 90% (14487/16096) remote: Compressing
> > objects: 91% (14648/16096) remote: Compressing objects: 92%
> > (14809/16096) remote: Compressing objects: 93% (14970/16096)
> > remote: Compressing objects: 94% (15131/16096) remote:
> > Compressing objects: 95% (15292/16096) remote: Compressing
> > objects: 96% (15453/16096) remote: Compressing objects: 97%
> > (15614/16096) remote: Compressing objects: 98% (15775/16096)
> > remote: Compressing objects: 99% (15936/16096) remote:
> > Compressing objects: 100% (16096/16096) remote: Compressing
> > objects: 100% (16096/16096), done.
> > Receiving objects: 0% (1/62166) fatal: write error: No space left on
> > device
> > fatal: index-pack failed
> >
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1721)
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:72)
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:405)
> > at
> >
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
> > at
> >
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
> > at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> > at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> > at hudson.remoting.Request$2.run(Request.java:369)
> > at
> >
> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > at java.lang.Thread.run(Thread.java:748)
> > Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
> call
> > to H21
> > at
> > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> > at
> >
> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
> > at hudson.remoting.Channel.call(Channel.java:955)
> > at
> >
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
> > at sun.reflect.GeneratedMethodAccessor918.invoke(Unknown
> > Source)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:498)
> > at
> >
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
> > at com.sun.proxy.$Proxy119.execute(Unknown Source)
> > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:886)
> > at
> > hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> > at hudson.scm.SCM.checkout(SCM.java:504)
> > at
> > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> > at
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> > at
> > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> > at
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> > at hudson.model.Run.execute(Run.java:1794)
> > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> > at
> > hudson.model.ResourceController.execute(ResourceController.java:97)
> > at hudson.model.Executor.run(Executor.java:429)
> > ERROR: Error fetching remote repo 'origin'
> > Retrying after 10 seconds
> >> git rev-parse --is-inside-work-tree # timeout=10
> > Fetching changes from the remote Git repository
> >> git config remote.origin.url
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git #
> timeout=10
> > ERROR: Error fetching remote repo 'origin'
> > hudson.plugins.git.GitException: Failed to fetch from
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
> > at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> > at hudson.scm.SCM.checkout(SCM.java:504)
> > at
> hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> > at
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> > at
> > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> > at
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> > at hudson.model.Run.execute(Run.java:1794)
> > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> > at
> > hudson.model.ResourceController.execute(ResourceController.java:97)
> > at hudson.model.Executor.run(Executor.java:429)
> > Caused by: hudson.plugins.git.GitException: Command "git config
> > remote.origin.url
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git" returned
> > status code 4:
> > stdout:
> > stderr: error: failed to write new configuration file <
> >
> https://builds.apache.org/job/guacamole-client-coverity/ws/guacamole-client/.git/config.lock
> >>
> >
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1970)
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1966)
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1597)
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1609)
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.setRemoteUrl(CliGitAPIImpl.java:1243)
> > at hudson.plugins.git.GitAPI.setRemoteUrl(GitAPI.java:160)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:498)
> > at
> >
> hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:929)
> > at
> >
> hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:903)
> > at
> >
> hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:855)
> > at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> > at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> > at hudson.remoting.Request$2.run(Request.java:369)
> > at
> >
> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > at java.lang.Thread.run(Thread.java:748)
> > Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
> call
> > to H21
> > at
> > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> > at
> >
> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
> > at hudson.remoting.Channel.call(Channel.java:955)
> > at
> >
> hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:283)
> > at com.sun.proxy.$Proxy118.setRemoteUrl(Unknown Source)
> > at
> >
> org.jenkinsci.plugins.gitclient.RemoteGitImpl.setRemoteUrl(RemoteGitImpl.java:295)
> > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:876)
> > at
> > hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> > at hudson.scm.SCM.checkout(SCM.java:504)
> > at
> > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> > at
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> > at
> > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> > at
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> > at hudson.model.Run.execute(Run.java:1794)
> > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> > at
> > hudson.model.ResourceController.execute(ResourceController.java:97)
> > at hudson.model.Executor.run(Executor.java:429)
> > ERROR: Error fetching remote repo 'origin'
>
> --
Matteo Merli
<mm...@apache.org>
Re: No space on H21 (Was: Fwd: Build failed in Jenkins:
guacamole-client-coverity #73)
Posted by Matteo Merli <ma...@gmail.com>.
Chris, are these `ws-cleanup` jobs something that is created internally by
Jenkins? The job we have is `pulsar_precommit_cpp`.
Is there any setting in Jenkins that could explain that these files are not
removed?
On https://builds.apache.org/job/pulsar_precommit_cpp/configure , we
already have "Days to keep builds == 1".
For "Days to keep artifacts" is left blank. Should that be set to 1 day
too?
Any other hint at things we can try to ensure builds are cleaned up
promptly?
Thanks,
Matteo
On Sun, Sep 30, 2018 at 3:57 PM Chris Lambertus <cm...@apache.org> wrote:
> The main culprit on H31 appears to be pulsar ws-cleanup jobs consuming
> almost 40% of the disk. (The problem on H21 was something docker-related.)
>
>
> This is the second project I have found (Brooklyn was the first) which is
> leaving ws-cleanup directories all over the workspace and filling up the
> disks. Each one of these was 500+ MB. Pulsar, please address ASAP.
>
>
>
>
> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:19
> pulsar_precommit_cpp@2_ws-cleanup_1535737112280
> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:18
> pulsar_precommit_cpp@2_ws-cleanup_1536174576053
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 06:04
> pulsar_precommit_cpp@2_ws-cleanup_1537292379178
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 22:45
> pulsar_precommit_cpp@2_ws-cleanup_1537397120862
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> pulsar_precommit_cpp@2_ws-cleanup_1537403378279
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 21 23:31
> pulsar_precommit_cpp@2_ws-cleanup_1537572701829
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 26 16:04
> pulsar_precommit_cpp@2_ws-cleanup_1537977879341
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 22:23
> pulsar_precommit_cpp@2_ws-cleanup_1538087035027
> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:14
> pulsar_precommit_cpp_ws-cleanup_1535699830230
> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:16
> pulsar_precommit_cpp_ws-cleanup_1535735511423
> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:13
> pulsar_precommit_cpp_ws-cleanup_1535786064397
> drwxr-xr-x 52 jenkins jenkins 4096 Sep 19 06:18
> pulsar_precommit_cpp_ws-cleanup_1536096963586
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 21 06:47
> pulsar_precommit_cpp_ws-cleanup_1536114938910
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 21 06:45
> pulsar_precommit_cpp_ws-cleanup_1536173165909
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:38
> pulsar_precommit_cpp_ws-cleanup_1536175584418
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:50
> pulsar_precommit_cpp_ws-cleanup_1536180707506
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:38
> pulsar_precommit_cpp_ws-cleanup_1536182927940
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:51
> pulsar_precommit_cpp_ws-cleanup_1536185193716
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 22 05:50
> pulsar_precommit_cpp_ws-cleanup_1536273709029
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 05:59
> pulsar_precommit_cpp_ws-cleanup_1536316964794
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:06
> pulsar_precommit_cpp_ws-cleanup_1536675650893
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 28 06:08
> pulsar_precommit_cpp_ws-cleanup_1536879616962
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 30 19:53
> pulsar_precommit_cpp_ws-cleanup_1536891056968
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 30 19:47
> pulsar_precommit_cpp_ws-cleanup_1536966466683
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 18 00:43
> pulsar_precommit_cpp_ws-cleanup_1537216387387
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 00:45
> pulsar_precommit_cpp_ws-cleanup_1537292309413
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 19 00:45
> pulsar_precommit_cpp_ws-cleanup_1537308526251
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> pulsar_precommit_cpp_ws-cleanup_1537331780420
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> pulsar_precommit_cpp_ws-cleanup_1537379886715
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
> pulsar_precommit_cpp_ws-cleanup_1537386197893
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
> pulsar_precommit_cpp_ws-cleanup_1537397073464
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:46
> pulsar_precommit_cpp_ws-cleanup_1537403247575
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:43
> pulsar_precommit_cpp_ws-cleanup_1537413954888
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
> pulsar_precommit_cpp_ws-cleanup_1537417322151
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
> pulsar_precommit_cpp_ws-cleanup_1537421381129
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:44
> pulsar_precommit_cpp_ws-cleanup_1537469348852
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 00:45
> pulsar_precommit_cpp_ws-cleanup_1537489399468
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 22 00:44
> pulsar_precommit_cpp_ws-cleanup_1537566528660
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 21 23:12
> pulsar_precommit_cpp_ws-cleanup_1537571549978
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 22 00:44
> pulsar_precommit_cpp_ws-cleanup_1537573605258
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 00:43
> pulsar_precommit_cpp_ws-cleanup_1537855939828
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 11:18
> pulsar_precommit_cpp_ws-cleanup_1537874305575
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 12:41
> pulsar_precommit_cpp_ws-cleanup_1537879296622
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 01:39
> pulsar_precommit_cpp_ws-cleanup_1537925955967
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 26 13:17
> pulsar_precommit_cpp_ws-cleanup_1537967826838
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 15:55
> pulsar_precommit_cpp_ws-cleanup_1537977326736
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 26 17:34
> pulsar_precommit_cpp_ws-cleanup_1537983268177
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 22:04
> pulsar_precommit_cpp_ws-cleanup_1538085846793
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 28 00:20
> pulsar_precommit_cpp_ws-cleanup_1538094024821
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 28 17:42
> pulsar_precommit_cpp_ws-cleanup_1538156543343
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:02
> pulsar_precommit_integrationtests_ws-cleanup_1536298125841
> drwxr-xr-x 53 jenkins jenkins 4096 Sep 25 06:00
> pulsar_precommit_integrationtests_ws-cleanup_1536968128660
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 18 00:43
> pulsar_precommit_integrationtests_ws-cleanup_1537218541598
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 20 00:45
> pulsar_precommit_integrationtests_ws-cleanup_1537331703324
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 22 00:43
> pulsar_precommit_integrationtests_ws-cleanup_1537553748859
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 00:43
> pulsar_precommit_integrationtests_ws-cleanup_1537730402937
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 07:48
> pulsar_precommit_integrationtests_ws-cleanup_1537775336586
> drwxr-xr-x 54 jenkins jenkins 4096 Sep 24 23:30
> pulsar_precommit_integrationtests_ws-cleanup_1537831851711
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 25 23:08
> pulsar_precommit_integrationtests_ws-cleanup_1537916928485
> drwxr-xr-x 55 jenkins jenkins 4096 Sep 27 15:21
> pulsar_precommit_integrationtests_ws-cleanup_1538061702243
>
>
> -Chris
> ASF Infra
>
>
>
>
>
> > On Sep 30, 2018, at 3:12 PM, Chris Thistlethwaite <
> cthistlego@outlook.com> wrote:
> >
> > I've been working on H21 and H31. They both had a some docker issues
> where the process was locked and I couldn't do anything with a few
> containers that were over 8 days old.
> >
> > Thought we might be good this morning so I threw them back into
> rotation. Guess not, so I'll take them back out.
> >
> > -Chris T.
> > ________________________________
> > From: Mike Jumper <mj...@apache.org>
> > Sent: Sunday, September 30, 2018 6:03:51 PM
> > To: builds@apache.org
> > Subject: No space on H21 (Was: Fwd: Build failed in Jenkins:
> guacamole-client-coverity #73)
> >
> > FYI - we're currently seeing disk space issues on H21.
> >
> > - Mike
> >
> >
> > ---------- Forwarded message ---------
> > From: Apache Jenkins Server <je...@builds.apache.org>
> > Date: Sun, Sep 30, 2018, 14:37
> > Subject: Build failed in Jenkins: guacamole-client-coverity #73
> > To: <de...@guacamole.apache.org>
> >
> >
> > See <
> >
> https://builds.apache.org/job/guacamole-client-coverity/73/display/redirect
> >
> >
> > ------------------------------------------
> > Started by user mjumper
> > [EnvInject] - Loading node environment variables.
> > Building remotely on H21 (ubuntu xenial) in workspace <
> > https://builds.apache.org/job/guacamole-client-coverity/ws/>
> > [WS-CLEANUP] Deleting project workspace...
> > Cloning the remote Git repository
> > Cloning repository
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> >> git init <
> >
> https://builds.apache.org/job/guacamole-client-coverity/ws/guacamole-client
> >
> > # timeout=10
> > Fetching upstream changes from
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> >> git --version # timeout=10
> >> git fetch --tags --progress
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > +refs/heads/*:refs/remotes/origin/*
> > ERROR: Error cloning remote repo 'origin'
> > hudson.plugins.git.GitException: Command "git fetch --tags --progress
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > +refs/heads/*:refs/remotes/origin/*" returned status code 128:
> > stdout:
> > stderr: remote: Counting objects: 62166, done.
> > remote: Compressing objects: 0% (1/16096) remote: Compressing
> > objects: 1% (161/16096) remote: Compressing objects: 2%
> > (322/16096) remote: Compressing objects: 3% (483/16096)
> > remote: Compressing objects: 4% (644/16096) remote:
> > Compressing objects: 5% (805/16096) remote: Compressing
> > objects: 6% (966/16096) remote: Compressing objects: 7%
> > (1127/16096) remote: Compressing objects: 8% (1288/16096)
> > remote: Compressing objects: 9% (1449/16096) remote:
> > Compressing objects: 10% (1610/16096) remote: Compressing
> > objects: 11% (1771/16096) remote: Compressing objects: 12%
> > (1932/16096) remote: Compressing objects: 13% (2093/16096)
> > remote: Compressing objects: 14% (2254/16096) remote:
> > Compressing objects: 15% (2415/16096) remote: Compressing
> > objects: 16% (2576/16096) remote: Compressing objects: 17%
> > (2737/16096) remote: Compressing objects: 18% (2898/16096)
> > remote: Compressing objects: 19% (3059/16096) remote:
> > Compressing objects: 20% (3220/16096) remote: Compressing
> > objects: 21% (3381/16096) remote: Compressing objects: 22%
> > (3542/16096) remote: Compressing objects: 23% (3703/16096)
> > remote: Compressing objects: 24% (3864/16096) remote:
> > Compressing objects: 25% (4024/16096) remote: Compressing
> > objects: 26% (4185/16096) remote: Compressing objects: 27%
> > (4346/16096) remote: Compressing objects: 28% (4507/16096)
> > remote: Compressing objects: 29% (4668/16096) remote:
> > Compressing objects: 30% (4829/16096) remote: Compressing
> > objects: 31% (4990/16096) remote: Compressing objects: 32%
> > (5151/16096) remote: Compressing objects: 33% (5312/16096)
> > remote: Compressing objects: 34% (5473/16096) remote:
> > Compressing objects: 35% (5634/16096) remote: Compressing
> > objects: 36% (5795/16096) remote: Compressing objects: 37%
> > (5956/16096) remote: Compressing objects: 38% (6117/16096)
> > remote: Compressing objects: 39% (6278/16096) remote:
> > Compressing objects: 40% (6439/16096) remote: Compressing
> > objects: 41% (6600/16096) remote: Compressing objects: 42%
> > (6761/16096) remote: Compressing objects: 43% (6922/16096)
> > remote: Compressing objects: 44% (7083/16096) remote:
> > Compressing objects: 45% (7244/16096) remote: Compressing
> > objects: 46% (7405/16096) remote: Compressing objects: 47%
> > (7566/16096) remote: Compressing objects: 48% (7727/16096)
> > remote: Compressing objects: 49% (7888/16096) remote:
> > Compressing objects: 50% (8048/16096) remote: Compressing
> > objects: 51% (8209/16096) remote: Compressing objects: 52%
> > (8370/16096) remote: Compressing objects: 53% (8531/16096)
> > remote: Compressing objects: 54% (8692/16096) remote:
> > Compressing objects: 55% (8853/16096) remote: Compressing
> > objects: 56% (9014/16096) remote: Compressing objects: 57%
> > (9175/16096) remote: Compressing objects: 58% (9336/16096)
> > remote: Compressing objects: 59% (9497/16096) remote:
> > Compressing objects: 60% (9658/16096) remote: Compressing
> > objects: 61% (9819/16096) remote: Compressing objects: 62%
> > (9980/16096) remote: Compressing objects: 63% (10141/16096)
> > remote: Compressing objects: 64% (10302/16096) remote:
> > Compressing objects: 65% (10463/16096) remote: Compressing
> > objects: 66% (10624/16096) remote: Compressing objects: 67%
> > (10785/16096) remote: Compressing objects: 68% (10946/16096)
> > remote: Compressing objects: 69% (11107/16096) remote:
> > Compressing objects: 70% (11268/16096) remote: Compressing
> > objects: 71% (11429/16096) remote: Compressing objects: 72%
> > (11590/16096) remote: Compressing objects: 73% (11751/16096)
> > remote: Compressing objects: 74% (11912/16096) remote:
> > Compressing objects: 75% (12072/16096) remote: Compressing
> > objects: 76% (12233/16096) remote: Compressing objects: 77%
> > (12394/16096) remote: Compressing objects: 78% (12555/16096)
> > remote: Compressing objects: 79% (12716/16096) remote:
> > Compressing objects: 80% (12877/16096) remote: Compressing
> > objects: 81% (13038/16096) remote: Compressing objects: 82%
> > (13199/16096) remote: Compressing objects: 83% (13360/16096)
> > remote: Compressing objects: 84% (13521/16096) remote:
> > Compressing objects: 85% (13682/16096) remote: Compressing
> > objects: 86% (13843/16096) remote: Compressing objects: 87%
> > (14004/16096) remote: Compressing objects: 88% (14165/16096)
> > remote: Compressing objects: 89% (14326/16096) remote:
> > Compressing objects: 90% (14487/16096) remote: Compressing
> > objects: 91% (14648/16096) remote: Compressing objects: 92%
> > (14809/16096) remote: Compressing objects: 93% (14970/16096)
> > remote: Compressing objects: 94% (15131/16096) remote:
> > Compressing objects: 95% (15292/16096) remote: Compressing
> > objects: 96% (15453/16096) remote: Compressing objects: 97%
> > (15614/16096) remote: Compressing objects: 98% (15775/16096)
> > remote: Compressing objects: 99% (15936/16096) remote:
> > Compressing objects: 100% (16096/16096) remote: Compressing
> > objects: 100% (16096/16096), done.
> > Receiving objects: 0% (1/62166) Receiving objects: 1% (622/62166)
> > Receiving objects: 2% (1244/62166) Receiving objects: 3%
> > (1865/62166) Receiving objects: 4% (2487/62166) Receiving objects:
> > 5% (3109/62166) Receiving objects: 6% (3730/62166) Receiving
> > objects: 7% (4352/62166) Receiving objects: 8% (4974/62166)
> > Receiving objects: 9% (5595/62166) Receiving objects: 10%
> > (6217/62166) Receiving objects: 11% (6839/62166) Receiving objects:
> > 12% (7460/62166) Receiving objects: 13% (8082/62166) Receiving
> > objects: 14% (8704/62166) Receiving objects: 15% (9325/62166)
> > Receiving objects: 16% (9947/62166) Receiving objects: 17%
> > (10569/62166) Receiving objects: 18% (11190/62166) Receiving
> objects:
> > 19% (11812/62166) Receiving objects: 20% (12434/62166) Receiving
> > objects: 21% (13055/62166) Receiving objects: 22% (13677/62166)
> > fatal: write error: No space left on device
> > fatal: index-pack failed
> >
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1721)
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:72)
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:405)
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:614)
> > at
> >
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
> > at
> >
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
> > at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> > at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> > at hudson.remoting.Request$2.run(Request.java:369)
> > at
> >
> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > at java.lang.Thread.run(Thread.java:748)
> > Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
> call
> > to H21
> > at
> > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> > at
> >
> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
> > at hudson.remoting.Channel.call(Channel.java:955)
> > at
> >
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
> > at sun.reflect.GeneratedMethodAccessor918.invoke(Unknown
> > Source)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:498)
> > at
> >
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
> > at com.sun.proxy.$Proxy142.execute(Unknown Source)
> > at
> > hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1146)
> > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> > at hudson.scm.SCM.checkout(SCM.java:504)
> > at
> > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> > at
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> > at
> > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> > at
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> > at hudson.model.Run.execute(Run.java:1794)
> > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> > at
> > hudson.model.ResourceController.execute(ResourceController.java:97)
> > at hudson.model.Executor.run(Executor.java:429)
> > ERROR: Error cloning remote repo 'origin'
> > Retrying after 10 seconds
> >> git rev-parse --is-inside-work-tree # timeout=10
> > Fetching changes from the remote Git repository
> >> git config remote.origin.url
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git #
> timeout=10
> > Fetching upstream changes from
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> >> git --version # timeout=10
> >> git fetch --tags --progress
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > +refs/heads/*:refs/remotes/origin/*
> > ERROR: Error fetching remote repo 'origin'
> > hudson.plugins.git.GitException: Failed to fetch from
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
> > at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> > at hudson.scm.SCM.checkout(SCM.java:504)
> > at
> hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> > at
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> > at
> > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> > at
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> > at hudson.model.Run.execute(Run.java:1794)
> > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> > at
> > hudson.model.ResourceController.execute(ResourceController.java:97)
> > at hudson.model.Executor.run(Executor.java:429)
> > Caused by: hudson.plugins.git.GitException: Command "git fetch --tags
> > --progress https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > +refs/heads/*:refs/remotes/origin/*" returned status code 128:
> > stdout:
> > stderr: remote: Counting objects: 62166, done.
> > remote: Compressing objects: 0% (1/16096) remote: Compressing
> > objects: 1% (161/16096) remote: Compressing objects: 2%
> > (322/16096) remote: Compressing objects: 3% (483/16096)
> > remote: Compressing objects: 4% (644/16096) remote:
> > Compressing objects: 5% (805/16096) remote: Compressing
> > objects: 6% (966/16096) remote: Compressing objects: 7%
> > (1127/16096) remote: Compressing objects: 8% (1288/16096)
> > remote: Compressing objects: 9% (1449/16096) remote:
> > Compressing objects: 10% (1610/16096) remote: Compressing
> > objects: 11% (1771/16096) remote: Compressing objects: 12%
> > (1932/16096) remote: Compressing objects: 13% (2093/16096)
> > remote: Compressing objects: 14% (2254/16096) remote:
> > Compressing objects: 15% (2415/16096) remote: Compressing
> > objects: 16% (2576/16096) remote: Compressing objects: 17%
> > (2737/16096) remote: Compressing objects: 18% (2898/16096)
> > remote: Compressing objects: 19% (3059/16096) remote:
> > Compressing objects: 20% (3220/16096) remote: Compressing
> > objects: 21% (3381/16096) remote: Compressing objects: 22%
> > (3542/16096) remote: Compressing objects: 23% (3703/16096)
> > remote: Compressing objects: 24% (3864/16096) remote:
> > Compressing objects: 25% (4024/16096) remote: Compressing
> > objects: 26% (4185/16096) remote: Compressing objects: 27%
> > (4346/16096) remote: Compressing objects: 28% (4507/16096)
> > remote: Compressing objects: 29% (4668/16096) remote:
> > Compressing objects: 30% (4829/16096) remote: Compressing
> > objects: 31% (4990/16096) remote: Compressing objects: 32%
> > (5151/16096) remote: Compressing objects: 33% (5312/16096)
> > remote: Compressing objects: 34% (5473/16096) remote:
> > Compressing objects: 35% (5634/16096) remote: Compressing
> > objects: 36% (5795/16096) remote: Compressing objects: 37%
> > (5956/16096) remote: Compressing objects: 38% (6117/16096)
> > remote: Compressing objects: 39% (6278/16096) remote:
> > Compressing objects: 40% (6439/16096) remote: Compressing
> > objects: 41% (6600/16096) remote: Compressing objects: 42%
> > (6761/16096) remote: Compressing objects: 43% (6922/16096)
> > remote: Compressing objects: 44% (7083/16096) remote:
> > Compressing objects: 45% (7244/16096) remote: Compressing
> > objects: 46% (7405/16096) remote: Compressing objects: 47%
> > (7566/16096) remote: Compressing objects: 48% (7727/16096)
> > remote: Compressing objects: 49% (7888/16096) remote:
> > Compressing objects: 50% (8048/16096) remote: Compressing
> > objects: 51% (8209/16096) remote: Compressing objects: 52%
> > (8370/16096) remote: Compressing objects: 53% (8531/16096)
> > remote: Compressing objects: 54% (8692/16096) remote:
> > Compressing objects: 55% (8853/16096) remote: Compressing
> > objects: 56% (9014/16096) remote: Compressing objects: 57%
> > (9175/16096) remote: Compressing objects: 58% (9336/16096)
> > remote: Compressing objects: 59% (9497/16096) remote:
> > Compressing objects: 60% (9658/16096) remote: Compressing
> > objects: 61% (9819/16096) remote: Compressing objects: 62%
> > (9980/16096) remote: Compressing objects: 63% (10141/16096)
> > remote: Compressing objects: 64% (10302/16096) remote:
> > Compressing objects: 65% (10463/16096) remote: Compressing
> > objects: 66% (10624/16096) remote: Compressing objects: 67%
> > (10785/16096) remote: Compressing objects: 68% (10946/16096)
> > remote: Compressing objects: 69% (11107/16096) remote:
> > Compressing objects: 70% (11268/16096) remote: Compressing
> > objects: 71% (11429/16096) remote: Compressing objects: 72%
> > (11590/16096) remote: Compressing objects: 73% (11751/16096)
> > remote: Compressing objects: 74% (11912/16096) remote:
> > Compressing objects: 75% (12072/16096) remote: Compressing
> > objects: 76% (12233/16096) remote: Compressing objects: 77%
> > (12394/16096) remote: Compressing objects: 78% (12555/16096)
> > remote: Compressing objects: 79% (12716/16096) remote:
> > Compressing objects: 80% (12877/16096) remote: Compressing
> > objects: 81% (13038/16096) remote: Compressing objects: 82%
> > (13199/16096) remote: Compressing objects: 83% (13360/16096)
> > remote: Compressing objects: 84% (13521/16096) remote:
> > Compressing objects: 85% (13682/16096) remote: Compressing
> > objects: 86% (13843/16096) remote: Compressing objects: 87%
> > (14004/16096) remote: Compressing objects: 88% (14165/16096)
> > remote: Compressing objects: 89% (14326/16096) remote:
> > Compressing objects: 90% (14487/16096) remote: Compressing
> > objects: 91% (14648/16096) remote: Compressing objects: 92%
> > (14809/16096) remote: Compressing objects: 93% (14970/16096)
> > remote: Compressing objects: 94% (15131/16096) remote:
> > Compressing objects: 95% (15292/16096) remote: Compressing
> > objects: 96% (15453/16096) remote: Compressing objects: 97%
> > (15614/16096) remote: Compressing objects: 98% (15775/16096)
> > remote: Compressing objects: 99% (15936/16096) remote:
> > Compressing objects: 100% (16096/16096) remote: Compressing
> > objects: 100% (16096/16096), done.
> > Receiving objects: 0% (1/62166) fatal: write error: No space left on
> > device
> > fatal: index-pack failed
> >
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1721)
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:72)
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:405)
> > at
> >
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
> > at
> >
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
> > at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> > at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> > at hudson.remoting.Request$2.run(Request.java:369)
> > at
> >
> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > at java.lang.Thread.run(Thread.java:748)
> > Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
> call
> > to H21
> > at
> > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> > at
> >
> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
> > at hudson.remoting.Channel.call(Channel.java:955)
> > at
> >
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
> > at sun.reflect.GeneratedMethodAccessor918.invoke(Unknown
> > Source)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:498)
> > at
> >
> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
> > at com.sun.proxy.$Proxy119.execute(Unknown Source)
> > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:886)
> > at
> > hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> > at hudson.scm.SCM.checkout(SCM.java:504)
> > at
> > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> > at
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> > at
> > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> > at
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> > at hudson.model.Run.execute(Run.java:1794)
> > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> > at
> > hudson.model.ResourceController.execute(ResourceController.java:97)
> > at hudson.model.Executor.run(Executor.java:429)
> > ERROR: Error fetching remote repo 'origin'
> > Retrying after 10 seconds
> >> git rev-parse --is-inside-work-tree # timeout=10
> > Fetching changes from the remote Git repository
> >> git config remote.origin.url
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git #
> timeout=10
> > ERROR: Error fetching remote repo 'origin'
> > hudson.plugins.git.GitException: Failed to fetch from
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git
> > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
> > at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> > at hudson.scm.SCM.checkout(SCM.java:504)
> > at
> hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> > at
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> > at
> > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> > at
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> > at hudson.model.Run.execute(Run.java:1794)
> > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> > at
> > hudson.model.ResourceController.execute(ResourceController.java:97)
> > at hudson.model.Executor.run(Executor.java:429)
> > Caused by: hudson.plugins.git.GitException: Command "git config
> > remote.origin.url
> > https://git-wip-us.apache.org/repos/asf/guacamole-client.git" returned
> > status code 4:
> > stdout:
> > stderr: error: failed to write new configuration file <
> >
> https://builds.apache.org/job/guacamole-client-coverity/ws/guacamole-client/.git/config.lock
> >>
> >
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1970)
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1966)
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1597)
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1609)
> > at
> >
> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.setRemoteUrl(CliGitAPIImpl.java:1243)
> > at hudson.plugins.git.GitAPI.setRemoteUrl(GitAPI.java:160)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:498)
> > at
> >
> hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:929)
> > at
> >
> hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:903)
> > at
> >
> hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:855)
> > at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> > at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> > at hudson.remoting.Request$2.run(Request.java:369)
> > at
> >
> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > at java.lang.Thread.run(Thread.java:748)
> > Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote
> call
> > to H21
> > at
> > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> > at
> >
> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
> > at hudson.remoting.Channel.call(Channel.java:955)
> > at
> >
> hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:283)
> > at com.sun.proxy.$Proxy118.setRemoteUrl(Unknown Source)
> > at
> >
> org.jenkinsci.plugins.gitclient.RemoteGitImpl.setRemoteUrl(RemoteGitImpl.java:295)
> > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:876)
> > at
> > hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
> > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
> > at hudson.scm.SCM.checkout(SCM.java:504)
> > at
> > hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
> > at
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> > at
> > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> > at
> >
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> > at hudson.model.Run.execute(Run.java:1794)
> > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> > at
> > hudson.model.ResourceController.execute(ResourceController.java:97)
> > at hudson.model.Executor.run(Executor.java:429)
> > ERROR: Error fetching remote repo 'origin'
>
> --
Matteo Merli
<mm...@apache.org>