You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Dan Hecht (JIRA)" <ji...@apache.org> on 2017/09/21 22:15:01 UTC
[jira] [Resolved] (IMPALA-2106) ec2 build slave git operations
sometimes fail
[ https://issues.apache.org/jira/browse/IMPALA-2106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dan Hecht resolved IMPALA-2106.
-------------------------------
Resolution: Invalid
Per previous comment.
> ec2 build slave git operations sometimes fail
> ---------------------------------------------
>
> Key: IMPALA-2106
> URL: https://issues.apache.org/jira/browse/IMPALA-2106
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 2.3.0
> Reporter: Dan Hecht
> Priority: Critical
> Labels: broken-build
>
> When jenkins jobs running on EC2 try to do git operations, they sometimes fail. e.g.:
> http://sandbox.jenkins.cloudera.com/job/impala-s3-cdh5-2.2.0_5.4.x/470/console
> {code}
> Receiving objects: 45% (116561/256286), 915.46 MiB | 679 KiB/s
> Receiving objects: 45% (116563/256286), 916.60 MiB | 846 KiB/s
> error: index-pack died of signal 15
> 00:20:25 fatal: index-pack failed
> 00:20:25
> 00:20:25 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1325)
> 00:20:25 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1186)
> 00:20:25 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$200(CliGitAPIImpl.java:87)
> 00:20:25 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:257)
> 00:20:25 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:413)
> 00:20:25 at org.jenkinsci.plugins.gitclient.AbstractGitAPIImpl.clone(AbstractGitAPIImpl.java:60)
> 00:20:25 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.clone(CliGitAPIImpl.java:87)
> 00:20:25 at hudson.plugins.git.GitSCM$2.invoke(GitSCM.java:1006)
> 00:20:25 at hudson.plugins.git.GitSCM$2.invoke(GitSCM.java:942)
> 00:20:25 at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2439)
> 00:20:25 at hudson.remoting.UserRequest.perform(UserRequest.java:118)
> 00:20:25 at hudson.remoting.UserRequest.perform(UserRequest.java:48)
> 00:20:25 at hudson.remoting.Request$2.run(Request.java:328)
> 00:20:25 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> 00:20:25 at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> 00:20:25 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 00:20:25 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 00:20:25 at java.lang.Thread.run(Thread.java:745)
> 00:20:25 Trying next repository
> 00:20:25 ERROR: Could not clone repository
> 00:20:25 hudson.util.IOException2: remote file operation failed: /data/jenkins/workspace/impala-s3-cdh5-2.2.0_5.4.x/repos/Impala at hudson.remoting.Channel@4c192f2e:impala-boost-static-burst-slave-1ee3.vpc.cloudera.com
> 00:20:25 at hudson.FilePath.act(FilePath.java:907)
> 00:20:25 at hudson.FilePath.act(FilePath.java:884)
> 00:20:25 at hudson.plugins.git.GitSCM.determineRevisionToBuild(GitSCM.java:942)
> 00:20:25 at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1108)
> 00:20:25 at org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:118)
> 00:20:25 at hudson.model.AbstractProject.checkout(AbstractProject.java:1411)
> 00:20:25 at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:652)
> 00:20:25 at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88)
> 00:20:25 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:561)
> 00:20:25 at hudson.model.Run.execute(Run.java:1665)
> 00:20:25 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
> 00:20:25 at hudson.model.ResourceController.execute(ResourceController.java:88)
> 00:20:25 at hudson.model.Executor.run(Executor.java:246)
> 00:20:25 Caused by: java.io.IOException: Could not clone
> 00:20:25 at hudson.plugins.git.GitSCM$2.invoke(GitSCM.java:1019)
> 00:20:25 at hudson.plugins.git.GitSCM$2.invoke(GitSCM.java:942)
> 00:20:25 at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2439)
> 00:20:25 at hudson.remoting.UserRequest.perform(UserRequest.java:118)
> 00:20:25 at hudson.remoting.UserRequest.perform(UserRequest.java:48)
> 00:20:25 at hudson.remoting.Request$2.run(Request.java:328)
> 00:20:25 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> 00:20:25 at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> 00:20:25 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 00:20:25 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 00:20:25 at java.lang.Thread.run(Thread.java:745)
> 00:20:25 Retrying after 10 seconds
> 00:20:36 Checkout:impala-s3-cdh5-2.2.0_5.4.x / /data/jenkins/workspace/impala-s3-cdh5-2.2.0_5.4.x - hudson.remoting.Channel@4c192f2e:impala-boost-static-burst-slave-1ee3.vpc.cloudera.com
> 00:20:36 Using strategy: Default
> 00:20:36 Last Built Revision: Revision 34e828f80255d2318596b6c8e2fd458def9eff33 (origin/cdh5-2.2.0_5.4.x)
> 00:20:37 > git rev-parse --is-inside-work-tree
> 00:20:37 Fetching changes from 1 remote Git repository
> 00:20:37 > git config remote.origin.url git://github.mtv.cloudera.com/CDH/Impala.git
> 00:20:37 Fetching upstream changes from origin
> 00:20:37 > git config --get remote.origin.url
> 00:20:37 > git fetch -t git://github.mtv.cloudera.com/CDH/Impala.git +refs/heads/*:refs/remotes/origin/*
> 00:30:37 ERROR: Timeout after 10 minutes
> 00:30:37 ERROR: Problem fetching from origin / origin - could be unavailable. Continuing anyway
> 00:30:37 hudson.plugins.git.GitException: Command "git fetch -t git://github.mtv.cloudera.com/CDH/Impala.git +refs/heads/*:refs/remotes/origin/*" returned status code 143:
> 00:30:37 stdout:
> 00:30:37 stderr:
> 00:30:37 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1325)
> 00:30:37 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1186)
> 00:30:37 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1110)
> 00:30:37 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1101)
> 00:30:37 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.fetch(CliGitAPIImpl.java:289)
> 00:30:37 at hudson.plugins.git.GitAPI.fetch(GitAPI.java:235)
> 00:30:37 at hudson.plugins.git.GitAPI.fetch(GitAPI.java:239)
> 00:30:37 at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:793)
> 00:30:37 at hudson.plugins.git.GitSCM.access$000(GitSCM.java:57)
> 00:30:37 at hudson.plugins.git.GitSCM$2.invoke(GitSCM.java:977)
> 00:30:37 at hudson.plugins.git.GitSCM$2.invoke(GitSCM.java:942)
> 00:30:37 at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2439)
> 00:30:37 at hudson.remoting.UserRequest.perform(UserRequest.java:118)
> 00:30:37 at hudson.remoting.UserRequest.perform(UserRequest.java:48)
> 00:30:37 at hudson.remoting.Request$2.run(Request.java:328)
> 00:30:37 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> 00:30:37 at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> 00:30:37 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 00:30:37 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 00:30:37 at java.lang.Thread.run(Thread.java:745)
> 00:30:37 ERROR: Could not fetch from any repository
> 00:30:37 hudson.util.IOException2: remote file operation failed: /data/jenkins/workspace/impala-s3-cdh5-2.2.0_5.4.x/repos/Impala at hudson.remoting.Channel@4c192f2e:impala-boost-static-burst-slave-1ee3.vpc.cloudera.com
> 00:30:37 at hudson.FilePath.act(FilePath.java:907)
> 00:30:37 at hudson.FilePath.act(FilePath.java:884)
> 00:30:37 at hudson.plugins.git.GitSCM.determineRevisionToBuild(GitSCM.java:942)
> 00:30:37 at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1108)
> 00:30:37 at org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:118)
> 00:30:37 at hudson.model.AbstractProject.checkout(AbstractProject.java:1411)
> 00:30:37 at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:652)
> 00:30:37 at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88)
> 00:30:37 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:561)
> 00:30:37 at hudson.model.Run.execute(Run.java:1665)
> 00:30:37 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
> 00:30:37 at hudson.model.ResourceController.execute(ResourceController.java:88)
> 00:30:37 at hudson.model.Executor.run(Executor.java:246)
> 00:30:37 Caused by: java.io.IOException: Could not fetch from any repository
> 00:30:37 at hudson.plugins.git.GitSCM$2.invoke(GitSCM.java:986)
> 00:30:37 at hudson.plugins.git.GitSCM$2.invoke(GitSCM.java:942)
> 00:30:37 at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2439)
> 00:30:37 at hudson.remoting.UserRequest.perform(UserRequest.java:118)
> 00:30:37 at hudson.remoting.UserRequest.perform(UserRequest.java:48)
> 00:30:37 at hudson.remoting.Request$2.run(Request.java:328)
> 00:30:37 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> 00:30:37 at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> 00:30:37 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 00:30:37 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 00:30:37 at java.lang.Thread.run(Thread.java:745)
> 00:30:37 Retrying after 10 seconds
> 00:30:48 Checkout:impala-s3-cdh5-2.2.0_5.4.x / /data/jenkins/workspace/impala-s3-cdh5-2.2.0_5.4.x - hudson.remoting.Channel@4c192f2e:impala-boost-static-burst-slave-1ee3.vpc.cloudera.com
> 00:30:48 Using strategy: Default
> 00:30:48 Last Built Revision: Revision 34e828f80255d2318596b6c8e2fd458def9eff33 (origin/cdh5-2.2.0_5.4.x)
> 00:30:49 > git rev-parse --is-inside-work-tree
> 00:30:49 Fetching changes from 1 remote Git repository
> 00:30:49 > git config remote.origin.url git://github.mtv.cloudera.com/CDH/Impala.git
> 00:30:49 Fetching upstream changes from origin
> 00:30:49 > git config --get remote.origin.url
> 00:30:49 > git fetch -t git://github.mtv.cloudera.com/CDH/Impala.git +refs/heads/*:refs/remotes/origin/*
> 00:40:49 ERROR: Timeout after 10 minutes
> 00:40:49 ERROR: Problem fetching from origin / origin - could be unavailable. Continuing anyway
> 00:40:49 hudson.plugins.git.GitException: Command "git fetch -t git://github.mtv.cloudera.com/CDH/Impala.git +refs/heads/*:refs/remotes/origin/*" returned status code 143:
> 00:40:49 stdout:
> 00:40:49 stderr:
> 00:40:49 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1325)
> 00:40:49 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1186)
> 00:40:49 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1110)
> 00:40:49 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1101)
> 00:40:49 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.fetch(CliGitAPIImpl.java:289)
> 00:40:49 at hudson.plugins.git.GitAPI.fetch(GitAPI.java:235)
> 00:40:49 at hudson.plugins.git.GitAPI.fetch(GitAPI.java:239)
> 00:40:49 at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:793)
> 00:40:49 at hudson.plugins.git.GitSCM.access$000(GitSCM.java:57)
> 00:40:49 at hudson.plugins.git.GitSCM$2.invoke(GitSCM.java:977)
> 00:40:49 at hudson.plugins.git.GitSCM$2.invoke(GitSCM.java:942)
> 00:40:49 at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2439)
> 00:40:49 at hudson.remoting.UserRequest.perform(UserRequest.java:118)
> 00:40:49 at hudson.remoting.UserRequest.perform(UserRequest.java:48)
> 00:40:49 at hudson.remoting.Request$2.run(Request.java:328)
> 00:40:49 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> 00:40:49 at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> 00:40:49 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 00:40:49 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 00:40:49 at java.lang.Thread.run(Thread.java:745)
> 00:40:49 ERROR: Could not fetch from any repository
> 00:40:49 hudson.util.IOException2: remote file operation failed: /data/jenkins/workspace/impala-s3-cdh5-2.2.0_5.4.x/repos/Impala at hudson.remoting.Channel@4c192f2e:impala-boost-static-burst-slave-1ee3.vpc.cloudera.com
> 00:40:49 at hudson.FilePath.act(FilePath.java:907)
> 00:40:49 at hudson.FilePath.act(FilePath.java:884)
> 00:40:49 at hudson.plugins.git.GitSCM.determineRevisionToBuild(GitSCM.java:942)
> 00:40:49 at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1108)
> 00:40:49 at org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:118)
> 00:40:49 at hudson.model.AbstractProject.checkout(AbstractProject.java:1411)
> 00:40:49 at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:652)
> 00:40:49 at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88)
> 00:40:49 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:561)
> 00:40:49 at hudson.model.Run.execute(Run.java:1665)
> 00:40:49 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
> 00:40:49 at hudson.model.ResourceController.execute(ResourceController.java:88)
> 00:40:49 at hudson.model.Executor.run(Executor.java:246)
> 00:40:49 Caused by: java.io.IOException: Could not fetch from any repository
> 00:40:49 at hudson.plugins.git.GitSCM$2.invoke(GitSCM.java:986)
> 00:40:49 at hudson.plugins.git.GitSCM$2.invoke(GitSCM.java:942)
> 00:40:49 at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2439)
> 00:40:49 at hudson.remoting.UserRequest.perform(UserRequest.java:118)
> 00:40:49 at hudson.remoting.UserRequest.perform(UserRequest.java:48)
> 00:40:49 at hudson.remoting.Request$2.run(Request.java:328)
> 00:40:49 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> 00:40:49 at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> 00:40:49 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 00:40:49 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 00:40:49 at java.lang.Thread.run(Thread.java:745)
> 00:40:49 [CHECKSTYLE] Skipping publisher since build result is FAILURE
> 00:40:49 [FINDBUGS] Skipping publisher since build result is FAILURE
> 00:40:49 [TASKS] Skipping publisher since build result is FAILURE
> 00:40:49 Archiving artifacts
> 00:40:50 Recording test results
> 00:40:55 Email was triggered for: Failure
> 00:40:55 Sending email for trigger: Failure
> 00:41:00 Sending email to: impala-jenkins@cloudera.com
> 00:41:00 Finished: FAILURE
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)